Spurred by the need to find new ways to maintain safe customer experiences during the coronavirus pandemic, companies are keen to adopt new touchless interfaces that help employees and customers work and play effectively in ways that move beyond the traditional, flatter realm of 2D computing. From automatic speech recognition to 3D eye tracking and onto gesture-based computing, companies are investigating how these technologies can be used to break a reliance on touching screens – without negatively impacting the quality of customer experiences. And that’s without considering the rise of entirely virtual worlds, such as the so-called metaverse, where entertainment and commerce (according to its enthusiasts at least) will take place minus a physical presence at all. SEE: CIO priorities: 10 challenges to tackle in 2022 Consultant Capgemini says organisations recognise that touchless interfaces are integral to high-quality experiences in a health- and safety-conscious environment: 75% of business leaders believe that increasing customer appetite for non-touch practices will persist in the post-pandemic world. This move towards new forms of interaction is a trend that resonates with Mia Sorgi, director of digital product and experience at food and drink giant PepsiCo Europe, whose company ran a gesture-based project recently that allowed customers in a KFC restaurant to be served by moving their hands, with no contact required. “I’m really proud of the work we did here,” she says. “I believe that gesture is a very important emerging interface option. I think it is something that we will be doing more of in the future. I think it’s really valuable to get an understanding of how to win in that space, and how to create something successful that people can use.” While PepsiCo’s gesture-based project received fresh impetus during the pandemic, Sorgi explains to ZDNet that the company has been experimenting with touchless technology for the past three years. Those initial investigations into gesture were scaled up and explored in a business environment last year. The front-end application, which was developed by PepsiCo, took a year to develop and allows customers in a restaurant to order food. Customers stood at a kiosk and made gestures in front of a menu. Hand-tracking technology interpreted the gestures and created orders that were sent through to the restaurant’s food-ordering system. Sorgi’s team worked with fast-food specialist AmRest and trialled the touchless system in a KFC restaurant in Poland. They also worked with design and engineering agency Method and used gesture-control technology from specialist company Ultraleap. The trial has shown the benefits – and some of the issues – of touchless technology. “It’s increasingly reliable, and there are many ways to implement it; there are many technologies you can leverage. But it’s also important to recognise that design around gesture is still very challenging,” she says. The project was implemented in three phases. In the first phase, the team discovered how difficult it is to design a touchless menu that provides a user experience that’s as good – if not better – than a traditional touchscreen interface. “That’s a technically complicated process, but really it’s a user experience problem,” says Sorgi. “Will the users accept it? Will they understand what they need to do? How do you change the menu to reflect what they’re trying to achieve? Can you fit everything in? All of those things are uncharted territory.” After a couple of months, Sorgi’s team created what she refers to as a passable prototype: “It wasn’t perfect, it was still very slippery and fiddly. But we thought, ’there’s something in this, and we need to keep going.’” That level of progress led to a second phase of development through summer and autumn 2020. Here, the team tested the design with real users during a respite in lockdown. The team then moved to the third stage, which was a trial in a public setting. Lockdown challenges meant the three-week trial in the KFC restaurant in Poland could only begin in June 2021. Sorgi says any technical issues were minor and corrected through iterative development. She says service design is crucial to effective touchless technology, especially signalling to people where they should go and what they should do. “This is a new experience,” she says. “Everybody wants to touch the screen because the past 20 years we’ve taught people to touch screens. In 1990, no one would have touched the screen. People had to unlearn that behaviour, so how do you message around the new touchless experience? What do you put on the home screen?” Generally, customers had a positive impression of touchless technology. “What we were able to show is that the system worked,” she says. “And that was not a completely safe assumption to make given the complexity of the design challenge.” SEE: The Metaverse is a human rights dilemma One of the other things Sorgi’s team has discovered is that gesture technology can be ergonomically beneficial. “That extra bit of reaching to push a touchscreen, it actually doesn’t do you any favours,” she says. “Yes, you get a bit of feedback with your finger, but if you can successfully communicate that a choice has been made in mid-air, it’s a little easier to use than a touchscreen – as long as the cameras pick up the gesture.” Research suggests other companies should run similar experiments. Deloitte goes as far as to suggest the single most important factor that differentiates the coronavirus crisis from previous ones is the rise of the contactless economy. While PepsiCo’s initial trial is now over, Corgi says work around the project continues. “What PepsiCo learned in general is that a gesture-based interface is viable in a commercial setting. In a high-traffic, fast-food setting, this is a viable option to pursue. This technology will become more common as people master the design and the technologies,” she says.
