5 lessons for the future success of virtual and augmented reality

Virtual and augmented reality (VR/AR) have been around for decades in university laboratories and military facilities, but in 2014 the technology turned a corner. Mainstream corporations poured billions of dollars into the hardware, and for the first time in history there were millions of affordable systems in the hands of businesses and consumers. This is just the tip of the iceberg. According to IDC, the AR/VR market is projected to take off over the next five years with a compound average growth rate of over 75%, growing to over $120 billion by 2022.

It’s been five years since this so-called consumer revolution started, and now is the time to step back and take stock of which applications have scaled successfully from prototype to product, and have demonstrated a positive impact on business or society. The World Economic Forum has convened a Global Futures Council on VR and AR, with members from industry, government, academia, nonprofits, and content studios from around the world, who among them have expertise in public policy, education, art, ethics, computer science, innovation and medicine. The goal of this piece is to summarize the inaugural work of the council and to highlight opportunities and pitfalls of large-scale VR/AR use.

We should learn from history. Here are five key lessons from the past that should shape our journey towards a more virtual and augmented future.

1) Practice makes perfect

Training in VR/AR is the use case which has stood the test of time, beginning with the flight simulator, introduced by Edwin Link in 1929. Simulators allow for repetitions on demand and without danger during training, allowing people to experience very rare and intense situations in a safe environment. Indeed, since the 1990s perhaps the most robust use-case has been training surgeons to use a variety of hand tools. Hundreds of studies have demonstrated the efficacy of VR/AR surgery training, and thousands of surgeries each year are actually performed by doctors who are either remotely located and use networked VR/AR to perform the surgery or alternatively are in the room with the patient but use the technology to augment their vision.

Corporations are catching on. In 2018, over 200,000 Walmart employees used VR to become better at their jobs, practicing customer service during the holiday rush, learning to spot safety violations around the stores quickly, and learning how to use new sales systems. In 2019 the number of Walmart employees trained in VR will grow to over a million, in more than 4,500 stores across the U.S. Similarly, Verizon is using VR to train thousands of store employees in how to safely navigate a robbery. Nationwide has cut its training of insurance adjusters down from three hours to just under 15 minutes, by having them inspect accidents in VR. In 2018, STRIVR, the company that facilitated these training modules, logged over 600,000 minutes of corporate training - that’s more than a year of continuous training.

2) Doing the impossible pays dividends

When thinking about use cases for VR/AR, “doing the impossible” should be a constant theme. A famous TED Talk by Chris Milk calls VR “The Ultimate Empathy Machine”. People can walk a mile in the shoes of another, and - by taking this novel perspective - empathize in a way that can change attitudes towards race, gender, age and other domains. There have been dozens of academic experiments over the past 15 years showing that intense VR experiences change attitudes and behaviour more than watching movies or doing role-playing exercises.

Clouds over Sidra, according to a recent email from the VR film’s creators, has been translated into over 15 languages, continues to double donations to UNICEF, and has brought in at least an extra one billion dollars to charities for Syrian refugees. The Emmy award winning VR film Collisions by Lynette Wallworth highlights the little known impact of atomic tests on indigenous Australians in the remote South Australian desert in the 1950s. This powerful VR experience, with the World Economic Forum, as an Executive Producer, was used at the Comprehensive Test Ban Treaty Organization meetings in Vienna in 2016 to encourage ratification by signatories to the Treaty and shown at the UN General Assembly in advance of an historic vote on a resolution to ban nuclear weapons.

Companies are now starting to use these empathy simulations in their diversity and inclusion curricula. For example, in a VR journey developed by Columbia Professor Courtney Cogburn, 1,000 Cut Journey, the viewer experiences implicit racial bias while “wearing” the body of a black male across three different stages of his life.

Not all of these impossible experiences need to simulate dire situations. The Red Sox and New England Sports Network used VR simulation produced by Intel to show fans a baseball game at Fenway park. For some fans this is just a fun gadget, but for others the pure normalcy is what makes it so special. One fan in particular who has multiple sclerosis was particularly taken by “watching the game with everyone else”.

3) Use body movements

We can read email pretty well on phones and computers; we don’t need to be immersed. What makes VR/AR special is its body movements—people turn their heads to see other people, use their hands to grab objects, and their feet to walk around. The most successful video games in VR/AR have realized this. Pokémon Go is an augmented reality game played by tens of millions each month, in which players peer through phones and tablets to find game characters, and they walk and drive miles to find them. What makes the game a success is not the graphics, which are fairly mundane, but the integration of the physical and the digital.

People love to find digital objects nestled in their physical world, and the travel component is likely what drives the game’s success. Indeed, a recent study by a Stanford University computer scientist estimated that Pokémon Go added a total of 144 billion steps to the physical activity of its users. Similarly, in VR one of the most successful games in terms of sales is Beat Saber, in which players swing their arms to smash boxes with light sabers and jump side-to-side to avoid flying walls. The graphics and narrative are simple compared to most VR games. What makes Beat Saber a success is that it leverages what makes VR/AR technology unique: moving one’s body.

4) Diversify teams

Given that VR/AR is a unique medium, and not just an extension of video games and film, the teams that produce the best applications tend to cross boundaries. From a production side, there are three components needed: information technologists to help guide hardware and software, domain experts to guide the content for impact and accuracy, and production teams that can provide story and directorial expertise. Moreover, diversity in ethnicity, geography, and other factors on projects is critical to ensure VR applications will scale and be most effective.

5) Avoid the downsides

VR is distracting. The first death in VR occurred in December 2017, when a Moscow game player was so consumed he forgot the constraints of the physical room, fell through a glass table and bled to death. Addiction could become an issue. Moreover, the privacy issues plaguing social media will be magnified in VR/AR, which captures biometric data via body movements and uses cameras that are constantly filming.

But the features of the technologies that produce these downsides can be used for strategic advantage. Consider the distraction issue. It turns out that VR is so consuming that it transports people, psychologically, out of their bodies. They literally feel the physical world less. One of the most compelling use cases is distracting patients from acute pain procedures, whether it is changing bandages for burn victims or doing painful dental surgery.

Dozens of peer-reviewed academic articles have demonstrated the effectiveness of this Today, after 25 years of research, these products are beginning to emerge in hospitals and healthcare centres. For example, at Lucille Packard Children’s Hospital Stanford, thousands of patients have used VR as part of their regular treatment over the past two years, and that is only one of many hospitals supported by Oculus to treat patients.

VR and AR present a significant market opportunity. To realize this growth, industry and policymakers must collaborate to find ways to mitigate the potential negative effects. Our best advice is to start with a problem that needs to be solved, and then decide if VR/AR technology is a fit. Not everything needs to be immersive, and when choosing where to start we should look to the amazing and enduring success cases from the past.

Information about the Authors:

Sandra Lopez leads Immersive Experiences for Intel and Jeremy Bailenson leads the Virtual Human Interaction Lab at Stanford. Jeremy is also the author of Experience on Demand.