Could the metaverse help predict and produce movie blockbusters?

Technology, as it evolves, has enabled industries to reinvent and reimagine the way consumers experience their products and offerings. The entertainment and media industry in the last few years has transformed the way viewers experience content across formats, and forecasts suggest that it is only going to get better. The use of technology to deliver, for example, cinema or web series is now ubiquitous. With cutting-edge 5G connectivity and people stuck at home because of the pandemic, adoption of OTT platforms, especially streaming, has seen a huge surge.

At the same time, content producers are upping the game by adopting technology to rapidly increase quality and, most importantly, at an economical cost. This includes the use of algorithm-driven editing tools, auto voice simulation for singers, movies shot entirely on handheld phones, and many other innovations.

Digital entertainment platforms are accelerating hyper-personalized content suggestions to better engage users. This is often powered by an AI engine that provides insights on user journeys and time spent across content. The use of AI in entertainment could go much further: According to Gartner, a major blockbuster film will be released by 2030 with 90% of the film generated by AI.

With the metaverse around the corner, the entertainment industry has been one of the early adopters of the technology. Netflix created a metaverse experience of the movie The Gray Man on the platform Decentraland, enabling users to navigate using facts from the movie’s plot. Technology giant Meta have released a new wearable headgear, Meta Quest VR, for its users to watch cinema, concerts and other supported content together.

The metaverse as a technology enhances the interactivity and engagement of its consumers. Through immersive extended reality (XR) applications like virtual reality (VR), augmented reality (AR) and mixed reality (MR), it transforms the sensory experience of the content you watch, play or participate in; the metaverse is a digital world that will change how people interact, work and play, and is immersive, interoperable, and synchronous. Currently widely discussed, many industries are experimenting with how to improve the metaverse’s throughput and quality. In India alone, the metaverse is expected to enable a $10 billion opportunity in media and entertainment, according to a BCG report.

Beyond merely offering consumers an entertaining and engaging experience, the metaverse can assist in developing digital representations and simulating behaviours to reflect a particular thing or person. In the field of video production, it can help create a story plot with the treatment envisioned by the director. Musician Justin Bieber played live in a metaverse concert using a virtual version of himself that replicated his physical movements on an immersive XR platform.

In cinema, software companies are developing platforms that provide building blocks for models that can be reused time and again to create storylines in the metaverse. Digital avatars of actors and actresses can be built with attributes like expressions, reactions, gestures and other detailed aspects – captured from existing photo and video – which then can be used to simulate casting in a specific script. Additionally, the platforms can depict physical infrastructure, sets, outdoor locations as building blocks to choose a location, plan and organize the real-life production shoot; the virtual environment created using scanning technology such Trimble or NavVis. James Cameron has developed a similar system for his Avatar films, using a virtual camera to overlay an AR version of the planet Pandora on the actors’ performances in the studio.

Such simulations can significantly lower the logistical and environmental costs for a production, reducing travel to locations, thus lowering CO2 emissions, and reducing the need for non-recyclable objects to build sets and backgrounds. Digital models also provide new revenue opportunities for actors, similar to music label rights.

As well as the prospective set and locations, the metaverse can allow a director and team to virtually plot out a story to present the intended emotional effect and message of the finished product. This can help the director present his case and play it through for intended stakeholder groups for their feedback.

Upon finalizing the story in the metaverse, the next most important aspect is to simulate how the audience might perceive the movie, an impact investors will be interested in. A film is said to be successful when its viewers empathize with the characters and identify with the story. By taking a cue from how AI-driven deep learning engines are used to process natural language to understand customer sentiment for products or services, movie producers can simulate the metaverse story with viewer sentiments available in form of reviews of movies with similar genres or theme to generate scene-by-scene feedback.

The data set could also capture the emotions of how people reacted to a two-hour movie plot with a graph plotted on the highs and lows of engagement. For improved accuracy, the metaverse version can run by diverse individuals of different age groups and background, capturing their emotional response to run through a neural network algorithm to generate feedback.

A metaverse version of a film can also help understand whether the director’s vision of the plot is communicated effectively and what sections or scenes need refinement. The metaverse thus can effectively be leveraged across all phases of film-making, supporting the creative, feedback and financial aspects. The industry needs to create an organization to manage the digital rights of actors, artists who design the sets and digitize the locations, and the platforms which will continuously train the models, story/ideas conceptualized etc.

With the future of mixed reality on the roll, the way we use consume entertainment in the next five to 10 years will be exciting, engaging and interactive.