Over the years, technology has been instrumental in the progression of film-making, from innovations in camera technology to motion capture, VFX, and digital compositing. Now we are seeing virtual production beginning to make an impact.

What is virtual production?

There are many forms of virtual production, which is why it can be difficult to get your head around it all. It’s become a catch-all term for computer-aided production and visualisation filmmaking methods.

Rear-projection, which dates back to the 1930s, was the original method of faking environmental backgrounds on a soundstage. This was largely replaced by green screen technology, allowing directors to replace the environment in post-production, along with the ability to deliver increasingly complex visual effects sequences.

Nowadays, modern virtual production uses high-resolution LED screens and game engine technology to give creatives control over an entire scene — from the weather to the viewpoint. Recent films that have used virtual production include The Jungle Book, The Lion King, and The Mandalorian (2019).

Ben Grossmann, Oscar-winning and Emmy award-winning Visual Effects Supervisor and co-founder of Magnopus was the Virtual Production Supervisor onThe Lion King. He describes virtual production as creating a game called “Filmmaking” in which you are building the world of the film. Ben further explained it as, “Instead of guns or jet packs, we give the players traditional film equipment like cameras, dollies, and lights. When you’re using VR to make the movie, it’s as though you’re standing on a film set in the real world, even though it’s all virtual. You can pick up a tree and move it. You can grab the sun and change the light. You can become a character and give a different performance.”

THE LION KING — (Pictured) Caleb Deschanel. Photo by: Michael Legato.
© Disney Enterprises, Inc. All Rights Reserved.

What are the benefits?

Virtual production blurs the line between pre-production, production, and post. This non-linear, collaborative, and efficient process can be used to overcome a number of practical, creative, and budgetary constraints.

With virtual production, the workflow moves from a “fix it in post” to a “solve it in prep” approach. It empowers the filmmakers to experiment and visualise shots with immediate feedback. For the actors, it empowers them to deliver better performances as they can actually immerse themselves in the environment and see the virtual actors they are supposed to be interacting with.

Why does virtual production feel like a buzzword right now?

Recent advances in real-time computer graphics coupled with restrictions imposed by the COVID crisis have created a unique opportunity for production companies. The critical advantage at the moment is that virtual movie sets can be assembled by people working from home and the detail of a shoot can be manipulated by remote directors.

Using game engines like Unreal and Unity, combined with high-power graphics cards, camera tracking, as well as VR and AR, filmmakers are now able to create scenes across the physical and digital worlds.

It’s not just the film industry that has been quick to embrace the advantages of virtual production. The tools and techniques are used by many other industries including broadcast, music, fashion, and advertising.

Recently, we worked on a virtual production for the Cannes Lions keynote in partnership with Ryot Studio and Verizon Media. We acted as the virtual art department and our role was to design and create digital worlds which brought to life the content of the presentation — how technology is augmenting the sports fan experience. The result was a highly engaging experience.

What is a VAD?

Like a traditional art department, the virtual art department (VAD) is focused on shot design, layout, visual development, and creating production-ready digital assets to be seen on screen.

This description, taken from The Virtual Production Field Guide compiled by Epic Games, helps to further explain the role of a VAD: “Think of the VAD as a transitional team bridging the work of a more traditional art department and a regular previs department via the advent of real-time animation.”

Why is the VAD important for virtual production?

Taking designs and digital renderings from the art department, the VAD transforms them into virtual 3D replicas. By creating high-quality and accurate assets early on and knowing how they will interact with their surroundings, it is possible for the director and production team to fully visualise how the final shot will appear, before physical production. When the director does step foot on set, he or she will understand how everything will look and react with the VFX.

Which departments does the VAD work with?

The VAD works in close collaboration with the director, cinematographer, production designers, and traditional art departments in order to ensure the digital worlds integrate seamlessly with the vision of all the creative stakeholders.

An additional benefit

As high-quality assets are being produced, they can also be leveraged for promotional imagery, games, and further post-production without the need to rebuild assets for each new use. These digital artefacts can simply be recalled and reused in future productions and sequels.

The future

There is still much progress to be made. Advancements in CPUs and GPUs will continue to push the quality and current rendering limitations of real-time rendering.

In time, the demands of virtual production will also help to open up the creative possibilities to the wider industry. These technologies will become cheaper and more accessible, allowing a wider range of creatives and industries with smaller budgets to also adopt virtual production to create new and exciting content.

Virtual Production and the role of the virtual art department