The Lion King:
Virtual Production

The solution

To solve this, we knew we needed to put the filmmakers back inside their movies, and put the traditional tools of cinema back in their hands. By building the world of the film in a real-time game engine we could put the filmmakers inside that world through virtual reality, and by connecting interfaces to their traditional live-action cinematography equipment, we could give them and the crew an experience similar to live-action filmmaking. Instead of standing over someone’s shoulder looking at a monitor, we could put them “inside the monitor.” This approach could be used by the Director, Assistant Director, and all members of the camera department, art department, animation, and visual effects, and would even allow traditional live-action crew, like grips and gaffers, to contribute their expertise.

The filmmakers would start their day “scouting the set” which was prepared by the Virtual Art Department (VAD) in multiplayer VR using the virtual production tools. From this, shot lists, lighting notes, animation feedback, and design changes can be made live, or in some cases, requested as revisions by the VAD or Animation teams. When a scene was rehearsed or blocked and ready to shoot, the departments would reassemble in the scene to shoot out the scene using a traditional live-action workflow. As each shot was captured, every detail and change was precisely recorded and sent to the appropriate departments. 

Video streams were sent to editorial, where they could be cut in minutes after being shot. Camera animations (as well as the movements of every piece of film equipment like dollies, gear heads, fluid heads, and cranes) would be saved as keyframes with each take and sent to visual effects, along with any changes made to lighting by the cinematographer. Changes to the assets in the scene made by the Art Department could happen on a per-shot basis with full continuity tracking to post-production as well, so that no detail was lost. In addition to providing clear creative direction to post production, the keyframes of the live-action camera operation were used as the final rendered keyframes in visual effects, providing the human-touch that the filmmakers required.