Movies today are filled with visual effects. If you watch a Hollywood movie, almost every shot has some form of computer-generated images, or CGI, even if it’s just adding in a digital background.
Let’s say you want to film a scene at a small outdoor café in Paris, but due to budget or other production limitations, you can’t fly the whole production to Paris. How do you achieve the authentic look of an outdoor café in Paris?
The way this has been achieved in Hollywood for years is to build a set of a little cafe, put a green screen (or blue screen depending on the number of factors including the color of the actor’s hair, their wardrobe, and the color of other objects in the scene) in the background, and replace the green screen with footage of the Eiffel Tower (fun fact, this footage is called a background plate).
Now, this technique has been perfected by the pros, but it comes with some small quirks as we continue to learn here at VMG Studios, a Seattle-based creative marketing agency that specializes in video production.
Recently, new technology has been developed that will fundamentally change the way movies, and other types of videos such as a marketing video, are made. This article will dive into the definition of virtual production and the techniques used to achieve certain aesthetics.
What is Virtual Video Production?
Virtual Production is tracking the position and orientation of a camera in the 3D space of a film set in order to place your actor in a virtual set.
This enables the director to see, in real-time, what the actor looks like on the virtual set, letting them fix any problems with the lighting or digital set.
This behind-the-scenes video from Parasite, which won Best Picture at the 92nd Academy Awards, shows virtual production in action.
Taking this back to our Parisian café example, with virtual production, you’d be able to control where the sun would be coming from depending on the time of day of your scene, or you could decide that you don’t quite like where the Eiffel Tower is and shift its positioning in the shot to make it look better.
The best example of how virtual production has recently changed the way sets are built, either physically or digitally, was on the recent Disney+ series The Mandalorian. They used this technique extensively, which gave the show’s creators significantly more freedom in their story.
How to Achieve the Virtual Production Aesthetic
Currently, there are two main ways to place your actor in a digital environment with virtual production:
1. Place the actor on a giant green screen and use software to key out (replace) the green. This is the more cost-effective method, but it has drawbacks that we’ll cover shortly.
2. Film the actor in front of or surrounded by walls of LED panels, essentially making a giant TV screen. This method is more expensive but yields far superior results.
There are pros and cons of each approach, so let’s dive into some of the quirks that come with the traditional route of green screen compositing (which, as mentioned, is a more cost-effective option) and how virtual production solves them, or at the very least, helps manage the pain.
How Virtual Video Production Works with a Green Screen
1. Camera Movement
When you key out a green screen, all you’re doing is telling the computer, “Take this color of green, make it transparent so you can see through those pixels like you’re looking through a window, and then you put in your desired background.”
The problem with this comes up when you want to move your camera.
Let’s say your actor is delivering an impassioned speech and you want the camera to slowly push in to pack the scene with some more emotional punch. To keep the replaced background looking like it’s supposed to be there, you need to track the green screen to follow the camera movement.
This involves placing tracking markers on the green screen (if you’ve seen any BTS photos from any of the Marvel movies, you’ve probably seen little orange T’s on the green screens), tracking those markers in virtual 3D space, and then replacing those markers with a green color so they can also be keyed out.
2. Realistic Lighting
One of the easiest ways to tell that something has been green screen composited is when the lighting on your actors doesn’t match the background. If your actor is lit like they’re standing under the sun at noon, but the background plate is of a moody sunset, your audience isn’t going to buy the composite.
If you’re using a green screen, virtual production will help a bit, especially when it comes to being able to see what the final image looks like on set.
On the other hand, this is where the LED wall method really shines. When you are in an environment, inevitably, some of the light bouncing around will hit your subject.
For example, when you’re in a forest, the light has a green tint to it, or when you’re standing next to a sunlit white wall, the light bounces onto your face.
Because of this, any light wrap, any reflections that occur on the set look natural and only help to sell the compositing effect.
3. Fixing Bad Keys
Lighting a green screen is hard. You have to make sure that the whole green area is as flat as possible, meaning there are no sections that are significantly brighter than others. This ensures that the green color the software uses to key is as uniform as possible, otherwise you run the risk of the key picking up some unwanted parts of the frame.
You also have to make sure that the green screen isn’t casting any green color onto your actor’s face. This can make the key significantly harder to pull, as well as making the actors look sickly.
The video below highlights what can happen when a green screen is poorly lit.
Admittedly, these issues persist in virtual production if you are using a green screen to composite in your digital background, but if you’re able to see the problems on set in real-time, you can make the necessary adjustments during production rather than fighting with a “bad key” when you get to post-production.
This is also another huge benefit of using the LED wall method. No green screen means no keying. Instead, you get very realistic composites, almost like your actors are in the computer world.
Odds are most of us aren’t going to working on virtual productions of that caliber, but the principles are going to have a huge impact on all of video production.
Moving Forward with Virtual Video Production
The way movies and videos are shot now is far from the early days of a single shot from a steady camera in a built set.
Now, hundreds of people may be on-set during big Hollywood productions, and more often than not, special effects are being used in real-time to capture the scene.
While it may not be within budget to use mixed reality, projection mapping, or LED walls for your marketing or corporate video, a green screen can help elevate the final product. A green screen enables the opportunity to add that “wow factor” to your video, much more cost-effectively.
There’s a learning curve to figuring out how best to use a green screen, and luckily, we’ve been at it for years! We have our own green screen here at VMG Studios and can show you endless possibilities for creating a killer marketing video while answering any questions you may have about how it works.
