DanielChoi is a writer who’s currently pursuing a BA in Film & Television from New York University. With a background in amateur film production, Daniel is fascinated by how artists’ cultural backgrounds inform their work, subconsciously or not, and how that work is then perceived by different audiences across time and space. He joined Hollywood Insiderto promote its mission statement of substantive entertainment journalism, and hopes to enrich readers’ understandings of cinema through insightful analysis.
If you’ve seen ‘Pulp Fiction’ or ‘Austin Powers’, then you’ve seen parodic examples of rear projection, the old cinematic technique wherein a moving background is projected through a screen behind the actors, often for driving scenes. It’s largely been replaced by chroma key compositing, but the basic premise of the concept seems logical and intuitive–why not use pre-existing footage to create background images that would otherwise be difficult, expensive, or impossible to produce?
Of course, all of the minor, technical hiccups accumulate–classic instances of rear projection, dating back to 1930 and ubiquitously used until the ‘60s, appear cheesy and visibly outdated to modern sensibilities. Backgrounds appear washed out due to being projected through a screen, and accurately matching the foreground with the background is no easy feat–how do you create dynamic foreground lighting that perfectly matches with the background, or cause the prop car to bump and sway with each corresponding movement of the image?
The invention of Scotchlite, a type of retroreflective sheeting developed by 3M, would lead to a new technique called front projection, prominently utilized in ‘2001: A Space Odyssey’ and ‘Superman’, that had a number of advantages over rear projection–not only was the image sharper and more saturated, but the whole set-up also occupied less space. However, it was digital compositing that ultimately replaced in-camera projection techniques as the industry standard for VFX, and “green screen” has become a household term.
One of the main differences between digital compositing and older projection techniques is that projection techniques are done “in-camera,” meaning the visual effect exists on the original negative or recording. Green screen is capable of many things, but one definitive shortcoming is its inability to authentically interact with the actors in front of it–on the set of ‘The Hobbit’, for instance, Sir Ian McKellen cried with frustration and lamented, “This is not why I became an actor.”
Christopher Nolan, known for his preference of practical effects over CGI, would go to impressive lengths to ensure his actors wouldn’t be stuck with a green screen throughout the production of ‘Interstellar’–revamping the front projection process, Nolan and VFX company DNEG (then known as Double Negative) wrapped a projection screen 30 feet in height by 300 feet in length around the main set of the Endurance spacecraft, as well as two additional screens around smaller spacecraft. Using Barco projectors attached to a forklift, the background image could be projected anywhere on the screen.
Damien Chazelle’s‘First Man’, whose VFX was also handled by DNEG, utilized a similar technique to shoot its spacecraft exteriors. Unlike ‘Interstellar’, however, ‘First Man’ used a 35 feet-high-by-60-feet-diameter LED screen that formed a 180-degree semicircular background. One key advantage of this set-up was that the LED screen provided its own light which fell on the actors–the horizon of the Earth reflected in Ryan Gosling’s visor, for instance, is the reflection of the CGI animation displayed on the LED.
The pixel pitch (distance between RGB pixel clusters) of the LED panels used was too large to avoid moiré and offer photo-realistic imagery, so they were instead used purely for lighting. In doing so, ILM had realized the means of using LED panels to bypass a classic technical challenge presented by both projection techniques and digital compositing–accurate, dynamic lighting.
Over half of ‘The Mandalorian’ Season 1 was shot in the Volume, which refers to the 20-feet-high-by-180-feet-circumference LED video wall that created a 270-degree semicircular background. Serving as both background and lighting, the Volume removed not only the need for digital compositing in post but also the time-consuming task of setting up lights and sets for each shot.
However, what really sets StageCraft apart from its predecessors is its incorporation of motion-capture technology and Epic Games’ Unreal Engine (the company and engine behind ‘Fortnite’)–this allows StageCraft, unlike previous techniques, to allow camera movement. But how?
The dilemma that previous filmmakers faced was that, even if the set were completely encapsulated by photorealistic digital projections, their static nature would cause inconsistencies in perspective, also known as a failure to achieve parallax. To clarify, if there’s a ball behind a tree in real life, you can walk to the right and the ball will emerge from behind the tree. With a digital projection on screen, there’s no real 3-dimensional space and so the same effect cannot be achieved.
StageCraft circumvents this limitation through a multi-step process that first involves photographic scanning of a location to create virtual environments–thousands of photos are taken from different viewpoints to generate a 3-D photographic model. These photos are combined with computer-rendered sets through a technique commonly known as tiling or photogrammetry in order to create a photorealistic background that’s camera-ready.
Finally, the 3-D environment is rendered by Unreal Engine while the camera’s position is tracked by infrared cameras–according to the camera’s angle and position, the engine renders the image with proper 3-D parallax in real-time, finally allowing for camera movement in conjunction with in-camera compositing.
Shortcomings of StageCraft and the Future of VFX
StageCraft is undeniably a miracle in video production that not only perfects the art of in-camera compositing but also eliminates much of the time-consuming processes of lighting and production design.
However, there’s always room for improvement–as advanced as it is, the technology isn’t completely there yet. The Unreal Engine is only capable of rendering a photorealistic projection for the portion of the screen that’s on-camera, and there’s a slight delay that occurs between the camera movement and parallax adjustment–this means that it can’t yet handle fast camera movements.
Another problem is that the LED panels don’t have the power to re-create the quality or intensity of direct sunlight, and so any ‘Mandalorian’ scene that required such lighting was simply shot on a Hollywood backlot.
Its arrival seems particularly timely against the backdrop of the ongoing pandemic, as productions continue to suffer from lockdowns and travel bans. ILM provides “pop up” virtual production services, as was the case with ‘The Midnight Sky’, which seems like a perfect solution for any production that’s unable to travel or shoot on location.
An excerpt from the love letter: Hollywood Insider’s CEO/editor-in-chief Pritan Ambroase affirms, “Hollywood Insider fully supports the much-needed Black Lives Matter movement. We are actively, physically and digitally a part of this global movement. We will continue reporting on this major issue of police brutality and legal murders of Black people to hold the system accountable. We will continue reporting on this major issue with kindness and respect to all Black people, as each and every one of them are seen and heard. Just a reminder, that the Black Lives Matter movement is about more than just police brutality and extends into banking, housing, education, medical, infrastructure, etc. We have the space and time for all your stories. We believe in peaceful/non-violent protests and I would like to request the rest of media to focus on 95% of the protests that are peaceful and working effectively with positive changes happening daily. Media has a responsibility to better the world and Hollywood Insider will continue to do so.”
DanielChoi is a writer who’s currently pursuing a BA in Film & Television from New York University. With a background in amateur film production, Daniel is fascinated by how artists’ cultural backgrounds inform their work, subconsciously or not, and how that work is then perceived by different audiences across time and space. He joined Hollywood Insiderto promote its mission statement of substantive entertainment journalism, and hopes to enrich readers’ understandings of cinema through insightful analysis.
DanielChoi is a writer who’s currently pursuing a BA in Film & Television from New York University. With a background in amateur film production, Daniel is fascinated by how artists’ cultural backgrounds inform their work, subconsciously or not, and how that work is then perceived by different audiences across time and space. He joined Hollywood Insiderto promote its mission statement of substantive entertainment journalism, and hopes to enrich readers’ understandings of cinema through insightful analysis.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.Ok
