Sky replacement is among the cheapest and easiest extensions that can be made to a shot. This opens up various possibilities to shoot faster and more cheaply. Not only do you not have to wait for ideal climate conditions, you can swap in not only a different sky but even an extended physical skyline.
After all, skies are part of the story, often a subliminal one but occasionally a starring element. In Vanilla Sky, for example, the surreal-looking sky was the first clue that maybe Tom Cruise's character was no longer inhabiting the real world. An interior with a window could be anywhere, but show a Manhattan or San Francisco skyline outside the window and locals will automatically gauge the exact neighborhood and city block of that location, along with the time of day, time of year, weather, outside temperature, and so on—possibly without ever really paying conscious attention to it. Why spend extra production money on background conditions for a scene that could be shot cheaper elsewhere? You could spend tens or even hundreds of thousands of dollars for that view from the apartment on Central Park East for a scene at golden hour (the beautiful "hour" of sunset that typically lasts about 20 minutes and is missing on an overcast day). The guerilla method would be to use your friend's apartment, light it orange, shoot all day, and add the sunset view in post. In many cases, the real story is elsewhere, and the sky is a subliminal (even if beautiful) backdrop that must serve that story (see Figure 8).
Figure 8 For an independent film with no budget set in San Francisco, the director had the clever idea of shooting it in a building lobby across the bay in lower-rent Oakland (top left), pulling a matte from the blue sky (top center), and match-moving a still shot of the San Francisco skyline from street level (top right) for a result that anyone familiar with that infamous pyramid-shaped building would assume was taken in downtown San Francisco (bottom). (Images courtesy of The Orphanage.)
The Sky Is Not (Quite) a Bluescreen
Only on the clearest, bluest days does the sky become a candidate for bluescreen keying. Look at an actual sky nearby as you read this—even better, study reference images—and you may notice that the blue color desaturates near the horizon, cloudless skies are not always so easy to come by, and even clear blue skies are not as saturated as they might sometimes seem.
Still, some combination of a color keyer, such as Keylight, and a hi-con luminance matte pass or a garbage matte, as needed, can remove the existing sky in your shot, leaving nice edges around the foreground.
The first step of sky replacement is to remove the existing "sky" (which may include other items at infinite distance, such as buildings and clouds) by developing a matte for it. As you do this, place the replacement sky in the background. A sky matte typically doesn't have to be as exacting as a bluescreen key, because the replacement sky often bears a resemblance to the source (see Figure 9).
Figure 9 This rather poor bluescreen matte is acceptable for this type of shot because the color range and contrast of the target background are not so different from the source. The holes in the matte are the result of reflected sky color in the highlights of the foreground; this looks acceptable in the final shot.
A locked-off shot can be completed with the creation of the matte and a color match to the new sky. If there's camera movement in the shot, however, you might assume that a 3D track is needed to add a proper new sky element. Typically, that's overkill. Instead, consider these options:
- When matching motion from the original shot, if anything in the source sky can be tracked, by all means track the source.
- If only your foreground can be tracked, apply a track to a 3D camera: Move the replacement sky to the distant background (via a Z Position value well into four or five digits, depending on camera settings). Scale up to compensate for the distance. This is all done by eye.
- A push or zoom shot may be more easily re-created using a tracked 3D camera.
The basic phenomenon to re-create is that scenery at infinite distance moves less than objects in the foreground. This is the parallax effect, which is less pronounced with a long, telephoto lens, and much more obvious with a wide angle. For the match in Figure 8, a still shot (no perspective) was skewed to match the correct angle and tracked in 2D; the lens angle was long enough and the shot brief enough that they got away with it.