Mediating the Background and the Foreground – From Green Screen and Chroma-Key Effects to Virtual Sets

It may be hard to remember now, but the first digital cameras only started to appear on the shelves in 1990, to be replaced for many just a decode later by camera replacing smartphones. Prior to that, cameras were film based, or produced “self-developing” polaroid photos, printed by the camera itself. Many film based cameras required the film to be manually “wound on” between taking one photograph and the next. Failure to do this could result in a particular piece of the film being double exposed, with the result that two photographs could be superimposed. Such tricks were well know to photographers and film makers alike, and multiple exposure techniques, along with other tricks of the photographer’s trade, were widely used for creating otherwise impossible to record scenes.

In Behind the Scenes of Sports Broadcasting – Virtual Sets, Virtual Signage and Virtual Advertising, we saw how sports broadcasters could make use of virtual sets to enhance outside, on-location settings. Virtual sets are increasingly used by broadcasters for a wide range of other live television formats, such as news and politics, with digital objects often appearing in front of the presenters. This contrasts with a traditional green screen effect where the background behind the presenter is replaced. In film studios, virtual “backlots” may make extensive use of green screens to replace the need for unwieldy physical sets with digital ones that are rendered in post production.

So how do green screen, or “chroma key” effects work?

Green Screen Effects

Green screen style effects have a long history in film and television and were available long before digital green screen effects became available. The effect relies on producing a matte, or travelling matte, from an image that allows elements from two separate images to be combined in a single image, a process referred to as compositing. By making part of one image transparent, it can be layered on top of another background image.

So what happened next?

(The earlier parts of the above video also retrace the history of the green screen effect.)

Exercise: for a simple demonstration of how green screen compositing works, the post Simple Demo of Green Screen Principle in a Jupyter Notebook Using MyBinder links to an interactive activity using the python programming language (no coding/programming skills required!) showing how to add a background apparently behind a greenscreened television newscaster.

Even without digital technologies and the introduction of virtual digital objects, compositing multiple takes of a few human actors can be used to generated a visual scene that appears to include a cast of thousands:

The following showreel provides some examples of how the green screen effect has been put to use as a virtual backlot for movies:

Exercise: see if you can find behind the scenes footage of the visual effects – VFX – used to create one or two recent of your most recent favourite films.

One problem with the chroma key approach is that much of the magic is done in post-production, and not in real time. But as you will know from watching TV weather reports, chroma-key key effects can be used for real time mediation of video imagery. And increasingly, green screen technique can be used to produce a virtual studio or virtual set in real time, with no post production required:

SAQ: What are the similarities and differences between virtual studio or virtual set and chroma key techniques?

The virtual set itself is a 3D digital model that is rendered around the human presenter(s).

An important part of the system is the ability to track the location of the camera, as well as physical objects within the set.

Replacing part of the visual scene using a chroma-key effect is a tried and trusted technique, and as work on virtual sets shows can be used to support real-time mediated reality effects. Tracking objects within the set allows digital objects to be overlaid on those tracked physical objects. But object tracking in the form of motion capture, and the even more refined performance capture, can be used as the basis for far more elaborate visual effects, as we’ll see in another post.

But first, let’s step aside for a moment, and see how the notion of image layers can be used to transform a single photograph into a short video…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.


%d bloggers like this: