From Video Pixels to Multi-pass CGI compositing

[I SHALL REVISIT THIS SPACE AFTER I HAVE SHOT MY GREEN SCREEN PLATES AND PUT IN IMAGES TO SHOW MY WORKFLOW]

Video pixels are naturally separated into Luminescence and Chrominance. A Luma matte looks at the brightness of the pixel whereas Chroma matte looks at the colour.

Luma keying views the image as grey-scale and sets a threshold value for each pixel, equal to or greater than 100% white, and the rest are black (empty space). It is possible to separate out RGB channels and create a Luma matte for each value which can then be composited together to create a matte to key out backgrounds that are difficult to key with the original image. You would effectively be using the brightness in e.g. 50% red channel matte, 30 % blue channel matte and 10% red channel matte. This can be used to create a hard contrast matte which would have strong edges but a soft interior. And the threshold can be altered to create the opposite; the 2 mattes can then be composited together to create the Ulitmatte.

Garbage matte- a rough/quick matte.

Chroma Keying looks at RGB, hue, saturation and value (brightness). Looking at all this could create softer edges automatically. If one matte does not work it is a good idea to over lay a few Chroma key mattes with different thresholds and max them together. You could also raise the amount of green as oppose to Red and Blue and use this for a cleaner matte.

To remove grain from the plate (Green/Blue screen shot) effectively, remove grain from either the green or blue channel. Most of the grain noise comes from the blue channel and majority of the image detail is held in the red and green channels.

Over exposed or hotspots on the green screen?- lower the brightness to pull the matte.

Under exposed, too dark?- raise the brightness to pull the matte.

Despill- the removal of baking colour spill contaminating a foreground shot, e.g. Chroma Keying hair. Fix this by a hue shift or brightness drop on one of the RGB channels.

Spill Map- a monochrome image that contains excess green from the Green screen plate. Subtracting this from the plate will give a despilled image. Clipping the green channel is another method, by keeping the green lower than the red channel, but all methods are really just creating a series of spill maps.

The Composite= (Green Screen x Matte = scaled foreground) + (Background x Inverted Matte = scaled background).

Compositing this way makes a cleaner edge blend. Blending the two images into one is like punching a hole in the background and filling it with the foreground.

Alpha channel- matte rendered with a 4 channel CGI image for compositing.

Key- matte rendered to composite 2 video layers.

Both the above are for the same purpose but with different names.

Mix composite- Add mix method gives edges more intensity or removes dark edges around the foreground. This is done by adding a colour curve to the BG and FG mattes before using them in the composite.

Edge blend- use the two mattes to create an edge outline of the FG and blur from 1-3 pixels, then add this over the final composite.

Light wrap- blur the FG matte, invert, multiply by the origional matte and then multiply by BG to create a Light wrap.

Soft/Hard comp:

1. Create a soft matte with edge detail but bad density in the middle.

2. Multiply soft matte over BG to create soft comp.

3. Hard matte is a tight matte from FG with good middle density.

4. Add one matte over the other to create the hard comp.

Layer integration- selling the shot by integrating items from similar BG's to put into the image, creating depth and then colour correcting to make the shot look seamless.

CGI compositing:

Premultiplied- all passes rendered into one, unable to edit highlights/flares without editing the rest of the image especially in 8 and 16-bit channels because the RGB values exceed the alpha values. Colour correcting here will cause dark edges, this is fine for small (far away) images.

Unpremultiplied- Different passes of the same render. Rendering highlights on a separate pass stops highlight clipping (dead white spots). Easier to colour correct only the colour pass, without effecting shadows or highlights.

Render layer- rendering multiple CGI objects on separate layers can be more cost/time effective than having 1 CGI layer, because if a problem arises, only one part has to be re rendered.

Render pass- different surface attributes of a CGI object, when combined in a composition they create an editable object.

Beauty pass-(colour pass or diffuse pass)- a full colour pass of the object with coloured texture maps and lighting, normally no shadows, highlights or reflection.

Specular pass- (highlight pass)- this has highlights/special effects. Typically screened or added to final composite.

Reflection pass- all reflections! Typically screened or added,

Occlusion pass- (ambient occlusion)- all white version of the object with dark corners/cracks. Typically multiplied by colour pass.

Shadow pass- often rendered as a monochrome image to act as a tint over a colour pass.

Alpha pass- (matte)- one channel image. White over black.

Light pass- rendering lights in separate passes allows for more control by the compositor to match the live action lighting.

Data pass- creates depth of field, e.g. gradual blur by using a mask that fades from white to black.

Matte pass- used to isolate particular parts of the CGI image which can be edited by the compositor. 4 channels can be put in each pass, RGB and alpha.

Multi Chanel Files- previously 3D departments would use TIFF files, which were a series of different passes. Then along came EXR files, which is one image holding multiple passes like layers in Photoshop. This makes it perfect for Multi-pass CGI compositing.