Production and In Camera Compositing

Matte and Matteless Compositing

The dvMatte algorithm uses an advanced, unique method of detail extraction. By combining a traditional color difference matte keyer, along with a matteless extraction, a truly stunning composite can be achieved.

Complete Edge Control

Edges are where a composite is made or broken. Previzion provides a complete set of edge controls, including:

• Add/mix composite controls, to fix bright or dark edges
• Edge coloration, to retint the edges and blend into a background plate
• Dynamic spill suppression, to handle alternating dark/light areas

Drawn Mattes

For fast, on the fly mattes, Previzion provides a powerful drawn matte system. The mattes are drawn with traditional pen type tools, but track and work in 3D space, including automatic extensions off the visible screen. This provides a staggeringly powerful tool for rapid scene modification.

Surveyed Mattes

For objects that are in fixed positions in the scene, such as greenscreen backdrops and floors, surveying the corners of the object and entering these coordinates into Previzion provides an efficient way of creating mattes that move with the object.

This works well for extending the virtual world off the edge of the greenscreen, to create a large virtual workspace with a smaller greenscreen for the live action work.

Garbage Mattes

‘Garbage’ mattes let the operator remove live action objects that interfere with the shot (mic boom poles, light stands on the edge of the screen, the edge of the greenscreen fabric) and replace them with the virtual background area.

Floor Mounted Targets

Previzion’s optical/inertial tracking can be set up in any orientation, including looking straight down at the floor.

New Breakthroughs in Rendering Capabilities with Windows 7

With the introduction of our Win7 systems, our rendering capabilities have exploded.

Polygon Rendering:

• at 30p, Previzion can render a 12million triangle object without dropping frames
• at 60i, Previzion can render a 4 million triangle object without dropping frames

Texture Rendering and Look-up

• Previzion can load and render three unique 1GPix textures with no major hit to render time
• The largest tested single file size that Previzion can load is 1.1GPix (we will eventually test 1.21Gixels!!!)
• Previzion has no known texture dimension limits. So a texture whose width is 300,000 pixels can be loaded and rendered

Rendering Transforms

• At both 60i and 30p, Previzion was able to support rendering of 6000(+/-) transforms in a scene without dropping frames

Depth of Field

The accuracy of Previzion’s lens calibration and rendering system means that depth of field of the virtual elements matches the depth of field of the live action elements.

Previzion’s real time 3D renderer simulates optical depth of field by calculating the actual circle of confusion for each individual pixel, based on the current focal length, sensor size, focus distance, and even the iris value.

Motion Blur

To avoid ‘judder’ type artifacts, the virtual background must be correctly blurred to match the motion blur that naturally occurs in the live action elements. Previzion automatically calculates and renders motion blur in the virtual background.

Interlaced Rendering

To correctly match 60i interlaced rendering, Previzion renders the background at 60 matched fields/second.

In addition, when using interlaced MPEG2 footage as virtual backgrounds, Previzion correctly uses alternating even and odd scan lines to generate the virtual background, preserving the distinctive look of 60i imagery.

Reflections and Transparency

Both reflective and transparent objects can be rendered cleanly in Previzion, with HD video texture maps updating in real time.

Live Action Integration

As seen above, Previzion’s integrated 2D/3D rendering pipeline enables the live action subjects to be placed behind the transparent windshield and reflections, integrating them into the virtual scene.

Interactive Shader Control

The key to seamless integration of CG and live action is fast, interactive control of the separate elements. Previzion fully supports adjustable parameters in CgFx shaders, so that the lighting and surface properties can all be tweaked in real time for on-set adjustment.

Video Integration

Previzion lets any of the video sources in the system, including pre-recorded HD video or live HDSDI video streams, to be mapped to 3D scene textures specified as video destinations by the video_ modifier, such as video_myTexture.jpg.

Transparency is also possible in video elements, by using a shader to combine a MPEG2 color foreground and a matching video alpha background.

Selectable HDSDI Outputs

Previzion has 2 4:2:2 HDSDI outputs that can be individually configured to output the following signals:

• Foreground

• Matte

• Background with Blend (matteless layer)

• Composite

Each of these contains embedded HDSDI timecode and audio buffers.

Embedded HDSDI Timecode and Audio

Each Previzion 4:2:2 output contains embedded timecode and audio buffers, to enable a frame-accurate match back to the original greenscreen feed.

Timecode is embedded according to the SMPTE RP188 standard, which is compatible with most HDSDI capable decks and recorders.

8 separate channels of audio HDSDI data are also transferred through the system, preserving audio sync with the original greenscreen feed.

Live HDSDI Inputs

Previzion has 2 4:2:2 HDSDI inputs, which can be independently assigned. Input #1 is typically sent to the keyer, as the live action foreground.

Input #2 can be routed to any of the video destinations available in the scene, including any of the 3D textures that are designated as video destinations.

In this way, advanced effects (such as mapping feeds from head mounted cameras onto virtual character faces) can be easily achieved.

Multiple HD Video Playback

Previzion uses parallel processing to decode and render up to 8x 1920×1080 MPEG2 HD files in real time. Files can be triggered to start in sync, and individually paused or restarted.

Interactive controls enable the in and out points of MPEG2 clips to be adjusted on set.

Dual Link 4:4:4 I/O [Under Development]

Previzion’s built in keyer can produce production-quality composites with most 4:2:2 video feeds. For the ultimate in keying performance, Previzion will support real time 4:4:4 I/O and log space camera data inputs, such as Sony’s S-Log and Panavision’s Panalog formats, with correct conversion into linear scene referred color space for compositing.

Automated Photo Matching

High quality photogrammetry requires capture and matching of large numbers of high resolution images. Doing this manually can be very slow.

Previzion’s pipeline makes uses of GPU accelerated photogrammetry tools like BundlerMatcher to automatically solve the position from where the image was taken, and to create a point cloud that describes the scene.

From there, our automatic undistortion tools remove distortion from hundreds of images at a time, and bring in the image-aligned cameras and the solved point cloud into Maya for the actual modeling process.

Real Time Projection Shaders

Once the camera positions are imported into Maya, Lightcraft’s photogrammetry pipeline uses real time CgFx projection shaders written to run inside Maya.

The undistorted images are applied to the shaders, which then project the images onto geometry for instant modeling feedback, dramatically speeding up the background creation process while still retaining artistic control over the process.

Projection Baking

The projection baking process bakes the image data from the projection cameras into UV mapped textures for the virtual objects in the scene.

Lightcraft created a special Maya C++ plugin to accelerate this process, and generate layered image files that are then ready for final paint fixes.

Background Color Controls

Instantly recolor background images with the toolset, including R, G, and B gamma adjustments, and basic brightness/contrast controls.