Matte-Extraction - In some sequences you have a colorful foreground object that is moving in front of a solid color (often blue) background. Matte-extraction (or blue-screen matting) refers to the process of separating the foreground element from the background element.
Your matte-extraction plug in will take a single (r,g,b,a) frame as input and produce a single output (r,g,b,a) frame. The alpha value of the input frame will be 1 for every pixel. The alpha value of the output frame should vary between 0 and 1, where 0 represents a background pixel and 1 represents a completely foreground pixel. Pixels at the edge between background and foreground should contain fractional alpha values. This plug-in does not require any parameters. In general matte-extraction is an under-specified problem, but given certain assumptions it can be performed successfully.
Suppose Co = (Ro,Go,Bo,Ao) is the color of an input frame pixel, Cf
= (Rf,Gf,Bf,Af) is the color of the foreground element and Cb = (Rb,Gb,Bb,Ab)
is the color of the background element.
The matte-extraction plug-in must take a frame containing Co at every
pixel as input and produce a frame containing Cf at every pixel as output.
Let us assume that the input image was created by compositing the foreground
element OVER the background element. Let us also assume that the blue channel
of the foreground element is 0 (that is Bf=0) and that the background is
a contstant shade of pure blue (that is Cb = (0,0,Bb,1)). Under these conditions
determining Cf requires solving a simple linear equation.
Compositing - Implement three of the compositing operators described in in handout #11, Compositing Digital Images by Porter and Duff. One of the three must be the Over operator. You are free to choose the other two operators from the list they provide. Each compositing operator will take two frames containing (r,g,b,a) pixels as input and produce one output frame containing (r,g,b,a) pixels. The input frame will not contain pre-multiplied alphas and the compositing plug-ins should be written so that the output frame also does not have pre-multiplied alphas. Again no parameters are required for this plug-in.
Magnify and Translate - The magnify plug-in scales the image up by a factor ranging from 1.0 to 5.0. You should allow a continuous range of scale factors (i.e. floating point values between 1 and 5) and the scale factor should be attached to the slider. One good method for magnifying an image is to map each output image pixel position its corresponding position in the input image. For example if the magnification factor is 2.25, then the output pixel position Po = (10,23) would map to Pi = Po/s = (10/2.25, 23/2.25) = (4.444,10.222) in the input image. To determine the pixel value at Pi we then bilinearly interpolate the values of the four closest input pixels, which would be the pixels at input image locations (4,10), (4,11), (5,10) and (5,11).
For translation, the plug-in should translate an input frame by some amount (x,y), and as described earlier you should set both these parameters in the plug-in callback function. Like the magnification scale factor the translation offsets (x,y) should be floating point values and you should use bilinear interpolation to determine the output image pixel values.
Blur - The blur plug-in will take one frame as input and produce a blurred version of that frame. In general blur is a filtering operation that involves replacing each pixel in the input image with a weighted average of a set of pixels centered at the original pixel. The kernel of the blur filter is defined as the set of pixels that are averaged together and is often described as an NxN.
The simplest blur filter (also called a box filter) replaces each pixel with the unweighted average value of the pixels in the blur kernel. You must implement a box filter blur plug-in that allows the kernel size N to vary between 1 and 21, rounding off to the nearest odd integer (It is more difficult to blur with even size box filters, can you explain why?). The kernel size N should be attached to the slider provided by the video-editor.
Minify - Scale down an input frame by a scale factor ranging from 1 to 1/5. In this case as for Magnify, the scale factor is a floating point value. With minification each pixel of the output image combines many pixels of the input image. You should average together the input image pixels that correspond to each out image pixel.
Rotation - Rotate the input frame by some angle theta. Theta will be an arbitrary floating point value between 0 and 360 degrees.
Non-Linear Warp - Warp the input frame using some non-linear warp (i.e. a sinusoidal warp). Check out section chapter 10 of GV for more ideas on this plug-in. One of the example images (and videos) is a magnifying glass that was rendered over a blue background. After doing matte extraction on this data, you will have with a fractional alpha in the interior of the glass; use this to select a region of the image to warp in a way that simulates magnification as a magnifying glass would do.
Sharpen - Write an image sharpening plug-in. See section 6.5 of GV for more details on this.
Impressionistic Painting - Write a plug-in that automatically creates an impressionistic version of the input frame using one of the basic brush types from assignment 1.
Color Balance - Adjust the white balance in a sequence to fix an overly red or blue tinge to an image. White or grey areas of an image should have color components of nearly equal intensity. However this is not always true in the video sequences we have given you. For example, suppose a white pixel has values (R220,G250,B250). You could scale the red intensities in the image by 250/220 to restore correct color balance. While the effects of gamma in the camera make exact correction a nonlinear process, good results can often be obtained by simple linear scaling.