High Speed Video Using a Dense Camera Array

High Speed Video Using a Dense Camera Array
Bennett Wilburn Stanford University	Neel Joshi Stanford University	Vaibhav Vaish Stanford University	Marc Levoy Stanford University	Mark Horowitz Stanford University
Presented at CVPR 2004
Abstract We demonstrate a system for capturing multi-thousand frame-per-second (fps) video using a dense array of cheap 30fps CMOS image sensors. A benefit of using a camera array to capture high speed video is that we can scale to higher speeds by simply adding more cameras. Even at extremely high frame rates, our array architecture supports continuous streaming to disk from all of the cameras. This allows us to record unpredictable events, in which nothing occurs before the event of interest that could be used to trigger the beginning of recording. Synthesizing one high speed video sequence using images from an array of cameras requires methods to calibrate and correct those cameras' varying radiometric and geometric properties. We assume that our scene is either relatively planar or is very far away from the camera and that the images can therefore be aligned using projective transforms. We analyze the errors from this assumption and present methods to make them less visually objectionable. We also present a method to automatically color match our sensors. Finally, we demonstrate how to compensate for spatial and temporal distortions caused by the electronic rolling shutter, a common feature of low-end CMOS sensors.

Figure 1: Our 52 camera high speed array.		Figure 2: 1560fps image of a popping balloon.
Paper Adobe Acrobat PDF (300KB)
Videos fan_even.mpg 1560 fps video of a fan. Aligning rolling shutter images causes temporal shifts, which make the fan motion non-uniform. fan_shifted.mpg Changing the camera trigger times to offset the temporal shifts makes the fan motion uniform again. Rolling shutter spatial distortions are still evident in the misshapen fan blades. fan_shifted_sliced.mpg Rolling shutter images are diagonal planes in the spatiotemporal volume. Slicing along planes of constant time produces an undistorted video. The fan spins uniformly, and the blades have the correct shape. balloon1_distorted.mpg The first of two popping balloons in one sequence. We have not corrected for the rolling shutter distortion, so the balloon appears to pop from the bottom instead of from where we struck it. balloon2_distorted.mpg The second popping balloon, also uncorrected. balloon2_sliced.mpg After slicing, the balloon motion is shown correctly. balloons.mpg The final corrected video sequence showing both popping balloons.