Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform

 

Vaibhav Vaish

Stanford University

Gaurav Garg

Stanford University

Eino-Ville Talvala

Stanford University

Emilio Antunez

Stanford University

Bennett Wilburn

Stanford University

Mark Horowitz

Stanford University

Marc Levoy

Stanford University

 

To appear in A3DISS 2005 (at CVPR), San Diego.

 

Abstract

 

Synthetic aperture focusing consists of warping and adding together the images in a 4D light field so that objects lying on a specified surface are aligned and thus in focus, while objects lying off this surface are misaligned and hence blurred. This provides the ability to see through partial occluders such as foliage and crowds, making it a potentially powerful tool for surveillance. If the cameras lie on a plane, it has been previously shown that after an initial homography, one can move the focus through a family of planes that are parallel to the camera plane by merely shifting and adding the images. In this paper, we analyze the warps required for tilted focal planes and arbitrary camera configurations. We characterize the warps using a new rank-1 constraint that lets us focus on any plane, without having to perform a metric calibration of the cameras. We also show that there are camera configurations and families of tilted focal planes for which the warps can be factorized into an initial homography followed by shifts. This homography factorization permits these tilted focal planes to be synthesized as efficiently as frontoparallel planes. Being able to vary the focus by simply shifting and adding images is relatively simple to implement in hardware and facilitates a real-time implementation. We demonstrate this using an array of 30 video-resolution cameras; initial homographies and shifts are performed on per-camera FPGAs, and additions and a final warp are performed on 3 PCs.

 

 

   
Image from one of the cameras Synthetic aperture image, using tilted focal planes using the homography factorization.
 

Paper

Adobe Acrobat PDF (2.4 MB)

 

Video

(viewable by Windows Media Player, and Mplayer under unix)

A demonstration of real-time synthetic aperture video