Panoramas and Omnidirectional imaging

1. Motivation

  - General goal: omnidirectional images
     - set of samples of rays through point
     - cover entire directional sphere uniformly
 
  - Implications for photography
     - photos of an entire landscape
         (mars pathfinder)
     - photos of an enclosed space
         http://www.apple.com/quicktime/products/gallery/

  - Implications for vision
     - No more rotation/translation ambiguity
     - Much more robust calibration, reconstruction, localization

  - Recall from lens design lecture:
     - lenses naturally perform well for narrow FOV
     - increasing FOV leads to increasing cost, compromises
     - wide angle perspective images have limited usefulness
     - special lenses (fisheye) can achieve wide FOV
        - but low resolution
        - but high cost (nonstandard, much glass)
           Nikon 50mm/1.4 : $265
           Nikon 50mm/1.8 : $110
           Nikon 16mm/2.8 fisheye : $750



2. Simultaneous-exposure omnidirectionality

  - Catadioptric systems

     - mirror / lens optical systems
     - old technique: mirror ball for environment maps
     - general idea: point perspective lens at curved mirror

     - derivation of mirror shapes
        - want to focus rays through one point to another point
        - turns out conics are the whole story
	   - plane: duh
           - sphere, cone: deg
           - ellipsoid, hyperboloid: useful
        - example: Columbia omnicam

     - produces annular image
     - can be resampled locally to perspecive if desired

  - Camera arrays

     - try to pack for SVP
     - or use mirrors for SVP
           
        



3. Sequential-exposure omnidirectionality

  - Slit cameras

     - panoramic cameras
        - film
        - digital

     - fun games with non-SVP
        - stereo off-center panoramas
           - slit scan with off-center rotation
           - forward and back
           - any point is seen by two tangents
           - disparity is then 180 - angle
        - concentric mosaics
           - same idea but 1D family of axis distances
           - can capture using one camera (or small number)
           - 3D light-field-like object, can be resampled for
              planar set of camera positions

  - Image mosaics

     - overview: register, blend, resample

     - registration
        - parameterized by allowed motions
           - 1D translation (cylindrical panorama) [2 in practice]
           - 2D translation & rotation (satellite / aerial mosaics)
           - 8D projective motion (arbitrary camera motion)
           - 3D camera rotation (projective for fixed viewpoint)
           - many-D spline warp
        - optimization
           - generally gradient descent on pixelwise errors
              - alternative is feature matching (not usual though)
           - easier than stereo because images actually do match
           - use coarse-to-fine approach
        - global registration
           - errors will accumulate
           - solve global system
           - solve for calibration parameters too if you want
              - e.g. focal length
        - initialization
           - human places photos approximately
           - human takes photos in a regular way
	      - "stitch assist" mode

     - blending
        - simple abutment
        - feathering
        - pyramid blending
        - deghosting

     - resampling
        - onto cylinder
           - good for panning but not much else 
           - does not see poles
        - into spherical coords
           - not terribly convenient for resampling
        - onto cube
           - resample with texmap hardware
        - or other polygon mesh