(finish up pyramid blending from last time)


Camera calibration and tracking


  Goals of calibration
    - establish quantitative relationship between world and image
       - needed for reconstruction, matching, view interpolation, ...
    - accuracy
       - in image plane
       - relative to real parameters
    - automation
    - minimial target requirements
       - 3D targets
       - 2D targets
       - no targets (self-calibration)


  Camera models

    projective model
    graphics approach to modeling: camera transform and params
    vision approach to modeling: M = P [R t]
      P = [al_1  0   u_0 0
            0   al_2 v_0 0
            0    0    1  0]
    models for lens distortion
      radial distortion p = (1 + k|p|) p
                        p = (1 + k_1 |p| + k_2 |p|^3) p
                        p = (1 / 1 + k |p|) p


  Direct camera calibration

    input: 3D points and corresponding 2D points
    output: M
    postprocess: decompose M into P, R, t
    11 unknowns

    direct linear transform (DLT) solution [Sutherland 63 sketchpad TR]  

    more general framework: minimize cost function
      3D points accurate: one-way error
      3D points noisy
        - introduce "true" 3D points as variables
        - minimize covariance-weighted sum of 3D and 2D errors
      DLT as special case (less than ideal)
      ** use DLT as initialization for nonlinear solver

    estimating covariance

    radial distortion calibration
      - include in nonlinear minimization
      - use images of straight lines, separately

    practicalities
      depth range of points (not nearly coplanar)
      ROT: factor of 5 more points than needed (28)


  Self-calibration and bundle adjustment

    general idea of self-calibration

    bundle adjustment: general formulation
    use of B.A. to calibrate camera parameters
      - usefulness of minimizing the number of free parameters
      - generally pretty unstable with unconstrained cameras

    initializing B.A. -- Tomasi & Kanade

    medium approach between bundle adjustment and direct calibration
      - use planar target (no good for calibration from one image)
      - take images with various poses
      - solve for one set of internals and a number of poses
         - like B.A. but w/o 3D points as variables
      - go ahead and toss in radial distortion

    example results from Zhang TR


  Camera tracking

    the fundamental matrix

    estimating F from point correspondences
      - can use automatically located features 
         (if the cameras are not too far apart)
      - computation is much like camera calibration
         - x' F x = 0 gives one constraint for each pair
         - solve n by 9 system (n >= 8) with unit norm constraint
         - enforce singularity (SVD)

    robustness
      iterated outlier rejection
        example with 2D line fitting
      RANSAC
        example with 2D line fitting

    Keble College example from Hartley & Zisserman


  Bibliography
 
    Fagueras. Three-dimensional Vision: A Geometric Viewpoint.  MIT Press,
       1993.

    Hartley and Zisserman. Multiple View Geometry in Computer
       Vision. Cambridge University Press, 2000 

    Tomasi and Kanade. Shape and motion from image streams under orthography:
       A factorization approach.  IJCV 9(2):137, 1992.

    Fischler and Bolles. Random Sample Consensus: A Paradigm for Model Fitting
       with Applications to Image Analysis and Automated Cartography.  CACM
       24(6):381, 1981.