WYSIWYG Computational Photography via Viewfinder Editing

Jongmin Baek, Ph.D. Dissertation, Stanford University, December 2013.

Abstract:

The past decade witnessed a rise in the ubiquity and capability of digital photography, paced by the advances in embedded devices, image processing and social media. Along with it, the popularity of computational photography also grew. Many computational photography techniques work by first capturing a coded representation of the scene---a stack of photographs with different settings, an image obtained via a modified optical path, et cetera---and then computationally decoding it later as a post-process according to the user's specification.

However, the coded representation, available to the user at the time of capture, is often not sufficiently indicative of the decoded output that will be produced later. Depending on the type of the computational photography technique involved, the coded representation may appear to be a distorted image, or may not even be an image at all. Consequently, these techniques discard one of the most significant attractions of digital photography: the what-you-see-is-what-you-get (WYSIWYG) experience.

In response, this dissertation explores a new kind of interface for manipulating images in computational photography applications, called viewfinder editing. With viewfinder editing, the viewfinder more accurately reflects the final image the user intends to create, by allowing the user to alter the local or global appearance of the photograph via stroke-based input on a touch-enabled digital viewfinder, and propagating the edits spatiotemporally. Furthermore, the user specifies via the interface how the coded representation should be decoded in computational photography applications, guiding the acquisition and composition of photographs and giving immediate visual feedback to the user. Thus, the WYSIWYG aspect is reclaimed, enriching the user's photographing experience and helping him make artistic decisions before or during capture, instead of after capture.

This dissertation realizes and presents a real-time implementation of viewfinder editing on a mobile platform, constituting the first of its kind. This implementation is enabled by a new spatiotemporal edit propagation method that meaningfully combines and improves existing algorithms, achieving an order-of-magnitude speed-up over existing methods. The new method trades away spatial locality for efficiency and robustness against camera or scene motion.

Finally, several applications of the framework are demonstrated, such as high-dynamic-range (HDR) multi-exposure photography, focal stack composition, selective colorization, and general tonal editing. In particular, new camera control algorithms for stack metering and focusing are presented, which takes advantage of the knowledge of the user's intent indicated via the viewfinder editing interface and optimizes the camera parameters accordingly.

Dissertation:

WYSIWYG Computational Photography via Viewfinder Editing PDF (38 MB)

Related Papers:

WYSIWYG Computational Photography via Viewfinder Editing
Jongmin Baek and Dawid Pająk and Kihwan Kim and Kari Pulli and Marc Levoy
Siggraph Asia 2013, November 2013.

Focal Stack Compositing for Depth of Field Control
David E. Jacobs and Jongmin Baek and Marc Levoy
Stanford Computer Science Tech Report CSTR 2012-01, 2012.

Fast High-Dimensional Filtering Using the Permutohedral Lattice
Andrew Adams and Jongmin Baek and Abe Davis
Eurographics 2010, March 2010.


jbaek@cs.stanford.edu