Signal Processing - September 2016 - 79

illumination, the motion a rotating object can also enable
recovery of the object BRDF. [16]. For most of these estimation methods, results are evaluated by comparing renderings
to ground truth. That is, the fidelity of the representation when
compared to ground truth is the performance metric.

Hiding information within appearance
When considering appearance as measured by reflectance,
novel opportunities arise to hide information in the angular
space of reflected light. In this context, the high dimensionality of the local appearance is an advantage. Using changes of
refractive index, [13] presents a unique method to create
transparent watermarks that are hidden to the human eye but
can be detected by a multiview camera. By encoding a watermark with different refractive indices, the minimum reflected
intensity as defined by the Brewster angle occurs at a different angle for different spatial regions of the watermark. This
subtle effect cannot be visualized unless multiple angles are
viewed simultaneously as done with a curved mirror-based
reflectance capture device. Another approach to hiding information using reflectance angular space is Bokode [37], where
a single image can capture a hidden message embedded as a
spatial pattern and a lenslet. The key idea is that refocusing
the image from the captured reflectance reveals the message.
Photographic steganography [64] refers to hiding information
in electronically displayed images that can be decoded with a
camera receiver either using intensity [5], [61], [64], high-frequency modulation [28], [43], or color [60].

Conclusions
Measuring apparatus for appearance has evolved past ordinary
cameras to devices for more general light capture. Specialized
cameras, devices, and algorithms can more fully capture patterns of light from a scene. Dense sampling of reflectance with
robotic devices and light/camera domes has evolved to computational imaging procedures for faster capture in a more compact device and a more compact representation. A large
majority of the appearance capture methods make improvements that are generically useful (e.g., faster, more compact,
more complete). An open research topic for future trends is tuning computational appearance for a particular task, creating a
feedback from application/algorithm to capture method. For
example, the image signal processors within a camera that handle demosaicing, denoising, and other image processing tasks
have been optimized in an end-to-end manner [24] to optimize
image quality. There are many more possibilities in the concept
of combining optimization and appearance capture. Consider
machine-learning methods such as deep learning and convolutional neural networks that can use appearance measurements
as input. An interesting paradigm is machine learning affecting
appearance capture, driving what aspects of appearance are
most relevant for the task at hand. Since the computational
camera is a tunable device, the question of how to tune it leads
to new cost functions that balance algorithm requirements,
hardware constraints, physical and optical constraints, and
application goals. Future trends for appearance capture are not

replicating what the eye would see, but rather tuning the sampling of light to meet what an algorithm requires.

Author
Kristin J. Dana (kdana@ece.rutgers.edu) received her B.S.
degree in 1990 from the Cooper Union, New York; her M.S.
degree from the Massachusetts Institute of Technology in
1992; and her Ph.D. degree from Columbia University, New
York, in 1999. She is a full professor in the Department of
Electrical and Computer Engineering at Rutgers University.
Her research interests in computer vision include computational photography, machine learning, illumination modeling,
texture and reflectance, bioimaging, motion estimation, optical devices, optimization in vision, and applications of robotics. She is the recipient of the 2001 National Science
Foundation Career Award and a team recipient of the Charles
Pankow Innovation Award in 2014 from the American Society
of Civil Engineers.

References

[1] E. H. Adelson and J. R. Bergen, Computational Models of Visual Processing.
Cambridge, MA: MIT Press, 1991, pp. 3-20.
[2] M. Aittala, T. Weyrich, and J. Lehtinen, "Practical SVBRDF capture in the frequency domain," ACM Trans. Graph., vol. 32, no. 4, pp. 110:1-110:12, July 2013.
[Online]. Available: http://doi.acm.org/10.1145/2461912.2461978
[3] M. Aittala, T. Weyrich, and J. Lehtinen, "Two-shot SVBRDF capture for stationary materials," ACM Trans. Graph., vol. 34, no. 4, pp. 110:1-110:13, July 2015.
[Online]. Available: http://doi.acm.org/10.1145/2766967
[4] M. A. Ali, I. Sato, T. Okabe, and Y. Sato, "Toward efficient acquisition of
BRDFs with fewer samples," in Computer Vision-ACCV 2012, New York:
Springer, 2013, pp. 54-67.
[5] A. Ashok, S. Jain, M. Gruteser, N. Mandayam, W. Yuan, and K. Dana,
"Capacity of screen-camera communications under perspective distortions,"
Pervasive Mob. Comput., vol. 16, pp. 239-250, Jan. 2015.
[6] M. Ben-Ezra, J. Wang, B. Wilburn, X. Li, and L. Ma, "An LED-only BRDF
measurement device," in Proc. IEEE Conf. Computer Vision and Pattern
Recognition, 2008, pp. 1-8.
[7] J. F. Blinn and B. T. Phong, "Models of light reflection for computer synthesized
pictures," in Proc. 4th Annu. Conf. Computer Graphics and Interactive
Techniques, SIGGRAPH '77, 1977, pp. 192-198.
[8] O. G. Cula and K. J. Dana, "Compact representation of bidirectional texture
functions," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001,
vol. 1, pp. 1041-1067.
[9] O. G. Cula, K. J. Dana, and D. K. Pai, and D. Wang, "Polarization multiplexing
for bidirectional imaging," in Proc. IEEE Conf. Computer Vision and Pattern
Recognition, June 2005, pp. 1116-1123.
[10] O. G. Cula, K. J. Dana, F. P. Murphy, and B. K. Rao, "Bidirectional imaging
and modeling of skin texture," IEEE Trans. Biomed. Eng., vol. 51, no. 12, pp. 2148-
2159, Dec. 2004.
[11] K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink, "Reflectance
and texture of real-world surfaces," Columbia University, New York, Tech. Rep.
CUCS-048-96, Dec. 1996.
[12] K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink, "Reflectance
and texture of real world surfaces," ACM Trans. Graph., vol. 18, no. 1, pp. 1-34,
Jan. 1999.
[13] K. J. Dana, G. Livescu, and R. Makonahalli, "Transparent watermarking using
bidirectional imaging," in Proc. IEEE Computer Society Conf. Computer Vision
and Pattern Recognition Workshops, 2009, pp. 31-38.
[14] K. J. Dana and J. Wang, "Device for convenient measurement of spatially
varying bidirectional reflectance," J. Optical Soc. Am. A, vol. 21, pp. 1-12, Jan.
2004.
[15] P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar.
"Acquiring the reflectance field of a human face," in Proc. 27th Annu. Conf.
Computer Graphics and Interactive Techniques, SIGGRAPH '00, New York,
2000, pp. 145-156.
[16] Y. Dong, C. Guojun, P. Pieter, Z. Jiawan, and T. Xin, "Appearance-frommotion: Recovering spatially varying surface reflectance under unknown lighting,"
ACM Trans. Graph., vol. 33, no. 6, pp. 193, 2014.

IEEE SIgnal ProcESSIng MagazInE

September 2016

http://doi.acm.org/10.1145/2461912.2461978 http://doi.acm.org/10.1145/2766967

Table of Contents for the Digital Edition of Signal Processing - September 2016