Signal Processing - September 2016 - 78

Figure 10. Basal cell carcinoma as seen under a set of illumination angles. Each panel depicts the same patient at the same time. Only the illumination
angle varies, yet the structural change in appearance is significant [10].

primal-dual coding [47] uses liquid crystal displays (LCDs)
dermatology, the visual impression of the clinical dermatolofor modulation, structured light transport uses DMDs and,
gist is the main tool of initial diagnosis. A typical question in
therefor, the projection masks and pixel masks are binary.
evaluating subtle change may be: Has a mole changed by 3%
over the course of a year? A clinical practitioner may have difficulty in making such a precise quantitative assessment based
Estimating geometry, reflectance, and illumination
on observation. Figure 10 illustrates an example of the depenIn general, an image depends on object geometry (shape),
dence of skin appearance on viewing and illumination
reflectance, and illumination. If two are known, the other
direction [9]. This image set shows basal
can be measured directly. Alternatively,
cell carcinoma where each image depicts
when more than one of the elements are
An interesting paradigm is unknown, auxilliary information can be
the same skin region captured under difmachine learning affecting used for estimation. Just as binocular stereo
ferent illumination directions. Notice that
appearance capture,
the appearance changes significantly
can be used to determine geometry of a
across the image set, revealing unique surscene, photometric stereo uses multiple
driving what aspects of
face structures. For example, the raised
light sources to determine local surface
appearance are most
translucent border is only visible in a subshape in terms of local surface normals.
relevant for the task
set of the illumination directions. These
Implicit in traditional methods of photoat hand.
images are multiangle reflectance meametric stereo is a Lambertian reflectance
surements that capture skin appearance
model. Lambertian reflectance provides a
and provides a digital representation to store, share, and
simple expression for reflectance as a function of the incident
compare appearance in a computationally meaningful way.
illumination direction and the surface normal. Measured
reflectance for known illumination directions can be used to
create a set of equations to estimate the surface normal. More
Separating reflectance components
recent methods remove the restriction that the reflectance
Light in a scene originates not only from the primary source
must be Lambertian and consider unknown reflectance. More
but also from light scattering due to translucency and from
generally, recent research considers what can be simultaneinterreflections within the scene. Fast separation of direct and
ously estimated about reflectance, illumination, and shape.
indirect reflectance [40] uses high-frequency binary illuminaWhen surface shape and lighting direction are known, the
tion patterns projected on the scene to recover reflection of
reflectance can be directly measured according to the referthe light source separately from reflection due to scattering
ence coordinate frame defined by the local surface normal. In
and interreflections. This method creates two components of
this case, an image of the surface provides the reflectance and
appearance that can be used for scene analysis such as detectthe angles i i, z i, i v, z v are known with respect to the given
ing shadowed areas and areas obscured by translucent objects.
Primal-dual coding [47] also projects illumination patterns
surface normals (known shape) of the object. Recent work in
onto the scene and modulates the light capturing elements so
reflectance and illumination recovery [33] addresses the probthat there is control over which light paths (between projector
lem of estimating object reflectance and scene illumination
and camera) contribute to the image and how much they are
from a single image when the object geometry is known. Algomodulated. By controlling the relative modulations of the
rithms in this area can also predict both the surface reflectance
direct and indirect light components, this method creates a
and the surface geometry simultaneously when the illuminatuned image that can rebalance the different scene appearance
tion is known [49]. Joint reflectance and shape can be estimatcomponents. Structured light transport [46] creates a live
ed [48] when the illumination is known but uncontrolled as
video stream of indirect reflection using a high-speed imagin measured natural illumination. Estimation of a homogenous
ing system comprising a digital micromirror device (DMD)
BRDF from a single image of a known shape in unknown realprojector and DMD mask. The method projects a pattern and
world illumination can be accomplished by utilizing natural
places a modulating pixel mask at the camera. Where
image statistics [51]. In the case of known shape and unknown
78

IEEE SIgnal ProcESSIng MagazInE

September 2016

Table of Contents for the Digital Edition of Signal Processing - September 2016