Signal Processing - September 2016 - 131

documents, assesses, interprets, and conserves culturally signifiassess style and attribution of an artwork from aspects of the
cant artifacts housed in museums around the world.
painted surface not visible to the naked eye [1]. This trend
continued with other wavelengths of illumination. Specifically, ultraviolet (UV)-induced visible fluorescence helped
Multispectral and hyperspectral imaging
reveal areas of loss/repair and provided a general sense of
Human eyes only perceives visible light (380 nm + 750 nm) with
chemical composition [2]. By the late 1960s, infrared (IR)
three types of color-sensitive cones: "red," "green," and "blue."
reflectography was in routine use in museMultispectral and hyperspectral techniques
ums to reveal hidden underdrawings and
extend the measurable spectrum from visiNew ways of engaging
preparatory marking in paintings [3]. More
ble light to UV (10 nm + 380 nm) and IR
with objects from our
specialized techniques, such as autoradiog(750 nm + 1 mm) lights with increased resoshared cultural heritage
raphy achieved by neutron bombardment of
lution: typically multispectral imagery has
a work of art, opened up the possibility of
three to ten bands, while hyperspectral
are possible with
combining elemental composition together
imagery could have hundreds or even thouadvances in computation
with imaging for the first time [4]-a techsands of narrower (e.g., 10 nm) bands. Multiand imaging that allow
nique that would inspire further developspectral and hyperspectral imaging provide
scientists to analyze art
ments in X-ray fluorescence imaging
a wealth of information across space and
noninvasively, historians
several decades later [5]. By the 1980s, new
wavelengths comprising large swaths of the
to pose new social
three-dimensional (3-D) acquisition techelectromagnetic spectrum. The techniques
niques were being explored for the 3-D docare also flexible since they can be scaled
questions about the
umentation and display of cultural objects,
from the imaging of landscapes, when used
art, and the public to
both of which remain relevant subjects of
on satellites and telescopes, down to the
explore and interact
investigation to this day [6].
microscopic. Also, importantly, these imagwith art in ways never
The recent explosion in imaging of culing spectroscopies are nondestructive under
before possible.
tural heritage has grown mainly out of the
the normal conditions of their implementafields of remote sensing and color science.
tion. Liang's recent review [17] should be
Of particular note is the use of hyperspectral and multispectral
consulted for developments in the field through 2012, but a brief
imaging instruments for pixel-by-pixel material characterizaintroduction is provided here.
tion [7]. A parallel development has been the use of synchroThere are typically three principal ways of obtaining multi/
tron-based X-ray fluorescence and diffraction imaging that has
hyperspectral data sets:
grown in conjunction with the diversification of users of these
1) imaging the entire object at once through a series of differlarge-scale facilities from all research disciplines. Around the
ent filters (or through a single filter whose bandpass charsame time, computational illumination techniques were develacteristics may be controlled), e.g., in 2004, Lumiere
oped to dynamically relight works of art in postcapture [8],
Technology (http://www.lumiere-technology.com/Pages/
[9]. With the advent of inexpensive digital scanners, several
Services/services2.htm) used 13 filters from UV to IR and
researchers have focused on digitization of existing X-radioa 240-megapixel camera to image the famous "Mona Lisa"
graphs of canvas paintings, enabling recent advances in image
in the Louvre Museum
processing algorithms to be applied to these historical works,
2) scanning a linear slit view of the object through a grating
such as in the canvas weave project initiated at Cornell Unithat spreads the relevant spectral region onto a two-dimenversity [10], [11]. The proliferation of inexpensive digital sensional (2-D) sensing array
sors have been allowing museums to capture large amounts of
3) scanning the entire object point by point across its x-y surhigh-resolution photographs in multiple modalities that were
face [18].
then computationally stitched together to provide seamless
Aspects of cost, time, and instrumental design parameters
image mosaics with unprecedented detail [12]. Optical coherwill dictate the choice of image acquisition method.
ence tomography [13] and THz imaging [14] provide in-situ 3-D
The resulting x-y surface images are "stacked" as a function
reconstructions of microscopically thin layers of paint comprisof wavelength thus creating an image cube that may be intering pictures and drawings. At the other extremes of scale, the
rogated in two ways. If the cube is "sliced" parallel to the x-y
popularity of both light detection and ranging (LiDAR) [15] and
image face, one can analyze each of the images taken at each
structure-from-motion (SfM) techniques [16] have allowed us to
wavelength. Such an analysis at infrared wavelengths might
search for ancient cities and document the historic landscapes of
readily provide an image of an underdrawing beneath the surmodern ones.
face of a painting. If the stack is rotated by 90° degrees to x-y
In this article, recent developments are discussed in four core
image face of the cube, one can obtain the detected spectrum
areas that have served to advance the field of cultural heritage
at every pixel. Because chemical components have distinguishinto new territory: multispectral and hyperspectral imaging,
able spectral responses, multivariate statistical methods such
3-D shape scanning and recovery, image relighting, and macro
as principal component analysis (PCA) can provide informaX-ray imaging. Key developments in each of these areas have
tion on the spatial distribution of different materials. Examindramatically changed the landscape of how one noninvasively
ing the PCA images, spectral angle maps of end members, or
IEEE SIgnal ProcESSIng MagazInE

|

September 2016

|

131
http://www.lumiere-technology.com/Pages/
Table of Contents for the Digital Edition of Signal Processing - September 2016
