Signal Processing - September 2016 - 8

special reports
John Edwards

Signal Processing Brings Fresh capabilities to Digital Imaging
Research is leading to more sophisticated and useful imaging techniques

y speeding up processing and postprocessing tasks, improving storage,
and cutting costs, digital imaging has
all but eliminated traditional film photography over the past several years. Now,
with the help of increasingly sophisticated signal processing techniques, digital
imaging is poised to become even more
flexible and useful, generating images
that would be far more difficult, if not
impossible, to create with conventional
analog photography.

On-the-go processing
Back in the days when film was the
most popular photographic imaging
medium, signs promising "same day
processing" or "60-minute processing"
could be seen almost everywhere. Most
of today's smartphone and tablet photographers are far less patient, however.
Not only do they demand to see their
photos instantly, they want to have the
ability to customize the images on the
spot to make them look more attractive
or to express a particular mood. Technology makes all of these wishes possible, but at a cost.
The seemingly inescapable drawback to mobile-device-based photo
editing is that image correction and
manipulation tasks are generally computationally intensive, capable of quickly draining a mobile device's relatively
small battery. Several mobile apps
Digital Object Identifier 10.1109/MSP.2016.2580718
Date of publication: 2 September 2016

attempt to solve this problem by sending
image files to a central server for off-site
postprocessing. Yet this approach frustrates many users by creating significant
delays as well as adding costs for
increased data usage.
Late last year, researchers from the
Massachusetts Institute of Technology
(MIT), Stanford University, and San
Jose, California-based Adobe Systems
unveiled an experimental system that
promises to slash the bandwidth consumed by server-based image-processing systems by as much as 98.5% and
mobile device power consumption by
up to 85%. The system works by sending a highly compressed version of the
image to a central server. The server, after creating simple instructions for modifying the original image, then sends an
even smaller file back to the mobile device, which uses the new data to create
the enhanced image.
Applying the modifications to the
original image does demand some extra
processing power from the phone but
not as much energy as uploading and
downloading high-resolution files. In
researchers' experiments, the energy
savings were generally between 50 and
85%, and the time savings ranged between 50 and 70% (Figure 1).
Cloud image processing is often proposed as a solution to the limited computing power and battery life of mobile
devices. "It allows complex algorithms
to run on powerful servers with virtually
unlimited energy supply," says Michaël
IEEE SIgnal ProcESSIng MagazInE

September 2016

Gharbi, a graduate student in electrical
engineering and computer science at
MIT. "Unfortunately, this overlooks the
time and energy cost of uploading the
input and downloading the output images. When transfer overhead is accounted
for, processing images on a remote server becomes less attractive, and many applications do not benefit from cloud
offloading," noted Gharbi, who was the
lead author on a recent paper describing
the system. Gharbi's coauthors were his
thesis advisor, Frédo Durand, an MIT
professor of computer science and engineering; YiChang Shih, who received
his Ph.D. degree in electrical engineering and computer science from MIT in
March; Gaurav Chaurasia, a former
postdoc in Durand's group who's now at
Disney Research; Jonathan Ragan-Kelley, who has been a postdoc at Stanford
since graduating from MIT in 2014; and
Sylvain Paris, who was a postdoc with
Durand before joining Adobe.
Gharbi describes the overall process as
a transform recipe. "The main criterion in
the design of transform recipes is that
they are compact-less data to transfer
over the network-but they should be
able to represent faithfully many image
transformations," he notes. "At equivalent
quality, our recipes are much more compact than JPEG images."
Gharbi says that the process is relatively straightforward. "First, we decompose the input image into luminance
and chrominance," he states. Luminance
characterizes the brightness variations in
1053-5888/16©2016IEEE

Table of Contents for the Digital Edition of Signal Processing - September 2016