IEEE Spectrum October, 2012 - 60

That example may sound simplistic,
but in one of our early experiences, our
system kept recommending the Beatles'
"White Album" to nearly every visitor.
Statistically this was a great recommendation: The customers had never purchased this item from the e-commerce
site, and most customers rated it highly.
And yet the recommendation was useless. Everyone who was interested in the
"White Album" already owned a copy.
Most recommender rules are more
subtle, of course. When John recently
searched for an action movie on Netflix,
for instance, he wasn't offered The
Avengers, because the blockbuster was
not yet available for rental, and so the suggestion wouldn't have profited Netflix.
Instead it steered him to Iron Man 2,
which was available for streaming.
Other business rules prevent recommenders from suggesting loss leaders-
products that sell below cost to draw
people into the site-or conversely
encourage them to recommend products that are overstocked. During our
time at Net Perceptions, we worked
with a client who did just that: He used
his recommender system to identify-
with considerable success-potential
customers for his overstocked goods.
This kind of thing quickly gets
tricky, however. A system that simply pushes high-margin products isn't
going to earn the customers' trust. It's
like going to a restaurant where the
waiter steers you toward a particular
fish dish. Is it really his favorite? Or did
the chef urge the staff to push out the
fish before its sell-by date?
To build trust, the more sophisticated
recommender systems strive for some
degree of transparency by giving customers an idea of why a particular item
was recommended and letting them correct their profiles if they don't like the recommendations they're getting.
You can, for instance, delete information from your Amazon prof ile
about things you purchased as gifts;
after all, those don't ref lect your
tastes. You can also f ind out why
certain products have been offered
t hrough t he recom mender. A f ter
Amazon selected Jonathan Franzen's
novel Freedom for John, he clicked on
the link labeled "Explain." He then got
a brief explanation that certain books
on John's Amazon wish list had triggered the recommendation. But as
John hadn't read any of the wish list
books, he discounted the Freedom
spectrum.ieee.org

suggestion. Explanations like these
let users know how reliable a given
recommendation is.
But profile adjustments and explanations often aren't enough to keep a
system on track. Recently Amazon
bombarded Joe with e-mails for largescreen HDTVs-as many as three a
week for months. Besides sending
him more e-mail on the topic than
he could possibly want, the retailer
didn't recognize that he'd already
purchased a TV through his wife's
account. What's more, the e-mails
did not offer an obvious way for Joe to
say, "Thanks, but I'm not interested."
Eventually, Joe unsubscribed from
certain Amazon e-mails; he doesn't
miss the messages, and he has more
time to actually watch that TV.
sO HOw well do recommenders
ultimately work? They certainly are
increasing online sales; analyst Jack
Aaronson of the Aaronson Group
estimates that investments in recommenders bring in returns of 10 to
30 percent, thanks to the increased
sales they drive. And they still have a
long way to go.
Right now the biggest challenge for
those of us who study recommender systems is to figure out how best to judge the
new approaches and algorithms. It's not
as simple as benchmarking a microprocessor, because different recommenders
have very different goals.
The easiest way to evaluate an
algorithm is to look at the difference
between its predictions and the actual
ratings users give. For instance, if John
gives the teen-romance novel Twilight
one star, Amazon might note that it
had predicted he would give it two
stars, based on the ratings of other similar users, and so its recommender was
off by a star. But sellers care much more
about errors on highly rated items than
errors on low-rated items, because the
highly rated items are the ones users are
more likely to buy; John is never going
to purchase Twilight, so scoring this rating contributes little to understanding
how well the recommender works.
Another common measure is the
extent to which recommendations
match actual purchases. This analysis can also be misleading, however,
because it erroneously rewards the recommender for items users managed to
find on their own-precisely the items
they don't need recommendations for!

http://spectrum.ieee.org

Table of Contents for the Digital Edition of IEEE Spectrum October, 2012