IEEE Spectrum October, 2012 - 57

Items that are rated
similarly by many users form
"neighborhoods" of items.

User profiles come
from many sources.

User data
users who rate items
similarly form
neighborhoods of users.

Your data comes from items you rate,
enlarge, or look at multiple times;
items you place on wish lists; and
items you actually purchase.

the recommender
system builds a
profile of long-term
preferences.
recommenders store nearly
every action you take when
logged into a site, which raises
privacy concerns.

Recommenders can
use these different
sources of preference
information together
or separately.

navigation
recommenders follow
your path through a site.

the resulting taste signature pretty accurately-
if abstractly-represents user tastes.

For instance, if Jim and Jane each give the movie Tron five
stars, their distance is zero. If Jim then gives Tron: Legacy five
stars, while Jane rates it three stars, their distance increases.
Users whose tastes are relatively "near" each other according
to these calculations are said to share a "neighborhood."
But the user-user approach doesn't work that well. For one
thing, it's not always easy to form neighborhoods that make
sense: Many pairs of users have only a few ratings in common
or none at all, and in the case of movies, these few ratings in
common tend to be of blockbusters that nearly everyone likes.
Also, because the distance between users can change rapidly,
user-user algorithms have to do most of their calculations on
the spot, and that can take more time than someone clicking
around a website is going to hang around.
So most recommenders today rely on an "item-item" algorithm, which calculates the distance between each pair of books
or movies or what have you according to how closely users who
have rated them agree. People who like books by Tom Clancy are
likely to rate books by Clive Cussler highly, so books by Clancy
and Cussler are in the same neighborhood. Distances between
pairs of items, which may be based on the ratings of thousands or
millions of users, tend to be relatively stable over time, so recommenders can precompute distances and generate recommendations more quickly. Both Amazon and Netflix have said publicly
that they use variants of an item-item algorithm, though they
keep the details secret.
One problem with both user-user and item-item algorithms
is the inconsistency of ratings. Users often do not rate the same
item the same way if offered the chance to rate it again. Tastes
spectrum.ieee.org

change, moods change, memories fade. MIT conducted one
study in the late 1990s that showed an average change of one
point on a seven-point scale a year after a user's original rating. Researchers are trying different ways to incorporate
such variables into their models; for example, some recommenders will ask users to rerate items when their original ratings seem out of sync with everything else the recommender
knows about them.
But the user-user and item-item algorithms have a bigger
problem than consistency: They're too rigid. That is, they can
spot people who prefer the same item but then miss potential pairs who prefer very similar items. Let's say you're a fan
of Monet's water lily paintings. Of the 250 or so paintings of
water lilies that the French impressionist did, which is your
favorite? Among a group of Monet fans, each person may like
a different water lily painting best, but the basic algorithms
might not recognize their shared taste for Monet.
About a decade ago, researchers figured out a way to factor
in such sets of similar items-a process called dimensionality
reduction. This method is much more computationally intensive
than the user-user and item-item algorithms, so its adoption has
been slower. But as computers have gotten faster and cheaper, it
has been gaining ground.
To understand how dimensionality reduction works, let's
consider your taste in food and how it compares with that of
a million other people. You can represent those tastes in a
huge matrix, where each person's taste makes up its own row
and each of the thousands of columns is a different food. Your
row might show that you gave grilled filet mignon five stars,
october 2012 * Ieee Spectrum * NA

http://spectrum.ieee.org

Table of Contents for the Digital Edition of IEEE Spectrum October, 2012