IEEE Spectrum October, 2012 - 57
Items that are rated
similarly by many users form
"neighborhoods" of items.
User profiles come
from many sources.
User data
users who rate items
similarly form
neighborhoods of users.
Your data comes from items you rate,
enlarge, or look at multiple times;
items you place on wish lists; and
items you actually purchase.
the recommender
system builds a
profile of long-term
preferences.
recommenders store nearly
every action you take when
logged into a site, which raises
privacy concerns.
Recommenders can
use these different
sources of preference
information together
or separately.
navigation
recommenders follow
your path through a site.
the resulting taste signature pretty accurately-
if abstractly-represents user tastes.
For instance, if Jim and Jane each give the movie Tron five
stars, their distance is zero. If Jim then gives Tron: Legacy five
stars, while Jane rates it three stars, their distance increases.
Users whose tastes are relatively "near" each other according
to these calculations are said to share a "neighborhood."
But the user-user approach doesn't work that well. For one
thing, it's not always easy to form neighborhoods that make
sense: Many pairs of users have only a few ratings in common
or none at all, and in the case of movies, these few ratings in
common tend to be of blockbusters that nearly everyone likes.
Also, because the distance between users can change rapidly,
user-user algorithms have to do most of their calculations on
the spot, and that can take more time than someone clicking
around a website is going to hang around.
So most recommenders today rely on an "item-item" algorithm, which calculates the distance between each pair of books
or movies or what have you according to how closely users who
have rated them agree. People who like books by Tom Clancy are
likely to rate books by Clive Cussler highly, so books by Clancy
and Cussler are in the same neighborhood. Distances between
pairs of items, which may be based on the ratings of thousands or
millions of users, tend to be relatively stable over time, so recommenders can precompute distances and generate recommendations more quickly. Both Amazon and Netflix have said publicly
that they use variants of an item-item algorithm, though they
keep the details secret.
One problem with both user-user and item-item algorithms
is the inconsistency of ratings. Users often do not rate the same
item the same way if offered the chance to rate it again. Tastes
spectrum.ieee.org
change, moods change, memories fade. MIT conducted one
study in the late 1990s that showed an average change of one
point on a seven-point scale a year after a user's original rating. Researchers are trying different ways to incorporate
such variables into their models; for example, some recommenders will ask users to rerate items when their original ratings seem out of sync with everything else the recommender
knows about them.
But the user-user and item-item algorithms have a bigger
problem than consistency: They're too rigid. That is, they can
spot people who prefer the same item but then miss potential pairs who prefer very similar items. Let's say you're a fan
of Monet's water lily paintings. Of the 250 or so paintings of
water lilies that the French impressionist did, which is your
favorite? Among a group of Monet fans, each person may like
a different water lily painting best, but the basic algorithms
might not recognize their shared taste for Monet.
About a decade ago, researchers figured out a way to factor
in such sets of similar items-a process called dimensionality
reduction. This method is much more computationally intensive
than the user-user and item-item algorithms, so its adoption has
been slower. But as computers have gotten faster and cheaper, it
has been gaining ground.
To understand how dimensionality reduction works, let's
consider your taste in food and how it compares with that of
a million other people. You can represent those tastes in a
huge matrix, where each person's taste makes up its own row
and each of the thousands of columns is a different food. Your
row might show that you gave grilled filet mignon five stars,
october 2012 * Ieee Spectrum * NA
57
http://spectrum.ieee.org
Table of Contents for the Digital Edition of IEEE Spectrum October, 2012
IEEE Spectrum October, 2012 - Cover1
IEEE Spectrum October, 2012 - Cover2
IEEE Spectrum October, 2012 - 1
IEEE Spectrum October, 2012 - 2
IEEE Spectrum October, 2012 - 3
IEEE Spectrum October, 2012 - 4
IEEE Spectrum October, 2012 - 5
IEEE Spectrum October, 2012 - 6
IEEE Spectrum October, 2012 - 7
IEEE Spectrum October, 2012 - 8
IEEE Spectrum October, 2012 - 9
IEEE Spectrum October, 2012 - 10
IEEE Spectrum October, 2012 - 11
IEEE Spectrum October, 2012 - 12
IEEE Spectrum October, 2012 - 13
IEEE Spectrum October, 2012 - 14
IEEE Spectrum October, 2012 - 15
IEEE Spectrum October, 2012 - 16
IEEE Spectrum October, 2012 - 17
IEEE Spectrum October, 2012 - 18
IEEE Spectrum October, 2012 - 19
IEEE Spectrum October, 2012 - 20
IEEE Spectrum October, 2012 - 21
IEEE Spectrum October, 2012 - 22
IEEE Spectrum October, 2012 - 23
IEEE Spectrum October, 2012 - 24
IEEE Spectrum October, 2012 - 25
IEEE Spectrum October, 2012 - 26
IEEE Spectrum October, 2012 - 27
IEEE Spectrum October, 2012 - 28
IEEE Spectrum October, 2012 - 29
IEEE Spectrum October, 2012 - 30
IEEE Spectrum October, 2012 - 31
IEEE Spectrum October, 2012 - 32
IEEE Spectrum October, 2012 - 33
IEEE Spectrum October, 2012 - 34
IEEE Spectrum October, 2012 - 35
IEEE Spectrum October, 2012 - 36
IEEE Spectrum October, 2012 - 37
IEEE Spectrum October, 2012 - 38
IEEE Spectrum October, 2012 - 39
IEEE Spectrum October, 2012 - 40
IEEE Spectrum October, 2012 - 41
IEEE Spectrum October, 2012 - 42
IEEE Spectrum October, 2012 - 43
IEEE Spectrum October, 2012 - 44
IEEE Spectrum October, 2012 - 45
IEEE Spectrum October, 2012 - 46
IEEE Spectrum October, 2012 - 47
IEEE Spectrum October, 2012 - 48
IEEE Spectrum October, 2012 - 49
IEEE Spectrum October, 2012 - 50
IEEE Spectrum October, 2012 - 51
IEEE Spectrum October, 2012 - 52
IEEE Spectrum October, 2012 - 53
IEEE Spectrum October, 2012 - 54
IEEE Spectrum October, 2012 - 55
IEEE Spectrum October, 2012 - 56
IEEE Spectrum October, 2012 - 57
IEEE Spectrum October, 2012 - 58
IEEE Spectrum October, 2012 - 59
IEEE Spectrum October, 2012 - 60
IEEE Spectrum October, 2012 - 61
IEEE Spectrum October, 2012 - 62
IEEE Spectrum October, 2012 - 63
IEEE Spectrum October, 2012 - 64
IEEE Spectrum October, 2012 - 65
IEEE Spectrum October, 2012 - 66
IEEE Spectrum October, 2012 - 67
IEEE Spectrum October, 2012 - 68
IEEE Spectrum October, 2012 - 69
IEEE Spectrum October, 2012 - 70
IEEE Spectrum October, 2012 - 71
IEEE Spectrum October, 2012 - 72
IEEE Spectrum October, 2012 - 73
IEEE Spectrum October, 2012 - 74
IEEE Spectrum October, 2012 - 75
IEEE Spectrum October, 2012 - 76
IEEE Spectrum October, 2012 - 77
IEEE Spectrum October, 2012 - 78
IEEE Spectrum October, 2012 - 79
IEEE Spectrum October, 2012 - 80
IEEE Spectrum October, 2012 - Cover3
IEEE Spectrum October, 2012 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1217
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1117
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1017
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0917
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0817
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0717
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0617
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0517
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0417
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0317
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0217
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0117
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1216
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1116
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1016
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0916
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0816
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0716
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0616
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0516
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0416
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0316
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0216
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0116
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1215
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1115
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1015
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0915
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0815
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0715
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0615
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0515
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0415
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0315
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0215
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0115
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1214
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1114
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1014
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0914
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0814
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0714
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0614
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0514
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0414
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0314
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0214
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0114
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1213
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1113
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1013
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0913
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0813
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0713
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0613
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0513
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0413
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0313
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0213
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0113
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1212
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1112
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1012
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0912
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0812
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0712
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0612
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0512
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0412
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0312
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0212
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0112
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1211
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1111
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1011
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0911
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0811
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0711
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0611
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0511
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0411
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0311
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0211
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0111
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1210
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1110
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1010
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0910
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0810
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0710
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0610
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0510
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0410
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0310
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0210
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0110
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1209
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1109
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1009
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0909
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0809
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0709
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0609
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0509
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0409
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0309
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0209
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0109
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1208
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1108
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1008
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0908
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0808
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0708
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0608
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0508
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0408
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0308
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0208
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0108
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1207
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1107
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1007
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0907
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0807
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0707
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0607
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0507
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0407
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0307
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0207
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0107
https://www.nxtbookmedia.com