IEEE Spectrum November, 2013 - 53
and contexts. for instance, locating all [of experience] was zero," says lex Machina's former chief technolcases related to patent infringement was complicated by the fact that ogy officer, Mihai surdeanu, who had previously worked on naturalthe exact term didn't always appear in a document's text. "It was
language processing at yahoo labs. "there was nobody doing this."
a matter of translating upwards and understanding the concept of
existing machine-learning techniques don't work very well on
infringement regardless of the words they were using," Manning says. litigation data, he says. even a relatively simple process like normalanother difficulty the researchers encountered was that each court izing names poses a challenge. the computer has to recognize, for
website uses its own variant of electronic filing. they therefore had to
example, that IBM, International Business Machines, and IBM Corp.
design a web crawler for each one. once collected, the data then had
all refer to the same company. More problematic were law firm
to be standardized to account for the variations in the way courts file
names, because firms sometimes change their names when they
data. and in many instances, the data were just plain wrong; in more
merge with other firms or when partners join or leave. even a firm's
than half of all cases, the final decision in the case had been incorrectly
own attorneys can get the name wrong, surdeanu says. "one of
coded, according to walker, who served as lex Machina's first Ceo
the firms in our database has 89 different legal spellings," he adds.
and is now an attorney at the law firm simpson thacher & bartlett.
the system must also be able to handle complex legal constructs.
the way the cases were tagged-as patent, copyright, or trademark unlike baseball, which is a numbers game, the legal world is based
infringement, for example-was also often wrong. and there were no
on qualitative information, subtle distinctions, and most of all, words.
tags for certain types of cases, such as those involving trade secrets. "People argue about the meanings of words and make arguments
the researchers had to manually sort through, categorize, and with paragraphs of text," explains Manning. the machine needs to
correct the data. "there were hundreds of thousands of legal judg- understand phrases and strings of commonly used legal language as
ments that had to be made" as they sifted through the informa- well as context so that it can tell the difference between, for examtion, walker says. In total, it took the team about 100 000 hours.
ple, the summary judgment document (in which a judge determines
once the team had cleaned up the data and understood its many which party wins the case or at least certain issues in the case) and a
complexities, the engineers designed algorithms to automatically minor procedural filing that simply mentions the summary judgment.
review each document and sort the results. similar algorithms are
to help parse the legalese, lex Machina has developed a set of rules-
used in web searches, but interpreting legal documents requires
a sort of legal grammar for the machine. the company does this
more sophistication. "from a science perspective, the baseline
through an iterative process: a legal analyst reviews the algorithms'
continued froM page 34 |
40
th
AnnIvErsAry
of
IEEE Foundation
Inspiring the Future
Donate and Enable the Impact of IEEE through IEEE Foundation
EducATIon * InnoVATIon * PrEsErVATIon
Your generous donations motivate students and young professionals,
enable innovators to make a difference, promulgate technology's influence
on the world and inspire the future.
IEEE Foundation
Dedicated to providing philanthropic services to support the core purpose
of IEEE-Advancing Technology For Humanity.
Visit ieeefoundation.org to learn more.
Be an inspiration. donate Today. ieee.org/donate
13-FND-0046h-IEEE-Foundation-Ad-7x4.75-FINAL.indd 1
SPECTRUM.IEEE.ORG
|
nORTh aMERICan
|
3/20/13 3:39 PM
nOv 2013 | 53
http://www.ieeefoundation.org
http://www.ieee.org/donate
http://SPECTRUM.IEEE.ORG
Table of Contents for the Digital Edition of IEEE Spectrum November, 2013
IEEE Spectrum November, 2013 - Cover1
IEEE Spectrum November, 2013 - Cover2
IEEE Spectrum November, 2013 - 1
IEEE Spectrum November, 2013 - 2
IEEE Spectrum November, 2013 - 3
IEEE Spectrum November, 2013 - 4
IEEE Spectrum November, 2013 - 5
IEEE Spectrum November, 2013 - 6
IEEE Spectrum November, 2013 - 7
IEEE Spectrum November, 2013 - 8
IEEE Spectrum November, 2013 - 9
IEEE Spectrum November, 2013 - 10
IEEE Spectrum November, 2013 - 11
IEEE Spectrum November, 2013 - 12
IEEE Spectrum November, 2013 - 13
IEEE Spectrum November, 2013 - 14
IEEE Spectrum November, 2013 - 15
IEEE Spectrum November, 2013 - 16
IEEE Spectrum November, 2013 - 17
IEEE Spectrum November, 2013 - 18
IEEE Spectrum November, 2013 - 19
IEEE Spectrum November, 2013 - 20
IEEE Spectrum November, 2013 - 21
IEEE Spectrum November, 2013 - 22
IEEE Spectrum November, 2013 - 23
IEEE Spectrum November, 2013 - 24
IEEE Spectrum November, 2013 - 25
IEEE Spectrum November, 2013 - 26
IEEE Spectrum November, 2013 - 27
IEEE Spectrum November, 2013 - 28
IEEE Spectrum November, 2013 - 29
IEEE Spectrum November, 2013 - 30
IEEE Spectrum November, 2013 - 31
IEEE Spectrum November, 2013 - 32
IEEE Spectrum November, 2013 - 33
IEEE Spectrum November, 2013 - 34
IEEE Spectrum November, 2013 - 35
IEEE Spectrum November, 2013 - 36
IEEE Spectrum November, 2013 - 37
IEEE Spectrum November, 2013 - 38
IEEE Spectrum November, 2013 - 39
IEEE Spectrum November, 2013 - 40
IEEE Spectrum November, 2013 - 41
IEEE Spectrum November, 2013 - 42
IEEE Spectrum November, 2013 - 43
IEEE Spectrum November, 2013 - 44
IEEE Spectrum November, 2013 - 45
IEEE Spectrum November, 2013 - 46
IEEE Spectrum November, 2013 - 47
IEEE Spectrum November, 2013 - 48
IEEE Spectrum November, 2013 - 49
IEEE Spectrum November, 2013 - 50
IEEE Spectrum November, 2013 - 51
IEEE Spectrum November, 2013 - 52
IEEE Spectrum November, 2013 - 53
IEEE Spectrum November, 2013 - 54
IEEE Spectrum November, 2013 - 55
IEEE Spectrum November, 2013 - 56
IEEE Spectrum November, 2013 - 57
IEEE Spectrum November, 2013 - 58
IEEE Spectrum November, 2013 - 59
IEEE Spectrum November, 2013 - 60
IEEE Spectrum November, 2013 - 61
IEEE Spectrum November, 2013 - 62
IEEE Spectrum November, 2013 - 63
IEEE Spectrum November, 2013 - 64
IEEE Spectrum November, 2013 - 65
IEEE Spectrum November, 2013 - 66
IEEE Spectrum November, 2013 - 67
IEEE Spectrum November, 2013 - 68
IEEE Spectrum November, 2013 - 69
IEEE Spectrum November, 2013 - 70
IEEE Spectrum November, 2013 - 71
IEEE Spectrum November, 2013 - 72
IEEE Spectrum November, 2013 - 73
IEEE Spectrum November, 2013 - 74
IEEE Spectrum November, 2013 - 75
IEEE Spectrum November, 2013 - 76
IEEE Spectrum November, 2013 - 77
IEEE Spectrum November, 2013 - 78
IEEE Spectrum November, 2013 - 79
IEEE Spectrum November, 2013 - 80
IEEE Spectrum November, 2013 - 81
IEEE Spectrum November, 2013 - 82
IEEE Spectrum November, 2013 - 83
IEEE Spectrum November, 2013 - 84
IEEE Spectrum November, 2013 - Cover3
IEEE Spectrum November, 2013 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1217
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1117
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1017
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0917
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0817
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0717
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0617
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0517
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0417
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0317
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0217
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0117
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1216
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1116
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1016
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0916
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0816
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0716
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0616
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0516
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0416
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0316
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0216
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0116
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1215
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1115
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1015
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0915
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0815
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0715
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0615
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0515
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0415
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0315
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0215
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0115
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1214
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1114
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1014
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0914
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0814
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0714
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0614
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0514
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0414
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0314
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0214
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0114
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1213
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1113
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1013
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0913
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0813
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0713
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0613
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0513
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0413
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0313
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0213
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0113
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1212
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1112
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1012
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0912
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0812
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0712
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0612
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0512
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0412
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0312
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0212
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0112
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1211
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1111
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1011
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0911
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0811
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0711
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0611
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0511
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0411
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0311
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0211
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0111
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1210
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1110
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1010
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0910
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0810
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0710
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0610
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0510
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0410
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0310
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0210
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0110
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1209
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1109
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1009
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0909
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0809
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0709
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0609
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0509
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0409
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0309
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0209
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0109
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1208
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1108
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1008
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0908
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0808
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0708
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0608
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0508
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0408
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0308
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0208
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0108
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1207
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1107
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_1007
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0907
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0807
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0707
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0607
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0507
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0407
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0307
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0207
https://www.nxtbook.com/nxtbooks/ieee/spectrum_na_0107
https://www.nxtbookmedia.com