IEEE Spectrum November, 2013 - 53

and contexts. for instance, locating all [of experience] was zero," says lex Machina's former chief technolcases related to patent infringement was complicated by the fact that ogy officer, Mihai surdeanu, who had previously worked on naturalthe exact term didn't always appear in a document's text. "It was
language processing at yahoo labs. "there was nobody doing this."
a matter of translating upwards and understanding the concept of
existing machine-learning techniques don't work very well on
infringement regardless of the words they were using," Manning says. litigation data, he says. even a relatively simple process like normalanother difficulty the researchers encountered was that each court izing names poses a challenge. the computer has to recognize, for
website uses its own variant of electronic filing. they therefore had to
example, that IBM, International Business Machines, and IBM Corp.
design a web crawler for each one. once collected, the data then had
all refer to the same company. More problematic were law firm
to be standardized to account for the variations in the way courts file
names, because firms sometimes change their names when they
data. and in many instances, the data were just plain wrong; in more
merge with other firms or when partners join or leave. even a firm's
than half of all cases, the final decision in the case had been incorrectly
own attorneys can get the name wrong, surdeanu says. "one of
coded, according to walker, who served as lex Machina's first Ceo
the firms in our database has 89 different legal spellings," he adds.
and is now an attorney at the law firm simpson thacher & bartlett.
the system must also be able to handle complex legal constructs.
the way the cases were tagged-as patent, copyright, or trademark unlike baseball, which is a numbers game, the legal world is based
infringement, for example-was also often wrong. and there were no
on qualitative information, subtle distinctions, and most of all, words.
tags for certain types of cases, such as those involving trade secrets. "People argue about the meanings of words and make arguments
the researchers had to manually sort through, categorize, and with paragraphs of text," explains Manning. the machine needs to
correct the data. "there were hundreds of thousands of legal judg- understand phrases and strings of commonly used legal language as
ments that had to be made" as they sifted through the informa- well as context so that it can tell the difference between, for examtion, walker says. In total, it took the team about 100 000 hours.
ple, the summary judgment document (in which a judge determines
once the team had cleaned up the data and understood its many which party wins the case or at least certain issues in the case) and a
complexities, the engineers designed algorithms to automatically minor procedural filing that simply mentions the summary judgment.
review each document and sort the results. similar algorithms are
to help parse the legalese, lex Machina has developed a set of rules-
used in web searches, but interpreting legal documents requires
a sort of legal grammar for the machine. the company does this
more sophistication. "from a science perspective, the baseline
through an iterative process: a legal analyst reviews the algorithms'
continued froM page 34 |

AnnIvErsAry

of
IEEE Foundation

Inspiring the Future

Donate and Enable the Impact of IEEE through IEEE Foundation

EducATIon * InnoVATIon * PrEsErVATIon
Your generous donations motivate students and young professionals,
enable innovators to make a difference, promulgate technology's influence
on the world and inspire the future.

IEEE Foundation
Dedicated to providing philanthropic services to support the core purpose
of IEEE-Advancing Technology For Humanity.
Visit ieeefoundation.org to learn more.

Be an inspiration. donate Today. ieee.org/donate
13-FND-0046h-IEEE-Foundation-Ad-7x4.75-FINAL.indd 1

SPECTRUM.IEEE.ORG

nORTh aMERICan

3/20/13 3:39 PM
nOv 2013 | 53

http://www.ieeefoundation.org http://www.ieee.org/donate http://SPECTRUM.IEEE.ORG

Table of Contents for the Digital Edition of IEEE Spectrum November, 2013