Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

A Long History
Innovation in Law

One of the First to Teach
Law with Computers

Helped Launch the Premier
State Wide Legal Aid Website

Helped 3.5 million+ Users
Seek Access to Justice
Guided
Interview
Completed
Document
A2J AUTHOR
www.a2jauthor.org
LOGIC
Used over
3.5
Million
times
2.1 Million
Documents generated
IMPACT

American
Federal
Judiciary
American
Law Professoriate
Building New Algorithms
Large Scale
Judicial Studies
Scientiﬁc
Research

3D HD Visualization of Supreme
Court Citation Network
Campaign Contributions and
Legislative Ecosystems
Six Degrees
of
Marbury
v.
Madison
Electronic
World
Treaty
Index
Radial
SCOTUS
Citation
Network
Scientiﬁc
Research

http://www.quantitativemethodsclass.com/Professor Daniel Martin Katz
Intro Class

http://www.legalanalyticscourse.com/Professor Daniel Martin Katz
Professor Michael J. Bommarito II Advanced Class

Building
Ties with the
Legal Industry

FinLegalTechConference.comNovember 4, 2016

Fin(Legal)Tech Conference
finlegaltechconference.com
@Illinois Tech - Chicago Kent College of Law

Fin(Legal)Tech
Conference
October 19 2017
FinLegalTechConference.com
@LawLaboratory
Chicago, IL

Today —
a session
in ﬁve parts …

A Reset on Robot LawyersI.
The Rise of #LegalAnalyticsII.
The Killer Use Case(s) - Fin (Legal) Tech)III.
The Infrastructure for #LegalAnalyticsIV.
Building a Legal Data StrategyV.

A Reset on
Robolawyers
Part I< >

There has been lots of recent
interest in applying
artiﬁcial intelligence to law

and there is a bit of confusion
as to where we stand today
and where we are headed

data driven AI rules based AI
Competing Orientations in
Artiﬁcial Intelligence

expert
systems
Computational Law
Data Driven Rules Based
prediction
models
and
methods
network
analytic
methods
natural
language
processing
self
executing
law
visual
law
computable
codes

we see a decent amount of
rules based AI
in legal industry

Three Examples
of Rules Based A.I.

Among other things Neota
has been used to create
decision trees to support
lawyers / non lawyers

What do I do if there has been
An issue in Human Resources ?
A potential FCPA violation?
A potential data breach?

Decision Trees are a step by step
memorialization of best practices

At my home institution -
Illinois Tech Chicago-Kent Law
has a platform that allows
individuals to automate
various legal forms, etc.

used by a variety of
legal aid organizations

PROCESS
Guided
Interview
Completed
Document

Used over
3.5
Million
times
2.1 Million
Documents generated
IMPACT

Expert Systems  
(together with data)  
will eventually
become Chatbots …

Client Intake
More Seamless Client Interaction
via Tech Platform
Providing Legal Information
to Non-Lawyers in Large
Organizations

so although we see a
decent amount of
rules based AI
in legal industry

I am pretty bearish
on Rules Based A.I. for most
(but not all) applications …

my views are informed by
the history of A.I. in general

lots of issues
with expert systems
and/or
rules based A.I.
(without data or an evolutionary dynamic)

rules based A.I. data driven A.I.
1980’s, 1990’s, Early 2000’s
>

1980’s, 1990’s, Early 2000’s
>
2005 - Present
<
~

Ultimately we are trying to learn
the rules / dynamics that
underlie some class of activity

With that understanding we want to
be able to mimic / predict

There are some use cases
for Rules Based AI /
Expert Systems

Practically
ZERO
Top Tier
Companies
Building
Expert
Systems

A.I. State of the Art
purely data centric

purely data centric
augment expert forecasts w/ data

iterative data < > rules
purely data centric
augment expert forecasts w/ data

Again -
I like Chatbots because they
end up being a massive data
collection effort …
iterative data < > rules

In the Rules vs. Data
Debate in A.I.

Data Won
the War
(Terms of Surrender
are Available)

The Rise of #LegalAnalytics
Part II< >

Law is a relatively small vertical
and there is lots of diversity
among tasks lawyers undertake …

Given large ﬁxed costs
infrastructure
+
human capital
(data scientists)

harder to successfully deploy
high quality enterprise
applications for relatively
narrow (sub)verticals

in addition
there is a
borderline
pathological
numerophobia
among lawyers

plus the implicit (explicit)
challenge of partnership as
the dominant form of the
organization within our market

taken together this has
challenged the deployment  
of analytics in legal

Analytics /
Quant Legal Prediction
has come to law
Notwithstanding these head winds—

#LegalAnalytics
Quantitative Legal Prediction

In a real sense, this represents
just a narrow set of products

#ContractAnalytics

#JudicialAnalytics

#PredictiveCoding #E-Discovery

General Counsels as Legal
Procurement Specialists
TyMetrix/ELM -
Using $50 billion+ in Legal
Spend Data to Help GC’s
Look for Arbitrage
Opportunities, Value
Propositions in Hiring Law
Firms
#LegalSpendAnalytics

#LegalAnalytics
https://lexsemble.com/

#NegotiationAnalytics

Here are just a subset of the
tasks that we are trying
to accomplish in legal …

#Predict Case Outcomes
Data Driven Legal Underwriting

#Predict Legal Costs
Data Driven Legal Operations

#Predict Relevant Documents
Data Driven EDiscovery/Due Diligence
(Predictive Coding)

(Predictive Coding)
#Predict Rogue Behavior
Data Driven Compliance

(Predictive Coding)
Data Driven Compliance
#Predict Contract Terms/Outcomes
Data Driven Transactional Work
#Predict Rogue Behavior

A Deeper Dive
on Predicting
Predicting Case Outcomes
(other problems can be
solved using similar methods)

Supreme Court of United States
#PredictSCOTUS

There are only 3 ways  
to predict something
Experts
Crowds
Algorithms

Columbia Law Review
October, 2004
Theodore W. Ruger, Pauline T. Kim,
Andrew D. Martin, Kevin M. Quinn
Legal and Political Science
Approaches to Predicting
Supreme Court Decision
Making
The Supreme Court
Forecasting Project:

Case Level Prediction
Justice Level Prediction
67.4% experts
58% experts
From the 68
Included
Cases
for the
2002-2003
Supreme
Court Term

these experts probably
overﬁt

they ﬁt to the noise
and
not the signal

if this were
ﬁnance this
would be
trading
worse than
S&P500

like many other forms
human endeavor
law is full of  
noise predictors …

we need to
evaluate
experts and
somehow
benchmark
their
expertise

from a pure
forecasting
standpoint

the best
known
SCOTUS
predictor is

the law
version of
superforecasting

https://fantasyscotus.lexpredict.com/case/list/
We can
generate
Crowd
Sourced
Predictions

Just like the
Market
the Crowd is
collectively
terrible …
< No Alpha >

however,
not all
members of
crowd are
made equal

we maintain
a ‘supercrowd’
which is the top n%
of predictors
up to time t

the
‘supercrowd’
outperforms
the overall
crowd
(and the best
single player)

not
enough
crowd
based
decision
making in
institutions
(law included)

“Software developers were asked on
two separate days to estimate the
completion time for a given task, the
hours they projected differed by 71%,
on average.
W h e n p a t h o l o g i s t s m a d e t wo
assessments of the severity of biopsy
results, the correlation between their
ratings was only .61 (out of a perfect
1.0), indicating that they made
inconsistent diagnoses quite frequently.
Judgments made by different people
are even more likely to diverge.”

not
enough
crowd
based
decision
making in
institutions
(aka manual
underwriting)

Brief Aside
About
Crowd
Sourced
Prediction
#LegalCrowdSourcing

(most pundits did not
identify as a serious
candidate him until
mid-January 2017)
Neil Gorsuch was #1
o n o u r F a n t a s y
Platform 12 Days after
Donald Trump was
elected President
(i.e Nov 20)

Our algorithm is a special version
of random forest (time evolving)
http://journals.plos.org/
plosone/article?id=10.1371/
journal.pone.0174698
available at
RESEARCH ARTICLE
A general approach for predicting the
behavior of the Supreme Court of the United
States
Daniel Martin Katz1,2
*, Michael J. Bommarito II1,2
, Josh Blackman3
1 Illinois Tech - Chicago-Kent College of Law, Chicago, IL, United States of America, 2 CodeX - The Stanford
Center for Legal Informatics, Stanford, CA, United States of America, 3 South Texas College of Law Houston,
Houston, TX, United States of America
* dkatz3@kentlaw.iit.edu
Abstract
Building on developments in machine learning and prior work in the science of judicial pre-
diction, we construct a model designed to predict the behavior of the Supreme Court of the
United States in a generalized, out-of-sample context. To do so, we develop a time-evolving
random forest classifier that leverages unique feature engineering to predict more than
240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015).
Using only data available prior to decision, our model outperforms null (baseline) models at
both the justice and case level under both parametric and non-parametric tests. Over nearly
two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the jus-
tice vote level. More recently, over the past century, we outperform an in-sample optimized
null model by nearly 5%. Our performance is consistent with, and improves on the general
level of prediction demonstrated by prior work; however, our model is distinctive because it
can be applied out-of-sample to the entire past and future of the Court, not a single term.
Our results represent an important advance for the science of quantitative legal prediction
and portend a range of other potential applications.
Introduction
As the leaves begin to fall each October, the first Monday marks the beginning of another term
for the Supreme Court of the United States. Each term brings with it a series of challenging,
important cases that cover legal questions as diverse as tax law, freedom of speech, patent law,
administrative law, equal protection, and environmental law. In many instances, the Court’s
decisions are meaningful not just for the litigants per se, but for society as a whole.
Unsurprisingly, predicting the behavior of the Court is one of the great pastimes for legal
and political observers. Every year, newspapers, television and radio pundits, academic jour-
nals, law reviews, magazines, blogs, and tweets predict how the Court will rule in a particular
case. Will the Justices vote based on the political preferences of the President who appointed
them or form a coalition along other dimensions? Will the Court counter expectations with an
unexpected ruling?
PLOS ONE | https://doi.org/10.1371/journal.pone.0174698 April 12, 2017 1 / 18
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Katz DM, Bommarito MJ, II, Blackman J
(2017) A general approach for predicting the
behavior of the Supreme Court of the United
States. PLoS ONE 12(4): e0174698. https://doi.
org/10.1371/journal.pone.0174698
Editor: Luı´s A. Nunes Amaral, Northwestern
University, UNITED STATES
Received: January 17, 2017
Accepted: March 13, 2017
Published: April 12, 2017
Copyright: © 2017 Katz et al. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: Data and replication
code are available on Github at the following URL:
https://github.com/mjbommar/scotus-predict-v2/.
Funding: The author(s) received no specific
funding for this work.
Competing interests: All Authors are Members of
a LexPredict, LLC which provides consulting
services to various legal industry stakeholders. We
received no financial contributions from LexPredict
or anyone else for this paper. This does not alter
our adherence to PLOS ONE policies on sharing
data and materials.

From a Pure
Machine Learning Perspective —
Much of this is not novel
EXCEPT the time evolving
element of the
Random Forest

https://github.com/mjbommar/
scotus-predict-v2/

243,882
28,009
Case Outcomes
JusticeVotes
Final Version of #PredictSCOTUS
1816-2015
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174698

Final Version of #PredictSCOTUS
1816-2015
case accuracy
70.2%
71.9%
justice accuracy
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174698

http://www.sciencemag.org/news/
2017/05/artiﬁcial-intelligence-prevails-
predicting-supreme-court-decisions
Professor Katz noted
that in the long term
…“We believe the
blend of experts,
crowds, and
algorithms is the
secret sauce for the
whole thing.”
May 2nd 2017

For most problems ...
ensembles of these streams
outperform any single stream

the non-trivial question
is how to optimally assemble
such streams for particular problems

Humans
+
Machines
Humans
or
Machines
>

Here is what we are
working on right now …

expert
forecast
crowd
forecast
learning problem is to discover how to blend streams of intelligence
algorithm
forecast
ensemble method
ensemble model

expert
forecast
crowd
forecast
learning problem is to discover how to blend streams of intelligence
algorithm
forecast
ensemble method
ensemble model
via back testing we can learn the weights
to apply for particular problems

By the way, you
might ask why does
one care about
marginal improvements
in prediction ?
#Fin(Legal)Tech

It is a fair question
because in the
private market …
improvements in
performance must
be linked up to an
actual business
model …

Fin (Legal) Tech
is the killer use case
Part III< >

Given our ability to offer
forecasts of judicial
outcomes, we wondered
if this information could
inform an event driven
trading strategy ?

Revise + Resubmit @
http://arxiv.org/abs/1508.05751
available at
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2649726

lots of litigation decisions
are just a version of this basic idea
law = ﬁnance

from an asset valuation standpoint
lots of litigation decisions
are actually implicit litigation ﬁnance
(or self insurance)
#ﬁn(legal)tech

Consider for example …
Litigation Reserves Setting Under
FASB ASC 450-20-25
#ﬁn(legal)tech

But there are many
other places where …
law = ﬁnance

Three Types of Lawyers
(as described by paul lippe)

play “whack-a-mole”, reacting to
problems by creating fear and
friction within organizations and
the impression that there is a
legal risk around every corner.
Mediocre Lawyers

can help clients shape
(perhaps distort)
external perception of risk.
Merely Clever Lawyers

design systems that
balance risk and improve
transparency, helping clients
correctly price risk internally
Great Lawyers

law = ﬁnance
(insuranceaswell)

law < > ﬁnance
many elements in law look like
ﬁnance did 25 - 50 years ago
(on the long road from Black-Scholes to algorithmic trading)

Lawyer VALUE PROPOSITION
(From the Client’s Perspective)
(internal or external client)

help price risk /
help reduce information asymmetries
transactional =

litigation =
characterize (predict) risk/exposure
shift the expected value of a lawsuit
help price risk /
transactional =

litigation =
compliance = identify + prevent rogue behavior
monitor behavior in (near) real time
help price risk /
transactional =

litigation =
compliance = identify + prevent rogue behavior
monitor behavior in (near) real time
help price risk /
transactional =
regulatory =
help identify (predict) the decisions
of regulators / law makers and the risk
associated with various outcomes

Dominant Model in Law
expert centered pricing of risk

Dominant Model in Law
lots of unintentional self insurance
rarely (if ever) based upon
explicit risk models

Cult of one  
(or very small # of)
person(s) thinking drives
decisions with serious
ﬁnancial consequences

Claim:
ﬁn(tech) offers lessons
for many areas
in law

thesis statement:
the ﬁnancialization
of the law will be
an important vector
of the next decade(s)

#Fin(Legal)Tech
application of those ideas and
technologies to a wide range of
law related spheres including
litigation, transactional work
and compliance.

Here are
just a few
of many
examples

www.burfordcapital.com/
http://www.gerchenkeller.com/
http://www.fulbrookmanagement.com/
http://www.longfordcapital.com/
http://www.benthamimf.com/
Litigation Finance

Outside of M+A
Requires Mapping of Deal Terms
to actual substantive outcomes
#legaldata
#legalanalytics

Being able to compute the
change in risk as a function
of a change in deal terms

Trading Desk is
all about alpha -
using data,
predictions,
process, etc.

Not about simply buying
tools off the shelf and
deploying them …

The Infrastructure
for Legal Analytics -
#MLaaS and the
Enterprise Open
Source Movement
Part IV< >

Lots of folks ask me what is
next in legal analytics …

A big part of the answer
comes from one of the most
dominant vectors in tech

both those in positions of
leadership and those in technical
positions need to take stock

the democratization of
machine learning is underway

Emerging Business Model -
Machine Learning as a Service
#MLaaS

IBM Watson (per se)
IBM Watson (as early #MLaaS)
vs.

IBM WATSON
First major effort at #MLaaS

#MLaaS

Machine
Learning as
a Service
#MLaaS

#MLaaS
Enterprise Open Source Movement
#OpenSource
+

Enterprise Open Source Movement
#OpenSource

https://techcrunch.com/2016/06/19/the-next-wave-in-software-is-open-adoption-software/

historically one needed to
build the full stack (i.e end to
end) for an application

Standing on  
the Shoulders of Giants

The (Emerging) Last Mile Problem
in (Legal) Analytics

Off the
Shelf
#MLaaS, etc.
(perhaps with some
conﬁguration
and/or
customization)
Unique Domain
Speciﬁc Offering

Piece together the
combinations of  
#MLaaS + open source

to build enterprise applications
which are unique combinations
drawn from across the
#MLaaS / open source spectrum

First Wave vs.
Second Wave
Legal Tech

Second Movers can
catch up faster …

Second Movers
need less capital …

Second Movers
who start now
will have lower
ﬁxed costs …

Major Implication
The Best Legal Tech
is Yet to Be Built …

We are beginning
to see the ﬁrst wave
of #MLaaS
Implementation
Companies in
General

https://computationallegalstudies.com/2017/05/07/machine-learning-
service-mlaas-ecosystem-grows-bonsai-mlaas-implentation-company/
“AI startup Bonsai has raised $7.6
million to grow its platform that
simpliﬁes open-source machine
learning library TensorFlow to
help businesses construct their
own artiﬁcial intelligence models
and incorporate AI into their
business.”

And this is in Part
What My Company
LexPredict
will be (already is)
doing within law …

https://www.slideshare.net/lexpredict/
contraxsuite-why-were-opensourcing-
contraxsuite-and-product-overview
#OpenSourceLegal

"We are increasingly thinking that there's room in
legal tech for a Red Hat in legal — companies that
really focus on development of software by providing
wraparound services, but offer their software open
source," Michael J Bommarito II said.
Michael J. Bommarito
Co-Founder
CEO @ LexPredict

Will you resell the software to third parties?
YES%NO%
How much does ContraxSuite cost?
Will you keep derivative work open?Free%
YES%NO%
Free%$12K/year%
50% in trust for open source grants ! 50% for ContraxSuite, LLC!

If you are just
buying tools from
vendors you likely
have no alpha

Building a Legal
Data Strategy
Part V< >
(A Role for Law Librarians?)

every organization in law
needs a data strategy

Capture, Clean, Regularize Data
to support a range of tasks

Deploy Data for Speciﬁc
Enterprise Applications
Develop a
data roadmap

Statement and Framework
Data Strategy: Defined
! ! ! ! A! data! strategy! combines! a! top2down! mission! statement!
acknowledging! the! value! of! an! organiza(on’s+ data! with! a!
framework!for!developing!data.driven+capabili(es.+
!
! ! ! ! While! data! strategies! are! built! on! lists! of! principles! and!
technologies,! they! address! much! more:! strategic!
communica=on! and! change! management,! process!
improvement,!knowledge!management,!and!risk!management,!
to!name!a!few.!

D - I - K - W
From Data Strategy to Wisdom
Data$
Informa+on$
Knowledge$
Wisdom$
Direct'record'of'fact,'
signal,'symbol'
Indirect'record'or'
descrip6on$
Interpreta6on'of'
informa6on$
Ac6onable'inference'or'
heuris6c$
Data-Information-Knowledge-Wisdom
Data$
Readings'from'a'temperature'
sensor'in'Tahoe.$
Informa+on$
The'average'temperature'in'the'
month'of'December'is'32.2F.$
Knowledge$
Snow'is'likely'to'accumulate'in'
December.'
Wisdom$
January'is'a'good'month'to'plan'
a'ski'trip'to'Tahoe.$

When is Data Valuable?
Even when it’s not
LOW$ HIGH$
HIGH$LOW$
IMPACT'
FREQUENCY'
High3frequency,'high3impact'3'best'use'case'for'data'
•  Systema/c'understanding'and'treatment'
•  Standardized$reporEng'and'sta/s/cal$treatment'
•  PotenEal'for'automaEon'and'predicEon'
Example:'
•  Labor'&'Employment'for'a'large'employer'
•  Patent'Defense'for'a'large'tech'company'

Some%Organiza-ons%Have%Publically%
Commi8ed%Themselves%to%Use%Data%
to%Become%‘Best%in%Class’%%
Legal%Departments%%

33!
“Now! we! have! program! managers,!
data! analysts,! business! analysts,!
data! scien9sts,! opera9ons!
managers,!I!mean,!we!have!a!ton!of!
stuﬀ.! That's! the! key! for! me,! is!
thinking! about! the! right! people!
doing! the! right! tasks.! That's! the!
people!part.!And!then!how!they!do!
them,! is! the! process,! and! then,!
automa9ng! parts,! is! kind! of! that!
next,!ﬁnal!step.!!
"
And$ all$ of$ that$ is$ underpinned$ by$
d a t a ." Y o u$ c a n ' t$ d o$ a n y$
improvements$ unless$ you$ have$
data.$ You$ can't$ automate$ unless$
you$have$good$data.”!

36!
“From!se)lement!informa0on!and!
contracts! to! sensi0ve! client! data!
and! beyond,! Liberty! Mutual!
creates! and! stores! ever:growing!
volumes! of! unorganized! data!
across! its! worldwide! oﬃces! and!
databases.”!
“I've!seen!a!real!transforma0on!in!
the! legal! department! just! having!
t h a t! i n f o r m a 0 o n! v i s u a l l y!
available."!
“The' legal' department' is' now'
w o r k i n g' p r e d i c 7 v e' a n d'
prescrip7ve' analy7cs,"' i.e.' ways'
to' analyze' data' that' enable'
forecas7ng'for'legal'issues.”'

37!
“I"believe"strongly"that"data"analy2cs"is"
a"new"fron2er"in"the"legal"space.”"
Susie!Lees!
General!Counsel!!
Allstate!!
“Leveraging" data," not" only" that" we"
possess" but" that" our" law" ﬁrms" have"
amassed"over"the"years,"oﬀers"a"plethora"
of" un<tapped" opportuni=es—not" simply"
to" help" us" forecast" and" manage" legal"
expenses," but" also" to" help" our" clients"
make"more"informed"business"decisions.”"

Five reasons to care
Can you answer these questions?
1. How&many&legal&ma.ers&did&you&handle&last&year?&
2. How&much&poten:al&legal&liability&did&you&handle&last&year?&
3. How&many&hours&per&legal&ma.er&did&you&spend&last&year?&
4. How&many&dollars&per&legal&ma.er&did&you&spend&last&year?&
5. How&much&value&did&you&protect&or&create&last&year?&

47!
Methods for Using (Legal) Data
Historical reporting in legal
Historical analytics in legal
Predictive analytics in legal

48!
Historical reporting
in legal
Ques'on:+ What! did! we!
spend! on! se.lements! and!
legal!expenses!last!quarter?!
$1.2M+
Ques'on:+ On! average,! how!
many!eﬀort!hours!does!staﬀ!
counsel! spend! on! the!
discovery! phase! of! a! non>
compete!dispute?!
25+
hours+

49!
Ques&on:!What!
factors!drove!
se3lement!
amounts!last!
quarter?!
•  F o r ! l a b o r ! a n d!
employment!disputes,!the!
length! of! employment!
a n d! p r e s e n c e! o f!
retaliatory! or! sexual!
harassment! claims! are!
posi&vely! related! to!
se3lement!amount!
•  Disputes! origina&ng! in!
region!X!have!abnormally!
higher! se3lements! than!
expected,! given! their!
facts!
Ques&on:!What!
factors!drove!
legal!expenses!
last!quarter?!
•  An! increase! in! ma3ers! in!
highCcost! jurisdic&ons! is!
posi&vely! related! to! total!
legal!expenses!
•  A! decrease! in! arbitra&on/
media&on! u&liza&on! is!
posi&vely! related! to! total!
legal!expenses!
Historical analytics in legal

50!
Ques'on:!Should!we!se,le!this!dispute!at!outset?!
•  The!counterparty!is!expected!to!accept!an!ini8al!offer!
•  The!dispute!is!predicted!to!se,le!for!$100k,!with!legal!
expenses!of!$15k!
•  If!an!ini8al!offer!is!not!made,!this!dispute!is!expected!to!
cost!$50k!in!legal!expenses!and!has!a!25%!chance!of!going!
to!jury!trial.!
Ques'on:!How!many!effort!hours!will!
we!spend!on!this!ma,er?!
•  An!es8mate!of!18!hours,!with!90%!confidence!that!the!
dispute!will!fall!between!13!and!30!hours!
Predictive analytics in legal

37!
Stages!of!Legal!!
Data!Strategy!Maturity!
Chaotic
Managed
Defined
Data-Driven Continuously Improvin
5!1! 2! 3! 4!

51!
(be able to do so without a herculean effort)
1. !Measure,!monitor,!and!manage!your!resources!and!service!providers.!
!
2. !Using!data!+!experts,!model!and!improve!the!processes!you!execute.!
3. !Allocate!tasks!across!internal/external!resources!and!assess!cost!and!quality.!
4. !Manage!risk!and!be!able!to!formally!characterize!the!risks!avoided.!
5. !Jus&fy)and)explain)performance)to)the)clients.!
Five Goals for Every Legal Organization

Deploying a Legal
Data Strategy for a
Discrete Problem

Y = βo +/- β1 ( X1 ) +/- β2 ( X2 ) +/- β3 ( X3 ) +/- β4 ( X3 ) +/- β5 ( X3 ) + ε
Y = $151 + $15 ( ) + 161 ( ) + 95 ( ) + 34 ( ) +/- β5 ( ) + ε
Per
100
Lawyers
If Tier 1
Market
is True
Partner
Status
is True
Per
10
Years
Practice
Area

1. Deﬁne the Parameter Space
3. Select a Model/Method
4. Validate Out of Sample
2. Collect / Normalize Data
(typically using experts)

Work with experts to
deﬁne relevant variables
that drive outcomes on
some problem
(experts are strong at identifying
relevant variables but have
trouble applying weights)

Figure out how to
collect or normalize
relevant data

Yes
No
f( )
Outcome?
binary
f( )
Outcome?
continuous
machine learning is the approach to
‘learn’ the best performing f ( )
select a model/method
then validated out of sample

developing-a-legal-data-strategy-learning-
to-see-data-as-a-strategic-business-asset
legal-data-strategy-maturity-assessing-
capabilities-and-planning-improvements

Daniel Martin Katz
@ computational
computationallegalstudies.com
lexpredict.com
danielmartinkatz.com
illinois tech - chicago kent college of law@
thelawlab.com

Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

Similaire à Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz

Similaire à Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz (20)

Plus de Daniel Katz

Plus de Daniel Katz (14)

Dernier

Dernier (20)

Can Law Librarians Help Law Become More Data Driven ? An Open Question in Need of a Solution — Professor Daniel Martin Katz