how to get smart about e-discovery
DESCRIPTION
How is Big Data changing litigation and what challenges does that create for electronic discovery? This program talks about the challenges of e-discovery and how technology is helping to overcome those challenges. The program was presented by Mark Noel, Managing Director of Professional Services at Catalyst Repository Systems, and Shawn Cheadle, General Counsel, MilItary Space, at Lockheed Martin Space Systems co.TRANSCRIPT
catalystsecure.com
How to Get Smart About E-Discovery
Shawn Cheadle, J.D. Mark Noel, J.D.
Mark Noel
Presenters
Shawn Cheadle
Managing Director, Professional Services Catalyst Repository Systems [email protected]
General Counsel Military Space, Lockheed Martin Space Systems Co. [email protected]
Agenda
How big data is changing litigation
E-discovery 101 and its challenges
Court decisions that open the door to cost-saving technology
Lessons learned from larger corporations
What this means to business leaders
Q&A
Recent E-Discovery Sanctions
Rambus — $250 million (2013)
In re Pradaxa Products — $1 million (2013)
Victor Stanley v. Creative Pipe — $1.05 million (2011)
Soaring Costs and Greater Risk
E-discovery costs skyrocketing
Review costs skyrocketing
Data spread across multiple cases, firms, and vendors
Exploding content
The Problem:
1.8 Zettabytes a year Library of Congress: 30 Terabytes
60 Million Libraries a year...
... and growing
“The 1% Problem”
“The 1% Problem”
Facing the Corporate Challenges
Three Broad Categories:
Document Review Tasks
1. Classify
2. Protect
3. Discover
Recall
“How complete was my catch?”
Precision
“How pure was my catch?”
Which bucket does each document go into?
1. Classify
Recall? Reasonably high
Precision? Reasonably high
Think Sedona Conference and FRCP 1, 26
1. Classify
2. Protect
Recall? 100% – nothing escapes
Precision? Usually high, especially
if you have to log it all
Make sure no sensitive or privileged info gets out
2. Protect
3. Discover Recall? Don’t really care
Precision? Really good – don’t waste our time with junk.
Most -like.
What can we learn from the documents’ contents?
3. Discover
“It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.”
– Mark Twain
Research on Search Terms
20%
75% Attorneys worked with experienced paralegals to develop search terms. Upon finishing, they estimated that they had retrieved at least three quarters of all relevant documents.
What they actually retrieved:
Blair & Maron, An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System (1985).
Research on Search Terms
Counsel and others need interactive access to the ESI population in order to achieve at least 50% retrieval of responsive ESI.
– Some Lessons Learned To Date from the TREC Legal Track
Employing different combinations of search technologies (and others) results in the retrieval of approximately 78% of the total number of relevant documents. In effect, using a variety of methods and tools will increase the search results substantially.
– In Search of the Perfect Search
Whether search terms or “keywords” will yield the information sought is a complicated question involving the interplay, at least, of the sciences of computer technology, statistics and linguistics …
United States v O’Keefe, 537 F. Supp. 2d 14, 24 (D.D.C. 2008)(Facciola, J.); See also Equity Analytics, LLC v. Lundin, 248 F.R.D. 331, 333 (D.D.C. 2008)(Facciola, J.).
The Gold Standard?
Human Review
“The idea that exhaustive manual review is the most effective—and therefore the most defensible—approach to document review is strongly refuted. Technology assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort.”
Maura Grossman and Gordon Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient than Exhaustive Manual Review, Richmond Journal of Law and Tech, Vol XVII, Issue 3 (2011).
Not Really...
Reality
Danziger, Leva and Avnaim-Pesso, Extraneous factors in judicial decisions, PNAS (2011).
Physiological Factors
“Justice is what the judge ate for breakfast” – Jerome Frank
Review accuracy may be what the junior associates ate for lunch
Executive function influenced by blood glucose levels, breaks, positive mood, and viewing pictures of nature.
What is Technology Assisted Review?
“By computer-assisted coding, I mean tools that use sophisticated algorithms to enable the computer to determine relevance, based on interaction with a human reviewer.”
Da Silva Moore v. Publicis Groupe (S.D.N.Y. 2012)
What Goes on Under the Hood? The computer builds a big, complex search!
What terms are most likely to be associated with good documents?
What terms are most likely to be associated with bad documents?
What is Computer Assisted Coding? By computer-assisted coding, I mean tools that
use sophisticated algorithms to enable the computer to determine relevance, based on
interaction with a human reviewer.
Da Silva Moore v. Publicis Groupe (S.D.N.Y. 2012)
It’s About Mathematics Support Vector Machines
Naïve Bayes
K-Nearest Neighbor
Geospatial Predictive Modeling
Latent Semantic
“I may be less interested in the science behind the ‘black box’ than in whether it produced responsive documents with reasonably high recall and high precision.”
– Peck, M.J. (SDNY)
Corporate e-discovery landscape
Lessons from Larger Corporations
Responding to the big data challenge
Increasing number and size of discovery events
Corporations taking control of discovery spend
Decreasing number of providers/firms across limitation portfolio
Fixed fee models
Managed service models
Greater diversity and overlap between corporate and cloud data sources
Changing approach to e-discovery
Lessons from Larger Corporations
Matter-based approach Each goes through EDRM model independently Reactive and linear process Fire-and-forget budgets
Manage entire discovery process as a proactive, unified, interactive, iterative lifecycle
Traditional:
Emerging:
Controlling litigation spending
Industry Overview
Centralized cross-functional environment Manage global collections and cases Single-instance storage Reduce and reuse Predictive analytics Technology-assisted review Intelligent synchronization and feedback
Control of data, reduced cost and risk
Business Benefits
One-time processing
Single-instance savings and efficiency
Leverage prior work-product for review and quality control
Get new matters started quickly — turnkey design
Cross-matter standardization
Retain document history
Key Points Effective use of all available technologies is crucial
Tools should be combined and workflows adjusted depending on what you’re trying to accomplish
You don’t have to be an expert to succeed with e-discovery
Mark Noel
Questions?
Shawn Cheadle
Managing Director, Professional Services Catalyst Repository Systems [email protected]
General Counsel Military Space, Lockheed Martin Space Systems Co. [email protected]