James D. Shook, Esq., EMC eDiscovery Expert

By James D. Shook, Esq. 

The evidence just keeps piling up on why in-house lawyers must be involved with their organization’s retention policies and e-discovery processes.  We have already discussed the benefits to legal as well as the idea of a “shield” in litigation.  But now there’s even more evidence.

The latest comes from Scentsy v. Chase et. al., 2012 WL 4523112 (10/12/12 D. Idaho), a case that included claims of spoliation by defendant Harmony against Scentsy for allegedly deleting email and files relevant to the dispute.  Scentsy’s corporate policy was to delete email messages after 60 days but permitted documents stored on laptops, desktops and fileshares to be retained indefinitely — until deleted by the employee.  Further, a relaxed litigation hold process consisted of the general counsel speaking with employees that might have relevant information, merely “request[ing] that those documents not be deleted.”

Read the full Article…

With the acquisition of Kazeon by EMC, the Kazeon blog will be moving!!! - Get all your eDiscovery, Compliance, Regulatory, and Governance insights on the EMC SourceOne Insider Blog - http://emcsourceoneinsider.wordpress.com 

According to a new market report published by Transparency Market Research,  the global e-discovery market was worth $US3.6 billion in 2010 and is expected to reach $US9.9 billion in 2017. In the overall global market, the US is expected to maintain its lead position in terms of revenue with 73% of global e-discovery market share in 2017.

Discover the Full Article

James D. Shook, Esq.

Judge Scheindlin’s most recent opinion in National Day Laborer Organizing Network et al. v. United States Immigration and Customs Enforcement Agency, et al. 2012 U.S. Dist. Lexis 97863 (S.D.N.Y., July 13, 2012) (“NDLON V”) is another nail in the coffin of custodian self-collection for e-Discovery.  The decision deals mostly with the adequacy of the various defendants’ processes in locating information responsive to FOIA requests.  Judge Scheindlin takes the defendants to task for their reliance on end-users to locating the information, along the way noting the similarities between the FOIA and civil eDiscovery collection processes.  But the real surprise in this case is that self-collection remains so alive and well.

Custodian self-collection is relying upon individual employees to locate and collect electronically stored information (ESI) that is relevant to a dispute.  There’s no “standard” for self-collection and the level of reliance on employees can vary.  In some cases, employees are given only very basic information, such as the general parameters of a dispute (i.e. an allegation of wrongful termination, a breached contract, etc.) and asked to collect and forward any relevant ESI to counsel.  At the other end of the spectrum, a more thorough process might include more specific factual information about the case (“Smith alleges that we fired him because he’s considered too old”) with even specific instructions on what should be searched and how the search should be conducted to maximize finding relevant ESI.

The problems inherent in self-collection are clear to anyone who gives them even a little bit of thought:

-        Failing to understand the facts of a case.  An employee may have information related to the Acme contract dispute — except that she knows about the matter only through her relationship with XYZ Corporation (which could be a sister company, a predecessor/successor, etc.).  A generic request for information about Acme may not even trigger a response.

-        Failing to understand the legal scope of a case.  Most employees do not have a strong legal background.  An employee asked to collect information about a breach of a lease may have no idea that mitigation of damages is a key issue, and so he does not provide any information about the company’s efforts to lease the property.

-        Lack of time or prioritization.  Busy employees may miss, skim or not understand the importance of an email or other notification from the legal department about a legal hold.   Or they may think it’s just another internal request for information that they have successfully ignored or “gone through the motions” in the past.

-        Lack of IT knowledge.  Even employees that try to do the right thing, and understand the basic facts and legal issues, may fail miserably in the collection process. As Judge Scheindlin notes in NDLON V, an employee conducting an email search may unknowingly use the wrong search fields (subject field v. subject field and message body), choose poor logical connectors (AND v. OR) or take other technical mis-steps, substantially impacting the usefulness of their work (NDLON V at 34).  Further, they may not understand that a certain search covers only their mail on the email server and ignores their email archive, laptop, departmental fileshares, PST files, etc.

-        Contrary interest.  Employees may be self-interested in a case and not willing to hand over damaging information; or they may not be interested in the case but not want to disclose information that is personally embarrassing – such as criticizing a supervisor or executive, admitting a mistake, etc.

-        Departed employees.  As Judge Scheindlin noted in the well-known Pension Committee[1] decision, the ESI of employees that are no longer with the company may be relevant to a case.  But if that employee is no longer part of the company, then by definition they cannot identify or collect ESI in a self-collection process.

-        Technical Issues.  Even if everything else falls into place, having custodians perform their own collection may create technical issues related to the actual ESI.  For example, it’s likely that the custodian’s collection will not preserve system metadata for the ESI (e.g. the accessed and modified dates).  In addition, if it becomes an issue, it may be difficult to obtain helpful testimony about how the collection was performed since the employee (probably) does not do this work on a regular basis.  In addition, if the custodian collecting the data later leaves the company, it may be very difficult to authenticate or admit the evidence based on a business records exception.

The problems with self-collection, and Judge Scheindlin’s criticism, are not new.   There are many cases, old and new, showing that the judiciary is concerned with self-collection processes[2].  Here is a sampling of a few well-known cases:

* Pension Committee, supra:  Among other problems with the eDiscovery process, Defendants were sanctioned for over-reliance on self-collection processes by employees;

* Green v. Blitz U.S.A. Inc, 2011 U.S. Dist. LEXIS 20353 (E.D. Tex. Mar. 1, 2011):  Severe sanctions awarded for various eDiscovery mis-steps, including too much reliance on end-users with contrary interests and/or without knowledgeable of the systems or even the facts of the case;

* Cache La Poudre Feed, LLC v. Land O’Lakes Inc., 244 F.R.D. 614 (D. Colo. 2007): A failure to follow-up with key employees and monitor IT’s activities in the collection process was unreasonable, resulting in sanctions;

* Jones v. Bremen High School District, 228, 2010 WL 2106640 (N.D. Ill. May 25, 2010):  Reliance on self-interested employees to collect ESI in a wrongful termination case was unreasonable and resulted in sanctions;

*  Suntrust Mortgage, Inc. v. AIG United Guaranty Corp., 2011 U.S. Dist. LEXIS 33118 (E.D. Va. Mar. 29, 2011):  Company sanctioned after the key employee in a case, who self-collected messages, was found to have changed their content to favor the company (and her own actions).

Why does self-collection survive?  There are undoubtedly a host of reasons – although not all of them are very good.  First, self-collection is similar to how lawyers conducted discovery when it was paper-based.  Employees were (mostly) on their own to review files and provide information that was relevant to a case.  There is some comfort in doing things the way we’ve always done them.  Second, the process fits nicely with many lawyers lack of IT knowledge.  If the lawyers don’t understand how IT systems function in general, and specifically within the company, then there’s no perceived benefit to becoming more involved in the eDiscovery process and reliance on end-users makes sense.  Third, it’s easy and cheap.  Relying on end-users to do the work takes very little time from the legal or IT departments.  Also, there’s no real expense or disruption in the process.

Finally, keep in mind that there is nothing inherently wrong with using the expertise and factual knowledge of your end-user employees in the discovery process.  In fact, understanding what your employees know about a case and where relevant information is located is a required part of the process.  It’s just cannot be the whole process.  The problem with self-collection really only occurs when there is over-reliance and a lack of checkpoints.

So remember:

-        Your employees should be informed about litigation holds and they should be asked to preserve information.  In fact, although it has come under great criticism, Pension Committee mandates that employees be provided with written notification of legal hold information (and it’s a recognized best practice);

-        Reliance on employees should be limited to reasonable levels and in reasonable situations.  For example, in a case involving hundreds (or thousands) of employees, it might be reasonable to ask “outlying” custodians – those that are not in the center of the dispute and less likely to have relevant information – to self-preserve and self-collect.  Similarly, in lower “value” cases, a proportionality analysis might even weigh heavily in favor of self-collection processes;

-        Don’t rely on employees to collect where it appears that they are or may be self-interested in a case;

-        Do provide oversight and spot-checking of the eDiscovery process, even when it seems to be low risk and everything is going well;

-        Do consider other, more systematic processes – such as enterprise tools for eDiscovery search and collection – as part of your eDiscovery toolset.

Over the last several years, corporate awareness of the costs and risks of e-Discovery has grown. In response, many companies have taken some or all of their e-Discovery work in-house to make it more efficient, repeatable and less expensive. At the same time, data volume has exploded and the number of target information repositories for e-Discovery has grown. So more is expected and needed from in-house e-Discovery solutions.

While most in-house eDiscovery teams (legal and IT) have become adept at handling basic tasks, many do not realize that more advanced techniques are possible with sophisticated solutions such as EMC’s SourceOne Kazeon. These techniques can help to expand control and knowledge over corporate ESI. Let’s take a look at a few.

Objective Privilege

The process of determining which documents are protected by attorney-client and/or work-product privilege — and therefore not produced to other parties – is difficult. For that reason, most privilege review is done on a document-by-document basis using skilled lawyers, making it expensive and inefficient. After documents are identified as privileged, they are listed in a privilege log which is provided to the other party. In many cases, disputes follow over whether the documents were actually privileged or if they merely involved communication with lawyers (which may not be privileged), which drives up the cost and difficulty of the process.

In response, some parties have begun to employ a process known as “objective privilege.” The parties agree to a set of objective criteria to determine whether something is “likely” to be privileged , typically using senders or recipients of messages or creators of documents (e.g. in-house and outside counsel); that were sent or created within certain date ranges ; and sometimes including specific keywords. Documents meeting these criteria are deemed to be presumptively privileged and are listed in a privilege log, which is provided to the other party. The presumption of privilege can be challenged as necessary. For the producing party, the process results in far less expense in review, but it also benefits the recipient by enabling production in a much shorter window of time.

Your in-house e-Discovery platform should enable this process very readily. You can quickly run, review and modify searches to determine an appropriate set of criteria to identify a “privilege rich” set of documents. For example, in most cases searches using the term “privileged” would quickly be determined to be useless, because it will identify all emails sent by most attorneys (the term “privileged” is typically found in a disclaimer in email messages). But terms like “advice” or “advise” might be more useful in narrowing the scope, and in many cases a project name or other more unique identifier might be useful. Fast, transparent searches will help to quickly identify terms that are useful – and useless.

Once you have settled on a useful set of criteria – which will probably involve a process of showing the other party why certain terms do not work well – you can quickly mark all “hits” with a tag like “Objective Privilege”. Better systems will then automatically generate a privilege log of such items this and – perhaps most importantly — insure that any later productions of data exclude information from this set.

Custodian Identification and Validation

To create an effective litigation hold process, you must determine the people who have knowledge and information related to the facts of the case. This normally requires interviewing key witnesses, but a truly diligent process will include some review of those custodians’ ESI to confirm (or expand) on that initial base.

Unfortunately, this task is not as easy as it might seem. A litigation hold needs to begin upon “reasonable anticipation of litigation”, not a few months after receiving the Complaint or when discovery begins in earnest. Reviewing ESI early in the process requires the ability to quickly access that information, not waiting weeks for collection, processing and hosting of data. You should be able to quickly access email, always a rich trove of information, and in many cases due some checking on fileshares, desktops and perhaps even Sharepoint.

Let’s look at a quick example. You have an employment discrimination case with “John Doe” alleging discriminatory treatment. In determining your custodian list, it’s useful to quickly locate all of John Doe’s emails (sent to or by him) during the relevant time period and see the people that he communicates with on a normal basis. If you have access to a full email archive, it’s even more useful to generate emails threads from any interesting messages, to see if “John” sent copies of the messages to others (or an external mailbox for safekeeping) and even to review later messages from that thread where John may have been cut of the conversation. Even when this type of analysis does not turn up any new information, it’s a useful step to establish the level of diligence performed in reasonably establishing the bounds of the legal hold.

Keyword Search Development /Query Analysis

Today’s e-Discovery is, for better or worse, largely based upon keyword search terms. Once the basic repositories and custodians of relevant data have been determined, parties will frequently negotiate sets of keyword searches that will (hopefully) produce information on specific issues. For example, in a breach of contract case where the shipment of “K506 widgets” is an issue, one party might suggest that any emails or documents with the terms “shipment” or “ship” or “shipping” or “mail” or “deliver” (etc.) that are close in proximity to the term “K506 widgets” would yield important information. Similarly, the act of delivering could be specified in so many different ways – deliver, delivery, deliverable – and even misspelled – delivereble, delivere, etc. – that if it’s an important term you might be required to search for any words that begin with the common stem, “deliv”, using what is normally called a “wildcard search”.

Better eDiscovery solutions will assist the parties in determining useful keyword searches by providing fast and detailed feedback about the search terms. First, you want fast searches that let you immediately see results so that you can keep modifying a search until you achieve the necessary results, whether related to quality, volume or another requirement. Second, you should be able to see the precise terms that are generating “hits”, especially if you are including a wildcard. Maybe the search for a heart monitoring product, using “heart*” has returned thousands of hits. But if most of those are “heartache” or “heartburn”, it may not be a good search and it’s useful to have the system tell you how your terms are being expanded so that you can modify them to make them better.

Conclusions

A good in-house e-Discovery tool can provide many different capabilities to improve your litigation process. Spend some time learning about advanced features and get creative on how you use them. (And if you already have some good ideas, let us know with the feedback button below).

Discover More

Director, EMC eDiscovery and Compliance Field Practice

—-

We talk with companies every day about how they can be better at managing their enterprise information.  Good policies, with technology to enable and enforce them, can help insure that records and compliance information are retained for the right amount of time, while also enabling the deletion of stale and useless information which has outlived its retention period.  Good information management processes insure that protected information is stored in the right place, operational efficiencies are enhanced by focusing on useful information and the e-Discovery process is easier and more efficient.

Many organizations know that they should implement information management initiatives, but often have difficulty in providing concrete reasons to the business.  If your organization is looking for more reasons why good information management is valuable, two recent cases provide some great reasons:

-        If you have an information governance policy, it may help you to defeat a claim for sanctions even if data has been deleted; and

-        If you don’t have an information governance policy, and you delete data that was subject to compliance requirements, the lack of a policy can help to establish the bad faith necessary to award sanctions.

Diligence As A Shield

In Danny Lynn Electrical & Plumbing, LLC v. Veolia Es Solid Waste Southeast, Inc., 2012 U.S. Dist. LEXIS 62510 (M.D. Ala. May 4, 2012), the plaintiff requested sanctions for the defendants’ alleged failure to properly implement a litigation hold.  Specifically, the plaintiff claimed that defendants had deleted nine email accounts and kept in place an auto-delete function which removed email from the trash after 10 days.  They also alleged that the defendants improperly sent notifications to employees on legal hold that they should continue to delete email messages to comply with email account size limitations.

The court found it significant that the defendants had deployed an email archive to capture all of its email messages.  (Interestingly, the court did not discuss or make any findings about how the archive had been setup, configured or managed).  In addition, in finding that there was no bad faith (a requirement in the 11^th Circuit), the court found it important that defendants “began using a software system that archives all emails”:

The court’s impression is that the defendants have expended great effort to insure that the plaintiffs receive information from both their live and archived email system by providing document review technology and allowing access to its database.

All of these factors added up to the court finding that no sanctions were warranted.

Lack of Diligence Can Be A Final Straw

The flip side to the protection offered by information management can be found in FDIC v. Malik, 2012 U.S. Dist. LEXIS 41178 (E.D.N.Y. Mar. 26, 2012) where the court also considered a spoliation motion for the deletion of emails.  The email messages related to a law firm’s prior representation of a mortgage company.

In determining whether bad faith was present to enable sanctions, the court noted that the subject email messages were required to have been preserved not initially for litigation hold, but under compliance requirements — professional responsibility and ethical rules.  The court found that retention under the compliance requirement was especially important to this case:

A regulation requiring retention of certain documents can establish the preservation obligation necessary for an adverse inference instruction where the party seeking the instruction is ‘a member of the general class of persons that the regulatory agency sought to protect in promulgating the rule.

The court held off a final decision pending an evidentiary hearing.

Being Proactive With Information Management

We all know that litigation holds are difficult to implement and are almost never perfect.  Sometimes something bad actually does occur– a custodian is inadvertently omitted, a handful of emails are lost.  But more often, nothing bad happens at all.  Still, even in those cases it can be difficult (and time-consuming and expensive) to fight off the other side’s claim that something “must have been lost.”  A good information management policy, with tools and education to enable it, can go a long way towards showing good faith and protecting your organization from harm.

Discover More

Wednesday, April 18, 2012, 8:00 am PT / 11:00 am ET

To understand why compliance tops the list of business concerns, just look at the headlines: fraud, data theft, lost information, corporate espionage, and leaks. The costs and risks of unmanaged content are staggering. EMC can help you identify and protect costly and potentially risky content throughout your organization so you can avoid the consequences of unmanaged information and meet compliance requirements.

Attend this webcast and learn how you can extend your current EMC solutions to:

Reduce the costs associated with managing unstructured content

Mitigate risk by identifying and securing content based on its value to the organization

Ensure compliance by automatically enforcing policies that govern access and retention—inside and outside the enterprise

Speakers:

Garth Landers
Director, Cross Category Marketing, EMC Information Intelligence Group

Lori McKellar
Director, Product Marketing,  EMC Information Intelligence Group

Register Now>>>>

Thursday, March 22, 2012
Networking & breakfast: 8:00 – 8:30 a.m. EST
Program: 8:30 – 10:00 a.m. EST

Register>>>

Location: Venable LLP
Rockefeller Center 1270 Avenue of the Americas (between 50th and 51st Streets)
Twenty-Fourth Floor New York, NY 10020

Speakers: Edwin M. Larkin, Esq. Partner at Venable LLP, James E. Nelson, Esq.
Partner at Venable LLP, James D. Shook, Esq.  Director of the eDiscovery and Compliance Team at EMC Corporation

In litigation matters, you are responsible for any relevant data that is in your “care, custody or control.” But what happens when that data is stored in “The Cloud”? Under such conditions, traditional notions of ownership and control no longer apply in the same manner as they have for physical goods, such as paper documents. What happens to your data – and your obligation to a court or regulator – when that provider will no longer access “your” data because they have terminated your account for non-payment or alleged mis-use? What if your provider needs 3 weeks to collect your data, but you have to respond within 3 days?

Join our panelists for a thought-provoking discussion in which we will discuss: the basic framework of regulatory and compliance issues that you need to know when you entrust your data with a third party;
questions to ask and contract terms to consider when you are entering into a “cloud” agreement; and
different types of cloud models to consider that might fit your needs.

Register>>>>

For questions about this event, please contact Dawn M. Mixon at DMMixon@Venable.com

Heidi Maher, Esq., eDiscovery Expert

At a recent a conference for compliance and IT professionals working in the state government sector, it quickly become evident that one of their main concerns was the tremendous increase in the number of open records requests that they have to deal with.   Both the federal and state governments give much lip service to the theory of transparency but few have made the necessary changes to properly deal with the onslaught of requests that appear almost daily.  Wisconsin’s Governor, Scott Walker’s administration has already produced 60,586 pages of open records in response to 222 requests in 13 months.  Compare that to 312 requests filled during the previous governor’s first 4 years[1].  It’s not just Wisconsin that is dealing with an explosion of open records and FOIA requests.  The U.S. Department of Defense received 67,434 in 2009 compared to 74,573 in 2010 and the National Archives and Records administration received 14,075 in 2008 compared to 18,129 in 2011[2].  Most government entities handle open records requests the same as they handle eDiscovery for litigation, manually and on an ad hoc basis.  Unfortunately for government agencies, the turnaround for a response is much quicker than for litigation.  Federal agencies have a statutory requirement to respond to requests within 20 business days [3].  State agencies have time limits ranging from 10-30 days or within “a reasonable time.”  For this reason, IT departments are struggling to keep up and there is a substantial backlog at most agencies.

Adding to their concerns, metadata could be a factor in public disclosure requests. Within the hard drive of any standard computer, metadata is created with each underlying electronic document. Metadata describes the document’s history, tracking and management. In Arizona and Washington that metadata, when requested, is now also subject to a public disclosure, along with the underlying document itself.  On a national level, a ruling by Judge Scheindlin in February of 2011 stated that responses by the federal government to FOIA requests must include metadata and be in a searchable format [4].  Although, she withdrew her opinion later that year (she said it was not based on a full and complete record), her original ruling will undoubtedly influence other courts grappling with public disclosure disputes, especially as they become more technologically savvy.

At this same conference, we heard from the CIO of a large state agency who revealed the tremendous cost of dealing with open records requests especially in a year where his agency has been the subject of several news stories and litigation [5].  The agency took several steps to reduce the cost and time associated in responding to these requests.  The first step was to perform file remediation on data that was not a record and met no legal, regulatory or business requirement for retention.  Next, they began the process of implementing an email archive in order to enforce retention and have one repository of record for all emails instead of dealing with local email storage on each hardware device.  In the meantime, they have installed in-house search technology that has allowed the agency to find and copy the requested information in a matter of minutes whereas the same action used to take several days.  When the occasional litigation notice came through, they have been able to utilize the same technology to put the requested information on hold.

Another concern for government agencies is the prospect of moving some or all of their data to the cloud. In fact, federal agencies were directed by President Obama to consider cloud based services or storage systems for records keeping [6].  The challenge then becomes how to facilitate cloud management of that information while still responding quickly to public record requests.  Any agency contemplating that move must ensure that the data being managed by the cloud provider is maintained in an easily accessible manner and that the provider is contractually bound to have technology in place for easy and fast retrieval of data for responding to eDiscovery.  Otherwise each request may be billed as a special project and the cost savings initially realized can quickly dissipate.

A possible step in the right direction is the common web portal for FOIA requests launching in the fall of 2012.  According to The FOIA Ombudsman, the $1.3 million portal, being built mostly with funds from the Environmental Protection Agency and the Commerce Department, with some participation from NARA, could save the federal government $200 million over 5 years were it to be adopted government wide[7].  This is a big step toward giving the public a self service model (similar to a tool utilized by government agencies in Mexico[8]).  However a portal is only as good as the data behind it so only time will tell if this can serve as a national model.

Although the government is notoriously behind the private sector in modernizing its technology, the public’s need for an open and transparent government does appear to be speeding up the process to the benefit of agency budgets and more importantly, the taxpaying public.

[1] http://bit.ly/z1x2hf

[2] http://www.foia.gov/

[3] 5 U.S.C. § 552(a)(6)(A)

[4] Nat. Day Laborer Org. Network v. U.S. Immigration and Customs Enforcement Agency (“NDLON”) 2011 WL 381625 (S.D.N.Y. Feb. 7, 2011)

[5] While public disclosure rules allow for collecting fees and recovering costs, some requesters who qualify for placement in favored fee categories may be charged less or may not be charged at all. Educational, news media and noncommercial scientific requesters typically pay no search or review fees and only duplication costs after a certain number of pages (usually 100 or more).  The amount of paper that is created by these responses is unacceptable.  Taxpayers are right to question why so much of their money is spent creating paper documents when 93%+ of all communication is in an electronic format (David W. Degnan, Accessing Arizona’s Government: Open Records Requests for Metadata and other Electronically Stored Information after Lake v. Phoenix, 3 Phoenix L. Rev. 69 (2010)).

[6] http://www.whitehouse.gov/the-press-office/2011/11/28/presidential-memorandum-managing-government-records

[7] http://blogs.archives.gov/foiablog/2012/01/09/foia-portal-moving-from-idea-to-reality/

[8] http://indices.ifai.org.mx/wb2, also http://www.dhs.gov/xlibrary/assets/privacy/privacy_wkshop_04-2006_personal.pdf

James D. Shook, Esq., EMC eDiscovery Expert

By James D. Shook, Esq.

In light of Magistrate Judge Andrew Peck’s recent decision in Da Silva Moore v. Publicis, much has been written and discussed about the idea of using machine learning techniques to automatically classify documents during review, a process sometimes known as “predictive coding” or even “computer assisted review”.    (Although these terms may actually imply different technologies and processes this article adopts Judge Peck’s umbrella use of the term “predictive coding”).  This article explores some of the key issues around this promising intersection of law and technology.

What Is Predictive Coding?  How is It Used?

At a simple level, predictive coding is just a technological “lever” that allows a (relatively) small amount of review work – usually by humans — to be leveraged across a much larger set of documents.  Let’s say that we have a class action where the identification phase of eDiscovery has located about twenty million electronically stored information (ESI) items – email messages, word processing files, spreadsheets, powerpoint presentations, etc. – that are likely to be relevant, discoverable information for the issues in our case.

Traditionally, we have had a few choices about what to with all of this ESI.  First, we could just hand it all over to the other party without reviewing any of it for actual relevance or even privilege.  Second, we could negotiate search terms with the other side, which we would then run against the ESI in an attempt to locate the most relevant information.  All non-privileged ESI that was a “hit” with the search terms would be handed over to the other side.  Third, we could have human reviewers reach each item to determine what is relevant (and non-privileged), and then produce that information.  Each of these approaches has its benefits and problems, including the amount of time that the process takes, the cost of reviewing millions of items, the validity and usefulness of the process, etc.

With predictive coding, we have another choice to help determine what information is relevant.  In this approach, we carefully review a small subset of the 20,000,000 item set – maybe as few as 2,500 items depending upon the margin of error that we can tolerate in our result.  We then use predictive coding technology to “learn” from that set, applying that “knowledge” to the remaining 19,997,500 items.

Is Predictive Coding Worth the Trouble?

The effective use of predictive coding impacts the litigation process by substantially reducing the cost and time for review.   In our example, in applying a predictive coding process we might manually review just 2,500 documents of the 20,000,000 set, depending on the specific technology, case requirements and tolerance for potential errors.  If we assume a cost of $5 per document for high-level manual review (by a more experienced attorney) and just $0.50 for bulk review (by contract attorneys and paralegals), we would spend about $12,500 in review (plus the costs of the predictive coding technology process).  This cost is in stark contrast to a complete manual review – which in this case could cost up to $10,000,000!  (Note that in Da Silva, there were about 3.2 million documents.  At one point the parties estimated a cost of $5 document to produce a projected set of 40,000 documents.  Hearing Transcript at 62; Order at 6).

The predictive coding process takes less time because the computers handling the review are much faster, more consistent and can work longer hours than people.  If we assume a review rate of two documents per minute, then the manual phase of our predictive coding process requires about 2 people-days (the machine-based review would come after this phase).  In contrast, a complete, eyes-on review process would take 20,833 people-days!  Even with a large team of 100 reviewers, that process would take over 200 working days and require strong project management to complete properly.

Perhaps more important, according to recent studies the predictive coding process is also more effective than human or keyword review.  Unfortunately, it is difficult to determine the true accuracy of human review because opinions, even among experts, can vary on whether a document is relevant to a case.  (Maura Grossman & Gordon Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, XVII Rich. J.L. & Tech. 11 92011 at 9).  But the bulk of available information implies that machine coding is better.  In fact, some studies put a human reviewer’s recall– the percentage of relevant documents actually located – at less than 50%.  The use of basic keywords is even worse, dropping recall to about 25%.  (Grossman/Cormack at 18-19).  Some predictive coding studies indicate that the process is far more accurate, in the range of 70% recall (Grossman/Cormack at 36-37).  Given the lower cost and speed, recall that’s even close to the human level should be acceptable.  (Note that other measures, such as precision and F1 (the harmonic mean of recall and precision) – are also important in this process.  For more information see Grossman/Cormack at 9).

What are the Barriers to Using Predictive Coding?

Despite Judge Peck’s opinion, there are number of real-world barriers to using predictive coding on a regular basis:

-        The underlying technology and math can appear complicated, and Judges and lawyers may not jump at the chance to use predictive coding until they better understand the process and there is more guidance from the bench on when and how to use it.  Although Judge Peck has stated that the requirements of Daubert do not apply to predictive coding, there remains a comfort level and learning curve that will probably take some time to achieve;

-        There are not yet many studies establishing that predictive coding is better than human or keyword review, even though many believe that is the case.  Further, “predictive coding” is just an umbrella term for

-        There is not a clear process of how to approach the problem when one side wants to use predictive coding by is opposed by the other party.  In Da Silva Moore v. Publicis, both parties agreed to use predictive coding and Judge Peck’s order addresses issues related to the protocol and process of how it is to be used in the case.  However, in Kleen Products LLC v. Packaging Corp. of America, Magistrate Judge Nan Nolan will be focusing on that issue:  can a party require computer-assisted review over the objection of another party?  Stay tuned — Kleen Products has another hearing scheduled for April 2012.

What Are the Barriers to Not Using Predictive Coding?

Interestingly, the continued massive growth of data may ultimately force the use of predictive coding technologies.  IDC projects that most companies will be dealing with 50 times the amount of data in 2020 than they had in 2011. People working in the eDiscovery industry also know that most cases today ignore many potentially relevant repositories of data, either by agreement or thru lacking of knowledge.  If the additional repositories are included, along with the enormous projected growth of data, it’s likely that the amount of ESI in many cases will soon be too expensive for “normal” review by humans — or it will take too long.

In addition, while it may seem farfetched today, the requirements of proportionality could soon mandate that parties use predictive coding technologies to insure that the litigation process remains just, speedy, and inexpensive.

What is the Future?

With cases like Kleen Products on the near horizon, it seems likely that we will be receiving some judicial guidance on predictive coding over the next year.  That’s certainly good news, as the more guidance we receive, the more likely that lawyers and litigants grow more comfortable with the process.

In addition, predictive coding technologies show promise outside of the litigation process to help with our information management overload issues.  Imagine training your email, fileshare or Sharepoint archive, or even your records management system, to recognize and automatically classify information as it is received or created.  “I’ll get milk on the way home” messages could be flagged for quick deletion, while employee reviews, contract modifications and other records could be stored according to your file classification plan.  (This technology already exists; improvements, higher comfort level and better understanding of the technologies caused by their use in litigation will help with the adoption rate).

For more, check out the references in “Dive Into Machine Classification and Coding”, part of my New Year’s Wish list.