Professional Documents
Culture Documents
Textual search of electronically stored information ("ESI") is a necessary part of many business and legal endeavors Not all native computer applications offer adequate text search functionality Textual search of ESI requires special indexing of content for effective and efficient results Not all text search tools or environments are the same There are other technologies used for analyzing ESI textual content, which may or may not be directly related to search The where, when and why of search requires consideration of the tools, environment, consequences and practicality of a search endeavor
Where
When
Post-Triggering Event Pre-Review, Concurrent w/ Review Post-Review Normal Course of Business
Why
Post-Triggering Event Pre-Review, Concurrent w/ Review Post-Review Normal Course of Business
Knowledge Management
Assessment / Scoping
success
The why may introduce legal implications, and the necessity for
document identification will likely not require proof of the comprehensiveness of the search attempt, or require documentation of a reasonable search process if the document or specific fact is actually returned by the search
High Legal Consequence: Search used to determine the ESI that
will be reviewed and produced in an e-discovery event, and conversely that which will be left out of review requires a documented process to enable improved accuracy and comprehensiveness of the search attempt
make determinations on scope or approach in handling a matter; likely requires less scrutiny than operationalizing a process resulting from such assessment
Review efficiency search measure of success is adding efficiency to the review
of an already determined review population where inaccuracy will not necessarily result in missing documents in production
assessment of comprehensiveness of the search result to withstand a defensibility challenge balanced with an avoidance of over-inclusiveness
Privilege search as a form of review efficiency search this has a special legal
impact as the potential for waiver of privilege may depend upon the reasonableness of the search process and results so the measure of success is similar to review determination.
How the when and where can adversely affect search with legal implications
Search "in-place" can introduce issues related to dynamic search
indices and data sets affecting the comprehensiveness of the search attempt
Are the search indices complete at the time of search? Is the source data stable, or are there possibly changing file locations, pointers or
The available tools may not account for all items sufficiently for
How the when and where can adversely affect search with legal implications
Available tools may lend themselves to one search objective more
than another
Ad Hoc Search (suitable for fact finding and assessment searching) Process-driven search, allow measurement of results at the macro and micro
documentation of criteria modification, and validation of results (suitable for effective search process involving legal implications)
unstructured data
Recall search results are inclusive enough (defensibility) Precision search results are not overinclusive (proportionality)
Ad hoc search is a largely manual and interactive way of searching a data set in which a user constructs a single query with varying degrees of complexity and retrieves the results of that one query for immediate perusal.
Suitable for:
Research Testing Fact finding Document location
Process-driven search allows scripting of a series of concurrent queries with varying degrees of complexity, recording the results to allow measurement of the interaction of individual queries and overall impact of the set
Suitable for:
Large numbers of queries Iteration of entire sets of queries Measure impact of sets of queries Auditing, tracking and documentation of iterative search attempts Practical sampling and validation opportunities
Search Term Results Under-inclusive search. Good Candidate for defensibility challenge Not an unduly expensive, but yet incomplete review scenario
Targeted search. Unlikely candidate for defensibility challenge Right-sized review scenario as to cost and efficiency
Case Law
Victor Stanley Inc. v. Creative Pipe Inc.
250 F.R.D. 251 (D. Md. 2008)
all keyword searches are not created equal; and there is a growing body of literature that highlights the risks associated with conducting an unreliable or inadequate keyword search [parties] do not assert that any sampling was done of the text searchable ESI files to see if the search results were reliable. Common sense suggests that even a properly designed and executed keyword search may prove to be overinclusive or under-inclusive [t]he only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents in order to arrive at a comfort level that the categories are neither overinclusive nor under-inclusive. [excerpted and highlighted for emphasis]
the only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents - Judge Grimm
Case Law
In Re Fannie Mae Securities Litigation,
552 F.3d 814 (D.C. Cir. 2009)
OFHEO will work with the Individual Defendants to provide the necessary information (without individual document review) to develop appropriate search terms.
OFHEO did not participate in the determination of appropriate search terms, but rather allowed opposing party to use untested search terms. They presumably assumed it was the responsibility of defendant to determine appropriateness. This resulted in a large volume of documents to review, which proved very costly, and OFHEO in fact did not complete the task. The court disagreed with assertion that defendant was limited by the stipulated orders language related to appropriateness.
direct[s] OFHEO and the individual defendants to work together, but only to facilitate OFHEO's provision of information to assist in developing search terms
Over-inclusive search. This caused OFHEO to spend large sums on review. OFHEO had an opportunity as per the stipulated order to provide information that may have resulted in more targeted review
Case Law
William A. Gross Construction Associates, Inc. v. American Manufacturers Mutual Insurance Co.
256 F.R.D. 134 (S.D.N.Y. 2009)
Judge Andrew Peck where counsel are using keyword searches for retrieval of ESI, they at a minimum must carefully craft the appropriate keywords, with input from the ESI's custodians as to the words and abbreviations they use, and the proposed methodology must be quality control tested to assure accuracy in retrieval and elimination of false positives. It is time that the Bar-even those lawyers who did not come of age in the computer era-understand this.
Case Law
United States v. OKeefe,
537 F. Supp. 2d 14 (D.D.C. 2008)
for lawyers and judges to dare opine that a certain search term or terms would be more
likely to produce information than the terms that were used is truly to go where angels fear to tread if defendants are going to contend that the search terms used by the government were insufficient, they will have to specifically so contend in a motion to compel and their contention must be based on evidence that meets the requirements of Rule 702 of the Federal Rules of Evidence
To liberally paraphrase Judge Facciola one might say prove it. If parties want to contend that particular search terms are sufficient or insufficient a formalized search process capable of documentation, measurement, and validation should be available, accompanied by individuals wellversed in such information retrieval efforts.
- Daegis EDAC
[sampling techniques] can help the review team to rank the precision and recall of various terms or concepts.
defensibility in court will very likely depend on the implementation of, and adherence to, processes developed for use with a search and retrieval technology.
Best Practices Commentary on the Use of Search and Information Retrieval Methods in E-Discovery (08/2007)
There are several validation methodologies that should be used throughout the development of the search criteria to be used for selection of documents for attorney review. [Many involve the case team] reviewing samples of documents to determine litigation relevance to classify documents as Responsive or Not Responsive to the issues of the case and therefore increasing the precision of the search results.
Questions