Generative AI-Assisted Review: Hello, World!

From the earliest days of computer programming, users have dreamed of computers speaking to them, and responding in natural language.  Since at least 1972, computer programmers have been testing new programming languages and computer systems by writing “Hello, World!” programs, making it a tradition for “hello” to be the first thing a new computer system “says.”

Today, the advent of Generative AI (GenAI) has finally given the “power of speech” to computers in a way that seems almost like science fiction, and in the context of eDiscovery, GenAI represents the latest advancement in the two-decade long struggle between legal technology and the ever-evolving and expanding world of electronically-stored information (ESI).  In particular, GenAI’s power of speech makes it a powerful new tool to aid in legal document review.

From TAR to GenAI

For more than a decade, the legal industry has successfully leveraged various technology-assisted review (TAR) tools to aid in legal document review, from predictive coding to continuous active learning (CAL).  In any of these use cases and workflows, the underlying tool – regardless of which algorithm – is acting as a classifier.  It is classifying the documents into the buckets that have been defined for it, usually one binary pair of buckets at a time, like relevant documents to be reviewed and not relevant documents to be set aside.

Like TAR and CAL tools before it, GenAI can also function as a classifier.  Various research and testing efforts have now shown that GenAI tools are capable of achieving results comparable to those achieved by human reviewers, which means GenAI tools can be used to enhance and support document review efforts in the same ways and workflows as TAR tools:

• Accelerating Review
  • Speed is often crucial in legal document review – not just speed to completion, but speed to finding the most important information. Like TAR tools, GenAI can be used to identify and prioritize the most important documents so they get reviewed first.

• Quality Control
  • Quality control of legal document review is even more important than speed, to ensure you produce all that you must and no more. Like TAR tools, GenAI can be used to provide an enhanced quality control layer for human document review.  Agreements between the human and GenAI classifiers suggest correctness, while disagreements indicate where senior team members should look for issues.

• Reducing Review Volume
  • TAR tools were initially introduced to the legal industry as a better way than keyword searches to divide large collections of documents into two piles: those potentially-relevant documents that must be reviewed and those likely-irrelevant documents that could be ignored. GenAI can fulfill this role too, and its classifications can be validated using the same well-established methods to measure recall and elusion.

Potential Benefits of GenAI-Assisted Review

Leveraging GenAI tools in workflows for document review – instead of TAR 3.0, let’s call it GenAI-assisted review (GAR) – can offer the same kinds of speed, efficiency, and quality benefits as leveraging TAR, but it can do so while also providing one major benefit not available from TAR:

• Increased Speed
  • As noted above, GAR can offer dramatic increases in speed over unassisted human review. Exact throughput will depend on the tool you are using and how you are using it, but speeds so far seem comparable to those possible with TAR and CAL, and that speed may increase over time.

• Increased Efficiency
  • Like TAR, GAR is a force multiplier that multiplies the effort of a few into the effort of many and the effort of a few hours into the effort of hundreds. Each case team will need to consider when to use that increased capacity to spend less and when to use to increase quality by doing more.

• Increased Quality
  • It is axiomatic that two heads are better than one, and classifiers are no exception. Like TAR, GAR offers a second classifier that can work in concert with your attorney reviewers to produce higher quality results than either classifier could produce on its own, the comparison revealing what’s certain and what needs another look.

GAR Says “Hello, World!”

Beyond providing benefits like those of TAR, GAR offers an entirely new kind of benefit.  Because GenAI provides the power of speech, GAR can do more than just tell you what, it can also tell you why.

When a TAR tool evaluates a document for relevance it provides a score, a numerical value representing how confident it is that this document is responsive to your inquiry.  When a GenAI tool evaluates a document for relevance, however, it can provide an explanation of its rationale and the supporting excerpt(s) from the document.  This additional ability has potential benefits throughout the document review process, for example:

• If a GenAI tool has been used to prioritize a attorney review, the human reviewers may benefit from seeing why each document was included and where to focus their attention.
• If a GenAI tool has been used to decide what will and won’t be reviewed by humans, the explanations and excerpts may enhance the evaluation and analysis of its results.
• If a GenAI tool is used as an aid to quality control of human review, senior reviewers may gain insight and efficiency from knowing why the tool disagreed with the human reviewer.

Key Takeaways

Like TAR and CAL tools before it, GenAI can also function as a classifier, which means GenAI tools can be used to enhance and support document review efforts in the same ways and workflows as TAR tools: acceleration, volume reduction, and quality control.  Leveraging Gen AI-assisted review (GAR) can offer the same kinds of speed, efficiency, and quality benefits as leveraging TAR, but GenAI’s ability to “speak” can also offer you the powerful, new benefit of an explanation and supporting excerpts.