Co-Founder & Chief Operating Officer
Privilege review - the process of identifying which documents should be considered covered by attorney-client privilege - is a key element of litigation matters. But the time and expense spent on poring through thousands of documents for privilege review has long been a pain point for counsel. In conjunction with the ACC Central Ohio Technology Committee and Litigation Network, Text IQ convened four industry leaders as part of our The Inevitable 2020 Series to discuss whether artificial intelligence (AI) can help play a broader role in litigation matters - beyond reducing the headache of manual privilege review.
For the GC panelists, their initial misgivings were that AI-based approaches couldn’t match the understanding of context and content that human reviewers provided. Instead, what they discovered was that the benefits extended beyond bringing down the overall cost of privilege review through automation. The tool helped mitigate risk since it picked up clues that humans had missed to more consistently identify and classify a larger number of potentially privileged communications and documents. And, the speed at which the tool functioned has the overall impact of getting to the substance of the case quicker.
Moderated by Eversheds Sutherland veteran Litigation Partner Robert Owen, the discussion moved from how AI works, how the technology is applied to litigation procedures and what the future potentially holds. Joining Robert were:
● Associate General Counsel at GlaxoSmithKline, Dispute Resolution & Prevention (DRP) Group, John O’Tuel
● David Orensten, Associate General Counsel of Litigation for Cardinal Health, and
● Text IQ CEO and Co-Founder Apoorv Agarwal
What is Artificial Intelligence and How Does it Work?
Apoorv, an original member of the IBM Watson team, kicked off the discussion helping to demystify AI and the various types of machine learning algorithms currently in use.
He traces the beginnings of artificial intelligence as a rule-based system executing against rules established by programmers – but not learning from data – to current day AI which is statistical in nature and uses various algorithms to learn. “As soon as the machine looks at the data, and uses statistics to learn from it automatically, that is machine learning,” notes Apoorv.
"Whenever we're getting humans to create a seed set, there's a limitation on the amount of data humans can label and bias really comes from this limitation."
Importantly, he sheds light on the limitations of “supervised” machine learning and the introduction of “bias,” helping to elucidate the arguments surrounding “seed sets” created by subject matter expert reviewers intended to “teach” the machine.
How are Litigators Using AI to Mitigate Risk and Lower Cost?
Associate General Counsel of Litigation for Cardinal Health David Orensten notes that “AI has shown to be better at identifying Potentially Privileged and other confidential documents.”
"A lot of the ways I've seen it used aren't necessarily that far down the road in terms of relying on AI to make privileged calls, but I've seen the benefits related to risk mitigation."
David talks through the Proof of Concept he used to test Text IQ’s PRIV IQ tool against a prior document review.
Not only was the result generated through AI, 100% inclusive of all documents that were recorded on the privilege log in the prior manual review, but thousands of additional potentially privileged documents were found that human reviewers had not identified.
For Cardinal Health, the efficacy of AI in privilege review was important in reducing cost, but the broader value was in risk mitigation (by flagging privileged documents that would have otherwise been missed) and in speeding up the entire process of litigating the case at hand.
What Does the Future Hold?
Over the years we moved in a stepwise approach first utilizing more rules-based AI systems to do QC on human reviews as well as check the precision and recall search methodology notes GlaxoSmithKline, Dispute Resolution & Prevention (DRP) Group’s, John O’Tuel.
"AI has the ability to really help out with staging review such that you get to those more meaningful documents earlier... It has it has a cost benefit in traditional litigation, but even more so, I think just getting down to the substance of the case quicker."
From there we began utilizing technology assisted reviews to prioritize, bucket, and identify hot docs from opponent productions and finally using AI to prioritize, stage, and bucket our documents for review but still using humans for the actual review.
O’Tuel continues: In terms of future state, when we talk about trying to find needles in a haystack or certain categories of documents – whether it's for privacy purposes or compliance – if you're trying to create a program related to insider trading or Foreign Corrupt Practices Act, I think AI has uses to be able to try to find those sorts of communications where manually it might be very difficult.
There is much more that is talked about in a no-no nonsense manner that is infused with the experience of these subject matter experts.