On October 30th Text IQ brought together a lawyer, a general counsel, and a legal service provider to discuss testing the value proposition of artificial intelligence (AI) through the use of a proof of concept as part of our The Inevitable 2020 Series.
Our three panelists share quite a bit in common. This combination of professionals is the quintessential "dream team" for document review. Their focus is discovery and discovery-related issues. They also recently collaborated on a proof of concept (POC) using artificial intelligence to identify privileged and personal information.
Briordy Meyers serves as Director & Senior Counsel - E-discovery for Boehringer Ingelheim USA. Briordy also works closely with Boehringer’s data privacy officers to ensure compliance with U.S. data privacy laws and cross-border transfers of data subject to the GDPR.
Sidley Austin LLP, Data Analytics & Discovery Counsel, Matthew Jackson advises Fortune 100 clients regarding best practices on information management, preservation, discovery-readiness solutions, and defensible deletion. He is a member of Sidley’s E-Discovery Task Force and a published author and speaker on e-discovery issues.
James Calvert is Discovery Counsel at Troutman Pepper eMerge where he advises clients on all phases of discovery, negotiates discovery agreements, and advocates for clients involved in discovery disputes. Prior to focusing on e-discovery, his practice included antitrust and trade regulation and commercial litigation.
Why Do an AI Proof of concept?
“Looking for savings, looking for a return on investment. If you're in-house like I am you are part of a business...a cost center. So how do you show savings and show your value? That's just self-interest for our department. The more on the nose pressure points? In our workflows, whether internal or external case-specific triggers, things keep coming up: triggers for change.” ~Meyers
Briordy continues noting that Boehringer Ingelheim, a German company, hosts a lot of data in Germany making that data subject to GDPR. Managing compliance with GDPR and the competing demands and potential conflicts with US-based litigation was “something that we really wanted to solve for.”
“It’s innovate or die,” adds Sidley Austin’s Jackson. The velocity, variety, and sheer volume of data generation are coming into play, not just the discovery, but for data privacy as well. “We have to figure out ways to continue to be cheaper, faster, better [to] meet our client's demands.”
How Do You Do a POC?
Matt notes that a crucial decision have to make is, “what am I testing and what are the parameters of what I'm testing?” Getting as close to a live experience with all the variables of a live investigation or litigation that you deal with on a daily basis is key for a good test.
Trautman’s Calvert goes a step further. His preference is to test on an actual project with contingencies in place in the event that the POC does not play out.
“I think the perfect scenario to test a new technology would be where if the test works out, as you hope it will, you're going to be able to work with those results. I think that's the ideal dataset from my standpoint.” ~Calvert
Of course, prior to actually running a proof of concept you need to get buy-in from your key stakeholders. Here the panelists offer some crucial insights into change management, clearly articulating the value of the proposed solution, and gaining not just acceptance, but more significantly, the commitment of key stakeholders.
The audience is privy to these insights from three key vantage points: law firm, corporate legal department, and vendor. They also discuss how to integrate new technologies like AI into existing workflows. It is well worth the listen.
How Do You Measure Success?
“Designing it takes, some targeted focus...to be able to report on [the outcome] and compare it to what it would have been in the traditional workflow. So that everybody involved: law firm, client, and the technology provider are going to be able to understand what those different metrics are and help inform decisions.” ~Calvert
Jim goes on to note that designing your POC with evaluative metrics in mind changes “the leap of faith to a well-informed decision.” Briordy notes that key performance indicators (KPIs) will be different for everyone and will include hard metrics as well as softer attributes like “easier.”
Key Performance Metrics for a POC the panelists agree on include:
- Cost Reduction
- Risk Mitigation
- Time Savings
- Labor Savings
- Ease of Use
All of these benefits are a function of AI accuracy compared to traditional review methods. More on that can be found here. Another important attribute? The ability of the AI solution to seamlessly integrate with human-based workflows and support attorneys and others in their decision-making capacity. Here again the audience given a vantage point from three crucial perspectives: legal advisor, client, and vendor.
Establishing a meaningful POC has always been difficult in legal technology. Viable POCs that truly reflect the reality of e-discovery workflows and challenges, particularly as they pertain to privilege and PII reviews, exceptionally so.
Learning how our three panelists solved for this challenge is must-listen material. Learn more about doing a POC here.
Watch the webinar on-demand here.