Last year, firms were testing on average as many as three to five genAI models in the hope of reducing workloads, and that also meant employing supportive resources such as innovation teams and knowledge management professionals, Gartner’s Friedmann said.
People have been talking about the demise of the billable hour for about 30 years “and nothing’s killed it yet,” said Ryan O’Leary, research director for privacy and legal technology at IDC. “But if anything will, it’ll be this.”
However, there are still a lot of issues with genAI that need to be settled before it could automate legal services, O’Leary cautioned — not least of which is how much genAI may cost to use and how accurate and secure it can be.
“The cost of using AI may be as much as using an associate,” O’Leary said.
Is genAI cheaper and more accurate than an attorney?
Along with AI’s ability to perform tasks previously accomplished by attorneys and other legal workers, there remains a big concern over accuracy, security, and hallucinations. As in the healthcare industry, the stakes are high when it comes to client confidential information.
“There are big issues around copyright protection and whether these large language models are being trained on copyrighted materials,” Harvard’s Wilkins said. “So, what you’re seeing is a lot of experimentation with trying to build customized AI models and large language models. AI providers claim their models are trained exclusively on legal materials, cutting down on hallucinations.”
While law firms are aware of AI’s pitfalls, attorneys are still going to use the technology, Wilkins said, whether or not that’s in line with a corporate policy. GenAI is simply too “transformative” a technology to not use simply because there are risks, he said.
One problem in comparing human workers to the technology is that the bar is often set too high for AI, Wilkins said.
“I’ve heard people say, ‘We could never use this unless it’s 98% effective and reliable.’ I said, ‘Well, does it have the reliability of sending an associate to a windowless warehouse in Phoenix, Arizona to find documents related to a case? Is that 99% accurate?’” Wilkins said.
In the end, whether genAI assists in a task or not, ultimately the attorneys involved will be held responsible for the outcome — good or bad. Whether the technology will replace attorneys and legal aides remains to be seen.
“Our experience has been — and we’ve kicked tires on a lot of language models and purpose-designed tools — [genAI tools] are not good enough to replace people for a lot of the work we do,” ClearyX’s Swansburg said. “For something like due diligence…, you often must be [100%] right. You need to know whether you can get consent to transfer something. In other use cases, such as summarization and initial drafting, that sort of thing is a little more accessible.
“In my world, it’s not really replacing jobs yet, but it’s changing how you do jobs,” she continued. “So, it’s allowing people to move up the value chain a little bit. It’s taking away rote and repetitive work.”
Harvard’s Wilkins placed the adoption of AI by law firms and other legal entities as still being “in the Stone Age” but with massive potential.
“The potential efficiencies are great,” he said. “We’re just working out what are the real advantages.”