AI v. Human Lawyers
It's hard to ignore the ways artificial intelligence (AI) has already bested humans, from making incredibly convincing internet videos to beating humanity's best Go players. AI is expected to outpace humans in a number of occupations; some of which might be a little unexpected.
A new study released this week from LawGeex, a leading AI contract review platform, has revealed a new area in which AI outperforms us: Law. Specifically, reviewing Non-Disclosure Agreements (NDAs) and accurately spotting risks within the legal documentation.
For the study, 20 human attorneys were pitted against LawGeex's AI in reviewing 5 NDAs. The controlled conditions of the study were designed to resemble how lawyers would typically review and approve everyday contracts.
After two months of testing, the results were in: the AI finished the test with an average accuracy rating of 94 percent, while the lawyers achieved an average of 85 percent. The AI's highest accuracy rating on an individual test was 100 percent, while the highest rating a human lawyer achieved on a single contract was 97 percent.
As far as accuracy goes, the study showed that humans can (for the most part) keep up with AI in reviewing contracts. The same couldn't be said when it came to speed, however.
On average, the lawyers took 92 minutes to finish reviewing the contracts. The longest time taken by an individual lawyer was 156 minutes and the shortest 51 minutes.
LawGeex's AI, on the other hand, only needed 26 seconds.
Law & Order: AI
Gillian K. Hadfield, Professor of Law and Economics at the University of Southern California, and one of the consultants on the study says the efficiency gap between the two groups may be even wider than we realize. In the study, the lawyers were completely and singularly focused on the task at hand. In the real world, however, they would have other responsibilities to tend to, distractions, and interruptions which would likely increase the actual time it takes them to review contracts.
"This experiment may actually understate the gain from AI in the legal profession," says Hadfield. "The lawyers who reviewed these documents were fully focused on the task: it didn’t sink to the bottom of a to-do list, it didn’t get rushed through while waiting for a plane or with one eye on the clock to get out the door to pick up the kids."
While law might be a fairly new domain for AI, it's hardly the only area in which AI has been shown to outperform humans in terms of accuracy and efficiency. Medicine, too, has demonstrated the many potential applications for AI. For example, researchers from the John Radcliffe Hospital and the startup Optellum are working on AI systems that can diagnose heart disease and lung cancer earlier and more accurately than human doctors. At the Singapore National Eye Center, researchers have created an AI that can spot eye disease sooner than human doctors can.
The medical field isn't the last stop for AI, either. In February, AI proved capable of detecting more earthquakes in Oklahoma than traditional methods. And as this year's Winter Olympics wrapped up, the International Gymnastics Federation announced it intends to use AI to assist judges during the 2020 Olympics.
AI still has to prove itself in many areas, so it's hardly "case closed" — but for attorneys who might be looking to hand off some of their administrative drudgeries, there's a clear verdict on AI's usefulness in the law office, at least.
Share This Article