Gen AI and lawyers do not work well together yet

According to a survey by Bain Research, legal executives are becoming less satisfied with generative AI tools because they don’t meet their performance expectations and produce poor-quality work. Despite the technology’s perceived importance for future business strategies, this situation persists.

There is a lot of talk about how generative AI will change business, and one important theme for lawyers is coming up: Please show us the value. A quarterly survey by Bain Research found that GenAI isn’t meeting the needs of both in-house and outside counsel in terms of performance reviews.

The survey revealed that legal executives were the least satisfied with GenAI software tools. Over five months, the number of lawyers who said GenAI had “met or exceeded expectations” dropped by 18%. About half of lawyers (53%) said the tools met or went beyond what they expected.

The satisfaction level of people working in operations who used GenAI went down by 16%, but it went up by 6% for people working in sales and sales operations.

43% of those who were disappointed said GenAI couldn’t do a needed task with “sufficient performance,” and 42% said it produced “poor quality output.”

Another 38% said they didn’t know how to use the tools, and 29% said the vendor and its product were of “low quality.” The final finding, having a negative opinion of the quality of a vendor or tool, showed the biggest change in how people felt compared to the last survey.

“Lawyers have a high bar,” Mary O’Carroll, chief community officer at San Francisco-based Ironclad, which makes contract management software, told Corporate Counsel as one reason why both in-house and outside counsel are exhausted of the deal. She said, “I really think there’s a great use case for AI here, and the early-stage startups that are trying to solve it haven’t met the bar yet.”

GenAI Among Top Priorities for Businesses

More than 60% of businesses surveyed by Bain said that GenAI was one of their top three priorities for 2025, and 87% said it was one of their top five priorities for the next three to four years.

According to Bain’s data, around 100 people at companies are using the new AI technology in some way. Large companies have as many as 240 team members working with AI.

“As with other digital transformations, automating a process or moving it to a new, more efficient technology doesn’t always bring enough benefits to make the investment worthwhile,” the Bain report says.

Understanding how new technology alters not only the methods of doing tasks but also their nature is crucial.

The results come from a survey of 200 company executives taken in October 2023 and February 2024.