AI coding agents are cranking out significantly more code. But those efficiency gains are being wiped out by human review “bottlenecks”.
As reported in ArsTechnica (AI coding agents generate more code, but not more software, written by Kyle Orland and available here), a study covering more than 700 firms found that adopting agents increased code volume by 30%, commits by 20%, and pull requests by 23%, but did not significantly improve completion rates for tracked issues or features.
Why?
Because review time increased 49%. More granular data from the researchers showed “the share of pull requests with changes requested nearly doubl[ing], and the number of comments per pull request increas[ing] by 35%” following the AI agent shift.
While AI could also theoretically help with this review process, the researchers found that, so far, that impact has been marginal. Although 80% of measured firms used some form of AI code review, AI agents were only responsible for 23.3% of all review comments and 10.8% of all pull requests, suggesting humans were still responsible for the vast majority of this work.
This doesn’t seem to have an effect on employment. Wjhile the researchers found a 14% increase in the share of workers performing code reviews after AI agents’ introduction, they also write that they “cannot attribute significant employment changes to AI” after looking at total active workers across aggregated analytics data firm Jellyfish and cross-referencing with LinkedIn data at those firms.
One caveat is that the study has a March 2026 data cutoff, so it doesn’t reflect any findings from the last six months, which is a long time in today’s rapidly moving AI world.
Is this a case of “trust, but verify” running amok? Or are the AI coding agents sacrificing quality for quantity?
I’ve seen similar discussions related to using GenAI to facilitate review in discovery. Some people say that you need to do so much manual review to validate the results that you’re offsetting the gains from the technology. While others say that statistical sampling methods (like those we’ve utilized in predictive coding) are enough to validate the results. I side with the latter group.
Of course, a 90+ percent recall on review in discovery is an excellent metric. 90 percent of your code being correct means you still have a lot of problems and your program doesn’t run (or worse, it runs badly and creates more problems). The additional review time may be warranted for AI coding agents.
So, what do you think? Are people being too cautious with AI agents? Please share any comments you might have or if you’d like to know more about a particular topic.
Image created using ChatGPT, using the term “two humans reviewing programming code on a screen while a robot programmer stands by looking at its watch”.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.
Discover more from eDiscovery Today by Doug Austin
Subscribe to get the latest posts sent to your email.
