🔍 Read the full analysis: Research Looks At What It Takes To Review AI’s Work on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Georgia Tech cognitive engineer Karen Feigh argues that AI’s productivity gains can be offset by the time and mental effort people spend checking its work. Research she cites reports mixed results: AI slowed experienced developers in one trial, sped up consultants on tasks within its capabilities, and led to more errors on a task outside them.
Georgia Tech cognitive engineer Karen Feigh argued in a November 16 essay that workers’ productivity gains from AI can be offset by the time and mental effort needed to check whether its output is accurate and appropriate. The issue matters as organizations use AI for coding, writing and knowledge work, but the studies she cites show that the effects vary by task and do not establish a single productivity result for all users.
Feigh calls the work of checking AI output the “verification burden.” In The Conversation, she describes an executive assistant who uses AI for itineraries and briefings but has encountered enough problems to set aside time to check for inaccuracies and bias. Feigh compares the new responsibility to supervising a fast, polished intern: the system can produce useful work, but also errors in the same confident tone, and it does not bear responsibility for the consequences.
One study she cites, a randomized trial by the research group METR, involved 16 experienced open-source developers completing 246 tasks in large software projects. The trial found that developers took 19% longer on average when AI tools were allowed. Before the study, participants expected AI to reduce completion time by 24%; afterwards, they still estimated that it had saved them 20%. The trial used tools including Cursor Pro and Claude 3.5 and 3.7 Sonnet during February to June 2025.
METR also reported that developers accepted fewer than 44% of AI code generations, while 75% said they read every line of generated code. On a subset of 44 tasks with usable screen recordings, about 9% of AI-allowed working time went to reviewing and cleaning up AI output. These measures describe this particular software trial, not all AI use. Other studies Feigh cites found gains in different settings, including faster professional writing and consulting tasks that fell within the AI’s abilities.
AI and work · explainer
The verification loop: where AI’s time savings can go
The draft is fast. Knowing whether it is right is the work that follows.
Time for one task (schematic)
Draft: secondsgood and bad parts in the same confident voice About 9% of the timewent to reviewing and cleaning AI output in METR’s trial with AI allowed Speed is not productivitycount the whole loop, errors included
How the hidden cost builds up
- DraftThe AI answers in seconds, fluent and confident“few clues about where scrutiny is needed”
- CheckDecide what to check and how, without knowing if or where an error iscognitive load
- FixFinding an error does not end the work, it starts a correction cycletime
- Re-checkEvery fix needs checking again; when nothing is wrong, the effort feels wastedmotivation
- Count it allProductivity is the whole human-AI workflow, not the first draftnet, not gross
Source: Karen Feigh, Georgia Tech, The Conversation, Nov 16, 2026 (the three burdens, the quotes); METR, arXiv 2507.09089, p. 20 (time spent reviewing and cleaning AI output). The bars are a schematic, not data.
Graphic: Powered by Thorsten Meyer AI
Why Verification Changes Productivity
The findings show why the speed of an AI-generated first draft is not, on its own, a measure of time saved. Workers may need to inspect, test, correct or discard output before they can use it. That follow-up work can take enough time to reduce or erase gains, as the METR trial found for its participants and tasks.
The risk is not limited to time. In the consulting experiment Feigh cites, AI users performed better on tasks within the system’s capabilities but were 19% less likely to produce a correct solution on a task designed to fall outside them. That result makes task fit and review quality relevant to organizations deciding where AI can be used and who remains accountable for its output. The studies do not establish that AI is generally harmful or unproductive; they show that results depend on the work and on whether people can detect mistakes.
Products Worth Considering

Bose QuietComfort Wireless Noise-Cancelling Headphones, Sandstone

Jackery Explorer 300 Portable Power Station, 292Wh

Amazon Kindle Scribe (16GB) with Premium Pen, Tungsten
As an affiliate, we earn on qualifying purchases.
Different Tasks, Different Results
The research Feigh discusses does not point to one uniform effect. In a study of mid-level professional writing tasks, researchers found that ChatGPT users completed work 40% faster and produced work rated 18% higher in quality. In a separate experiment involving Boston Consulting Group and 758 knowledge workers, participants using AI completed 12.2% more tasks and worked 25.1% faster on tasks within the AI’s capabilities. On a complex task outside those capabilities, they were 19% less likely to give a correct answer.
A separate experiment with 2,784 people examined how well reviewers detected errors in AI-style suggestions. Participants usually caught typos, but their accuracy was 31% for errors requiring an understanding of the task’s rules. That finding suggests that surface proofreading may not be enough to identify substantive problems. It does not mean that every reviewer or workplace will have the same accuracy; the result belongs to the specific experiment.
Feigh also connects the issue to a longstanding concern in automation: a system may exercise practical influence over decisions while a human retains responsibility. Her essay is an expert perspective drawing together existing research, rather than a new experiment. The studies differ in participants, tasks, tools and measures, so their results should not be treated as directly interchangeable.
“AI’s productivity gains “are often countered by time and mental energy spent determining whether the output is accurate and appropriate.””
— Karen Feigh, cognitive engineer at Georgia Tech, in The Conversation
Where the Evidence Has Limits
The evidence does not show that AI always slows work or that review costs outweigh productivity gains in every setting. METR’s result came from a small group of experienced developers working on real tasks in mature repositories with particular tools available in 2025. The authors cautioned that their findings do not show that current AI systems are useless in other economically relevant settings, or rule out gains from future systems.
The studies also do not establish a standard amount of time or training needed to review AI output safely. Results from writing, consulting and coding tasks involve different measures and cannot be combined into a single estimate of workplace productivity. It remains unclear how well the findings apply to other occupations, newer AI models, less experienced users or organizations with different review procedures.
Measure the Full Work Cycle
Feigh’s practical conclusion is that employers should measure the whole workflow, rather than only how quickly AI produces an initial draft. That includes the time spent checking, testing, correcting and integrating output, as well as whether the final work is accurate and appropriate.
She also argues that workers who remain accountable for AI-assisted work need time and training to review it. The research cited does not set out a single review standard or predict when AI will produce net time savings across an organization. Further evidence will need to show how those outcomes vary by task, tool and review process.
Key Questions
What does Karen Feigh mean by the “verification burden”?
She means the time and mental effort workers spend checking AI output for accuracy and appropriateness, which can offset some of the time saved generating it.
Did AI make developers slower in the METR trial?
In the randomized trial cited by Feigh, 16 experienced open-source developers took 19% longer on average when AI tools were allowed across 246 tasks. That finding applies to the study’s participants, tools and tasks, not every software project.
Did the research find that AI always reduces productivity?
No. Other studies cited in Feigh’s essay found faster completion on writing and consulting tasks. The results varied by task, including a consulting task outside the AI’s capabilities where AI users were less likely to answer correctly.
Why can AI errors be hard to spot?
In one experiment with 2,784 people, reviewers were generally better at catching typos than errors that required understanding task rules. Accuracy for the latter errors was 31% in that experiment.
What should employers measure when evaluating AI tools?
Feigh argues they should track the full work cycle: generation, review, correction and integration, along with the quality of the final work. The research does not prescribe one universal measurement method.
Source: ThorstenMeyerAI.com
