81% of teams merge AI code or tests they do not fully understand
TestMu AI found that 81% of teams merge AI-written code or tests they do not fully understand, while 44% report more bugs reaching production as AI-written code becomes more common.
Why it matters
- 81% of software teams have merged AI-written code or tests they did not fully understand.
- AI writes more than half of production code at 40% of teams.
- 44% report more bugs reaching production, while only 41% run an automated pre-merge check.
- AI-generated tests are "almost right, but not quite" a median 65% of the time.
- 77% build AI agents to test, but 37% have no systematic evaluation method.
TestMu AI's State of AI in Quality Engineering 2026 report found that 81% of software teams have merged AI-written code or tests they did not fully understand.
The same report found that AI writes more than half of production code at 40% of teams. TestMu AI points to that production share as evidence of a broader shift in how software is built and tested.
Verification has not matched that adoption. Some 44% of teams said more bugs were reaching production as AI-written code became more common. Only 41% said they run any automated check on AI-written code before it is merged.
The findings put software quality controls at the center of the story. TestMu AI argues that organisations are expanding AI in software development faster than they are putting checks around its output. That makes verification a production-governance issue for mainstream software teams, rather than a concern limited to specialist AI companies.
What happens before AI output reaches production?
Manual oversight remains common. Fifty-five percent of teams rely on manual review, while 51% depend on a QA hand-off after code is written. The report interprets these practices as verification that often occurs only after AI has generated software or tests.
TestMu AI treats that mix as a timing problem rather than an absence of scrutiny. Its concern is not that people never examine AI output; it is that many examinations happen after code or tests already exist, while automated pre-merge checks remain uncommon. That is why the report calls independent validation the practical response.
How exposed are developers?
Developers appeared to face the greatest exposure in the survey. Among developers, 93% said they had merged AI output they did not fully understand, well above the 81% figure across software teams overall.
That result concentrates the report's verification problem in the teams closest to generation. The 93% figure records admitted gaps in understanding among the people best placed to inspect the code before release.
Does continuous integration reduce escaped defects?
Teams without separate continuous integration tests were more likely to report escaped defects. Forty-nine percent of those teams said bugs reached production more often, compared with 38% of teams that run those tests.
The report makes the control architectural: keep generation and validation separate, then repeat validation whenever AI output changes. The proposed test applies before a merge, not only after a defect appears in production. That sequence addresses the report's broader concern that checks lag behind generation.
Can one AI model verify another?
TestMu AI found that teams using what it described as LLM-as-a-judge methods regularly merged AI output they did not understand at nearly twice the rate of other teams. The split was 29% versus 15%.
The report says many teams still verify AI output either with a person after the fact or with another AI model. That practice raises a direct question about whether the checking process is sufficiently independent from the process that generated the output. Model-on-model review does not remove that dependency, according to TestMu AI.
How often are AI-generated tests correct?
The report also found that AI-generated tests were "almost right, but not quite" a median 65% of the time. The report says that pattern creates a practical challenge for engineering teams, which may receive plausible-looking output but still need to decide whether it is accurate enough for production use.
TestMu AI says teams must still judge whether an almost-correct test is accurate enough to ship, even when the generated result looks usable. The finding turns apparent success into a review obligation rather than a finished test.
Where does the AI testing budget go?
Half of teams said they spend most of their AI testing budget on producing output, while only 17% said verification takes the largest share. The report says that split may help explain why stronger controls have not spread as quickly as generation.
That allocation puts the emphasis on output rather than verification. The survey therefore links budget choices to the operational gap: teams may direct most AI testing resources toward generating code and tests while few make verification their largest category.
Only 13% said they have an incident-response plan for AI errors. That leaves most surveyed organisations without a formal procedure for handling mistakes introduced by AI-generated code or tests. The report treats the missing plan as evidence that governance has not matched the spread of AI in testing.
Is AI testing becoming universal?
The report found that 97% of teams surveyed now use AI in testing or plan to do so. That near-universal expected adoption moves AI testing beyond specialist AI companies into mainstream software teams across sectors.
For mainstream software teams, the stakes are escaped defects, release controls and incident readiness. TestMu AI treats independent validation, incident planning and systematic agent evaluation as operational requirements for ordinary software delivery, not as specialist research controls. The nearly universal adoption figure makes those controls relevant across sectors.
How are teams testing AI agents?
The research also examined AI agents, which development teams increasingly need to test. It found that 77% of teams build AI agents they need to test, but 37% have no systematic way to evaluate them.
Only 28% said they test agents against users who try to break them. TestMu AI interprets those figures as showing that structured methods for stress-testing agents remain limited even as agents enter software products and workflows.
The survey does not say that the remaining teams perform no agent checks; it says 37% lack a systematic evaluation method. Only 28% use the specific stress test of users attempting to break agents. That leaves a measurable gap between building agents for products and testing them under adversarial conditions.
What independence standard does TestMu AI advocate?
Across code generation, test generation and release decisions, the survey shows teams leaning heavily on AI at several stages of development. TestMu AI says that dependence can create blind spots, with teams accepting output that appears useful without fully understanding how it works or whether it will hold up in production.
Asad Khan, TestMu AI's Co-founder and CEO, said:
"Software teams now rely on AI to write their code, generate their tests, and increasingly determine whether both are ready to ship. Verification, however, has not kept pace with that shift. We believe any output AI produces should be validated by a system independent of the model that produced it, before it merges and every time it changes."
"The teams in our survey that already follow this discipline see fewer defects reach production, and we expect it to become the standard for engineering organisations shipping with AI."
With 44% already reporting more production bugs, TestMu AI frames independent validation as the control point for the next stage of AI-assisted development.
Source: itbrief.com.au
More from Rebecca Stone
Show full bio
Correspondent covering companies, deals and strategy in artificial intelligence and machine learning at AI In Context.
802 articles