AI agents are being trusted with more decisions than companies can actually verify

Key points

• In a June 2026 survey of 157 enterprise teams, 50% reported shipping an AI agent or AI feature that passed internal testing and then caused a customer-facing failure.
• One in four of those companies experienced that kind of failure more than once.
• 66% of surveyed enterprises already allow some AI deployment without human review, or plan to within 12 months.
• Only 5% of respondents said they fully trust the automated testing systems that decide whether an AI is ready to ship.
• Larger companies with 2,500 or more employees are moving fastest toward zero-human oversight, and they are also reporting more customer failures than smaller firms.

Picture a quality inspector who admits, quietly, that the checklist is mostly wrong. That is roughly the situation a survey published by VentureBeat describes across a broad slice of corporate AI teams right now.

The June 2026 VB Pulse survey asked 157 people at companies with 100 or more employees about how they test and release AI agents, software programs that can carry out multi-step tasks on their own, things like handling a customer refund, updating a database record, or drafting and sending a message without a human pressing send. The sample was self-selected rather than randomly chosen, so the numbers should be treated as a signal, not a census.

The signal is uncomfortable. Half of the companies surveyed had already shipped an agent that passed every internal test and still went on to hurt a customer experience. A quarter had done it more than once.

Why does a passing test not guarantee a working agent?

Because traditional software testing checks whether a fixed input produces an expected output, and AI agents do not work that way. An agent chooses its own sequence of steps, calls external tools, pulls in live data, and may behave differently each time it runs, even on the same task.

An agent can get five steps exactly right and then leak sensitive information on the sixth. It can draft a valid refund and send it without waiting for approval. Each individual decision looks fine. The final outcome is a problem.

The US National Institute of Standards and Technology made a similar observation in its Generative AI Profile guidance: behaviour measured in a controlled test environment often does not hold once real users, unpredictable prompts, and live data enter the picture. The guidance calls for field testing and ongoing monitoring after launch, not just a pre-release score.

The survey respondents themselves seem to understand this. When asked why they distrust automated testing, the most common answer, given by 29%, was that test scores do not align with what actually happens in the real world.

And yet the autonomy keeps expanding. Two thirds of the companies surveyed are already running, or actively building toward, deployments with no human in the loop at all. Only 5% fully trust the automated checks that would be doing the job a human reviewer used to do.

That gap, growing autonomy sitting on top of shaky assurance, is the core problem.

Anthropics guidance on agent evaluation draws a useful line here: a system that succeeds once is not the same as a system that succeeds reliably. For anything touching customers or money or data, reliability is the only bar that matters.

What should companies actually do? Run every scenario multiple times. Vary the wording and the context. Simulate tool failures. When a real incident happens in production, turn it into a permanent test case so the same failure cannot quietly recur. And expand zero-human oversight only where the consequences of a mistake are genuinely low, drafting an internal summary is not the same as approving a financial transaction.

Removing a human from a process does not remove the risk. It just removes the person who might have caught it.