Bridgeworks CISO, Mihai Poppa features in this article from The Stack to discuss the best methods for managing an AI agent when a mistake has been made.
September 4, 2026
Human beings aren’t perfect, so it should be no surprise that any AI system designed and coded by humans can go wrong.
Last July, Jason Lemkin, a tech entrepreneur and founder of the SaaS community SaaStr, documented his experiment with Replit’s AI-assisted “vibe coding” tool, which managed to delete a production database during a standard code freeze. An AI developer agent bypassed that code freeze, generated fake user records to hide its mistake, and wiped a live production database after it incorrectly triggered a command.
As we learned from the OpenAI-Hugging Face incident, there are still a lot of things that enterprise tech does not understand about AI agent behavior. The incident highlighted the need for a precise system rollback tool to undo an AI agent’s specific actions – in Replit’s case, developers were forced to manually restore old backups, and that can cause a massive amount of operational downtime.
According to reports at the time, Lemkin was able to manually roll back the damage caused by the AI agent within a couple of hours. Replit responded immediately too, with Replit CEO Amjad Masad publicly apologising on X and calling the incident “unacceptable.”
In a separate incident later that year, Amazon’s Kiro AI autonomous coding tool accidentally bypassed approval steps and wiped a production environment, which it also decided to recreate, which caused a 13-hour outage for the Cost Explorer service in China. Amazon attributed the incident to user error and misconfigured access controls, saying that the engineer had granted the agent overly broad permissions.
Both events highlighted the importance of traceability when working with AI agents, which is easier said than done.
All about trust
According to Gartner analysts Chris Howard, Svetlana Sicular, and Val Sribar, it’s better to be aware of any potential AI failures and to prevent them before they happen and cause friction, as they imply in their report, “Resolve AI-Created Friction Before It Damages Trust.”
“Friction can come from the AI itself. A model returns a confident wrong answer. An agent takes a step no one sanctioned. A tool draws on data it should not have touched. That is the friction this brief is about. Its source is the system, not the people around it,” the authors wrote.
The trouble is that restarting an agent is not the same as undoing what it has already done.
When an email has been sent erroneously, a price changed, a refund issued, code merged or a record updated, reverting the agent does not reverse the business event; it can still cause damage, either financially, in lost services, or lost time.
“Initially, agents were mostly read-only. They were about reading different data sources and summarizing information. Now they are graduating to systems that actually change the state of the external world,” said Maxim Fateev, co-founder and chief technology officer of Temporal. “Also, they frequently run at very large scale, and the more scale you have, the more likely failures become the norm rather than the exception, so losing state and not recovering is not an option anymore.”
Automated troubleshooting
To tackle these issues, Langchain launched the LangSmith Engine earlier this year to resolve the agent development lifecycle bottleneck, and to automate troubleshooting. The tool watches production traces and cluster agent failures, and can propose bug fixes.
Mihai Popa, CISO of Bridgeworks, said the LangSmith Engine is designed to identify recurring patterns and suggest fixes rather than expecting engineers to find those patterns one trace at a time.
“Without that capability, the cost is not just engineering time,” he said. “A subtle failure can continue affecting customers for days because each incident looks isolated. You also risk fixing individual symptoms without recognising the underlying pattern.”
James Massey, co-founder of Zylox, agreed. “Once agents are used in production, the challenge shifts from investigating isolated failures to understanding patterns across many runs. A single trace might show what happened once, but a collection of traces can reveal that a particular tool, prompt or type of input is repeatedly causing problems.”
“Without these capabilities, teams can end up relying on user reports and debugging incidents individually,” he said. “That is slow, but the larger cost is that recurring problems can remain hidden while the overall success rate still looks reasonable.”
Questions about convergence
To reduce the potential incidents that are caused when an AI agent does something wrong, the industry is starting to converge on standards around durable execution and checkpointing.
Temporal built a foundational software development framework that supplies state-management tools, such as tracking prompts, states, and checkpoints, to monitor and counteract large scale errors created by AI applications. It addresses the scaling challenges of modern agent-based software.
“LangGraph, Temporal and Pydantic AI do different things at their core, but once an agent becomes a long-running workflow the same questions recur: what state was saved, which step failed, what can be retried, and where a person needs to intervene,” Massey said.
But according to Samuel Colvin, founder & CEO at Pydantic, that’s not exactly the case. “There’s a lot of confusion as we don’t have our own durable execution solution. Pydantic AI integrates with existing ones, mainly Temporal along with others like DBOS, Prefect, and more,” he said.
“LangGraph and Temporal aren’t doing the same thing either. Checkpointing, which is what LangGraph offers, resumes an agent from a saved point on its own process. Durable execution, what Temporal provides, is a different and more complete idea: it separates business logic from I/O through workflows and activities, so the whole system, not just the model call, can survive a crash, a deploy, or a rate limit and pick back up exactly where it left off.”
He says that distinction matters because checkpointing alone doesn’t solve the reliability problem it’s marketed as solving: “It gets you back to a saved state and doesn’t give you the guarantees that come from treating durable execution as a first-class architectural concern, which is what systems like Temporal were built for from ground up.”
Pydantic AI doesn’t try to reinvent that, he claimed, and so it is independent. “We’d rather integrate with systems that already do it well than build a worse version ourselves,” he suggests.
Importance of durable execution
According to Qian Li, co-founder and CEO of DBOS, “durable execution is essential for production AI agents. The more useful an agent becomes, the longer it runs and the more real-world tasks it performs.” After all, in the real world, everything fails all the time, as Amazon CTO Werner Vogels likes to put it.
“Without checkpointing, an agent may lose its progress and repeat expensive work,” she said. “That is why frameworks such as Pydantic AI integrate with DBOS: if an agent fails halfway through, DBOS can resume it from the last completed step rather than starting over.”
While AI has brought incredible speed and scale, it doesn’t always equate to quality. So when there is an error, it scales that workload too. “Humans could also have sent wrong emails or deleted wrong files but agents can send thousands of wrong emails at once – the scale of the possible error is the problem and the only way to counteract, monitor and fix it is by using the same technology,” explained Maria Sukhareva, founder of AI Realist.
She said that the convergence in standards is “due to the fact that a lot of software and applications are being built now on the agentic basis, thus it requires similar monitors like prompts, agent states, checkpoints which older frameworks for general software development were created to solve.” She also stressed that a human in the loop can only function in this case as a liability sponge if they are not armed with automatic tools.
Lessons from security teams
When it comes to an agent incident response, Fred Hebert, staff site reliability engineer at Honeycomb, said there is a lesson to be learned from security teams. “It’s been known for a while that phishing tests are not necessarily useful because there’s always going to be someone you can properly social engineer,” he explained.
The best response training usually comes from simply assuming an account has been compromised and then figuring out how we trace, isolate, and repair what has been done; it shifts from purely preventive interventions into rapid detection and reaction, he said.
“If we think agents cannot be fully controlled through prompts and pre-planned defences only, exercises may benefit by assuming a compromised or misbehaving agent, and then figuring out how to deal with the situation once it happens: what are the signs, how do we know it’s happening, how do we constrain the possible actions, and what does the organisation’s response and communications look like in this situation?” he said.
Resilience to failure
Durable execution makes agents resilient to failure, but it doesn’t automatically make their decisions reversible, Li said. There is no universal rollback for the outside world, because once an agent crosses into another system, recovery depends on idempotency, human in the loop, and application-specific compensation.
Restarting an AI agent may restore its workflow, but it cannot reliably undo changes already made in the real world. Durable execution and checkpointing improve resilience by preserving state and preventing completed work from being repeated; reversibility, however, must be designed separately.
“Theoretically if the versioning and backups are in-place one could recover the previous state,” Sukhareva said. “If the agent is done for irreversible decisions – then one should accept the risk of errors and consider if the benefits outweigh the damage.”
Some actions will remain irreversible, so leaders should weigh expected benefits against potential harm before granting authority. The enterprise benchmark is not simply whether agents complete tasks reliably, but whether failures can be detected, contained, explained, and repaired.
“Sometimes you can [roll it back], sometimes you can’t,” Hebert said. “A serious question to ask is therefore how certain you can be that the right action is about to be taken, and consequently how much uncertainty do you tolerate before giving up?”
