How to Make AI Agents Verify Their Own Work Before Handoff
Learn how to build verification loops into AI agent workflows using screenshots, browser tests, and eval sets so outputs land closer to done.
Why AI agents rarely finish the job on the first try
Ask an AI agent to complete a task and you typically get something in the range of 60 to 70 percent done. Then you give feedback, it fixes something, you give more feedback, and you slowly climb toward 90 or 95 percent through manual back and forth. The fix isn’t a better prompt. It’s building a verification step into the workflow itself, so the agent checks its own output against a clear standard before handing it back to you, instead of you doing the checking every time.
TL;DR
- Verification loops let an agent test, screenshot, or score its own output and keep iterating until it meets a defined bar, instead of stopping at a first draft.
- The setup question is simple: if a human reviewed this work, what would they actually do to approve it, and can the agent do that same check itself?
- For visual or interactive work like websites, a screenshot and click-through loop (checking layout, mobile view, button functionality, form submissions) catches obvious failures before a human ever sees them.
- AI evals turn “I think this is better” into a measurable yes or no by scoring agent output against a set of known-good examples.
- Grading can be code-based for objective answers or use an LLM as a judge when the correctness call requires reasoning.
- Tool access matters as much as instructions: an agent that can technically send an email will eventually send one, so verification has to include what the agent is allowed to touch, not just what it produces.
- The payoff is fewer rounds of manual correction: one prompt, a wait, and a deliverable that’s already close to finished instead of a first draft that needs three more passes.
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
What does a verification loop actually look like?
A verification loop is a step (or set of steps) an agent runs after producing output, specifically to check that output against a standard, before considering the task done. It’s the same instinct as a human double-checking their own work, just automated.
The practical way to design one: think through how you’d personally review the deliverable if a person handed it to you. Would you look at it? Test it? Click through a signup flow? Compare it against an example you know is correct? Whatever that manual review process is, an AI agent can usually do a version of it.
For code and websites, that means the agent can:
- Take screenshots and check that layout elements are in bounds and render correctly on both desktop and mobile.
- Click through the interface to confirm buttons work, forms submit properly, and data lands where it’s supposed to (like the right webhook).
- Run automated tests against the output rather than just eyeballing it.
For research or writing tasks, verification might mean the agent compares its output against a rubric or a set of prior good examples and scores itself before returning an answer.
The point isn’t a single check. It’s a loop: the agent verifies, finds a gap, fixes it, and re-verifies, without needing you in the middle of every cycle.
How do you set up verification without a QA team?
You don’t need a dedicated testing team to get most of the benefit. The setup is smaller than it sounds:
- Define what “done” actually looks like. Vague instructions produce vague stopping points. If the agent doesn’t know the finish line, it will stop whenever it feels reasonably confident, which is often well short of 100 percent.
- Give the agent the tools to check itself. A browser-operating tool for websites, a test suite for code, a comparison method for written output. If the agent can operate a browser, it can screenshot its own work and evaluate it the way a human would.
- Loop the check back into the fix. The agent shouldn’t just flag a problem, it should be instructed to fix it and re-check, repeating until the condition is met or it hits a reasonable limit.
- Have it argue with itself. Because models tend to be agreeable, asking “is this good?” often gets a yes regardless of quality. Assigning the review to a different persona, a skeptical user, an engineer who has to maintain the thing, a competitor sizing it up, surfaces problems a single straightforward review misses.
This shifts the workflow from “one shot, then manual cleanup” to “agent submits something it has already stress-tested.”
What are AI evals and why do they matter for verification?
A single successful run doesn’t tell you much. Because AI models are non-deterministic, the same task can produce different results across multiple runs, and a small change to a prompt, tool, or model can shift performance in either direction without warning.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
AI evals solve this by giving you a repeatable way to measure quality across many runs, not just one.
The process:
- Collect a set of real examples with known good answers. This becomes your source of truth, sometimes called a golden dataset. Even a small set, twenty or so solid examples, is enough to start.
- Run the agent against those examples and score the output.
- Use code-based grading when the correct answer is objective (a number, a specific format, a pass/fail condition).
- Use an LLM as a judge when scoring requires some reasoning, such as evaluating whether a written response actually addresses a customer’s question the way a human-written one would.
- Re-run the eval every time you change something, whether that’s the prompt, the tools available, or the underlying model.
This turns “I think this update made things better” into an actual measured comparison. It also catches the surprising cases where a change you were confident would help actually lowers the score, which is far better to discover in testing than in front of real users.
Is verification worth the extra setup time?
Yes, for anything that matters beyond a one-off task. The upfront cost is building the checking logic (test scripts, screenshot loops, an eval set), but the payoff shows up immediately in reduced back-and-forth. Instead of sending a request, getting a rough draft, and spending several rounds correcting it, you send one request and get something back that has already been checked against a real standard.
The setup cost also scales down well. A small project might only need a basic screenshot check. A production agent handling customer support or financial data warrants a proper eval set with dozens or hundreds of known-good examples. Match the rigor to the stakes.
Does verification replace permission controls?
No, and this is a common gap. Verifying output quality is a separate problem from restricting what an agent is allowed to do. An agent can produce a technically perfect draft and still have access to a tool that lets it take an action nobody asked for, like sending an email instead of just drafting one.
Because models are non-deterministic, a rule written into a prompt (“only draft, never send”) is a suggestion, not a restriction. If the underlying tool access still allows sending, the agent may eventually use it. The reliable fix is scoping the actual tool or API key so the action is physically impossible, not just discouraged in the instructions.
So a complete verification setup covers two things: does the output meet the bar, and is the agent even capable of doing something it shouldn’t. Skipping the second one means a well-verified output can still come with an unwanted side effect.
Frequently Asked Questions
What’s the difference between verification and just giving better feedback?
Feedback is manual correction after the fact, done by a person each time. Verification is a check built into the workflow itself, so the agent tests or scores its own output and fixes issues before you ever see a draft. Feedback doesn’t scale; verification does.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
Do I need coding skills to build a verification loop?
Basic ones help but aren’t required for simple cases. Many verification tasks (screenshotting a webpage, checking mobile layout, clicking through a form) can be handled by an agent operating a browser tool with plain-language instructions. More rigorous setups, like running a formal eval against hundreds of examples, benefit from some scripting.
How many examples do I need for an AI eval?
There’s no fixed minimum, but even a small set, around twenty known-good examples, is enough to start catching regressions and comparing versions. Larger, higher-stakes systems benefit from bigger datasets, sometimes in the hundreds, to get a reliable success rate.
Can verification fully replace human review?
Not entirely, especially early on. Verification gets output much closer to done before a human sees it, but human spot checks still matter, particularly for judgment calls an LLM-as-judge might miss or for high-stakes actions where an error is costly.
What should I check first if an agent’s output looks wrong?
Check the eval or golden dataset comparison first to see if the failure is consistent or a one-off due to non-determinism. Then check tool and permission scope, since an agent behaving unexpectedly is often a sign it had access to do something nobody intended, not just a bad prompt.