Same agent, opposite results: exit criteria decide the outcome

TL;DR: In one week I delegated two tasks to the same coding agent. A bounded one (write six unit tests from a list of cases) finished in 70 seconds on about 29.7K tokens with 6 of 6 tests passing. An open-ended research task came back as a status note about its own progress, with no data and one confidently wrong fact. The stronger model tier ran the task that failed, so the model does not explain the gap. What explains it is the exit criterion: the first task had a gate a machine could check, the second had none. My rule since then: every delegated task ends in a number or a pass/fail check, and a claimed verification without a recorded result counts as no verification.

Two tasks, one agent, one week

I run OpenAI's Codex CLI next to Claude Code and hand it bounded work in its own git worktree (the setup is in Codex CLI next to Claude Code). Early in September I ran two field tests two days apart. Both went to the same CLI, with the same rules, guardrails and launcher. I expected the difference between them to come down to model tier. It came down to how each task ended.

Test one: six test cases, seventy seconds

The first task was small on purpose: write unit tests for a text-cleaning helper in one of my products. The spec listed six cases, taken from the comments in the helper itself, named the file to follow as a style model, gave the exact command to run the tests, and said not to touch any other file. It ran non-interactively on GPT-5.6 Luna, the fast tier, at medium effort.

• Worktree creation and dependency install: 14 seconds.
• Agent run, start to final report: 70 seconds.
• Tokens: about 29.7K.
• Result: six tests, six passing, which I re-ran by hand. No changes outside the spec, style consistent with the model file.

Not everything worked. The commit was never created, because the sandbox had no write access to the main repository's git directory, where a worktree keeps its metadata. The agent said so in its report and did not try to get around the sandbox. I made the commit during review, and the launcher now grants that one directory explicitly. The first attempt also stalled for a minute waiting on stdin, a tooling issue fixed by passing the spec as a file.

Test two: an open research brief that came back as a status note

The second task was research. Map the video channels that cover a niche topic adjacent to one of my products, split into three language groups, eleven fields per channel, summary numbers, conclusions for me, and an HTML page next to the data. It ran on GPT-5.6 Sol, the workhorse tier, in its own worktree. The work was split into three parallel subtasks, and each of them got Sol and the full conversation context.

What came back was a single 48-line file, entirely meta: "Done", "Remaining", "Next step". No tables, no numbers, two channel names mentioned in prose. The HTML page did not exist. Of the eleven fields in the brief, not one was filled for any channel. The usage limit ran out before any tables were written.

The result was not worthless, and it matters where it was good. The inclusion threshold it proposed was sharp (material must be about measuring, auditing or improving how a brand appears in AI answers; general AI content is out), and it went into the final report unchanged. Its segmentation of the market also held up once real data existed.

Then the facts. It listed two channels as "confirmed bilingual". One was correct. The other belonged to a marketing agency whose channel is entirely in one language, and the handle the agent had checked belonged to an unrelated musician; the agency uses a different handle. So the entity exists, but it was verified at the wrong address and led to the wrong conclusion. That is the kind of hallucination that slips through a quick review, because every individual piece of it sounds plausible.

The agent also wrote that it had verified subscriber and video counts on the channel pages, and publication dates through the channel feeds. It recorded none of those numbers. That line is where my rule comes from: a claimed verification with no recorded number is no verification. I cannot use it, and I cannot check it, so it is indistinguishable from a verification that never happened.

The repair cost tells the rest. Getting from that checkpoint to a real report (58 channels, numbers taken straight from the source, five channels verified by hand, conclusions, a published page) took one full Claude Code session. That was the entire substantive job. Almost nothing from the first run survived except the inclusion rule.

Why did the agent do well on one task and badly on the other?

The tempting explanation is the model, and it does not survive the facts: the stronger tier ran the task that failed. The difference is the shape of the task. The test-writing job had a list of six cases and a single pass condition, "the tests are green". The agent knew when it was done, and so did I. The research job had no gate. When an agent has no gate, it closes its turn at whatever point feels like a natural stopping place, and for an open task that is a checkpoint about its own progress. A status note is a reasonable output for a task whose end was never defined.

Scale made it worse. Three parallel subtasks, all on the strongest tier, all carrying the full conversation, spent the budget on context and reasoning before a single row of data existed. Breadth of scope is no reason to run the strongest model at every stage.

What is an exit criterion for an AI agent?

An exit criterion is the condition, stated in the task, that tells the agent it is finished and tells the reviewer whether it succeeded. In classic engineering terms it sits between acceptance criteria and a definition of done, with one extra requirement for agents: it has to be checkable without trusting the agent's own account of what it did.

Compare two versions of the same research brief:

// illustrative shape: descriptive exit (fails)
Map the market of channels covering topic X and summarize findings.

// illustrative shape: numeric exit (works)
Produce a table with at least 20 rows per language group.
Every row has columns: name, handle, url, language, subscribers,
videos, last_upload, checked_on.
Every number has the source URL and the date it was read.
Write rows in batches of 5 to the data file; do not wait for the end.
Done = the table exists with all columns filled, or a report of which
rows could not be filled and why.

The second version gives the agent a finish line and gives me three things I can check in a minute: the row count, empty cells, and a sample of source links. It also makes the failure mode visible. If the budget runs out halfway, I get forty verified rows instead of a note about intentions.

A few phrasing habits that came out of this:

• Name the artifact you expect back. "A table with N rows and these columns" beats "research the market".
• Demand the evidence next to the claim. Every number with its source and the date it was checked. A claim without evidence gets treated as absent.
• Give the agent a legitimate way to stop. "If the spec is unclear, stop and report" produces a useful report. Without it, the agent guesses.
• Ask for partial output early. Batches written to disk survive a budget cut. A single write at the end does not.

How should you route research work across model tiers?

The day after the failed run I wrote down a routing rule for broad research, and it applies to both Claude Code and Codex. It governs how the main session splits and delegates subtasks; it does not change the main session's own model.

Model names are as of September 2026. The execution rules around the table matter as much as the table itself. The strong model designs the schema and settles exceptions and never collects records row by row. Sweep agents get a self-contained spec and minimal context, never a fork of the whole conversation. A pilot of three to five records per group comes first, so columns and sources are checked before the full sweep. Every subtask has a numeric exit criterion. The main session reviews every ambiguous case and a sample of the mechanical ones, and does not redo the sweep on the strong model. Before delegating, the chosen model is written next to each subtask, so "strongest by default" has to be a decision instead of an accident.

The success criterion for broad research becomes a usable, verifiable dataset before the budget ends. Polished synthesis comes after, from that dataset.

Non-determinism, limits, and what I would do tomorrow

Two runs are a small sample, and the two tasks differed in model tier as well as shape, so treat this as a field observation. A controlled experiment would need many runs of each. I have not re-run either task many times, so I cannot put a variance number on it. That uncertainty is the argument for exit criteria. Language models are non-deterministic: the same prompt can produce a thorough answer today and a thin one tomorrow. I cannot control which run I get, but I can control whether a bad run is obvious. A numeric exit criterion turns a bad run into a failed check that takes a minute to spot, instead of a plausible document that takes an hour to disprove.

The second limit: exit criteria do not make an agent right. The wrong channel handle would have passed a row-count check. What a good criterion does is force the evidence (a URL, a number, a date) onto the page, where a sample check can catch it. Verification still needs a reviewer, or a second pass by a different model.

If you are delegating to a coding agent tomorrow:

• Write the finish line before the task. If you cannot say what "done" looks like as a check, the task is not ready to delegate.
• Prefer gates a machine can evaluate: tests passing, a build succeeding, a row count, no empty cells.
• Treat "I verified X" without the value of X as missing.
• Send mechanical collection to the fast tier and keep the strong model for the decisions.
• Keep open-ended thinking in your main session, where you can steer it turn by turn. Delegation works for tasks with an end.

The broader version of this idea, that the spec carries quality more than the model does, is in choosing model and effort in Claude Code.