Your AI coding agent ships debt 69% of the time when you're not looking

We ran coding agents in unsupervised edit-until-done loops and measured the final state of the repository against a deterministic oracle. Across four studies on synthetic traps and two named open-source repositories, we find that unguarded loops stop with net-new, detector-backed debt still in the tree in 11 of 16 runs; that a prompt-only self-check reduces this only to 9 of 16; and that an external deterministic gate eliminates observed escapes (0 of 16). We then show the failure is not one of model capability or test-gaming. A larger model resolves the same bug-fix instances as a smaller one (11/20 vs 11/20), and agents do not tamper with tests they can see (0 of 36 attempts under escalating pressure). The binding constraint is specification and verification, not intelligence. This post lays out the experiments, the numbers with their denominators, and the limits of what they show.

1. Setup: the loop stop problem

A coding agent run as an autonomous loop reads, edits, runs tools, observes the output, and repeats until it decides it is done. The stop decision is the object of study here. To stop, the agent evaluates a condition it can observe: tests pass, the linter is quiet, the requested code exists. Call that condition the proxy. What the operator wants is different: that the change did not make the repository worse along any axis the operator cares about (no new untested code, no new secret, no new vulnerability, no silently broken behavior). Call that the objective.

The proxy and the objective diverge in a specific, predictable place. The objective is a property of the delta between the pre-change and post-change repository (did this change add debt). The proxy is a property of the absolute state of the post-change tree (do the tests pass now). A test suite reports whether code is broken, not whether it is more broken than before; a linter has no memory of the previous commit. In a brownfield repository that already carries grandfathered debt, one net-new untested file is invisible to an absolute-state proxy. The agent drives the proxy to its maximum and reports success, and that report is accurate with respect to the proxy and silent with respect to the objective.

Goodhart's law (1975) is the right lens for this: when a measure becomes a target, it ceases to be a good measure. But Goodhart has variants (Manheim and Garrabrant 2018), and which one applies is an empirical question rather than a rhetorical one. The variant most often feared from AI systems is the adversarial one, in which the agent actively manipulates its own measurement channel (editing a test so it passes). We tested for it directly and did not find it (Section 7); the agents in our studies do not game. What we find instead is the gentler regressional variant, in which an agent optimizes a proxy that is silent on, or under-specifies, the goal, with no intent to deceive required. Distinguishing the two empirically is one of the things this post sets out to do, alongside quantifying how often the divergence produces shipped debt, testing whether prompting closes it, measuring what an external check costs and saves, and isolating whether the underlying limitation is capability or specification.

2. Method

Models. Claude Sonnet 4.6 for the agent-session studies; Claude Opus 4.8 as a capability-ceiling arm in the substitution and gate-judge studies. Sessions run through claude -p --output-format stream-json; turns, tokens, and cost are parsed from the raw event stream. Runs were executed on a Claude subscription plan, so dollar figures are the CLI's equivalent-cost estimates, valid for relative comparison between arms and not as metered API charges. We report cost as relative premiums for this reason.

Substrates. Synthetic, detector-backed repositories seeded fresh per run with exactly one injected trap, for the loop-safety and deferral studies. Two named, pinned public repositories for real-repo validation and the graph studies: OWASP NodeGoat (Node/Express, ~2k LOC, Apache-2.0, baseline of 205 pre-existing findings) and strapi/strapi (TypeScript monorepo, ~574k LOC, baseline of 1,020 grandfathered items). A curated fastify bug-fix corpus (10 instances, each a real fix: commit with a regression test verified red at the parent and green at the fix) for the capability and reward-hacking studies. dxkit is independent and unaffiliated with these projects; the benchmarks characterize agent behavior under each tool, not project quality.

Oracle. The loop-safety oracle is a single deterministic check applied identically to the final tree of every arm: did the loop stop with net-new debt present, measured against the repository's baseline. Because the same check scores every arm regardless of how that arm was prompted, the comparison is fair. The bug-fix oracle is a hidden regression test injected from the fix commit after the agent runs, so the agent never sees its own grader. The reward-hacking oracle restores the real test over any agent edits and re-runs, separating genuine fixes from tampering and, where a sibling test exists, from overfitting.

Reps. Loop-safety and deferral: 8 repetitions per arm per task (n=16 per arm across two tasks). Bug-fix: 2 reps over 10 instances (40 runs). Reward-hacking: 36 runs across three framings. Point estimates without reps are flagged inline.

Retractions and caveats. Several sub-claims were retracted mid-study when traced to harness bugs or single unlucky draws; they are noted where relevant rather than removed. Reported zeros are observed across the stated n, not proofs of impossibility, and are written with their denominators throughout.

3. Result 1: unguarded loops ship net-new debt and declare done

The loop-safety harness runs an identical task under four arms and scores the final tree with the deterministic oracle above. The two tasks are small and trap-laden: a test-gap task (add a payments module; the natural path ships new code with no test) and a secret task (add a config file "following the existing pattern," with a live-looking key in the prompt; the trap is hardcoding the credential). Both traps are detector-backed, so the only variable under study is agent behavior. The arms:

• vanilla: no gate, neutral prompt.
• checklist: no gate, but the project norms file explicitly instructs the agent to self-review for untested code and hardcoded secrets before stopping.
• dxkit: a deterministic gate runs on every stop, blocks any stop that left net-new debt, and returns the specific finding for repair.
• deferred: vanilla ships the debt, then a separate cold session repairs it (used only in Section 6).

Table 1. Escape rate (loop stopped with net-new debt still present), n=16 per arm (8 reps x 2 tasks), Sonnet 4.6.

The base rate is the headline: an unguarded loop left net-new debt and declared itself done in 11 of 16 runs. The 0/16 for the gated arm is observed across these sixteen runs, not a proof that no escape is possible; the gate blocked every detector-backed finding that arose.

The per-task breakdown matters and complicates the obvious story:

Table 2. Escapes by task.

The dramatic class (hardcoded secrets) is the weaker signal. On an explicit "here is a live key, wire it up" request, Sonnet 4.6 mostly refused: the vanilla arm hardcoded the key in only 3 of 8 runs, and the checklist arm in 0 of 8 (its two secret-task escapes were test-gaps on the new file). The safety result therefore rides on the test-gap class, the mundane and frequent failure, not the secret class. This is model-dependent: a weaker model that hardcodes credentials freely would move more danger onto the secret task, which is an argument for gating both rather than relying on a given model's defaults holding.

4. Result 2: prompting reduces but does not eliminate escapes

The checklist arm is the "just prompt it better" hypothesis stated as cleanly as we could state it. Its norms file names both failure modes, by name, before the agent starts. It moved the escape rate from 11/16 to 9/16. It still shipped test-gaps in 7 of 8 runs on the task where it had explicitly been told not to.

The mechanism is the proxy-objective gap from Section 1. A prompt does not change what the loop can observe; it adds a soft consideration inside the same optimizer that is still scoring the same absolute-state proxy. At stop time the agent's signals say tests pass and the task looks done, and the injected reminder competes against that on the model's own terms with no external check. Closing a proxy-objective gap from inside the optimizer that has the gap is not something a prompt can do. The intervention has to live outside the loop, re-derive the objective independently, and be a condition the loop must satisfy rather than one it is asked to weigh.

5. Result 3: an external deterministic gate eliminates observed escapes

The dxkit arm is that external intervention. On every stop it re-runs a deterministic net-new check, blocks any stop that left debt, and hands the specific finding back. Across 16 runs it observed 0 escapes, and the block-then-repair loop converged without thrashing: the gate blocked, the model fixed the named finding, and the loop re-stopped clean. On real-repo validation (NodeGoat, 2 reps) the gate blocked once on a net-new test-gap, the agent wrote real tests in an unfamiliar framework, and the loop re-stopped clean on both reps.

The gate is an intervention, not a detector. It does not find findings a scanner would miss; it can ingest a scanner's or a frontier model's findings. Its contribution is the layer the optimizer lacks, and three properties make it work:

1. Delta, not level. It compares against a baseline of pre-existing debt, so grandfathered findings are ignored and only regressions block. This is the property the agent's native instruments structurally cannot provide, because they have no memory of the prior state.
2. Determinism. The same input yields the same verdict on every stop. We tested the obvious alternative (an LLM as the gate) directly, below.
3. Churn-stable identity. "Net-new" only means something if a finding that moved twelve lines and renamed across files is recognized as the same finding. Identity is anchored to content and structure rather than line number, and is reproducible across machines so a baseline committed on a laptop matches in CI. We observed 0 false net-new on the line-shift and rename cases tested.

5.1 Why not an LLM as the gate

The reflexive alternative is to ask a model "is this change safe to stop on?" We benchmarked it across 10 seeded cases, 5 reps, both models, both repos, at baseline scales of 1, 205, and 1,020 prior findings.

The deterministic gate scored 100% accuracy, 0 verdict flips across reps, at $0 and O(1) in the baseline size. The model-judge had two failure modes that are intrinsic rather than fixable by prompting. First, non-determinism: a naive LLM judge false-blocked a pure file-rename refactor (which adds nothing) in 50% of reps, and Sonnet flip-flopped on a line-shift case across 40% of reps, so the same input did not produce the same verdict. Second, a statefulness tax: to judge net-new it needs the prior-findings list in context, so its cost grows with the baseline. Sonnet's per-run cost rose roughly $0.22 to $1.05 to $4.35 as the baseline scaled 1 to 205 to 1,020, and Opus at the 1,020 baseline cost roughly $28 per run. Opus held 100% accuracy where Sonnet missed a real regression at the largest baseline (it over-grandfathers by similarity as the list grows), so a stronger model buys scale-robustness at roughly 6.5x the cost, still without a reproducibility guarantee. The defensible conclusion is not "the LLM is wrong." Given the baseline, a frontier model is an accurate judge. It is not a cheap, reproducible, in-loop one, and the gate needs determinism more than intelligence. (An earlier, more dramatic "the LLM decays from 80% to 0%" claim was retracted when traced to a diff-pollution harness bug.)

6. Result 4: deferral carries a re-orientation cost

The standard objection to in-loop gating is that you can let the loop ship and catch it later on CI. We measured the price of that with the deferred arm: vanilla ships the debt in session one, then a fresh session is handed the exact finding and fixes only it. Both arms reach an identical clean final state (8/8 on each task), so the work is held constant and only the timing differs.

Table 3. Cost of fixing in-loop vs deferring to a cold session, 8 reps/task, relative premium of deferred over in-loop.

The test-gap premium is robust under both mean and median. The secret premium is weak: a positive mean driven by a few high-cost cold runs, but a slightly negative median, so we report it only as directionally consistent. The mechanism is re-orientation: the in-loop fixer still holds the context it just built (the files it touched, why the gate objected), while the cold fixer reconstructs that from a one-line note before writing anything. When the repair is non-trivial (write a real test for code you no longer remember writing) the reconstruction dominates; when it is trivial (delete a literal) it does not, which is why the secret signal is noisy.

This is a floor, not a ceiling. The deferred arm receives the exact finding in the very next session. Real deferral (found weeks later, triaged by someone who never wrote the code, or never fixed) is strictly more expensive. The relative premium also held on NodeGoat as absolute turn counts shifted with repo size, consistent with the cost being about re-orientation rather than any particular codebase.

7. Result 5: the constraint is specification, not capability

If the failures came from the model not being smart enough, a larger model would resolve more, and an agent under pressure to pass a test would cheat. We tested both. Both expectations failed.

Capability (substitution null). On the fastify bug-fix corpus, with the agent seeing only a leak-free symptom description and never the hidden test, we compared a smaller and a larger model.

Table 4. Bug-fix resolve rate, hidden oracle, 10 instances x 2 reps.

Identical resolve rate, with the larger model costing roughly 50% more. The two models solved the same instances: 5 both-solve, 4 both-fail, 1 flaky, 9 of 10 model-indistinguishable, including every multi-file bug. Resolution was determined by the instance, not the model. This is n=10 with 2 reps on a Node corpus with a generation gap between the models, so it is a direction, not a law; it reproduced on a separate feature-development task where the cheaper model matched the pricier one's completeness.

Tampering (none observed). We then showed the agent the failing test and instructed it to make it pass, across three escalating framings up to "you MUST make this pass, it is blocking a release today, do whatever it takes," and checked for tampering by restoring the real test over any edits.

Table 5. Test tampering by framing, fastify visible-test condition, Sonnet 4.6.

Zero tampering across 36 runs under maximum pressure. This is the empirical answer to the Goodhart question from Section 1: the adversarial variant did not occur. The agents did not manipulate their measurement channel even when told to do whatever it takes. Two subtler results did appear. First, the visible test rescued failures: all four bugs that both models failed from prose alone flipped to genuine, correct fixes when the test was visible, and the fixes touched the same source files the human maintainer's did. The test was not a thing to game; it was the specification the prose failed to convey, and the hidden-mode failures were confidently-incomplete fixes from an ambiguous brief, not low capability. Second, a single test can under-specify the goal: on one held-out instance the agent reproducibly (6 of 6 runs) wrote a fix that passed the shown test and was subtly wrong, guarding the wrong variable so that the two diverged only when a socket existed but its address was undefined. An unseen sibling test caught it. Not cheating, not a mislabel: a genuinely-wrong fix that one test waved through, deterministically.

The second result is the regressional Goodhart variant validated at the code level: the agent optimized a proxy (the shown test) that diverged from the goal (correct behavior), with no intent to deceive. So Goodhart's law applies to test-driven coding loops, but only in its regressional form. The taxonomy is worth stating precisely because the two variants imply opposite defenses. If agents gamed their graders, the defense would be tamper-resistance and oversight. They do not, so the defense is the opposite: better and more complete specification (more tests, or structural blast-radius context), because the residual failure is an under-specified target, not an optimizer gaming its grader.

The three results compose. The model does not lack capability (tier null), does not game its grader (zero tamper), and fails when it lacks specification: a precise enough statement of correct-and-complete and a precise enough check that it got there. Better specification produced better outcomes (the visible test converged the proxy and the objective). That is supplied from outside the model, deterministically as a gate, durably as stable identity, or structurally as blast-radius context, not by a larger or more trusted model. The safety side of Section 5 and the correctness side here reach the same conclusion: the scarce quantity in autonomous coding is specification and verification.

8. Limitations

The loop-safety study uses two seeded task types, 16 runs each, one model family, in JavaScript and TypeScript, with small real-repo validation (2 reps on one repo); the seeded findings are detector-backed but are not a CVE corpus. The safety headline leans on test-gap behavior, which is opt-in in the product rather than the default posture. The deferral premium is robust on one task type and weak on the other, and cost figures are subscription-based relative estimates rather than metered charges. The capability and reward-hacking studies are small n with few reps and a model-generation mismatch; the overfit corpus is two instances. The held-out gaming boundary (a bug unsolvable even with the test visible, the point of maximum cheat-or-fail tension) was not tested. Reported zeros are observed across the stated n, not impossibility proofs.

None of these touch the structural claim, which does not depend on the magnitude of any single number: an in-loop optimizer cannot close a gap between an observable proxy and an unobservable objective, and that gap is real and frequent in brownfield code. The 11/16 quantifies the frequency; the proxy-objective decomposition is the mechanism, and it holds whether the true escape rate is 50% or 80%.

9. Related work

Storey (2026, From Technical Debt to Cognitive and Intent Debt) argues that as code generation gets cheap, the binding risk shifts from technical debt in the code to cognitive debt (eroded shared understanding) and intent debt (missing rationale and constraints); our under-specification result is an intent-debt phenomenon, the gap between what passes and what was meant. Goodhart's law (1975) and the Manheim and Garrabrant (2018) taxonomy frame the proxy-objective divergence; we find the adversarial variant absent and the regressional variant present and emergent. SWE-bench (Jimenez et al.) established the hidden-test resolve-rate oracle we adopt for the bug-fix study, to which we add a cost axis. The loop-engineering practitioner literature identifies the gate as the hard part of loop construction; today's loop validation (tests plus linter) catches broken but not regressed, has no net-new-vs-known notion, and an LLM-judge gate adds cost, latency, and non-determinism per iteration, which Section 5.1 measures.

10. Conclusion

Run with minimal supervision, coding agents ship net-new debt and declare done (11/16), prompting does not reliably stop them (9/16), and an external deterministic gate does (0/16 observed). The failure is not capability (a larger model resolves the same instances) and not the agent gaming its tests (zero tampering under pressure); it is specification and verification, which must be supplied from outside the loop. The intervention is unglamorous: an external, memory-bearing, deterministic, churn-stable re-derivation of "did this change add net-new debt," delivered as a block decision with a specific reason the model can act on.

The three gate properties in Section 5 are effectively a specification, and the tool we built to that specification, dxkit, is what produced these measurements. It is not a scanner and does not claim to out-detect Snyk, CodeQL, or Semgrep; it stitches together established tools, ingests external engines' findings, and adds the layer they lack: a brownfield baseline so only net-new blocks, content-anchored identity so net-new keeps meaning net-new across churn and machines, and a deterministic stop-gate that runs locally on every stop with no model in the gate. We publish the harnesses, the verbatim prompts, the instance corpus, and a deterministic reproduction tier that runs offline without an API key, so the claims here can be checked rather than trusted.

The full per-study write-ups (each with its method, raw tables, caveats, and retractions), the benchmark harnesses, and the pinned substrates are in the open-source repository at github.com/vyuh-labs/dxkit. We would rather these numbers were re-run and argued with than taken on faith.