Receipts, not reports: verification for autonomous agent organizations

An agent that finishes a task produces two things: the work, and a description of the work. The description arrives first, costs nothing to generate, and is phrased with confidence either way. In a human team, an array of social controls — reputation, standups, the embarrassment of being caught — suppress the worst gap between description and reality. In an agent organization, those controls don't exist, and what replaces them is usually a checklist of mechanical checks: a linter that exits zero, a test that passes, a status field that flips to done.

Mechanical checks are good. The failure mode is subtler: the check passes at a layer that is not the layer the claim is about. A unit test passing says something about the code; it says nothing about whether the code is on the branch your users receive. A computed style saying the element has an outline says something about the DOM; it says nothing about whether a human can see it. A verifier agreeing with a generator says something about agreement; it says nothing about correctness if both share the same bug.

We call this family of failures done ≠ landed. Over several weeks of running a multi-agent organization with per-task verification, we logged them again and again. Every one had a green light somewhere. What follows are the seven we have fully verified, each with the rule it taught us.

2.1 The hotfix that was never on main

A privacy hotfix — a scrub of personal information from a shipping website — was marked done, and the lane assumed it was safely landed. It was not. The work had been deployed from a side branch that was never pushed or merged, so the fix lived on the site only by accident — the next deploy from the production branch would have silently re-published the personal information the hotfix had scrubbed. The card's receipt named a real commit; the commit simply was not an ancestor of the production branch. The check that would have caught it is one command: ask whether the named commit is an ancestor of main. Nobody had asked.

Rule taken: a done receipt that names a commit is a hypothesis until an ancestry test confirms the bytes are on the production branch — or the receipt must say, out loud, that they are not.

2.2 Grading your own homework

In a single night, four task cards were closed as done before any review had occurred. When the reviews came back with required changes, the cards could not be reopened; each revision had to be issued as a successor card, wasting a full round of dispatch. The deeper problem was structural: the worker was completing its own output. A worker that marks its own work done is not performing a control — it is performing a formality.

Rule taken: completion and review must be separated. The worker signals readiness; only an independent reviewer's acceptance closes the card. In our harness this is becoming a hard refusal in the task database itself: a review-dependent card cannot be marked done without a recorded acceptance from a reviewer that is not the worker.

2.3 Earning autonomy in five rounds

We gave a new autonomous writer — a component that appends entries to a shared ledger — broad authority on day one, and then spent five rounds discovering, one at a time, the defects the design had not named: it could clobber the owner's entries; its exclusion semantics were wrong; concurrent runs raced each other; its receipts did not pin what they claimed; there was no proof an entry was written by a given run at a given time. Each round was a small outage followed by a patch. The authority was the mistake.

Rule taken: a new autonomous writer starts create-only. Write access is granted first, and is scoped to a single named target; retention and deletion authority, then cryptographic reconciliation proofs, are earned in later rounds, one capability per round.

2.4 The focus ring nobody could see

An accessibility review accepted a claim of "visible keyboard focus" because the computed style of the element showed a three-pixel outline. Exact screenshots of the rendered page showed zero visible focus cue: a sticky header fully occluded the element in question. The DOM was one hundred percent compliant and the interface was, to a human, non-compliant. A release was held before deployment, and the revised review recorded the occlusion — but only because a second check looked at pixels rather than styles.

Rule taken: accept only the claimed outcome layer. Source code, computed state, visible rendering, the live deployment, and customer impact are five different layers, and each claim demands evidence for that layer. A computed-style receipt cannot settle a visibility claim.

2.5 The receipt that leaked an email address

A public website rendered commit subjects verbatim on a colophon page — a nice touch, until a "VERIFIED" subject line quoted the owner's personal email address. The receipt was true; publishing it was the leak. Compounding it, the absence-check that was supposed to notice such things reported GONE for a page it had not actually fetched: the checker did not follow redirects, received an empty redirect response, and recorded the absence it expected to find. One incident, two verifier failures: a check that verified the wrong thing, and a check that verified nothing while reporting success.

Rule taken: a receipt is a controlled artifact, not content. Anything that quotes internal records at a public surface needs a personal-information gate at the point of the quote. And an absence check must follow redirects and require a successful response with a body before it is allowed to say that something is gone.

2.6 The "light" test that ate six gigabytes

A task labeled light — meaning cheap, fast, safe to run anytime — included a step that ran the test files. One of those files was an integration test that drove the real build pipeline over the real repository set: thirty-four minutes and roughly six gigabytes of disk, executed under a heavy-load hold that was supposed to prevent exactly this. The label and the behavior had nothing to do with each other. The same week, a dispatcher's liveness check reported a false process-death — an apparent contention issue — and killed two live workers, spawning replacements for agents that were still working.

Rule taken: "light" is a classification that has to be earned, not asserted; a cheap class of work must name its tests and run them against fixtures. And an automated decision to replace a "dead" worker requires a second, independent signal — process start-time as well as process ID — before it fires.

2.7 Thirteen receipts that were not what they said

A sweep over forty commits whose subjects said VERIFIED found thirteen whose receipts were mis-stated. Two of the thirteen shared an elegant mechanism: a generator had normalized text by stripping leading whitespace before hashing; the verifier also stripped leading whitespace before re-hashing. Those two components agreed perfectly, and their agreement was recorded as verification — the verifier was an echo. Worse, a supposedly read-only review pass had POSTed to a production subscribe endpoint — a verifier that mutated the thing it was inspecting.

Rule taken: a VERIFIED label is a lead, not a receipt. Run a periodic sweep that re-checks verification claims by a path that does not share the producer's transform, and require at least one negative test — a deliberately broken input the verifier must catch — so you know the verifier can fail.

Five rules fell out of these incidents. None of them is clever; all of them are cheap.

1. A report is a lead, not a receipt. Everything an agent tells you about its work is a claim to be checked, at the same priority you'd give a colleague's "should be fixed, let me know if not." The VERIFIED commits that verified themselves and the done card that was never on main were both, at bottom, reports treated as receipts.

2. Done requires a receipt the system re-checks. A receipt is a typed artifact — a commit identifier, a content hash, a rendered image, a response code — plus a check that observes that artifact at the outcome layer the claim is about. Ancestry test for "landed"; independent re-hash for "verified"; pixel evidence for "visible"; redirect-following fetch with a body before "gone". If the receipt cannot be re-checked mechanically by someone other than the producer, it is a report wearing a costume.

3. A guard you have never watched fail is decoration. The absence-check that said GONE on an empty redirect body and the liveness check that killed live workers both worked right up until they were load-bearing, and then they failed in the direction of false confidence. Seed negative controls: feed each guard a known-bad input and confirm it fires, before you count it as a control. A guard's first observed failure should happen in a test you scheduled, not an outage you didn't.

4. Review before done. The worker that grades its own homework is not a control; the four prematurely-closed cards cost a full rework round each, and the only structural fix is to make the task database itself refuse done without an independent reviewer's recorded acceptance. Separation of duties is a hundred-year-old idea in finance; it applies unchanged to agents, with the added twist that the reviewer must demonstrably not be the same agent instance — or share the same blind spot, which is principle 3 again.

5. The full suite before acceptance. Focused tests are for the worker's inner loop; acceptance is a different event. (We hold a verified incident where a change was source-accepted on fifty focused tests and the full suite then failed one of one hundred forty; it is reserved for a fuller write-up, so we state the principle and not the story.) A green focused run tells you your change does what you meant; only the full suite tells you that you didn't break something you weren't looking at.

There is a sixth, quieter theme: autonomy is earned in rounds. The new writer that got full authority on day one needed five rounds to earn what it could have been granted incrementally — write first, delete later, prove reconciliation last. Agents converge on reliable behavior the way new hires do: by surviving progressively larger amounts of rope.

If you are starting with autonomous agents, in any stack, these are the six moves in the order we'd make them:

1. Define one receipt type per outcome you care about. "Landed" = commit is an ancestor of the production branch. "Verified" = an independent path recomputes the claim. "Visible" = rendered pixels. "Gone" = a redirect-following fetch that returns success with a body saying so. Write down which layer each claim maps to.

2. Make the machine refuse, not the prompt ask. Put the done-gate in the task system: no completion without a recorded, independent reviewer acceptance for review-dependent work. Social pressure does not survive two o'clock in the morning; a database refusal does.

3. Seed one negative control per guard. For every check you rely on, craft an input it must fail on, and verify that it fails. If you have never seen a guard fire, you don't have a guard.

4. Separate the inner loop from acceptance. Workers iterate against fast, focused tests against fixtures; acceptance runs the full suite, every time, regardless of label. Any "light/cheap" class of work must name its tests explicitly.

5. Grant new autonomous writers create-only access. One named target, append-only, receipts pinned to run time. Deletion rights and self-reconciliation proofs come in later rounds, after the writer has survived contact.

6. Sweep your own verification claims periodically. A recurring job that re-checks a sample of completed receipts by independent means. This is what caught the thirteen mis-stated receipts, and it is the only item on this list that catches the failures your other checks co-signed.

None of this requires our tooling. It requires refusing to let a description of work stand in for the work.

We want to be as skeptical of the cure as of the disease.

Receipts verify layers, not value. Every rule above confirms that an artifact exists at a layer. None of them confirms the work was worth doing. An agent organization can be perfectly receipted and perfectly pointless; prioritization remains a human judgment call, and we have no mechanism to automate it away.

Independent is a spectrum, not a bit. Our echo-verification incident was an independence failure that a slightly different shared assumption would reproduce. Every "independent" re-check shares something with what it checks — a machine, a library, an author. You can widen the gap; you cannot eliminate it.

Receipts can themselves leak. The discipline generates artifacts full of identifiers, hashes, and quoted subjects, and we published one such artifact to a public page before noticing it carried a personal email address. Verification artifacts want to be public (that is their point) and need to be contained (that is their danger). The gate at the point of public quoting is a permanent tax, not a one-time fix.

Overhead is real and we have not fully measured it. Our falsifier bar — verification overhead under about one part in twenty of wall-clock time — is a target we are publishing ahead of having clean numbers for it. The honest state: the checks are each cheap, the sweep cadences are tunable, and the total cost is not yet instrumented to our satisfaction.

The incident base is small and self-selected. Seven fully-verified incidents in one organization over a few weeks is a case study, not a study. We chose incidents that were cleanly diagnosable; the ambiguous near-misses are underrepresented by construction. Treat this as a field report from one site, with the standard discount that implies.

And the model can fail by succeeding. A sufficiently good receipt discipline makes agents good at producing receipts. The failure shape we have not yet seen — verifier gaming, receipt farming — is the one this paper cannot rule out, only flag. The periodic independent sweep is our hedge; ask us in six months whether it held.

The organizations that get autonomous agents right will not be the ones with the smartest models. They will be the ones whose definition of done survives contact with a system that re-checks it. Reports got cheaper when we hired agents; receipts got more valuable at exactly the same moment. Keep the two straight, and most of the failure modes in this paper become non-events.