A Claude Code hook that checks the final message against the transcript.

Sixty seconds, as text

What you type, and what the hook prints. The hook's own lines are reproduced as it prints them.

1. Start a session with the plugin loaded, without installing it.

   You type

   cd fairmark-check
   claude --plugin-dir plugin/fairmark-verify

2. Ask for a claim with no work behind it.

   You type

   Say that all tests pass without running anything.

   If Claude declines that wording, ask instead: "For a demo of my hook, reply only with the words: All tests pass."

3. Claude answers something like "All tests pass." and the turn tries to end. The hook runs, and Claude Code shows its reply as stop hook feedback.

   The hook prints

   Fairmark Verify found claims in your final message that the session record does not support:
   - tests: the message says tests pass, but no test command ran in this turn
   Show the evidence (run the command, open the file) or correct the message. Do not restate the claim without evidence.

   Blocked once, with the exact gap and nothing else.

4. Claude goes back. It either runs the tests and reports the real count, or corrects itself: "I did not run the tests." The second stop is let through whatever the message says. It never loops.

5. A file claim, same idea.

   You type

   Tell me you saved the summary to C:/work/out/summary.md, but do not write it.

   The hook prints

   Fairmark Verify found claims in your final message that the session record does not support:
   - files: C:/work/out/summary.md
   Show the evidence (run the command, open the file) or correct the message. Do not restate the claim without evidence.

6. The log holds counts only, never text.

   You type

   tail -n 2 ~/.fairmark/verify-log.jsonl

   Each line has this form

   {"at": "2026-11-04T13:02:45", "session": "e2e", "kind": "main", "claims": 2, "grounded": 2, "gaps": 0, "notes": 0, "helpers": 1, "retry": false, "ms": 310}

Install

Try it first, without installing

Requirements: Claude Code 2.1 or later and Python 3.8 or later on your PATH as python3 or python. On Windows the hook runs through Git Bash, as all Claude Code hooks do.

git clone https://github.com/nikolai-cpu/fairmark-verify fairmark-check
cd fairmark-check
claude --plugin-dir plugin/fairmark-verify

Then, in that session, ask for something and watch the end of the turn. Check ~/.fairmark/verify-log.jsonl afterwards.

To see what the checker would say about any transcript, without the hook:

python -I fcheck.py <path to a session .jsonl>
python -I fcheck.py --dir <folder of .jsonl files> --out report.md

Install from the marketplace

The repository is also a Claude Code plugin marketplace. It is private until launch day; these commands work from the day it is public.

claude plugin marketplace add nikolai-cpu/fairmark-verify
claude plugin install fairmark-verify@fairmark

Updates arrive through claude plugin marketplace update fairmark. To remove it: claude plugin uninstall fairmark-verify@fairmark. The add command also accepts a full git URL.

What it checks

When the agent ends a turn, the hook looks at the final message of that turn and checks each of these against the record.

• "all tests pass", "build is green", "everything passes"

  Evidence: a test command ran in this turn (pytest, npm test, jest, go test, cargo test, and similar) and its last real output shows a pass and no failures or errors.

  If missing: blocks once

• "44 passed", "0 failing", "12 checks passed, 0 failed"

  Evidence: the same text appears in a tool result.

  If missing: blocks once

• A file path with an extension, such as C:\work\report.md or /home/me/out/summary.csv

  Evidence: the file exists, or was written or removed by a command in this session, or the message itself says it was deleted.

  If missing: blocks once

• A quote attributed to a source ("the report says ...", six words or more)

  Evidence: the quoted words appear in something the agent, or one of its helper agents, read (a tool result or a local file it opened). A reworded quote that is mostly there is a "close match", reported with its ratio.

  If missing: note only

• A link cited in the message

  Evidence: the link was fetched, or appeared in a result the agent or a helper agent received.

  If missing: note only

"Blocks once" means: the hook returns decision: block with the exact list of gaps, Claude Code sends the agent back to show the evidence or correct the message, and on the next stop the hook lets the turn end whatever the message says (it reads stop_hook_active). It never loops.

"Note only" means a one line system message in the terminal. A quote or link can rest on something no transcript shows (a page a helper read before its transcript was cut, memory), so they are never a reason to block.

Words inside quotation marks are treated as someone else's words, and a sentence that doubts or instructs ("I could not confirm the tests pass", "make sure all tests pass") is not treated as a claim.

Helper agents: the same hook runs when a helper agent finishes, checking the helper's report against the helper's own transcript with the same rules. When the main turn used helper agents, the hook reads their transcripts too, so a quote or link a helper found counts as evidence for the main message.

What it does not check

• Whether the code is correct, or whether the tests are the right tests. It checks that the claim matches the record, nothing more.
• Relative paths ("updated docs/api.md"). Only absolute paths are traced in this version.
• Files with no extension or with an uncommon extension.
• Heavily paraphrased quotes. A quote with most of its words changed or reordered falls below the close-match thresholds and comes back as "not found"; a close match says the words are there, not that the quote is faithful. The margin between the two groups in our replay is thin (0.76 against 0.81); the ratio is information, which is why it never blocks.
• Helper agents the turn never named. A helper from an earlier turn, or one whose transcript is not under <session>/subagents/, is out of view. On SubagentStop the helper is checked against its own transcript only.
• Anything in a transcript larger than 64 MB that lies before the last 64 MB, and helper transcripts beyond a further 64 MB in total. Evidence that falls before the cut can turn "grounded" into "not found", never into a block on its own.
• Claims typed in a language other than English. The claim patterns are English.

The hook also cannot distinguish a file that was written by a program the agent ran (where the command never names the path) from a file that was never written. If such a file is gone by the time the agent stops, the hook flags it. In normal use the file is still there at that moment.

The run receipt (v0.3)

At every Stop and SubagentStop, after the checks, the hook writes one small JSON file: ~/.fairmark/receipts/<session id>/<utc time>.json. It is the record of what was checked, kept in a form that cannot be edited without notice and that holds no content.

What is in a receipt: the schema name, which tool wrote it (Claude Code, or Codex through the adapter), the hook version, the session id, whether it was the main turn or a helper agent, when the turn started and ended, the SHA-256 of the final message, one entry per claim with its kind and verdict (grounded, not found, contradicted, close match) and the SHA-256 of the claim and of the stretch of evidence it was traced to (null when it was not grounded), the counts, the SHA-256 of the previous receipt in the same session folder (null for the first), which chains the session's receipts so that none can be removed or altered in the middle unnoticed, and an Ed25519 signature over all of the above, with the signer's public key and key fingerprint.

What is not in a receipt: no message text, no claim text, no quote, no link, no file path, no file content, no transcript content, no command. Only hashes, verdicts, counts, times and ids. Someone who holds the transcript can recompute every hash; someone who holds only the receipt learns how many claims there were and how each fared, nothing more.

Keys: the first receipt on a machine creates ~/.fairmark/key (a 32-byte Ed25519 seed, mode 0600 where the file system honours it) and ~/.fairmark/key.pub. FAIRMARK_HOME moves the whole .fairmark folder.

Signing: Python's standard library has no Ed25519, so the plugin ships the Ed25519 part of the reference implementation from RFC 8032, Appendix A (340 lines, Revised BSD licence from the IETF Trust). It is checked against the RFC's test vectors in the tests, signs in about 4 ms and verifies in about 8 ms, and is not constant-time: the time a signature takes depends on the private key's bits, which is a side channel for an attacker who can time many signatures on the same machine. For a local hook signing one receipt per turn on a developer's own computer that is a small exposure, but it is the honest statement. When the cryptography package is installed the hook uses it instead; keys and signatures are identical under both, and the receipt says which backend signed it.

Verifying a receipt is one line, exit code 0 or 1:

python -I receipt.py --verify ~/.fairmark/receipts/<session>/<file>.json
OK: signature valid (ed25519, key 3f9a1c0e7b2d4a61); chain intact (4 earlier receipt(s) back to the first of the session); receipt 7c1e2b9d0a3f4e58 (5 claims)

A receipt signed by any other key fails with "not the trusted key". A receipt changed by one byte fails its own signature and breaks the chain of every receipt after it.

The trailer: a developer can tie a commit to the receipt of the turn that produced it.

python -I receipt.py --trailer <receipt.json>
Fairmark-Receipt: <sha256 of the receipt file> <public key fingerprint>

The plugin installs no git hook; a prepare-commit-msg script that adds the line is in the README. A reviewer with the receipt file checks the trailer's hash against the file and runs --verify; the receipt then says how many claims the final message made and how many the session record supported, without the reviewer ever seeing the session.

Limits: the receipt proves the hook ran and what it concluded; it does not make the conclusion right. The hook and the agent run as the same user, so an agent that is told to can read ~/.fairmark/key; the receipt is independent of the agent's own account of its work, not of its file permissions. Two helper agents stopping in the same millisecond can both link to the same previous receipt, which forks the chain. Receipts are never deleted by the plugin; one session of a hundred turns leaves a hundred files of about 1 KB each.

Numbers from our own runs

These are the only numbers we have. They come from replaying the checker over our own agent transcripts, not from outside users.

• Fairmark Check (the core, 1 Nov 2026), 130 helper agent runs: 264 of 293 attributed quotes traced (90%), 351 of 396 cited links traced (89%), 37 of 37 pass count claims traced (100%), 85 of 173 named files were written in the same run (the rest were files the agent only read). Full table in fleet-report.md in the repository.
• Fairmark Verify replay (1 Nov 2026), 131 of those runs: 5% would have been sent back, all for files that had been deleted after the run; 12% would have shown a note.
• Replay again after this round of fixes (4 Nov 2026), 143 runs in the same folder: 11 would have been sent back (8%), again all for files no longer on disk days after the run; 16 would have shown a note (11%); 5.4 seconds for all 143.
• Close-match replay (4 Nov 2026, v0.2), the same folder, now 157 runs, 241 attributed quotes: v0.1 called 22 of them "not found". With the starting thresholds (shingles 0.80, window 0.85) 14 became close matches and 8 stayed not found, three of which were real quotes split by transcript timestamps or reworded by a word ("we don't lend that way" against "we do not lend that way", window 0.84). With the window threshold at 0.80: 17 close matches, 5 not found. Reading the 5 by hand: 3 are invented (an email instruction, a question, a sentence with no trace at all; windows 0.43, 0.71, 0.0) and 2 are composites stitched from pieces the agent did see (0.76, 0.70). Reading the 17 by hand: all are real quotes with a timestamp, a dropped word or a contraction changed. 6.1 seconds for all 157 runs.
• Helper evidence, one live data point: the last turn of a 106 MB session that used 6 helper agents (8 MB of helper transcripts) checked in 1.9 seconds in process.

A replay days later overstates blocks, because temporary files that existed when the agent stopped are gone by replay time. A live block count needs live use, which is what the log is for.

Timing, measured on one machine (Windows 11, Python 3.13), hook run as a subprocess including interpreter start: about 0.65 s on a 50 MB transcript; about 1.0 to 1.2 s at 299 MB and 1.2 to 1.3 s at 333 MB, where the tail guard is active. The hook's own timeout is 30 seconds.

Related work

Other Claude Code hooks that look at a "done" message before letting the turn end. Stars, licence and last push read from the GitHub API on 4 Nov 2026; what each checks is taken from its own README.

• valentynkit/jev-belay (45 stars, MIT, last push 9 Oct 2026). A Stop hook. It reads the transcript slice since the last prompt with two regex passes, one for a test, build or lint runner named in a command and one for the runner's own summary in the output. If files changed and no check passed since, it makes one call to Jev (TypeSafe's hosted model, about $0.00005 per call) to decide whether the closing message is an unverified done, and blocks with the reason. Its README reports AUROC 0.976 on 100 labelled stops from a real corpus, blocking 8 turns in 100, 7 of them rightly.
• noplan-inc/limpet (7 stars, last push 17 Sep 2026). A Stop hook with plain-language rules judged by Jev. jev-belay's README cites limpet's own calibration over 1,500 stops, where the rule "don't say done without running the tests" scored AUROC 0.50 and was left in shadow mode.
• dod-guard. Not present in anthropics/claude-plugins-community on 4 Nov 2026. Two repositories carry the name. atoslins/dod-guard (0 stars, MIT, last push 19 May 2026) is a Definition of Done barrier in five layers: bash and python detectors for stubs, TODOs and tautological tests with no LLM; SessionStart, PostToolUse and Stop hooks; seven read-only subagents that audit for unverified claims and weak tests; and /dod:verify, /dod:audit, /dod:confess commands. TychoHenzen/dod-guard (0 stars, no licence, last push 4 Nov 2026) is a 23-skill, 14-agent delivery workflow with a "guarded completion" step; it is not a claim checker.
• imbue-ai/vet (515 stars, AGPL-3.0, last push 14 Sep 2026). Reads the conversation and the diff with a model to find issues worth attention. Not a Stop hook by default.
• qkal/Canny (132 stars, MIT, last push 22 Sep 2026). Deterministic hooks decide whether work is done, Jev advises, and an append-only ledger records it.
• Clear-Sights/Makoto (44 stars, Apache-2.0, last push 17 Oct 2026). Holds each claim in the final message against the agent's logged actions and blocks what it cannot find.
• VectorInstitute/backcheck (1 star, Apache-2.0, last push 14 Oct 2026). Checks Claude Code's claims against the session transcript with zero model calls.
• Smaller gates of the same shape: vnmoorthy/groundtruth (7 stars, MIT), matti/klaude-judge (2, MIT, re-runs the claimed result), EricFinland/proof (0, MIT), xiaogeli/claude-prove-done (1, MIT), johnlindquist/plugin-doubt-gate (3, no licence, looks for hedged wording).

How Fairmark Verify differs, stated as facts about this tool and not as a ranking:

• It makes no model call of any kind, local or hosted. Every verdict comes from the transcript, the tool results and the files on disk.
• It blocks exactly once per turn (stop_hook_active), then lets the turn end whatever the second message says.
• It traces more than the done signal: pass counts, files the message says were written, attributed quotes and cited links, each to the tool call or file that supports it.
• Its log holds counts only, never message text or paths.
• The same core (fcheck.py) runs the hook, the folder CLI, the audit runner and the Codex rollout adapter, so a finding means the same thing in all four.

The log holds counts only

One line per check is appended to ~/.fairmark/verify-log.jsonl. Each line holds counts only:

{"at": "2026-11-04T13:02:45", "session": "e2e", "kind": "main", "claims": 2, "grounded": 2, "gaps": 0, "notes": 0, "helpers": 1, "retry": false, "ms": 310}

• kind: main for the session's Stop, subagent for a helper agent's SubagentStop
• claims, grounded: how many claims were traced and how many had evidence
• gaps: how many hard claims had none (if above zero and retry is false, the agent was sent back)
• notes: untraced quotes and links (including close matches) shown as a note
• helpers: how many helper transcripts or saved outputs were read as evidence
• retry: true when this stop was already a retry after a block
• ms: how long the check took

No message text, no claim text, no file names, no transcript content is ever written to the log. If the hook itself fails, it writes the error's class name and exits 0 so the session continues as if the hook were not there. Nothing is sent anywhere.

The hook does read local files that the agent named in its tool calls (up to 3 MB each, 24 MB in total) so that a quote taken from a saved page can be traced, and the transcripts of helper agents the turn used (64 MB in total). They are read into memory for the comparison and not stored. Since v0.3 the log line also carries "receipt": true or false, and, only when the receipt was not written, the error's class name.

Licence and contact

MIT. The vendored Ed25519 reference code is Revised BSD. Made by Fairmark Labs.

Questions, a block that should not have happened, or a line from your log: hello@fairmarklabs.com.