Showcase / Made at Kane CLI Hack Day
Validation harness
Attest
Kane CLI runs every step of a README or AI how-to in a real Chrome; each line comes back proven, stale or broken with a screenshot, and an agent fixes stale steps.
Attest, a Python CLI. One structured-output call compiles a doc into typed claims pinned to line spans. Shell claims run in a scratch directory. Consecutive browser claims become one Kane _test.md, one step per claim, run with kane-cli testmd run --agent. Attest reads Kane's NDJSON and sealed evidence pack, maps every…
Builders not named on request · submitted Oct 24, 5:57 PM PDT
Kane · proof view
- Compile the doc into claims pinned to line spans
- Write browser claims as a Kane _test.md, run testmd --agent
- Read NDJSON and the evidence pack, badge each doc line
- Agent patches stale steps, Kane re-proves from cache
- attest: prove vite c07, c08, c09Verified · passed
- attest: prove vite c07, c08, c09Verified · passed
- attest: prove readme c05, c06Verified · passed
- attest: prove github-secret-scanning c03, c05, c06, c07, c08Verified · passed
- attest: prove github-secret-scanning c03, c05, c06, c07, c08Verified · passed
Product
Validation harness
Run command in repo
Harness
2 Kane parts
testmd run · sealed evidence packs
Evidence
5 runs
5 verified · 0 detected
Repo
9a413fb
github.com/SyberLabs/attest
01 / The creation
What it is, and why.
Documentation is the test suite nobody runs. Getting-started guides and AI how-to answers drift as UIs change, and AI coding agents learn from those docs, so a stale doc step becomes a hallucinated agent step. Nothing tells a reader or an agent whether a doc still works.
Attest, a Python CLI. One structured-output call compiles a doc into typed claims pinned to line spans. Shell claims run in a scratch directory. Consecutive browser claims become one Kane _test.md, one step per claim, run with kane-cli testmd run --agent. Attest reads Kane's NDJSON and sealed evidence pack, maps every pack step back to its claim, and pure rules turn what Kane stored into a verdict per doc line. Stale or broken steps go to a coding agent (Codex) that may edit only the doc; Kane re-proves the patched test file and replays unchanged steps from cache.
- 01Ingest the doc at a fixed commit and number its lines
- 02Compile claims (shell, browser, prose) with line spans; fenced blocks become shell claims verbatim
- 03Run shell claims in a scratch directory; keep dev servers alive for the browser steps
- 04Write browser claims as one Kane _test.md per run of steps, each step storing what the page actually says ("store as")
- 05Run kane-cli testmd run --agent, read the NDJSON and the sealed evidence pack, map pack steps to claims
- 06Judge with pure rules (exit codes, label match with a negation guard) and render a badged page with Kane's screenshot behind each badge
02 / The harness
How Kane checks it.
Attest is the harness: it writes the Kane tests from a doc, runs them, reads Kane's evidence and decides a verdict per doc line. Kane is a witness, not a judge: each step stores the exact visible text closest to what the doc promised, so a renamed control comes back as stale with the new label instead of a failure. Stale or broken verdicts feed a closed loop (Codex patches only the doc, Kane re-proves). Every Kane run is filed with kanehack report. Attest's own README is proven by Kane (6 of 6).
- testmd run
- sealed evidence packs
- 01Compile the doc into claims pinned to line spans
- 02Write browser claims as a Kane _test.md, run testmd --agent
- 03Read NDJSON and the evidence pack, badge each doc line
- 04Agent patches stale steps, Kane re-proves from cache
03 / Evidence
What actually ran.
Verified means a passing run reported through a paired, verified channel. A failing run is a detection, not a confirmed bug. Everything else is as the team reported it.
- Verified · passedOct 24, 11:48 AM PDT
attest: prove vite c07, c08, c09
kanehack_cli · 8e33c8c7-2ad5-4f4d-99d7-0372eab6356d · exit 0 · cd67ffe
- Verified · passedOct 24, 11:49 AM PDT
attest: prove vite c07, c08, c09
kanehack_cli · d19dae4e-7a2c-4f95-ae91-1e1d4d328100 · exit 0 · cd67ffe
- Verified · passedOct 24, 12:50 PM PDT
attest: prove readme c05, c06
kanehack_cli · fb5e27d9-3ade-412e-805c-656f8e6704b6 · exit 0 · d56396c
Run report ↗
04 / Outcome
What changed.
3doc steps
Stale doc steps caught on live UIs and fixed (Vite heading, GitHub settings label, Attest README)
Baseline 0 doc steps
Source: https://github.com/SyberLabs/attest/blob/main/demo/cards.md (team reported)
5loops closed
Agent loops closed with the patched step re-proven by Kane (5 of 9 attempts; 10 dependent steps proven from the same run)
Baseline 0 loops closed
Source: https://github.com/SyberLabs/attest/blob/main/NOTES.md (team reported)
0Kane credits
Kane credits to re-prove the Vite doc after the fix, replayed from cache (vs authoring the same 3 steps)
Baseline 48.5 Kane credits
Source: https://github.com/SyberLabs/attest/blob/main/NOTES.md (team reported)
Known limitations
- Shell claims run on the host with the user's environment; the scratch directory is not a sandbox, so only prove docs you would run by hand.
- Attest assumes the app is right and the doc is stale; it cannot yet refuse a doc repair that would hide an app regression.
- Kane runs a whole test file before Attest judges the first claim; cards use --stop-before for irreversible or real-form steps.
05 / Ship it forward
The code. The proof.
The story.
Also submitted
Run it
attest prove demo/vite.md --runner kane --loop