01 · Creation
Kane CLI Hack Day
Attest
Kane CLI runs every step of a README or AI how-to in a real Chrome; each line comes back proven, stale or broken with a screenshot, and an agent fixes stale steps.
- Documentation is the test suite nobody runs. Getting-started guides and AI how-to answers drift as UIs change, and AI coding agents learn from those docs, so a stale doc step becomes a hallucinated ag
https://youtu.be/b6pe-q9gbLs
01 / 05 · Attest
02 · See it work
Kane CLI Hack Day
See it work
- Ingest the doc at a fixed commit and number its lines
- Compile claims (shell, browser, prose) with line spans; fenced blocks become shell claims verbatim
- Run shell claims in a scratch directory; keep dev servers alive for the browser steps
https://youtu.be/b6pe-q9gbLs
02 / 05 · Attest
03 · The harness
Kane CLI Hack Day
The intelligence behind shipping
Attest is the harness: it writes the Kane tests from a doc, runs them, reads Kane's evidence and decides a verdict per doc line. Kane is a witness, not a judge: each step stores the exact visible text closest to what the doc promised, so a renamed control comes back as stale with the new label instead of a failure. Stale or broken verdicts feed a closed loop (Codex patches only the doc, Kane re-proves). Every Kane run is filed with kanehack report. Attest's own README is proven by Kane (6 of 6).
- 01Compile the doc into claims pinned to line spans
- →
- 02Write browser claims as a Kane _test.md, run testmd --agent
- →
- 03Read NDJSON and the evidence pack, badge each doc line
- →
- 04Agent patches stale steps, Kane re-proves from cache
Components: testmd · sealed_evidence
03 / 05 · Attest
04 · Proof
Kane CLI Hack Day
What Kane actually verified
- ✓ VerifiedKane run passed: attest: prove github-secret-scanning c03, c05, c06, c07, c08run 128 · commit 9a413fbf21 · kane_run:kanehack_cli
- ✓ VerifiedKane run passed: attest: prove github-secret-scanning c03, c05, c06, c07, c08run 124 · commit 9a413fbf21 · kane_run:kanehack_cli
- ✓ VerifiedKane run passed: attest: prove readme c05, c06run 82 · commit d56396cfed · kane_run:kanehack_cli
- ✓ VerifiedKane run passed: attest: prove vite c07, c08, c09run 9 · commit cd67ffeec0 · kane_run:kanehack_cli
- ✓ VerifiedKane run passed: attest: prove vite c07, c08, c09run 8 · commit cd67ffeec0 · kane_run:kanehack_cli
Not yet verified: Shell claims run on the host with the user's environment; the scratch directory is not a sandbox, so only prove docs you would run by hand. · Attest assumes the app is right and the doc is stale; it cannot yet refuse a doc repair that would hide an app regression. · Kane runs a whole test file before Attest judges the first claim; cards use --stop-before for irreversible or real-form steps.
04 / 05 · Attest
05 · Outcome
Kane CLI Hack Day
What improved
Kane CLI runs every step of a README or AI how-to in a real Chrome; each line comes back proven, stale or broken with a screenshot, and an agent fixes stale steps.
3 doc steps Stale doc steps caught on live UIs and fixed (Vite heading, GitHub settings label, Attest README) (baseline 0; source: https://github.com/SyberLabs/attest/blob/main/demo/cards.md)
5 loops closed Agent loops closed with the patched step re-proven by Kane (5 of 9 attempts; 10 dependent steps proven from the same run) (baseline 0; source: https://github.com/SyberLabs/attest/blob/main/NOTES.md)
0 Kane credits Kane credits to re-prove the Vite doc after the fix, replayed from cache (vs authoring the same 3 steps) (baseline 48.5; source: https://github.com/SyberLabs/attest/blob/main/NOTES.md)
https://github.com/SyberLabs/attest
05 / 05 · Attest