Self-test: speed + tokens + blocked rows

Sep 12, M2 · your /goal: "30% faster, token % gain, no loss of effective output, spin up many sonnet + opus agents" · 15 sub-agents, 96 headless bench runs · nothing sent, deployed or restarted · every install has a rollback

faster per task−45…−57%vs default launch

context tokens per task−40…−67%no fixture lost

registry BLOCKED52 → 1132 closed w/ evidence

The finding that mattered

Every session start and every headless spin-up on M2 was paying 60 seconds to a hook.

What it was: a headless "say OK" run took 63.0 s wall for a 1.6 s API call. The SessionStart hook closeout-longrun.py --drain was retrying four queued closeout files to Mini (unreachable, expired tailnet key), one TCP timeout per row.

Fixed: the drain preflights each host once (4 s cap) and leaves unreachable rows queued; the hook runs detached. Drain 60.1 s → 4.6 s, session floor 63 s → 6.5 s. Block B104 recorded. Nano never had the hook.

Where the tokens are

A headless run pays a harness floor before the agent card. The card is 1.6–6.5k; the floor was 7–25× that. Launch mode is the lever; card trims are second-order.

default launch

43,240

no user settings

29,768

lean (+no MCP, agent tools)

17,169

bare (+own system prompt)

8,798

Now live: bash ~/AI_OS/agents/spin-up.sh <ID> --lean "<task>" (or --bare). Proven from the installed tree: A13 md5 task in 10 s, 17,255 tokens, correct answer. CLAUDE.md rule: headless work runs lean by default.

Bench: same task, different launch × model

8 real fixtures (md5 proof, HTTP codes, registry rulings with evidence paths, packet index, Spanish translation, a script that must run) × sonnet/opus. Baseline = today's default launch on opus. Tokens = context re-sent across all turns, what you actually pay.

mode / model	pass	wall s	Δ	tokens	Δ	cost Δ
full / opus (baseline)	8/8	35.9	—	157k	—	—
full / sonnet	8/8	25.1	−30%	192k	+22%	−66%
lean / opus	8/8	16.7	−53%	51k	−67%	−42%
bare / opus	8/8	18.6	−48%	79k	−50%	−55%
bare / sonnet	8/8	23.4	−35%	83k	−47%	−88%
slim-lean / opus (trimmed cards)	8/8	15.3	−57%	95k	−40%	−49%
slim-bare / sonnet	8/8	19.2	−47%	74k	−53%	−86%
lean / sonnet · slim-lean / sonnet	7/8	26.7 · 16.9	−26% · −53%	79k · 109k	−50% · −30%	−79%

Loss check: full 16/16, bare 16/16, slim-bare 16/16, lean 15/16, slim-lean 15/16. The single lean miss is one STALE verdict returned with no evidence path where the check demands one; the same row passes in full, bare and lean/opus, with old and new cards alike. Fixture strictness, not lost context. Pre-including the Always set in a full-floor prompt ("card" mode) did not help: +18% tokens, the floor dominates.

Sonnet vs opus: on mechanical fixtures sonnet passes everything at a fifth of the cost. On judgment (49 registry rulings) sonnet's first pass held 35 of 49 (71%) under opus adversarial review; the overturns were mostly "STALE" calls on topics still ACTIVE or PARKED in lanes.json. Rule that falls out: mechanical rows → sonnet lean; rulings → sonnet draft + opus verify, or opus direct.

Cards trimmed

Fleet card + Always set 70.7k → 54.3k tokens (−24%); median agent 3.5k → 2.65k; 11 of 19 now under the 3k target (was 8); in-session copy prompts 9.7k → 3.2k.

How: process boilerplate, Footprint and spin-up lines left the cards. A2/A4/A5 load one client block via agents/client-section.sh <slug> instead of all of CLIENTS.md. A9/A11 load two 200-token refs instead of all of CLAUDE.md. A0 loads a 1k role sheet instead of ROSTER (icm/agents.json turned out to be 11.4k, not compact). Still heavy and untouched: Ak DESIGN-INDEX 4.2k, A7 PERMISSION-TIERS 2.4k, A9 icm/v2 README 2.5k.

Rollback: gen-agents.mjs.pre-slim, REGISTRY.json.pre-slim, cards.pre-slim.tgz, ~/.claude/agents.pre-slim.

What "dozens of pending blocks" was, and what happened

pool	before	after	how
registry rows status BLOCKED	52	11	49 packet rows ruled by 5 sonnet agents; every RESOLVED / STALE / NEEDS_SEAN verdict re-read by an opus verifier (14 overturned); rulings applied through a new overlay that survives rebuilds. 11 RESOLVED · 21 STALE · 9 NEEDS-SEAN · 11 STILL_BLOCKED with the action named. Registry open 983 → 951.
the 3 "blocked" lanes	3	0 real	booking-mailer: not broken — token and calendar fetch return 200; the calendar has zero bookings; "fetch failed" is <1% transient noise (109 of ~12,000 cycles). sync-conflict-cleanup: the 18-file list died with Nano's removed home; 8 of 18 recovered, 7 keep-M2, 1 merge needs you. m2-snapshot: superseded — writer retired 05-16, snapshots/ retired 09-09.
task cards blocked on you	4 (+5)	9 questions	one INTERVIEW table, recommended answers first; 1 was already decided (Pocket Control PIN).

Pattern worth keeping: a "BLOCKED:" line in an old packet is not a live block — 32 of 52 were history. The registry now records verdicts (icm/v2/10-REGISTRY/rulings.json) instead of re-deriving them every rebuild.

Bugs found on the way

The staged headless runner had never completed a real run — run-as-agent.sh split its JSON summary on spaces and died every time. Fixed while wiring --lean.
--setting-sources "" also drops the tools allowlist, so headless Bash blocks on a prompt nobody can answer. The wiring now writes a 90-byte per-agent settings file granting exactly the agent's tools.
Two broken MCP servers in the home-directory project scope (claude-peers 9.2 s cold and exits 1; gws 7.8 s npx cold start) and two dead paths in workspace/.mcp.json. Any session started from ~ pays them. Report-only, nothing touched.
Bench harness: unresolved output dir broke system-prompt paths; greedy JSON extractor. Both fixed before the matrix ran.

Your part

NEEDS-YOU.md in the packet — 9 one-word answers: keychain service name (A migrate / B keep), relay Anthropic-API switch (y/n), Gmail marketing sweep scope, secrets storage (age vs plain), Motion transcripts lost?, and five "greenlight now?" items. Plus 9 registry rows now marked NEEDS-SEAN, mostly GO on staged things: experior domain remap, cesar restyle flavor, draft-badge removal, Mixpost reinstall-or-retire, proraider lab dependency, booking-mailer From-address brand mismatch.

Optional GO (report-only so far): remove or repair the 4 broken MCP entries; disable 3 unused plugins (claude-md-management, code-review, feature-dev — est. up to 14k tokens per session); merge the duplicated path-guard PreToolUse entry.

Not done: Mini sync (unreachable all session); DESIGN-INDEX / PERMISSION-TIERS / icm README trims; the 11 STILL_BLOCKED actions are routed to agents, not started (goguru extraction 0/40, CF zone-create token, access passphrase rollout, Clark CA forms, DDR field map, Codex review owed…).

Installed on M2, synced to Nano (md5-verified)

what	where	rollback
hook fix	`ops/scripts/closeout-longrun.py`, `~/.claude/settings.json`	.bak-20260912
lean/bare spin-up	`agents/spin-up.sh`, `ops/scripts/run-as-agent.sh`, `agents/spin-up-bundle.mjs`, SPIN-UP.md §2.5	.bak-20260912
rulings overlay	`icm/v2/10-REGISTRY/build-commitments.mjs` + rulings.json (52) + rebuilt commitments.json	.bak-20260912
slim cards	gen-agents.mjs, REGISTRY (6 Always edits), all cards, aios-* copies, agents/refs, client-section.sh	*.pre-slim
self-test harness	`agents/selftest/` (bench.mjs, README, baseline, hook audit)	delete dir
rules + memory	CLAUDE.md token-metric line; memory reference_lean_spinup_and_session_floor; block B104	—

Full packet with evidence: ~/AI_OS/comms/outputs/2026-09-13-selftest-speed-tokens/ — REPORT.md, bench/ (96 runs), cuts/, rulings/ (batches, verified, final), lanes/, NEEDS-YOU.md, HOOK-AUDIT.md, SAVE-STATE.md. Re-run any time: node ~/AI_OS/agents/selftest/bench.mjs.