Strategy. Correctness first, then self-hosting: writing the compiler in Nikaia tests the language for real and finds bugs; faster CI serves those two and counts only as far as it speeds them.
Formula. The next topic is the one with the highest
Priority = (strategic value + operational value) / total costover one horizon. Strategic value: self-hosting progress, compiler bugs found and fixed, fewer known open bugs, trust in the language. Operational value: tokens saved by faster CI, less manual translation, less debugging and fewer workarounds, shorter feedback loops, fewer regressions. Cost: tokens for implementation, tests and integration, added maintenance, and the self-hosting progress delayed meanwhile. Progress is counted in Rust lines replaced, not Nikaia tokens, since Nikaia code is shorter. Bugs first is the default; the score overrides it when the figures say so. The estimate per ticket are in docs/priorities.md.
Consulting agent. Proposals for how we work no longer come only from the owner or from outside advisers asked from experience: a consulting agent on the team reads our runs (transcripts, CI steps) and proposes measures, ranked. Each measure the architect takes is a row below, with its hope and its measurement kept apart.
We keep changing the way we work, and each change is a row below. The table says what we did, what we hoped for, and what was measured; where nobody counted, it says so.
| Iteration | Date | Action taken | Hope | Measured |
|---|---|---|---|---|
| 1 | 2026-10-11 | Self-hosting by long agents without a turn limit | Move Rust into Nikaia fast | Share 43.8 % to 49.8 % in 24 commits (0.0.787-0.0.810); 120-190k context tokens per moved line; agents of 695 and 768 turns ended at 566-626k context, 71-78 % of the cost after 300k |
| 2 | 2026-10-11 | Short Opus planning agents decide what builders were stuck on | Not recorded | 7-21 M tokens each against 219-276 M for a long builder; value counted in builders unblocked, not in lines |
| 3 | 2026-10-11 | One agent, one slice, restart at about 120 turns | Computed: about one third of the cost | 71k against 143k tokens per moved line (11 fresh builders, 205 M tokens, 2,903 net lines; two long agents, 495 M, 3,459 lines); target below 50k; share 54.3 % at 0.0.823. Two cohorts, not yet a trend |
| 4 | 2026-10-11 | rs2nika draft as the default start of a move |
Builder starts from a draft, not an empty file | About one turn saved per unit (C-5b); the twin file in the other module saved more |
| 5 | 2026-10-11 | Research brief (at most 80 lines) before the builder | Not recorded | Not measured |
| 6 | 2026-10-11 | Set-up of models: Sonnet medium and Opus medium tried; agents now Opus low, main session Sonnet 5.5 low with a 200,000-token window | Owner read that Opus low beats Sonnet medium; the window keeps the session under 300k | Not measured; no model or effort choice was benchmarked |
| 7 | 2026-10-17 | Ten build agents measured | Find where the tokens go | 52k tokens at start, 35-40k of it descriptions of 41 unused tools; growth about 1k per turn; 7 of 10 ran past 200 turns; 10-20 % of cost spent polling |
| 8 | 2026-10-17 | builder agent type with six tools; waiting in one call |
Start context from 52k to about 15k; no polling cost | Start context about 15k (the size CLAUDE.md gives); Not re-measured on a run |
| 9 | 2026-10-21 | --profile fast (#724): release optimisation, incremental |
Shorter build after a one-file change | Stated in CLAUDE.md: about 18 s instead of 70 s for the binary; no count taken |
| 10 | 2026-10-24 | Rule: count, never time; callgrind for performance | Figures that survive a change of machine | The same binary moved 1.4-1.9 times between days; counts do not |
| 11 | 2026-10-24 | Architect role (Opus, few tools, GitHub issues), main session as carrier, stage 1: one builder at a time | Not recorded (docs/architect.md, 97385b53, states the job: decide what is built next, keep issues current) |
First run row: 38 turns, 1.2 M tokens, no line moved. Not enough to judge |
| 12 | 2026-10-24 | Per-file cache keys for the lowering (#667) | No stale or needless re-lowering | Built; a review found a stale-hit risk in the first form and filed it as #801 before harm |
| 13 | 2026-10-24 | lower-std: one Rust module per .nika (0.0.1207) |
A one-file change recompiles one file | 1 of 256 objects changed on a one-file change; was one 7.4 MB unit |
| 14 | 2026-10-24 | Test binaries 267 into one (--test it) |
Fewer links and less disk per change | 267 binaries of 16-20 MB become 1; saving in links read from the tree, disk not re-measured |
| 15 | 2026-10-24 | CI: tests built in a step of their own (#795) | Compiling separated from running | Not measured; one slip (YAML error) fixed in 8d56f29c |
| 16 | 2026-10-24 | Reading three CI runs (docs/ci-cache.md) |
Decide whether a CI cache pays | Up to 40 % of a run is queue wait; Test is 68-83 % of a job; the owner parked the cache plan |
| 17 | 2026-10-24 | Long jobs: CI is the verify service (docs/long-jobs.md) |
Builders stop running the long jobs | Touched tests about 10 min, full local suite about 45 min before the single binary; no counts |
| 18 | 2026-10-24 | Some(x) and None over a T? refused (ADR-352 D5, NK2704, NK2705); the tools write x and null |
ADR-352 D5: a T? has no constructors (ADR-278 D5), so a pattern names null and the value itself |
36 Some(x) arms in the tools rewritten (3573f202); sequences ceiling 17 to 16 |
| 19 | 2026-10-25 (57a13199) | Architect chooses the topic and writes one brief per builder to a file; the supervisor starts the builder with only the path | Not recorded | First runs: architect topic and brief 37,681 tokens, 17 tool uses; #824 builder 46,492 tokens, 32 tool uses; table-rewrite builder 35,050 tokens, 11 tool uses. No comparison row yet |
| 20 | 2026-10-24 (#826 filed); 2026-10-25 brief rule (57a13199) | Disk: #826 decides to bound the Nikaia cache and delete clones when an agent ends; until steps 1-2 are built only the brief rule applies (NIKAIA_CACHE_LIMIT=2G, own target/ deleted, at most two clones with target/) |
#826: the disk ran out several times; bound the cache and delete clones instead of a shared target dir | Before (#826, 2026-10-24): 4.6 GB free; Nikaia cache 6.0 GB, 38,125 lower-std entries; one builder clone 7.8 GB; old clones 5.5 GB. After: not measured |
| 21 | 2026-10-25 (named in 57a13199; no run recorded) | Pilot named: QA agent (Sonnet low, runs the suite like CI); no separate review agent unless asked | Not recorded | No run recorded |
| 22 | 2026-10-25 | Consulting agent on the team: reads 30 agent transcripts and the CI steps, proposes ten ranked measures (consulting-speed note), passed to the architect one at a time |
Proposals from our own data, not only from experience | Its own run: 120,645 tokens, 22 tool uses (harness); the measures’ effects are counted in their own rows |
| 23 | 2026-10-25 | Consultant’s measure 1: one canonical lower-std / build / lower-std cycle as one tool call in builder.md; no resetting lowered/, no clone, no polling |
1-4 fewer build cycles for builders that looped (#757, #792, #824); fewer turns | Before, from the transcripts: 2-10 cycles per builder (#792: 10, #757: 7), three resets of lowered/. After: to be counted per builder (cycles = cargo build calls, tool uses and tokens from the harness) over the next five bug builders. Counted since: one cargo build in each of #830, #831, #832 (before 2-10 per builder); no reset of lowered/ |
| 24 | 2026-10-25 | Consultant’s measure 2: keep target/nikaia-project-tests (the std build of the emitted programs, 283 MB) between test runs; delete it only above 1 GB or under 3 GB free disk (CLAUDE.md). (correction, same day: the directory is the tests’ Nikaia cache, NIKAIA_CACHE_DIR, and the compiler’s own least-recently-used prune keeps it to NIKAIA_CACHE_LIMIT; the size check is dropped, only the free-disk guard stays) |
Each test call reuses the std build instead of compiling it cold; fewer turns lost to it |
Four bug builders after the change (#762, #764, #765, #758). Compiling lines in a touched-test call: 1 in each of the three that use the directory (only the nikaia crate; #762’s tests do not use it). Size 256, 485, 497 MB, so it levels off below the 1 GB guard; free disk 14-15 GB throughout. Tokens per builder 45k, 28k, 41k, 74k against 42-75k before: no change measurable in tokens. Kept. Then #831 grew it from 497 MB to 1.2 GB in one builder, and the builder did not delete it: the rule was prose. Now the limit is 2 GB (15 GB free, the 3 GB-free guard stays) and the check is one command in builder.md that every builder runs at the end. Then #776 left it at 1.9 GB: it grows by 0.5-0.7 GB per builder that runs project tests, probably because each compiler change adds a new set of artefacts and Cargo removes none, which the guard would have reset; measured in the directory, it sits at 1.9 GB because the compiler’s prune holds it under the 2 GB limit, entry by entry (two compiled-std trees of 230-240 MB, one per setting; build-time grammar targets 563 and 396 MB, 132 MB of it incremental; 51 small programs, 179 MB) |
| 25 | 2026-10-25 | Consultant’s measure 3: clippy runs beside the last touched-test run, in one tool call (builder.md); they build in different target directories |
One tool call and turn fewer per builder; idle cores in the tests’ tail used | Three bug builders after it (#830, #831, #832): clippy ran beside the tests in all three, so it was no extra call. Tool uses 30, 47, 28, tokens 41k, 51k, 38k, against 26-64 calls and 42-75k before: inside the earlier spread, so no gain can be shown in tokens. Kept: it costs nothing |
| 26 | 2026-10-25 | Consultant’s measure 4 (the owner’s #821 for new tests): a new program-output or refusal test is a tests/compiler/tests/*.nika + .stdout/.stderr, run by the built binary; it/*.rs only for API or lowered-text checks (builder.md) |
No it crate recompile for a data-only test; more tests already in Nikaia |
The same three builders: 5 new tests as .nika, none in it/*.rs; no it rebuild was needed for them (Compiling count not applicable). Kept. Effect on tokens not separable from measure 3 |
| 27 | 2026-10-25 | Consultant’s measure 5: no sysroot test among a builder’s touched tests unless sysroot.rs, described.contracts or a std .nika changed (builder.md); the cycle’s second lower-std with an empty git status stands in, CI runs it |
Two or three whole-toolchain lowerings fewer per builder that ran it (#760, #787, E10-4, #758, #765 did) | Not counted before. To count: builders running sysroot (from their Tests line), and CI runs red on sysroot after a builder skipped it; the next three builders |
| 28 | 2026-10-25 | Consultant’s measure 6: the accepted-programs sweep (#705) moves from bootstrap_compiler to arm64 when arm64 runs; it stays on x86 when arm64 is off (nikaia.yml). No caching change (parked 2026-10-24) |
The critical path loses the sweep, about 2 of about 12 minutes per run; arm64 ended about 2.5 minutes earlier | Before: the sweep step after Test in bootstrap_compiler in every run. To check over the next five runs with arm64: the sweep step’s place and result in each job, and which job ends last. Job lengths are times, so they are only used to see the order, not quoted as figures |
| 29 | 2026-10-25 | Consultant’s measure 7 (clippy as its own parallel CI job): not taken. Clippy is about 25 s of a 12-minute critical path; a job of its own needs its own build (clippy’s dev-profile artefacts, not in the release cache) on one more runner per run, and caching is parked (2026-10-24). Builders already run clippy before every push (CLAUDE.md, measure 3) |
A lint red about 1.5 minutes sooner, 25 s off the path | Not built, nothing to measure. Revisit if CI runs turn red on clippy (count them) |
| 30 | 2026-10-25 | Consultant’s measure 8, owner’s decision (option C): the architect writes two or three briefs for independent bugs in different files; the main session starts builder N+1 when N says done; the batch is cut on Found, a Deviation other than none or BLOCKED, and the rest goes back to the architect |
The architect’s share per fixed bug drops to about half (one round per two or three bugs) | Before: one architect round per bug (55-90k tokens, consultant). To count: tokens per fixed bug, architect plus builder (harness), over the next six bugs |
| 31 | 2026-10-25 | Owner’s decision on the test cache target/nikaia-project-tests (option A): incremental stays everywhere; no pruning inside an entry; no measuring job for now. The compiler’s own prune of whole entries (least recently used, to NIKAIA_CACHE_LIMIT) is the mechanism |
Fast rebuilds after a std change kept; the directory bounded |
Counted on disk (1.9 GB): incremental 620 MB, of which older sessions 11 MB; stale files in all (deps/ old versions about 18 MB, duplicate crate directories 6 MB, .fingerprint/ 9 MB) about 2 %; the only larger lever, incremental for one-off test programs in run/ (165 MB), not taken |
| 32 | 2026-10-25 | Owner’s decision: the builders’ cache limit rises from 2 to 4 GB (NIKAIA_CACHE_LIMIT=4G); the default for users stays 8 GiB (#826 closed on that point) |
Room for one more generation of cache keys, so a compiler change no longer pushes out an entry that is needed again at once | Before: the live set of the test cache was about 1.84 GB against the 2 GB limit (two std trees 460 MB, bundles 241 and 396 MB, parsers 563 MB, 51 programs 179 MB); a 563 MB entry whose key never changes was created anew that day, which fits an eviction but is not proved. To count: large entries newly created per builder (new report line Cache:) |
| 33 | 2026-10-25 | Consultant’s measure 9, owner’s decision (option B): the main session raises the version itself (scripts/bump.py new with the architect’s changelog lines), no agent of its own; the rule that raises are the supervisor’s stays |
One agent start fewer per raise | Before: an agent per raise (e731ba79: 23k tokens, 5 tool uses) and its own CI run. To count: agents per raise (should be 0) |
| 34 | 2026-10-25 | Consultant’s measure 10 (CI tests split over two runners with cargo-nextest): rejected by the owner. His direction: make the process itself faster and use all CPUs of the one runner provided, not more CI runs or shards. A push notification to the owner’s phone: parked | Not built | Nothing to measure. Follow-up: whether the CI test step already uses all of its runner’s CPUs |