Status: Accepted Related: ADR-002 D3 (the germ of this), ADR-003 D2 (the orchestrator owns caching), ADR-001 D1 (the toolchain D2 records), ADR-005 D3 and D8 (the ledger D10 keeps separate), ADR-020 (the ledger as its own file)
The specification promised fast builds in three places and never said how.
nikaia.lock was a “deterministic cache key” for compile-time I/O; the ledger
was said to double as an incremental cache key; the orchestrator was where
“hash-based incremental compilation checks” happened. Three descriptions of
caching, no statement of what is hashed, what invalidates what, or where
anything is stored.
A cache key is the one thing that cannot be widened later without invalidating everything built under it, and both ways of getting a key wrong are silent.
An external compilation cache is the obvious alternative, and it is not taken —
not because such caches work badly: they cache the wrong
layer. They memoise rustc invocations, while the expensive repeated work in a
Nikaia build is macro evaluation, grammar expansion and lowering — all of which
happen before rustc is reached, and none of which rustc ever sees.
No second caching system to configure and keep in step. ADR-003 D2 already makes the orchestrator the owner; this makes that ownership exclusive.
Cargo’s own fingerprinting continues to cover the Rust workspace and is not replaced or wrapped.
nikaia.lock holds, per entry: the SHA256 of every external file read at
compile time, the SHA256 of every .nika source that took part, resolved
dependency versions, the exact toolchain used, and the compiler’s own
identity (D3).
Splitting the record — assets here, toolchain there — produces a lockfile that cannot answer the only question a lockfile exists for: does this build the same thing for you as for me? Leave out an asset or the toolchain and the answer is no, and the file does not know it.
Stated separately because it is the most dangerous omission available. The emitter is a pure function from source to Rust — which is what makes caching sound at all — but it is a function we edit. Same input, same toolchain, changed emitter, different output. Without it the cache serves artifacts from the previous emitter and the build is silently stale. The failure lands only on compiler developers, which is precisely why it would survive undetected.
“Version” has to mean a fingerprint, not a version string. The obvious
reading is CARGO_PKG_VERSION, and it is wrong: it stays 0.1.0 across every
edit to the emitter, so the key would sit still exactly while the thing it
identifies moved. What goes in is a SHA256 over the sources that decide the
output — the compiler’s own src/, and the std ledger it bakes in. A version
string is a claim about the compiler; a fingerprint is a measurement of it,
which is the same distinction D4 draws between a manifest and a lock.
nikaia.toml declares what the project requires; nikaia.lock records what was
resolved and used. A constraint and a measurement can coexist; two constraints
cannot — which is what resolves the drift that would otherwise appear between
rust-toolchain.toml and the lock.
The toolchain field in the ledger is not part of that drift. Per D10 the
ledger travels with a published package while the lock does not, so it arrives
downstream alone and must describe itself; otherwise a consumer cannot tell a
contract diff caused by a compiler upgrade from one caused by their own code.
Duplication a file needs in order to be readable on its own is not drift.
Changed by ADR-292 D7: there is no ordering switch, so it is not a dimension; the two build switches that remain are.
The two build switches, opt-level and backend are decisions made at invocation time, not facts about the inputs. Hashed into the key, never persisted — a lockfile that records them is rewritten whenever a developer switches either, producing a committed diff carrying no information and conflicts between developers who build differently.
Both must be in the key. Either setting lowers the same source to different Rust, so a key without them hands back one build’s artifact for the other — and the test that exists to catch exactly that divergence would compare an artifact against itself and pass trivially. A cache is allowed to make a test faster. It is never allowed to make a test vacuous.
The key for a unit is the hash of that unit’s inputs, not the hash of
nikaia.lock as a whole. Hashing the whole file is the easy implementation and
destroys the property the cache exists for: one changed asset would invalidate
every unit. Smart invalidation is not a later optimisation — it is a consequence
of choosing the granularity correctly the first time.
A key can be wrong in two directions, and both are silent:
So the key is assembled from an explicit, reviewable list. Nothing is captured incidentally, and nothing that determines the output is left to be remembered.
For a git dependency, hash content, not the revision: Cargo fingerprints such a dependency on its commit, so local edits to the checkout are silently inert. A bug to learn from, not to reproduce.
target/nikaia/cache/, addressed by the D6 key, git-ignored. The lockfile
records inputs; the store holds outputs. Because the key is derivable from the
lockfile plus the D5 choices, the store needs no index of its own — and the
absence of a second bookkeeping structure is the point, since an index that can
disagree with the lockfile is a cache that can be wrong.
The precondition, determinism, already holds: it is why --explain rebuilds the
source map from source rather than storing it (ADR-300 D2).
Measured on this workspace, Cranelift is fully compatible and buys about one
second on an incremental rebuild and nothing on a full build: at opt-level = 0
the cost is the dependency graph and the proc macros, not codegen. Defaulting to
it would oblige every contributor and CI job to install a toolchain component
these numbers cannot justify.
It is not switchable from this repository: the switch is [profile.dev]
codegen-backend, which Cargo has not stabilised, so turning it on costs a
nightly toolchain Nikaia does not use (ADR-001 D1). Revisit when a
single crate becomes large enough for codegen to dominate its rebuild.
A backend nothing implements is refused with an error naming this decision, never accepted and quietly replaced by the default. The message offers no remedy, because there is none: a refusal that named something to install would send the reader after a build that does not exist.
nikaia.contracts does not merge into nikaia.lock. Two properties make that
impossible rather than merely inadvisable:
--locked would mean two contradictory things. For the ledger it means
regenerate and compare byte for byte, sound because the ledger is a pure
function of (source tree, toolchain). For a lockfile it means the reverse,
do not re-resolve, and it must: resolved dependency versions are not a
function of the source tree at all. That is the entire reason lockfiles exist.The shared phrase “cache key” is retired here, because it names two different mechanisms:
| consulted | question it answers | |
|---|---|---|
nikaia.lock |
before any work | Do I need to start at all? — a skip decision |
nikaia.contracts |
after inference | Which callers must be re-checked? — a blast radius |
The lock can be read and answered without compiling anything. The ledger cannot: it is compared against contracts that exist only once inference has run. They are complementary stages of one build, not two implementations of one idea.
Reusing an unchanged lowering is the difference in feel between a Nikaia build
and a Rust one, and a default nobody types is not that difference. --no-cache
is the way out, kept for debugging the emitter.
Being the default turns where files go into a requirement: nothing may be written into a directory the user did not ask us to write into.
nikaia.lock at the root, committed; the store in
target/nikaia/cache/. A unit is named by its path below the root, never
absolutely, since D7’s first failure direction is a key that moves with the
checkout.A read-only checkout, a full disk, a CI sandbox, a corrupt lockfile: each must cost the next build its speed and leave this one working. A cache that turns a succeeding build into a failing one has taken something the user had, and no amount of speed pays for that.
A store that cannot be opened is reported on stderr and the build proceeds; a
lookup that cannot read is a miss rather than an error. The library still
returns Result — it should be able to say what went wrong — and the caller
decides the answer is not fatal.
D7 says dimensions coexist; they do not share, and the compiled-std tree is
where that costs real disk: it is a whole Cargo target directory, and every new
toolchain, target or codegen table starts another one. Unbounded, that reached
gigabytes and failed builds for want of disk: a cache with no eviction is a
disk leak — coexisting is right, coexisting forever is the defect.
The bound is the cache’s size, and only the last use orders what goes.
Every kind of entry - compiled-std trees, bundles, parsers, files run
on their own - is held to one limit together, 8 GiB unless NIKAIA_CACHE_LIMIT
says otherwise, because the disk is what runs out, not a count of trees. Over
the limit, the entry used longest ago goes first, however young it is; a
month-old entry that is still used stays. Rejected: a count per kind (eight
small parsers and eight 240 MB trees are not the same cost, and a count did not
stop 15 GB) and an age limit (it removes what a project still uses after a quiet
month, and keeps a day of churn in full).
The last use is a stamp the compiler writes into the entry, not the file
system’s access time, which most mounts do not keep. An entry a running build
holds is never removed: each user takes a shared flock on the entry and keeps
it until it exits, and Cargo holds its own lock on a target directory while it
builds; the pass takes the entry’s lock exclusively before removing it, and skips
what it cannot lock. That includes the entry the pruning build itself just used.
The pass runs when a build has added an entry, after Cargo is done with it, so
the new entry’s size is counted; a build that only reused the cache does not walk
it.
A program leaves the tree with its project. Every project builds its program
in the shared tree, under a crate name carrying the project’s tag, and the
binaries are tens of megabytes each. A build records which directory its tag
belongs to and removes the artifacts of every tag whose directory is gone: a
project that no longer exists is not building, so nothing is taken from a build in
flight. Rejected: one tree per project, which is the per-project std this tree
exists to avoid.
Ordered by a marker file this writes, not by the directory’s own timestamp: Cargo writes into subdirectories, so a tree used every day can carry the mtime of the day it was made. And a sweep that fails is D12’s rule again — a fuller disk, never a failed build.
A hit serves the whole build: nothing is parsed, sync-checked, lowered or
inferred. Sound for exactly one reason — only a build that passed every check
is ever recorded. A failing check returns before anything reaches the store,
so an entry’s existence is the statement that this input passed. Skipping the
check on a hit is reading its memoised result.
The price is D7 applied strictly: every input those checks consult must be a dimension. If a future check reads something not on that list — an environment variable, a file found at run time — it becomes a dimension in the same commit that adds the check, or the cache starts approving inputs nobody checked.
Two things a hit does not skip:
--locked. Whether the committed ledger still matches is the user’s
question about the working tree, not the compiler’s about the source. A cached
build has as much to answer for there as a fresh one.Bought: one answer to “what determines this build”, in one reviewable file;
invalidation proportional to the change; and a cache over macro evaluation and
lowering — the work that is actually repeated — rather than only the rustc
call at the end.
Paid: disk, twice over. Dimensions coexist rather than share, so each combination of build-time choices occupies its own entries. That is the correct trade and it is stated rather than discovered: the cache is larger so that it is never wrong and never thrashes. D12.5 bounds the user cache.
Not decided: the lockfile’s concrete syntax, the hash framing, how the store
under target/nikaia/cache/ is collected. Those follow the implementation. The one structural question
decided rather than deferred is D10, because deferring it would have left the
orchestrator free to merge two mechanisms whose distribution and --locked
semantics contradict.
This workspace, dev profile.
Cargo keys its cache per dimension, and dimensions never evict one another —
the property D7 protects. Steady state costs nothing; a first cargo clippy
after a cargo build rebuilds 56 crates not as waste but as a parallel artifact
set. Dimensions coexist; they do not share.
Cranelift builds the whole workspace, passes the test suite and produces identical output, at 25 s against 26 s cold and 1 s against 2 s incremental (D9).
orchestrator::cache holds the key, the store and the lockfile; every dimension
of D7’s list is named in Key::build and nowhere else.
D2’s resolved dependency versions come from Cargo: a project build generates a
Cargo.toml (ADR-002 D1) and Cargo resolves a Cargo.lock beside
it under target/. Without carrying them across, nikaia.lock would answer
does this build the same thing for you as for me? only about the Nikaia half.
[dependencies] in nikaia.lock, the whole resolved graph
rather than the lines the author wrote, one entry per name holding a list of
versions. The whole graph because a transitive crate that resolved
differently on two machines is a different build; a list because two majors
of one crate in one graph is ordinary, and a record that keeps one of them is
not the whole answer. Read out of Cargo’s answer rather than computed, which
is D4’s distinction in the small: nikaia.toml declares version = "1.5"
and the lock records the 1.13.1 that turned into.Written after cargo returns, because that is when the versions exist, and
only for a build that succeeded. A failure to write it warns and leaves the
build alone (D12), and an unchanged record is not rewritten, so a committed
file does not turn up modified after a build that changed nothing about it.
What --locked would mean for the lock is fixed by D10 — do not re-resolve —
and it is not passed to Cargo; the flag is the ledger’s (ADR-005 D8).