ADR-002: A Toolchain That Wraps Cargo, and Metaprogramming Without Proc-Macros

Status: Accepted; D6 (codegen units are the project’s, the CPU level the build command’s) added October 20, 2026 (#692) Supersedes in part: ADR-014 D1’s when — std’s Nikaia half is lowered at release time, not when the crate is built (D4) Related: ADR-021 (works out D3, and whose key D4 extends), ADR-310 (what a build may read), ADR-177 and ADR-287 (what runs while it is built), ADR-296 (the multi-language goal), ADR-014 D1 (the .nika half of std D4 pre-lowers), ADR-312 D10 (the Shared that would break D4’s constraint), ADR-005 D8 (why the ledger travels with the compiler)

1. Context

Nikaia is one language on a toolchain meant to host several. Two things had to be true before any of them: a Nikaia project must consume crates.io without reinventing dependency resolution, and compile-time metaprogramming must not cost what Rust’s proc-macros cost — a separate dynamic library compiled before the program that uses it.

2. Decisions

D1 — One CLI wraps and instruments Cargo.

nikaia.toml is translated to a compliant Cargo.toml, native Rust dependencies passing through unchanged. Compilation is intercepted with RUSTC_WORKSPACE_WRAPPER: Rust crates go to the native rustc, .nika files to the Nikaia pipeline. The build switches of ADR-037 set opt-level and the panic strategy from there.

Dependency resolution and linking are solved problems, and a language that cannot reach crates.io starts with no ecosystem.

The interception is RUSTC_WORKSPACE_WRAPPER because Cargo applies it to workspace members only: a crate from crates.io is compiled by the real rustc with nothing in between, and a command line without a .nika file in it is passed through untouched. The generated manifest is not in the build cache’s key — opt-level, lto and a dependency version change the machine code, not the lowering, and Cargo’s own fingerprinting covers that half (ADR-021 D1). The resolved versions are recorded in nikaia.lock and not keyed (ADR-021 §5). A Nikaia dependency is refused (§5).

D2 — Compile-time code runs in an interpreter, not as a compiled plugin.

Changed by ADR-177 D1: a grammar that runs while the program is built is compiled from the parser it generates, not interpreted; the program’s own build-time bodies are still interpreted in the compiler’s process (ADR-287 D18).

Quasi-quoting, dsl … { … } validation and reading a file at build time are evaluated by an AST walker in the compiler’s own process.

A proc-macro is a crate that must be built, linked and loaded before the crate that uses it, which is where Rust’s build latency comes from. An interpreter has none of that, and the compiler already holds the AST.

What a build may read is ADR-310; what may run while it is built is ADR-287.

D3 — A build that reads the world needs a deterministic key over what it read.

Every external file a compile-time evaluation touches is recorded by content hash in a lockfile, and an unchanged hash skips the evaluation.

ADR-021 is this decision worked out: what is hashed, what invalidates what, and where artifacts live. Where the two differ, ADR-021 wins.

D4 — A project reaches std through a sysroot: sources, pre-lowered, compiled into a keyed rlib cache.

nikaia-std is not published, so D1’s “Cargo resolves it” does not apply to it and never will until a registry is decided. What replaces the path to a checkout is a sysroot: a directory that travels with the compiler and holds std’s sources, with the Nikaia half already lowered to .rs and the .nika kept beside it. NIKAIA_SYSROOT names one; the default is the checkout the compiler was built from.

What decided it: while std lowered its own .nika files in a build script (ADR-014 D1), it had the compiler as a build-dependency, so a hello-world’s lockfile held 97 packages, 58 of them only to build nikaia-std — Cargo compiled the compiler again inside every project’s target/.

std ships as sources, with the .nika half pre-lowered, and the .nika kept. A binary install compiles the .rs and needs no compiler to do it. A from-source install may re-lower, by invoking the installed nikaia binary (nikaia lower-std) and never by linking the compiler as a library — that linkage is the defect. One artifact serves both installation styles; the alternative is two release pipelines that can disagree.

Built locally into an rlib cache, not shipped pre-built per platform. The Rust toolchain an installation needs is about 600 MiB (ADR-001 D1) against about 2 MB for nikaia-std’s rlib, so installation size cannot decide this; and building the compiler from source is about 40 s, so shipping pre-built artifacts per platform buys little while costing a matrix.

The cache is keyed with the toolchain, as ADR-021’s key is, plus the target and the codegen flags. Key::sysroot is the one place those dimensions are named, beside Key::build, for ADR-021 D7’s reason. The codegen flags are in because hardware optimisation is the one axis where std matters more than the compiler: std’s code runs inside every user program, where the compiler’s own speed only costs build time. So -C target-cpu=native is a dimension — a second entry that coexists — rather than a reason to rebuild the first one, which is ADR-021 §4’s “dimensions coexist; they do not share” applied one layer up.

ADR-021 D7’s “not too much, not too little” keeps three plausible dimensions out. user_parallelism is out because the switch reaches std as a runtime value — UserCode::Sequential or ::Concurrent handed to rt::start in the fn main the emitter writes — and never as a cfg or a feature. The one switch that does reach std as a feature is reentrancy-check, which is a dimension of the compiled tree for that reason (ADR-281 D37); its only other cfg is target_os. So there is one compiled std per machine and re-entrancy setting, not one per user_parallelism. The toolchain’s own package in src/tools is lowered at yes and outside that rule (Changed by ADR-363 D1). The lowering backend is out because it decides what a .nika file lowers to, and std’s Rust is not lowered by a project’s build at all. And the content hash of the std tree is out: the key names a directory, and inside it Cargo’s own fingerprinting already decides what is stale (ADR-021 D1). Hashing the tree as well would start a fresh directory, and rebuild everything, for every edit to std — to guard against a staleness that cannot occur.

The compiler’s own fingerprint is out too, for the same reason, although ADR-021 D3 puts it in the unit key. std is compiled from its committed .rs, so the compiler writes no byte of it, and with the fingerprint in, every edit to the compiler started a fresh directory and compiled std and everything under it again. What the compiler does write in that directory is the program’s own crates, lowered by the rustc wrapper, which Cargo knows by its path and not by what it writes. So the fingerprint is a line of the switches file instead, the file the wrapper already puts into every lowered crate’s dependency file (ADR-021 D5): another compiler makes those crates stale, and nothing else. Rejected: keeping it in the key, which pays a full rebuild of what did not change to make stale the one crate that did.

The compiled std lives in the user’s cache directory and not under target/, because being shared between projects is what it is for. Two prices follow: rm -rf target does not give a cold build, and two builds sharing one entry serialise on Cargo’s lock on that directory.

NIKAIA_SYSROOT replaces NIKAIA_STD_PATH. The old name said crate directory, and a crate directory can answer only one of the questions this mechanism has to answer. A sysroot is a directory with nikaia-std/ in it, which is why crates/ is one and the in-tree flow needs no special case.

The constraint this creates

std’s Nikaia half is lowered at one setting of the build switches — Build::default(), which is target = x86_64-linux, user_parallelism = no — whatever the consuming program is built at:

std’s Nikaia half is lowered once, at one setting. This is sound only while nothing switch-sensitive appears in it.

Shared is what would break it: its count is plain at user_parallelism = no and atomic at yes (ADR-312 D10), so a Shared in a .nika file here would be lowered with a plain count and linked into a program built at yes.

It is checked, not merely written down: every .nika file in std is lowered at both settings and the bytes have to agree, so the day something switch-sensitive arrives it is a red build instead of a miscompilation.

This holds until std needs something switch-sensitive, and then it is decided anew. The test going red is the signal; the choice then (lower std per setting, or lower it always at the atomic floor, or keep the code out of std) is not made in advance.

Committed, not generated by a release step

The pre-lowered .rs is committed, beside the .nika. Committing puts a generated file in a reviewer’s diffs; generating adds a release step that must not be forgotten. What breaks the tie is that cargo test --workspace must work from a fresh checkout with no extra steps: at that moment a generated .rs does not exist, and the only mechanism that could produce it in time is the build-dependency on the compiler this decision removes.

The file is made unable to lie: a test regenerates it with the compiler and fails if the committed bytes differ, naming the command that fixes it — the same standard ADR-005 D8 sets for the ledger.

Where the ledger lives

std.contracts stays inside the compiler, baked in with include_str!, and the compiler answers from that copy and never from the sysroot. ADR-005 D8 makes ledger stability across toolchain versions explicitly not required, so a std that could be paired with a different compiler would let the ledger describe a compiler that is not there. The file in the sysroot is the source it was built from — a reviewer reads it there — and a test requires the two to be equal, so they cannot drift.

D5 — The one profile is written whole: no debug assertions, and incremental is the project’s to switch.

A build has one Cargo profile, dev, because Nikaia has no dev/release division, not because a build is a debug build. Everything that profile means for the machine code is therefore written into the generated manifest rather than inherited from Cargo’s defaults for dev:

Replaced by D6: codegen-units stays Cargo’s: with the two above settled, 256 units against 16 or 1 moved 1BRC by one or two instructions a row. Nothing here changes what a program means, so none of it enters the lockfile (ADR-021 D5).

D6 — The unit count is a key of the project; the CPU level is an option of the build command.

[build.x86_64-linux]
opt-level = 3
codegen-units = 1
nikaia build --cpu=x86-64-v3

codegen-units is a key of [build.<target>], beside opt-level, lto, incremental and debug: a whole number from 1, the number of pieces rustc cuts each crate into. Absent, rustc chooses, as it does today: 16 where incremental is off, 256 where it is on.

The CPU level is chosen when the compiler is called, never in nikaia.toml. --cpu=LEVEL on build, run and test names the least CPU the program will run on, and the build may use every instruction that level has, in the program and in std. On x86-64 the levels are x86-64 (the default), x86-64-v2, x86-64-v3 and x86-64-v4. Every target takes native, the CPU of the machine that builds. A target without named levels takes only its baseline and native. A level the target does not have fails the build, and the message names the ones it has. The level is a dimension of the compiled std’s cache key (D4); for native the key holds the CPU that was found, not the word. It is the level by which ADR-371 D2 allows or refuses an instruction of one architecture. Where the user’s environment also sets rustflags (RUSTFLAGS, CARGO_ENCODED_RUSTFLAGS), they apply and --cpu comes after them and wins: the level named at the call is never dropped silently.

Why. On 1BRC at 0.0.1130 one unit against 16 moved examples/1brc.nika from 231.2 to 273.9 instructions a row and the tuned Rust program from 312.4 to 270.9: which functions the inliner sees together decides it, in either direction, so no default is right for every program and a project has to be able to try. It changes nothing a program means, as lto does not. The CPU level is different in kind: one source is shipped as several binaries for several machines, and which machine a binary is for is a property of that build, not of the project; a level in the manifest would make every build of the project one that older machines cannot run. --target is chosen at the same place, for the same reason. native is allowed because the person building chooses it for their own machine; the cache keeps its result apart from every named level.

Rejected: a cpu key in [build.<target>], which fixes per project what is chosen per binary; a default unit count of 16 written into the profile, which rustc already chooses where incremental is off and which is wrong where it is on and the project asked for speed of rebuilding; RUSTFLAGS as the way, which reaches past the manifest and is not part of the cache key.

Decided October 20, 2026 (#692).

3. Consequences

Rust’s own guarantees are inherited rather than rebuilt — the borrow checker verifies the lowered program, which is what lets Nikaia promise memory and thread safety without implementing either.

The price is that a target language has to fit Rust’s ownership model. A language that assumes pervasive garbage collection does not.

5. A Nikaia dependency

Part III 13.3 shows http-server = "1.2" beside a Rust crate, and nothing decides what resolving it would mean — no registry, no name space, no distribution format. So D1 refuses it, with that reason, rather than handing it to Cargo, which would look for it on crates.io and fail somewhere that explains nothing.

This site is open source. Improve this page.