Status: Superseded by ADR-303 — its D1-D7 are D1-D7 there
Related: ADR-289 (what an HTTP handler sees — this is the
“runtime binding” it waits on), ADR-297 (Cleanup parks with the
runtime, so D4 is what makes D3 of that record buildable),
ADR-292 (the wake-up cost that decided D3),
ADR-037 D2 (user_parallelism decides what D4 starts),
ADR-002 D1 (crates.io, which D7 is about),
ADR-296 (the grammar protocol D6 parses HTTP with)
Three records wait on “the runtime binding” and none of them says what it is.
tokio appears in a parenthesis in ADR-289 and as a checkbox
in the roadmap — chosen by repetition rather than decided, which is the shape of
dependency this project has spent a cleanup removing.
Two findings make the choice non-obvious and both are ours.
ADR-292: overlapping two operations costs a thread wake-up, about 46 µs, and that is an operating-system number no user-space scheduler removes. A runtime we wrote would not beat it.
But epoll cannot read a file at all. A regular file is always “ready” and
the read blocks in the kernel anyway, which is why every readiness-based runtime
serves file I/O from a thread pool. That pool is where the 46 µs comes from, and
io_uring removes it — not by being a faster pool, but because no thread is
involved: the kernel performs the read and reports when it is done.
(§5’s measurement sharpens this sentence, and the sharpening matters: a pre-started pool removes thread creation and still pays ~38 µs a pair for thread wake-up. Starting early is necessary and not sufficient; only having no thread at all makes a pair free.)
So the two halves of I/O have different answers, and the HTTP server sits on exactly the half where the finished libraries are.
hyper is not adopted.A finished server would have to be given I/O it can ask for data from, and
io_uring reports data instead. Bridging the two costs a copy through an
intermediate buffer, which spends most of what io_uring was for. Adopting one
therefore does not postpone the I/O decision — it makes it, for sockets, and
unmaking it later means replacing the HTTP implementation rather than the
plumbing under it.
The positive reason is larger. An HTTP request is a request line, then headers, then a body, delimited by fixed byte sequences: it is a grammar over bytes with a frame boundary, which is what ADR-296 exists for. And it is the one program that exercises ADR-010 (request bytes are Untrusted by construction, not by a special case) and ADR-017 (escaping is the template’s contract) on something real. A language whose HTTP server is written in it makes a different claim from a language with a binding to somebody else’s.
rustls is bound, as a Rust dependency.Writing TLS is years of security-critical work where a mistake is not slowness.
rustls is bound through ADR-002 D1’s passthrough like any other
Rust crate.
It constrains nothing here, and that is worth recording: rustls takes bytes
and produces bytes, and has no opinion about how they arrive. Unlike a finished
HTTP server, it does not decide D3.
Files go to io_uring where the machine has it, with the thread pool as the
fallback where it does not — an older kernel, a sandbox that forbids the
syscalls, wasm32 where neither exists. Sockets are readiness-based.
A regular file is the exception and everything else is the rule, which is
the better way round to state this: a pipe, a terminal and a socket all signal
readiness honestly, and a read on one blocks only while there is nothing there.
Only a regular file lies — it is always “ready” and the read blocks in the
kernel anyway. So standard input (ADR-019) belongs to the
readiness half with the sockets, however much it looks like std::fs’s shape
minus what a stream cannot keep.
This is where the measurement points: the cost ADR-292 measured is
a file cost, and the two reads of its own example are two files. What
io_uring adds for sockets is mainly batching syscalls, which is worth
double-digit percentages rather than an order of magnitude — and is not
something this project has measured.
Both live behind one std surface, for the reason
ADR-292 gave when it put the overlap behind
nikaia_std::task::both: the next change of mechanism must be a std change
and not a compiler change. (That surface is what lets
ADR-292 D6 lower a statement pair onto a completion pair without
the emitter knowing which mechanism will serve it — the choice is made at
startup, inside std, and a compiler could not have made it at all.)
main, and user_parallelism says what starts.An I/O thread that is already running costs nothing per operation; one started per pair of operations costs a wake-up per pair, which is the whole of §8.4’s finding. So it is running before the first statement the user wrote.
At user_parallelism = no that is all that starts: one I/O thread, and
everything the user wrote on the main thread. At yes a pool for user code
starts with it. The asymmetry is ADR-037 D2’s, restated where it
becomes a thread count: the compiler’s own threads were never bounded by that
switch, and the I/O thread is one of the compiler’s.
nikaia.toml keeps what the compiler must know — target and
user_parallelism (ADR-037 D5) — and nothing else. How many I/O
workers, how large the pool for user code, which I/O method, and how long
shutdown drains are read when the program starts, by the person running it on
the machine it runs on. A build-time key cannot be tuned by the operator, who is
not the person who compiled it.
Four settings and no fifth. The I/O method’s default is auto, and it can be
pinned — the same shape --ordering strict has: the default decides, and a
person may overrule it to rule something out in the field.
cleanup-deadline moves here from nikaia.toml, where Part III 13.3 put it.
How long a program waits at exit for pending cleanup
(ADR-297 D6) is an operating property, and it was in the wrong
file.
HTTP/1.1 is the grammar D1 is about. HTTP/2 is header compression, flow control and stream multiplexing — a different order of work, and the place where “production-ready” becomes expensive.
Part III’s std::http promises “a production-ready HTTP/1.1 and HTTP/2 server
and client” with no status note. Until HTTP/2 exists that sentence is the kind
of promise docs/README.md calls a defect, and it carries a status note from
here on.
And the risk is named rather than discovered. A fresh HTTP/1.1 parser’s
failure mode is request smuggling: a Content-Length and a Transfer-Encoding
that disagree, a header sent twice, a chunked body with an edge case — and two
servers in a chain then see two different requests. It is a test corpus and a
fuzzer written alongside the parser, not an audit afterwards.
ADR-002 D1 promises crates.io, so a program can depend on a crate
that starts threads and an event loop of its own — hyper among them. That is
allowed, and two things about it are decided rather than discovered:
A value may only cross into a foreign thread if it may cross any thread.
At user_parallelism = no, Shared is Rc (ADR-037 D3); a
foreign runtime that moves it to another thread would make that unsound. The
structural Send check ADR-037 §3 names is what decides it, and
it applies at both settings — which is why that section already says a library
is checked against the rules parallel code needs whatever it was built at.
That check is built, and ADR-005 §5 is what it does: a value this
rule refuses is refused here as NK2502, in the .nika file, at both
settings — user_parallelism bounds what the program runs at once, and a
foreign runtime’s threads are not the program’s.
The rule reaches only as far as the foreign crate is honest, and no frontend
check can change that. A Rust API that writes unsafe impl Send about a type
that is not puts the value on another thread with nothing complaining, and a
structural check would not have caught it either: where Nikaia can see the
value it is crossable by declaration, and only the callee’s body is wrong.
docs/history/foreign-runtime.md is the program that
demonstrates it — smuggled/ reads a plain reference count on a foreign
worker thread and builds without complaint. This is
ADR-292 D3’s “reached through unsafe” in a second place, and
what it means for this rule is that the honest statement of it is conditional:
a crossing is refused where the signature that carries it is true.
A Cleanup may not be owned by a foreign task. ADR-297 D5
parks a cancelled cleanup with our runtime, and a foreign runtime’s shutdown
will not drain that queue. A Cleanup handed across is a cleanup that may
never run, which is the one thing that record exists to prevent.
What needs no rule: the ledger knows nothing about a foreign crate’s effects, so
touches is absent, so it reaches everything and orders against everything
(ADR-292 D3). Fail-closed polarity means interop cannot silently
break the ordering guarantee — it only makes programs that use it slower.
Evidence — the experiment §4 asked for has run, and it changes nothing here.
A Nikaia program starts a hyper server and serves one request:
examples/foreign-runtime/, written up in
docs/history/foreign-runtime.md. Twenty-six crates
resolve from crates.io and none reaches the Nikaia pipeline, so
ADR-002 D1’s promise and this record hold at the same time. Three
things the experiment found that this record did not say:
hyper’s cannot be spelled — async, generics
over futures, trait bounds — so the shim is a narrowing of a generic Rust API
to i64 and String, which is also where the first rule’s soundness
actually lives: the signature that decides what crosses is hand-written Rust
no Nikaia analysis reads.Send check did not exist when the experiment ran, in this
compiler or anywhere in crates/. Nothing was unsound yet, for a reason that is
not the rule: with Shared unbuilt every value Nikaia can construct is Send,
and the emitted Rust contains no unsafe. What refused an illegal crossing was
rustc, reporting E0277 against a generated file — which Part III C.1 calls a
bug in this compiler. It is built now (ADR-005 §5): NK2501 and
NK2502 refuse what the frontend can decide, and the rest reaches the user
against the .nika line, because a trait bound is now among the backend classes
ADR-005 D7 has a translation for.unsafe impl Send about a type that is not puts an Rc
on another thread with nothing complaining, and a structural check in the
frontend would not have caught it either — the value is Send by declaration
where Nikaia can see it. That is ADR-292 D3’s “reached through
unsafe” in a second place.The third paragraph above was confirmed empirically and is asserted on every
cargo test. The second rule could not be run — ADR-297 is
unbuilt — and the argument for it is stronger than the reason given here: on the
normal path the inserted cleanup() is never emitted at all, and
ADR-297 D6’s drain cannot name what it did not park. The notes also
name a sound alternative to refusing the crossing that this record did not
consider.
Three records stop waiting. ADR-289 can be built once D4 and D6
exist; ADR-297 D5’s parked cleanup has a queue to park in;
ADR-292’s prediction — that the overlap is worth more at
user_parallelism = no, because a running I/O thread carries no per-pair
wake-up — becomes measurable rather than predicted. (Measured, and the
prediction’s reason was the wrong half: a running thread still costs a wake-up,
and what costs nothing is having no thread. ADR-292 D6.)
The dependency surface grows by rustls and, on Linux, an io_uring binding.
Neither reaches a .nika file: D3’s std surface is what a program sees.
D3, D4 and D5. The runtime foundation: files complete, sockets signal
readiness, both behind one std surface, and the whole of it is running before
the first statement the user wrote. D1, D2, D6 and D7 are still decisions and
not deliveries — see §4.5.
mainfn main belongs to the runtime. The emitter renames the program’s own entry
point to __nikaia_main and writes the one Rust runs:
fn main() -> Result<(), Box<dyn std::error::Error>> {
let nikaia_runtime = nikaia_std::rt::start(nikaia_std::rt::UserCode::Sequential);
let outcome = __nikaia_main();
nikaia_runtime.finish();
outcome
}Three things about that shape are decided rather than incidental. The wrapper
is generated code and not a link-time constructor, because finish is the
other half — ADR-297 D6’s drain has an outcome to report, and
an .init_array entry has nowhere to put one. UserCode is written in by the
compiler, because user_parallelism is a build switch and D5 has four
operating settings of which this is not one (ADR-037 D2). And
only the plain shapes — fn main() and fn main() throws, no parameters, no
returned value — are wrapped; anything else keeps its own name, because a
wrapper that guessed at a signature would be worse than no wrapper.
What starts is what ADR-037 D2 allows. At
user_parallelism = no: the I/O workers, and everything the user wrote on the
main thread. At yes: a pool for user code as well.
io_uring where the machine has it: the kernel performs
the read and one io_uring_enter collects it. fs::read,
fs::read_to_string, fs::write and fs::read_both go through it.epoll says when a descriptor
can be read or written; the caller does the transfer.io_uring runs on one without it: the probe
is IoUring::new answering ENOSYS on an older kernel or EPERM in a
sandbox that forbids the syscalls. The only compile-time gate is on the
bindings, which do not exist off Linux at all.fs::map is untouched. It is a mapping, it hands back memory it does not
own, and it never reaches the ring.The I/O thread is the compiler’s, and the code says so. A worker’s inbox is
a channel of Op, a closed enum private to nikaia_std::rt, and no variant of
it carries a closure or anything else a program could put its own code into.
Outside that module there is no way to name a value the channel accepts — so
ADR-037 D2’s promise that nothing the user wrote runs
concurrently at no is a property of the types rather than a convention review
has to keep.
ADR-292 predicted that the overlap carries no per-pair wake-up on a runtime that is already running, and said in the same sentence that there was nothing to time. There is now.
Two reads of the same two files, three vehicles, the same std, 20 000 pairs
per repeat, nine repeats, 4 vCPU. What each vehicle adds per pair over
reading the two files one after the other:
Read the signs and the ratios, not the microseconds. Re-running this same binary on the same box a day later put every figure below, the baseline included, 1.4–1.9× higher — a shared VM does not hold an absolute. What reproduced exactly is what the three columns are for: a floor in the first, a zero in the second, a flat tax in the third.
docs/history/runtime-cost.md§6 has both runs.
| added per pair | task::both( rayon::join) |
fs::read_both,completion |
fs::read_both,blocking fallback |
|---|---|---|---|
| 5 B/file | +59.2 µs (sd 1.4) | −0.3 µs (sd 0.7) | +37.9 µs (sd 0.9) |
| 4 KiB/file | +60.3 µs (sd 1.7) | −0.5 µs (sd 0.6) | +37.7 µs (sd 2.3) |
| 64 KiB/file | +67.3 µs (sd 4.6) | −0.6 µs (sd 1.2) | +37.3 µs (sd 1.1) |
| 256 KiB/file | +95.9 µs (sd 7.2) | −0.9 µs (sd 4.6) | −77.9 µs (sd 10.9) |
Three things come out of it, and the third was not predicted.
The first column reproduces §8.4. It is higher than that section’s 46 µs
because fixed measured a pair with nothing in it: here each closure also
performs a read, so the stolen closure’s wake-up sits on the critical path of
real work rather than beside it. Same floor, same shape, a little more of it.
The second column is D4 confirmed, and it is zero. Not “small” — the mean
is negative at every size up to a quarter megabyte and the spread straddles
zero. The kernel performs both reads; no thread is started and none is woken;
fs::read_both costs what one io_uring_enter costs, and the ring was already
there because the runtime started before main. ADR-292’s “none in
principle” is now “none, measured”.
The third column is the finding, and it is D3’s own argument arriving as a
number. The fallback does not deliver D4’s zero — it pays ~38 µs a pair. A
pre-started thread removes thread creation; it cannot remove thread
wake-up, and §8.4 already said the floor was the wake-up. So “the runtime
starts before main” is necessary and not sufficient: what makes a pair
free is that io_uring involves no thread at all. That is exactly why D3
splits I/O by kind rather than putting files on a pool, and it is the first
evidence for the split from this project rather than from the literature.
The fallback’s 38 µs is also flat — ~38 µs at 5 bytes, at 4 KiB and at 64 KiB — and then goes negative at 256 KiB, where the overlap starts paying for itself. That crossover lands where ADR-292 put it, near a quarter megabyte, which is what a fixed per-pair cost amortised against a growing payload has to look like. The completion path has no crossover to find, because it has nothing to amortise.
The method, the machine, every repeat, the load the box was under, and one
confound named rather than hidden are in
docs/history/runtime-cost.md; benches/overlap/runtime.sh
is the script that produced them.
nikaia-runtime.toml, read at startup, four keys and no fifth: io-workers
(default 1), user-pool (default 0, meaning as many as the machine has),
io-method (default auto, pinnable to uring or blocking) and
cleanup-deadline (default 30s, 0 disables draining). A key outside the
four is a typo and fails, the same rule nikaia.toml’s [build] follows.
NIKAIA_RUNTIME_CONFIG names the file outright, for one binary in several
deployments.
Pinning uring on a machine that has none is a failure to start, not a
silent fallback: the point of pinning a mechanism is to find out whether it is
there.
cleanup-deadline in nikaia.toml is still accepted and prints where it went.
Refusing it would fail a manifest somebody wrote to the specification that
documented it.
The file’s name and search path, which D5 had left open, are now decided:
nikaia-runtime.toml, looked for in the working directory, with
NIKAIA_RUNTIME_CONFIG naming a file outright for one binary serving several
machines. The keys are spelled as nikaia.toml’s are — io-workers,
user-pool, io-method, cleanup-deadline — because a person who has written
one of the two files has already learned the convention.
D1’s HTTP server, D2’s rustls and D6’s HTTP/1.1 parser are untouched. (Since
written: D1’s server is built at 0.0.166, and D6’s parser is a Nikaia grammar at
0.0.248 - nikaia-std/src/tools/http1.nika, by
ADR-290’s route. D2 is still untouched.) D7’s
first rule now has the check it named: ADR-005 §5’s structural
Send check refuses what it can decide as NK2502, at both settings, and the
rule’s reach is bounded by the honesty of the foreign signature as D7 now says.
D7’s second rule — a Cleanup may not be owned by a foreign task — is still
untouched, and cannot be built before ADR-297 is.
The order §4’s first draft implied still holds, minus the part that
now exists: a socket layer that keeps registrations rather than answering one
readiness question at a time, then a minimal HTTP/1.1 server on it — which is
also what gives ADR-292 the real program its debt has been
waiting for — then the parsing moved into Nikaia, then rustls, then HTTP/2.
D7’s two rules were the exception worth checking early rather than late, because
a program that starts hyper is the cheapest test of whether
ADR-002 D1’s crates.io promise and this record can hold at the
same time. That check ran, and §2’s D7 carries what it found.
And one thing is deliberately not built, because it is not this record’s to
decide. fs::read_both exists and is measured, and no lowering uses it: at
user_parallelism = no, ordering = "effects" still degrades to strict
(ADR-292b). §8.5 called that degradation’s reason “no event
loop, no sidecar and no async lowering yet”, and two of the three are now gone —
but whether a statement pair may be lowered onto a completion-based pair is
ADR-292’s decision, and making it here would be deciding another record’s
question in this one’s delivery.
ADR-292 has since decided it — D10, and the answer is yes for a pair of
file reads, at both settings of user_parallelism. Two things about that are
worth having here rather than only there. The lowering does not name
fs::read_both: that function overlaps whatever the mechanism costs, and D10
refuses to pay the fallback’s ~38 µs for an overlap the program did not ask for,
so it names task::read_pair, which is this record’s §4.3 third column turned
into a policy — free on the completion path, the written order on the fallback.
And the measurement that settles it reproduces §4.3’s shape and not its
absolute numbers on a different day: see docs/history/runtime-cost.md §6, where the
same script on the same box measures join − seq at +112 µs where §4.3 has
+59 µs. The signs, the flatness and the ratios hold; the box does not.