Status: Superseded by ADR-283 — its D1, D2 and D5 are D1-D3 there, D6’s inspection half is D4, D7-D8 are D5-D6, its two D9s are D7 and D8; D3, D4’s representation and D6’s assertion are not carried (displaced by ADR-209 and ADR-201)
Refines: ADR-005 D4 — the rule was stated over storage, and
the event that matters is escape
Related: ADR-005 D3 (the ledger) and D8 (determinism),
ADR-007 D8 (interner vs. tether), ADR-006
(Cleanup, whose timing D8 changes)
ADR-005 D4 settled what happens to a slice a parser yields and something stores. It did not settle the user’s own struct:
pub struct Reading { name: &str, temp: i32 } // built ~10^9 times
pub struct Summary { stations: HashMap[&str, Stats] } // ~10^4 entriesThe rule was over the wrong event. Part I 6.6 said a slice “stored in a
struct” is tethered. Taken literally, every Reading carries a Shared handle
on the mapped file — and at user_parallelism = yes Shared is Arc, so a
billion-row run
performs a billion atomic increment/decrement pairs on one refcount word
shared by every worker: a single contended cache line, lock xadd per row. The
worst access pattern a parallel loop can have, and the rule as written mandated
it.
Yet nothing in that program escapes. The mapping outlives the totals, so the
correct lowering for both the field and the map key is a plain &'buf str at
zero cost. Storage is not the event. Escape is.
That leaves a second question: whether Nikaia should distinguish views (&str)
from owned values (String) at all, given that there are no lifetime
annotations. Four designs were available — views plus escape analysis (1); one
String type with automatic promotion on escape (2a); one type with a hard
error on escape (2b); explicit region binding (3).
&str in a field is a view marker, not a lifetime.Two syntactic categories: owned (String, List[T], Bytes) and views
(&str, &[u8], &T). A struct with a view field declares that it may point
into someone else’s buffer. That single sigil is the whole annotation burden —
no region name, no parameter, no propagation into the struct’s type.
ADR-005 D1 stays absolute.
With one type instead, the tether state becomes a hidden type parameter
inferred from use, and that is worse in three ways. The declaration stops
carrying what the diagnostic needs, so every cost is explained at a site remote
from it — the action at a distance ADR-005 §3 rejected in-source
annotations to avoid. The struct’s representation becomes an emergent
whole-program property, so one added call site in a distant module changes
Reading’s layout, or under 2b turns a compiling program into a rejected one,
with no edit anywhere near. And it breaks every existing example — not decisive,
but the 0.0.7 authors reached for &str without being told to.
Every view has one of three states, and the user writes none of them.
| State | Representation | Cost | Chosen when |
|---|---|---|---|
| Borrowed | native &'r str (ptr, len) |
zero | every use provably lies inside the buffer’s live range |
| Tethered | buffer handle + (offset, len) |
one Shared handle per container |
the value escapes that range and the buffer is Shared-capable |
| Owned | String |
one allocation + copy | only where the source says .to_owned() |
Borrowed ⊑ Tethered ⊑ Owned, and inference computes the least state that makes
the program valid:
Transient = borrow. Escaping = tether. Copying = yours to ask for.
“Escapes” means outliving the buffer’s owning scope: returned past it, captured
by a @detached lambda, written into a longer-lived container or a global. A
struct field is not an escape by itself, which is what makes the 1BRC inner loop
free.
All view fields of an instance share one state, so a struct with k view fields has 3 layouts, not 3^k, and monomorphisation is bounded by the lattice height.
The cost is a struct that would want one borrowed and one tethered field; it is lifted to the join. That is a representation choice and never an allocation, so it cannot silently cost a copy.
When a value tethers, the
Sharedhandle is taken by the owner of the escape — the struct, collection or task that outlives the buffer — not by each slice inside it.
This is what makes tethering affordable at 10^9 rows. A HashMap[&str, Stats]
that outlives its buffer holds a small buffer table, one handle per distinct
source buffer, and its keys stay (buffer_index, offset, len). Where the
compiler proves a single buffer — the common case, including chunks that are all
subranges of one map — the index is elided and a key is 8 bytes. Inserting a
million keys performs zero refcount traffic; merging two containers merges
two tables.
Handle traffic is therefore O(containers × buffers), never O(elements), and the failure mode in §1 cannot occur by construction. It also recovers what explicit regions offer — a collection bound to a buffer’s life — by inference, with no region name.
Eq/Hash on views are content-based, never (buffer, range) identity.
Two chunks yield different offsets for Hamburg, and a map keyed by identity
would silently split stations across workers.
Owned is never automatic.ADR-005 §3 already bans compiler-inserted clones; here is the
performance argument for the struct case. An implicit .to_string() on escape
has unbounded invisible cost that scales with the data — a refactor three
modules away turns 10^9 zero-copy slices into 10^9 mallocs — and it can inject
allocation into a sync body the user believes is allocation-free, which is the
opposite of what sync is for. A tether’s cost is bounded by D4 and independent
of the row count.
Where neither Borrowed nor Tethered applies — the buffer is not Shared-capable
(a stack array, an FFI pointer, a slice the caller lent us) — it is a hard
error naming .to_owned() and the two other ways out. That is 2b’s error kept
for the case where it is the truth, rather than made the default rule.
@borrowed is the opt-in “this never tethers” assertion.@borrowed
pub struct Reading { name: &str, temp: i32 }Predictability is 2b’s one genuine advantage: a hot path can never quietly
change representation. @borrowed takes that without the default cost — an
escape that would promote is a compile error naming the escape site, the buffer
and the scope that outlives it. It changes no semantics, only forbids a state
transition, so it can be added to a hot struct after profiling and removed again
without touching anything else.
The inverse tool is inspection, not assertion: because the state lands in the
ledger (D7), nikaia explain --tethers prints the solved state of every view,
and a change shows up as a ledger diff in review.
Recorded per function: the state of every view in its signature. Per struct: the solved state and the buffer-table shape.
Determinism is free. The lattice is a 3-element chain and the constraints are
monotone, so this is the same monotone system as the region solver, with the
same unique least fixpoint. ADR-005 D8 applies unchanged — no new
determinism surface. Switch neutrality holds: the state is identical at
every setting of either switch; only Shared’s expansion differs.
Barriers, not blowup. Crossing a dyn boundary or a published non-generic
API is a state barrier: the value widens to Tethered — the widest
representation, never to Owned. So separate compilation and dynamic dispatch
need one ABI rather than one per state, and the trait-method join stays sound
without ever costing an allocation.
A tether keeps the entire buffer alive: a 12-byte station name out of a
13 GB map pins 13 GB. This is bytes::Bytes’ known space-leak shape. The
compiler warns where a small extract escapes a large buffer at a scope boundary,
with .to_owned() as the fix-it. An accepted, named cost rather than a surprise.
It also moves a cleanup: a tethered container keeps the mapped file’s handle
alive, so ADR-006’s Cleanup runs when the last tether dies, not
at the end of the mapping scope. The diagnostic must say so where the two orders
differ.
let mut stats = HashMap::new() in file_scope is honest and predictable, but
the region name is a lifetime with a friendlier spelling: viral (every function
touching stats must name it), it leaks buffer provenance through abstraction
boundaries — the caller of a grammar would have to know which scope the
parser’s buffer lives in — and it reopens the door D1 nailed shut. D4 delivers
the same guarantee by inference.
Kept on the shelf in one narrower form: an explicit arena for owned data as a future allocator feature, which is an allocation strategy and not a lifetime construct.
| Value | Escapes? | State | Cost |
|---|---|---|---|
NAME slice inside measurement |
no | Borrowed | zero |
Reading.name (≈10^9 constructions) |
no — dies inside the fold |
Borrowed | zero, no atomics |
Summary.stations key (≈10^4) |
no — the totals die before the map | Borrowed | zero |
a chunk from split_aligned |
no | Borrowed | one handle per chunk, 8–128 total |
that Summary returned past main’s scope |
yes | Tethered | one handle for the map, keys 8 bytes |
Nothing is allocated per row and no refcount traffic occurs in the parallel section. Under option 2a the last row would have been a billion allocations, chosen silently.
sync.
See D5.'static.fn name() -> &str { "Ada" }emitted fn name() -> &str, and rustc refused it: “missing lifetime
specifier — this function’s return type contains a borrowed value, but there is
no value for it to be borrowed from”, about a file nobody wrote (Part III, C.1).
A view’s lifetime comes from the input, and Rust’s elision takes it from a
reference among the arguments. A function with no reference among them — and no
receiver — has nothing to take it from, and 'static is the only lifetime that
can be written there: a view handed back by a function that borrowed nothing can
only point at something outliving the program.
So this is a derivation rather than a choice, and it is safe in both
directions. It never refuses a right program: rustc still checks the body, so
a body that cannot honour the 'static is refused about the body — measured,
fn first(xs: Vec[String]) -> &str comes back as “cannot return value
referencing function parameter xs“, on the Nikaia line, which is the message a
reader wants. And it never accepts a wrong one, because nothing about the
signature is taken on trust.
A receiver counts, and so does any parameter that is a view or holds one:
where either is present the result keeps the elision it had, since writing
'static there would demand more of the caller than the body needs.
Part I 6.6 is restated over escape rather than storage and gains the lattice;
Part II 10.6 inherits it; @borrowed and the retention lint enter Part III.
The region solver gains a second, tiny lattice, and the ledger schema gains a state column and a buffer-table shape per struct.
Open: the buffer table’s index width and its growth policy for containers fed by many distinct buffers — a streaming parser over thousands of files. The single-buffer case is fully determined.
All three, since ADR-209 (0.0.185). Tethered is a keep — an append-only store the buffer moves into — owned by the nearest frame that outlives the views, by a handle a task carries, or by each view of a container that drops entries; the views stay plain references. D3’s three layouts and D4’s positions were not needed: ADR-209 §2 says why. What follows is what stood before, kept for the record.
Two of the three states, and the middle one is a refusal.
examples/1brc.nika and examples/inventory are §3’s free case and they run..to_owned(), written by the program and never inserted (D5).NK2302 for a naked view parameter that is stored, and
the backend on the Nikaia line for a view of a local that escapes. That is
D5’s residual hard error standing in for the state that is missing — the
honest answer, and it names .to_owned().@borrowed is parsed and vacuous, which follows: it forbids a state
transition and there is no transition to forbid. The emitter writes a comment
on every such struct saying so.D9 is built, and for a result that carries a view as well as one that is
one. It asked the question of the written type, so -> Vec[Entry] — where
Entry holds a &str — went on eliding and the answer was Vec<Entry<'_>>:
missing lifetime specifier about a file nobody wrote, which is the defect D9
exists to close, one type in. The declaration is what tells the two apart and the
emitter has it. Only in a signature that declares no lifetime of its own,
which is D9’s own case: inside an impl that declares 'a the result is the
subject’s, and 'static there is a promise the impl cannot keep —
examples/1brc.nika’s anonymous constructor is what said so.
D2 is solved and D7’s first half is written: every view in a signature
carries its state in the ledger, under views, and nothing reads it. That
order is the point rather than a half-measure — an analysis whose answers
nothing depends on can be held against the whole corpus and read, and being
wrong costs a wrong line in a file rather than a wrong program.
Owned is never the analysis’s word (D5). A .to_owned() hands back a
String, which is not a view at all, so no view position is ever Owned and the
lattice this solves over is the two states below it. The top exists for the
program to reach, not for the compiler to assign.
The question, once the caller’s buffers are out of the picture. A parameter
is lent for the call, so a view in one borrows; a result borrows where a
receiver or a parameter carries a view, because that buffer is the caller’s.
Where nothing does, the view points either at something that outlives the
program — D9’s fn name() -> &str { "Ada" } — or at a buffer this body made,
and the second is the escape. Does this body own a buffer is therefore the
whole of it, and it is read off the ledger: a let whose initialiser hands back
a String, a Bytes, a Mapped or a run of u8, plus to_owned and
to_string by name.
It errs towards Tethered, which is D7’s own polarity for the case that record names: a barrier widens to Tethered, the widest representation, never to Owned. An unresolved call is a buffer this walk could not rule out. The one shape it does not decide is a buffer built element by element into a list whose element type the ledger does not name; no program in the tree writes one.
What it found: the whole corpus is the free case. Every view in every
signature of examples/ and benches/ solves to Borrowed — §3’s worked
check, read off the analysis rather than asserted. A test holds it as a ceiling,
so a program that starts needing the state that is not built says so.
D7’s second half is not written. Per struct: the solved state and the buffer-table shape is a fact about a representation, and the buffer table is named in the same breath; a type already records which of its fields hold a view. It belongs with the layouts.
D6’s inspection half is built: --tethers prints what was solved, beside
--trust, --overlaps and --sharing. The assertion half is not, for the
reason above — there is no transition to forbid.
#196 carries what Tethered would take, in seven parts, and says which of them decides whether the rest is worth starting.