overlap { … } Is How a Program Asks for LessStatus: Superseded by ADR-292 — its D1-D7 are D1-D7 there
Related: ADR-033 (what this displaces: statement overlapping,
seq, and the ordering build switch), ADR-038 D3 and D4 (the
runtime this needs, and the part of it that is built),
ADR-021 D5 (why ordering became a cache dimension — it stops being
one), ADR-027 D1 (the sync property, which is what D6 reads to
know which branch to start first), ADR-037 D2 (the switch, and why
D6 reaches the same schedule at both settings), Part I 1.2 (the count of build
switches, which returns to two), Part I 8.1.1, Part II 12.2 and 12.4–12.7 (the
concurrency forms this joins)
ADR-033 made two operations that touch disjoint resources unordered:
the compiler may overlap them, and a program says otherwise with seq { … }. It
has read provisional and unmeasured since it was accepted, and
open-decisions.md carried the question of when it would be measured; this
record is the answer, so the entry is gone from that file.
The measurement could not settle it. The only corpus is this repository’s own examples, written by the people who designed the language; a number off them describes those programs rather than the language. A decision conditioned on such a number is a decision deferred to an event that will not make it.
And the state it deferred is the worst of the available ones. Asked of a
program, the analysis turns out to reach only adjacent statements: a pair of file
reads with their failures caught overlaps, and one line between them removes the
pair from consideration entirely — not refused with a reason, never examined. A
read whose failure is uncaught overlaps with nothing at all, because the statement
after it might never have run. So the wins are narrow and hard to predict, and
where they do not arrive a program has no way to ask for them: spawn starts
something that outlives the call, select takes the first to finish, par_iter
runs one operation over many elements, and seq only forces an order. The one
shape every other language offers first — run these and wait for all of them — is
missing.
Unpredictable wins, and no recourse. That is what this record changes.
The automatic reordering is withdrawn. Two statements run in the order they are written whether or not they meet on anything, and no analysis stands between the source and the schedule.
The reason is not a number, and it is visible in the design being withdrawn.
ADR-033 ships two escapes: seq { … }, to force an order the
analysis cannot see, and --ordering strict, to turn the whole thing off. Two
escapes are an admission — that the analysis can be incomplete, and that the
correction is by hand, after the fact, by somebody who noticed. A rule that changes
what a program means, resting on an analysis admitted to be incomplete and
corrected manually, is the shape of a defect source rather than of a guarantee.
Against that stands an optimisation whose reach nobody has been able to state.
overlap { … } runs its branches together and waits for all of them.let (user, rights, prefs) = overlap {
db::load_user(id)
db::load_rights(id)
cache::load_prefs(id)
}Each statement in the block is one branch. The block starts every branch, waits for every branch, and its value is the tuple of their results in written order.
A branch is an expression. Several steps in one branch are a block expression inside it.
The touch sets ADR-020 and ADR-033 built are kept, and asked the other way round. They no longer decide whether the compiler may overlap two statements; they decide whether the programmer was right to say so.
A pair of branches that reach a common resource is refused, and the diagnostic names the resource:
error: these two branches cannot run together — they meet on the output
3 | println("x")
4 | println("y")
= two writes to one resource have an order, and `overlap` says they have none
= if you meant them in order, write them as ordinary statementsThis is the part no other language has. Every mainstream form for run these together takes the programmer’s word for the independence; here the claim is checked against what the ledger records each call touching. That is the return on the machinery ADR-033 built, kept after the inference it was built for is gone.
overlap block is not a task, and nothing outlives it.The block ends before the function continues. Nothing is moved into a branch, borrowing works inside one, and the structural crossing check (ADR-005 §1 Group B) has nothing to do, because nothing crosses out.
That is what makes the form lighter than two spawns, and why it earns a form of
its own rather than being a pattern written with the ones that exist.
Failure propagates as it does anywhere else (ADR-023 D8): a branch
whose failure is uncaught makes the block fail. Where more than one branch fails,
the error is the one from the first branch in written order — the only order
the source has, and therefore the only one that makes the result reproducible —
and the others are attached to it as its secondary list, in written order
(ADR-115).
Per-branch handling is catch inside the branch, unchanged.
I/O is not user code, so any number of operations may be in flight at once at
either setting of user_parallelism (ADR-037 D2). What no
bounds is user computation: one piece at a time. So a block holding two reads and
one computation costs max(reads, computation) rather than their sum, at both
settings, and the setting changes only whether two computations may also overlap.
Reaching that bound requires an order of starting, and the naive one does not:
let (n, a, b) = overlap {
expensive(x) // does not suspend
fs::read_to_string("a") // suspends
fs::read_to_string("b") // suspends
}Running branches in written order until each suspends would run expensive to
completion before either read is submitted, and the block would cost the sum. So:
a branch is started up to its first suspension point before any branch that
cannot suspend is run. The reads go out, then the computation runs while they are
in flight, then the results are collected.
Which branches those are is read off machinery that exists: a call that can pause
is exactly what ADR-027 D1’s sync property already answers, per
function, over the same call graph.
This changes nothing observable, because D3 has established that the branches
meet on nothing. It is a choice of schedule and not of meaning — how, never
what (Part I 1.2).
seq { … } and [build] ordering are withdrawn with the reordering.seq exists to force an order between statements that would otherwise be
reordered. Under D1 nothing reorders them, so it has nothing to do and goes.
ordering in [build] exists to turn the reordering off. There is nothing left to
turn off, so it goes as well — and Part I 1.2’s “there are exactly two things to
choose” becomes true again, having been three.
ADR-021 D5’s cache dimension goes with it.
What this displaces. ADR-033 D1’s unordering of disjoint
operations, D7’s seq, and the ordering switch; Part I 8.1.1’s “Order Is Kept
Where It Can Be Seen”, whose title is the qualification D1 removes; Part II 12.2’s
closing sentence, which points at seq for an order the touch sets cannot see —
under D1 there is no such case, because nothing is taken away to begin with. Part I
1.2’s count of switches returns to two.
What it keeps. The touch sets, their ledger entries, and --overlaps, which
becomes a report about overlap blocks rather than about statements the compiler
chose. The analysis survives its original purpose because D3 gives it a better one:
checking a claim is a stronger use of it than making one.
What a program loses. The overlap it used to get without asking, on adjacent independent I/O with failures caught. That is the narrow case measured in §1, and D2 covers it by being written down.
let a = … in a
branch rather than a tuple outside. The tuple is the smaller form and admits the
other later.overlap is for a fixed, small number of
different things; the same operation over many elements is par_iter
(Part II 12.6).D2 through D6 — the construct. overlap { … } parses, each statement is a
branch, every branch is in flight at once, and the block’s value is their results
in written order.
D3 is the one that earns the form, and it is built on exactly the machinery
D1 withdraws the inference for: each branch is reduced to an
ADR-033 Operation and every pair is asked for a verdict. A pair
that is not Overlap is refused as NK2104, and the message is that verdict’s
own words — “both reach stdout, and one writes it”. A branch this compiler
cannot account for is not refused, because reading an absence as “they meet”
would refuse a correct program on nothing (Part III C.4). And a branch that
binds is refused too: the block’s value already carries every branch’s
result, so a let inside one would name a thing that leaves by two doors.
D6 is the argument order, which is what makes it cheap: std has one
overlap<n> per arity that polls every branch in one pass, in the order it is
given, and the emitter hands over the branches that can pause first. Nested
binary joins would have made the naive order structural — interleave(a,
interleave(b, c)) polls a to completion before b is started, which is the
sum D6 names and refuses. The results are put back into written order at the
call, with a comment naming this record; a block whose branches are all of one
kind pays for no permutation.
D5 is the language below’s own control flow. Where any branch can fail out of
itself, every branch is wrapped in Ok and the tuple’s elements each take a ?
in written order — and ? returns at the first Err, so “the first in
written order wins” is not a comparison this compiler makes. Per-branch handling
is a catch inside the branch, which the fallibility walk does not count. The
error type is written rather than inferred, because an async block with a ?
and nothing to infer from is “type annotations needed” about a file nobody
wrote (Part III C.1).
D4 is visible in the emitted Rust: a branch is an async block and not an
async move one, so nothing is moved in and a branch borrows what is around it
exactly as an ordinary statement does. That is what makes the form lighter than
two spawns, and it is a line of generated code rather than a claim.
D1 and D7 — the withdrawal. The order §5 gave was the runtime binding, then
D2–D6 on it, then this; withdrawing the reordering before there was a way to ask
for overlap would have left the language with neither, and there is a way now.
So: the emitter no longer groups statements, seq { … } is not a construct and
seq is not a reserved word, ordering is not a switch in the manifest or on
the command line, and ADR-021 D5’s cache dimension for it is gone.
Part I 1.2’s count of switches is two again, having been three.
What survives is the half D3 needs, which is the whole point of the order
this was taken in. contracts::order keeps the touch sets, Operation,
verdict and the vocabulary — what goes is Vehicle, the grouping, and the
lowerings the compiler chose. --overlaps is a report about overlap blocks
now: which branches a program wrote, and which of them are started first,
which is the one thing D6 decides that the source does not show.
Un-reserving seq is the direction that costs nothing. Reserving a word
narrows what parses and un-reserving one widens it, so no program that compiled
before stops compiling — and Part I 1334’s own example names a parameter seq,
which is a line the withdrawal makes legal rather than one it breaks.
One limit, with its reason in the refusal. std writes one vehicle per arity
and eight are written, because the branches have different types and a tuple of
futures is what that means in the language below. A block of more branches is
refused naming the limit rather than miscompiled. A block of one is refused
too: it has nothing to overlap with.
And a gap this met rather than made. Part I 8.1.2’s own example writes let
(user, rights, prefs) = overlap { … }, and let takes one name. Part II 12.5’s
let (tx, rx) = channel::bounded(100) writes the same form, so the destructuring
let is a construct the specification uses twice and defines nowhere —
#224 carries it. An overlap is reached by its tuple in the meantime.