Nikaia
Menu
v0.0.1343pre-alpha GitHub

ADR-yaml-1: yaml Is an Official Package in packages/yaml/: a Grammar Over Bytes Builds a Zero-Copy Tree With Spans, YAML 1.2 Core Schema, Limits On by Default; the serde_yaml Facade and stream Come Later

Status: Accepted Date: October 30, 2026 (#928; the owner’s decisions of 2026-10-28 and 2026-10-29 recorded there; D5’s conversion of numbers, D11 and D4’s private Scalar value are the owner’s of 2026-10-30) Answers: where the official package yaml lives, what its native API is, what it is tested against, and in which order its later layers come (#928) Supersedes / Related: ADR-344 D1, D2 (an official package lives in packages/<name>/ with its own version, taken in by path); ADR-338 D2 (formats are official packages; yaml joins the list), §6 (editing a text in place: answered here for yaml); ADR-grammar-1 (rule parameters, bounds, guards: YAML’s indentation n and context c); ADR-296 (a grammar is part of the language, lowered to the winnow-grammar engine); ADR-282 D12, D13 (a field holding only views is a view position); ADR-016 D5 (a grammar over a mapping checks UTF-8 as it reads)

1. Question

What is the yaml package, as a program sees it, and what must stand before its parts can be built? #928 fixed the direction (a native API nobody fills well, and a drop-in for the archived serde_yaml 0.9 for Rust users) and the owner decided the shape (one package that is also the crate’s core; features serde, tree, stream; licences). Without this record a builder does not know the input type, the tree’s shape, what a limit is, nor which parts wait on the toolchain.

2. How others do it

Library Its answer What it costs there Source
serde_yaml 0.9 (Rust) from_str::<T>, Value that owns its strings; YAML 1.2 core schema (yes is a string); duplicate key an error; << merged only by apply_merge; a recursion limit every string allocated; no spans in Value; no comments kept; archived its docs and source, 0.9.34
saphyr / yaml-rust2 (Rust) an event parser with markers, and a loader building an owned tree owned tree; events carry a position, not a span from memory, not checked
ruamel.yaml (Python) round-trip mode keeps comments and order on edit slow, fragile formatting rules from memory, not checked
toml_edit (Rust, TOML) a document model that keeps the text’s formatting, edited in place a second API beside toml’s its docs

None offers a tree that points into the input, carries spans and is edited in place in one model.

3. Facts that bear on it

Probed with target/fast/nikaia on develop (2026-10-30):

4. Decision

D1 — yaml is packages/yaml/, a package with its own version; nothing here moves the language’s version.

packages/yaml/ holds nikaia.toml (name = "yaml"), src/ and tests/, like packages/json/ (ADR-344 D1, D2). It is taken in by path. A change there raises only the package’s version. This record carries no Part I-III text, so it raises nothing either. A compiler defect met on the way is fixed in crates/ as its own commit, which raises the toolchain’s version as any such fix does.

D2 — The input is ref Bytes; positions are byte offsets.

The parse reads a ref Bytes (a mapped file, a buffer read from a pipe), so spans exist today and a mapped file is read without a copy. A byte that is not UTF-8 is an error at its offset. Line and column are computed from an offset when an error or a tool asks. An entry taking String comes with text::Pos (#743).

D3 — One grammar builds the document tree, written after the specification’s productions.

The grammar is a Nikaia grammar with rule parameters for the indentation n and the context c (ADR-grammar-1), its rules named after YAML 1.2.2’s productions where one stands for one (s-indent(n) is Indent(n)), so a reviewer can hold it against the specification. One grammar, no hand-written scanner beside it.

D4 — The tree points into the input and carries spans.

pub struct Node { pub start: u32, pub end: u32, pub anchor: String?, pub tag: String?, pub value: Value }
pub enum Value { Null, Bool(bool), Int(String), Float(String), Str(Scalar), Seq(Vec[Node]), Map(Vec[Entry]), Alias(String) }
pub struct Entry { pub key: Node, pub value: Node }
pub struct Scalar { pub raw: String, pub style: Style, value: String }   // Plain, Single, Double, Literal, Folded; `value` read through `text()`

Names may change in the build; the rules may not:

D5 — Plain scalars resolve by the YAML 1.2 core schema; a quoted scalar is a string.

null, Null, NULL, ~ and the empty node are null; true, True, TRUE, false, False, FALSE are booleans; [-+]?[0-9]+, 0o[0-7]+, 0x[0-9a-fA-F]+ are integers; the core schema’s float forms, .inf and .nan in their three spellings, are floats; everything else is a string. yes, no, on, off are strings. The core tags (!!null, !!bool, !!int, !!float, !!str, !!seq, !!map) fix a node’s kind; any other tag is kept on the node.

A number of a known shape is converted for that shape, and its shape is checked once (owner, 2026-10-30). The core-schema resolution in the grammar’s action has already checked an Int’s or a Float’s shape, so as_i64 and as_f64 check nothing per digit, as benches/brc/fast.nika reads its temperature: a hexadecimal digit’s value is (c & 15) + 9 * (c >> 6), branch free over a digit already checked; whether 0x… or 0o… fits an i64 is decided by the count of its significant digits (and, at the limit, the first one), not by checked arithmetic; a decimal integer stays text::parse_i64. No generic radix parse enters std for this (left open for later, not refused).

D6 — The parsed tree is the text as written; load resolves it.

The tree from the parse keeps Alias nodes, anchors and tags as written: it is what spans, events and editing work on. load gives the tree a program reads: each alias replaced by its anchored node (the structure copied, the strings still views), a key given twice in one mapping an error naming both positions (keys compared by kind and value), and << an ordinary key: a merge happens only on request, by a separate function, as in serde_yaml 0.9.

That function (architect, 2026-10-31, as serde_yaml 0.9’s apply_merge and the YAML 1.1 merge key): merge(node: Node, input: ref Bytes) -> Node sync throws over a tree from load, every mapping at every depth. A mapping’s << entry is taken out; its value is a mapping, or a sequence of mappings, whose entries are added after the mapping’s own, in order, each only when its key (compared as load compares keys) is not there yet: the mapping’s own keys win, then the first merged mapping over a later one. Merged nodes are merged themselves first. Any other value of << (a scalar, null, a sequence holding a non-mapping, an alias left unresolved) is an Error at that value’s position, input giving its line and column, as for every error of the package. A << that is a quoted scalar ("<<") is an ordinary key, as in serde_yaml.

D7 — Events are a walk of the parsed tree, in the yaml-test-suite’s vocabulary, each with its span.

events turns the parsed tree into a list: stream, document (explicit or not), mapping and sequence start and end (block or flow), scalar (style, raw text), alias, each with anchor, tag and span. Printed one per line they are the suite’s test.event form (+STR, +DOC ---, +MAP {}, =VAL :a, =ALI *x), which is what the tests compare. Events one at a time with fixed memory are the stream stage (D10).

What the walk needs and the tree did not yet keep is kept in the tree, as written (D6), so the walk reads nothing from the input again (architect, 2026-10-31): a Node has pub flow: bool, true for a flow sequence or mapping, false otherwise, so every match on Value stays as it is; and documents(input; limits) gives Vec[Document], a Document holding its root: Node and whether --- and ... were written (start: bool, end: bool). parse stays the call for a program that wants the nodes: it is documents with each root taken out, so it pays for nothing it does not read. events walks documents; an event’s line form is the package’s own function, not the caller’s.

A scalar keeps how it was written whatever it resolves to (architect, 2026-10-31, y21’s report; Language decisions of docs/supervisor.md: D6 already says the tree is the text as written, and D7’s flow set the form). ~, NULL and null are one Value::Null, True and true one Value::Bool(true), and !!int "1" an Value::Int with no quotes; the suite’s =VAL :~, =VAL :True, =VAL <tag:yaml.org,2002:int> "1 need the difference. So a Node has pub raw: String, the scalar’s text as Scalar’s raw takes it (a view: between the quotes, after a block header), and pub style: Style, for every scalar node, whatever its Value; a collection, an alias and an empty node have "" and Style::Plain (the suite writes an empty node =VAL :). Not payloads on Value::Null and Value::Bool: that changes every match on Value and still loses a tagged scalar’s quotes. A Value::Str’s Scalar keeps its own raw and style, which text() reads.

D8 — Limits are on by default, a value the caller may pass, and an exceeded limit is an error.

Limits holds the nesting depth (default 128), the nodes an alias expansion may produce (default ten times the document’s own nodes plus 10 000: a billion laughs stops, an ordinary anchor does not), and the input’s size (default the span type’s range, 4 GiB minus one). The parse checks depth and size; load checks aliases. An error names the limit, its value and the position. A caller who passes a Limits raises or lowers them; none switches them off by omission.

D9 — The tests are the package’s own, then the yaml-test-suite vendored byte for byte.

First, the package’s own tests/*.nika with .stdout, run by nikaia test --project packages/yaml. Then the yaml-test-suite’s generated data branch (one directory per case: in.yaml, test.event, in.json, error), vendored unchanged in packages/yaml/tests/suite/ with the suite’s MIT License and a file naming the source commit; a test program reads the cases and compares, and the cases it passes are listed in a pass list outside the vendored directory, which only grows. The data branch carries no licence file of its own (it is generated from main at tag v2022-01-17, whose License is the MIT text); the owner confirmed that main’s licence covers it, with that License beside it (2026-10-30, #928).

The vendored suite is never changed (owner, 2026-10-28, again 2026-10-30): it is copied byte for byte and stays so. The reason is a licence, not only honest tests: the suite’s Spec Example cases are copied from the YAML 1.2 specification, whose terms allow copying it only unmodified; MIT alone would allow changes, the specification’s terms do not. No case is edited, reformatted, renamed, re-encoded, trimmed or deleted, not even one the parser fails or one that looks wrong; no cargo fmt, nikaia fmt, line-ending or whitespace fix touches the directory. What the package does not pass goes into the pass list beside it, never into the suite. An update is a fresh byte-for-byte copy of a newer upstream commit, with the commit file changed to name it.

D10 — The serde_yaml facade, stream and editing are later stages, each waiting on a named step.

Stage Waits on
events and load over the tree (D6, D7) #1000 (a struct built from a ref parameter’s views)
the facade crate nikaia-yaml: serde_yaml 0.9’s surface and behaviour, hand-written Rust over the generated crate, same version (owner, 2026-10-28, 2026-10-29) #978 (artifact = "rust-library", [features] with serde default, tree, stream)
stream: events from a file, pipe or socket in fixed memory #978, and the lowering writing winnow-grammar’s parse_<rule>_reader, #[stream] and #[stream(each)]
format-preserving editing (comments, order, indentation kept) the tree carrying comments and every token’s span; an ADR of its own for the edit model
writer matching serde_yaml 0.9’s output the facade’s tests
typed yaml::read[T] on the Nikaia side #859, as json::read[T]

D11 — An escape is checked and decoded once, by the grammar, where the knowledge is.

(Owner, 2026-10-30.) An invalid escape is a syntax error in YAML 1.2.2, so the grammar checks it and the error carries line and column: the digits of \x, \u and \U by a guard per digit (x:any if(is_hex(x)), two, four or eight of them; not waiting for hex_digit, #1002), and the range by a guard on the value: no surrogate after \u, nothing above 0x10FFFF after \U; \x is in range by its two digits. The grammar’s action decodes the escapes into the scalar’s value, so the guard’s fact reaches scalar::from(n), whose precondition (n a Unicode scalar value, #1003) is then proved, not checked again. A second check of the same digits in text() or a helper is what this decision excludes; where the prover does not carry a guard’s fact into the action, that is a compiler defect, fixed before the decoding is built.

5. Consequences

6. What this does not decide

This site is open source. Improve this page.