yaml, also a Rust crate: a serde_yaml 0.9 drop-in layer over a native API (zero-copy, spans, format-preserving edits, limits)Open and answer this issue on GitHub.
ready, wip · opened 2026-10-28 · updated 2026-10-31 · 11 comments
An official package yaml (ADR-338 D2, in packages/yaml/) that Nikaia programs use, and that is published as a crate Rust programs use on its own, in two layers:
serde_yaml 0.9. Switching is one line in Cargo.toml:
serde_yaml = { package = "nikaia-yaml", version = "0.1" }Surface: from_str / from_slice / from_reader, to_string / to_writer, Value, Mapping, Number, Error with a position, and the serde Deserializer / Serializer impls.
Behaviour matters more than the API; serde_yaml’s defaults are kept, or existing configs break silently:
yes / no / on are strings, not booleans (the “Norway problem”).<< are not resolved by themselves, only on request.Every deviation goes on a migration page. Better error messages may differ.
Value<'a> pointing into the input, and deserializing &'a str without allocating (serde_yaml cannot).toml_edit. Missing in Rust; ruamel.yaml fills it poorly in Python. Tools such as Renovate, Helm helpers, CI bots need it: probably the strongest selling point.Later, the same principle for Python (drop-in for PyYAML safe_load / safe_dump; no unsafe load; a ruamel-like round-trip API) and JS (parse / stringify as the yaml package, plus a document model that keeps comments).
Benchmark shows both layers: “drop-in, x times faster than serde_yaml” and “native API, another y times faster” (counted with callgrind, CLAUDE.md “Measuring performance”, against tuned Rust: serde_yaml 0.9, serde_yml, saphyr / yaml-rust2).
json, toml, csv, html, http1, db; yaml is not on the list. Publishing as nikaia_yaml follows ADR-338 D2 / ADR-286 D23 / ADR-336 D2 (crates.io treats nikaia-yaml and nikaia_yaml as one name), so the owner’s Cargo.toml line fits the prefix rule.yaml (and is the model for toml, which nikaia.toml tooling wants).json): read[T] / write[T] wait for #859 (type parameters, T::kind, building a value from fields). yaml’s typed read on the Nikaia side waits for the same.pub fn from_str<'de, T: Deserialize<'de>>, impl serde::Deserializer<'de> for &mut Deserializer, impl Serialize for Value, Value<'a> with a lifetime, std::io::Read / Write. Nikaia cannot implement a foreign Rust trait today, and the lowered names are not a stable public API. Options:
T: Deserialize<'de>), so the facade is Nikaia too — larger, and a language change;packages/yaml/bindings/?) or separate; built on the same Rust crate (PyO3, napi / wasm).Decided: the owner’s comments below (2026-10-28: option (a), one crate one version, licences; 2026-10-29: features serde/tree/stream, streaming via winnow-grammar) and ADR-yaml-1 (docs/specification/adr/adr-yaml-1.md). Built by the lane scripts/drive.nika -- docs/briefs/yaml (briefs in docs/briefs/yaml/, order in its batch.txt).
ref Bytes into the zero-copy tree with spans (ADR-yaml-1 D2-D5), the package’s own tests:
y1-tree-core.md, 642eedf; 0x/0o and \x \u \U left for 2c);0x/0o converted for their shape (ADR-yaml-1 D5, D11; owner 2026-10-30): scalar::from0x/0oy7b-1009-let-reads.mdy4c-1003-finish.mdhex4 promises its range with an assert (ADR-359 D3), scalar::from(cp), then dfe7bac6 (an integer as scalar refused, NK1241), Fixes #1003;y7c-match-join.mddecision, nothing needs it): #1021 (found by y7b: a match arm’s assignment to a let mut is not joined as after an if, ADR-361 D9); a defect, not needed by json/yaml, built here since it touches only the prover;let d = ds[k] read (spike probes b_acc_ret, c_acc_for);y8-1026-grammar-label-struct.mdn:(P), reaches rustc; nothing in the package needs it): y9-1027-group-label.mdString reaches rustc; nothing in the package needs it): y10-1030-literal-label.mdy2c-properties.md (y2 step 2 from the draft on wip/y2-properties, ea068fa4; run 1 blocked on two compiler defects, filed as #1032 (a call in a grammar guard is no call site for the text tiers, NK1185) and #1033 (a guard cannot read a text(..) label that binds names, rustc): y11-1032-guard-call-tiers.mdy12-1033-guard-text-label.mdref yaml::Node gets one lifetime, ADR-353 D7): y13-1037-dependency-places.mdwip/y2c-properties, b223f972) but the fmt scan is red on #1015 (nikaia fmt breaks a labelled guard on a long rule line into text that does not parse): y15-1015-fmt-guard-label.mdy2c-properties.mdtext(..) (NK1117), #1013 step 4: y14-1013-guard-inner-label.mdtext(d:P+) or a name below a repeat read by an action reaches rustc; nothing in the package needs it): y16-1039-unbound-text-names.mdy17-1041-entry-takes-a-parameter.mdLimits for depth and size (y2b-keys-directives-limits.md)peek(..)/not(..) passes check, fails at run), #1050 (a guard handing an optional label to a by-value Node? parameter, rustc), #1051 (nikaia fmt breaks before a postfix *); the package needs none of them (it uses peek(Dash) and t: ref Node?): y18-1049-1050-grammar-builtins-and-guard.mdt.clone() for an optional; nothing needs it): y20-optional-copy-help.mdy19-1002-hex-digit.mdHex guard replaced by the built-in hex_digit, #1002 built in 705a5db2; two error offsets of a bad \x/\u escape now point at the offending character, accepted by the architect: “where reading stopped”).ref parameter), which steps 4-5 need: y3-1000-views-from-ref.mdlowered/project_members.rs drift fix that kept CI’s sysroot tests red from 263d13d0). Its Found (an owned field moved out through an index, ns[0].raw, reaches rustc) is commented on #927; nothing needs it.Node.flow and documents gives Document with its ---/... markers, D7’s paragraph of 2026-10-31): blocked on #1063 (found by the architect probing this step: a view field of a lent struct, n.anchor of a ref Node, stored in a new struct is refused NK2106 and cascades into NK1106 across the grammar; load needs it too). y22-1063-view-field-copy.mdy21-events.md run 1 built Node.flow and documents (2a303980) and stopped on the tree dropping a scalar’s text and style when it resolves to Null/Bool/Int/Float (~ = null, !!int "1" loses its quotes); decided by the architect (ADR-yaml-1 D7, last paragraph: Node.raw, Node.style for every scalar node). y23-scalar-written.mdNode.raw, Node.style). y21-events.md run 2 drafted events (branch wip/y21-events, 0433d662) and stopped on three view-checker defects, filed #1070 (a by-value field-less enum beside a view taken for a view source, NK2302), #1071 (a list of views handed back refused NK2302 as a literal, started non-empty, or grown with extend), #1072 (a view field off a match payload of a lent struct, NK1106). y24-1070-by-value-enum-not-a-view.mdy25-1071-list-of-views-literal-extend.mdy26-1072-payload-view-field.mdy21-events.md run 3 finished the draft (wip/y21-events, 1c1199d6: Event.value a view, Event.text the owned text of a Value::Str) and stopped on #1085 (a list or struct of views handed back through an if/match is refused NK2302; the architect checked the rest of the draft builds and its events test runs). y28-1085-views-back-through-if-match.mdy21-events.mdEvent, events, event_line; tags written as the tree keeps them, resolving them later). y27-load.md run 1 built load (load(input; limits), aliases expanded per document within the alias limit (checked before copying), an unknown or recursive alias an error, a duplicate key an error naming both positions (keys by kind and value: a="a", 16=0x10), << an ordinary key; D6, D8) on local branch wip/y27-load (eef3a2e7) and stopped on #1089 (a local bound by m.get(k) ?? <jump> / match m.get(k) { null => <jump> x => x } is not lent; a match on its field lowers without &, rustc E0507). y29-1089-match-on-a-get-bound-local.mdmatch moves out is inferred lent, rustc E0507; nothing needs it). y27-load.mdload). y31-merge.md run 1 (merge(node, input), the << key on request; ADR-yaml-1 D6, paragraph of 2026-10-31) stopped on #1094 (a call result built from a by-value view-holding argument is charged to a ref argument beside it: NK2302 on a struct-literal store, rustc “missing lifetime” through a method). Batch of 2026-10-31 (twelfth): y32-1094-result-from-struct-argument.md (Fixes #1094), y31-merge.md run 2, y30-1092-match-moves-owned-field.md (Fixes #1092). Next: tag resolution for events.data branch vendored byte for byte with its License and source commit, a test program over every case, a pass list that only grows (D9). Waits for the owner: the data branch (6ad3d2c, generated from main 45db50ae = v2022-01-17) carries no licence file of its own; main’s License (MIT, Ingy döt Net) is byte-identical on main and data-flat.to_string) matching serde_yaml 0.9’s output where a program could depend on it.nikaia-yaml (serde_yaml 0.9 surface and behaviour, ported tests with their licence, migration page): waits on #978.stream: waits on #978 and the lowering writing winnow-grammar’s parse_<rule>_reader, #[stream], #[stream(each)].benches/yaml/ (serde_yaml 0.9 vs. drop-in vs. native API, instructions counted).Found on the way (y21 run 2, 2026-10-31): #1070, #1071, #1072 (above). y21 run 3: #1085 (above). y27 run 1: #1089 (above). y29: #1092 (above). y31: #1094 (above). Found on the way (y4b/y5/y6, 2026-10-30): #1009 (above), #1010, #1013 (above), #1014 (lowering cache key misses an inferred sync), #1015 (nikaia fmt breaks if(cond, "label") rule lines), #1016 (a ref Vec[T] result into a ref Vec[T] parameter); y7b (2026-10-31): #1021 (above); y7c: #1022 (what a match guard gives its arm, decision); y2 (2026-10-31): #1026 (above); y8: #1027 (above); y9: #1030 (above); y2c: #1032, #1033 (above); y12: #1013 step 4 (above); architect probing y2c: #1037 (above), #1038 (a dependency’s view field returned beside an owned text, NK1104; nothing needs it yet); y14: #1039 (above); architect probing y2b: #1041 (above), #1040 (a return in a grammar action reaches rustc; decision, nothing needs it); y2b: #1049, #1050, #1051 (above); none else blocks the package beyond those named in 2c.
Found earlier: #999 (a tuple result mixing String with another type refuses views; the package uses a struct), #1000 (above), #1001 (a group with branches of different types reaches rustc), #1002 (hex_digit undefined; the package checks digits with a guard instead, by decision), #1003 (scalar::from, decided 2026-10-30; steps 1-2 on develop, step 3 in y4c), #1004 (a temporary Bytes(..) entry argument). Only #1003 blocks the package (2c); #1000 blocks step 4; #1001, #1002, #1004 are compiler bugs the package does not need and come after y3.
Every gap found on the way in the compiler, lowering or std is a finding fixed in the compiler (CLAUDE.md), not routed around in the package.
Generated by Claude Code
packages/yaml/ is the Rust crate nikaia-yaml: lowering the package fills the crate’s src/ with the generated .rs files, and the hand-written Rust files lie beside them. The Nikaia package and the crate carry the same version. Reason: when Rust trait support arrives, the hand-written files move into Nikaia inside the same package, with no migration between two crates.decision until then.crates/nikaia-std is that shape today: hand-written .rs modules next to .nika sources whose lowered .rs files (src/tools/lowered/*.rs, src/text.rs, …) are committed and written by nikaia lower-std; one Cargo.toml, one version. What is missing for a package:
std) into its own src/ the way lower-std does, with the lowered files committed (ADR-290 §”not decided”: publishing a Nikaia-written module on its own);nikaia.toml’s version and Cargo.toml’s), checked like the badges;nikaia_std, so the crate depends on nikaia_nikaia_std (on crates.io from 0.1, ADR-336 D2). A Rust user of nikaia-yaml pulls that in; the runtime it needs should stay small (no I/O workers started by a parse).| Source | Licence | Compatible with our Apache-2.0 | What taking it in requires |
|---|---|---|---|
serde_yaml 0.9.34 (dtolnay/serde-yaml): API shape, behaviour, tests |
MIT OR Apache-2.0 (Cargo.toml, LICENSE-MIT, LICENSE-APACHE) |
yes | ported tests keep the copyright/permission notice (author: David Tolnay; LICENSE-MIT has no copyright line of its own) and say they were changed (Apache 4(b)); a LICENSE-serde_yaml / NOTICE entry beside them |
| yaml-test-suite (yaml/yaml-test-suite) | MIT, “Copyright (c) 2016-2020 Ingy döt Net” | yes | ship its License file with the data |
| … its “Spec Example” cases, copied from the YAML 1.2 spec | the spec says “This document may be freely copied, provided it is not modified” (yaml-spec 1.2.2) | yes if the files are taken unchanged | vendor the data files byte for byte; our tests read them, never rewrite them |
| libyaml / unsafe-libyaml (serde_yaml’s parser) | MIT | yes | only a reference for behaviour; no code is taken |
Not checked: the data-* release branches of the yaml-test-suite (GitHub’s API did not answer from here); their licence is to be confirmed before one of them is the vendored form.
Generated by Claude Code
target/fast/nikaia on develop)| Need | State | Blocks |
|---|---|---|
Zero-copy tree: parse(input) -> Value whose strings are views |
works: lowers to enum Value<'a> { Str(&'a str), .. }, fn parse(input: &str) -> Value<'_> |
nothing |
Pull / event parser: a struct holding the input and a position, next returning views |
broken: checker accepts, rustc refuses (lifetime; move while borrowed); a ref String field refuses a text read at run time (NK1106) while NK1261 calls ref redundant. Filed as #929 |
the event parser, the editing model |
Cutting text at positions: text::Pos, text::Cursor, before / after / between, find(…; from:) |
ADR-374 and ADR-320 accepted, not built (#453); json works around it by copying the text into a Vec[scalar] |
a zero-copy scanner that is not rewritten later |
| Streaming input | exists: io::read_to_string(), stdin line by line and in chunks (ADR-019), fs::map. serde_yaml 0.9’s from_reader reads everything first, so the drop-in needs nothing more. An incremental parser over chunks is the package’s design, not a language gap |
nothing for the start |
A type as a parameter, T::kind, building a value from fields (#859) |
not built | only the typed yaml::read[T] / write[T] on the Nikaia side; the Rust layer uses serde |
Lowering a package into its own crate (as lower-std does for std), one version for nikaia.toml and Cargo.toml |
not built | publishing, not the core |
| Bindings | nikaia bind python / node exist (ADR-284 D26: one C library, ctypes / N-API over it): an alternative to PyO3 / napi for step 8 |
nothing |
Order proposed: #929 and the text positions (ADR-374 / #453) before the parser core; the ADR can be written now; #859 and the package lowering run alongside.
Generated by Claude Code
The owner accepts the licence check above. Test data and ported tests may enter the repository on these terms, with no further approval:
License file goes with the data; the files are vendored byte for byte and never changed (the Spec Example cases are copied from the YAML 1.2 spec, which allows copying only unmodified). Our tests read them, never rewrite them. Before a data-* release branch is the vendored form, its licence is confirmed to be the same.With decisions 1-4 above and this one, nothing here waits on the owner any more: label ready. Before the parser core, two language steps: #929 step 3 (does not block the parser) and the text positions (ADR-374 / #453).
Generated by Claude Code
Entscheidungen vom 2026-10-29 zur Gestalt des Pakets:
nikaia-yaml, die handgeschriebene serde_yaml-Fassade. Sie hängt von einem Krate ab, das aus dem Nikaia-Paket erzeugt wird; das ist ein Implementierungsdetail wie serde_derive. Beide tragen dieselbe Version.serde ist Standard und schaltet tree mit ein;tree ist der Value-Baum ohne Kopie und das Editieren;stream sind Events aus Datei, Pipe oder Socket mit festem Speicher.default-features = false mit ["stream"] oder ["tree"] ergibt ein schlankes Krate.artifact = "rust-library" und [features]).Generated by Claude Code
Streaming über winnow-grammar, entschieden am 2026-10-29 (Owner, nach dem Review des Upstream-Agenten):
#[frame].
--- am Zeilenanfang.Partial auf dem vollständigen Pfad? Kostet es nichts, gibt es nur einen Codepfad. Sonst eine Deklaration, die unabhängig von input bytes; ist.parse_<rule>_reader, Form prefix ELEM* suffix). Getestet wird jede Eingabe, geteilt an jedem Byte-Offset und in 1-Byte-Häppchen.{"items": [...]} in JSON, ein XML-Wurzelelement mit vielen Kindern.intern.Alle Punkte sind allgemein formuliert, YAML ist jeweils nur der erste Testfall. Das YAML-Paket beginnt mit vollständiger Eingabe und fs::map und bekommt stream, sobald Stufe 0 da ist.
Generated by Claude Code
Die Engine steht jetzt auf winnow-grammar 3b2dd3d (4853408, 0.0.1304). Upstream hat Stufe 0 und den statischen Pfad gebaut (dort ADR 27):
parse_<rule>_reader für par_fold-Regeln;#[stream] für ein fold, das über einen statischen Pfad erreicht wird, darunter YAML-Dokumente und eine lange Sequenz auf oberster Ebene;#[stream(each)], damit der Aufrufer jedes Element sieht.Nikaia schreibt #[stream] noch nicht. Das kommt mit dem Feature stream des Pakets.
Generated by Claude Code
data branch (checked 2026-10-30)data (6ad3d2c, “Regenerated data from main 45db50ae”, 2022-01-17; main 45db50ae is tag v2022-01-17): one directory per case (in.yaml, test.event, in.json, out.yaml, error), no License file and no other licence statement in the branch.main and data-flat carry the same License (MIT, “Copyright (c) 2016-2020 Ingy döt Net”), byte-identical.Question for the owner: may data be vendored byte for byte into packages/yaml/tests/suite/ with main’s License beside it and a file naming both commits (the data is the copyright holder’s own generated form of the MIT-licensed main)? Until then #928 step 5 waits; steps 2-4 use the package’s own tests.
ADR-yaml-1 is on develop; the lane’s first batch is docs/briefs/yaml/batch.txt.
Generated by Claude Code
Owner, 2026-10-30: yaml-test-suite licence answered, and a rule. The data branch may be vendored, with main’s MIT License beside it. It must be vendored byte for byte and is never changed: no case edited, reformatted, renamed, re-encoded, trimmed or deleted, not even a failing one; no formatter or whitespace fix touches the directory. Failing cases go into the pass list beside it. An update is a fresh byte-for-byte copy of a newer upstream commit. Recorded in ADR-yaml-1 D9 (76cccc1). The vendoring step also adds a .gitattributes with -text for packages/yaml/tests/suite/**, so git never converts line endings.
Generated by Claude Code
Owner’s decisions, 2026-10-30, recorded in ADR-yaml-1 (5914134): (1) no generic radix parse in std for now; numbers of a known shape are converted for that shape, checking nothing per digit (hex digit (c & 15) + 9 * (c >> 6), 0x/0o overflow by the count of significant digits, decimal stays text::parse_i64), D5. (2) scalar::from(n) with a precondition, n as scalar refused (#1003). (3) Escapes are checked (digits, range) and decoded once, in the grammar, so an invalid escape is a parse error with line and column and the guard’s fact proves scalar::from’s precondition; only a double-quoted scalar with an escape or line break gets text of its own, D4/D11. Next lane batch: y4 (#1003), y5 (escapes and radix, starting with a probe whether the prover carries a guard’s fact into the action), y2. To build updated.
Generated by Claude Code
Owner’s decisions, 2026-10-30 (recorded by the architect):
ready with a To build; Part III 14.2 comes with the build (y7a-639-loop-ranges.md). Built together with #1009 and the untyped-let read (y7b). It frees json’s \u and so #1003 step 3 (y4c).text(..) are supported (s:text(parts:Part*): the raw view and the inner bindings in one pass). This settles #1013 point 1 as “support”; the yaml escapes use it instead of peek(text(..)), so nothing is parsed twice. Part II 10 text comes with the build (y6-1013-text-bindings.md). The peek binding refusal in winnow-grammar is reported upstream by the main session and does not block.Scalar.value is a private field, read through text(). ADR-282 D22 stays as it is, and ADR-yaml-1 D4 is amended. Whether the value is a view or owned stays internal (y5b-escapes.md).Lane batch (docs/briefs/yaml/batch.txt): y6-1013-text-bindings, y5b-escapes, y7a-639-loop-ranges. After them: y7b-1009-let-reads, y4c-1003-finish, y2-tree-properties, y3-1000-views-from-ref (To build above).
Generated by Claude Code
95a690fe (anchors, tags and aliases, y2 step 2) is part of #928; its message reads Fixes none by a slip of the report (builder rule fixed in .claude/agents/builder.md).
Generated by Claude Code