span(p), here, same_line; then the compiler’s own grammar (parser/mod.rs) moves to NikaiaOpen and answer this issue on GitHub.
language, selfhosting, grammar, ready · opened 2026-10-10 · updated 2026-11-02 · 3 comments
The parser self-hosting session (2026-10-10, #125) moved everything around the compiler’s grammar! block (numbers, parse errors, tree building, the interpreter) and stopped at the block itself: 3 185 lines, 318 rules, written in Rust. It uses three things Part II 10.8 does not offer: the span a rule matched (@=, 52 rules, 78 uses), a declared parser state (state Trivia: newline before a token, /// docs) and hand-written rules (extern rule).
ADR-339 and Part II 10.8, on the owner’s acceptance of the proposal:
span(p) yields the byte range p matched (a Span, two u32, offered by std::text); here yields a position and consumes nothing.same_line matches nothing and fails where the whitespace skipped just before it held a line break. The generated parser records it by position, so it survives backtracking./// is an ordinary rule; a /// where no item, field or variant follows is a parse error, as in rustc.extern rule: a lexical rule and an action that may fail (fail("…")) do what they did.Not built yet; the specification text is ahead of the compiler.
ParseError. Test: rule n -> i64 = t:text(digit+) { parse(t) } where parse fails, inside a | b.span(p) and here: grammar front end, checker, lowering. Tests: span(p) equals the range the Rust _span gives for the same rule on examples/json input; here consumes nothing. Span is declared in std::text and tools/ast.nika uses that one.same_line: the generated parser records by position what its last WS skip held. Tests: a\n[0] and f\n(x) are not an index and not a call (ADR-307 D4); a [0] is..nika grammar, running tests/ and tests/errors/ after each; /// as a rule (ADR-135 D3’s cases and a stray /// as a parse error); the number rules become lexical rules calling parse_numbers.grammar! block and the Trivia state. The commit that does so says Fixes #557.Node ids decided (owner, 2026-10-28): ADR-383, built by #903. The grammar draws no ids from Trivia; a numbering pass right after the parse gives them and writes the span table. What ADR-383 leaves to this issue: semi, ran, comment/block_comment (ADR-337 D2’s comment list) and the trailing-trivia cut PrTrivia::cut that at applies to a span - step 2 (span(p)) must end a span at its last token for the span table to stay byte-identical.
Generated by Claude Code
The order for steps 4-5 is decided in #1188 (owner, 2026-11-02):
grammar! block call a rule of the lowered Nikaia grammar? The answer decides whether the switch goes chunk by chunk or in one commit.parse_numbers), error conversion and the actions’ logic move to Nikaia.tools/parser_grammar.nika, each compared on every tests/compiler and tests/errors program.Generated by Claude Code
Step 4a is answered: no. The Rust grammar! block cannot call a rule of another grammar! block, including a lowered Nikaia grammar, at winnow-grammar 4f8e6691.
&mut path. Another grammar has no such value.Leaf::parse_word returns the entry parser, so rustc reports E0593.parse_word_inner is private (E0603) and generic over __WgEr (E0308).rt::entry. That resets the outer parse’s state, eats trailing whitespace without the outer Trivia seeing it, and demands end of input. In the probe, a:word_ext b:word_ext on "x \n y" fails with “expected end of input”.Calling across grammars chunk by chunk would need a sub-rule export in winnow-grammar: a pub inner rule generic over the caller’s error type, with no entry, no trailing skip and no end-of-input check. That would be a change to the owner’s crate, so I am not making it.
So, per #1188, the plan is:
tests/compiler and tests/errors program.Step 4b, the helpers, starts now, beginning with the error conversion and the findings.
Generated by Claude Code