Issue #539: Formats and protocols are official packages (json, toml, csv, html, http1), not std; listed in std and taken in by their first use (ADR-338)

Open and answer this issue on GitHub.

libraries, grammar, ready · opened 2026-10-07 · updated 2026-11-03 · 3 comments

Records: Part III’s list of key modules (std::json: serialization using compile-time code generation), ADR-331 (@json::Name("createdAt") is its first example), ADR-014 (half of std is Nikaia).

Where it stands. One of the README’s five promises is formats are part of the language. The examples prove it at program level: examples/json (a tree of unbounded depth) and examples/config (INI with its own whitespace) are grammars and test themselves. std holds no format at all. std::json is named in Part III and not built, and CSV and TOML aren’t named anywhere. A program that reads a config file or a spreadsheet export starts by writing a grammar for a format everyone already agrees on.

Proposal. Each of these is a grammar in std’s Nikaia half, so that:

• it’s the showcase: the reference grammar for the format is the one a user reads to learn grammars;
• it’s dogfooding of the kind the self-hosting road does for the checker: every format moved into std is a test of the grammar protocol’s corners (CSV’s quoted fields and "", TOML’s dotted keys and tables, JSON’s escapes and zero-copy strings);
• the typed half reads @json::Name and its siblings through ADR-331, so a struct is read from JSON without a hand-written walk.

Order and what it waits on

1. std::json: already specified, examples/json as the starting point. Needs a grammar a package offers, #520 (ADR-296 D25), and how a dsl block names it, #485. #518 shows the lowering’s side.
2. std::csv: the smallest; mostly a question of the row type, so it shares #477’s matching rules (a block’s rows are a declared struct, matched by name).
3. std::toml: nikaia.toml is read by the toolchain through the toml crate today (ADR-250’s adapters), so this one also takes an adapter off the self-hosting road.

To decide. Whether CSV and TOML belong in std at all, or in packages beside it once there’s a registry (#99). The argument for std: the toolchain needs TOML itself, and each one is a reference grammar. Also whether a format module is only the grammar plus the attribute-driven reader, or also a writer.

Comments

keywan-ghadami, 2026-10-09

Decided (owner, 2026-10-09): not in std. Official packages, taken in by their first use

How it went. The proposal put JSON, CSV and TOML in std. The owner asked first whether a large std makes every program larger, then whether it makes the modules inflexible, and pointed out that std::html, std::http1 and std::json had entered std without anyone deciding it.

Measured (release build, 2026-10-09). A program carries only what it calls:

• a hello world has 519 674 bytes of code and no symbol of std::html or std::http1;
• with html::escape it has one, and 2 244 bytes more.

Size is no argument either way. What decided it is that a module in std moves with the language’s version, so its mistakes stay (Java’s Date; Go’s encoding/json until a second version). Deno (@std/json, separately versioned) and Kotlin (kotlinx) keep formats official but versioned on their own.

Side finding. The same hello world binary is 57 MB on disk, of which about 0.5 MB is code. The rest is debug information, in a release build.

Where it is written: ADR-338 and Part III 17, 17.3 (0.0.714).

• std is the core the language stands on.
• json, toml, csv, html and http1 are official packages:
  • on crates.io as nikaia_json and so on (the prefix rule, ADR-336 D2);
  • shipped with the toolchain, so they work offline;
  • each versioned on its own.
• std carries the list of official packages with their versions. A use json that nothing else names takes it in at that version and records it in nikaia.lock.
• The toolchain’s own Nikaia uses toml as any program would.

Still open (ADR-338 §6):

• whether each format package also writes. The recommendation is that JSON and CSV read and write, and TOML reads first, since writing with comments and order kept is its own design question;
• whether std::db’s protocol traits stay in std.

To build

1. The list and the automatic use.
   • std carries the list (name → shipped version).
   • The resolver takes in a listed package for a use that [dependencies] and the project’s directories do not name.
   • The resolved version goes into nikaia.lock.

   Tests:

   • use json with no manifest entry builds offline;
   • a [dependencies] entry wins over the list;
   • an unlisted, undeclared name is still refused.
2. html and http1 out of std. Move them into packages under the toolchain’s shipped packages, and change the callers (dsl html lowering, examples/http, ADR-017’s escape call). Outputs stay the same.
3. json as a grammar package, starting from examples/json. It waits on #520 and #485.
4. csv, sharing #477’s row rules.
5. toml (reading), then the toolchain’s nikaia.toml reader on it instead of the toml crate.

---

Generated by Claude Code

keywan-ghadami, 2026-10-09

Amended (owner, 2026-10-09, 0.0.715): json, toml and csv all read and write (ADR-338 D5). Writing makes a new text from a value. Editing an existing text in place while keeping its comments and order is left open.

To build, step 5 becomes: toml reads and writes, then the toolchain’s nikaia.toml reader uses it.

---

Generated by Claude Code

keywan-ghadami, 2026-10-13

json’s shape is decided in ADR-344 and its build order is #604: first packages/json/ by path dependency (Value, read[T], write[T], a build-time schema); this issue’s ADR-338 D3 and the grammar export (#520, #485) come last.

---

Generated by Claude Code

Back to Open Issues