Issue #743: A text position is a text::Pos: find gives a Match, cuts are before/after/between, moves name their unit (ADR-374)

Open and answer this issue on GitHub.

ready · opened 2026-10-20 · updated 2026-10-27 · 1 comment

How it came up

The 1BRC study. Gap 03 of the 1BRC study: a hand-written row loop over a mapped file carries its position from one turn to the next, and the cut is refused.

let mut k = 0
while k < t.len() {
    let e = t.find("\n"; from: k) ?? return
    let row = t[k..<e]            // NK1214: `k` came through the loop
    k = e + "\n".len()
}

The form the help offers, t.bytes[k..<e].text(), checks every row as UTF-8 again, about 130 instructions a row, and the program must handle a NotText error that cannot happen.

Probes on 0.0.1123 (ADR-320 D5 followed where a number came from):

Case	Result	Correct?
a mut local carried through a loop	refused	no, too much refused
e + "xy".len() after find("\n")	accepted	no, too little proved
e - "x".len()	accepted	no, too little proved
t[..<at - 1]	refused	yes

The cuts accepted wrongly stopped the program at run time (byte position 3 is inside a scalar). Every text cut is still checked at run time: nikaia_std::index::text_cut tests both ends with is_char_boundary. So there is no memory or UTF-8 hole, but the static rule proved less than its wording claims.

Decision round (owner, October 20, 2026)

The owner’s direction, in order:

1. A position should be a type that knows its text, lowered to a plain integer.
2. Brackets on a text invite arithmetic, so cut with methods instead.
3. Group the operations so a programmer can hold them in mind: the text, a position, a view.

Survey of the repository:

• Text cuts:
  • 38 in crates/nikaia-std/src/tools/*.nika, all expressible with no change of meaning:
    • 20 before or after a match;
    • 16 after a match’s end: the added literal or number always equals the needle;
    • 2 at the smallest of three finds.
  • 6 in tests/language: indexing.nika tests the mismatched-literal rule, and sequence_shapes.nika:110 cuts at a literal’s length.
  • 2 in Part I.
  • None in examples/, benches/ or guide/.
• find results: none is printed or kept in a field. One is compared with byte numbers from a table (comptime_driver.nika:144).
• Grammars: here and span(p) hand out u32 byte offsets.

Decision: ADR-374 replaces ADR-320 D5 and changes D6.

• text::Pos is a boundary in exactly one text. It takes no arithmetic and is never made from a number.
• find and rfind give a text::Match with start, end and text().
• A text is cut with before, after, between and split_at(Pos). There is no […] on text.
• A position moves with forward/back(scalars:|graphemes:|bytes: n) -> Pos?.
• The compiler tracks which text a position belongs to with the machinery it uses for a view’s sources. A position of another text, or of a text chosen at run time, is NK1214.
• A position cannot be a field (NK1284). text::Cursor holds a text and a position (at, go, until, rest, text).
• A grammar over text hands out Pos and Match.
• A cut at proven positions is not checked at run time.

Spec: Part I 2.6, Part II’s grammar table, Part III (NK1214 reworded, NK1284, NK1285).

When to build (owner, 2026-10-27): the architect decides, by what moves self-hosting forward (#125). It may come after more of the compiler has moved, so the moved code is migrated once. This replaces the earlier “do not start building before the owner says so”.

To build

1. Types:
   • text::Pos, a byte offset below the language, and text::Match in std.
   • s.start() and s.end().
   • find(needle; from: Pos) -> Match? and rfind -> Match? on text. Bytes.find stays i64?.
   • Contracts in std.contracts.
2. Cuts:
   • before, after, between and split_at(Pos) lower to &s[a..b] with no check (D8).
   • between with q < p stops the program.
   • Remove text […]: a bracket on a text becomes NK1214, and the help names before/after/between.
3. Moves:
   • forward and back with exactly one of scalars:, graphemes:, bytes:; otherwise NK1285.
   • bytes: returns null inside a scalar and past either end.
4. Tracking a position’s text:
   • Give a Pos sources, as a view has (Part I 6.6), across parameters, results and branches.
   • Refuse a cut, move or search with a position of another text, or of ref(a | b) (NK1214).
   • Refuse a field of type Pos (NK1284).
   • Replace the number-trail analysis in check_core_places.nika (check_core_places_text_cut, check_core_places_text_position).
5. text::Cursor: Cursor(t), at(), go(p), until(needle) -> ref String?, rest(), text().
6. Grammars over text: here gives a text::Pos and span(p) a text::Match.
7. Migrate:
   • the 38 cuts and every find in tools/*.nika; comptime_driver.nika:144 uses rust.before(p).bytes.len();
   • benches/brc/naive.nika (split_at);
   • tests/language/indexing.nika and sequence_shapes.nika, rewritten to forward;
   • run lower-std.
8. Tests:
   • the loop with a mut position is accepted;
   • m.end.forward(scalars: 4) on "abcdefg" gives abcdef;
   • forward(bytes: 1) inside ä gives null;
   • a position of a used on b → NK1214;
   • ref(a | b) → NK1214;
   • a number in before → NK1214;
   • t[..] → NK1214;
   • a Pos field → NK1284;
   • forward() and forward(scalars: 1, bytes: 1) → NK1285;
   • a cut at proven positions emits no boundary check (look at the lowering);
   • Cursor.until over three rows.
9. Measure: the 1BRC row loop as a cut by Pos against t.bytes[k..<e].text(), counting instructions with callgrind.

Comments

keywan-ghadami, 2026-10-27

Owner’s direction, 2026-10-27. This replaces “do not start building before the owner says so.”

The architect decides when #743 is built and where it goes in the score (docs/priorities.md). The measure is what moves self-hosting forward (#125), not a release by the owner.

It may make sense to build it after more of the compiler has moved into Nikaia. The moved code would then be migrated to text::Pos in one pass (step 7) instead of twice. That is the architect’s call.

The spec (Part I 2.6) already describes text::Match? while std’s find gives an i64?. This gap stays until #743 is built. Lane G of #877 met it (s.before(m.start) is NK1171).

---

Generated by Claude Code

Back to Open Issues