Issue #751: No word is read past the end of a Bytes; a program pads the end itself (ADR-320 D13)

Open and answer this issue on GitHub.

blocked · opened 2026-10-20 · updated 2026-10-20 · 0 comments

How it came up

The 1BRC study needs this. The fast programs read the station name at the start of a row as one 8-byte word (u64_le), then mask off everything from the ; onwards. ADR-320 D12 checks such a read the way it checks the cut b[i..<i+8]. In the last row of a file, a short name often lies less than 8 bytes before the end, so the program stops there. ADR-320 §5 had left the question open: should a word be read past the end, either filled with zeros or from a margin that std guarantees behind a mapping?

Checked:

• fs::map maps exactly the file’s length (crates/nikaia-std/src/fs.rs).
• Linux and Windows fill the rest of the last page with zeros. A file whose size is a multiple of 4096 has no such rest, and a read beyond it gets SIGBUS.
• simdjson requires a padded input type. Rust and the fastest Java solutions handle the end as a path of their own.
• Building a padded copy works today. Bytes(v) does not exist yet; it is decided in #718.

Options put to the owner:

• A: stay as D12 is.
• B: four zero-filling reads.
• C: a margin fs::map guarantees.
• D: B now, C undecided.

Decision (owner, October 20, 2026)

None of them. B grows std by four documented methods for one loop per program, and C is a platform promise the type would have to carry, against #718. The owner wants a manual way out, even if it is longer, without growing std:

• No word is read past the end.
• A loop that reads whole words up to its last row runs over a copy of the last bytes, with zeros behind them, as Bytes(tail).

Recorded in ADR-320 D13, with a short pointer in Part I’s text on word reads.

To build (blocked on #718, which builds Bytes(v))

1. Add a test in tests/language/src/. It runs one function over a Bytes to its last row: once over the data up to a cut, and once over the padded copy of the rest. Its output must equal a byte-by-byte reading.
2. The 1BRC study’s fast.nika uses this pattern instead of a separate path for the end.

Back to Open Issues