Issue #755: ADR-371 D4, the rest: vectors at the build’s width, the mask, a fixed width, the CPU-level query, x86:: instructions, crc32c, prefetch

Open and answer this issue on GitHub.

decision · opened 2026-10-20 · updated 2026-10-20 · 0 comments

What it is

ADR-371 decided the direction: vector lanes, crc32c and prefetch are portable std, and an instruction of one architecture is a safe function of std::backend::x86. D4 left the API to a record of its own. Its first part, searching whole buffers, is decided in ADR-375 (#754). Everything else is open, knowingly: the language needs it, but it is not decided yet.

Why it is left open now

The owner asked whether 1BRC needs these parts to beat or match the fastest Rust program. The measurement in the study says no. Each Rust building block has a counterpart in Nikaia that ADR-375 or the existing word reads already give:

• SIMD row splitting: −64 to −85 instructions a row;
• hashing whole words: −21 to −53;
• the SWAR temperature parse: −17 to −20;
• an own table: −15 to −19.

pext was worse than SWAR. Prefetch cannot be measured by counting. The owner set the priority on the 1BRC builds (October 20, 2026), so this record is to be decided later.

Open questions, with the round’s findings so far

1. Vectors a program holds (level 2), at the build’s width.
   • The width is a constant per --cpu level, as in Highway ScalableTag, C# Vector<T>, Java SPECIES_PREFERRED and Zig suggestVectorLength.
   • The owner rejected a fixed width as the main form: it contradicts the best performance on the target.
   • Open: whether only operations whose result does not depend on the width are offered (lane-local, integer reductions, the mask read only through positions), so that the same output at every width is guaranteed.
2. The mask. Read as positions (first, next, count), not as a number. bits() -> u64 breaks on SVE, which has up to 256 lanes.
3. A fixed width (level 3), only where an algorithm demands it:
   • hashes defined over blocks (xxh3, SHA);
   • 16-entry tables (pshufb, UTF-8 validation, base64), probably only in std::backend;
   • data whose form is fixed (RGBA, UUID).
4. How a program asks its CPU level (ADR-371 D2, --cpu from ADR-002 D6). The proposal is build-time bool constants (x86::has_bmi2) in an ordinary if, with a call refused where it is reachable at a level without the instruction.
5. The first x86:: instructions: pext, pdep and bzhi were proposed. pext is microcoded on AMD Zen 1/2, and in 1BRC it lost to SWAR.
6. crc32c. The proposal is std::hash::crc32c(acc: u32, word: u64) -> u32 plus a bytes form, with a table fallback. In 1BRC it was worth 10–17 instructions a row at v3 on top of a word hash. If fast.nika falls short of the Rust program by about that much, this is the first part to decide.
7. Prefetch. Its signature (prefetch[T](of: ref Array[T], at: i64)) is open, and so is the rule for accepting a change that counting cannot show.
8. From ADR-375 §4: count and rfind over a set, a set’s complement (not), and what text’s search takes (ASCII bytes or scalars, after ADR-374).

Material

The decision basis and the measurement are in the 1BRC study’s scratchpad (the D4 basis, and stable-Rust variants v0–v5 with callgrind counts at x86-64/x86-64-v3 and 1 and 16 units). They should be moved into the study’s record before this issue is decided.

Back to Open Issues