Open and answer this issue on GitHub.
performance, note · opened 2026-10-06 · updated 2026-10-06 · 1 comment
After the MVP. Nothing here is decided, and nothing here is built for the MVP. This issue keeps the ideas from the decision round on #114 / ADR-299, so that the rules for rows and target structs, decided now, leave these optimisations open.
Take a chain of many-to-one links: accounts.user → users.address → addresses.country → countries { id, name, code }. A query asks for every account together with its country’s name and code:
struct Land { name: String, code: String }
struct Konto { id: i64, iban: String, c: Land }
let q = dsl sqlite(schema: APP) -> Konto {
SELECT a.id, a.iban, c.name, c.code
FROM accounts AS a
JOIN users AS u ON a.user = u.id
JOIN addresses AS d ON u.address = d.id
JOIN countries AS c ON d.country = c.id
} eodJOIN gives one row per account, so the number of rows does not multiply."Deutschland" 100 000 times once the program collects them.for loop that drops each row holds only one row at a time.ORMs solve this with an identity map. Nikaia has no ORM (ADR-299 D5), so the question is whether the language can still let a program avoid the repetition.
A program can run two queries and assemble the result itself with Shared[T] (Part I 6.2):
struct Land { id: i64, name: String, code: String }
struct KontoZeile { id: i64, iban: String, land: i64 }
struct Konto { id: i64, iban: String, land: Shared[Land] }
let nach_id = laender.execute(db).map(|l| (l.id, Shared(l))).collect()
let alle = konten.execute(db)
.map(|z| Konto { id: z.id, iban: z.iban, land: nach_id[z.land] })
.collect()missing_fields stays what it is: a missing ? field becomes null.Shared(l).Shared[T] on a field or a group of a target structstruct Konto {
id: i64,
iban: String,
c: Shared[Land], // a group: equal countries once
stadt: Shared[String], // one column, no key: equal names once
}The compiler-written conversion (ADR-299 D8) would merge equal values within one execute.
Why value equality is enough, without a notion of record identity. A Shared[T] cannot be changed, and as far as the specification goes a program cannot ask whether two handles point at the same memory (there is no ptr_eq). So merging is invisible: it saves memory and means nothing.
A primary or unique key from the schema would then only speed up the comparison (compare the id instead of every field), not define it. The DDL grammar reads CREATE TABLE and so sees PRIMARY KEY and UNIQUE, but meta::column(name, type) does not pass them on. A call such as meta::key(table, columns) would.
Costs:
Seq ends. A loop that drops rows would then hold every distinct value of a column, which is worse than without Shared for a column of mostly unique values.
execute. A program-long interner, like the grammars’ Symbol table (Part II 10.6), never frees anything, which is wrong for rows from a database.Today a group field of type Shared[T] is refused because it is not a struct. Turning that refusal into a permission later changes no existing program, so the target-struct rules do not block this.
Where merging happens matters less than when:
The second needs the Connection trait to hand back a row as a view into the driver’s buffer, valid until the next row. That is what SQLite (sqlite3_column_text, valid until the next sqlite3_step) and libpq hand back anyway. Without Shared it also saves one copy per cell: today it would be into a db::Value and then into the field.
Open:
db::Value“. Whether a db::Value owns its memory is not defined, and std::db does not exist yet.rusqlite’s ValueRef<'stmt>.| Database | How a row arrives | "Deutschland" for 100 000 accounts |
|---|---|---|
| SQLite (in process) | a pointer into SQLite’s buffer, valid until the next row | rewritten 100 000 times into the same buffer |
| PostgreSQL | one DataRow message per row, each column as length and bytes; no dictionary in the standard protocol, as far as known |
100 000 times over the wire |
| MySQL | as PostgreSQL; the wire can be compressed (zlib/zstd) | fewer bytes on the wire, 100 000 times after unpacking |
| Columnar (Arrow Flight SQL, DuckDB, ClickHouse) | blocks per column, text often as a dictionary plus indices | once, plus 100 000 small numbers |
With row protocols a driver can find repeats only by hashing, as the conversion can. Only columnar protocols deliver the merge for free, and using that needs a trait that hands back blocks rather than rows. That is a separate question about the shape of std::db.
The normalisation lives only in the database file. The JOIN undoes it on the server.
extern "C" (ADR-302)examples/sqlite/main.nika binds SQLite directly; crates/nikaia/tests/foreign_pointers.rs builds and runs it. What being in process offers:
sqlite3_column_text and sqlite3_column_bytes: pointer and length).
foreign::CStr that is copied (D8).CStr has no lifetime the language knows. One kept past the next sqlite3_step points at freed memory, and ADR-302 checks nothing there.sqlite3_prepare_v2 on first execute, then sqlite3_reset). This is the driver’s work and needs no decision.sqlite3_prepare against an empty database built from schema.sql, plus sqlite3_column_table_name / sqlite3_column_origin_name) would know exactly which table each column comes from.
std’s Rust half run at build time. The driver’s grammar parses the SQL itself.INTEGER can hold text unless the table is STRICT (3.37+). The conversion has to check each value’s type at run time (sqlite3_column_type), whatever the schema said at build time.Shared[T] in a target struct is refused today. That refusal may later become a permission.db::Value and of the Connection trait’s rows (owned values or views) is not fixed by the MVP in a way that excludes views.Why SQLite may not check a statement while the program is built (asked by the owner).
The idea would be: the driver’s grammar opens an in-memory database from schema.sql, calls sqlite3_prepare_v2 on the statement, and reads sqlite3_column_origin_name / _table_name / _decltype. Four things stand in the way, in order:
touches.sqlite3_prepare_v2 is refused at build time by D14 already (NK1152).system() or unlink(). The compiler cannot tell sqlite3_prepare_v2 from system(): both are names in an extern "C" block. That is Rust’s build.rs / proc-macro hole, a supply-chain path onto the developer’s machine and CI.examples/sqlite links the machine’s libsqlite3, and versions accept different SQL (RETURNING since 3.35, STRICT since 3.37). One project would build on one machine and not on another, and the build cache key (ADR-321 D4) does not contain the system library.What it costs: the grammar has to replicate SQLite’s name and type resolution.
sqlite3_prepare_v2 reports the error at the first execute at run time.Neither gives a wrong value.
The one way it could be allowed, not decided: the compiler ships and vouches for one fixed SQLite, as it does for std’s Rust half (ADR-020 D5). The price:
Generated by Claude Code