Sqlite

A SQLite database: queries, prepared statements, and their typed outcomes.

This is what makes a visualization address a dataset rather than a file. A query returns rows, not a blob to parse, so the database does the filtering and aggregation and the app draws the answer.

These effects wait. Every one of them is legal in init!, where it blocks startup until the answer is in -- which is what opening a database and loading the first screen of data wants -- and in tasks, where it parks the task while the frame loop keeps drawing. They are refused in update! and render!, with a message naming the effect and the fix.

That refusal holds even for a query that usually finishes in well under a millisecond. What a query costs depends on the page cache, the size of the result, and whether another process holds the write lock -- a contended write can wait for the whole busy_timeout_ms. An effect is classified by what it can do, not by what it usually does.

init! : App.Init(Model, [StoreFailed])
init! = App.init(
    App.default.with_title("scores"),
    |_startup| {
        db = Sqlite.Db.open!("scores.db") ? |_err| StoreFailed
        Sqlite.exec_script!(db, "CREATE TABLE IF NOT EXISTS runs(name TEXT, score INTEGER)") ? |_err| StoreFailed
        rows = Sqlite.query!({ db, query: "SELECT name, score FROM runs", bindings: [] }) ? |_err| StoreFailed
        leader = match List.first(rows) {
            Ok(row) => Sqlite.Row.str(row, "name") ? |_err| StoreFailed
            Err(_) => "nobody"
        }
        Ok({ db, rows, leader })
    },
)

update! = |model, input| {
    if input.devices.key_pressed(KeyEnter) {
        Task.spawn!(
            input,
            || match Sqlite.execute!({
                db: model.db,
                query: "INSERT INTO runs VALUES (:name, :score)",
                bindings: [
                    { name: ":name", value: String("player") },
                    { name: ":score", value: Integer(10) },
                ],
            }) {
                Ok(_outcome) => Saved
                Err(SqliteErr(code, message)) => SaveFailed("${Sqlite.errcode_to_str(code)} (${message})")
                Err(_) => SaveFailed("the platform refused the write")
            },
        )
    }
    Ok(model)
}

Each call fails with a union of its own, and none of those unions is open, so ? cannot merge them: every one is mapped to the app's own StoreFailed before it propagates. examples/sqlite_scores maps them to a Str through Sqlite.errcode_to_str instead, which is what an app that shows the reason on screen wants.

Two shapes reach the same effects. The handle receivers -- Sqlite.Db.open!, Sqlite.Stmt.query!, Sqlite.Stmt.execute! -- run a statement the app prepared once and kept, which is what a query run every frame or every save wants: the parse and the plan are paid at prepare!. The free functions take a record naming the connection and the SQL -- Sqlite.query!({ db, query, bindings }) -- and prepare, run, and discard the statement in one call, which is what a one-off wants: a schema change, a screen loaded once, a query whose text the app just built.

Transactions are SQL, so they are written as SQL. Sqlite.exec_script!(db, "BEGIN"), then the parameterised execute! calls, then Sqlite.exec_script!(db, "COMMIT") -- N inserts through one prepared statement inside one transaction is how a bulk write stays fast, because SQLite otherwise pays a disk sync per statement. "ROLLBACK" undoes the batch when one of those calls fails; a task that ends without committing leaves the transaction open on that connection, so match every BEGIN.

A Db is a reference-counted handle to a host-owned connection. Copying it shares the connection; releasing the last reference closes it, so there is no close to remember. Sqlite.Db.close! exists only for the app that wants to pay a large WAL checkpoint at a moment it chooses rather than whenever the handle happens to go out of scope.

Paths are used as the app gives them, resolved against the process working directory, exactly as Files resolves one, and nothing here is sandboxed. ":memory:" opens a private in-memory database, which is what tests want.

Eight connections and sixty-four prepared statements may be open at once; past that, open! and prepare! answer TooManyConnections and TooManyStatements. A single query may return at most a million cells and sixteen megabytes of text and blobs, which Config.max_result_bytes changes per connection. Those caps are refusals rather than truncations: a result cut short decodes into wrong data rather than into an error.

SQL is the app's to write. This module does not build queries, cache results, or interpret a schema; it binds parameters, runs statements, and hands back rows.

default_config : Config

Read/write/create, a five second busy timeout, sixteen megabytes of result payload.

prepare! : Db, Str => Try(Stmt, PrepareErr)

Compile one statement for repeated use.

The string must hold exactly one statement; several is MultipleStatements, and exec_script! is the call that runs those.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

execute! : { db : Db, query : Str, bindings : List(Binding) } => Try(Outcome, ExecuteErr)

Run one statement that changes data and does not return rows.

This does not occupy a prepared-statement slot: the statement is compiled, run, and finalized inside the call. Use prepare! when the same SQL runs many times.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

query! : { db : Db, query : Str, bindings : List(Binding) } => Try(List(Row), QueryErr)

Run one query and decode every row it returns.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

query_exactly_one! : { db : Db, query : Str, bindings : List(Binding) } => Try(Row, ExactlyOneErr)

Run one query that must return exactly one row.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

exec_script! : Db, Str => Try({  }, [SqliteErr(ErrCode, Str)])

Run every statement in a script, for schema setup and migrations.

Takes no bindings and returns no rows, because a script is SQL the app wrote rather than SQL assembled from input. Anything that needs a parameter, or answers with data, is a query.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

errcode_to_str : ErrCode -> Str

Describe an error code, for a log line or an error screen.

Value : [
    Null,
    Integer(I64),
    Real(F64),
    String(Str),
    Bytes(List(U8)),
]

A value stored in, or read out of, a database column.

ValueKind : [Null, Integer, Real, String, Bytes]

The five things a Value can be, without the payload. Reported by UnexpectedType when a column held something other than what a decoder asked for.

Binding : { name : Str, value : Value }

A named parameter binding. name includes SQLite's parameter prefix, so { name: ":id", value: Integer(42) } binds :id.

A binding whose name the statement does not mention is a SqliteErr(OutOfRange, _) rather than a silent no-op: a typo in a parameter name would otherwise run the query with a NULL nobody asked for.

ErrCode : [
    Error,
    Internal,
    Perm,
    Abort,
    Busy,
    Locked,
    NoMem,
    ReadOnly,
    Interrupt,
    IOErr,
    Corrupt,
    NotFound,
    Full,
    CanNotOpen,
    Protocol,
    Empty,
    Schema,
    TooBig,
    Constraint,
    Mismatch,
    Misuse,
    NoLFS,
    AuthDenied,
    Format,
    OutOfRange,
    NotADatabase,
    Notice,
    Warning,
    Row,
    Done,
    Unknown(I64),
]

SQLite's own result codes.

Extended result codes are reduced to the primary code they extend, so a UNIQUE violation is Constraint rather than a number an app would have to know. The accompanying Str carries SQLite's message, which is where the detail went.

Interrupt is what shutdown looks like from inside a query. Rather than making the window wait for a long statement to finish, the host interrupts every connection with work in flight, and each of those calls answers SqliteErr(Interrupt, _). A task that treats it as a database failure will report one on the way out; a task that is about to be cancelled anyway has nothing to report.

OpenErr : [SqliteErr(ErrCode, Str), TooManyConnections]

Why a connection was not opened.

TooManyConnections means eight are already open; releasing a Db the app no longer needs frees a slot.

PrepareErr : [SqliteErr(ErrCode, Str), TooManyStatements, MultipleStatements]

Why a statement was not prepared.

MultipleStatements is a query string holding more than one statement; exec_script! is the call that runs several. TooManyStatements means sixty-four are already prepared.

QueryErr : [SqliteErr(ErrCode, Str), ResultTooLarge, MultipleStatements]

Why a query produced no rows to decode.

ResultTooLarge is the per-connection cap on a single result, and it is a refusal: nothing is delivered, because a truncated result set is wrong data rather than an error. Narrow the query, or raise Config.max_result_bytes.

ExecuteErr : [SqliteErr(ErrCode, Str), ResultTooLarge, MultipleStatements, RowsReturnedUseQueryInstead]

Why a statement that should change data did not.

RowsReturnedUseQueryInstead is a SELECT handed to execute!, which has nowhere to put the rows.

ExactlyOneErr : [SqliteErr(ErrCode, Str), ResultTooLarge, MultipleStatements, NoRowsReturned, TooManyRowsReturned]

Why a query did not produce exactly one row.

DecodeErr : [NoSuchField(Str), UnexpectedType(ValueKind), IntOutOfBounds]

Why a column did not decode.

NoSuchField names a column the row does not have -- usually a name that does not match the SELECT. UnexpectedType reports what was actually there. IntOutOfBounds is an integer that does not fit the width asked for.

Mode : [ReadWriteCreate, ReadWrite, ReadOnly]

How a connection is opened.

ReadWriteCreate creates the file if it is not there. ReadOnly opens an existing database for reading only, and the host locks that connection down further: schema-rewriting tricks are disabled and ATTACH cannot reach a second file, so a connection opened to visualize someone else's data cannot be talked into writing.

Config : { mode : Mode, busy_timeout_ms : U64, max_result_bytes : U64 }

Per-connection limits.

busy_timeout_ms is how long a statement waits for another process's write lock before answering SqliteErr(Busy, _).

max_result_bytes caps the text and blob payload of one query. A query that would exceed it fails with ResultTooLarge rather than returning part of its rows.

A plain record; build one with { ..Sqlite.default_config, mode: ReadOnly } rather than a chain of with_* calls.

Outcome : { changes : I64, last_insert_rowid : I64 }

What a statement that changes data did.

changes counts the rows the statement inserted, updated, or deleted. last_insert_rowid is the rowid of the most recent successful insert on this connection, and is only meaningful after one.

Db

Sqlite.Db :: # (opaque)

An open database connection.

The host owns the connection; this is a reference-counted handle to it. Keep it in the model, copy it freely, and let the last reference close it.

open! : Str => Try(Db, OpenErr)

Open or create a database under default_config.

The parent directory must already exist. Unlike Files.write_text!, which builds the tree on its way, opening a database does not create one: a database file is normally placed beside an application rather than into a directory the application is inventing, and a mistyped path should be SqliteErr(CanNotOpen, _) rather than a new empty tree. Create it with a write if the app owns that decision.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

open_with! : Str, Config => Try(Db, OpenErr)

Open a database with explicit limits and access mode.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

close! : Db => Try({  }, [SqliteErr(ErrCode, Str)])

Close this connection now rather than when its last handle is released.

Releasing the final handle closes the connection anyway, so this is never required for safety. It exists because closing a database in WAL mode may checkpoint, which is real disk work: doing it here, on a task, keeps it off the frame the last handle happened to be dropped in.

Statements prepared against this connection keep working until they are released; SQLite defers the close until the last one is gone.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

stub : Db

Resource-free connection value for pure tests.

The handle never resolves to an open database, so every call through it fails as SqliteErr(Misuse, _), the same way a call through a released handle does. It exists for the app that keeps a Db in its model, to let a pure expect build that model. Do not use it to test queries or resource lifetime.

Row

:= { names : List(Str), values : List(Value) }

One row of a result, with its column names.

A row is ordinary Roc data by the time an app sees it: the whole result crossed the boundary at once, so reading a column is a lookup rather than an effect. Decode with the receivers below.

is_eq : _

Two rows are equal when their names and values are. Worth having so a decoded result can be compared to an expected one in a test.

names : Row -> List(Str)

This row's column names, in the order the query selected them.

values : Row -> List(Value)

This row's values, in column order.

value : Row, Str -> Try(Value, [NoSuchField(Str), ..])

The value in a named column, still tagged.

str : Row, Str -> Try(Str, DecodeErr)

Decode a TEXT column.

f64 : Row, Str -> Try(F64, DecodeErr)

Decode a REAL column.

i64 : Row, Str -> Try(I64, DecodeErr)

Decode an INTEGER column.

i32 : Row, Str -> Try(I32, DecodeErr)

Decode an INTEGER column that must fit in an I32.

i16 : Row, Str -> Try(I16, DecodeErr)

Decode an INTEGER column that must fit in an I16.

i8 : Row, Str -> Try(I8, DecodeErr)

Decode an INTEGER column that must fit in an I8.

u64 : Row, Str -> Try(U64, DecodeErr)

Decode a non-negative INTEGER column.

u32 : Row, Str -> Try(U32, DecodeErr)

Decode an INTEGER column that must fit in a U32.

u16 : Row, Str -> Try(U16, DecodeErr)

Decode an INTEGER column that must fit in a U16.

u8 : Row, Str -> Try(U8, DecodeErr)

Decode an INTEGER column that must fit in a U8.

for_tests : List(Str), List(Value) -> Row

Build a row without a database, for pure tests.

Row decoding is where an app's own logic usually sits -- which column means what, and what to do when one is missing -- so it is worth an expect. names and values are paired by position.

Stmt

Sqlite.Stmt :: # (opaque)

A compiled statement, ready to run again with different bindings.

Preparing once and running many times skips re-parsing the SQL, which is what a per-frame or per-record write wants. The host owns the compiled statement; the last handle released finalizes it, and the connection it came from stays open at least that long.

execute! : Stmt, List(Binding) => Try(Outcome, ExecuteErr)

Run this statement, which must not return rows.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

query! : Stmt, List(Binding) => Try(List(Row), QueryErr)

Run this statement and decode every row it returns.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

query_exactly_one! : Stmt, List(Binding) => Try(Row, ExactlyOneErr)

Run this statement, which must return exactly one row.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!.

stub : Stmt

Resource-free statement value for pure tests. See Db.stub.