# qubes-sqlite

A **SQLite engine in Rust, compiled to wasm for the browser** — the
structured-state engine for the [qubepods](https://github.com/qubepods/qubepods)
agent harness, packaged as a Continuum **library qube**.

It is built on [**Turso**](https://github.com/tursodatabase/turso) (the Rust
rewrite of SQLite) via its `turso_core` crate, and runs inside an OPFS Web
Worker. Per the harness design of record (`qubepods/docs/agent-harness.md`),
this is the queryable store for the agent loop: checkpoints, the run
transcript/event log, the tool-call + QAD audit, token/budget counters, FTS,
and vector recall.

## Status — v0 (in-memory)

- ✅ Real `turso_core` engine, in-memory (`MemoryIO`, `:memory:`).
- ✅ Portable engine (`src/engine.rs`) — host-agnostic, native `cargo test`.
- ✅ Browser surface (`src/lib.rs`, `wasm_bindgen`) — a `Sqlite` class callable
  from a Web Worker.
- ✅ WIT contract (`wit/sqlite.wit`) — the Continuum identity/surface.
- ⏳ **Next:** durable persistence — snapshot the database image to the host's
  blob store (OPFS cache ⇄ R2 source-of-truth), via the `wasi:blobstore`-style
  import sketched in the WIT world. See the harness doc's persistence model.

## API (browser / Web Worker)

```js
import init, { Sqlite } from "./pkg/qubes_sqlite.js";

await init();                       // load the wasm
const db = new Sqlite();            // fresh in-memory database

db.execBatch("CREATE TABLE t(a INTEGER, b TEXT);");

const { changes, lastInsertRowid } =
  db.run("INSERT INTO t VALUES (?, ?)", [1, "hi"]);

const { columns, rows } =
  db.query("SELECT a, b FROM t WHERE a = ?", [1]);
// columns: ["a", "b"]
// rows:    [[1, "hi"]]
```

Parameters accept `null`, number, bigint, boolean, string, and `Uint8Array`.
Result cells come back as native JS values; integers outside ±2^53 are returned
as `BigInt` to avoid precision loss. Blobs come back as `Uint8Array`.

The methods are synchronous — fine in a Web Worker (the engine is in-memory, so
no I/O blocks), and exactly what a worker-side message handler wants.

## Build

```sh
# Native tests (the portable engine, real turso_core):
cargo test -p qubes-sqlite

# Browser bundle → ./pkg (cargo wasm32 + wasm-bindgen --target web):
./build.sh

# Regression check the built bundle in wasm (node + a real browser Worker):
node examples/verify.mjs
```

`examples/verify.mjs` runs the engine two ways: directly in Node's wasm runtime
(joins, transactions, constraints, NULLs, BigInt) and inside a real Chromium
Web Worker via Playwright (skipped, not failed, if Playwright/Chromium is
absent). Exit code is non-zero only if a layer that actually ran had a failure.

`build.sh` pins the `wasm-bindgen` CLI to the crate version and installs it on
demand. Output (`pkg/qubes_sqlite.js` + `pkg/qubes_sqlite_bg.wasm`) is a
gitignored build artifact — load it straight into a worker.

## Why `turso_core`, not `turso`

The high-level `turso` SDK kit pulls `tracing-appender` (file logging via
symlinks), which does not compile to wasm. `turso_core` is the engine itself
and builds cleanly for `wasm32-unknown-unknown`. We keep `fs` enabled (the
crate's always-compiled paths assume it) but drive a `MemoryIO` at runtime, so
there are no filesystem syscalls.

## The clock patch (`vendor/turso_core`)

turso_core 0.6.1 calls `std::time::Instant::now` / `SystemTime::now` directly,
and those **panic** on `wasm32-unknown-unknown` (`time not implemented on this
platform`) — the Turso-beta browser gap the harness design anticipated. All of
turso_core's time routes through two functions in `io/clock.rs`, so the fix is
surgical: we vendor the crate (`vendor/turso_core`, wired via the workspace
`[patch.crates-io]`) and swap `std::time` → [`web-time`](https://crates.io/crates/web-time)
(`performance.now` / `Date.now`) at those two sites plus the SQL `date('now')`
path. **Native builds are byte-for-byte upstream** — only the wasm time reads
change (guarded by `#[cfg(target_arch = "wasm32")]`). When upstream Turso ships
a browser clock, the vendor dir and the patch stanza drop out.

Verified: the built bundle runs INSERT/SELECT with bound params, blobs, JSON
functions, and `date('now')` end-to-end in a JS runtime (see the smoke test in
the commit that landed this).

## Publishing to the Continuum — status

Not yet published. Two things gate it, both external to this qube:

1. **A q64 Continuum publish PAT.** The registry write endpoints
   (`POST /v1/qubes/{name}`, `POST /v1/auth/token`) require
   `Authorization: Bearer <qube_pat_…>`; minting one needs a q64 identity
   login. The Cloudflare / R2 / GitHub credentials a cloud session carries do
   **not** authenticate to the q64 registry, and the prod Continuum's own
   D1/R2 must not be written behind the API's back.
2. **Foreign-component support in the Continuum.** Today the registry is
   q64-source-centric — `qube publish` compiles the `.q` entry and the server
   re-indexes effects from source. This qube is a prebuilt Rust component (no
   `.q` entry), which is exactly the "source-not-wasm" / OCI gap tracked in
   qubepods `TODO.md` §"Continuum (OCI) serves the engine wasm" and q64
   `todo.md` §"Continuum — OCI compliance".

Until both land, this qube lives here and is consumed **local-path** from
qubepods (`apps/app`), per the harness design's sequencing (prototype local →
Continuum-publish once `qube wac` / OCI distribution land).

## Bundle size

`pkg/qubes_sqlite_bg.wasm` is **~5.3 MB raw / ~1.8 MB gzipped** — and gzip (or
brotli) is what the browser actually transfers, once, then caches. That is
turso_core's footprint: a full Rust SQLite engine is larger than C SQLite
(sql.js/wa-sqlite ≈ 1 MB raw) because it ships the whole Rust engine, not the
~600 KB hand-tuned C amalgamation.

How we got there (from the initial 6.7 MB):
- `opt-level = "z"` + `lto = "fat"` + `strip` + `panic = "abort"` (workspace
  release profile).
- Dropped turso_core features this qube doesn't use (`series`). `time`, `json`,
  `uuid`, `fs`, `pure-rust-crypto` stay (the first three are used SQL surface;
  `uuid` is referenced in non-gated turso_core code; `fs` is assumed by its
  always-compiled paths — we still run `MemoryIO`).
- `wasm-opt -Oz` (binaryen) — `build.sh` runs it when present, enabling the
  post-MVP wasm features rustc emits (bulk-memory, sign-ext, …) so older
  binaryen doesn't reject the module.

Further trimming would mean forking more of turso_core (its `incremental`/`mvcc`
modules compile unconditionally) — deferred; the gzip transfer size is already
reasonable for a one-time, cached Worker load.

## Layout

```
sqlite/
├── qube.json5      # Continuum manifest (type: library, WIT world)
├── wit/sqlite.wit  # the WIT contract — the qube's surface + identity
├── src/
│   ├── engine.rs   # portable engine over turso_core (native-testable)
│   └── lib.rs      # wasm-bindgen browser surface (Sqlite class)
├── tests/engine.rs # native integration tests
├── examples/       # Web Worker host (worker.js) + demo page (index.html)
├── vendor/         # turso_core, vendored + clock-patched for wasm
├── build.sh        # browser bundle builder
└── README.md
```
