openvibers/openvibe.sdk Commits
0.18.0: moving a service off better-sqlite3 becomes mostly mechanical. Ambient transactions (on by default): inside db.tx(fn), plain db calls, db.prepare statements and library stores handed db join the running transaction through AsyncLocalStorage; a db.tx inside is a savepoint; a promise the transaction did not await runs on the pool afterwards; db.detached(fn) leaves on purpose; db.inTransaction() tells. db.prepare(text) gives async better-sqlite3-shaped statements (get, all, run with changes, rows and lastInsertRowid from RETURNING, pluck) with ? and @name/:name parameters compiled to $n, leaving strings, identifiers, dollar quotes, comments and :: casts alone. importSqlite verification treats JSON null and SQL NULL as equal (found in the Tips migration). The migration guide gains the short path and the Tips lessons. test/db-ambient.test.js passes on PGlite and through PgBouncer; 39/39 test files.
Changes
@@@ -5,6 +5,25 @@ release may change an API and says so here.++++## 0.18.0 (2026-09-28)++++This release makes the move from better-sqlite3 mostly mechanical: call sites keep their shape and gain `await`.++++- **`openvibe-sdk/db`: ambient transactions** (on by default; `createDb({ ambient: false })` turns them off).++ - Inside `db.tx(fn)`, plain `db.*` calls join the running transaction through AsyncLocalStorage. So do `db.prepare` statements and library stores that were handed `db`. Their writes are visible inside the transaction, roll back with it, and never wait on it.++ - A `db.tx` inside is a savepoint.++ - A promise the transaction did not await runs on the pool once it has ended.++ - `db.detached(fn)` leaves the transaction on purpose, and `db.inTransaction()` tells whether code is inside one.++ - The handle `fn` receives works as before.++- **`db.prepare(text)`**: async statements shaped like better-sqlite3's.++ - Methods: `get` (a row or `undefined`), `all`, `run` (`{ changes, rows, lastInsertRowid }`; the id needs `RETURNING`) and `pluck()`.++ - Parameters: `?`, or `@name`/`:name` from one object (a repeated name is one parameter).++ - The compile to `$n` leaves `'…'` strings, `"…"` identifiers, dollar quotes, comments and `::` casts alone.++- Tests: `test/db-ambient.test.js`, on PGlite and through PgBouncer.++- `importSqlite` verification treats a JSON `null` and SQL NULL as equal. JSON text `'null'` imported into `jsonb` had failed its checksum (found in the Tips migration).++- The guide gains a section of lessons from the Tips migration: `SUM(bigint)` returns a numeric string, test roles through PgBouncer, byte-exact replays, idempotency claims, NUL and surrogate cleaning, and pub/sub without Valkey.++## 0.17.0 (2026-09-28)- **`openvibe-sdk/geo`** (browser and server): `createGeoClient(client, { samples?, timeoutMs? })` finds the closest OpenVibe node (roadmap WS-X1 task 5).@@@ -3,7 +3,7 @@"stage": "alpha","code": true,"wave": "W2",-- "version": "0.17.0",++ "version": "0.18.0","plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30","deployed": false,"deployment": "none: a package released as git tags; services and apps pin a tag",@@@ -1,4 +1,4 @@--// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.17.0, MIT). Do not edit.++// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.18.0, MIT). Do not edit.// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects, geo). No server-only code: no service tokens, JWT verification, events, identity./* eslint-disable */@@@ -41,7 +41,7 @@ const { paginate, offsetPager } = __load(7);const semver = __load(6);/** The package version (test/version.test.js keeps it equal to package.json). */--const SDK_VERSION = '0.17.0';++const SDK_VERSION = '0.18.0';module.exports = {createClient,@@@ -40,6 +40,7 @@ Create `migrations/0001_initial.sql` (`-- phase: expand`). It reproduces today's| `TEXT` holding JSON | `jsonb` (and stop calling `JSON.parse` on it: rows come back parsed) || `TEXT` (UUID, ULID, slug) | `text` with the same `CHECK`s; `citext` or a `lower()` index where the code compared `COLLATE NOCASE` || `REAL` | `double precision`; money stays integer cents (`bigint`), never float |++| `SUM(int)`, `SUM(bigint)`, `AVG(…)` | PostgreSQL returns `numeric`, which the layer gives as a **string**. Cast sums that must stay numbers: `SUM(amount)::bigint` (found in the Tips migration) || `BLOB` | `bytea` || `CHECK (x IN (…))` | the same `CHECK` (or an enum type only if it never changes) |@@@ -51,13 +52,19 @@ Then:## 3. The code: async from end to end++**The short path (SDK ≥ 0.18).**++- Keep the statements' shape. `db.prepare(text)` returns async `get`, `all`, `run` and `pluck`, with the same `?` and `@name` parameters, so `q.x.get(id)` becomes `await q.x.get(id)`.++- Keep functions that run inside a transaction calling `db`. With ambient transactions, `db.transaction(fn)()` becomes `await db.tx(async () => fn())`, and every `db` call inside joins it.++- What still needs a person is the SQL itself (the table below) and `lastInsertRowid`, which needs `RETURNING`.++++| better-sqlite3 | openvibe-sdk/db ||---|---|| `db.prepare(q).get(a)` | `await db.maybe(sql\`… ${a}\`)` (or `one` when absence is a bug) || `db.prepare(q).all(a)` | `await db.many(sql\`…\`)` || `db.prepare(q).run(a)` / `.changes` | `await db.exec(sql\`…\`)` returns the count || `info.lastInsertRowid` | `INSERT … RETURNING id` with `db.one` |--| `db.transaction(fn)()` | `await db.tx(async (t) => { … })`: pass `t` to every call inside, never `db` |++| `db.transaction(fn)()` | `await db.tx(async (t) => { … })`: with ambient transactions (default) `db` calls inside join it; `t` also works || `INSERT OR IGNORE` | `INSERT … ON CONFLICT DO NOTHING` || `INSERT OR REPLACE` | `INSERT … ON CONFLICT (key) DO UPDATE SET …` (and `excluded.col` works) || `datetime('now')`, `strftime(…)`, `CURRENT_TIMESTAMP` | `now()`, `to_char(…)`, `extract(epoch from …)` |@@@ -125,3 +132,12 @@ It changes nothing in the SQLite file.- the test counts;- `STATUS.json` (the database) and the README (Depends on: PostgreSQL and Valkey);- the roadmap plan's WS-X2 migration list.++++## 7. Lessons from the first migrations++++- **Tips (2026-09-28)** is the reference: its `postgres` branch has the schema, the importer script, `test/helpers/db.js` and the multi-process integration test. Copy them.++- **Test roles on the shared containers.** End each per-run role's backends before dropping it, or PgBouncer keeps idle server connections and exhausts the container. `test/helpers/db.js` does this.++- **Replays compare bytes.** Keep a replayed response (idempotency records) as `text`: `jsonb` reorders keys.++- **Claim an Idempotency-Key before running the handler**, with `INSERT … ON CONFLICT DO NOTHING RETURNING`. That way two concurrent requests with the same key run once.++- **Clean text PostgreSQL refuses.** SQLite text can hold NUL and unpaired surrogates. Clean them in the importer's `map`, and report each column that needed it.++- **pub/sub without Valkey.** `createPubSub` has no local fallback when Valkey is down. A hub that must keep delivering on one process falls back to local delivery itself.@@@ -1,12 +1,12 @@{"name": "openvibe-sdk",-- "version": "0.17.0",++ "version": "0.18.0","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "openvibe-sdk",-- "version": "0.17.0",++ "version": "0.18.0","license": "MIT","devDependencies": {"@electric-sql/pglite": "^0.5.8",@@@ -1,6 +1,6 @@{"name": "openvibe-sdk",-- "version": "0.17.0",++ "version": "0.18.0","description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.","main": "index.js","types": "types/index.d.ts",@@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');const semver = require('./semver');/** The package version (test/version.test.js keeps it equal to package.json). */--const SDK_VERSION = '0.17.0';++const SDK_VERSION = '0.18.0';module.exports = {createClient,@@@ -13,10 +13,21 @@* await db.tx(async (t) => { … }, { isolation: 'serializable' }); // 40001/40P01 retried* await db.migrate({ dir: path.join(__dirname, 'migrations') }); // with DATABASE_DIRECT_URL*++ * Ambient transactions (default on): inside db.tx(fn), plain db.* calls (and db.prepare statements, and library stores++ * handed `db`) join the running transaction through AsyncLocalStorage, so code converted from better-sqlite3 need not++ * thread the handle through every function. A db.tx inside it is a savepoint. Work that outlives the transaction (a++ * promise it did not await) runs on the pool once it has ended; db.detached(fn) runs fn outside it on purpose.++ * createDb({ ambient: false }) turns this off. The handle fn receives works either way.++ *++ * const q = db.prepare('SELECT * FROM pages WHERE id = ?'); // better-sqlite3-shaped, async: get / all / run++ * const page = await q.get(id);++ ** Rules the pooler imposes (ADR-007 2026-09-24 rule 1): no session state between transactions (no session SET,* no LISTEN, no advisory locks held across transactions) on DATABASE_URL. Migrations use DATABASE_DIRECT_URL.*/++const { AsyncLocalStorage } = require('node:async_hooks');const { sql, isSql } = require('./sql');++const { prepare } = require('./prepare');const ISOLATION = { 'read committed': 'READ COMMITTED', 'repeatable read': 'REPEATABLE READ', serializable: 'SERIALIZABLE' };const RETRYABLE = new Set(['40001', '40P01']); // serialization failure, deadlock@@@ -113,13 +124,24 @@ function createDb(o = {}) {}}++ const als = o.ambient === false ? null : new AsyncLocalStorage();++ const current = () => {++ const cur = als && als.getStore();++ return cur && !cur.done ? cur : null;++ };++ const inScope = (scope, fn) => (als ? als.run(scope, fn) : fn());++async function tx(fn, opts = {}, conn = null, depth = 0) {++ if (!conn) {++ const cur = current();++ if (cur) { conn = cur.conn; depth = cur.depth; } // ambient: a savepoint in the running transaction++ }if (conn) {// Nested: a savepoint inside the running transaction.const name = `sp_${depth}`;await timed(conn, `SAVEPOINT ${name}`, []);try {-- const out = await fn(txApi(conn, depth + 1));++ const out = await inScope({ conn, depth: depth + 1, done: false }, () => fn(txApi(conn, depth + 1)));await timed(conn, `RELEASE SAVEPOINT ${name}`, []);return out;} catch (err) { await timed(conn, `ROLLBACK TO SAVEPOINT ${name}`, []).catch(() => {}); throw err; }@@@ -130,12 +152,15 @@ function createDb(o = {}) {for (let attempt = 0; ; attempt++) {const c = await adapter.acquire();stats.tx++;++ const scope = { conn: c, depth: 1, done: false };try {await timed(c, `BEGIN ISOLATION LEVEL ${iso}${opts.readOnly ? ' READ ONLY' : ''}`, []);-- const out = await fn(txApi(c, 1));++ const out = await inScope(scope, () => fn(txApi(c, 1)));++ scope.done = true;await timed(c, 'COMMIT', []);return out;} catch (err) {++ scope.done = true;await timed(c, 'ROLLBACK', []).catch(() => {});const code = err && (err.code || (err.cause && err.cause.code));if (RETRYABLE.has(code) && attempt < retries) {@@@ -151,12 +176,20 @@ function createDb(o = {}) {function txApi(conn, depth) {const t = queryApi((text, values) => timed(conn, text, values), {});t.tx = (fn) => tx(fn, {}, conn, depth);++ t.sql = sql;++ t.prepare = (text) => prepare(t, text);return t;}-- const db = queryApi((text, values) => timed(null, text, values), {});++ const db = queryApi((text, values) => { const cur = current(); return timed(cur ? cur.conn : null, text, values); }, {});db.sql = sql;db.tx = (fn, opts) => tx(fn, opts);++ /** Async statements shaped like better-sqlite3's (get / all / run / pluck), with ? and @name / :name parameters. */++ db.prepare = (text) => prepare(db, text);++ /** True while this code runs inside db.tx (ambient mode). */++ db.inTransaction = () => Boolean(current());++ /** Run fn outside any ambient transaction (its queries take a pool connection). */++ db.detached = (fn) => (als ? als.exit(fn) : fn());db.store = adapter.store;db.stats = () => ({ ...stats, pool: adapter.pool() });/** Readiness: a real round trip, and which store answered (never claimed from configuration). */nnew file mode 100644@@@ -0,0 +1,97 @@++'use strict';++/**++ * db.prepare(text): async statements shaped like better-sqlite3's, so a service moving from SQLite changes its++ * call sites by adding `await` (the SQL itself still has to be PostgreSQL). Parameters are written as SQLite code++ * wrote them and compiled once to PostgreSQL's $n:++ * ? positional, in order++ * @name :name named, bound from one plain object; a name used twice is one parameter++ * Question marks, @ and : inside '…' strings, "…" identifiers, dollar-quoted bodies and comments are left alone,++ * and so are :: casts.++ *++ * const q = db.prepare('SELECT * FROM pages WHERE space_id = @space AND slug = @slug');++ * await q.get({ space, slug }); // row or undefined++ * await q.all(...); // rows++ * await q.run(...); // { changes, rows, lastInsertRowid } (lastInsertRowid needs RETURNING)++ * await q.pluck().get(...); // the first column's value++ */++++function compile(text) {++ let out = '';++ const names = new Map(); // name -> $n++ let positional = 0;++ let n = 0;++ let i = 0;++ const len = text.length;++ while (i < len) {++ const c = text[i];++ const next = text[i + 1];++ if (c === "'" || c === '"') {++ const j = skipQuoted(text, i, c);++ out += text.slice(i, j); i = j; continue;++ }++ if (c === '-' && next === '-') { const j = text.indexOf('\n', i); const e = j < 0 ? len : j; out += text.slice(i, e); i = e; continue; }++ if (c === '/' && next === '*') { const j = text.indexOf('*/', i + 2); const e = j < 0 ? len : j + 2; out += text.slice(i, e); i = e; continue; }++ if (c === '$' && /[A-Za-z_$]/.test(next || '')) {++ const m = /^\$([A-Za-z_][A-Za-z0-9_]*)?\$/.exec(text.slice(i));++ if (m) { const end = text.indexOf(m[0], i + m[0].length); const e = end < 0 ? len : end + m[0].length; out += text.slice(i, e); i = e; continue; }++ }++ if (c === ':' && next === ':') { out += '::'; i += 2; continue; }++ if (c === '?') { n++; positional++; out += `$${n}`; i++; continue; }++ if ((c === '@' || c === ':') && /[A-Za-z_]/.test(next || '')) {++ const m = /^[A-Za-z_][A-Za-z0-9_]*/.exec(text.slice(i + 1));++ const name = m[0];++ if (!names.has(name)) { n++; names.set(name, n); }++ out += `$${names.get(name)}`;++ i += 1 + name.length;++ continue;++ }++ out += c; i++;++ }++ if (positional && names.size) throw new TypeError('openvibe-sdk/db prepare: use either ? or named parameters in one statement, not both');++ return { text: out, names: names.size ? [...names.entries()].sort((a, b) => a[1] - b[1]).map(([k]) => k) : null, count: n };++}++++function skipQuoted(text, i, q) {++ let j = i + 1;++ while (j < text.length) {++ if (text[j] === q) { if (text[j + 1] === q) { j += 2; continue; } return j + 1; }++ j++;++ }++ return j;++}++++const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v) && !(v instanceof Date) && !Buffer.isBuffer(v) && !ArrayBuffer.isView(v);++++function bind(stmt, args) {++ if (stmt.names) {++ const o = args[0];++ if (args.length !== 1 || !isPlainObject(o)) throw new TypeError('openvibe-sdk/db prepare: this statement takes one object of named parameters');++ return stmt.names.map((k) => {++ if (!(k in o)) throw new TypeError(`openvibe-sdk/db prepare: missing parameter @${k}`);++ return o[k] === undefined ? null : o[k];++ });++ }++ const flat = args.length === 1 && Array.isArray(args[0]) ? args[0] : args;++ if (flat.length !== stmt.count) throw new TypeError(`openvibe-sdk/db prepare: expected ${stmt.count} parameter(s), got ${flat.length}`);++ return flat.map((v) => (v === undefined ? null : v));++}++++function prepare(h, text, { pluck = false } = {}) {++ if (typeof text !== 'string') throw new TypeError('openvibe-sdk/db prepare: pass the SQL text');++ const stmt = compile(text);++ const first = (row) => (row ? row[Object.keys(row)[0]] : undefined);++ const api = {++ source: stmt.text,++ async get(...args) { const r = await h.query(stmt.text, bind(stmt, args)); const row = r.rows[0]; return pluck ? first(row) : row; },++ async all(...args) { const r = await h.query(stmt.text, bind(stmt, args)); return pluck ? r.rows.map(first) : r.rows; },++ async run(...args) {++ const r = await h.query(stmt.text, bind(stmt, args));++ const row = r.rows && r.rows[0];++ return { changes: r.rowCount, rows: r.rows || [], lastInsertRowid: row ? first(row) : undefined };++ },++ pluck(on = true) { return prepare(h, text, { pluck: on }); },++ };++ return api;++}++++module.exports = { prepare, compile };@@@ -77,7 +77,11 @@ function sortKeys(x) {}function canonical(v, udt) {if (v === null || v === undefined) return '\\N';-- if (udt === 'json' || udt === 'jsonb') return JSON.stringify(sortKeys(typeof v === 'string' ? JSON.parse(v) : v));++ if (udt === 'json' || udt === 'jsonb') {++ // JSON null and SQL NULL compare equal: a jsonb 'null' reads back as JS null, like a missing value.++ const parsed = typeof v === 'string' ? JSON.parse(v) : v;++ return parsed === null ? '\\N' : JSON.stringify(sortKeys(parsed));++ }if (udt === 'bytea') return Buffer.isBuffer(v) ? v.toString('hex') : Buffer.from(String(v)).toString('hex');if (udt === 'timestamptz' || udt === 'timestamp') return new Date(v).toISOString();if (udt.startsWith('_')) return JSON.stringify(v);nnew file mode 100644@@@ -0,0 +1,105 @@++'use strict';++/**++ * Ambient transactions and db.prepare: inside db.tx, plain db calls join the transaction (they see its writes,++ * roll back with it, and never wait on it); a db.tx inside it is a savepoint; a promise the transaction did not++ * await runs on the pool after it ends; db.detached leaves it on purpose. db.prepare compiles ? and @name/:name++ * to $n, leaves strings, identifiers, comments and :: casts alone, and gives get/all/run/pluck. PGlite always;++ * PostgreSQL through PgBouncer when OV_TEST_PG_URL is set.++ */++const assert = require('node:assert/strict');++const { run } = require('./helpers');++const { createDb, sql } = require('../src/db');++const { compile } = require('../src/db/prepare');++++function cases(label, open) {++ return [++ [`${label}: plain db calls inside db.tx join it; a rollback takes them back`, async () => {++ const { db, done } = await open();++ try {++ const ins = db.prepare('INSERT INTO items (name, qty) VALUES (?, ?) RETURNING id');++ const count = db.prepare('SELECT count(*)::int AS n FROM items').pluck();++ await assert.rejects(db.tx(async () => {++ await ins.run('a', 1);++ assert.equal(await count.get(), 1, 'the transaction sees its own write through db');++ assert.equal(db.inTransaction(), true);++ throw new Error('boom');++ }), /boom/);++ assert.equal(await count.get(), 0, 'rolled back');++ const id = await db.tx(async () => (await ins.run('b', 2)).lastInsertRowid);++ assert.equal(typeof id, 'number');++ assert.equal(db.inTransaction(), false);++ assert.equal(await count.get(), 1);++ } finally { await done(); }++ }],++ [`${label}: a db.tx inside is a savepoint; its failure undoes only itself`, async () => {++ const { db, done } = await open();++ try {++ const ins = db.prepare('INSERT INTO items (name, qty) VALUES (@name, @qty)');++ await db.tx(async () => {++ await ins.run({ name: 'outer', qty: 1 });++ await assert.rejects(db.tx(async () => { await ins.run({ name: 'inner', qty: 2 }); throw new Error('inner'); }), /inner/);++ await db.tx(async () => ins.run({ name: 'kept', qty: 3 }));++ });++ assert.deepEqual(await db.prepare('SELECT name FROM items ORDER BY id').pluck().all(), ['outer', 'kept']);++ } finally { await done(); }++ }],++ [`${label}: work the transaction did not await runs on the pool afterwards; detached leaves on purpose`, async () => {++ const { db, done } = await open();++ try {++ let late;++ await db.tx(async () => {++ await db.exec(sql`INSERT INTO items (name, qty) VALUES ('t', 1)`);++ late = new Promise((r) => setTimeout(r, 20)).then(() => db.value(sql`SELECT count(*)::int FROM items`));++ });++ assert.equal(await late, 1, 'ran after the commit, on the pool');++ if (label !== 'pglite') {++ await db.tx(async () => {++ await db.exec(sql`INSERT INTO items (name, qty) VALUES ('u', 1)`);++ assert.equal(await db.detached(() => db.value(sql`SELECT count(*)::int FROM items`)), 1, 'detached does not see the uncommitted row');++ });++ }++ } finally { await done(); }++ }],++ [`${label}: prepare binds positional, named (reused) and arrays; missing parameters are refused`, async () => {++ const { db, done } = await open();++ try {++ await db.prepare('INSERT INTO items (name, qty) VALUES (?, ?), (?, ?)').run(['x', 5, 'y', 7]);++ const q = db.prepare("SELECT name FROM items WHERE (@name::text IS NULL OR name = @name) AND qty >= :min AND name <> 'a?b' ORDER BY name");++ assert.deepEqual((await q.all({ name: null, min: 6 })).map((r) => r.name), ['y']);++ assert.deepEqual((await q.all({ name: 'x', min: 0 })).map((r) => r.name), ['x']);++ assert.equal(await db.prepare('SELECT name FROM items WHERE qty = ?').get(99), undefined, 'no row is undefined, as in better-sqlite3');++ assert.equal((await db.prepare('UPDATE items SET qty = qty + 1 WHERE qty > ?').run(0)).changes, 2);++ await assert.rejects(q.all({ name: 'x' }), /missing parameter @min/);++ await assert.rejects(db.prepare('SELECT ? AS a').get(), /expected 1 parameter/);++ } finally { await done(); }++ }],++ ];++}++++const tests = [['compile: strings, identifiers, comments, dollar quotes and casts are left alone', () => {++ assert.throws(() => compile('SELECT ?, @a'), /either \? or named/);++ const c = compile(`SELECT 'it''s ?', "@x", $$ :y ? $$, @a::int, :b, @a -- ?\n/* @c */`);++ assert.equal(c.text, `SELECT 'it''s ?', "@x", $$ :y ? $$, $1::int, $2, $1 -- ?\n/* @c */`);++ assert.deepEqual(c.names, ['a', 'b']);++ assert.equal(compile('SELECT ?, ?').text, 'SELECT $1, $2');++}]];++++const schema = 'CREATE TABLE items (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name text NOT NULL, qty integer NOT NULL)';++tests.push(...cases('pglite', async () => {++ const db = createDb({ pglite: true });++ await db.query(schema);++ return { db, done: () => db.close() };++}));++if (process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL) {++ tests.push(...cases('postgresql+pgbouncer', async () => {++ const owner = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL });++ await owner.query('DROP TABLE IF EXISTS items');++ await owner.query(schema);++ await owner.query('GRANT ALL ON items TO PUBLIC').catch(() => {});++ await owner.close();++ const db = createDb({ url: process.env.OV_TEST_PG_URL, max: 4 });++ return { db, done: () => db.close() };++ }));++} else console.log('db ambient through PgBouncer: skipped (OV_TEST_PG_URL not set; scripts/test-services.sh up)');++++run(tests);@@@ -27,16 +27,34 @@ export interface Queryable {value<T = unknown>(q: Sql | string, values?: unknown[]): Promise<T | null>;exec(q: Sql | string, values?: unknown[]): Promise<number>;}++/** An async statement shaped like better-sqlite3's: ? or @name / :name parameters. */++export interface Statement {++ readonly source: string;++ /** The first row, or undefined. */++ get<T = Row>(...params: unknown[]): Promise<T | undefined>;++ all<T = Row>(...params: unknown[]): Promise<T[]>;++ /** lastInsertRowid is the first column of the first returned row (needs RETURNING). */++ run(...params: unknown[]): Promise<{ changes: number; rows: Row[]; lastInsertRowid: unknown }>;++ /** The same statement returning each row's first column. */++ pluck(on?: boolean): Statement;++}export interface Tx extends Queryable {++ sql: SqlTag;/** A nested savepoint: rolls back alone when fn throws. */tx<T>(fn: (t: Tx) => Promise<T>): Promise<T>;++ prepare(text: string): Statement;}export interface TxOptions { isolation?: 'read committed' | 'repeatable read' | 'serializable'; retries?: number; readOnly?: boolean }export interface MigrateResult { applied: { id: string; name: string; phase: string; ms: number }[]; pending: { id: string; name: string; phase: string }[]; held: { id: string; reason: string }[] }export interface Db extends Queryable {sql: SqlTag;readonly store: 'postgresql' | 'pglite';++ /** Inside fn, plain db calls join the transaction (ambient mode, the default); a db.tx inside is a savepoint. */tx<T>(fn: (t: Tx) => Promise<T>, opts?: TxOptions): Promise<T>;++ prepare(text: string): Statement;++ inTransaction(): boolean;++ /** Run fn outside any ambient transaction. */++ detached<T>(fn: () => T): T;ready(): Promise<{ ok: true; detail: { store: string; pool: { total: number; idle: number; waiting: number } } } | { ok: false; error: string }>;migrate(o: { dir: string; windowDays?: number; dryRun?: boolean; now?: () => number; log?: { log(msg: string): void } }): Promise<MigrateResult>;stats(): { queries: number; errors: number; slow: number; retries: number; tx: number; pool: { total: number; idle: number; waiting: number } };@@@ -53,6 +71,8 @@ export interface CreateDbOptions {slowMs?: number;log?: { warn(msg: string): void; error(msg: string): void };registry?: object;++ /** Default true: db calls inside db.tx join it (AsyncLocalStorage). */++ ambient?: boolean;}export function createDb(opts?: CreateDbOptions): Db;export declare class DbError extends Error { code?: string; detail?: string; constraint?: string; table?: string; column?: string; statement?: string; cause?: unknown }
@@ -5,6 +5,25 @@ release may change an API and says so here.
@@ -5,6 +5,25 @@ release may change an API and says so here.
+
+## 0.18.0 (2026-09-28)
+
+This release makes the move from better-sqlite3 mostly mechanical: call sites keep their shape and gain `await`.
+
+- **`openvibe-sdk/db`: ambient transactions** (on by default; `createDb({ ambient: false })` turns them off).
+ - Inside `db.tx(fn)`, plain `db.*` calls join the running transaction through AsyncLocalStorage. So do `db.prepare` statements and library stores that were handed `db`. Their writes are visible inside the transaction, roll back with it, and never wait on it.
+ - A `db.tx` inside is a savepoint.
+ - A promise the transaction did not await runs on the pool once it has ended.
+ - `db.detached(fn)` leaves the transaction on purpose, and `db.inTransaction()` tells whether code is inside one.
+ - The handle `fn` receives works as before.
+- **`db.prepare(text)`**: async statements shaped like better-sqlite3's.
+ - Methods: `get` (a row or `undefined`), `all`, `run` (`{ changes, rows, lastInsertRowid }`; the id needs `RETURNING`) and `pluck()`.
+ - Parameters: `?`, or `@name`/`:name` from one object (a repeated name is one parameter).
+ - The compile to `$n` leaves `'…'` strings, `"…"` identifiers, dollar quotes, comments and `::` casts alone.
+- Tests: `test/db-ambient.test.js`, on PGlite and through PgBouncer.
+- `importSqlite` verification treats a JSON `null` and SQL NULL as equal. JSON text `'null'` imported into `jsonb` had failed its checksum (found in the Tips migration).
+- The guide gains a section of lessons from the Tips migration: `SUM(bigint)` returns a numeric string, test roles through PgBouncer, byte-exact replays, idempotency claims, NUL and surrogate cleaning, and pub/sub without Valkey.
+
## 0.17.0 (2026-09-28)
## 0.17.0 (2026-09-28)
- **`openvibe-sdk/geo`** (browser and server): `createGeoClient(client, { samples?, timeoutMs? })` finds the closest OpenVibe node (roadmap WS-X1 task 5).
- **`openvibe-sdk/geo`** (browser and server): `createGeoClient(client, { samples?, timeoutMs? })` finds the closest OpenVibe node (roadmap WS-X1 task 5).
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
"stage": "alpha",
"stage": "alpha",
"code": true,
"code": true,
"wave": "W2",
"wave": "W2",
- "version": "0.17.0",
+ "version": "0.18.0",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30",
"deployed": false,
"deployed": false,
"deployment": "none: a package released as git tags; services and apps pin a tag",
"deployment": "none: a package released as git tags; services and apps pin a tag",
@@ -1,4 +1,4 @@
@@ -1,4 +1,4 @@
-// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.17.0, MIT). Do not edit.
+// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.18.0, MIT). Do not edit.
// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces
// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces
// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects, geo). No server-only code: no service tokens, JWT verification, events, identity.
// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects, geo). No server-only code: no service tokens, JWT verification, events, identity.
/* eslint-disable */
/* eslint-disable */
@@ -41,7 +41,7 @@ const { paginate, offsetPager } = __load(7);
@@ -41,7 +41,7 @@ const { paginate, offsetPager } = __load(7);
const semver = __load(6);
const semver = __load(6);
/** The package version (test/version.test.js keeps it equal to package.json). */
/** The package version (test/version.test.js keeps it equal to package.json). */
-const SDK_VERSION = '0.17.0';
+const SDK_VERSION = '0.18.0';
module.exports = {
module.exports = {
createClient,
createClient,
@@ -40,6 +40,7 @@ Create `migrations/0001_initial.sql` (`-- phase: expand`). It reproduces today's
@@ -40,6 +40,7 @@ Create `migrations/0001_initial.sql` (`-- phase: expand`). It reproduces today's
| `TEXT` holding JSON | `jsonb` (and stop calling `JSON.parse` on it: rows come back parsed) |
| `TEXT` holding JSON | `jsonb` (and stop calling `JSON.parse` on it: rows come back parsed) |
| `TEXT` (UUID, ULID, slug) | `text` with the same `CHECK`s; `citext` or a `lower()` index where the code compared `COLLATE NOCASE` |
| `TEXT` (UUID, ULID, slug) | `text` with the same `CHECK`s; `citext` or a `lower()` index where the code compared `COLLATE NOCASE` |
| `REAL` | `double precision`; money stays integer cents (`bigint`), never float |
| `REAL` | `double precision`; money stays integer cents (`bigint`), never float |
+| `SUM(int)`, `SUM(bigint)`, `AVG(…)` | PostgreSQL returns `numeric`, which the layer gives as a **string**. Cast sums that must stay numbers: `SUM(amount)::bigint` (found in the Tips migration) |
| `BLOB` | `bytea` |
| `BLOB` | `bytea` |
| `CHECK (x IN (…))` | the same `CHECK` (or an enum type only if it never changes) |
| `CHECK (x IN (…))` | the same `CHECK` (or an enum type only if it never changes) |
@@ -51,13 +52,19 @@ Then:
@@ -51,13 +52,19 @@ Then:
## 3. The code: async from end to end
## 3. The code: async from end to end
+**The short path (SDK ≥ 0.18).**
+- Keep the statements' shape. `db.prepare(text)` returns async `get`, `all`, `run` and `pluck`, with the same `?` and `@name` parameters, so `q.x.get(id)` becomes `await q.x.get(id)`.
+- Keep functions that run inside a transaction calling `db`. With ambient transactions, `db.transaction(fn)()` becomes `await db.tx(async () => fn())`, and every `db` call inside joins it.
+- What still needs a person is the SQL itself (the table below) and `lastInsertRowid`, which needs `RETURNING`.
+
+
| better-sqlite3 | openvibe-sdk/db |
| better-sqlite3 | openvibe-sdk/db |
|---|---|
|---|---|
| `db.prepare(q).get(a)` | `await db.maybe(sql\`… ${a}\`)` (or `one` when absence is a bug) |
| `db.prepare(q).get(a)` | `await db.maybe(sql\`… ${a}\`)` (or `one` when absence is a bug) |
| `db.prepare(q).all(a)` | `await db.many(sql\`…\`)` |
| `db.prepare(q).all(a)` | `await db.many(sql\`…\`)` |
| `db.prepare(q).run(a)` / `.changes` | `await db.exec(sql\`…\`)` returns the count |
| `db.prepare(q).run(a)` / `.changes` | `await db.exec(sql\`…\`)` returns the count |
| `info.lastInsertRowid` | `INSERT … RETURNING id` with `db.one` |
| `info.lastInsertRowid` | `INSERT … RETURNING id` with `db.one` |
-| `db.transaction(fn)()` | `await db.tx(async (t) => { … })`: pass `t` to every call inside, never `db` |
+| `db.transaction(fn)()` | `await db.tx(async (t) => { … })`: with ambient transactions (default) `db` calls inside join it; `t` also works |
| `INSERT OR IGNORE` | `INSERT … ON CONFLICT DO NOTHING` |
| `INSERT OR IGNORE` | `INSERT … ON CONFLICT DO NOTHING` |
| `INSERT OR REPLACE` | `INSERT … ON CONFLICT (key) DO UPDATE SET …` (and `excluded.col` works) |
| `INSERT OR REPLACE` | `INSERT … ON CONFLICT (key) DO UPDATE SET …` (and `excluded.col` works) |
| `datetime('now')`, `strftime(…)`, `CURRENT_TIMESTAMP` | `now()`, `to_char(…)`, `extract(epoch from …)` |
| `datetime('now')`, `strftime(…)`, `CURRENT_TIMESTAMP` | `now()`, `to_char(…)`, `extract(epoch from …)` |
@@ -125,3 +132,12 @@ It changes nothing in the SQLite file.
@@ -125,3 +132,12 @@ It changes nothing in the SQLite file.
- the test counts;
- the test counts;
- `STATUS.json` (the database) and the README (Depends on: PostgreSQL and Valkey);
- `STATUS.json` (the database) and the README (Depends on: PostgreSQL and Valkey);
- the roadmap plan's WS-X2 migration list.
- the roadmap plan's WS-X2 migration list.
+
+## 7. Lessons from the first migrations
+
+- **Tips (2026-09-28)** is the reference: its `postgres` branch has the schema, the importer script, `test/helpers/db.js` and the multi-process integration test. Copy them.
+- **Test roles on the shared containers.** End each per-run role's backends before dropping it, or PgBouncer keeps idle server connections and exhausts the container. `test/helpers/db.js` does this.
+- **Replays compare bytes.** Keep a replayed response (idempotency records) as `text`: `jsonb` reorders keys.
+- **Claim an Idempotency-Key before running the handler**, with `INSERT … ON CONFLICT DO NOTHING RETURNING`. That way two concurrent requests with the same key run once.
+- **Clean text PostgreSQL refuses.** SQLite text can hold NUL and unpaired surrogates. Clean them in the importer's `map`, and report each column that needed it.
+- **pub/sub without Valkey.** `createPubSub` has no local fallback when Valkey is down. A hub that must keep delivering on one process falls back to local delivery itself.
@@ -1,12 +1,12 @@
@@ -1,12 +1,12 @@
{
{
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.17.0",
+ "version": "0.18.0",
"lockfileVersion": 3,
"lockfileVersion": 3,
"requires": true,
"requires": true,
"packages": {
"packages": {
"": {
"": {
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.17.0",
+ "version": "0.18.0",
"license": "MIT",
"license": "MIT",
"devDependencies": {
"devDependencies": {
"@electric-sql/pglite": "^0.5.8",
"@electric-sql/pglite": "^0.5.8",
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
{
{
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.17.0",
+ "version": "0.18.0",
"description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.",
"description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.",
"main": "index.js",
"main": "index.js",
"types": "types/index.d.ts",
"types": "types/index.d.ts",
@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');
@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');
const semver = require('./semver');
const semver = require('./semver');
/** The package version (test/version.test.js keeps it equal to package.json). */
/** The package version (test/version.test.js keeps it equal to package.json). */
-const SDK_VERSION = '0.17.0';
+const SDK_VERSION = '0.18.0';
module.exports = {
module.exports = {
createClient,
createClient,
@@ -13,10 +13,21 @@
@@ -13,10 +13,21 @@
* await db.tx(async (t) => { … }, { isolation: 'serializable' }); // 40001/40P01 retried
* await db.tx(async (t) => { … }, { isolation: 'serializable' }); // 40001/40P01 retried
* await db.migrate({ dir: path.join(__dirname, 'migrations') }); // with DATABASE_DIRECT_URL
* await db.migrate({ dir: path.join(__dirname, 'migrations') }); // with DATABASE_DIRECT_URL
*
*
+ * Ambient transactions (default on): inside db.tx(fn), plain db.* calls (and db.prepare statements, and library stores
+ * handed `db`) join the running transaction through AsyncLocalStorage, so code converted from better-sqlite3 need not
+ * thread the handle through every function. A db.tx inside it is a savepoint. Work that outlives the transaction (a
+ * promise it did not await) runs on the pool once it has ended; db.detached(fn) runs fn outside it on purpose.
+ * createDb({ ambient: false }) turns this off. The handle fn receives works either way.
+ *
+ * const q = db.prepare('SELECT * FROM pages WHERE id = ?'); // better-sqlite3-shaped, async: get / all / run
+ * const page = await q.get(id);
+ *
* Rules the pooler imposes (ADR-007 2026-09-24 rule 1): no session state between transactions (no session SET,
* Rules the pooler imposes (ADR-007 2026-09-24 rule 1): no session state between transactions (no session SET,
* no LISTEN, no advisory locks held across transactions) on DATABASE_URL. Migrations use DATABASE_DIRECT_URL.
* no LISTEN, no advisory locks held across transactions) on DATABASE_URL. Migrations use DATABASE_DIRECT_URL.
*/
*/
+const { AsyncLocalStorage } = require('node:async_hooks');
const { sql, isSql } = require('./sql');
const { sql, isSql } = require('./sql');
+const { prepare } = require('./prepare');
const ISOLATION = { 'read committed': 'READ COMMITTED', 'repeatable read': 'REPEATABLE READ', serializable: 'SERIALIZABLE' };
const ISOLATION = { 'read committed': 'READ COMMITTED', 'repeatable read': 'REPEATABLE READ', serializable: 'SERIALIZABLE' };
const RETRYABLE = new Set(['40001', '40P01']); // serialization failure, deadlock
const RETRYABLE = new Set(['40001', '40P01']); // serialization failure, deadlock
@@ -113,13 +124,24 @@ function createDb(o = {}) {
@@ -113,13 +124,24 @@ function createDb(o = {}) {
}
}
}
}
+ const als = o.ambient === false ? null : new AsyncLocalStorage();
+ const current = () => {
+ const cur = als && als.getStore();
+ return cur && !cur.done ? cur : null;
+ };
+ const inScope = (scope, fn) => (als ? als.run(scope, fn) : fn());
+
async function tx(fn, opts = {}, conn = null, depth = 0) {
async function tx(fn, opts = {}, conn = null, depth = 0) {
+ if (!conn) {
+ const cur = current();
+ if (cur) { conn = cur.conn; depth = cur.depth; } // ambient: a savepoint in the running transaction
+ }
if (conn) {
if (conn) {
// Nested: a savepoint inside the running transaction.
// Nested: a savepoint inside the running transaction.
const name = `sp_${depth}`;
const name = `sp_${depth}`;
await timed(conn, `SAVEPOINT ${name}`, []);
await timed(conn, `SAVEPOINT ${name}`, []);
try {
try {
- const out = await fn(txApi(conn, depth + 1));
+ const out = await inScope({ conn, depth: depth + 1, done: false }, () => fn(txApi(conn, depth + 1)));
await timed(conn, `RELEASE SAVEPOINT ${name}`, []);
await timed(conn, `RELEASE SAVEPOINT ${name}`, []);
return out;
return out;
} catch (err) { await timed(conn, `ROLLBACK TO SAVEPOINT ${name}`, []).catch(() => {}); throw err; }
} catch (err) { await timed(conn, `ROLLBACK TO SAVEPOINT ${name}`, []).catch(() => {}); throw err; }
@@ -130,12 +152,15 @@ function createDb(o = {}) {
@@ -130,12 +152,15 @@ function createDb(o = {}) {
for (let attempt = 0; ; attempt++) {
for (let attempt = 0; ; attempt++) {
const c = await adapter.acquire();
const c = await adapter.acquire();
stats.tx++;
stats.tx++;
+ const scope = { conn: c, depth: 1, done: false };
try {
try {
await timed(c, `BEGIN ISOLATION LEVEL ${iso}${opts.readOnly ? ' READ ONLY' : ''}`, []);
await timed(c, `BEGIN ISOLATION LEVEL ${iso}${opts.readOnly ? ' READ ONLY' : ''}`, []);
- const out = await fn(txApi(c, 1));
+ const out = await inScope(scope, () => fn(txApi(c, 1)));
+ scope.done = true;
await timed(c, 'COMMIT', []);
await timed(c, 'COMMIT', []);
return out;
return out;
} catch (err) {
} catch (err) {
+ scope.done = true;
await timed(c, 'ROLLBACK', []).catch(() => {});
await timed(c, 'ROLLBACK', []).catch(() => {});
const code = err && (err.code || (err.cause && err.cause.code));
const code = err && (err.code || (err.cause && err.cause.code));
if (RETRYABLE.has(code) && attempt < retries) {
if (RETRYABLE.has(code) && attempt < retries) {
@@ -151,12 +176,20 @@ function createDb(o = {}) {
@@ -151,12 +176,20 @@ function createDb(o = {}) {
function txApi(conn, depth) {
function txApi(conn, depth) {
const t = queryApi((text, values) => timed(conn, text, values), {});
const t = queryApi((text, values) => timed(conn, text, values), {});
t.tx = (fn) => tx(fn, {}, conn, depth);
t.tx = (fn) => tx(fn, {}, conn, depth);
+ t.sql = sql;
+ t.prepare = (text) => prepare(t, text);
return t;
return t;
}
}
- const db = queryApi((text, values) => timed(null, text, values), {});
+ const db = queryApi((text, values) => { const cur = current(); return timed(cur ? cur.conn : null, text, values); }, {});
db.sql = sql;
db.sql = sql;
db.tx = (fn, opts) => tx(fn, opts);
db.tx = (fn, opts) => tx(fn, opts);
+ /** Async statements shaped like better-sqlite3's (get / all / run / pluck), with ? and @name / :name parameters. */
+ db.prepare = (text) => prepare(db, text);
+ /** True while this code runs inside db.tx (ambient mode). */
+ db.inTransaction = () => Boolean(current());
+ /** Run fn outside any ambient transaction (its queries take a pool connection). */
+ db.detached = (fn) => (als ? als.exit(fn) : fn());
db.store = adapter.store;
db.store = adapter.store;
db.stats = () => ({ ...stats, pool: adapter.pool() });
db.stats = () => ({ ...stats, pool: adapter.pool() });
/** Readiness: a real round trip, and which store answered (never claimed from configuration). */
/** Readiness: a real round trip, and which store answered (never claimed from configuration). */
new file mode 100644
new file mode 100644
@@ -0,0 +1,97 @@
@@ -0,0 +1,97 @@
+'use strict';
+/**
+ * db.prepare(text): async statements shaped like better-sqlite3's, so a service moving from SQLite changes its
+ * call sites by adding `await` (the SQL itself still has to be PostgreSQL). Parameters are written as SQLite code
+ * wrote them and compiled once to PostgreSQL's $n:
+ * ? positional, in order
+ * @name :name named, bound from one plain object; a name used twice is one parameter
+ * Question marks, @ and : inside '…' strings, "…" identifiers, dollar-quoted bodies and comments are left alone,
+ * and so are :: casts.
+ *
+ * const q = db.prepare('SELECT * FROM pages WHERE space_id = @space AND slug = @slug');
+ * await q.get({ space, slug }); // row or undefined
+ * await q.all(...); // rows
+ * await q.run(...); // { changes, rows, lastInsertRowid } (lastInsertRowid needs RETURNING)
+ * await q.pluck().get(...); // the first column's value
+ */
+
+function compile(text) {
+ let out = '';
+ const names = new Map(); // name -> $n
+ let positional = 0;
+ let n = 0;
+ let i = 0;
+ const len = text.length;
+ while (i < len) {
+ const c = text[i];
+ const next = text[i + 1];
+ if (c === "'" || c === '"') {
+ const j = skipQuoted(text, i, c);
+ out += text.slice(i, j); i = j; continue;
+ }
+ if (c === '-' && next === '-') { const j = text.indexOf('\n', i); const e = j < 0 ? len : j; out += text.slice(i, e); i = e; continue; }
+ if (c === '/' && next === '*') { const j = text.indexOf('*/', i + 2); const e = j < 0 ? len : j + 2; out += text.slice(i, e); i = e; continue; }
+ if (c === '$' && /[A-Za-z_$]/.test(next || '')) {
+ const m = /^\$([A-Za-z_][A-Za-z0-9_]*)?\$/.exec(text.slice(i));
+ if (m) { const end = text.indexOf(m[0], i + m[0].length); const e = end < 0 ? len : end + m[0].length; out += text.slice(i, e); i = e; continue; }
+ }
+ if (c === ':' && next === ':') { out += '::'; i += 2; continue; }
+ if (c === '?') { n++; positional++; out += `$${n}`; i++; continue; }
+ if ((c === '@' || c === ':') && /[A-Za-z_]/.test(next || '')) {
+ const m = /^[A-Za-z_][A-Za-z0-9_]*/.exec(text.slice(i + 1));
+ const name = m[0];
+ if (!names.has(name)) { n++; names.set(name, n); }
+ out += `$${names.get(name)}`;
+ i += 1 + name.length;
+ continue;
+ }
+ out += c; i++;
+ }
+ if (positional && names.size) throw new TypeError('openvibe-sdk/db prepare: use either ? or named parameters in one statement, not both');
+ return { text: out, names: names.size ? [...names.entries()].sort((a, b) => a[1] - b[1]).map(([k]) => k) : null, count: n };
+}
+
+function skipQuoted(text, i, q) {
+ let j = i + 1;
+ while (j < text.length) {
+ if (text[j] === q) { if (text[j + 1] === q) { j += 2; continue; } return j + 1; }
+ j++;
+ }
+ return j;
+}
+
+const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v) && !(v instanceof Date) && !Buffer.isBuffer(v) && !ArrayBuffer.isView(v);
+
+function bind(stmt, args) {
+ if (stmt.names) {
+ const o = args[0];
+ if (args.length !== 1 || !isPlainObject(o)) throw new TypeError('openvibe-sdk/db prepare: this statement takes one object of named parameters');
+ return stmt.names.map((k) => {
+ if (!(k in o)) throw new TypeError(`openvibe-sdk/db prepare: missing parameter @${k}`);
+ return o[k] === undefined ? null : o[k];
+ });
+ }
+ const flat = args.length === 1 && Array.isArray(args[0]) ? args[0] : args;
+ if (flat.length !== stmt.count) throw new TypeError(`openvibe-sdk/db prepare: expected ${stmt.count} parameter(s), got ${flat.length}`);
+ return flat.map((v) => (v === undefined ? null : v));
+}
+
+function prepare(h, text, { pluck = false } = {}) {
+ if (typeof text !== 'string') throw new TypeError('openvibe-sdk/db prepare: pass the SQL text');
+ const stmt = compile(text);
+ const first = (row) => (row ? row[Object.keys(row)[0]] : undefined);
+ const api = {
+ source: stmt.text,
+ async get(...args) { const r = await h.query(stmt.text, bind(stmt, args)); const row = r.rows[0]; return pluck ? first(row) : row; },
+ async all(...args) { const r = await h.query(stmt.text, bind(stmt, args)); return pluck ? r.rows.map(first) : r.rows; },
+ async run(...args) {
+ const r = await h.query(stmt.text, bind(stmt, args));
+ const row = r.rows && r.rows[0];
+ return { changes: r.rowCount, rows: r.rows || [], lastInsertRowid: row ? first(row) : undefined };
+ },
+ pluck(on = true) { return prepare(h, text, { pluck: on }); },
+ };
+ return api;
+}
+
+module.exports = { prepare, compile };
@@ -77,7 +77,11 @@ function sortKeys(x) {
@@ -77,7 +77,11 @@ function sortKeys(x) {
}
}
function canonical(v, udt) {
function canonical(v, udt) {
if (v === null || v === undefined) return '\\N';
if (v === null || v === undefined) return '\\N';
- if (udt === 'json' || udt === 'jsonb') return JSON.stringify(sortKeys(typeof v === 'string' ? JSON.parse(v) : v));
+ if (udt === 'json' || udt === 'jsonb') {
+ // JSON null and SQL NULL compare equal: a jsonb 'null' reads back as JS null, like a missing value.
+ const parsed = typeof v === 'string' ? JSON.parse(v) : v;
+ return parsed === null ? '\\N' : JSON.stringify(sortKeys(parsed));
+ }
if (udt === 'bytea') return Buffer.isBuffer(v) ? v.toString('hex') : Buffer.from(String(v)).toString('hex');
if (udt === 'bytea') return Buffer.isBuffer(v) ? v.toString('hex') : Buffer.from(String(v)).toString('hex');
if (udt === 'timestamptz' || udt === 'timestamp') return new Date(v).toISOString();
if (udt === 'timestamptz' || udt === 'timestamp') return new Date(v).toISOString();
if (udt.startsWith('_')) return JSON.stringify(v);
if (udt.startsWith('_')) return JSON.stringify(v);
new file mode 100644
new file mode 100644
@@ -0,0 +1,105 @@
@@ -0,0 +1,105 @@
+'use strict';
+/**
+ * Ambient transactions and db.prepare: inside db.tx, plain db calls join the transaction (they see its writes,
+ * roll back with it, and never wait on it); a db.tx inside it is a savepoint; a promise the transaction did not
+ * await runs on the pool after it ends; db.detached leaves it on purpose. db.prepare compiles ? and @name/:name
+ * to $n, leaves strings, identifiers, comments and :: casts alone, and gives get/all/run/pluck. PGlite always;
+ * PostgreSQL through PgBouncer when OV_TEST_PG_URL is set.
+ */
+const assert = require('node:assert/strict');
+const { run } = require('./helpers');
+const { createDb, sql } = require('../src/db');
+const { compile } = require('../src/db/prepare');
+
+function cases(label, open) {
+ return [
+ [`${label}: plain db calls inside db.tx join it; a rollback takes them back`, async () => {
+ const { db, done } = await open();
+ try {
+ const ins = db.prepare('INSERT INTO items (name, qty) VALUES (?, ?) RETURNING id');
+ const count = db.prepare('SELECT count(*)::int AS n FROM items').pluck();
+ await assert.rejects(db.tx(async () => {
+ await ins.run('a', 1);
+ assert.equal(await count.get(), 1, 'the transaction sees its own write through db');
+ assert.equal(db.inTransaction(), true);
+ throw new Error('boom');
+ }), /boom/);
+ assert.equal(await count.get(), 0, 'rolled back');
+ const id = await db.tx(async () => (await ins.run('b', 2)).lastInsertRowid);
+ assert.equal(typeof id, 'number');
+ assert.equal(db.inTransaction(), false);
+ assert.equal(await count.get(), 1);
+ } finally { await done(); }
+ }],
+ [`${label}: a db.tx inside is a savepoint; its failure undoes only itself`, async () => {
+ const { db, done } = await open();
+ try {
+ const ins = db.prepare('INSERT INTO items (name, qty) VALUES (@name, @qty)');
+ await db.tx(async () => {
+ await ins.run({ name: 'outer', qty: 1 });
+ await assert.rejects(db.tx(async () => { await ins.run({ name: 'inner', qty: 2 }); throw new Error('inner'); }), /inner/);
+ await db.tx(async () => ins.run({ name: 'kept', qty: 3 }));
+ });
+ assert.deepEqual(await db.prepare('SELECT name FROM items ORDER BY id').pluck().all(), ['outer', 'kept']);
+ } finally { await done(); }
+ }],
+ [`${label}: work the transaction did not await runs on the pool afterwards; detached leaves on purpose`, async () => {
+ const { db, done } = await open();
+ try {
+ let late;
+ await db.tx(async () => {
+ await db.exec(sql`INSERT INTO items (name, qty) VALUES ('t', 1)`);
+ late = new Promise((r) => setTimeout(r, 20)).then(() => db.value(sql`SELECT count(*)::int FROM items`));
+ });
+ assert.equal(await late, 1, 'ran after the commit, on the pool');
+ if (label !== 'pglite') {
+ await db.tx(async () => {
+ await db.exec(sql`INSERT INTO items (name, qty) VALUES ('u', 1)`);
+ assert.equal(await db.detached(() => db.value(sql`SELECT count(*)::int FROM items`)), 1, 'detached does not see the uncommitted row');
+ });
+ }
+ } finally { await done(); }
+ }],
+ [`${label}: prepare binds positional, named (reused) and arrays; missing parameters are refused`, async () => {
+ const { db, done } = await open();
+ try {
+ await db.prepare('INSERT INTO items (name, qty) VALUES (?, ?), (?, ?)').run(['x', 5, 'y', 7]);
+ const q = db.prepare("SELECT name FROM items WHERE (@name::text IS NULL OR name = @name) AND qty >= :min AND name <> 'a?b' ORDER BY name");
+ assert.deepEqual((await q.all({ name: null, min: 6 })).map((r) => r.name), ['y']);
+ assert.deepEqual((await q.all({ name: 'x', min: 0 })).map((r) => r.name), ['x']);
+ assert.equal(await db.prepare('SELECT name FROM items WHERE qty = ?').get(99), undefined, 'no row is undefined, as in better-sqlite3');
+ assert.equal((await db.prepare('UPDATE items SET qty = qty + 1 WHERE qty > ?').run(0)).changes, 2);
+ await assert.rejects(q.all({ name: 'x' }), /missing parameter @min/);
+ await assert.rejects(db.prepare('SELECT ? AS a').get(), /expected 1 parameter/);
+ } finally { await done(); }
+ }],
+ ];
+}
+
+const tests = [['compile: strings, identifiers, comments, dollar quotes and casts are left alone', () => {
+ assert.throws(() => compile('SELECT ?, @a'), /either \? or named/);
+ const c = compile(`SELECT 'it''s ?', "@x", $$ :y ? $$, @a::int, :b, @a -- ?\n/* @c */`);
+ assert.equal(c.text, `SELECT 'it''s ?', "@x", $$ :y ? $$, $1::int, $2, $1 -- ?\n/* @c */`);
+ assert.deepEqual(c.names, ['a', 'b']);
+ assert.equal(compile('SELECT ?, ?').text, 'SELECT $1, $2');
+}]];
+
+const schema = 'CREATE TABLE items (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name text NOT NULL, qty integer NOT NULL)';
+tests.push(...cases('pglite', async () => {
+ const db = createDb({ pglite: true });
+ await db.query(schema);
+ return { db, done: () => db.close() };
+}));
+if (process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL) {
+ tests.push(...cases('postgresql+pgbouncer', async () => {
+ const owner = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL });
+ await owner.query('DROP TABLE IF EXISTS items');
+ await owner.query(schema);
+ await owner.query('GRANT ALL ON items TO PUBLIC').catch(() => {});
+ await owner.close();
+ const db = createDb({ url: process.env.OV_TEST_PG_URL, max: 4 });
+ return { db, done: () => db.close() };
+ }));
+} else console.log('db ambient through PgBouncer: skipped (OV_TEST_PG_URL not set; scripts/test-services.sh up)');
+
+run(tests);
@@ -27,16 +27,34 @@ export interface Queryable {
@@ -27,16 +27,34 @@ export interface Queryable {
value<T = unknown>(q: Sql | string, values?: unknown[]): Promise<T | null>;
value<T = unknown>(q: Sql | string, values?: unknown[]): Promise<T | null>;
exec(q: Sql | string, values?: unknown[]): Promise<number>;
exec(q: Sql | string, values?: unknown[]): Promise<number>;
}
}
+/** An async statement shaped like better-sqlite3's: ? or @name / :name parameters. */
+export interface Statement {
+ readonly source: string;
+ /** The first row, or undefined. */
+ get<T = Row>(...params: unknown[]): Promise<T | undefined>;
+ all<T = Row>(...params: unknown[]): Promise<T[]>;
+ /** lastInsertRowid is the first column of the first returned row (needs RETURNING). */
+ run(...params: unknown[]): Promise<{ changes: number; rows: Row[]; lastInsertRowid: unknown }>;
+ /** The same statement returning each row's first column. */
+ pluck(on?: boolean): Statement;
+}
export interface Tx extends Queryable {
export interface Tx extends Queryable {
+ sql: SqlTag;
/** A nested savepoint: rolls back alone when fn throws. */
/** A nested savepoint: rolls back alone when fn throws. */
tx<T>(fn: (t: Tx) => Promise<T>): Promise<T>;
tx<T>(fn: (t: Tx) => Promise<T>): Promise<T>;
+ prepare(text: string): Statement;
}
}
export interface TxOptions { isolation?: 'read committed' | 'repeatable read' | 'serializable'; retries?: number; readOnly?: boolean }
export interface TxOptions { isolation?: 'read committed' | 'repeatable read' | 'serializable'; retries?: number; readOnly?: boolean }
export interface MigrateResult { applied: { id: string; name: string; phase: string; ms: number }[]; pending: { id: string; name: string; phase: string }[]; held: { id: string; reason: string }[] }
export interface MigrateResult { applied: { id: string; name: string; phase: string; ms: number }[]; pending: { id: string; name: string; phase: string }[]; held: { id: string; reason: string }[] }
export interface Db extends Queryable {
export interface Db extends Queryable {
sql: SqlTag;
sql: SqlTag;
readonly store: 'postgresql' | 'pglite';
readonly store: 'postgresql' | 'pglite';
+ /** Inside fn, plain db calls join the transaction (ambient mode, the default); a db.tx inside is a savepoint. */
tx<T>(fn: (t: Tx) => Promise<T>, opts?: TxOptions): Promise<T>;
tx<T>(fn: (t: Tx) => Promise<T>, opts?: TxOptions): Promise<T>;
+ prepare(text: string): Statement;
+ inTransaction(): boolean;
+ /** Run fn outside any ambient transaction. */
+ detached<T>(fn: () => T): T;
ready(): Promise<{ ok: true; detail: { store: string; pool: { total: number; idle: number; waiting: number } } } | { ok: false; error: string }>;
ready(): Promise<{ ok: true; detail: { store: string; pool: { total: number; idle: number; waiting: number } } } | { ok: false; error: string }>;
migrate(o: { dir: string; windowDays?: number; dryRun?: boolean; now?: () => number; log?: { log(msg: string): void } }): Promise<MigrateResult>;
migrate(o: { dir: string; windowDays?: number; dryRun?: boolean; now?: () => number; log?: { log(msg: string): void } }): Promise<MigrateResult>;
stats(): { queries: number; errors: number; slow: number; retries: number; tx: number; pool: { total: number; idle: number; waiting: number } };
stats(): { queries: number; errors: number; slow: number; retries: number; tx: number; pool: { total: number; idle: number; waiting: number } };
@@ -53,6 +71,8 @@ export interface CreateDbOptions {
@@ -53,6 +71,8 @@ export interface CreateDbOptions {
slowMs?: number;
slowMs?: number;
log?: { warn(msg: string): void; error(msg: string): void };
log?: { warn(msg: string): void; error(msg: string): void };
registry?: object;
registry?: object;
+ /** Default true: db calls inside db.tx join it (AsyncLocalStorage). */
+ ambient?: boolean;
}
}
export function createDb(opts?: CreateDbOptions): Db;
export function createDb(opts?: CreateDbOptions): Db;
export declare class DbError extends Error { code?: string; detail?: string; constraint?: string; table?: string; column?: string; statement?: string; cause?: unknown }
export declare class DbError extends Error { code?: string; detail?: string; constraint?: string; table?: string; column?: string; statement?: string; cause?: unknown }