openvibers/openvibe.sdk Commits
SDK 0.14.0: the async data layer and the Valkey modules (ADR-035 and its amendment; roadmap WS-X2). openvibe-sdk/db: createDb({ url | pglite, service }) with query, many, maybe, one, value and exec on a safe sql tagged template (every value a bind parameter, nesting fragments, sql.ident/set/insert/join/json); tx(fn, { isolation, retries }) retrying 40001 and 40P01, with nested savepoints; ready() reporting the store that answered; query-time histogram, pool gauges and a slow-query log without values. node-postgres through PgBouncer in production and PGlite (real PostgreSQL in-process) in tests return identical rows: int8 as Number (refused past 2^53), timestamps as ISO strings, dates as YYYY-MM-DD, numeric as exact text, json parsed. db.migrate runs numbered expand/migrate/contract SQL files once, with checksums (edits refused), order kept, an advisory lock, contract migrations held until their expand is 7 days old, and -- no-transaction. importSqlite moves a service's SQLite file into its schema: parents first, conversion by target type, identities kept and sequences advanced, lost columns refused unless dropped, streamed verification of counts and checksums. openvibe-sdk/valkey (auto-pipelined, prefix-confined), cache (cross-process single-flight, tags), queue (at-least-once on streams: retries with backoff, dead letters, delays, dedupe, reclaim of hung workers' jobs), pubsub, and createValkeyLimitStore for limits shared by every process (in-process counting if Valkey fails); each has an in-process twin. Drivers are optional peers loaded on use. scripts/test-services.sh starts PostgreSQL 18 + PgBouncer (transaction, SCRAM) + Valkey 9 containers; CI runs the suite against them. npm test: 35/35 with the containers (the same db cases pass on PGlite and through PgBouncer; 8 concurrent serializable increments all land), 33/35 with 2 skipped without them.
Changes
@@@ -10,6 +10,8 @@ jobs:with:syntax-dirs: src scriptsextra: node scripts/esm.js --check # the generated ESM wrappers match src/++ # db, cache, queue, pubsub and limits against PostgreSQL 18 + PgBouncer + Valkey 9 containers (ADR-035)++ test: 'eval "$(scripts/test-services.sh up)" && npm test'# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo.security:uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@66b76f868ea8228aad7811309eafef3726b036f3@@@ -3,6 +3,27 @@All notable changes to `openvibe-sdk`. The package follows semver; while it is `0.x`, a minorrelease may change an API and says so here.++## 0.14.0 (2026-09-28)++++**The async data layer and Valkey modules** (ADR-035 and its amendment; roadmap WS-X2): every service moves to PostgreSQL 18 and Valkey, async-first.++- **`openvibe-sdk/db`:**++ - **Queries:** `createDb({ url | pglite, service })` has `query`, `many`, `maybe`, `one`, `value` and `exec` on a safe `sql` tagged template: every value is a bind parameter, fragments nest, and `sql.ident`, `sql.set`, `sql.insert`, `sql.join` and `sql.json` help build them.++ - **Transactions:** `tx(fn, { isolation, retries })` retries 40001 and 40P01 with backoff, and nested `t.tx()` calls are savepoints.++ - **Health:** `ready()` reports the store that actually answered, with the pool state. There are metrics (`db_query_seconds`, pool gauges) and a slow-query log that shows text only, never values.++ - **Adapters:** node-postgres, which production uses through PgBouncer in transaction mode, and PGlite for tests (real PostgreSQL in-process). Both return the same rows: int8 as Number (refused beyond 2^53), timestamps as ISO strings, dates as `YYYY-MM-DD`, numeric as exact text, json parsed.++- **Migrations:** `db.migrate({ dir })` runs numbered SQL files labelled `expand`, `migrate` or `contract`. It records them in a ledger with checksums, refuses edited or out-of-order files, serialises runs with an advisory lock, holds a contract migration until its expand is 7 days old (the N-1 window), and supports `-- no-transaction`.++- **`importSqlite()`:** the one-time move of a service's SQLite file into its PostgreSQL schema.++ - Tables go parents first, and values are converted by target type (0/1 to boolean, text or epoch to timestamptz, JSON text to jsonb, JSON arrays to arrays).++ - Identity values are kept and sequences advanced. A source column that would be lost is refused unless dropped on purpose.++ - Streamed verification compares counts and content checksums.++- **`openvibe-sdk/valkey`:** the shared connection. It is auto-pipelined, confined to `VALKEY_PREFIX`, has a `duplicate()` for blocking consumers, and `ready()`.++- **`openvibe-sdk/cache`:** `getOrSet` loads a key once across processes (single-flight plus a short lock), with tags, TTLs and an in-process fallback. A cache error never fails the caller.++- **`openvibe-sdk/queue`:** at-least-once jobs on Valkey streams. It has consumer groups, retries with exponential backoff, a dead-letter stream, delayed jobs, a 24-hour dedupe by id, and reclaiming of a job a dead or hung worker holds. It also has an in-process twin.++- **`openvibe-sdk/pubsub`:** fan-out across processes and hosts, inside the prefix.++- **`openvibe-sdk/limits`:** `createValkeyLimitStore(valkey)` makes every process and host count an actor together, in one atomic script per request. If Valkey fails, that request is counted in-process instead.++- **Dependencies:** `pg`, `@electric-sql/pglite`, `iovalkey` and `better-sqlite3` are optional peers, loaded only by the module that needs them.++- **Tests:** `scripts/test-services.sh up` starts PostgreSQL 18, PgBouncer and Valkey 9 in containers, and `test/db.test.js` and `test/valkey.test.js` run against them when their URLs are set. Otherwise they run on PGlite and the in-process stores, and print skip lines.++## 0.13.0 (2026-09-28)**Deliveries are v2 only**, as OpenVibe.Events sends them since 2026-09-28 (the v1 header, a replayable HMAC of the body alone, is retired: shim C-60). `signDeliveryHeaders` returns `X-OpenVibe-Timestamp` and `X-OpenVibe-Signature-V2` only, and the mock platform's `deliverEvents` sends no `X-OpenVibe-Signature`. A consumer test that asserted the v1 header must drop that assertion. `signDelivery` and `verifyDelivery` stay for testing a legacy receiver, and `parseDelivery` still accepts a v1-only delivery unless `requireV2`, which every production consumer sets.@@@ -338,6 +338,30 @@ The mock answers at the real public origins with real RS256 tokens and checks auHelpers: `signUserToken()`, `signServiceToken()`, `signAppToken()`, `authorize()`, `setAuthorization()`, `publishEvent(envelope, publisher, { projectId, env })` (a registered app's `app:<id>` publisher implies them), `pruneEvents()`, `deliverEvents()`, `dropRealtime()`, `dropJobStreams()`, `addTool()`, `addMediaObject()`, and `stats` and `state` for assertions. Its `fetch` rejects on an aborted signal, as `fetch` does. It is a fake. It has no persistence, its visibility rules are simplified, and it has no Chat (no WebSocket mock).++### Data: PostgreSQL and Valkey (server)++++Every service stores its data in PostgreSQL 18 through PgBouncer, and shares caches, limits, queues and fan-out through Valkey (ADR-035). The data role on the host provides both; `roles/data/add-service.sh` in OpenVibe.Host sets `DATABASE_URL`, `DATABASE_DIRECT_URL`, `VALKEY_URL` and `VALKEY_PREFIX`.++++```js++const { createDb, sql } = require('openvibe-sdk/db');++const { createValkey } = require('openvibe-sdk/valkey');++const { createCache } = require('openvibe-sdk/cache');++const { createQueue } = require('openvibe-sdk/queue');++++const db = createDb({ service: 'wiki' }); // DATABASE_URL; tests: createDb({ pglite: true })++await createDb({ url: process.env.DATABASE_DIRECT_URL }).migrate({ dir: 'migrations' });++const page = await db.maybe(sql`SELECT id, title FROM pages WHERE slug = ${slug}`);++await db.tx(async (t) => { … }, { isolation: 'serializable' });++++const valkey = createValkey(); // null without VALKEY_URL: in-process fallbacks++const cache = createCache({ valkey, namespace: 'pages' });++const html = await cache.getOrSet(`page:${slug}`, 300, () => render(slug));++const thumbs = createQueue({ valkey, name: 'thumbs' });++thumbs.process(async (job) => { … }, { concurrency: 4 });++```++++Install the drivers the service uses: `pg` and `iovalkey` (production), `@electric-sql/pglite` (tests), `better-sqlite3` (only for the one-time `importSqlite`).++### Browser without a bundler```html@@@ -3,7 +3,7 @@"stage": "alpha","code": true,"wave": "W2",-- "version": "0.12.0",++ "version": "0.14.0","plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30","deployed": false,"deployment": "none: a package released as git tags; services and apps pin a tag",@@@ -19,7 +19,9 @@"per-actor limiter for services (openvibe-sdk/limits)","the OpenVibe Frame for any app","mock platform for tests (openvibe-sdk/testing)",-- "ESM entries, TypeScript types and a browser bundle"++ "ESM entries, TypeScript types and a browser bundle",++ "db (ADR-035): the async PostgreSQL data layer (node-postgres through PgBouncer; PGlite in tests) with safe sql fragments, retrying transactions and savepoints, versioned expand/migrate/contract migrations, and importSqlite for the one-time SQLite move with verification",++ "valkey, cache, queue, pubsub and a shared limits store on Valkey 9, each with an in-process twin for tests"],"updated": "2026-09-28"}@@@ -1,4 +1,4 @@--// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.13.0, MIT). Do not edit.++// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.14.0, MIT). Do not edit.// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects). No server-only code: no service tokens, JWT verification, events, identity./* eslint-disable */@@@ -40,7 +40,7 @@ const { paginate, offsetPager } = __load(7);const semver = __load(6);/** The package version (test/version.test.js keeps it equal to package.json). */--const SDK_VERSION = '0.13.0';++const SDK_VERSION = '0.14.0';module.exports = {createClient,nnew file mode 100644@@@ -0,0 +1,7 @@++// Generated by scripts/esm.js from src/cache.js. Do not edit.++import cjs from '../src/cache.js';++++export const {++ createCache,++} = cjs;++export default cjs;nnew file mode 100644@@@ -0,0 +1,7 @@++// Generated by scripts/esm.js from src/db/index.js. Do not edit.++import cjs from '../src/db/index.js';++++export const {++ createDb, sql, DbError, importSqlite, PARSERS, ISOLATION,++} = cjs;++export default cjs;@@@ -2,6 +2,6 @@import cjs from '../src/limits.js';export const {-- createActorLimiter, defaultActor, WINDOWS,++ createActorLimiter, createValkeyLimitStore, defaultActor, WINDOWS,} = cjs;export default cjs;nnew file mode 100644@@@ -0,0 +1,7 @@++// Generated by scripts/esm.js from src/pubsub.js. Do not edit.++import cjs from '../src/pubsub.js';++++export const {++ createPubSub,++} = cjs;++export default cjs;nnew file mode 100644@@@ -0,0 +1,7 @@++// Generated by scripts/esm.js from src/queue.js. Do not edit.++import cjs from '../src/queue.js';++++export const {++ createQueue, backoff,++} = cjs;++export default cjs;nnew file mode 100644@@@ -0,0 +1,7 @@++// Generated by scripts/esm.js from src/valkey.js. Do not edit.++import cjs from '../src/valkey.js';++++export const {++ createValkey,++} = cjs;++export default cjs;@@@ -1,21 +1,58 @@{"name": "openvibe-sdk",-- "version": "0.13.0",++ "version": "0.14.0","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "openvibe-sdk",-- "version": "0.13.0",++ "version": "0.14.0","license": "MIT","devDependencies": {++ "@electric-sql/pglite": "^0.5.8","better-sqlite3": "^11.10.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0"++ "iovalkey": "^0.4.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",++ "pg": "^8.23.0"},"engines": {"node": ">=20"++ },++ "peerDependencies": {++ "@electric-sql/pglite": "^0.5.8",++ "better-sqlite3": ">=11",++ "iovalkey": "^0.4.0",++ "pg": "^8.23.0"++ },++ "peerDependenciesMeta": {++ "@electric-sql/pglite": {++ "optional": true++ },++ "better-sqlite3": {++ "optional": true++ },++ "iovalkey": {++ "optional": true++ },++ "pg": {++ "optional": true++ }}},++ "node_modules/@electric-sql/pglite": {++ "version": "0.5.8",++ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",++ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",++ "dev": true,++ "license": "Apache-2.0"++ },++ "node_modules/@iovalkey/commands": {++ "version": "0.1.0",++ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",++ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",++ "dev": true,++ "license": "MIT"++ },"node_modules/ajv": {"version": "8.20.0","resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz",@@@ -138,6 +175,34 @@"dev": true,"license": "ISC"},++ "node_modules/cluster-key-slot": {++ "version": "1.1.2",++ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",++ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",++ "dev": true,++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/debug": {++ "version": "4.4.3",++ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",++ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "ms": "^2.1.3"++ },++ "engines": {++ "node": ">=6.0"++ },++ "peerDependenciesMeta": {++ "supports-color": {++ "optional": true++ }++ }++ },"node_modules/decompress-response": {"version": "6.0.0","resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",@@@ -164,6 +229,16 @@"node": ">=4.0.0"}},++ "node_modules/denque": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",++ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",++ "dev": true,++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10"++ }++ },"node_modules/detect-libc": {"version": "2.1.2","resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",@@@ -274,6 +349,27 @@"dev": true,"license": "ISC"},++ "node_modules/iovalkey": {++ "version": "0.4.0",++ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",++ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "@iovalkey/commands": "^0.1.0",++ "cluster-key-slot": "^1.1.0",++ "debug": "^4.3.4",++ "denque": "^2.1.0",++ "lodash.defaults": "^4.2.0",++ "lodash.isarguments": "^3.1.0",++ "redis-errors": "^1.2.0",++ "redis-parser": "^3.0.0",++ "standard-as-callback": "^2.1.0"++ },++ "engines": {++ "node": ">=18.12.0"++ }++ },"node_modules/json-schema-traverse": {"version": "1.0.0","resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",@@@ -281,6 +377,20 @@"dev": true,"license": "MIT"},++ "node_modules/lodash.defaults": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",++ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",++ "dev": true,++ "license": "MIT"++ },++ "node_modules/lodash.isarguments": {++ "version": "3.1.0",++ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",++ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",++ "dev": true,++ "license": "MIT"++ },"node_modules/mimic-response": {"version": "3.1.0","resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",@@@ -311,6 +421,13 @@"dev": true,"license": "MIT"},++ "node_modules/ms": {++ "version": "2.1.3",++ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",++ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",++ "dev": true,++ "license": "MIT"++ },"node_modules/napi-build-utils": {"version": "2.0.0","resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",@@@ -358,6 +475,146 @@"node": ">=20"}},++ "node_modules/pg": {++ "version": "8.23.0",++ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",++ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "pg-connection-string": "^2.14.0",++ "pg-pool": "^3.14.0",++ "pg-protocol": "^1.16.0",++ "pg-types": "2.2.0",++ "pgpass": "1.0.5"++ },++ "engines": {++ "node": ">= 16.0.0"++ },++ "optionalDependencies": {++ "pg-cloudflare": "^1.4.0"++ },++ "peerDependencies": {++ "pg-native": ">=3.0.1"++ },++ "peerDependenciesMeta": {++ "pg-native": {++ "optional": true++ }++ }++ },++ "node_modules/pg-cloudflare": {++ "version": "1.4.0",++ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",++ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",++ "dev": true,++ "license": "MIT",++ "optional": true++ },++ "node_modules/pg-connection-string": {++ "version": "2.14.0",++ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",++ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",++ "dev": true,++ "license": "MIT"++ },++ "node_modules/pg-int8": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",++ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",++ "dev": true,++ "license": "ISC",++ "engines": {++ "node": ">=4.0.0"++ }++ },++ "node_modules/pg-pool": {++ "version": "3.14.0",++ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",++ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",++ "dev": true,++ "license": "MIT",++ "peerDependencies": {++ "pg": ">=8.0"++ }++ },++ "node_modules/pg-protocol": {++ "version": "1.16.0",++ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",++ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",++ "dev": true,++ "license": "MIT"++ },++ "node_modules/pg-types": {++ "version": "2.2.0",++ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",++ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "pg-int8": "1.0.1",++ "postgres-array": "~2.0.0",++ "postgres-bytea": "~1.0.0",++ "postgres-date": "~1.0.4",++ "postgres-interval": "^1.1.0"++ },++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/pgpass": {++ "version": "1.0.5",++ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",++ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "split2": "^4.1.0"++ }++ },++ "node_modules/postgres-array": {++ "version": "2.0.0",++ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",++ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",++ "dev": true,++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/postgres-bytea": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",++ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",++ "dev": true,++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-date": {++ "version": "1.0.7",++ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",++ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",++ "dev": true,++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-interval": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",++ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "xtend": "^4.0.0"++ },++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/prebuild-install": {"version": "7.1.3","resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",@@@ -428,6 +685,29 @@"node": ">= 6"}},++ "node_modules/redis-errors": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",++ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",++ "dev": true,++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/redis-parser": {++ "version": "3.0.0",++ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",++ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",++ "dev": true,++ "license": "MIT",++ "dependencies": {++ "redis-errors": "^1.0.0"++ },++ "engines": {++ "node": ">=4"++ }++ },"node_modules/require-from-string": {"version": "2.0.2","resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",@@@ -519,6 +799,23 @@"simple-concat": "^1.0.0"}},++ "node_modules/split2": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",++ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",++ "dev": true,++ "license": "ISC",++ "engines": {++ "node": ">= 10.x"++ }++ },++ "node_modules/standard-as-callback": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",++ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",++ "dev": true,++ "license": "MIT"++ },"node_modules/string_decoder": {"version": "1.3.0","resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",@@@ -595,6 +892,16 @@"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==","dev": true,"license": "ISC"++ },++ "node_modules/xtend": {++ "version": "4.0.2",++ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",++ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",++ "dev": true,++ "license": "MIT",++ "engines": {++ "node": ">=0.4"++ }}}}@@@ -1,7 +1,7 @@{"name": "openvibe-sdk",-- "version": "0.13.0",-- "description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, with deadlines, safe retries, problem errors and trace propagation.",++ "version": "0.14.0",++ "description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.","main": "index.js","types": "types/index.d.ts","browser": "browser.js",@@@ -76,6 +76,36 @@"import": "./esm/limits.mjs","default": "./src/limits.js"},++ "./db": {++ "types": "./types/db.d.ts",++ "browser": null,++ "import": "./esm/db.mjs",++ "default": "./src/db/index.js"++ },++ "./valkey": {++ "types": "./types/valkey.d.ts",++ "browser": null,++ "import": "./esm/valkey.mjs",++ "default": "./src/valkey.js"++ },++ "./cache": {++ "types": "./types/cache.d.ts",++ "browser": null,++ "import": "./esm/cache.mjs",++ "default": "./src/cache.js"++ },++ "./queue": {++ "types": "./types/queue.d.ts",++ "browser": null,++ "import": "./esm/queue.mjs",++ "default": "./src/queue.js"++ },++ "./pubsub": {++ "types": "./types/pubsub.d.ts",++ "browser": null,++ "import": "./esm/pubsub.mjs",++ "default": "./src/pubsub.js"++ },"./chat": {"types": "./types/chat.d.ts","import": "./esm/chat.mjs",@@@ -159,7 +189,30 @@"repository": "github:OpenVibers/OpenVibe.SDK","license": "MIT","devDependencies": {++ "@electric-sql/pglite": "^0.5.8","better-sqlite3": "^11.10.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0"++ "iovalkey": "^0.4.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",++ "pg": "^8.23.0"++ },++ "peerDependencies": {++ "pg": "^8.23.0",++ "@electric-sql/pglite": "^0.5.8",++ "iovalkey": "^0.4.0",++ "better-sqlite3": ">=11"++ },++ "peerDependenciesMeta": {++ "pg": {++ "optional": true++ },++ "@electric-sql/pglite": {++ "optional": true++ },++ "iovalkey": {++ "optional": true++ },++ "better-sqlite3": {++ "optional": true++ }}}@@@ -25,6 +25,11 @@ const ENTRIES = {'community.mjs': '../src/community.js','search.mjs': '../src/search.js','limits.mjs': '../src/limits.js',++ 'db.mjs': '../src/db/index.js',++ 'valkey.mjs': '../src/valkey.js',++ 'cache.mjs': '../src/cache.js',++ 'queue.mjs': '../src/queue.js',++ 'pubsub.mjs': '../src/pubsub.js','chat.mjs': '../src/chat.js','ai.mjs': '../src/ai.js','jobs.mjs': '../src/jobs.js',nnew file mode 100755@@@ -0,0 +1,38 @@++#!/usr/bin/env bash++# PostgreSQL 18 + PgBouncer (transaction pooling, SCRAM) + Valkey 9 in containers, the production data role's++# shape (ADR-035), for integration tests of openvibe-sdk/db, cache, queue, pubsub and limits.++#++# scripts/test-services.sh up start (or reuse) and print the env to export++# scripts/test-services.sh down remove the containers++#++# Tests read OV_TEST_PG_URL (through PgBouncer), OV_TEST_PG_DIRECT_URL and OV_TEST_VALKEY_URL; without them the++# integration cases print "<label>: skipped (…)" and the suite still runs on PGlite and the memory stores.++set -euo pipefail++NET=ovsdk-test; PW=ovtestpw++case "${1:-up}" in++up)++ docker network inspect $NET >/dev/null 2>&1 || docker network create $NET >/dev/null++ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-pg; then++ docker rm -f ovsdk-pg >/dev/null 2>&1 || true++ docker run -d --name ovsdk-pg --network $NET -p 127.0.0.1:55432:5432 \++ -e POSTGRES_USER=ov -e POSTGRES_PASSWORD=$PW -e POSTGRES_DB=ovtest -e POSTGRES_HOST_AUTH_METHOD=scram-sha-256 -e POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256 \++ postgres:18-alpine >/dev/null++ fi++ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-pgbouncer; then++ docker rm -f ovsdk-pgbouncer >/dev/null 2>&1 || true++ docker run -d --name ovsdk-pgbouncer --network $NET -p 127.0.0.1:56432:5432 \++ -e DATABASE_URL="postgres://ov:$PW@ovsdk-pg:5432/ovtest" -e POOL_MODE=transaction -e AUTH_TYPE=scram-sha-256 -e MAX_CLIENT_CONN=500 -e DEFAULT_POOL_SIZE=10 \++ edoburu/pgbouncer:latest >/dev/null++ fi++ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-valkey; then++ docker rm -f ovsdk-valkey >/dev/null 2>&1 || true++ docker run -d --name ovsdk-valkey --network $NET -p 127.0.0.1:56379:6379 valkey/valkey:9-alpine >/dev/null++ fi++ for i in $(seq 1 60); do docker exec ovsdk-pg pg_isready -U ov -d ovtest >/dev/null 2>&1 && break; sleep 0.5; done++ echo "export OV_TEST_PG_URL=postgres://ov:[email protected]:56432/ovtest"++ echo "export OV_TEST_PG_DIRECT_URL=postgres://ov:[email protected]:55432/ovtest"++ echo "export OV_TEST_VALKEY_URL=redis://127.0.0.1:56379/0"++ ;;++down) docker rm -f ovsdk-pg ovsdk-pgbouncer ovsdk-valkey >/dev/null 2>&1 || true; docker network rm $NET >/dev/null 2>&1 || true; echo removed ;;++*) echo "usage: $0 up|down" >&2; exit 2 ;;++esacnnew file mode 100644@@@ -0,0 +1,98 @@++'use strict';++/**++ * openvibe-sdk/cache — shared caching on Valkey, in-process without it (ADR-035; never authoritative).++ *++ * const cache = createCache({ valkey, namespace: 'pages', ttlSec: 60 });++ * const page = await cache.getOrSet(`slug:${slug}`, 300, () => db.maybe(sql`…`), { tags: [`space:${id}`] });++ * await cache.invalidateTag(`space:${id}`);++ *++ * getOrSet protects the database from a stampede twice: one load per key per process (single-flight), and++ * across processes a short lock, so while one process loads, the others wait briefly for its value instead of++ * all querying. Values are JSON. A Valkey error never fails the caller: the loader runs and the error is logged.++ */++++function memoryStore(max = 5000) {++ const map = new Map(); // key -> { v, exp }++ const tags = new Map(); // tag -> Set(key)++ const live = (e) => e && (!e.exp || e.exp > Date.now());++ return {++ async get(k) { const e = map.get(k); if (!live(e)) { map.delete(k); return undefined; } map.delete(k); map.set(k, e); return e.v; },++ async set(k, v, ttlSec, tagList) {++ map.set(k, { v, exp: ttlSec ? Date.now() + ttlSec * 1000 : 0 });++ if (map.size > max) map.delete(map.keys().next().value);++ for (const t of tagList || []) { if (!tags.has(t)) tags.set(t, new Set()); tags.get(t).add(k); }++ },++ async del(keys) { let n = 0; for (const k of keys) n += map.delete(k) ? 1 : 0; return n; },++ async lock() { return true; },++ async unlock() {},++ async invalidateTag(t) { const ks = tags.get(t); tags.delete(t); return ks ? this.del([...ks]) : 0; },++ };++}++++function valkeyStore(valkey, ns) {++ const c = valkey.client;++ const k = (key) => valkey.key('cache', ns, key);++ const tk = (tag) => valkey.key('cache', ns, 'tag', tag);++ return {++ async get(key) { const s = await c.get(k(key)); return s == null ? undefined : JSON.parse(s); },++ async set(key, v, ttlSec, tagList) {++ const m = c.multi().set(k(key), JSON.stringify(v), 'EX', Math.max(1, ttlSec));++ for (const t of tagList || []) m.sadd(tk(t), k(key)).expire(tk(t), Math.max(ttlSec, 86400));++ await m.exec();++ },++ async del(keys) { return keys.length ? c.del(...keys.map(k)) : 0; },++ async lock(key, ms) { return (await c.set(k(`lock:${key}`), '1', 'PX', ms, 'NX')) === 'OK'; },++ async unlock(key) { await c.del(k(`lock:${key}`)); },++ async invalidateTag(t) {++ const members = await c.smembers(tk(t));++ if (members.length) await c.del(...members); // members are full keys++ await c.del(tk(t));++ return members.length;++ },++ };++}++++function createCache({ valkey = null, namespace = 'default', ttlSec = 60, lockMs = 5000, waitMs = 1500, log = console, memoryMax } = {}) {++ const store = valkey ? valkeyStore(valkey, namespace) : memoryStore(memoryMax);++ const inflight = new Map();++ const stats = { hits: 0, misses: 0, loads: 0, errors: 0 };++ const safe = async (fn, fallback) => { try { return await fn(); } catch (err) { stats.errors++; log.warn(`[cache] ${err.message}`); return fallback; } };++++ async function getOrSet(key, ttl, loader, { tags } = {}) {++ const hit = await safe(() => store.get(key), undefined);++ if (hit !== undefined) { stats.hits++; return hit; }++ stats.misses++;++ if (inflight.has(key)) return inflight.get(key);++ const p = (async () => {++ const locked = await safe(() => store.lock(key, lockMs), true);++ if (!locked) {++ // Another process is loading it: wait briefly for its value.++ const until = Date.now() + waitMs;++ while (Date.now() < until) {++ await new Promise((r) => setTimeout(r, 50));++ const v = await safe(() => store.get(key), undefined);++ if (v !== undefined) { stats.hits++; return v; }++ }++ }++ try {++ stats.loads++;++ const v = await loader();++ if (v !== undefined) await safe(() => store.set(key, v, ttl || ttlSec, tags));++ return v;++ } finally { if (locked) await safe(() => store.unlock(key)); }++ })().finally(() => inflight.delete(key));++ inflight.set(key, p);++ return p;++ }++++ return {++ get: (key) => safe(() => store.get(key), undefined),++ set: (key, value, ttl, opts = {}) => safe(() => store.set(key, value, ttl || ttlSec, opts.tags)),++ del: (...keys) => safe(() => store.del(keys), 0),++ getOrSet,++ invalidateTag: (tag) => safe(() => store.invalidateTag(tag), 0),++ stats: () => ({ ...stats, store: valkey ? 'valkey' : 'memory' }),++ };++}++++module.exports = { createCache };@@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');const semver = require('./semver');/** The package version (test/version.test.js keeps it equal to package.json). */--const SDK_VERSION = '0.13.0';++const SDK_VERSION = '0.14.0';module.exports = {createClient,nnew file mode 100644@@@ -0,0 +1,261 @@++'use strict';++/**++ * openvibe-sdk/db — the async data layer every service uses (ADR-035, roadmap WS-X2).++ *++ * PostgreSQL is the only dialect. Production connects with node-postgres through PgBouncer in transaction mode++ * (DATABASE_URL); tests and embedded tools run real PostgreSQL in-process with PGlite, so what is tested is what++ * runs. Both adapters return identical rows: int8 as Number (refused beyond 2^53), timestamps as ISO-8601++ * strings, dates as 'YYYY-MM-DD', numeric as a string (exact), json/jsonb parsed.++ *++ * const { createDb, sql } = require('openvibe-sdk/db');++ * const db = createDb({ service: 'wiki' }); // DATABASE_URL, or { url }, or { pglite: true }++ * const page = await db.maybe(sql`SELECT id, title FROM pages WHERE slug = ${slug}`);++ * await db.tx(async (t) => { … }, { isolation: 'serializable' }); // 40001/40P01 retried++ * await db.migrate({ dir: path.join(__dirname, 'migrations') }); // with DATABASE_DIRECT_URL++ *++ * Rules the pooler imposes (ADR-007 2026-09-24 rule 1): no session state between transactions (no session SET,++ * no LISTEN, no advisory locks held across transactions) on DATABASE_URL. Migrations use DATABASE_DIRECT_URL.++ */++const { sql, isSql } = require('./sql');++++const ISOLATION = { 'read committed': 'READ COMMITTED', 'repeatable read': 'REPEATABLE READ', serializable: 'SERIALIZABLE' };++const RETRYABLE = new Set(['40001', '40P01']); // serialization failure, deadlock++++// One set of type parsers for both adapters (text-format values from the server).++function int8(v) {++ if (v === null || v === undefined) return v;++ const n = Number(v);++ if (!Number.isSafeInteger(n)) throw new RangeError(`openvibe-sdk/db: int8 value ${v} is beyond 2^53; select it as text`);++ return n;++}++const PARSERS = {++ 20: int8, // int8++ 1700: (v) => v, // numeric: exact, as text++ 1114: (v) => (v == null ? v : new Date(`${v.replace(' ', 'T')}Z`).toISOString()), // timestamp (UTC by convention)++ 1184: (v) => (v == null ? v : new Date(normaliseTz(v)).toISOString()), // timestamptz++ 1082: (v) => v, // date++ 114: (v) => (v == null ? v : JSON.parse(v)), // json++ 3802: (v) => (v == null ? v : JSON.parse(v)), // jsonb++};++function normaliseTz(v) {++ // '2026-09-28 10:47:22.411+00' → '2026-09-28T10:47:22.411+00:00'++ let s = v.replace(' ', 'T');++ if (/[+-]\d\d$/.test(s)) s += ':00';++ return s;++}++++function compileArgs(q, values) {++ if (isSql(q)) return q.compile();++ if (typeof q === 'string') return { text: q, values: values || [] };++ throw new TypeError('openvibe-sdk/db: pass sql`…` or (text, values)');++}++++class DbError extends Error {++ constructor(message, cause, extra = {}) { super(message); this.name = 'DbError'; this.cause = cause; Object.assign(this, extra); }++}++++/** The query methods, shared by the pool, a transaction and a savepoint. `run(text, values)` → { rows, rowCount }. */++function queryApi(run, self) {++ const api = {++ async query(q, values) { const c = compileArgs(q, values); return run(c.text, c.values); },++ async many(q, values) { return (await api.query(q, values)).rows; },++ async maybe(q, values) { const r = await api.query(q, values); return r.rows[0] || null; },++ async one(q, values) {++ const r = await api.query(q, values);++ if (!r.rows.length) throw new DbError('openvibe-sdk/db: expected one row, got none', null, { code: 'no_rows' });++ return r.rows[0];++ },++ async value(q, values) { const r = await api.query(q, values); const row = r.rows[0]; return row ? row[Object.keys(row)[0]] : null; },++ async exec(q, values) { return (await api.query(q, values)).rowCount; },++ };++ return Object.assign(self, api);++}++++/**++ * @param {object} [o]++ * @param {string} [o.url] postgres URL (default DATABASE_URL)++ * @param {boolean|string|object} [o.pglite] true: an in-memory PGlite; a directory: persisted PGlite; or a PGlite instance++ * @param {string} [o.service] application_name and metric label++ * @param {number} [o.max] pool size per process (default 10; PgBouncer multiplexes)++ * @param {number} [o.queryTimeoutMs] client-side cap per query (default 15000; the runtime role also has a server-side one)++ * @param {number} [o.slowMs] log statements slower than this (default 500), text only, never values++ * @param {object} [o.log] { warn, error } (default console)++ * @param {object} [o.registry] openvibe-shared/metrics registry: db_query_seconds histogram, pool gauges++ */++function createDb(o = {}) {++ const log = o.log || console;++ const service = o.service || process.env.OV_SERVICE || 'service';++ const slowMs = o.slowMs == null ? 500 : o.slowMs;++ const stats = { queries: 0, errors: 0, slow: 0, retries: 0, tx: 0 };++ let hist = null;++ if (o.registry && typeof o.registry.histogram === 'function') {++ try { hist = o.registry.histogram({ name: 'db_query_seconds', help: 'Database query time by kind', labelNames: ['kind'], buckets: [0.001, 0.005, 0.02, 0.1, 0.5, 2, 10] }); } catch { hist = null; }++ }++ const kindOf = (text) => (text.trimStart().slice(0, 6).toUpperCase().match(/^(SELECT|INSERT|UPDATE|DELETE|WITH)/) || ['OTHER'])[0].toLowerCase();++++ let adapter;++ if (o.pglite) adapter = pgliteAdapter(o.pglite);++ else adapter = pgAdapter({ url: o.url || process.env.DATABASE_URL, max: o.max || 10, service, queryTimeoutMs: o.queryTimeoutMs || 15000, log });++++ async function timed(conn, text, values) {++ const t0 = process.hrtime.bigint();++ try {++ const r = await adapter.run(conn, text, values);++ stats.queries++;++ return r;++ } catch (err) {++ stats.errors++;++ throw wrap(err, text);++ } finally {++ const s = Number(process.hrtime.bigint() - t0) / 1e9;++ if (hist) { try { hist.observe({ kind: kindOf(text) }, s); } catch { /* metrics never break a query */ } }++ if (s * 1000 > slowMs) { stats.slow++; log.warn(`[db] slow (${Math.round(s * 1000)} ms): ${text.replace(/\s+/g, ' ').slice(0, 160)}`); }++ }++ }++++ async function tx(fn, opts = {}, conn = null, depth = 0) {++ if (conn) {++ // Nested: a savepoint inside the running transaction.++ const name = `sp_${depth}`;++ await timed(conn, `SAVEPOINT ${name}`, []);++ try {++ const out = await fn(txApi(conn, depth + 1));++ await timed(conn, `RELEASE SAVEPOINT ${name}`, []);++ return out;++ } catch (err) { await timed(conn, `ROLLBACK TO SAVEPOINT ${name}`, []).catch(() => {}); throw err; }++ }++ const iso = ISOLATION[opts.isolation || 'read committed'];++ if (!iso) throw new TypeError(`openvibe-sdk/db: isolation must be one of ${Object.keys(ISOLATION).join(', ')}`);++ const retries = opts.retries == null ? (opts.isolation === 'serializable' ? 5 : 2) : opts.retries;++ for (let attempt = 0; ; attempt++) {++ const c = await adapter.acquire();++ stats.tx++;++ try {++ await timed(c, `BEGIN ISOLATION LEVEL ${iso}${opts.readOnly ? ' READ ONLY' : ''}`, []);++ const out = await fn(txApi(c, 1));++ await timed(c, 'COMMIT', []);++ return out;++ } catch (err) {++ await timed(c, 'ROLLBACK', []).catch(() => {});++ const code = err && (err.code || (err.cause && err.cause.code));++ if (RETRYABLE.has(code) && attempt < retries) {++ stats.retries++;++ await new Promise((r) => setTimeout(r, Math.min(1000, 10 * 2 ** attempt) + Math.random() * 20));++ continue;++ }++ throw err;++ } finally { adapter.release(c); }++ }++ }++++ function txApi(conn, depth) {++ const t = queryApi((text, values) => timed(conn, text, values), {});++ t.tx = (fn) => tx(fn, {}, conn, depth);++ return t;++ }++++ const db = queryApi((text, values) => timed(null, text, values), {});++ db.sql = sql;++ db.tx = (fn, opts) => tx(fn, opts);++ db.store = adapter.store;++ db.stats = () => ({ ...stats, pool: adapter.pool() });++ /** Readiness: a real round trip, and which store answered (never claimed from configuration). */++ db.ready = async () => {++ try {++ const v = await db.value(sql`SELECT 1 AS ok`);++ return v === 1 ? { ok: true, detail: { store: adapter.store, pool: adapter.pool() } } : { ok: false, error: 'unexpected answer' };++ } catch (err) { return { ok: false, error: `${adapter.store}: ${err.message}` }; }++ };++ db.migrate = (m) => require('./migrate').migrate({ db, ...m, log: m && m.log ? m.log : log });++ db.close = () => adapter.close();++ db._adapter = adapter;++ if (hist === null && o.registry && typeof o.registry.gauge === 'function') { /* histogram unsupported: counts still in stats() */ }++ if (o.registry && typeof o.registry.gauge === 'function') {++ try {++ o.registry.gauge({ name: 'db_pool_connections', help: 'Database pool connections by state', labelNames: ['state'],++ collect: () => { const p = adapter.pool(); return [{ labels: { state: 'total' }, value: p.total }, { labels: { state: 'idle' }, value: p.idle }, { labels: { state: 'waiting' }, value: p.waiting }]; } });++ } catch { /* registered already */ }++ }++ return db;++}++++function wrap(err, text) {++ if (err instanceof DbError) return err;++ const e = new DbError(err.message, err, { code: err.code, detail: err.detail, constraint: err.constraint, table: err.table, column: err.column });++ e.statement = text.replace(/\s+/g, ' ').slice(0, 200);++ return e;++}++++// ── node-postgres ──────────────────────────────────────────────────────────────────────────++function pgAdapter({ url, max, service, queryTimeoutMs, log }) {++ if (!url) throw new Error('openvibe-sdk/db: DATABASE_URL is not set (or pass { url } or { pglite: true })');++ let pg;++ try { pg = require('pg'); } catch { throw new Error('openvibe-sdk/db: install `pg` in the service (npm i pg)'); }++ const types = {++ getTypeParser(oid, format) {++ if (format === 'text' && PARSERS[oid]) return PARSERS[oid];++ return pg.types.getTypeParser(oid, format);++ },++ };++ const pool = new pg.Pool({ connectionString: url, max, application_name: service, idleTimeoutMillis: 30000, connectionTimeoutMillis: 5000, query_timeout: queryTimeoutMs, types, keepAlive: true });++ pool.on('error', (err) => log.error(`[db] idle client error: ${err.message}`));++ return {++ store: 'postgresql',++ run: (conn, text, values) => (conn || pool).query({ text, values }).then((r) => ({ rows: r.rows, rowCount: r.rowCount })),++ acquire: () => pool.connect(),++ release: (c) => c.release(),++ pool: () => ({ total: pool.totalCount, idle: pool.idleCount, waiting: pool.waitingCount }),++ close: () => pool.end(),++ };++}++++// ── PGlite (tests, embedded) ───────────────────────────────────────────────────────────────++function pgliteAdapter(opt) {++ let mod;++ try { mod = require('@electric-sql/pglite'); } catch { throw new Error('openvibe-sdk/db: install `@electric-sql/pglite` for { pglite } (tests)'); }++ const parsers = {};++ for (const [oid, fn] of Object.entries(PARSERS)) parsers[oid] = (v) => fn(v);++ const inst = typeof opt === 'object' && opt && typeof opt.query === 'function' ? opt++ : new mod.PGlite(typeof opt === 'string' ? opt : undefined, { parsers });++ // PGlite is one connection: transactions take turns through this queue so statements never interleave.++ let chain = Promise.resolve();++ let holder = null;++ const conn = { id: 'pglite' };++ return {++ store: 'pglite',++ async run(c, text, values) {++ if (c === null && holder) {++ // A statement outside a transaction waits for the running one to finish. Inside that transaction's++ // own code this would never finish (production would just take another pooled connection), so say so.++ let timer;++ const waited = await Promise.race([chain.then(() => true), new Promise((r) => { timer = setTimeout(() => r(false), 10000); })]);++ clearTimeout(timer);++ if (!waited) throw new Error('openvibe-sdk/db (PGlite): a query outside a transaction waited 10 s for it; inside db.tx(fn) use the handle fn receives');++ }++ // Several statements without parameters (a migration file) go through exec, as node-postgres's simple protocol does.++ if ((!values || !values.length) && text.replace(/;\s*$/, '').includes(';')) {++ const results = await inst.exec(text, { parsers });++ const last = results[results.length - 1] || { rows: [] };++ return { rows: last.rows || [], rowCount: last.affectedRows != null ? last.affectedRows : (last.rows || []).length };++ }++ const r = await inst.query(text, values, { parsers });++ return { rows: r.rows, rowCount: r.affectedRows != null ? r.affectedRows : r.rows.length };++ },++ acquire() {++ let release;++ const turn = new Promise((r) => { release = r; });++ const prev = chain;++ chain = prev.then(() => turn);++ return prev.then(() => { holder = { release }; return conn; });++ },++ release() { const h = holder; holder = null; if (h) h.release(); },++ pool: () => ({ total: 1, idle: holder ? 0 : 1, waiting: 0 }),++ close: () => inst.close(),++ instance: inst,++ };++}++++/** The one-time SQLite → PostgreSQL move (loaded on use: it needs better-sqlite3). */++function importSqlite(opts) { return require('./sqlite-import').importSqlite(opts); }++++module.exports = { createDb, sql, DbError, importSqlite, PARSERS, ISOLATION };nnew file mode 100644@@@ -0,0 +1,115 @@++'use strict';++/**++ * Versioned migrations for openvibe-sdk/db (ADR-035, ADR-028 expand/migrate/contract).++ *++ * A migrations directory holds files named `NNNN_description.sql` (NNNN: four or more digits, applied in order).++ * The first lines are headers:++ *++ * -- phase: expand | migrate | contract (required)++ * -- after: 0012 (contract only: the expand migration it completes)++ * -- no-transaction (for CREATE INDEX CONCURRENTLY and the like)++ *++ * Each migration runs once, in a transaction unless it says otherwise, and is recorded in `ov_migrations`++ * with its checksum; editing an applied file is refused. Runs are serialised by an advisory lock, so several++ * processes starting together apply each migration once. A contract migration is refused until the expand it++ * names has been applied for `windowDays` (default 7, the N-1 window of ADR-016), so a rollback to the previous++ * release still finds what it expects. Run it with the owner role (DATABASE_DIRECT_URL): the runtime role++ * behind PgBouncer may not change the schema, and advisory locks need a session.++ */++const crypto = require('crypto');++const fs = require('fs');++const path = require('path');++++const FILE_RE = /^(\d{4,})_([a-z0-9_-]+)\.sql$/;++const LOCK_KEY = 71_335_035; // one advisory lock id for every service's migrations (ADR-035)++++function parse(dir) {++ if (!fs.existsSync(dir)) return [];++ const files = fs.readdirSync(dir).filter((f) => f.endsWith('.sql')).sort();++ const seen = new Set();++ return files.map((f) => {++ const m = FILE_RE.exec(f);++ if (!m) throw new Error(`migrate: ${f} is not named NNNN_description.sql`);++ if (seen.has(m[1])) throw new Error(`migrate: two migrations numbered ${m[1]}`);++ seen.add(m[1]);++ const text = fs.readFileSync(path.join(dir, f), 'utf8');++ const header = text.split('\n').slice(0, 12).join('\n');++ const phase = (/^--\s*phase:\s*(expand|migrate|contract)\s*$/m.exec(header) || [])[1];++ if (!phase) throw new Error(`migrate: ${f} has no "-- phase: expand|migrate|contract" header`);++ const after = (/^--\s*after:\s*(\d{4,})\s*$/m.exec(header) || [])[1] || null;++ if (phase === 'contract' && !after) throw new Error(`migrate: ${f} is a contract migration and must name its expand ("-- after: NNNN")`);++ return {++ id: m[1], name: m[2], file: f, phase, after, text,++ transaction: !/^--\s*no-transaction\s*$/m.test(header),++ checksum: crypto.createHash('sha256').update(text.replace(/\r\n/g, '\n')).digest('hex'),++ };++ });++}++++/**++ * @param {object} o++ * @param {object} o.db a createDb() handle with schema rights (owner role, DATABASE_DIRECT_URL)++ * @param {string} o.dir the migrations directory++ * @param {number} [o.windowDays] N-1 window before a contract migration may run (default 7)++ * @param {boolean} [o.dryRun] report only++ * @param {() => number} [o.now]++ * → { applied: [{ id, name, phase, ms }], pending: [...], held: [{ id, reason }] }++ */++async function migrate({ db, dir, windowDays = 7, dryRun = false, now = () => Date.now(), log = console }) {++ const all = parse(dir);++ const out = { applied: [], pending: [], held: [] };++ const run = async (t) => {++ await t.query(`CREATE TABLE IF NOT EXISTS ov_migrations (++ id text PRIMARY KEY, name text NOT NULL, phase text NOT NULL CHECK (phase IN ('expand','migrate','contract')),++ checksum text NOT NULL, applied_at timestamptz NOT NULL DEFAULT now(), duration_ms integer NOT NULL DEFAULT 0)`);++ const done = new Map((await t.many('SELECT id, checksum, applied_at FROM ov_migrations')).map((r) => [r.id, r]));++ for (const m of all) {++ const prev = done.get(m.id);++ if (prev) {++ if (prev.checksum !== m.checksum) throw new Error(`migrate: ${m.file} changed after it was applied (write a new migration instead)`);++ continue;++ }++ const later = [...done.keys()].find((id) => id > m.id);++ if (later) throw new Error(`migrate: ${m.file} is older than applied migration ${later}; number new migrations after the last one`);++ if (m.phase === 'contract') {++ const exp = done.get(m.after);++ const ageDays = exp ? (now() - Date.parse(exp.applied_at)) / 86400000 : -1;++ if (!exp || ageDays < windowDays) {++ out.held.push({ id: m.id, reason: exp ? `expand ${m.after} applied ${ageDays.toFixed(1)} days ago; the N-1 window is ${windowDays}` : `expand ${m.after} is not applied` });++ break; // later migrations wait too: order is kept++ }++ }++ out.pending.push({ id: m.id, name: m.name, phase: m.phase });++ if (dryRun) continue;++ const t0 = Date.now();++ if (m.transaction) {++ await t.query('BEGIN');++ try {++ await t.query(m.text);++ await t.query('INSERT INTO ov_migrations (id, name, phase, checksum, duration_ms) VALUES ($1, $2, $3, $4, $5)', [m.id, m.name, m.phase, m.checksum, Date.now() - t0]);++ await t.query('COMMIT');++ } catch (err) { await t.query('ROLLBACK').catch(() => {}); throw new Error(`migrate: ${m.file} failed: ${err.message}`); }++ } else {++ try { await t.query(m.text); } catch (err) { throw new Error(`migrate: ${m.file} (no transaction) failed: ${err.message}`); }++ await t.query('INSERT INTO ov_migrations (id, name, phase, checksum, duration_ms) VALUES ($1, $2, $3, $4, $5)', [m.id, m.name, m.phase, m.checksum, Date.now() - t0]);++ }++ done.set(m.id, { id: m.id, checksum: m.checksum, applied_at: new Date().toISOString() });++ out.applied.push({ id: m.id, name: m.name, phase: m.phase, ms: Date.now() - t0 });++ log.log ? log.log(`[db] migration ${m.file} (${m.phase}) applied in ${Date.now() - t0} ms`) : null;++ }++ };++ // One session for the whole run: the advisory lock and the migrations share it.++ const adapter = db._adapter;++ const conn = await adapter.acquire();++ const session = {++ query: (text, values) => adapter.run(conn, text, values || []),++ many: async (text, values) => (await adapter.run(conn, text, values || [])).rows,++ };++ try {++ await session.query('SELECT pg_advisory_lock($1)', [LOCK_KEY]);++ try { await run(session); } finally { await session.query('SELECT pg_advisory_unlock($1)', [LOCK_KEY]).catch(() => {}); }++ } finally { adapter.release(conn); }++ return out;++}++++module.exports = { migrate, parse };nnew file mode 100644@@@ -0,0 +1,84 @@++'use strict';++/**++ * Safe SQL fragments for openvibe-sdk/db (ADR-035): a tagged template turns every interpolated value into a++ * bind parameter ($1, $2, …), so user data never becomes SQL text. Fragments nest, so a query can be++ * assembled from parts and still compile to one parameterised statement.++ *++ * sql`SELECT id, title FROM pages WHERE space_id = ${spaceId} AND id = ANY(${ids})`++ * sql`UPDATE pages SET ${sql.set({ title, body })} WHERE id = ${id}`++ * sql`INSERT INTO tags ${sql.insert(rows, ['page_id', 'tag'])} ON CONFLICT DO NOTHING`++ * sql`SELECT ${sql.join(cols.map(sql.ident))} FROM ${sql.ident('pages')}`++ *++ * Only sql.raw() puts text into the statement unchecked; keep it for constants you wrote yourself.++ */++++class Sql {++ constructor(parts, values) {++ this.parts = parts; // strings, one more than values++ this.values = values; // plain values or nested Sql fragments++ }++ /** { text, values } with $n placeholders numbered from `start`. */++ compile(start = 1) {++ const out = { text: '', values: [] };++ append(this, out, start);++ return out;++ }++ toString() { return this.compile().text; }++}++++function append(frag, out, start) {++ for (let i = 0; i < frag.parts.length; i++) {++ out.text += frag.parts[i];++ if (i >= frag.values.length) continue;++ const v = frag.values[i];++ if (v instanceof Sql) append(v, out, start);++ else { out.values.push(v === undefined ? null : v); out.text += `$${start + out.values.length - 1}`; }++ }++}++++function sql(strings, ...values) {++ if (!Array.isArray(strings) || !Array.isArray(strings.raw)) throw new TypeError('sql is a tagged template: sql`SELECT …`');++ return new Sql([...strings], values);++}++++const IDENT_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;++/** "name" or "schema"."name"; refuses anything that is not a plain identifier. */++sql.ident = (name) => {++ const parts = String(name).split('.');++ if (!parts.every((p) => IDENT_RE.test(p))) throw new TypeError(`sql.ident: not an identifier: ${name}`);++ return new Sql([parts.map((p) => `"${p}"`).join('.')], []);++};++/** Trusted text, inserted as is. Never pass user input. */++sql.raw = (text) => new Sql([String(text)], []);++/** Fragments (or values) joined by a separator fragment (default ", "). */++sql.join = (items, sep = sql`, `) => {++ const list = Array.from(items);++ if (!list.length) return new Sql([''], []);++ const parts = ['']; const values = [];++ list.forEach((item, i) => {++ if (i > 0) { values.push(sep); parts.push(''); }++ values.push(item); parts.push('');++ });++ return new Sql(parts, values);++};++/** `col = $1, col2 = $2` from an object (undefined keys are skipped). */++sql.set = (obj) => {++ const keys = Object.keys(obj).filter((k) => obj[k] !== undefined);++ if (!keys.length) throw new TypeError('sql.set: nothing to set');++ return sql.join(keys.map((k) => sql`${sql.ident(k)} = ${obj[k]}`));++};++/** `(a, b) VALUES ($1, $2), ($3, $4)` for a batch of rows; columns default to the first row's keys. */++sql.insert = (rows, columns) => {++ const list = Array.isArray(rows) ? rows : [rows];++ if (!list.length) throw new TypeError('sql.insert: no rows');++ const cols = columns || Object.keys(list[0]);++ const tuples = list.map((r) => sql`(${sql.join(cols.map((c) => (r[c] === undefined ? null : r[c])))})`);++ return sql`(${sql.join(cols.map(sql.ident))}) VALUES ${sql.join(tuples)}`;++};++/** A value as a jsonb bind parameter (objects and arrays are otherwise sent as PostgreSQL arrays). */++sql.json = (value) => sql`${value == null ? null : JSON.stringify(value)}::jsonb`;++++sql.Sql = Sql;++const isSql = (v) => v instanceof Sql;++++module.exports = { sql, Sql, isSql };nnew file mode 100644@@@ -0,0 +1,266 @@++'use strict';++/**++ * The one-time move of a service's SQLite data into its PostgreSQL schema (ADR-035 amendment, WS-X2 step 4).++ *++ * const { importSqlite } = require('openvibe-sdk/db');++ * const report = await importSqlite({ sqlite: '/opt/…/wiki.db', db: ownerDb, truncate: true });++ * if (!report.ok) throw new Error(JSON.stringify(report.problems));++ *++ * The target schema (created by the service's migrations) decides everything:++ * - which tables: every base table in `public` except ov_migrations that SQLite also has (or `tables`), copied++ * parents first (foreign-key order);++ * - which columns: those both sides have. A source column the target lacks is a problem unless the table lists++ * it in `dropColumns`; a target column the source lacks must have a default or be nullable;++ * - each value's conversion, by the target column's type: 0/1 → boolean, text or epoch → timestamptz (SQLite's++ * CURRENT_TIMESTAMP text is UTC), JSON text → jsonb (validated), JSON array text → PostgreSQL arrays,++ * Buffer → bytea, anything → text for text columns.++ * Rows go in batched multi-row INSERTs in one transaction (identity columns keep their values, then their++ * sequences move past the maximum). Verification compares each table's row count and a checksum of every row,++ * canonicalised the same way on both sides and walked in primary-key order (byte order for text keys).++ */++const crypto = require('crypto');++const { sql } = require('./sql');++++function toIso(v, col) {++ if (v == null || v === '') return null;++ if (typeof v === 'number' || (typeof v === 'string' && /^\d{9,}(\.\d+)?$/.test(v))) {++ const n = Number(v);++ return new Date(n > 1e11 ? n : n * 1000).toISOString();++ }++ const s = String(v).trim();++ let d;++ if (/^\d{4}-\d{2}-\d{2}$/.test(s)) d = new Date(`${s}T00:00:00Z`);++ else if (/^\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}(:\d{2}(\.\d+)?)?$/.test(s)) d = new Date(`${s.replace(' ', 'T')}Z`); // SQLite CURRENT_TIMESTAMP: UTC++ else d = new Date(s);++ if (Number.isNaN(d.getTime())) throw new Error(`${col}: "${s.slice(0, 40)}" is not a date/time`);++ return d.toISOString();++}++++/** Convert one SQLite value for a PostgreSQL column type (information_schema udt_name). */++function convert(v, udt, col) {++ if (v === undefined || v === null) return null;++ if (udt.startsWith('_')) { // array++ if (Array.isArray(v)) return v;++ if (typeof v === 'string' && v.trim().startsWith('[')) { const a = JSON.parse(v); if (!Array.isArray(a)) throw new Error(`${col}: not a JSON array`); return a; }++ if (v === '') return [];++ throw new Error(`${col}: cannot read "${String(v).slice(0, 40)}" as an array (store it as JSON text)`);++ }++ switch (udt) {++ case 'bool':++ if (typeof v === 'boolean') return v;++ if (v === 1 || v === '1' || v === 'true' || v === 't') return true;++ if (v === 0 || v === '0' || v === 'false' || v === 'f') return false;++ throw new Error(`${col}: "${v}" is not a boolean`);++ case 'int2': case 'int4': case 'int8': {++ const n = typeof v === 'bigint' ? Number(v) : Number(v);++ if (!Number.isSafeInteger(n)) throw new Error(`${col}: "${v}" is not an integer`);++ return n;++ }++ case 'float4': case 'float8': { const n = Number(v); if (Number.isNaN(n)) throw new Error(`${col}: "${v}" is not a number`); return n; }++ case 'numeric': return String(v);++ case 'timestamptz': case 'timestamp': return toIso(v, col);++ case 'date': return String(v).slice(0, 10);++ case 'json': case 'jsonb':++ if (typeof v === 'string') { if (v === '') return null; JSON.parse(v); return v; }++ if (Buffer.isBuffer(v)) { const s = v.toString('utf8'); JSON.parse(s); return s; }++ return JSON.stringify(v);++ case 'bytea': return Buffer.isBuffer(v) ? v : Buffer.from(String(v));++ default: return Buffer.isBuffer(v) ? v.toString('utf8') : String(v);++ }++}++++// The canonical text of a value for checksums (same rules on both sides).++function sortKeys(x) {++ if (Array.isArray(x)) return x.map(sortKeys);++ if (x && typeof x === 'object') return Object.keys(x).sort().reduce((o, k) => { o[k] = sortKeys(x[k]); return o; }, {});++ return x;++}++function canonical(v, udt) {++ if (v === null || v === undefined) return '\\N';++ if (udt === 'json' || udt === 'jsonb') return JSON.stringify(sortKeys(typeof v === 'string' ? JSON.parse(v) : v));++ if (udt === 'bytea') return Buffer.isBuffer(v) ? v.toString('hex') : Buffer.from(String(v)).toString('hex');++ if (udt === 'timestamptz' || udt === 'timestamp') return new Date(v).toISOString();++ if (udt.startsWith('_')) return JSON.stringify(v);++ if (udt === 'numeric') return String(Number(v));++ return String(v);++}++++async function schemaOf(db) {++ const cols = await db.many(sql`++ SELECT c.table_name, c.column_name, c.udt_name, c.is_nullable = 'YES' AS nullable,++ c.column_default IS NOT NULL OR c.is_identity = 'YES' OR c.is_generated = 'ALWAYS' AS has_default,++ c.is_identity = 'YES' AS identity, c.is_generated = 'ALWAYS' AS generated++ FROM information_schema.columns c++ JOIN information_schema.tables t ON t.table_schema = c.table_schema AND t.table_name = c.table_name++ WHERE c.table_schema = 'public' AND t.table_type = 'BASE TABLE' AND c.table_name <> 'ov_migrations'++ ORDER BY c.table_name, c.ordinal_position`);++ const pks = await db.many(sql`++ SELECT tc.relname AS table_name, a.attname AS column_name, array_position(i.indkey::int2[], a.attnum) AS pos++ FROM pg_index i JOIN pg_class tc ON tc.oid = i.indrelid JOIN pg_namespace n ON n.oid = tc.relnamespace++ JOIN pg_attribute a ON a.attrelid = tc.oid AND a.attnum = ANY(i.indkey)++ WHERE i.indisprimary AND n.nspname = 'public' ORDER BY 1, 3`);++ const fks = await db.many(sql`++ SELECT c.conrelid::regclass::text AS child, c.confrelid::regclass::text AS parent++ FROM pg_constraint c JOIN pg_namespace n ON n.oid = c.connamespace++ WHERE c.contype = 'f' AND n.nspname = 'public'`);++ const tables = new Map();++ for (const c of cols) {++ if (!tables.has(c.table_name)) tables.set(c.table_name, { name: c.table_name, columns: [], pk: [] });++ tables.get(c.table_name).columns.push(c);++ }++ for (const p of pks) if (tables.has(p.table_name)) tables.get(p.table_name).pk.push(p.column_name);++ // Parents first.++ const deps = new Map([...tables.keys()].map((t) => [t, new Set()]));++ for (const f of fks) { const ch = f.child.replace(/"/g, ''); const pa = f.parent.replace(/"/g, ''); if (deps.has(ch) && ch !== pa) deps.get(ch).add(pa); }++ const order = []; const state = new Map();++ const visit = (t) => { if (state.get(t) === 2) return; if (state.get(t) === 1) return; state.set(t, 1); for (const p of deps.get(t) || []) visit(p); state.set(t, 2); order.push(t); };++ for (const t of [...tables.keys()].sort()) visit(t);++ return order.map((t) => tables.get(t));++}++++/**++ * @param {object} o++ * @param {string|object} o.sqlite a path, or an open better-sqlite3 Database++ * @param {object} o.db createDb() handle with write rights on the tables (the owner role)++ * @param {object} [o.tables] { [target]: { from?, dropColumns?: [], where?: 'SQL filter on the source', map?: (row) => row } }++ * @param {string[]} [o.only] import only these target tables++ * @param {boolean} [o.truncate] empty the target tables first (rehearsals)++ * @param {boolean} [o.verify] compare counts and checksums (default true)++ * → { ok, tables: [{ table, source, rows, ms, checksum }], problems: [] }++ */++async function importSqlite({ sqlite, db, tables: opts = {}, only = null, truncate = false, verify = true, log = console } = {}) {++ let Database;++ const src = typeof sqlite === 'string' ? (() => { try { Database = require('better-sqlite3'); } catch { throw new Error('importSqlite: install better-sqlite3 to read the SQLite file'); } return new Database(sqlite, { readonly: true, fileMustExist: true }); })() : sqlite;++ const report = { ok: true, tables: [], problems: [] };++ const problem = (p) => { report.ok = false; report.problems.push(p); };++ try {++ const schema = (await schemaOf(db)).filter((t) => !only || only.includes(t.name));++ const srcTables = new Set(src.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all().map((r) => r.name));++ const plan = [];++ for (const t of schema) {++ const o = opts[t.name] || {};++ const from = o.from || t.name;++ if (!srcTables.has(from)) { log.log && log.log(`[import] ${t.name}: no source table ${from} (left empty)`); continue; }++ const srcCols = src.prepare(`PRAGMA table_info("${from.replace(/"/g, '""')}")`).all().map((c) => c.name);++ const drop = new Set(o.dropColumns || []);++ const insertable = t.columns.filter((c) => !c.generated);++ const cols = insertable.filter((c) => srcCols.includes(c.column_name));++ const lost = srcCols.filter((c) => !t.columns.some((x) => x.column_name === c) && !drop.has(c));++ if (lost.length) problem({ table: t.name, problem: `source columns with no target column (list them in dropColumns to drop them): ${lost.join(', ')}` });++ const unfillable = insertable.filter((c) => !srcCols.includes(c.column_name) && !c.nullable && !c.has_default);++ if (unfillable.length) problem({ table: t.name, problem: `target columns the source cannot fill: ${unfillable.map((c) => c.column_name).join(', ')}` });++ plan.push({ t, from, cols, o });++ }++ if (!report.ok) return report;++++ await db.tx(async (tx) => {++ if (truncate && plan.length) await tx.exec(sql`TRUNCATE ${sql.join(plan.map((p) => sql.ident(p.t.name)))} RESTART IDENTITY CASCADE`);++ for (const { t, from, cols, o } of plan) {++ const t0 = Date.now();++ const names = cols.map((c) => c.column_name);++ const hasIdentity = cols.some((c) => c.identity);++ const batch = Math.max(1, Math.min(1000, Math.floor(60000 / Math.max(1, names.length))));++ const stmt = src.prepare(`SELECT * FROM "${from.replace(/"/g, '""')}"${o.where ? ` WHERE ${o.where}` : ''}`);++ let rows = []; let n = 0;++ const flush = async () => {++ if (!rows.length) return;++ await tx.exec(sql`INSERT INTO ${sql.ident(t.name)} (${sql.join(names.map(sql.ident))}) ${hasIdentity ? sql`OVERRIDING SYSTEM VALUE ` : sql``}VALUES ${sql.join(rows.map((r) => sql`(${sql.join(r)})`))}`);++ n += rows.length; rows = [];++ };++ for (const raw of stmt.iterate()) {++ const row = o.map ? o.map({ ...raw }) : raw;++ if (!row) continue;++ rows.push(cols.map((c) => convert(row[c.column_name], c.udt_name, `${t.name}.${c.column_name}`)));++ if (rows.length >= batch) await flush();++ }++ await flush();++ for (const c of cols.filter((x) => x.identity || /nextval\(/.test(String(x.column_default || '')))) {++ await tx.query(sql`SELECT setval(pg_get_serial_sequence(${t.name}, ${c.column_name}), GREATEST((SELECT max(${sql.ident(c.column_name)}) FROM ${sql.ident(t.name)}), 1))`);++ }++ report.tables.push({ table: t.name, source: from, rows: n, ms: Date.now() - t0 });++ log.log && log.log(`[import] ${t.name}: ${n} rows in ${Date.now() - t0} ms`);++ }++ });++++ if (verify) {++ for (const entry of report.tables) {++ const { t, from, cols, o } = plan.find((p) => p.t.name === entry.table);++ const result = await verifyTable({ src, db, t, from, cols, o });++ entry.checksum = result.pg.sum;++ if (result.sqlite.rows !== result.pg.rows || result.sqlite.sum !== result.pg.sum) {++ problem({ table: t.name, problem: `verification failed: SQLite ${result.sqlite.rows} rows (${result.sqlite.sum}), PostgreSQL ${result.pg.rows} rows (${result.pg.sum})` });++ }++ }++ }++ return report;++ } finally {++ if (typeof sqlite === 'string') src.close();++ }++}++++/**++ * Stream both sides in primary-key order and hash each canonical row: SQLite iterates ORDER BY the key (its++ * BINARY collation is byte order, as COLLATE "C"), PostgreSQL reads keyset pages of 5000 (OFFSET pages for a++ * composite key). A table without a primary key is sorted in memory by every column.++ */++async function verifyTable({ src, db, t, from, cols, o }) {++ const names = cols.map((c) => c.column_name);++ const keys = t.pk.filter((k) => names.includes(k));++ const q = (x) => `"${String(x).replace(/"/g, '""')}"`;++ const text = new Set(cols.filter((c) => ['text', 'varchar', 'bpchar', 'citext', 'uuid'].includes(c.udt_name)).map((c) => c.column_name));++ const line = (vals) => vals.map((v, i) => canonical(v, cols[i].udt_name)).join('\t') + '\n';++ const convertRow = (row) => cols.map((c) => convert(row[c.column_name], c.udt_name, c.column_name));++++ const h1 = crypto.createHash('sha256'); let c1 = 0;++ const where = o.where ? ` WHERE ${o.where}` : '';++ if (keys.length) {++ for (const raw of src.prepare(`SELECT * FROM ${q(from)}${where} ORDER BY ${keys.map(q).join(', ')}`).iterate()) {++ const row = o.map ? o.map({ ...raw }) : raw;++ if (!row) continue;++ h1.update(line(convertRow(row))); c1++;++ }++ } else {++ const all = src.prepare(`SELECT * FROM ${q(from)}${where}`).all().map((raw) => (o.map ? o.map({ ...raw }) : raw)).filter(Boolean).map(convertRow);++ all.sort((a, b) => cmpRows(a, b, names.map((_, i) => i)));++ for (const r of all) { h1.update(line(r)); c1++; }++ }++++ const h2 = crypto.createHash('sha256'); let c2 = 0;++ const colList = sql.join(names.map(sql.ident));++ const orderBy = (ks) => sql.join(ks.map((k) => (text.has(k) ? sql`${sql.ident(k)} COLLATE "C"` : sql.ident(k))));++ const PAGE = 5000;++ if (keys.length === 1) {++ const k = keys[0]; let last = null;++ for (;;) {++ const cond = last === null ? sql`` : (text.has(k) ? sql`WHERE ${sql.ident(k)} COLLATE "C" > ${last}` : sql`WHERE ${sql.ident(k)} > ${last}`);++ const rows = await db.many(sql`SELECT ${colList} FROM ${sql.ident(t.name)} ${cond} ORDER BY ${orderBy([k])} LIMIT ${PAGE}`);++ for (const r of rows) { h2.update(line(names.map((n) => r[n]))); c2++; }++ if (rows.length < PAGE) break;++ last = rows[rows.length - 1][k];++ }++ } else {++ const ks = keys.length ? keys : names;++ for (let off = 0; ; off += PAGE) {++ const rows = await db.many(sql`SELECT ${colList} FROM ${sql.ident(t.name)} ORDER BY ${orderBy(ks)} LIMIT ${PAGE} OFFSET ${off}`);++ for (const r of rows) { h2.update(line(names.map((n) => r[n]))); c2++; }++ if (rows.length < PAGE) break;++ }++ }++ return { sqlite: { rows: c1, sum: h1.digest('hex').slice(0, 16) }, pg: { rows: c2, sum: h2.digest('hex').slice(0, 16) } };++}++++/** Order rows the way PostgreSQL does with COLLATE "C" (byte order for text, numeric for numbers). */++function cmpRows(a, b, idx) {++ for (const i of idx) {++ const x = a[i]; const y = b[i];++ if (x === y) continue;++ if (x === null) return 1; if (y === null) return -1; // NULLS LAST, PostgreSQL's default for ASC++ if (typeof x === 'number' && typeof y === 'number') return x - y;++ const bx = Buffer.from(String(x)); const by = Buffer.from(String(y));++ const c = Buffer.compare(bx, by);++ if (c) return c;++ }++ return 0;++}++++module.exports = { importSqlite, convert, canonical };@@@ -16,6 +16,11 @@* yourself; return null to skip counting (a trusted internal caller). Counters live in this process (a restart* forgets them) and are capped (maxActors, oldest forgotten first), so memory stays bounded under a flood.* Each limit is { minute?, hour?, day? }; a route's own limits replace the defaults window by window.++ *++ * Shared counters (ADR-035): pass `store: createValkeyLimitStore(valkey)` and every process and host counts one++ * actor together (one atomic script per request: check every window, then count). If Valkey errors, that++ * request is counted by this process's own counters instead, so a Valkey outage never opens the gates nor++ * blocks the service.*/const WINDOWS = { minute: 60, hour: 3600, day: 86400 };@@@ -46,7 +51,7 @@ function problem(res, retryAfter, detail) {* @param {(e: {actor, name, window, limit}) => void} [opts.onLimited] called on every refusal (metrics, logs)* @returns {((name: string, limits?: object) => Function) & { stats(): object, reset(): void }}*/--function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maxActors = 50000, now = () => Date.now(), onLimited = null } = {}) {++function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maxActors = 50000, now = () => Date.now(), onLimited = null, store = null, log = console } = {}) {for (const w of Object.keys(limits)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);const counters = new Map(); // `${name}|${actor}` -> { [window]: { start, count } }const stats = { allowed: 0, limited: 0, actors: 0 };@@@ -66,9 +71,29 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maconst merged = { ...limits, ...own };for (const w of Object.keys(merged)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);const windows = Object.entries(merged).filter(([, max]) => Number.isFinite(max) && max >= 0);++ const refuse = (res, who, worst) => {++ stats.limited++;++ if (onLimited) { try { onLimited({ actor: who, name, window: worst.window, limit: worst.max }); } catch { /* observers never block */ } }++ return problem(res, Math.max(1, worst.retry), `${name}: at most ${worst.max} per ${worst.window} for each caller`);++ };return function actorLimit(req, res, next) {const who = actor(req);if (who == null) return next();++ if (store) {++ const t = Math.floor(now() / 1000);++ return store.hit(name, who, windows.map(([w, max]) => [w, max, WINDOWS[w]]), t).then((r) => {++ if (r && r.limited) return refuse(res, who, r);++ stats.allowed++;++ return next();++ }, (err) => {++ stats.storeErrors = (stats.storeErrors || 0) + 1;++ if (stats.storeErrors === 1 || stats.storeErrors % 100 === 0) log.warn(`[limits] shared store failed (${err.message}); counting in this process`);++ return local(req, res, next, who);++ });++ }++ return local(req, res, next, who);++ };++ function local(req, res, next, who) {const t = Math.floor(now() / 1000);const c = touch(`${name}|${who}`);let worst = null;@@@ -80,19 +105,56 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maif (!worst || retry > worst.retry) worst = { window: w, max, retry };}}-- if (worst) {-- stats.limited++;-- if (onLimited) { try { onLimited({ actor: who, name, window: worst.window, limit: worst.max }); } catch { /* observers never block */ } }-- return problem(res, Math.max(1, worst.retry), `${name}: at most ${worst.max} per ${worst.window} for each caller`);-- }++ if (worst) return refuse(res, who, worst);for (const [w] of windows) c[w].count++;stats.allowed++;return next();-- };++ }}middleware.stats = () => ({ ...stats });middleware.reset = () => { counters.clear(); stats.actors = 0; };return middleware;}--module.exports = { createActorLimiter, defaultActor, WINDOWS };++// Check every window, then count every window, atomically. KEYS: one counter per window.++// ARGV: per window max, ttl (seconds), retry (seconds). Returns {0}, or the exceeded window with the longest++// wait as {index (1-based), retry}, as the in-process counters decide.++const HIT_SCRIPT = `++local n = #KEYS++local worst, wr = 0, -1++for i = 1, n do++ local c = tonumber(redis.call('GET', KEYS[i]) or '0')++ if c >= tonumber(ARGV[(i - 1) * 3 + 1]) then++ local r = tonumber(ARGV[(i - 1) * 3 + 3])++ if r > wr then worst = i; wr = r end++ end++end++if worst > 0 then return {worst, wr} end++for i = 1, n do++ if redis.call('INCR', KEYS[i]) == 1 then redis.call('EXPIRE', KEYS[i], ARGV[(i - 1) * 3 + 2]) end++end++return {0}`;++++/** A shared counter store on Valkey for createActorLimiter({ store }). */++function createValkeyLimitStore(valkey) {++ if (!valkey) return null;++ const c = valkey.client;++ if (typeof c.ovLimitHit !== 'function') c.defineCommand('ovLimitHit', { lua: HIT_SCRIPT });++ return {++ async hit(name, actor, windows, t) {++ const keys = []; const args = [];++ for (const [w, max, len] of windows) {++ const start = Math.floor(t / len) * len;++ keys.push(valkey.key('lim', name, w, String(start), actor));++ args.push(String(max), String(len + 5), String(start + len - t));++ }++ if (!keys.length) return { limited: false };++ const r = await c.ovLimitHit(keys.length, ...keys, ...args);++ if (!r || Number(r[0]) === 0) return { limited: false };++ const [w, max] = windows[Number(r[0]) - 1];++ return { limited: true, window: w, max, retry: Number(r[1]) };++ },++ };++}++++module.exports = { createActorLimiter, createValkeyLimitStore, defaultActor, WINDOWS };nnew file mode 100644@@@ -0,0 +1,55 @@++'use strict';++/**++ * openvibe-sdk/pubsub — fan-out between a service's processes and hosts on Valkey (ADR-035), in-process without++ * Valkey. For realtime delivery (chat rooms, live counters, cache invalidation), never for durable work: a++ * message published while nobody listens is gone (use openvibe-sdk/queue or OpenVibe.Events for that).++ *++ * const ps = createPubSub({ valkey });++ * const off = await ps.subscribe('room:42', (msg) => broadcast(msg));++ * await ps.publish('room:42', { text: 'hi' });++ *++ * Channels live inside the service's prefix (its Valkey user may use no others). Messages are JSON.++ */++const { EventEmitter } = require('events');++++function createPubSub({ valkey = null, log = console } = {}) {++ if (!valkey) {++ const ee = new EventEmitter(); ee.setMaxListeners(0);++ return {++ async publish(channel, msg) { const n = ee.listenerCount(channel); ee.emit(channel, JSON.parse(JSON.stringify(msg))); return n; },++ async subscribe(channel, fn) { ee.on(channel, fn); return async () => { ee.off(channel, fn); }; },++ async close() { ee.removeAllListeners(); },++ };++ }++ const handlers = new Map(); // full channel -> Set(fn)++ let sub = null;++ const full = (ch) => valkey.key('ps', ch);++ function subscriber() {++ if (sub) return sub;++ sub = valkey.duplicate();++ sub.on('message', (ch, raw) => {++ const set = handlers.get(ch);++ if (!set) return;++ let msg; try { msg = JSON.parse(raw); } catch { return; }++ for (const fn of set) { try { fn(msg); } catch (err) { log.warn(`[pubsub] handler on ${ch}: ${err.message}`); } }++ });++ return sub;++ }++ return {++ publish: (channel, msg) => valkey.client.publish(full(channel), JSON.stringify(msg)),++ async subscribe(channel, fn) {++ const ch = full(channel);++ if (!handlers.has(ch)) { handlers.set(ch, new Set()); await subscriber().subscribe(ch); }++ handlers.get(ch).add(fn);++ return async () => {++ const set = handlers.get(ch);++ if (!set) return;++ set.delete(fn);++ if (!set.size) { handlers.delete(ch); await subscriber().unsubscribe(ch).catch(() => {}); }++ };++ },++ async close() { handlers.clear(); if (sub) { try { await sub.quit(); } catch { sub.disconnect(); } sub = null; } },++ };++}++++module.exports = { createPubSub };nnew file mode 100644@@@ -0,0 +1,187 @@++'use strict';++/**++ * openvibe-sdk/queue — durable background jobs on Valkey streams (ADR-035), in-process without Valkey (tests).++ *++ * const q = createQueue({ valkey, name: 'thumbnails' });++ * await q.add({ objectId }, { delayMs: 0, attempts: 5, id: `thumb:${objectId}` });++ * const worker = q.process(async (data, job) => { … }, { concurrency: 4, visibilityMs: 60000 });++ * await worker.stop(); // finishes running jobs first++ *++ * At-least-once: a job leaves the stream only after its handler resolves. A worker that dies mid-job leaves it++ * pending, and another worker reclaims it after visibilityMs. A failure retries with exponential backoff++ * (1 s, 2 s, 4 s … capped at 5 min), and after `attempts` tries the job lands in the dead-letter stream with its++ * last error (`q.dead()`). `id` dedupes adds for 24 hours. Handlers must therefore be idempotent. Any number of++ * workers on any number of hosts share one queue through its consumer group.++ */++const crypto = require('crypto');++const os = require('os');++++const backoff = (attempt) => Math.min(300000, 1000 * 2 ** (attempt - 1));++const uid = () => crypto.randomBytes(6).toString('hex');++++// Move due delayed jobs into the stream. Member: "<uid>|<attempt>|<max>|<json>" (no JSON re-encoding in Lua).++const PROMOTE = `++local due = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1], 'LIMIT', 0, 200)++for _, m in ipairs(due) do++ local p1 = string.find(m, '|', 1, true)++ local p2 = string.find(m, '|', p1 + 1, true)++ local p3 = string.find(m, '|', p2 + 1, true)++ redis.call('XADD', KEYS[2], 'MAXLEN', '~', ARGV[2], '*', 'd', string.sub(m, p3 + 1), 'a', string.sub(m, p1 + 1, p2 - 1), 'm', string.sub(m, p2 + 1, p3 - 1))++ redis.call('ZREM', KEYS[1], m)++end++return #due`;++++function fieldsOf(arr) { const o = {}; for (let i = 0; i < arr.length; i += 2) o[arr[i]] = arr[i + 1]; return o; }++++function createQueue({ valkey = null, name, group = 'workers', maxLen = 100000, log = console } = {}) {++ if (!/^[a-z][a-z0-9_.-]{0,63}$/.test(String(name || ''))) throw new TypeError('queue: name must be a short lowercase identifier');++ return valkey ? valkeyQueue(valkey, name, group, maxLen, log) : memoryQueue(name, log);++}++++function valkeyQueue(valkey, name, group, maxLen, log) {++ const c = valkey.client;++ const key = valkey.key('q', name);++ const delayed = valkey.key('q', name, 'delayed');++ const dead = valkey.key('q', name, 'dead');++ const dedupe = (id) => valkey.key('q', name, 'id', id);++ if (typeof c.ovQueuePromote !== 'function') c.defineCommand('ovQueuePromote', { numberOfKeys: 2, lua: PROMOTE });++++ async function ensureGroup() {++ try { await c.xgroup('CREATE', key, group, '0', 'MKSTREAM'); } catch (err) { if (!/BUSYGROUP/.test(err.message)) throw err; }++ }++++ async function add(data, { delayMs = 0, attempts = 5, id = null } = {}) {++ if (id && (await c.set(dedupe(id), '1', 'EX', 86400, 'NX')) !== 'OK') return null;++ const json = JSON.stringify(data === undefined ? null : data);++ if (delayMs > 0) {++ await c.zadd(delayed, Date.now() + delayMs, `${uid()}|1|${attempts}|${json}`);++ return 'delayed';++ }++ return c.xadd(key, 'MAXLEN', '~', maxLen, '*', 'd', json, 'a', '1', 'm', String(attempts));++ }++++ function process(handler, { concurrency = 4, visibilityMs = 60000, blockMs = 5000 } = {}) {++ const consumer = `${os.hostname()}:${global.process.pid}:${uid()}`;++ const reader = valkey.duplicate();++ let stopped = false; let active = 0; let wake = null;++ const stats = { done: 0, failed: 0, retried: 0, dead: 0 };++++ async function run(id, raw) {++ const f = fieldsOf(raw);++ const attempt = Number(f.a) || 1; const max = Number(f.m) || 5;++ active++;++ try {++ await handler(JSON.parse(f.d), { id, attempt, queue: name });++ await c.multi().xack(key, group, id).xdel(key, id).exec();++ stats.done++;++ } catch (err) {++ stats.failed++;++ const reason = String((err && err.message) || err).slice(0, 500);++ const m = c.multi();++ if (attempt >= max) { m.xadd(dead, 'MAXLEN', '~', 10000, '*', 'd', f.d, 'a', String(attempt), 'e', reason, 'at', new Date().toISOString()); stats.dead++; log.warn(`[queue ${name}] job ${id} dead after ${attempt} attempts: ${reason}`); }++ else { m.zadd(delayed, Date.now() + backoff(attempt), `${uid()}|${attempt + 1}|${max}|${f.d}`); stats.retried++; }++ await m.xack(key, group, id).xdel(key, id).exec().catch((e) => log.warn(`[queue ${name}] could not requeue ${id}: ${e.message}`));++ } finally {++ active--;++ if (wake) { const w = wake; wake = null; w(); }++ }++ }++++ async function loop() {++ await ensureGroup();++ while (!stopped) {++ const free = concurrency - active;++ if (free <= 0) { await new Promise((r) => { wake = r; }); continue; }++ let res;++ try { res = await reader.xreadgroup('GROUP', group, consumer, 'COUNT', free, 'BLOCK', blockMs, 'STREAMS', key, '>'); } catch (err) {++ if (stopped) break;++ if (/NOGROUP/.test(err.message)) { await ensureGroup().catch(() => {}); continue; }++ log.warn(`[queue ${name}] read: ${err.message}`);++ await new Promise((r) => setTimeout(r, 1000));++ continue;++ }++ if (!res) continue;++ for (const [, entries] of res) for (const [id, raw] of entries) run(id, raw);++ }++ }++++ const promote = setInterval(() => { c.ovQueuePromote(delayed, key, String(Date.now()), String(maxLen)).catch(() => {}); }, 1000);++ const reclaim = setInterval(async () => {++ if (stopped || active >= concurrency) return;++ try {++ const [, entries] = await c.xautoclaim(key, group, consumer, visibilityMs, '0-0', 'COUNT', Math.max(1, concurrency - active));++ for (const [id, raw] of entries || []) if (raw) run(id, raw);++ } catch { /* the next tick tries again */ }++ }, Math.max(1000, Math.floor(visibilityMs / 2)));++ promote.unref(); reclaim.unref();++ const done = loop().catch((err) => log.error(`[queue ${name}] worker stopped: ${err.message}`));++++ return {++ stats: () => ({ ...stats, active }),++ async stop(timeoutMs = 30000) {++ stopped = true;++ clearInterval(promote); clearInterval(reclaim);++ reader.disconnect();++ if (wake) { const w = wake; wake = null; w(); }++ const until = Date.now() + timeoutMs;++ while (active > 0 && Date.now() < until) await new Promise((r) => setTimeout(r, 50));++ await done;++ },++ };++ }++++ return {++ name, add, process,++ async stats() {++ const [len, del, dd] = await Promise.all([c.xlen(key).catch(() => 0), c.zcard(delayed), c.xlen(dead).catch(() => 0)]);++ let pending = 0;++ try { const p = await c.xpending(key, group); pending = Number(p[0]) || 0; } catch { /* no group yet */ }++ return { waiting: Math.max(0, len - pending), pending, delayed: del, dead: dd };++ },++ async dead(count = 50) {++ const rows = await c.xrevrange(dead, '+', '-', 'COUNT', count);++ return rows.map(([id, raw]) => { const f = fieldsOf(raw); return { id, data: JSON.parse(f.d), attempts: Number(f.a), error: f.e, at: f.at }; });++ },++ };++}++++// ── In-process queue with the same behaviour (tests, single-process tools) ─────────────────++function memoryQueue(name, log) {++ const ready = []; const deadList = []; const seen = new Map(); const workers = new Set();++ let seq = 0;++ const kick = () => { for (const w of workers) w(); };++ function add(data, { delayMs = 0, attempts = 5, id = null } = {}) {++ if (id) { const t = seen.get(id); if (t && t > Date.now()) return Promise.resolve(null); seen.set(id, Date.now() + 86400000); }++ const job = { id: `m-${++seq}`, data: JSON.parse(JSON.stringify(data === undefined ? null : data)), attempt: 1, max: attempts };++ if (delayMs > 0) setTimeout(() => { ready.push(job); kick(); }, delayMs).unref();++ else { ready.push(job); kick(); }++ return Promise.resolve(delayMs > 0 ? 'delayed' : job.id);++ }++ function process(handler, { concurrency = 4, backoffMs = backoff } = {}) {++ let active = 0; let stopped = false;++ const stats = { done: 0, failed: 0, retried: 0, dead: 0 };++ const pump = () => {++ while (!stopped && active < concurrency && ready.length) {++ const job = ready.shift(); active++;++ Promise.resolve().then(() => handler(job.data, { id: job.id, attempt: job.attempt, queue: name })).then(() => { stats.done++; }, (err) => {++ stats.failed++;++ if (job.attempt >= job.max) { stats.dead++; deadList.push({ id: job.id, data: job.data, attempts: job.attempt, error: String((err && err.message) || err), at: new Date().toISOString() }); }++ else { stats.retried++; const next = { ...job, attempt: job.attempt + 1 }; setTimeout(() => { ready.push(next); kick(); }, backoffMs(job.attempt)).unref(); }++ }).finally(() => { active--; pump(); });++ }++ };++ workers.add(pump); pump();++ return {++ stats: () => ({ ...stats, active }),++ async stop(timeoutMs = 30000) { stopped = true; workers.delete(pump); const until = Date.now() + timeoutMs; while (active > 0 && Date.now() < until) await new Promise((r) => setTimeout(r, 10)); },++ };++ }++ void log;++ return {++ name, add, process,++ async stats() { return { waiting: ready.length, pending: 0, delayed: 0, dead: deadList.length }; },++ async dead(count = 50) { return deadList.slice(-count).reverse(); },++ };++}++++module.exports = { createQueue, backoff };nnew file mode 100644@@@ -0,0 +1,46 @@++'use strict';++/**++ * openvibe-sdk/valkey — the shared connection to Valkey (ADR-035): the service's user is confined to its key++ * prefix (VALKEY_PREFIX, e.g. "ov:wiki:"), so every key and channel this SDK touches goes through key().++ * Valkey is never authoritative (ADR-007 2026-09-24 rule 4): caches, limits, queues and fan-out only.++ *++ * const { createValkey } = require('openvibe-sdk/valkey');++ * const valkey = createValkey(); // VALKEY_URL + VALKEY_PREFIX, or { url, prefix }++ * await valkey.client.set(valkey.key('x'), '1', 'EX', 60);++ *++ * Commands are auto-pipelined (many callers in one tick share one round trip). A blocking consumer (queues) or a++ * subscriber gets its own connection through duplicate(). Without a URL, createValkey returns null, and the++ * cache, limits, queue and pubsub modules fall back to their in-process implementations.++ */++++function createValkey({ url = process.env.VALKEY_URL, prefix = process.env.VALKEY_PREFIX || '', lazyConnect = false, log = console, client = null } = {}) {++ if (!url && !client) return null;++ let Valkey;++ if (!client) {++ try { Valkey = require('iovalkey'); } catch { throw new Error('openvibe-sdk/valkey: install `iovalkey` in the service (npm i iovalkey)'); }++ }++ const opts = { maxRetriesPerRequest: 3, enableAutoPipelining: true, connectTimeout: 5000, lazyConnect };++ const c = client || new Valkey(url, opts);++ let lastError = null;++ c.on('error', (err) => { if (!lastError || lastError.message !== err.message) log.warn(`[valkey] ${err.message}`); lastError = err; });++ c.on('ready', () => { lastError = null; });++ const dupes = new Set();++ return {++ client: c,++ prefix,++ /** The full key (or channel) for a name inside this service's prefix. */++ key: (...parts) => prefix + parts.join(':'),++ /** A second connection with the same settings (for BLOCK reads and subscriptions). */++ duplicate() { const d = c.duplicate({ enableAutoPipelining: false }); d.on('error', () => {}); dupes.add(d); return d; },++ /** Readiness: a real PING within 1 s. */++ async ready() {++ try {++ const pong = await Promise.race([c.ping(), new Promise((_, rej) => setTimeout(() => rej(new Error('timeout')), 1000))]);++ return pong === 'PONG' ? { ok: true, detail: { store: 'valkey' } } : { ok: false, error: `unexpected ${pong}` };++ } catch (err) { return { ok: false, error: `valkey: ${err.message}` }; }++ },++ async close() { for (const d of dupes) { try { await d.quit(); } catch { d.disconnect(); } } try { await c.quit(); } catch { c.disconnect(); } },++ };++}++++module.exports = { createValkey };@@@ -76,7 +76,7 @@ run([for (const [re, why] of FORBIDDEN) if (re.test(src)) problems.push(`${path.relative(ROOT, file)}: ${why}`);}assert.deepEqual(problems, []);-- assert.deepEqual(blocked.sort(), ['./events', './identity', './limits', './testing', './vip']);++ assert.deepEqual(blocked.sort(), ['./cache', './db', './events', './identity', './limits', './pubsub', './queue', './testing', './valkey', './vip']);const rel = [...scanned].map((f) => path.relative(ROOT, f));for (const must of ['browser.js', 'src/core/client.js', 'src/auth/browser.js', 'src/realtime.js', 'src/media.js', 'src/community.js', 'src/modules.js', 'src/registry.js', 'src/jobs.js', 'src/tools.js', 'src/core/form.js', 'src/projects.js', 'browser/openvibe-sdk.mjs']) {assert.ok(rel.includes(must), `${must} is scanned`);nnew file mode 100644@@@ -0,0 +1,188 @@++'use strict';++/**++ * db (ADR-035): the sql builder binds every value; both adapters return the same row shapes (int8 → Number,++ * timestamps → ISO, dates → 'YYYY-MM-DD', numeric → exact text, json parsed); transactions commit, roll back++ * and nest as savepoints; serializable conflicts retry; migrations run once, in order, refuse edits, and hold a++ * contract migration inside the N-1 window; importSqlite copies a SQLite file into the schema parents first,++ * converts by target type, keeps identity values, and verifies counts and checksums. The suite runs on PGlite++ * always, and again through PgBouncer against PostgreSQL 18 when OV_TEST_PG_URL is set++ * (scripts/test-services.sh up).++ */++const assert = require('node:assert/strict');++const fs = require('node:fs');++const os = require('node:os');++const path = require('node:path');++const { run } = require('./helpers');++const { createDb, sql, importSqlite } = require('../src/db');++const { parse } = require('../src/db/migrate');++++const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ovsdk-db-'));++const mdir = (name, files) => { const d = path.join(tmp, name); fs.mkdirSync(d, { recursive: true }); for (const [f, text] of Object.entries(files)) fs.writeFileSync(path.join(d, f), text); return d; };++const PAGES = mdir('pages', {++ '0001_pages.sql': `-- phase: expand++CREATE TABLE spaces (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name text NOT NULL);++CREATE TABLE pages (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, space_id bigint REFERENCES spaces(id), slug text NOT NULL UNIQUE,++ title text NOT NULL, meta jsonb NOT NULL DEFAULT '{}', public boolean NOT NULL DEFAULT true, views bigint NOT NULL DEFAULT 0,++ price numeric(10,2), tags text[] NOT NULL DEFAULT '{}', created_at timestamptz NOT NULL DEFAULT now(), day date);++`,++});++++function suite(label, open) {++ return [++ [`${label}: sql binds values, fragments nest, identifiers are checked`, async () => {++ const q = sql`SELECT * FROM t WHERE a = ${1} AND b = ANY(${[2, 3]}) AND ${sql`c = ${'x'}`}`.compile();++ assert.equal(q.text, 'SELECT * FROM t WHERE a = $1 AND b = ANY($2) AND c = $3');++ assert.deepEqual(q.values, [1, [2, 3], 'x']);++ assert.equal(sql`${sql.ident('public.pages')}`.compile().text, '"public"."pages"');++ assert.throws(() => sql.ident('pages; DROP TABLE x'), /not an identifier/);++ const ins = sql`INSERT INTO t ${sql.insert([{ a: 1, b: null }, { a: 2, b: 'y' }])}`.compile();++ assert.equal(ins.text, 'INSERT INTO t ("a", "b") VALUES ($1, $2), ($3, $4)');++ assert.equal(sql`UPDATE t SET ${sql.set({ a: 1, skip: undefined, b: 2 })}`.compile().text, 'UPDATE t SET "a" = $1, "b" = $2');++ }],++ [`${label}: migrations, types, queries, transactions`, async () => {++ const { db, migrate } = await open();++ try {++ const m = await migrate(PAGES);++ assert.deepEqual(m.applied.map((a) => a.id), ['0001']);++ assert.equal((await migrate(PAGES)).applied.length, 0, 'applied once');++ const sp = await db.one(sql`INSERT INTO spaces (name) VALUES (${'main'}) RETURNING id`);++ await db.exec(sql`INSERT INTO pages ${sql.insert([++ { space_id: sp.id, slug: 'a', title: 'A', meta: JSON.stringify({ x: 1, list: [] }), price: '12.50', tags: ['x', 'y'], day: '2026-09-28' },++ { space_id: sp.id, slug: 'b', title: 'B', meta: '{}', public: false, price: null, tags: [], day: null },++ ])}`);++ const a = await db.one(sql`SELECT * FROM pages WHERE slug = ${'a'}`);++ assert.equal(typeof a.id, 'number');++ assert.deepEqual(a.meta, { x: 1, list: [] });++ assert.equal(a.price, '12.50', 'numeric stays exact text');++ assert.deepEqual(a.tags, ['x', 'y']);++ assert.match(a.created_at, /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/);++ assert.equal(a.day, '2026-09-28');++ assert.equal(a.public, true);++ assert.equal(await db.maybe(sql`SELECT 1 FROM pages WHERE slug = ${'zzz'}`), null);++ await assert.rejects(db.one(sql`SELECT 1 FROM pages WHERE slug = ${'zzz'}`), /expected one row/);++ assert.equal(await db.value(sql`SELECT count(*) FROM pages`), 2);++ assert.deepEqual((await db.many(sql`SELECT slug FROM pages WHERE slug = ANY(${['a', 'b']}) ORDER BY slug`)).map((r) => r.slug), ['a', 'b']);++ // A transaction commits; an inner savepoint rolls back alone; a thrown error rolls everything back.++ const title = await db.tx(async (t) => {++ await t.exec(sql`UPDATE pages SET views = views + 1 WHERE slug = 'a'`);++ await t.tx(async (s) => { await s.exec(sql`UPDATE pages SET title = 'X'`); throw new Error('inner'); }).catch(() => {});++ return t.value(sql`SELECT title FROM pages WHERE slug = 'a'`);++ });++ assert.equal(title, 'A');++ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'a'`), 1);++ await assert.rejects(db.tx(async (t) => { await t.exec(sql`UPDATE pages SET views = 99`); throw new Error('boom'); }), /boom/);++ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'a'`), 1, 'rolled back');++ await assert.rejects(db.tx(async () => {}, { isolation: 'nonsense' }), /isolation/);++ // Errors carry the PostgreSQL code and the statement, not the values.++ const err = await db.exec(sql`INSERT INTO pages (slug, title) VALUES (${'a'}, ${'dup'})`).catch((e) => e);++ assert.equal(err.code, '23505');++ assert.match(err.statement, /INSERT INTO pages/);++ // Serializable increments from 8 concurrent transactions all land (retries on 40001).++ await Promise.all(Array.from({ length: 8 }, () => db.tx(async (t) => {++ const v = await t.value(sql`SELECT views FROM pages WHERE slug = 'b'`);++ await t.exec(sql`UPDATE pages SET views = ${v + 1} WHERE slug = 'b'`);++ }, { isolation: 'serializable', retries: 30 })));++ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'b'`), 8);++ const r = await db.ready();++ assert.equal(r.ok, true);++ assert.equal(r.detail.store, label === 'pglite' ? 'pglite' : 'postgresql');++ } finally { await db.close(); }++ }],++ ];++}++++async function pgOpen() {++ // Migrations run as the owner on the direct connection; everything else through PgBouncer, as in production.++ const owner = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL, service: 'sdk-test' });++ await owner.query('DROP TABLE IF EXISTS pages, spaces, ov_migrations CASCADE');++ const db = createDb({ url: process.env.OV_TEST_PG_URL, service: 'sdk-test', max: 4 });++ const close = db.close;++ db.close = async () => { await close(); await owner.close(); };++ return { db, migrate: (dir) => owner.migrate({ dir, log: { log() {} } }) };++}++++const tests = [++ ...suite('pglite', async () => { const db = createDb({ pglite: true, service: 'sdk-test' }); return { db, migrate: (dir) => db.migrate({ dir, log: { log() {} } }) }; }),++ ['migration rules: phases, edits refused, order kept, contract held in the N-1 window, no-transaction', async () => {++ assert.throws(() => parse(mdir('bad1', { 'x.sql': '-- phase: expand\nSELECT 1;' })), /NNNN_description/);++ assert.throws(() => parse(mdir('bad2', { '0001_a.sql': 'SELECT 1;' })), /phase/);++ assert.throws(() => parse(mdir('bad3', { '0001_a.sql': '-- phase: contract\nSELECT 1;' })), /must name its expand/);++ const db = createDb({ pglite: true });++ const quiet = { log() {} };++ try {++ const d = mdir('rules', {++ '0001_add.sql': '-- phase: expand\nCREATE TABLE t (id int PRIMARY KEY, old text, new text);',++ '0002_fill.sql': '-- phase: migrate\nUPDATE t SET new = old;',++ '0003_drop.sql': '-- phase: contract\n-- after: 0001\nALTER TABLE t DROP COLUMN old;',++ '0004_idx.sql': '-- phase: expand\n-- no-transaction\nCREATE INDEX t_new ON t (new);',++ });++ let r = await db.migrate({ dir: d, log: quiet });++ assert.deepEqual(r.applied.map((a) => a.id), ['0001', '0002']);++ assert.match(r.held[0].reason, /N-1 window is 7/);++ r = await db.migrate({ dir: d, log: quiet, now: () => Date.now() + 8 * 86400000 });++ assert.deepEqual(r.applied.map((a) => a.id), ['0003', '0004'], 'after the window the contract runs, then the rest');++ fs.writeFileSync(path.join(d, '0002_fill.sql'), '-- phase: migrate\nUPDATE t SET new = upper(old);');++ await assert.rejects(db.migrate({ dir: d, log: quiet }), /changed after it was applied/);++ } finally { await db.close(); }++ const db2 = createDb({ pglite: true });++ try {++ const d2 = mdir('order', { '0001_add.sql': '-- phase: expand\nCREATE TABLE u (id int);', '0005_b.sql': '-- phase: expand\nSELECT 1;' });++ await db2.migrate({ dir: d2, log: quiet });++ fs.writeFileSync(path.join(d2, '0003_late.sql'), '-- phase: expand\nSELECT 1;');++ await assert.rejects(db2.migrate({ dir: d2, log: quiet }), /older than applied migration 0005/);++ } finally { await db2.close(); }++ }],++ ['importSqlite: parents first, conversions by target type, identities kept, verified', async () => {++ const Database = require('better-sqlite3');++ const file = path.join(tmp, 'legacy.db');++ const s = new Database(file);++ s.exec(`CREATE TABLE people (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT NOT NULL, is_admin INTEGER DEFAULT 0, settings TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, legacy_note TEXT);++ CREATE TABLE notes (id INTEGER PRIMARY KEY, person_id INTEGER REFERENCES people(id), body TEXT, pinned INTEGER, posted INTEGER, tags TEXT);++ CREATE TABLE loose (k TEXT, v TEXT);`);++ const ins = s.prepare('INSERT INTO people (id, username, is_admin, settings, created_at, legacy_note) VALUES (?, ?, ?, ?, ?, ?)');++ ins.run(3, 'ann', 1, '{"theme":"blue","n":[1,2]}', '2026-09-01 10:00:00', 'drop me');++ ins.run(7, 'bob', 0, null, '2026-09-02T11:30:00.000Z', null);++ ins.run(12, 'zoë', 0, '{}', '2026-09-03 09:15:00', null);++ s.prepare('INSERT INTO notes VALUES (?, ?, ?, ?, ?, ?)').run(1, 3, 'hello', 1, 1790000000, '["a","b"]');++ s.prepare('INSERT INTO notes VALUES (?, ?, ?, ?, ?, ?)').run(2, 7, 'ms time', 0, 1790000000123, '[]');++ s.prepare('INSERT INTO loose VALUES (?, ?)').run('b', '2');++ s.prepare('INSERT INTO loose VALUES (?, ?)').run('a', '1');++ s.close();++++ const db = createDb({ pglite: true });++ try {++ await db.migrate({ dir: mdir('imp', { '0001_s.sql': `-- phase: expand++CREATE TABLE people (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, username text NOT NULL, is_admin boolean NOT NULL DEFAULT false, settings jsonb, created_at timestamptz NOT NULL DEFAULT now());++CREATE TABLE notes (id bigint PRIMARY KEY, person_id bigint REFERENCES people(id), body text, pinned boolean, posted timestamptz, tags text[] NOT NULL DEFAULT '{}');++CREATE TABLE loose (k text, v text);` }), log: { log() {} } });++ const quiet = { log() {} };++ // A source column with no target column is a problem until it is dropped on purpose.++ let r = await importSqlite({ sqlite: file, db, log: quiet });++ assert.equal(r.ok, false);++ assert.match(r.problems[0].problem, /legacy_note/);++ r = await importSqlite({ sqlite: file, db, log: quiet, tables: { people: { dropColumns: ['legacy_note'] } } });++ assert.equal(r.ok, true, JSON.stringify(r.problems));++ assert.deepEqual(r.tables.map((t) => [t.table, t.rows]), [['loose', 2], ['people', 3], ['notes', 2]], 'parents before children');++ const ann = await db.one(sql`SELECT * FROM people WHERE id = 3`);++ assert.deepEqual([ann.username, ann.is_admin, ann.settings, ann.created_at], ['ann', true, { theme: 'blue', n: [1, 2] }, '2026-09-01T10:00:00.000Z']);++ const n2 = await db.one(sql`SELECT * FROM notes WHERE id = 2`);++ assert.deepEqual([n2.pinned, n2.posted, n2.tags], [false, new Date(1790000000123).toISOString(), []]);++ assert.equal((await db.one(sql`SELECT * FROM notes WHERE id = 1`)).posted, new Date(1790000000 * 1000).toISOString(), 'epoch seconds');++ const next = await db.one(sql`INSERT INTO people (username) VALUES ('new') RETURNING id`);++ assert.equal(next.id, 13, 'the identity continues after the imported maximum');++ // A second import onto the same rows fails loudly; truncate makes a rehearsal repeatable.++ r = await importSqlite({ sqlite: file, db, log: quiet, truncate: true, tables: { people: { dropColumns: ['legacy_note'] } } });++ assert.equal(r.ok, true);++ // A transformation that differs between copy and check shows up as a failed verification.++ let flip = 0;++ r = await importSqlite({ sqlite: file, db, log: quiet, truncate: true, only: ['loose'], tables: { loose: { map: (row) => ({ ...row, v: String(flip++) }) } } });++ assert.equal(r.ok, false);++ assert.match(r.problems[0].problem, /verification failed/);++ } finally { await db.close(); }++ }],++];++++if (process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL) tests.push(...suite('postgresql+pgbouncer', pgOpen));++else console.log('db through PgBouncer: skipped (OV_TEST_PG_URL not set; scripts/test-services.sh up)');++++run(tests).then(() => fs.rmSync(tmp, { recursive: true, force: true }));@@@ -66,7 +66,7 @@ run([})().catch((err) => { console.error(err); process.exit(1); });`;const res = execFileSync(process.execPath, ['-e', script], { cwd: app, encoding: 'utf8' });assert.match(res, new RegExp(`consumer ok: ${subpaths.length} subpaths \\+ the browser bundle`));-- assert.equal(subpaths.length, 21);++ assert.equal(subpaths.length, 26);assert.ok(!fs.existsSync(path.join(app, 'node_modules/openvibe-contracts')), 'the optional peer is not installed');} finally {fs.rmSync(tmp, { recursive: true, force: true });nnew file mode 100644@@@ -0,0 +1,168 @@++'use strict';++/**++ * valkey, cache, limits store, queue, pubsub (ADR-035): the in-process versions always; against Valkey 9 when++ * OV_TEST_VALKEY_URL is set (scripts/test-services.sh up). With Valkey: getOrSet loads once across two++ * connections, tags invalidate, two limiters share one count and fall back to their own when Valkey fails, the++ * queue retries with backoff, dead-letters, delays, dedupes and reclaims a job a hung worker holds, pub/sub++ * crosses connections, and every key stays inside the service prefix.++ */++const assert = require('node:assert/strict');++const crypto = require('node:crypto');++const { run, sleep } = require('./helpers');++const { createValkey } = require('../src/valkey');++const { createCache } = require('../src/cache');++const { createQueue } = require('../src/queue');++const { createPubSub } = require('../src/pubsub');++const { createActorLimiter, createValkeyLimitStore } = require('../src/limits');++++const quiet = { log() {}, warn() {}, error() {} };++function call(mw, req) {++ return new Promise((resolve) => {++ const res = { statusCode: 200, headers: {}, setHeader(k, v) { this.headers[k.toLowerCase()] = v; }, end: (body) => resolve({ status: res.statusCode, retry: res.headers['retry-after'], body }) };++ mw(req, res, () => resolve({ status: 200 }));++ });++}++const until = async (fn, ms = 5000) => { const end = Date.now() + ms; while (Date.now() < end) { if (await fn()) return true; await sleep(25); } return false; };++++const tests = [++ ['memory cache: single-flight, TTL, tags', async () => {++ const cache = createCache({ namespace: 'm', ttlSec: 60, log: quiet });++ let loads = 0;++ const vals = await Promise.all(Array.from({ length: 10 }, () => cache.getOrSet('k', 60, async () => { loads++; await sleep(20); return { v: 1 }; })));++ assert.equal(loads, 1);++ assert.deepEqual(vals[9], { v: 1 });++ await cache.set('t1', 'a', 60, { tags: ['space:1'] }); await cache.set('t2', 'b', 60, { tags: ['space:1'] });++ assert.equal(await cache.invalidateTag('space:1'), 2);++ assert.equal(await cache.get('t1'), undefined);++ assert.equal(cache.stats().store, 'memory');++ }],++ ['memory queue: runs, retries, dead-letters, dedupes', async () => {++ const q = createQueue({ name: 'jobs' });++ const seen = []; let flaky = 0;++ const w = q.process(async (d) => {++ if (d.kind === 'flaky' && flaky++ < 1) throw new Error('once');++ if (d.kind === 'bad') throw new Error('always');++ seen.push(d.n);++ }, { concurrency: 2, backoffMs: () => 5 });++ await q.add({ n: 1 }); await q.add({ n: 2 }); await q.add({ kind: 'flaky', n: 3 });++ await q.add({ kind: 'bad' }, { attempts: 2 });++ assert.equal(await q.add({ n: 9 }, { id: 'same' }) !== null, true);++ assert.equal(await q.add({ n: 9 }, { id: 'same' }), null, 'deduped');++ assert.ok(await until(async () => seen.length === 4 && (await q.stats()).dead === 1));++ assert.deepEqual(seen.sort(), [1, 2, 3, 9]);++ assert.equal((await q.dead())[0].error, 'always');++ await w.stop();++ }],++ ['memory pubsub', async () => {++ const ps = createPubSub();++ const got = [];++ const off = await ps.subscribe('room:1', (m) => got.push(m));++ await ps.publish('room:1', { t: 'hi' });++ await off();++ await ps.publish('room:1', { t: 'gone' });++ assert.deepEqual(got, [{ t: 'hi' }]);++ }],++];++++const URL = process.env.OV_TEST_VALKEY_URL;++if (!URL) console.log('valkey integration: skipped (OV_TEST_VALKEY_URL not set; scripts/test-services.sh up)');++else {++ const prefix = `ovsdk:test:${crypto.randomBytes(3).toString('hex')}:`;++ const open = () => createValkey({ url: URL, prefix, log: quiet });++ tests.push(++ ['valkey: ready, cache single-flight across connections, tags', async () => {++ const a = open(); const b = open();++ try {++ await a.client.flushdb();++ assert.equal((await a.ready()).ok, true);++ const ca = createCache({ valkey: a, namespace: 'pages', log: quiet }); const cb = createCache({ valkey: b, namespace: 'pages', log: quiet });++ let loads = 0;++ const loader = async () => { loads++; await sleep(150); return { title: 'A' }; };++ const [x, y] = await Promise.all([ca.getOrSet('slug:a', 60, loader), cb.getOrSet('slug:a', 60, loader)]);++ assert.deepEqual([x, y], [{ title: 'A' }, { title: 'A' }]);++ assert.equal(loads, 1, 'the second process waited for the first one\'s value');++ await ca.set('p1', 1, 60, { tags: ['space:7'] });++ assert.equal(await cb.invalidateTag('space:7'), 1);++ assert.equal(await ca.get('p1'), undefined);++ } finally { await a.close(); await b.close(); }++ }],++ ['valkey: two limiters share one count; a failing store falls back', async () => {++ const a = open(); const b = open();++ try {++ let t = 1_000_000_020_000;++ const la = createActorLimiter({ limits: { minute: 3 }, now: () => t, store: createValkeyLimitStore(a), log: quiet });++ const lb = createActorLimiter({ limits: { minute: 3 }, now: () => t, store: createValkeyLimitStore(b), log: quiet });++ const req = { user: { subject_id: 'usr_x' } };++ const out = [];++ for (const lim of [la, lb, la, lb]) out.push((await call(lim('wiki.page.write'), req)).status);++ assert.deepEqual(out, [200, 200, 200, 429], 'three per minute across both processes');++ t += 60_000;++ assert.equal((await call(lb('wiki.page.write'), req)).status, 200, 'the next window reopens');++ // Valkey gone: the limiter keeps counting in its own process.++ const dead = createValkey({ url: 'redis://127.0.0.1:1/0', prefix, log: quiet });++ dead.client.options.maxRetriesPerRequest = 0; dead.client.options.enableOfflineQueue = false;++ const lf = createActorLimiter({ limits: { minute: 1 }, now: () => t, store: createValkeyLimitStore(dead), log: quiet });++ assert.equal((await call(lf('x'), req)).status, 200);++ assert.equal((await call(lf('x'), req)).status, 429, 'counted locally while the store fails');++ dead.client.disconnect();++ } finally { await a.close(); await b.close(); }++ }],++ ['valkey queue: processes, retries with backoff, dead-letters, delays, dedupes', async () => {++ const v = open();++ try {++ const q = createQueue({ valkey: v, name: 'thumbs', log: quiet });++ const done = []; let flaky = 0;++ const w = q.process(async (d) => {++ if (d.kind === 'flaky' && flaky++ < 1) throw new Error('once');++ if (d.kind === 'bad') throw new Error('always');++ done.push(d.n);++ }, { concurrency: 3, blockMs: 200 });++ await q.add({ n: 1 }); await q.add({ n: 2 }); await q.add({ kind: 'flaky', n: 3 });++ await q.add({ kind: 'bad', n: 4 }, { attempts: 2 });++ await q.add({ n: 5 }, { delayMs: 300 });++ assert.notEqual(await q.add({ n: 6 }, { id: 'thumb:6' }), null);++ assert.equal(await q.add({ n: 6 }, { id: 'thumb:6' }), null, 'deduped');++ assert.ok(await until(async () => done.length === 5 && (await q.stats()).dead === 1, 8000), `done ${done}`);++ assert.deepEqual(done.sort(), [1, 2, 3, 5, 6]);++ const [d] = await q.dead();++ assert.deepEqual([d.data, d.attempts, d.error], [{ kind: 'bad', n: 4 }, 2, 'always']);++ const st = await q.stats();++ assert.deepEqual([st.waiting, st.pending, st.delayed], [0, 0, 0]);++ await w.stop();++ } finally { await v.close(); }++ }],++ ['valkey queue: a job a hung worker holds is reclaimed after the visibility timeout', async () => {++ const v1 = open(); const v2 = open();++ try {++ const q1 = createQueue({ valkey: v1, name: 'reclaim', log: quiet });++ const q2 = createQueue({ valkey: v2, name: 'reclaim', log: quiet });++ let took = false;++ const hung = q1.process(async () => { took = true; await new Promise(() => {}); }, { concurrency: 1, blockMs: 200, visibilityMs: 60000 });++ await q1.add({ n: 'x' });++ assert.ok(await until(async () => took));++ const got = [];++ const rescuer = q2.process(async (d, job) => { got.push([d.n, job.attempt]); }, { concurrency: 1, blockMs: 200, visibilityMs: 600 });++ assert.ok(await until(async () => got.length === 1, 6000), 'reclaimed');++ assert.deepEqual(got[0], ['x', 1]);++ await rescuer.stop(); await hung.stop(100);++ } finally { await v1.close(); await v2.close(); }++ }],++ ['valkey pubsub across connections; every key stays inside the prefix', async () => {++ const a = open(); const b = open();++ try {++ const pa = createPubSub({ valkey: a, log: quiet }); const pb = createPubSub({ valkey: b, log: quiet });++ const got = [];++ const off = await pb.subscribe('room:1', (m) => got.push(m));++ assert.ok(await until(async () => (await pa.publish('room:1', { t: 'hi' })) === 1));++ assert.ok(await until(async () => got.length >= 1));++ assert.deepEqual(got[0], { t: 'hi' });++ await off(); await pb.close();++ const keys = await a.client.keys('*');++ assert.ok(keys.length > 0);++ assert.deepEqual(keys.filter((k) => !k.startsWith(prefix)), [], 'no key outside the prefix');++ } finally { await a.close(); await b.close(); }++ }],++ );++}++++run(tests).then(() => process.exit(0));nnew file mode 100644@@@ -0,0 +1,12 @@++/** openvibe-sdk/cache — shared caching on Valkey, in-process without it; never authoritative (ADR-035). */++import type { Valkey } from './valkey';++export interface Cache {++ get<T = unknown>(key: string): Promise<T | undefined>;++ set(key: string, value: unknown, ttlSec?: number, opts?: { tags?: string[] }): Promise<void>;++ del(...keys: string[]): Promise<number>;++ /** One load per key across processes (single-flight plus a short lock). */++ getOrSet<T>(key: string, ttlSec: number, loader: () => Promise<T>, opts?: { tags?: string[] }): Promise<T>;++ invalidateTag(tag: string): Promise<number>;++ stats(): { hits: number; misses: number; loads: number; errors: number; store: 'valkey' | 'memory' };++}++export function createCache(opts?: { valkey?: Valkey | null; namespace?: string; ttlSec?: number; lockMs?: number; waitMs?: number; memoryMax?: number; log?: { warn(msg: string): void } }): Cache;nnew file mode 100644@@@ -0,0 +1,64 @@++/** openvibe-sdk/db — the async PostgreSQL data layer (ADR-035). */++export declare class Sql {++ readonly parts: string[];++ readonly values: unknown[];++ compile(start?: number): { text: string; values: unknown[] };++}++export interface SqlTag {++ (strings: TemplateStringsArray, ...values: unknown[]): Sql;++ /** "name" or "schema"."name"; throws on anything else. */++ ident(name: string): Sql;++ /** Trusted text, inserted as is. Never user input. */++ raw(text: string): Sql;++ join(items: Iterable<unknown>, sep?: Sql): Sql;++ set(obj: Record<string, unknown>): Sql;++ insert(rows: Record<string, unknown> | Record<string, unknown>[], columns?: string[]): Sql;++ json(value: unknown): Sql;++}++export const sql: SqlTag;++++export type Row = Record<string, any>;++export interface Queryable {++ query(q: Sql | string, values?: unknown[]): Promise<{ rows: Row[]; rowCount: number }>;++ many<T = Row>(q: Sql | string, values?: unknown[]): Promise<T[]>;++ maybe<T = Row>(q: Sql | string, values?: unknown[]): Promise<T | null>;++ /** Throws DbError (code 'no_rows') when there is none. */++ one<T = Row>(q: Sql | string, values?: unknown[]): Promise<T>;++ value<T = unknown>(q: Sql | string, values?: unknown[]): Promise<T | null>;++ exec(q: Sql | string, values?: unknown[]): Promise<number>;++}++export interface Tx extends Queryable {++ /** A nested savepoint: rolls back alone when fn throws. */++ tx<T>(fn: (t: Tx) => Promise<T>): Promise<T>;++}++export interface TxOptions { isolation?: 'read committed' | 'repeatable read' | 'serializable'; retries?: number; readOnly?: boolean }++export interface MigrateResult { applied: { id: string; name: string; phase: string; ms: number }[]; pending: { id: string; name: string; phase: string }[]; held: { id: string; reason: string }[] }++export interface Db extends Queryable {++ sql: SqlTag;++ readonly store: 'postgresql' | 'pglite';++ tx<T>(fn: (t: Tx) => Promise<T>, opts?: TxOptions): Promise<T>;++ ready(): Promise<{ ok: true; detail: { store: string; pool: { total: number; idle: number; waiting: number } } } | { ok: false; error: string }>;++ migrate(o: { dir: string; windowDays?: number; dryRun?: boolean; now?: () => number; log?: { log(msg: string): void } }): Promise<MigrateResult>;++ stats(): { queries: number; errors: number; slow: number; retries: number; tx: number; pool: { total: number; idle: number; waiting: number } };++ close(): Promise<void>;++}++export interface CreateDbOptions {++ /** Default: DATABASE_URL (through PgBouncer). */++ url?: string;++ /** true: in-memory PGlite (tests); a directory: persisted; or a PGlite instance. */++ pglite?: boolean | string | object;++ service?: string;++ max?: number;++ queryTimeoutMs?: number;++ slowMs?: number;++ log?: { warn(msg: string): void; error(msg: string): void };++ registry?: object;++}++export function createDb(opts?: CreateDbOptions): Db;++export declare class DbError extends Error { code?: string; detail?: string; constraint?: string; table?: string; column?: string; statement?: string; cause?: unknown }++export interface ImportTableOptions { from?: string; dropColumns?: string[]; where?: string; map?: (row: Row) => Row | null }++export interface ImportReport { ok: boolean; tables: { table: string; source: string; rows: number; ms: number; checksum?: string }[]; problems: { table: string; problem: string }[] }++/** The one-time SQLite → PostgreSQL move with verification (needs better-sqlite3). */++export function importSqlite(o: { sqlite: string | object; db: Db; tables?: Record<string, ImportTableOptions>; only?: string[]; truncate?: boolean; verify?: boolean; log?: { log(msg: string): void } }): Promise<ImportReport>;++export const PARSERS: Record<number, (v: string) => unknown>;++export const ISOLATION: Record<string, string>;@@@ -10,7 +10,10 @@ export interface ActorLimiter {stats(): { allowed: number; limited: number; actors: number };reset(): void;}--export function createActorLimiter(opts?: { limits?: WindowLimits; actor?: ActorOf; maxActors?: number; now?: () => number; onLimited?: (e: LimitedEvent) => void }): ActorLimiter;++export interface LimitStore { hit(name: string, actor: string, windows: [string, number, number][], nowSec: number): Promise<{ limited: false } | { limited: true; window: string; max: number; retry: number }> }++export function createActorLimiter(opts?: { limits?: WindowLimits; actor?: ActorOf; maxActors?: number; now?: () => number; onLimited?: (e: LimitedEvent) => void; store?: LimitStore | null; log?: { warn(msg: string): void } }): ActorLimiter;++/** Shared counters on Valkey: every process and host counts an actor together (null without Valkey). */++export function createValkeyLimitStore(valkey: import('./valkey').Valkey | null): LimitStore | null;/** req.principal.sub, else the signed-in person (user:<subject>), else ip:<address>. */export const defaultActor: ActorOf;export const WINDOWS: { minute: 60; hour: 3600; day: 86400 };nnew file mode 100644@@@ -0,0 +1,9 @@++/** openvibe-sdk/pubsub — fan-out between processes on Valkey, in-process without it (ADR-035). Not durable. */++import type { Valkey } from './valkey';++export interface PubSub {++ publish(channel: string, message: unknown): Promise<number>;++ /** Resolves to an unsubscribe function. */++ subscribe(channel: string, handler: (message: any) => void): Promise<() => Promise<void>>;++ close(): Promise<void>;++}++export function createPubSub(opts?: { valkey?: Valkey | null; log?: { warn(msg: string): void } }): PubSub;nnew file mode 100644@@@ -0,0 +1,14 @@++/** openvibe-sdk/queue — durable at-least-once jobs on Valkey streams, in-process without Valkey (ADR-035). */++import type { Valkey } from './valkey';++export interface JobInfo { id: string; attempt: number; queue: string }++export interface Worker { stats(): { done: number; failed: number; retried: number; dead: number; active: number }; stop(timeoutMs?: number): Promise<void> }++export interface Queue {++ name: string;++ /** Returns the job id, 'delayed', or null when `id` was already queued in the last 24 hours. */++ add(data: unknown, opts?: { delayMs?: number; attempts?: number; id?: string }): Promise<string | null>;++ process(handler: (data: any, job: JobInfo) => Promise<unknown>, opts?: { concurrency?: number; visibilityMs?: number; blockMs?: number; backoffMs?: (attempt: number) => number }): Worker;++ stats(): Promise<{ waiting: number; pending: number; delayed: number; dead: number }>;++ dead(count?: number): Promise<{ id: string; data: any; attempts: number; error: string; at: string }[]>;++}++export function createQueue(opts: { valkey?: Valkey | null; name: string; group?: string; maxLen?: number; log?: object }): Queue;++export function backoff(attempt: number): number;nnew file mode 100644@@@ -0,0 +1,13 @@++/** openvibe-sdk/valkey — the shared Valkey connection, confined to the service prefix (ADR-035). */++export interface Valkey {++ /** The iovalkey client (auto-pipelined). */++ client: any;++ prefix: string;++ /** The full key or channel name inside the prefix. */++ key(...parts: string[]): string;++ duplicate(): any;++ ready(): Promise<{ ok: true; detail: { store: 'valkey' } } | { ok: false; error: string }>;++ close(): Promise<void>;++}++/** null when no URL is configured (the cache, limits, queue and pubsub modules then run in-process). */++export function createValkey(opts?: { url?: string; prefix?: string; lazyConnect?: boolean; log?: { warn(msg: string): void }; client?: any }): Valkey | null;
@@ -10,6 +10,8 @@ jobs:
@@ -10,6 +10,8 @@ jobs:
with:
with:
syntax-dirs: src scripts
syntax-dirs: src scripts
extra: node scripts/esm.js --check # the generated ESM wrappers match src/
extra: node scripts/esm.js --check # the generated ESM wrappers match src/
+ # db, cache, queue, pubsub and limits against PostgreSQL 18 + PgBouncer + Valkey 9 containers (ADR-035)
+ test: 'eval "$(scripts/test-services.sh up)" && npm test'
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo.
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo.
security:
security:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@66b76f868ea8228aad7811309eafef3726b036f3
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@66b76f868ea8228aad7811309eafef3726b036f3
@@ -3,6 +3,27 @@
@@ -3,6 +3,27 @@
All notable changes to `openvibe-sdk`. The package follows semver; while it is `0.x`, a minor
All notable changes to `openvibe-sdk`. The package follows semver; while it is `0.x`, a minor
release may change an API and says so here.
release may change an API and says so here.
+## 0.14.0 (2026-09-28)
+
+**The async data layer and Valkey modules** (ADR-035 and its amendment; roadmap WS-X2): every service moves to PostgreSQL 18 and Valkey, async-first.
+- **`openvibe-sdk/db`:**
+ - **Queries:** `createDb({ url | pglite, service })` has `query`, `many`, `maybe`, `one`, `value` and `exec` on a safe `sql` tagged template: every value is a bind parameter, fragments nest, and `sql.ident`, `sql.set`, `sql.insert`, `sql.join` and `sql.json` help build them.
+ - **Transactions:** `tx(fn, { isolation, retries })` retries 40001 and 40P01 with backoff, and nested `t.tx()` calls are savepoints.
+ - **Health:** `ready()` reports the store that actually answered, with the pool state. There are metrics (`db_query_seconds`, pool gauges) and a slow-query log that shows text only, never values.
+ - **Adapters:** node-postgres, which production uses through PgBouncer in transaction mode, and PGlite for tests (real PostgreSQL in-process). Both return the same rows: int8 as Number (refused beyond 2^53), timestamps as ISO strings, dates as `YYYY-MM-DD`, numeric as exact text, json parsed.
+- **Migrations:** `db.migrate({ dir })` runs numbered SQL files labelled `expand`, `migrate` or `contract`. It records them in a ledger with checksums, refuses edited or out-of-order files, serialises runs with an advisory lock, holds a contract migration until its expand is 7 days old (the N-1 window), and supports `-- no-transaction`.
+- **`importSqlite()`:** the one-time move of a service's SQLite file into its PostgreSQL schema.
+ - Tables go parents first, and values are converted by target type (0/1 to boolean, text or epoch to timestamptz, JSON text to jsonb, JSON arrays to arrays).
+ - Identity values are kept and sequences advanced. A source column that would be lost is refused unless dropped on purpose.
+ - Streamed verification compares counts and content checksums.
+- **`openvibe-sdk/valkey`:** the shared connection. It is auto-pipelined, confined to `VALKEY_PREFIX`, has a `duplicate()` for blocking consumers, and `ready()`.
+- **`openvibe-sdk/cache`:** `getOrSet` loads a key once across processes (single-flight plus a short lock), with tags, TTLs and an in-process fallback. A cache error never fails the caller.
+- **`openvibe-sdk/queue`:** at-least-once jobs on Valkey streams. It has consumer groups, retries with exponential backoff, a dead-letter stream, delayed jobs, a 24-hour dedupe by id, and reclaiming of a job a dead or hung worker holds. It also has an in-process twin.
+- **`openvibe-sdk/pubsub`:** fan-out across processes and hosts, inside the prefix.
+- **`openvibe-sdk/limits`:** `createValkeyLimitStore(valkey)` makes every process and host count an actor together, in one atomic script per request. If Valkey fails, that request is counted in-process instead.
+- **Dependencies:** `pg`, `@electric-sql/pglite`, `iovalkey` and `better-sqlite3` are optional peers, loaded only by the module that needs them.
+- **Tests:** `scripts/test-services.sh up` starts PostgreSQL 18, PgBouncer and Valkey 9 in containers, and `test/db.test.js` and `test/valkey.test.js` run against them when their URLs are set. Otherwise they run on PGlite and the in-process stores, and print skip lines.
+
## 0.13.0 (2026-09-28)
## 0.13.0 (2026-09-28)
**Deliveries are v2 only**, as OpenVibe.Events sends them since 2026-09-28 (the v1 header, a replayable HMAC of the body alone, is retired: shim C-60). `signDeliveryHeaders` returns `X-OpenVibe-Timestamp` and `X-OpenVibe-Signature-V2` only, and the mock platform's `deliverEvents` sends no `X-OpenVibe-Signature`. A consumer test that asserted the v1 header must drop that assertion. `signDelivery` and `verifyDelivery` stay for testing a legacy receiver, and `parseDelivery` still accepts a v1-only delivery unless `requireV2`, which every production consumer sets.
**Deliveries are v2 only**, as OpenVibe.Events sends them since 2026-09-28 (the v1 header, a replayable HMAC of the body alone, is retired: shim C-60). `signDeliveryHeaders` returns `X-OpenVibe-Timestamp` and `X-OpenVibe-Signature-V2` only, and the mock platform's `deliverEvents` sends no `X-OpenVibe-Signature`. A consumer test that asserted the v1 header must drop that assertion. `signDelivery` and `verifyDelivery` stay for testing a legacy receiver, and `parseDelivery` still accepts a v1-only delivery unless `requireV2`, which every production consumer sets.
@@ -338,6 +338,30 @@ The mock answers at the real public origins with real RS256 tokens and checks au
@@ -338,6 +338,30 @@ The mock answers at the real public origins with real RS256 tokens and checks au
Helpers: `signUserToken()`, `signServiceToken()`, `signAppToken()`, `authorize()`, `setAuthorization()`, `publishEvent(envelope, publisher, { projectId, env })` (a registered app's `app:<id>` publisher implies them), `pruneEvents()`, `deliverEvents()`, `dropRealtime()`, `dropJobStreams()`, `addTool()`, `addMediaObject()`, and `stats` and `state` for assertions. Its `fetch` rejects on an aborted signal, as `fetch` does. It is a fake. It has no persistence, its visibility rules are simplified, and it has no Chat (no WebSocket mock).
Helpers: `signUserToken()`, `signServiceToken()`, `signAppToken()`, `authorize()`, `setAuthorization()`, `publishEvent(envelope, publisher, { projectId, env })` (a registered app's `app:<id>` publisher implies them), `pruneEvents()`, `deliverEvents()`, `dropRealtime()`, `dropJobStreams()`, `addTool()`, `addMediaObject()`, and `stats` and `state` for assertions. Its `fetch` rejects on an aborted signal, as `fetch` does. It is a fake. It has no persistence, its visibility rules are simplified, and it has no Chat (no WebSocket mock).
+### Data: PostgreSQL and Valkey (server)
+
+Every service stores its data in PostgreSQL 18 through PgBouncer, and shares caches, limits, queues and fan-out through Valkey (ADR-035). The data role on the host provides both; `roles/data/add-service.sh` in OpenVibe.Host sets `DATABASE_URL`, `DATABASE_DIRECT_URL`, `VALKEY_URL` and `VALKEY_PREFIX`.
+
+```js
+const { createDb, sql } = require('openvibe-sdk/db');
+const { createValkey } = require('openvibe-sdk/valkey');
+const { createCache } = require('openvibe-sdk/cache');
+const { createQueue } = require('openvibe-sdk/queue');
+
+const db = createDb({ service: 'wiki' }); // DATABASE_URL; tests: createDb({ pglite: true })
+await createDb({ url: process.env.DATABASE_DIRECT_URL }).migrate({ dir: 'migrations' });
+const page = await db.maybe(sql`SELECT id, title FROM pages WHERE slug = ${slug}`);
+await db.tx(async (t) => { … }, { isolation: 'serializable' });
+
+const valkey = createValkey(); // null without VALKEY_URL: in-process fallbacks
+const cache = createCache({ valkey, namespace: 'pages' });
+const html = await cache.getOrSet(`page:${slug}`, 300, () => render(slug));
+const thumbs = createQueue({ valkey, name: 'thumbs' });
+thumbs.process(async (job) => { … }, { concurrency: 4 });
+```
+
+Install the drivers the service uses: `pg` and `iovalkey` (production), `@electric-sql/pglite` (tests), `better-sqlite3` (only for the one-time `importSqlite`).
+
### Browser without a bundler
### Browser without a bundler
```html
```html
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
"stage": "alpha",
"stage": "alpha",
"code": true,
"code": true,
"wave": "W2",
"wave": "W2",
- "version": "0.12.0",
+ "version": "0.14.0",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30",
"deployed": false,
"deployed": false,
"deployment": "none: a package released as git tags; services and apps pin a tag",
"deployment": "none: a package released as git tags; services and apps pin a tag",
@@ -19,7 +19,9 @@
@@ -19,7 +19,9 @@
"per-actor limiter for services (openvibe-sdk/limits)",
"per-actor limiter for services (openvibe-sdk/limits)",
"the OpenVibe Frame for any app",
"the OpenVibe Frame for any app",
"mock platform for tests (openvibe-sdk/testing)",
"mock platform for tests (openvibe-sdk/testing)",
- "ESM entries, TypeScript types and a browser bundle"
+ "ESM entries, TypeScript types and a browser bundle",
+ "db (ADR-035): the async PostgreSQL data layer (node-postgres through PgBouncer; PGlite in tests) with safe sql fragments, retrying transactions and savepoints, versioned expand/migrate/contract migrations, and importSqlite for the one-time SQLite move with verification",
+ "valkey, cache, queue, pubsub and a shared limits store on Valkey 9, each with an in-process twin for tests"
],
],
"updated": "2026-09-28"
"updated": "2026-09-28"
}
}
@@ -1,4 +1,4 @@
@@ -1,4 +1,4 @@
-// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.13.0, MIT). Do not edit.
+// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.14.0, MIT). Do not edit.
// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces
// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces
// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects). No server-only code: no service tokens, JWT verification, events, identity.
// (auth, registry, modules, realtime, media, community, search, chat, ai, jobs, tools, projects). No server-only code: no service tokens, JWT verification, events, identity.
/* eslint-disable */
/* eslint-disable */
@@ -40,7 +40,7 @@ const { paginate, offsetPager } = __load(7);
@@ -40,7 +40,7 @@ const { paginate, offsetPager } = __load(7);
const semver = __load(6);
const semver = __load(6);
/** The package version (test/version.test.js keeps it equal to package.json). */
/** The package version (test/version.test.js keeps it equal to package.json). */
-const SDK_VERSION = '0.13.0';
+const SDK_VERSION = '0.14.0';
module.exports = {
module.exports = {
createClient,
createClient,
new file mode 100644
new file mode 100644
@@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
+// Generated by scripts/esm.js from src/cache.js. Do not edit.
+import cjs from '../src/cache.js';
+
+export const {
+ createCache,
+} = cjs;
+export default cjs;
new file mode 100644
new file mode 100644
@@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
+// Generated by scripts/esm.js from src/db/index.js. Do not edit.
+import cjs from '../src/db/index.js';
+
+export const {
+ createDb, sql, DbError, importSqlite, PARSERS, ISOLATION,
+} = cjs;
+export default cjs;
@@ -2,6 +2,6 @@
@@ -2,6 +2,6 @@
import cjs from '../src/limits.js';
import cjs from '../src/limits.js';
export const {
export const {
- createActorLimiter, defaultActor, WINDOWS,
+ createActorLimiter, createValkeyLimitStore, defaultActor, WINDOWS,
} = cjs;
} = cjs;
export default cjs;
export default cjs;
new file mode 100644
new file mode 100644
@@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
+// Generated by scripts/esm.js from src/pubsub.js. Do not edit.
+import cjs from '../src/pubsub.js';
+
+export const {
+ createPubSub,
+} = cjs;
+export default cjs;
new file mode 100644
new file mode 100644
@@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
+// Generated by scripts/esm.js from src/queue.js. Do not edit.
+import cjs from '../src/queue.js';
+
+export const {
+ createQueue, backoff,
+} = cjs;
+export default cjs;
new file mode 100644
new file mode 100644
@@ -0,0 +1,7 @@
@@ -0,0 +1,7 @@
+// Generated by scripts/esm.js from src/valkey.js. Do not edit.
+import cjs from '../src/valkey.js';
+
+export const {
+ createValkey,
+} = cjs;
+export default cjs;
@@ -1,21 +1,58 @@
@@ -1,21 +1,58 @@
{
{
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.13.0",
+ "version": "0.14.0",
"lockfileVersion": 3,
"lockfileVersion": 3,
"requires": true,
"requires": true,
"packages": {
"packages": {
"": {
"": {
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.13.0",
+ "version": "0.14.0",
"license": "MIT",
"license": "MIT",
"devDependencies": {
"devDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0"
+ "iovalkey": "^0.4.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
+ "pg": "^8.23.0"
},
},
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
+ },
+ "peerDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
+ "better-sqlite3": ">=11",
+ "iovalkey": "^0.4.0",
+ "pg": "^8.23.0"
+ },
+ "peerDependenciesMeta": {
+ "@electric-sql/pglite": {
+ "optional": true
+ },
+ "better-sqlite3": {
+ "optional": true
+ },
+ "iovalkey": {
+ "optional": true
+ },
+ "pg": {
+ "optional": true
+ }
}
}
},
},
+ "node_modules/@electric-sql/pglite": {
+ "version": "0.5.8",
+ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",
+ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",
+ "dev": true,
+ "license": "Apache-2.0"
+ },
+ "node_modules/@iovalkey/commands": {
+ "version": "0.1.0",
+ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",
+ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/ajv": {
"node_modules/ajv": {
"version": "8.20.0",
"version": "8.20.0",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz",
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.20.0.tgz",
@@ -138,6 +175,34 @@
@@ -138,6 +175,34 @@
"dev": true,
"dev": true,
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/cluster-key-slot": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
+ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
+ "dev": true,
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/debug": {
+ "version": "4.4.3",
+ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
+ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "ms": "^2.1.3"
+ },
+ "engines": {
+ "node": ">=6.0"
+ },
+ "peerDependenciesMeta": {
+ "supports-color": {
+ "optional": true
+ }
+ }
+ },
"node_modules/decompress-response": {
"node_modules/decompress-response": {
"version": "6.0.0",
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
@@ -164,6 +229,16 @@
@@ -164,6 +229,16 @@
"node": ">=4.0.0"
"node": ">=4.0.0"
}
}
},
},
+ "node_modules/denque": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
+ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
+ "dev": true,
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10"
+ }
+ },
"node_modules/detect-libc": {
"node_modules/detect-libc": {
"version": "2.1.2",
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
@@ -274,6 +349,27 @@
@@ -274,6 +349,27 @@
"dev": true,
"dev": true,
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/iovalkey": {
+ "version": "0.4.0",
+ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
+ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "@iovalkey/commands": "^0.1.0",
+ "cluster-key-slot": "^1.1.0",
+ "debug": "^4.3.4",
+ "denque": "^2.1.0",
+ "lodash.defaults": "^4.2.0",
+ "lodash.isarguments": "^3.1.0",
+ "redis-errors": "^1.2.0",
+ "redis-parser": "^3.0.0",
+ "standard-as-callback": "^2.1.0"
+ },
+ "engines": {
+ "node": ">=18.12.0"
+ }
+ },
"node_modules/json-schema-traverse": {
"node_modules/json-schema-traverse": {
"version": "1.0.0",
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",
"resolved": "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz",
@@ -281,6 +377,20 @@
@@ -281,6 +377,20 @@
"dev": true,
"dev": true,
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/lodash.defaults": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
+ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
+ "dev": true,
+ "license": "MIT"
+ },
+ "node_modules/lodash.isarguments": {
+ "version": "3.1.0",
+ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
+ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/mimic-response": {
"node_modules/mimic-response": {
"version": "3.1.0",
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
@@ -311,6 +421,13 @@
@@ -311,6 +421,13 @@
"dev": true,
"dev": true,
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/ms": {
+ "version": "2.1.3",
+ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
+ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/napi-build-utils": {
"node_modules/napi-build-utils": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
@@ -358,6 +475,146 @@
@@ -358,6 +475,146 @@
"node": ">=20"
"node": ">=20"
}
}
},
},
+ "node_modules/pg": {
+ "version": "8.23.0",
+ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
+ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "pg-connection-string": "^2.14.0",
+ "pg-pool": "^3.14.0",
+ "pg-protocol": "^1.16.0",
+ "pg-types": "2.2.0",
+ "pgpass": "1.0.5"
+ },
+ "engines": {
+ "node": ">= 16.0.0"
+ },
+ "optionalDependencies": {
+ "pg-cloudflare": "^1.4.0"
+ },
+ "peerDependencies": {
+ "pg-native": ">=3.0.1"
+ },
+ "peerDependenciesMeta": {
+ "pg-native": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/pg-cloudflare": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
+ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
+ "dev": true,
+ "license": "MIT",
+ "optional": true
+ },
+ "node_modules/pg-connection-string": {
+ "version": "2.14.0",
+ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
+ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
+ "dev": true,
+ "license": "MIT"
+ },
+ "node_modules/pg-int8": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
+ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
+ "dev": true,
+ "license": "ISC",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/pg-pool": {
+ "version": "3.14.0",
+ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
+ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
+ "dev": true,
+ "license": "MIT",
+ "peerDependencies": {
+ "pg": ">=8.0"
+ }
+ },
+ "node_modules/pg-protocol": {
+ "version": "1.16.0",
+ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
+ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
+ "dev": true,
+ "license": "MIT"
+ },
+ "node_modules/pg-types": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
+ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "pg-int8": "1.0.1",
+ "postgres-array": "~2.0.0",
+ "postgres-bytea": "~1.0.0",
+ "postgres-date": "~1.0.4",
+ "postgres-interval": "^1.1.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/pgpass": {
+ "version": "1.0.5",
+ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
+ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "split2": "^4.1.0"
+ }
+ },
+ "node_modules/postgres-array": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
+ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/postgres-bytea": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
+ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-date": {
+ "version": "1.0.7",
+ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
+ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-interval": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
+ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "xtend": "^4.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/prebuild-install": {
"node_modules/prebuild-install": {
"version": "7.1.3",
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -428,6 +685,29 @@
@@ -428,6 +685,29 @@
"node": ">= 6"
"node": ">= 6"
}
}
},
},
+ "node_modules/redis-errors": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
+ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/redis-parser": {
+ "version": "3.0.0",
+ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
+ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
+ "dev": true,
+ "license": "MIT",
+ "dependencies": {
+ "redis-errors": "^1.0.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
"node_modules/require-from-string": {
"node_modules/require-from-string": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@ -519,6 +799,23 @@
@@ -519,6 +799,23 @@
"simple-concat": "^1.0.0"
"simple-concat": "^1.0.0"
}
}
},
},
+ "node_modules/split2": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+ "dev": true,
+ "license": "ISC",
+ "engines": {
+ "node": ">= 10.x"
+ }
+ },
+ "node_modules/standard-as-callback": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
+ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
+ "dev": true,
+ "license": "MIT"
+ },
"node_modules/string_decoder": {
"node_modules/string_decoder": {
"version": "1.3.0",
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
@@ -595,6 +892,16 @@
@@ -595,6 +892,16 @@
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"dev": true,
"dev": true,
"license": "ISC"
"license": "ISC"
+ },
+ "node_modules/xtend": {
+ "version": "4.0.2",
+ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+ "dev": true,
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.4"
+ }
}
}
}
}
}
}
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
{
{
"name": "openvibe-sdk",
"name": "openvibe-sdk",
- "version": "0.13.0",
- "description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, with deadlines, safe retries, problem errors and trace propagation.",
+ "version": "0.14.0",
+ "description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.",
"main": "index.js",
"main": "index.js",
"types": "types/index.d.ts",
"types": "types/index.d.ts",
"browser": "browser.js",
"browser": "browser.js",
@@ -76,6 +76,36 @@
@@ -76,6 +76,36 @@
"import": "./esm/limits.mjs",
"import": "./esm/limits.mjs",
"default": "./src/limits.js"
"default": "./src/limits.js"
},
},
+ "./db": {
+ "types": "./types/db.d.ts",
+ "browser": null,
+ "import": "./esm/db.mjs",
+ "default": "./src/db/index.js"
+ },
+ "./valkey": {
+ "types": "./types/valkey.d.ts",
+ "browser": null,
+ "import": "./esm/valkey.mjs",
+ "default": "./src/valkey.js"
+ },
+ "./cache": {
+ "types": "./types/cache.d.ts",
+ "browser": null,
+ "import": "./esm/cache.mjs",
+ "default": "./src/cache.js"
+ },
+ "./queue": {
+ "types": "./types/queue.d.ts",
+ "browser": null,
+ "import": "./esm/queue.mjs",
+ "default": "./src/queue.js"
+ },
+ "./pubsub": {
+ "types": "./types/pubsub.d.ts",
+ "browser": null,
+ "import": "./esm/pubsub.mjs",
+ "default": "./src/pubsub.js"
+ },
"./chat": {
"./chat": {
"types": "./types/chat.d.ts",
"types": "./types/chat.d.ts",
"import": "./esm/chat.mjs",
"import": "./esm/chat.mjs",
@@ -159,7 +189,30 @@
@@ -159,7 +189,30 @@
"repository": "github:OpenVibers/OpenVibe.SDK",
"repository": "github:OpenVibers/OpenVibe.SDK",
"license": "MIT",
"license": "MIT",
"devDependencies": {
"devDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0"
+ "iovalkey": "^0.4.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
+ "pg": "^8.23.0"
+ },
+ "peerDependencies": {
+ "pg": "^8.23.0",
+ "@electric-sql/pglite": "^0.5.8",
+ "iovalkey": "^0.4.0",
+ "better-sqlite3": ">=11"
+ },
+ "peerDependenciesMeta": {
+ "pg": {
+ "optional": true
+ },
+ "@electric-sql/pglite": {
+ "optional": true
+ },
+ "iovalkey": {
+ "optional": true
+ },
+ "better-sqlite3": {
+ "optional": true
+ }
}
}
}
}
@@ -25,6 +25,11 @@ const ENTRIES = {
@@ -25,6 +25,11 @@ const ENTRIES = {
'community.mjs': '../src/community.js',
'community.mjs': '../src/community.js',
'search.mjs': '../src/search.js',
'search.mjs': '../src/search.js',
'limits.mjs': '../src/limits.js',
'limits.mjs': '../src/limits.js',
+ 'db.mjs': '../src/db/index.js',
+ 'valkey.mjs': '../src/valkey.js',
+ 'cache.mjs': '../src/cache.js',
+ 'queue.mjs': '../src/queue.js',
+ 'pubsub.mjs': '../src/pubsub.js',
'chat.mjs': '../src/chat.js',
'chat.mjs': '../src/chat.js',
'ai.mjs': '../src/ai.js',
'ai.mjs': '../src/ai.js',
'jobs.mjs': '../src/jobs.js',
'jobs.mjs': '../src/jobs.js',
new file mode 100755
new file mode 100755
@@ -0,0 +1,38 @@
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+# PostgreSQL 18 + PgBouncer (transaction pooling, SCRAM) + Valkey 9 in containers, the production data role's
+# shape (ADR-035), for integration tests of openvibe-sdk/db, cache, queue, pubsub and limits.
+#
+# scripts/test-services.sh up start (or reuse) and print the env to export
+# scripts/test-services.sh down remove the containers
+#
+# Tests read OV_TEST_PG_URL (through PgBouncer), OV_TEST_PG_DIRECT_URL and OV_TEST_VALKEY_URL; without them the
+# integration cases print "<label>: skipped (…)" and the suite still runs on PGlite and the memory stores.
+set -euo pipefail
+NET=ovsdk-test; PW=ovtestpw
+case "${1:-up}" in
+up)
+ docker network inspect $NET >/dev/null 2>&1 || docker network create $NET >/dev/null
+ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-pg; then
+ docker rm -f ovsdk-pg >/dev/null 2>&1 || true
+ docker run -d --name ovsdk-pg --network $NET -p 127.0.0.1:55432:5432 \
+ -e POSTGRES_USER=ov -e POSTGRES_PASSWORD=$PW -e POSTGRES_DB=ovtest -e POSTGRES_HOST_AUTH_METHOD=scram-sha-256 -e POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256 \
+ postgres:18-alpine >/dev/null
+ fi
+ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-pgbouncer; then
+ docker rm -f ovsdk-pgbouncer >/dev/null 2>&1 || true
+ docker run -d --name ovsdk-pgbouncer --network $NET -p 127.0.0.1:56432:5432 \
+ -e DATABASE_URL="postgres://ov:$PW@ovsdk-pg:5432/ovtest" -e POOL_MODE=transaction -e AUTH_TYPE=scram-sha-256 -e MAX_CLIENT_CONN=500 -e DEFAULT_POOL_SIZE=10 \
+ edoburu/pgbouncer:latest >/dev/null
+ fi
+ if ! docker ps --format '{{.Names}}' | grep -qx ovsdk-valkey; then
+ docker rm -f ovsdk-valkey >/dev/null 2>&1 || true
+ docker run -d --name ovsdk-valkey --network $NET -p 127.0.0.1:56379:6379 valkey/valkey:9-alpine >/dev/null
+ fi
+ for i in $(seq 1 60); do docker exec ovsdk-pg pg_isready -U ov -d ovtest >/dev/null 2>&1 && break; sleep 0.5; done
+ echo "export OV_TEST_PG_URL=postgres://ov:[email protected]:56432/ovtest"
+ echo "export OV_TEST_PG_DIRECT_URL=postgres://ov:[email protected]:55432/ovtest"
+ echo "export OV_TEST_VALKEY_URL=redis://127.0.0.1:56379/0"
+ ;;
+down) docker rm -f ovsdk-pg ovsdk-pgbouncer ovsdk-valkey >/dev/null 2>&1 || true; docker network rm $NET >/dev/null 2>&1 || true; echo removed ;;
+*) echo "usage: $0 up|down" >&2; exit 2 ;;
+esac
new file mode 100644
new file mode 100644
@@ -0,0 +1,98 @@
@@ -0,0 +1,98 @@
+'use strict';
+/**
+ * openvibe-sdk/cache — shared caching on Valkey, in-process without it (ADR-035; never authoritative).
+ *
+ * const cache = createCache({ valkey, namespace: 'pages', ttlSec: 60 });
+ * const page = await cache.getOrSet(`slug:${slug}`, 300, () => db.maybe(sql`…`), { tags: [`space:${id}`] });
+ * await cache.invalidateTag(`space:${id}`);
+ *
+ * getOrSet protects the database from a stampede twice: one load per key per process (single-flight), and
+ * across processes a short lock, so while one process loads, the others wait briefly for its value instead of
+ * all querying. Values are JSON. A Valkey error never fails the caller: the loader runs and the error is logged.
+ */
+
+function memoryStore(max = 5000) {
+ const map = new Map(); // key -> { v, exp }
+ const tags = new Map(); // tag -> Set(key)
+ const live = (e) => e && (!e.exp || e.exp > Date.now());
+ return {
+ async get(k) { const e = map.get(k); if (!live(e)) { map.delete(k); return undefined; } map.delete(k); map.set(k, e); return e.v; },
+ async set(k, v, ttlSec, tagList) {
+ map.set(k, { v, exp: ttlSec ? Date.now() + ttlSec * 1000 : 0 });
+ if (map.size > max) map.delete(map.keys().next().value);
+ for (const t of tagList || []) { if (!tags.has(t)) tags.set(t, new Set()); tags.get(t).add(k); }
+ },
+ async del(keys) { let n = 0; for (const k of keys) n += map.delete(k) ? 1 : 0; return n; },
+ async lock() { return true; },
+ async unlock() {},
+ async invalidateTag(t) { const ks = tags.get(t); tags.delete(t); return ks ? this.del([...ks]) : 0; },
+ };
+}
+
+function valkeyStore(valkey, ns) {
+ const c = valkey.client;
+ const k = (key) => valkey.key('cache', ns, key);
+ const tk = (tag) => valkey.key('cache', ns, 'tag', tag);
+ return {
+ async get(key) { const s = await c.get(k(key)); return s == null ? undefined : JSON.parse(s); },
+ async set(key, v, ttlSec, tagList) {
+ const m = c.multi().set(k(key), JSON.stringify(v), 'EX', Math.max(1, ttlSec));
+ for (const t of tagList || []) m.sadd(tk(t), k(key)).expire(tk(t), Math.max(ttlSec, 86400));
+ await m.exec();
+ },
+ async del(keys) { return keys.length ? c.del(...keys.map(k)) : 0; },
+ async lock(key, ms) { return (await c.set(k(`lock:${key}`), '1', 'PX', ms, 'NX')) === 'OK'; },
+ async unlock(key) { await c.del(k(`lock:${key}`)); },
+ async invalidateTag(t) {
+ const members = await c.smembers(tk(t));
+ if (members.length) await c.del(...members); // members are full keys
+ await c.del(tk(t));
+ return members.length;
+ },
+ };
+}
+
+function createCache({ valkey = null, namespace = 'default', ttlSec = 60, lockMs = 5000, waitMs = 1500, log = console, memoryMax } = {}) {
+ const store = valkey ? valkeyStore(valkey, namespace) : memoryStore(memoryMax);
+ const inflight = new Map();
+ const stats = { hits: 0, misses: 0, loads: 0, errors: 0 };
+ const safe = async (fn, fallback) => { try { return await fn(); } catch (err) { stats.errors++; log.warn(`[cache] ${err.message}`); return fallback; } };
+
+ async function getOrSet(key, ttl, loader, { tags } = {}) {
+ const hit = await safe(() => store.get(key), undefined);
+ if (hit !== undefined) { stats.hits++; return hit; }
+ stats.misses++;
+ if (inflight.has(key)) return inflight.get(key);
+ const p = (async () => {
+ const locked = await safe(() => store.lock(key, lockMs), true);
+ if (!locked) {
+ // Another process is loading it: wait briefly for its value.
+ const until = Date.now() + waitMs;
+ while (Date.now() < until) {
+ await new Promise((r) => setTimeout(r, 50));
+ const v = await safe(() => store.get(key), undefined);
+ if (v !== undefined) { stats.hits++; return v; }
+ }
+ }
+ try {
+ stats.loads++;
+ const v = await loader();
+ if (v !== undefined) await safe(() => store.set(key, v, ttl || ttlSec, tags));
+ return v;
+ } finally { if (locked) await safe(() => store.unlock(key)); }
+ })().finally(() => inflight.delete(key));
+ inflight.set(key, p);
+ return p;
+ }
+
+ return {
+ get: (key) => safe(() => store.get(key), undefined),
+ set: (key, value, ttl, opts = {}) => safe(() => store.set(key, value, ttl || ttlSec, opts.tags)),
+ del: (...keys) => safe(() => store.del(keys), 0),
+ getOrSet,
+ invalidateTag: (tag) => safe(() => store.invalidateTag(tag), 0),
+ stats: () => ({ ...stats, store: valkey ? 'valkey' : 'memory' }),
+ };
+}
+
+module.exports = { createCache };
@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');
@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');
const semver = require('./semver');
const semver = require('./semver');
/** The package version (test/version.test.js keeps it equal to package.json). */
/** The package version (test/version.test.js keeps it equal to package.json). */
-const SDK_VERSION = '0.13.0';
+const SDK_VERSION = '0.14.0';
module.exports = {
module.exports = {
createClient,
createClient,
new file mode 100644
new file mode 100644
@@ -0,0 +1,261 @@
@@ -0,0 +1,261 @@
+'use strict';
+/**
+ * openvibe-sdk/db — the async data layer every service uses (ADR-035, roadmap WS-X2).
+ *
+ * PostgreSQL is the only dialect. Production connects with node-postgres through PgBouncer in transaction mode
+ * (DATABASE_URL); tests and embedded tools run real PostgreSQL in-process with PGlite, so what is tested is what
+ * runs. Both adapters return identical rows: int8 as Number (refused beyond 2^53), timestamps as ISO-8601
+ * strings, dates as 'YYYY-MM-DD', numeric as a string (exact), json/jsonb parsed.
+ *
+ * const { createDb, sql } = require('openvibe-sdk/db');
+ * const db = createDb({ service: 'wiki' }); // DATABASE_URL, or { url }, or { pglite: true }
+ * const page = await db.maybe(sql`SELECT id, title FROM pages WHERE slug = ${slug}`);
+ * await db.tx(async (t) => { … }, { isolation: 'serializable' }); // 40001/40P01 retried
+ * await db.migrate({ dir: path.join(__dirname, 'migrations') }); // with DATABASE_DIRECT_URL
+ *
+ * Rules the pooler imposes (ADR-007 2026-09-24 rule 1): no session state between transactions (no session SET,
+ * no LISTEN, no advisory locks held across transactions) on DATABASE_URL. Migrations use DATABASE_DIRECT_URL.
+ */
+const { sql, isSql } = require('./sql');
+
+const ISOLATION = { 'read committed': 'READ COMMITTED', 'repeatable read': 'REPEATABLE READ', serializable: 'SERIALIZABLE' };
+const RETRYABLE = new Set(['40001', '40P01']); // serialization failure, deadlock
+
+// One set of type parsers for both adapters (text-format values from the server).
+function int8(v) {
+ if (v === null || v === undefined) return v;
+ const n = Number(v);
+ if (!Number.isSafeInteger(n)) throw new RangeError(`openvibe-sdk/db: int8 value ${v} is beyond 2^53; select it as text`);
+ return n;
+}
+const PARSERS = {
+ 20: int8, // int8
+ 1700: (v) => v, // numeric: exact, as text
+ 1114: (v) => (v == null ? v : new Date(`${v.replace(' ', 'T')}Z`).toISOString()), // timestamp (UTC by convention)
+ 1184: (v) => (v == null ? v : new Date(normaliseTz(v)).toISOString()), // timestamptz
+ 1082: (v) => v, // date
+ 114: (v) => (v == null ? v : JSON.parse(v)), // json
+ 3802: (v) => (v == null ? v : JSON.parse(v)), // jsonb
+};
+function normaliseTz(v) {
+ // '2026-09-28 10:47:22.411+00' → '2026-09-28T10:47:22.411+00:00'
+ let s = v.replace(' ', 'T');
+ if (/[+-]\d\d$/.test(s)) s += ':00';
+ return s;
+}
+
+function compileArgs(q, values) {
+ if (isSql(q)) return q.compile();
+ if (typeof q === 'string') return { text: q, values: values || [] };
+ throw new TypeError('openvibe-sdk/db: pass sql`…` or (text, values)');
+}
+
+class DbError extends Error {
+ constructor(message, cause, extra = {}) { super(message); this.name = 'DbError'; this.cause = cause; Object.assign(this, extra); }
+}
+
+/** The query methods, shared by the pool, a transaction and a savepoint. `run(text, values)` → { rows, rowCount }. */
+function queryApi(run, self) {
+ const api = {
+ async query(q, values) { const c = compileArgs(q, values); return run(c.text, c.values); },
+ async many(q, values) { return (await api.query(q, values)).rows; },
+ async maybe(q, values) { const r = await api.query(q, values); return r.rows[0] || null; },
+ async one(q, values) {
+ const r = await api.query(q, values);
+ if (!r.rows.length) throw new DbError('openvibe-sdk/db: expected one row, got none', null, { code: 'no_rows' });
+ return r.rows[0];
+ },
+ async value(q, values) { const r = await api.query(q, values); const row = r.rows[0]; return row ? row[Object.keys(row)[0]] : null; },
+ async exec(q, values) { return (await api.query(q, values)).rowCount; },
+ };
+ return Object.assign(self, api);
+}
+
+/**
+ * @param {object} [o]
+ * @param {string} [o.url] postgres URL (default DATABASE_URL)
+ * @param {boolean|string|object} [o.pglite] true: an in-memory PGlite; a directory: persisted PGlite; or a PGlite instance
+ * @param {string} [o.service] application_name and metric label
+ * @param {number} [o.max] pool size per process (default 10; PgBouncer multiplexes)
+ * @param {number} [o.queryTimeoutMs] client-side cap per query (default 15000; the runtime role also has a server-side one)
+ * @param {number} [o.slowMs] log statements slower than this (default 500), text only, never values
+ * @param {object} [o.log] { warn, error } (default console)
+ * @param {object} [o.registry] openvibe-shared/metrics registry: db_query_seconds histogram, pool gauges
+ */
+function createDb(o = {}) {
+ const log = o.log || console;
+ const service = o.service || process.env.OV_SERVICE || 'service';
+ const slowMs = o.slowMs == null ? 500 : o.slowMs;
+ const stats = { queries: 0, errors: 0, slow: 0, retries: 0, tx: 0 };
+ let hist = null;
+ if (o.registry && typeof o.registry.histogram === 'function') {
+ try { hist = o.registry.histogram({ name: 'db_query_seconds', help: 'Database query time by kind', labelNames: ['kind'], buckets: [0.001, 0.005, 0.02, 0.1, 0.5, 2, 10] }); } catch { hist = null; }
+ }
+ const kindOf = (text) => (text.trimStart().slice(0, 6).toUpperCase().match(/^(SELECT|INSERT|UPDATE|DELETE|WITH)/) || ['OTHER'])[0].toLowerCase();
+
+ let adapter;
+ if (o.pglite) adapter = pgliteAdapter(o.pglite);
+ else adapter = pgAdapter({ url: o.url || process.env.DATABASE_URL, max: o.max || 10, service, queryTimeoutMs: o.queryTimeoutMs || 15000, log });
+
+ async function timed(conn, text, values) {
+ const t0 = process.hrtime.bigint();
+ try {
+ const r = await adapter.run(conn, text, values);
+ stats.queries++;
+ return r;
+ } catch (err) {
+ stats.errors++;
+ throw wrap(err, text);
+ } finally {
+ const s = Number(process.hrtime.bigint() - t0) / 1e9;
+ if (hist) { try { hist.observe({ kind: kindOf(text) }, s); } catch { /* metrics never break a query */ } }
+ if (s * 1000 > slowMs) { stats.slow++; log.warn(`[db] slow (${Math.round(s * 1000)} ms): ${text.replace(/\s+/g, ' ').slice(0, 160)}`); }
+ }
+ }
+
+ async function tx(fn, opts = {}, conn = null, depth = 0) {
+ if (conn) {
+ // Nested: a savepoint inside the running transaction.
+ const name = `sp_${depth}`;
+ await timed(conn, `SAVEPOINT ${name}`, []);
+ try {
+ const out = await fn(txApi(conn, depth + 1));
+ await timed(conn, `RELEASE SAVEPOINT ${name}`, []);
+ return out;
+ } catch (err) { await timed(conn, `ROLLBACK TO SAVEPOINT ${name}`, []).catch(() => {}); throw err; }
+ }
+ const iso = ISOLATION[opts.isolation || 'read committed'];
+ if (!iso) throw new TypeError(`openvibe-sdk/db: isolation must be one of ${Object.keys(ISOLATION).join(', ')}`);
+ const retries = opts.retries == null ? (opts.isolation === 'serializable' ? 5 : 2) : opts.retries;
+ for (let attempt = 0; ; attempt++) {
+ const c = await adapter.acquire();
+ stats.tx++;
+ try {
+ await timed(c, `BEGIN ISOLATION LEVEL ${iso}${opts.readOnly ? ' READ ONLY' : ''}`, []);
+ const out = await fn(txApi(c, 1));
+ await timed(c, 'COMMIT', []);
+ return out;
+ } catch (err) {
+ await timed(c, 'ROLLBACK', []).catch(() => {});
+ const code = err && (err.code || (err.cause && err.cause.code));
+ if (RETRYABLE.has(code) && attempt < retries) {
+ stats.retries++;
+ await new Promise((r) => setTimeout(r, Math.min(1000, 10 * 2 ** attempt) + Math.random() * 20));
+ continue;
+ }
+ throw err;
+ } finally { adapter.release(c); }
+ }
+ }
+
+ function txApi(conn, depth) {
+ const t = queryApi((text, values) => timed(conn, text, values), {});
+ t.tx = (fn) => tx(fn, {}, conn, depth);
+ return t;
+ }
+
+ const db = queryApi((text, values) => timed(null, text, values), {});
+ db.sql = sql;
+ db.tx = (fn, opts) => tx(fn, opts);
+ db.store = adapter.store;
+ db.stats = () => ({ ...stats, pool: adapter.pool() });
+ /** Readiness: a real round trip, and which store answered (never claimed from configuration). */
+ db.ready = async () => {
+ try {
+ const v = await db.value(sql`SELECT 1 AS ok`);
+ return v === 1 ? { ok: true, detail: { store: adapter.store, pool: adapter.pool() } } : { ok: false, error: 'unexpected answer' };
+ } catch (err) { return { ok: false, error: `${adapter.store}: ${err.message}` }; }
+ };
+ db.migrate = (m) => require('./migrate').migrate({ db, ...m, log: m && m.log ? m.log : log });
+ db.close = () => adapter.close();
+ db._adapter = adapter;
+ if (hist === null && o.registry && typeof o.registry.gauge === 'function') { /* histogram unsupported: counts still in stats() */ }
+ if (o.registry && typeof o.registry.gauge === 'function') {
+ try {
+ o.registry.gauge({ name: 'db_pool_connections', help: 'Database pool connections by state', labelNames: ['state'],
+ collect: () => { const p = adapter.pool(); return [{ labels: { state: 'total' }, value: p.total }, { labels: { state: 'idle' }, value: p.idle }, { labels: { state: 'waiting' }, value: p.waiting }]; } });
+ } catch { /* registered already */ }
+ }
+ return db;
+}
+
+function wrap(err, text) {
+ if (err instanceof DbError) return err;
+ const e = new DbError(err.message, err, { code: err.code, detail: err.detail, constraint: err.constraint, table: err.table, column: err.column });
+ e.statement = text.replace(/\s+/g, ' ').slice(0, 200);
+ return e;
+}
+
+// ── node-postgres ──────────────────────────────────────────────────────────────────────────
+function pgAdapter({ url, max, service, queryTimeoutMs, log }) {
+ if (!url) throw new Error('openvibe-sdk/db: DATABASE_URL is not set (or pass { url } or { pglite: true })');
+ let pg;
+ try { pg = require('pg'); } catch { throw new Error('openvibe-sdk/db: install `pg` in the service (npm i pg)'); }
+ const types = {
+ getTypeParser(oid, format) {
+ if (format === 'text' && PARSERS[oid]) return PARSERS[oid];
+ return pg.types.getTypeParser(oid, format);
+ },
+ };
+ const pool = new pg.Pool({ connectionString: url, max, application_name: service, idleTimeoutMillis: 30000, connectionTimeoutMillis: 5000, query_timeout: queryTimeoutMs, types, keepAlive: true });
+ pool.on('error', (err) => log.error(`[db] idle client error: ${err.message}`));
+ return {
+ store: 'postgresql',
+ run: (conn, text, values) => (conn || pool).query({ text, values }).then((r) => ({ rows: r.rows, rowCount: r.rowCount })),
+ acquire: () => pool.connect(),
+ release: (c) => c.release(),
+ pool: () => ({ total: pool.totalCount, idle: pool.idleCount, waiting: pool.waitingCount }),
+ close: () => pool.end(),
+ };
+}
+
+// ── PGlite (tests, embedded) ───────────────────────────────────────────────────────────────
+function pgliteAdapter(opt) {
+ let mod;
+ try { mod = require('@electric-sql/pglite'); } catch { throw new Error('openvibe-sdk/db: install `@electric-sql/pglite` for { pglite } (tests)'); }
+ const parsers = {};
+ for (const [oid, fn] of Object.entries(PARSERS)) parsers[oid] = (v) => fn(v);
+ const inst = typeof opt === 'object' && opt && typeof opt.query === 'function' ? opt
+ : new mod.PGlite(typeof opt === 'string' ? opt : undefined, { parsers });
+ // PGlite is one connection: transactions take turns through this queue so statements never interleave.
+ let chain = Promise.resolve();
+ let holder = null;
+ const conn = { id: 'pglite' };
+ return {
+ store: 'pglite',
+ async run(c, text, values) {
+ if (c === null && holder) {
+ // A statement outside a transaction waits for the running one to finish. Inside that transaction's
+ // own code this would never finish (production would just take another pooled connection), so say so.
+ let timer;
+ const waited = await Promise.race([chain.then(() => true), new Promise((r) => { timer = setTimeout(() => r(false), 10000); })]);
+ clearTimeout(timer);
+ if (!waited) throw new Error('openvibe-sdk/db (PGlite): a query outside a transaction waited 10 s for it; inside db.tx(fn) use the handle fn receives');
+ }
+ // Several statements without parameters (a migration file) go through exec, as node-postgres's simple protocol does.
+ if ((!values || !values.length) && text.replace(/;\s*$/, '').includes(';')) {
+ const results = await inst.exec(text, { parsers });
+ const last = results[results.length - 1] || { rows: [] };
+ return { rows: last.rows || [], rowCount: last.affectedRows != null ? last.affectedRows : (last.rows || []).length };
+ }
+ const r = await inst.query(text, values, { parsers });
+ return { rows: r.rows, rowCount: r.affectedRows != null ? r.affectedRows : r.rows.length };
+ },
+ acquire() {
+ let release;
+ const turn = new Promise((r) => { release = r; });
+ const prev = chain;
+ chain = prev.then(() => turn);
+ return prev.then(() => { holder = { release }; return conn; });
+ },
+ release() { const h = holder; holder = null; if (h) h.release(); },
+ pool: () => ({ total: 1, idle: holder ? 0 : 1, waiting: 0 }),
+ close: () => inst.close(),
+ instance: inst,
+ };
+}
+
+/** The one-time SQLite → PostgreSQL move (loaded on use: it needs better-sqlite3). */
+function importSqlite(opts) { return require('./sqlite-import').importSqlite(opts); }
+
+module.exports = { createDb, sql, DbError, importSqlite, PARSERS, ISOLATION };
new file mode 100644
new file mode 100644
@@ -0,0 +1,115 @@
@@ -0,0 +1,115 @@
+'use strict';
+/**
+ * Versioned migrations for openvibe-sdk/db (ADR-035, ADR-028 expand/migrate/contract).
+ *
+ * A migrations directory holds files named `NNNN_description.sql` (NNNN: four or more digits, applied in order).
+ * The first lines are headers:
+ *
+ * -- phase: expand | migrate | contract (required)
+ * -- after: 0012 (contract only: the expand migration it completes)
+ * -- no-transaction (for CREATE INDEX CONCURRENTLY and the like)
+ *
+ * Each migration runs once, in a transaction unless it says otherwise, and is recorded in `ov_migrations`
+ * with its checksum; editing an applied file is refused. Runs are serialised by an advisory lock, so several
+ * processes starting together apply each migration once. A contract migration is refused until the expand it
+ * names has been applied for `windowDays` (default 7, the N-1 window of ADR-016), so a rollback to the previous
+ * release still finds what it expects. Run it with the owner role (DATABASE_DIRECT_URL): the runtime role
+ * behind PgBouncer may not change the schema, and advisory locks need a session.
+ */
+const crypto = require('crypto');
+const fs = require('fs');
+const path = require('path');
+
+const FILE_RE = /^(\d{4,})_([a-z0-9_-]+)\.sql$/;
+const LOCK_KEY = 71_335_035; // one advisory lock id for every service's migrations (ADR-035)
+
+function parse(dir) {
+ if (!fs.existsSync(dir)) return [];
+ const files = fs.readdirSync(dir).filter((f) => f.endsWith('.sql')).sort();
+ const seen = new Set();
+ return files.map((f) => {
+ const m = FILE_RE.exec(f);
+ if (!m) throw new Error(`migrate: ${f} is not named NNNN_description.sql`);
+ if (seen.has(m[1])) throw new Error(`migrate: two migrations numbered ${m[1]}`);
+ seen.add(m[1]);
+ const text = fs.readFileSync(path.join(dir, f), 'utf8');
+ const header = text.split('\n').slice(0, 12).join('\n');
+ const phase = (/^--\s*phase:\s*(expand|migrate|contract)\s*$/m.exec(header) || [])[1];
+ if (!phase) throw new Error(`migrate: ${f} has no "-- phase: expand|migrate|contract" header`);
+ const after = (/^--\s*after:\s*(\d{4,})\s*$/m.exec(header) || [])[1] || null;
+ if (phase === 'contract' && !after) throw new Error(`migrate: ${f} is a contract migration and must name its expand ("-- after: NNNN")`);
+ return {
+ id: m[1], name: m[2], file: f, phase, after, text,
+ transaction: !/^--\s*no-transaction\s*$/m.test(header),
+ checksum: crypto.createHash('sha256').update(text.replace(/\r\n/g, '\n')).digest('hex'),
+ };
+ });
+}
+
+/**
+ * @param {object} o
+ * @param {object} o.db a createDb() handle with schema rights (owner role, DATABASE_DIRECT_URL)
+ * @param {string} o.dir the migrations directory
+ * @param {number} [o.windowDays] N-1 window before a contract migration may run (default 7)
+ * @param {boolean} [o.dryRun] report only
+ * @param {() => number} [o.now]
+ * → { applied: [{ id, name, phase, ms }], pending: [...], held: [{ id, reason }] }
+ */
+async function migrate({ db, dir, windowDays = 7, dryRun = false, now = () => Date.now(), log = console }) {
+ const all = parse(dir);
+ const out = { applied: [], pending: [], held: [] };
+ const run = async (t) => {
+ await t.query(`CREATE TABLE IF NOT EXISTS ov_migrations (
+ id text PRIMARY KEY, name text NOT NULL, phase text NOT NULL CHECK (phase IN ('expand','migrate','contract')),
+ checksum text NOT NULL, applied_at timestamptz NOT NULL DEFAULT now(), duration_ms integer NOT NULL DEFAULT 0)`);
+ const done = new Map((await t.many('SELECT id, checksum, applied_at FROM ov_migrations')).map((r) => [r.id, r]));
+ for (const m of all) {
+ const prev = done.get(m.id);
+ if (prev) {
+ if (prev.checksum !== m.checksum) throw new Error(`migrate: ${m.file} changed after it was applied (write a new migration instead)`);
+ continue;
+ }
+ const later = [...done.keys()].find((id) => id > m.id);
+ if (later) throw new Error(`migrate: ${m.file} is older than applied migration ${later}; number new migrations after the last one`);
+ if (m.phase === 'contract') {
+ const exp = done.get(m.after);
+ const ageDays = exp ? (now() - Date.parse(exp.applied_at)) / 86400000 : -1;
+ if (!exp || ageDays < windowDays) {
+ out.held.push({ id: m.id, reason: exp ? `expand ${m.after} applied ${ageDays.toFixed(1)} days ago; the N-1 window is ${windowDays}` : `expand ${m.after} is not applied` });
+ break; // later migrations wait too: order is kept
+ }
+ }
+ out.pending.push({ id: m.id, name: m.name, phase: m.phase });
+ if (dryRun) continue;
+ const t0 = Date.now();
+ if (m.transaction) {
+ await t.query('BEGIN');
+ try {
+ await t.query(m.text);
+ await t.query('INSERT INTO ov_migrations (id, name, phase, checksum, duration_ms) VALUES ($1, $2, $3, $4, $5)', [m.id, m.name, m.phase, m.checksum, Date.now() - t0]);
+ await t.query('COMMIT');
+ } catch (err) { await t.query('ROLLBACK').catch(() => {}); throw new Error(`migrate: ${m.file} failed: ${err.message}`); }
+ } else {
+ try { await t.query(m.text); } catch (err) { throw new Error(`migrate: ${m.file} (no transaction) failed: ${err.message}`); }
+ await t.query('INSERT INTO ov_migrations (id, name, phase, checksum, duration_ms) VALUES ($1, $2, $3, $4, $5)', [m.id, m.name, m.phase, m.checksum, Date.now() - t0]);
+ }
+ done.set(m.id, { id: m.id, checksum: m.checksum, applied_at: new Date().toISOString() });
+ out.applied.push({ id: m.id, name: m.name, phase: m.phase, ms: Date.now() - t0 });
+ log.log ? log.log(`[db] migration ${m.file} (${m.phase}) applied in ${Date.now() - t0} ms`) : null;
+ }
+ };
+ // One session for the whole run: the advisory lock and the migrations share it.
+ const adapter = db._adapter;
+ const conn = await adapter.acquire();
+ const session = {
+ query: (text, values) => adapter.run(conn, text, values || []),
+ many: async (text, values) => (await adapter.run(conn, text, values || [])).rows,
+ };
+ try {
+ await session.query('SELECT pg_advisory_lock($1)', [LOCK_KEY]);
+ try { await run(session); } finally { await session.query('SELECT pg_advisory_unlock($1)', [LOCK_KEY]).catch(() => {}); }
+ } finally { adapter.release(conn); }
+ return out;
+}
+
+module.exports = { migrate, parse };
new file mode 100644
new file mode 100644
@@ -0,0 +1,84 @@
@@ -0,0 +1,84 @@
+'use strict';
+/**
+ * Safe SQL fragments for openvibe-sdk/db (ADR-035): a tagged template turns every interpolated value into a
+ * bind parameter ($1, $2, …), so user data never becomes SQL text. Fragments nest, so a query can be
+ * assembled from parts and still compile to one parameterised statement.
+ *
+ * sql`SELECT id, title FROM pages WHERE space_id = ${spaceId} AND id = ANY(${ids})`
+ * sql`UPDATE pages SET ${sql.set({ title, body })} WHERE id = ${id}`
+ * sql`INSERT INTO tags ${sql.insert(rows, ['page_id', 'tag'])} ON CONFLICT DO NOTHING`
+ * sql`SELECT ${sql.join(cols.map(sql.ident))} FROM ${sql.ident('pages')}`
+ *
+ * Only sql.raw() puts text into the statement unchecked; keep it for constants you wrote yourself.
+ */
+
+class Sql {
+ constructor(parts, values) {
+ this.parts = parts; // strings, one more than values
+ this.values = values; // plain values or nested Sql fragments
+ }
+ /** { text, values } with $n placeholders numbered from `start`. */
+ compile(start = 1) {
+ const out = { text: '', values: [] };
+ append(this, out, start);
+ return out;
+ }
+ toString() { return this.compile().text; }
+}
+
+function append(frag, out, start) {
+ for (let i = 0; i < frag.parts.length; i++) {
+ out.text += frag.parts[i];
+ if (i >= frag.values.length) continue;
+ const v = frag.values[i];
+ if (v instanceof Sql) append(v, out, start);
+ else { out.values.push(v === undefined ? null : v); out.text += `$${start + out.values.length - 1}`; }
+ }
+}
+
+function sql(strings, ...values) {
+ if (!Array.isArray(strings) || !Array.isArray(strings.raw)) throw new TypeError('sql is a tagged template: sql`SELECT …`');
+ return new Sql([...strings], values);
+}
+
+const IDENT_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
+/** "name" or "schema"."name"; refuses anything that is not a plain identifier. */
+sql.ident = (name) => {
+ const parts = String(name).split('.');
+ if (!parts.every((p) => IDENT_RE.test(p))) throw new TypeError(`sql.ident: not an identifier: ${name}`);
+ return new Sql([parts.map((p) => `"${p}"`).join('.')], []);
+};
+/** Trusted text, inserted as is. Never pass user input. */
+sql.raw = (text) => new Sql([String(text)], []);
+/** Fragments (or values) joined by a separator fragment (default ", "). */
+sql.join = (items, sep = sql`, `) => {
+ const list = Array.from(items);
+ if (!list.length) return new Sql([''], []);
+ const parts = ['']; const values = [];
+ list.forEach((item, i) => {
+ if (i > 0) { values.push(sep); parts.push(''); }
+ values.push(item); parts.push('');
+ });
+ return new Sql(parts, values);
+};
+/** `col = $1, col2 = $2` from an object (undefined keys are skipped). */
+sql.set = (obj) => {
+ const keys = Object.keys(obj).filter((k) => obj[k] !== undefined);
+ if (!keys.length) throw new TypeError('sql.set: nothing to set');
+ return sql.join(keys.map((k) => sql`${sql.ident(k)} = ${obj[k]}`));
+};
+/** `(a, b) VALUES ($1, $2), ($3, $4)` for a batch of rows; columns default to the first row's keys. */
+sql.insert = (rows, columns) => {
+ const list = Array.isArray(rows) ? rows : [rows];
+ if (!list.length) throw new TypeError('sql.insert: no rows');
+ const cols = columns || Object.keys(list[0]);
+ const tuples = list.map((r) => sql`(${sql.join(cols.map((c) => (r[c] === undefined ? null : r[c])))})`);
+ return sql`(${sql.join(cols.map(sql.ident))}) VALUES ${sql.join(tuples)}`;
+};
+/** A value as a jsonb bind parameter (objects and arrays are otherwise sent as PostgreSQL arrays). */
+sql.json = (value) => sql`${value == null ? null : JSON.stringify(value)}::jsonb`;
+
+sql.Sql = Sql;
+const isSql = (v) => v instanceof Sql;
+
+module.exports = { sql, Sql, isSql };
new file mode 100644
new file mode 100644
@@ -0,0 +1,266 @@
@@ -0,0 +1,266 @@
+'use strict';
+/**
+ * The one-time move of a service's SQLite data into its PostgreSQL schema (ADR-035 amendment, WS-X2 step 4).
+ *
+ * const { importSqlite } = require('openvibe-sdk/db');
+ * const report = await importSqlite({ sqlite: '/opt/…/wiki.db', db: ownerDb, truncate: true });
+ * if (!report.ok) throw new Error(JSON.stringify(report.problems));
+ *
+ * The target schema (created by the service's migrations) decides everything:
+ * - which tables: every base table in `public` except ov_migrations that SQLite also has (or `tables`), copied
+ * parents first (foreign-key order);
+ * - which columns: those both sides have. A source column the target lacks is a problem unless the table lists
+ * it in `dropColumns`; a target column the source lacks must have a default or be nullable;
+ * - each value's conversion, by the target column's type: 0/1 → boolean, text or epoch → timestamptz (SQLite's
+ * CURRENT_TIMESTAMP text is UTC), JSON text → jsonb (validated), JSON array text → PostgreSQL arrays,
+ * Buffer → bytea, anything → text for text columns.
+ * Rows go in batched multi-row INSERTs in one transaction (identity columns keep their values, then their
+ * sequences move past the maximum). Verification compares each table's row count and a checksum of every row,
+ * canonicalised the same way on both sides and walked in primary-key order (byte order for text keys).
+ */
+const crypto = require('crypto');
+const { sql } = require('./sql');
+
+function toIso(v, col) {
+ if (v == null || v === '') return null;
+ if (typeof v === 'number' || (typeof v === 'string' && /^\d{9,}(\.\d+)?$/.test(v))) {
+ const n = Number(v);
+ return new Date(n > 1e11 ? n : n * 1000).toISOString();
+ }
+ const s = String(v).trim();
+ let d;
+ if (/^\d{4}-\d{2}-\d{2}$/.test(s)) d = new Date(`${s}T00:00:00Z`);
+ else if (/^\d{4}-\d{2}-\d{2}[ T]\d{2}:\d{2}(:\d{2}(\.\d+)?)?$/.test(s)) d = new Date(`${s.replace(' ', 'T')}Z`); // SQLite CURRENT_TIMESTAMP: UTC
+ else d = new Date(s);
+ if (Number.isNaN(d.getTime())) throw new Error(`${col}: "${s.slice(0, 40)}" is not a date/time`);
+ return d.toISOString();
+}
+
+/** Convert one SQLite value for a PostgreSQL column type (information_schema udt_name). */
+function convert(v, udt, col) {
+ if (v === undefined || v === null) return null;
+ if (udt.startsWith('_')) { // array
+ if (Array.isArray(v)) return v;
+ if (typeof v === 'string' && v.trim().startsWith('[')) { const a = JSON.parse(v); if (!Array.isArray(a)) throw new Error(`${col}: not a JSON array`); return a; }
+ if (v === '') return [];
+ throw new Error(`${col}: cannot read "${String(v).slice(0, 40)}" as an array (store it as JSON text)`);
+ }
+ switch (udt) {
+ case 'bool':
+ if (typeof v === 'boolean') return v;
+ if (v === 1 || v === '1' || v === 'true' || v === 't') return true;
+ if (v === 0 || v === '0' || v === 'false' || v === 'f') return false;
+ throw new Error(`${col}: "${v}" is not a boolean`);
+ case 'int2': case 'int4': case 'int8': {
+ const n = typeof v === 'bigint' ? Number(v) : Number(v);
+ if (!Number.isSafeInteger(n)) throw new Error(`${col}: "${v}" is not an integer`);
+ return n;
+ }
+ case 'float4': case 'float8': { const n = Number(v); if (Number.isNaN(n)) throw new Error(`${col}: "${v}" is not a number`); return n; }
+ case 'numeric': return String(v);
+ case 'timestamptz': case 'timestamp': return toIso(v, col);
+ case 'date': return String(v).slice(0, 10);
+ case 'json': case 'jsonb':
+ if (typeof v === 'string') { if (v === '') return null; JSON.parse(v); return v; }
+ if (Buffer.isBuffer(v)) { const s = v.toString('utf8'); JSON.parse(s); return s; }
+ return JSON.stringify(v);
+ case 'bytea': return Buffer.isBuffer(v) ? v : Buffer.from(String(v));
+ default: return Buffer.isBuffer(v) ? v.toString('utf8') : String(v);
+ }
+}
+
+// The canonical text of a value for checksums (same rules on both sides).
+function sortKeys(x) {
+ if (Array.isArray(x)) return x.map(sortKeys);
+ if (x && typeof x === 'object') return Object.keys(x).sort().reduce((o, k) => { o[k] = sortKeys(x[k]); return o; }, {});
+ return x;
+}
+function canonical(v, udt) {
+ if (v === null || v === undefined) return '\\N';
+ if (udt === 'json' || udt === 'jsonb') return JSON.stringify(sortKeys(typeof v === 'string' ? JSON.parse(v) : v));
+ if (udt === 'bytea') return Buffer.isBuffer(v) ? v.toString('hex') : Buffer.from(String(v)).toString('hex');
+ if (udt === 'timestamptz' || udt === 'timestamp') return new Date(v).toISOString();
+ if (udt.startsWith('_')) return JSON.stringify(v);
+ if (udt === 'numeric') return String(Number(v));
+ return String(v);
+}
+
+async function schemaOf(db) {
+ const cols = await db.many(sql`
+ SELECT c.table_name, c.column_name, c.udt_name, c.is_nullable = 'YES' AS nullable,
+ c.column_default IS NOT NULL OR c.is_identity = 'YES' OR c.is_generated = 'ALWAYS' AS has_default,
+ c.is_identity = 'YES' AS identity, c.is_generated = 'ALWAYS' AS generated
+ FROM information_schema.columns c
+ JOIN information_schema.tables t ON t.table_schema = c.table_schema AND t.table_name = c.table_name
+ WHERE c.table_schema = 'public' AND t.table_type = 'BASE TABLE' AND c.table_name <> 'ov_migrations'
+ ORDER BY c.table_name, c.ordinal_position`);
+ const pks = await db.many(sql`
+ SELECT tc.relname AS table_name, a.attname AS column_name, array_position(i.indkey::int2[], a.attnum) AS pos
+ FROM pg_index i JOIN pg_class tc ON tc.oid = i.indrelid JOIN pg_namespace n ON n.oid = tc.relnamespace
+ JOIN pg_attribute a ON a.attrelid = tc.oid AND a.attnum = ANY(i.indkey)
+ WHERE i.indisprimary AND n.nspname = 'public' ORDER BY 1, 3`);
+ const fks = await db.many(sql`
+ SELECT c.conrelid::regclass::text AS child, c.confrelid::regclass::text AS parent
+ FROM pg_constraint c JOIN pg_namespace n ON n.oid = c.connamespace
+ WHERE c.contype = 'f' AND n.nspname = 'public'`);
+ const tables = new Map();
+ for (const c of cols) {
+ if (!tables.has(c.table_name)) tables.set(c.table_name, { name: c.table_name, columns: [], pk: [] });
+ tables.get(c.table_name).columns.push(c);
+ }
+ for (const p of pks) if (tables.has(p.table_name)) tables.get(p.table_name).pk.push(p.column_name);
+ // Parents first.
+ const deps = new Map([...tables.keys()].map((t) => [t, new Set()]));
+ for (const f of fks) { const ch = f.child.replace(/"/g, ''); const pa = f.parent.replace(/"/g, ''); if (deps.has(ch) && ch !== pa) deps.get(ch).add(pa); }
+ const order = []; const state = new Map();
+ const visit = (t) => { if (state.get(t) === 2) return; if (state.get(t) === 1) return; state.set(t, 1); for (const p of deps.get(t) || []) visit(p); state.set(t, 2); order.push(t); };
+ for (const t of [...tables.keys()].sort()) visit(t);
+ return order.map((t) => tables.get(t));
+}
+
+/**
+ * @param {object} o
+ * @param {string|object} o.sqlite a path, or an open better-sqlite3 Database
+ * @param {object} o.db createDb() handle with write rights on the tables (the owner role)
+ * @param {object} [o.tables] { [target]: { from?, dropColumns?: [], where?: 'SQL filter on the source', map?: (row) => row } }
+ * @param {string[]} [o.only] import only these target tables
+ * @param {boolean} [o.truncate] empty the target tables first (rehearsals)
+ * @param {boolean} [o.verify] compare counts and checksums (default true)
+ * → { ok, tables: [{ table, source, rows, ms, checksum }], problems: [] }
+ */
+async function importSqlite({ sqlite, db, tables: opts = {}, only = null, truncate = false, verify = true, log = console } = {}) {
+ let Database;
+ const src = typeof sqlite === 'string' ? (() => { try { Database = require('better-sqlite3'); } catch { throw new Error('importSqlite: install better-sqlite3 to read the SQLite file'); } return new Database(sqlite, { readonly: true, fileMustExist: true }); })() : sqlite;
+ const report = { ok: true, tables: [], problems: [] };
+ const problem = (p) => { report.ok = false; report.problems.push(p); };
+ try {
+ const schema = (await schemaOf(db)).filter((t) => !only || only.includes(t.name));
+ const srcTables = new Set(src.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all().map((r) => r.name));
+ const plan = [];
+ for (const t of schema) {
+ const o = opts[t.name] || {};
+ const from = o.from || t.name;
+ if (!srcTables.has(from)) { log.log && log.log(`[import] ${t.name}: no source table ${from} (left empty)`); continue; }
+ const srcCols = src.prepare(`PRAGMA table_info("${from.replace(/"/g, '""')}")`).all().map((c) => c.name);
+ const drop = new Set(o.dropColumns || []);
+ const insertable = t.columns.filter((c) => !c.generated);
+ const cols = insertable.filter((c) => srcCols.includes(c.column_name));
+ const lost = srcCols.filter((c) => !t.columns.some((x) => x.column_name === c) && !drop.has(c));
+ if (lost.length) problem({ table: t.name, problem: `source columns with no target column (list them in dropColumns to drop them): ${lost.join(', ')}` });
+ const unfillable = insertable.filter((c) => !srcCols.includes(c.column_name) && !c.nullable && !c.has_default);
+ if (unfillable.length) problem({ table: t.name, problem: `target columns the source cannot fill: ${unfillable.map((c) => c.column_name).join(', ')}` });
+ plan.push({ t, from, cols, o });
+ }
+ if (!report.ok) return report;
+
+ await db.tx(async (tx) => {
+ if (truncate && plan.length) await tx.exec(sql`TRUNCATE ${sql.join(plan.map((p) => sql.ident(p.t.name)))} RESTART IDENTITY CASCADE`);
+ for (const { t, from, cols, o } of plan) {
+ const t0 = Date.now();
+ const names = cols.map((c) => c.column_name);
+ const hasIdentity = cols.some((c) => c.identity);
+ const batch = Math.max(1, Math.min(1000, Math.floor(60000 / Math.max(1, names.length))));
+ const stmt = src.prepare(`SELECT * FROM "${from.replace(/"/g, '""')}"${o.where ? ` WHERE ${o.where}` : ''}`);
+ let rows = []; let n = 0;
+ const flush = async () => {
+ if (!rows.length) return;
+ await tx.exec(sql`INSERT INTO ${sql.ident(t.name)} (${sql.join(names.map(sql.ident))}) ${hasIdentity ? sql`OVERRIDING SYSTEM VALUE ` : sql``}VALUES ${sql.join(rows.map((r) => sql`(${sql.join(r)})`))}`);
+ n += rows.length; rows = [];
+ };
+ for (const raw of stmt.iterate()) {
+ const row = o.map ? o.map({ ...raw }) : raw;
+ if (!row) continue;
+ rows.push(cols.map((c) => convert(row[c.column_name], c.udt_name, `${t.name}.${c.column_name}`)));
+ if (rows.length >= batch) await flush();
+ }
+ await flush();
+ for (const c of cols.filter((x) => x.identity || /nextval\(/.test(String(x.column_default || '')))) {
+ await tx.query(sql`SELECT setval(pg_get_serial_sequence(${t.name}, ${c.column_name}), GREATEST((SELECT max(${sql.ident(c.column_name)}) FROM ${sql.ident(t.name)}), 1))`);
+ }
+ report.tables.push({ table: t.name, source: from, rows: n, ms: Date.now() - t0 });
+ log.log && log.log(`[import] ${t.name}: ${n} rows in ${Date.now() - t0} ms`);
+ }
+ });
+
+ if (verify) {
+ for (const entry of report.tables) {
+ const { t, from, cols, o } = plan.find((p) => p.t.name === entry.table);
+ const result = await verifyTable({ src, db, t, from, cols, o });
+ entry.checksum = result.pg.sum;
+ if (result.sqlite.rows !== result.pg.rows || result.sqlite.sum !== result.pg.sum) {
+ problem({ table: t.name, problem: `verification failed: SQLite ${result.sqlite.rows} rows (${result.sqlite.sum}), PostgreSQL ${result.pg.rows} rows (${result.pg.sum})` });
+ }
+ }
+ }
+ return report;
+ } finally {
+ if (typeof sqlite === 'string') src.close();
+ }
+}
+
+/**
+ * Stream both sides in primary-key order and hash each canonical row: SQLite iterates ORDER BY the key (its
+ * BINARY collation is byte order, as COLLATE "C"), PostgreSQL reads keyset pages of 5000 (OFFSET pages for a
+ * composite key). A table without a primary key is sorted in memory by every column.
+ */
+async function verifyTable({ src, db, t, from, cols, o }) {
+ const names = cols.map((c) => c.column_name);
+ const keys = t.pk.filter((k) => names.includes(k));
+ const q = (x) => `"${String(x).replace(/"/g, '""')}"`;
+ const text = new Set(cols.filter((c) => ['text', 'varchar', 'bpchar', 'citext', 'uuid'].includes(c.udt_name)).map((c) => c.column_name));
+ const line = (vals) => vals.map((v, i) => canonical(v, cols[i].udt_name)).join('\t') + '\n';
+ const convertRow = (row) => cols.map((c) => convert(row[c.column_name], c.udt_name, c.column_name));
+
+ const h1 = crypto.createHash('sha256'); let c1 = 0;
+ const where = o.where ? ` WHERE ${o.where}` : '';
+ if (keys.length) {
+ for (const raw of src.prepare(`SELECT * FROM ${q(from)}${where} ORDER BY ${keys.map(q).join(', ')}`).iterate()) {
+ const row = o.map ? o.map({ ...raw }) : raw;
+ if (!row) continue;
+ h1.update(line(convertRow(row))); c1++;
+ }
+ } else {
+ const all = src.prepare(`SELECT * FROM ${q(from)}${where}`).all().map((raw) => (o.map ? o.map({ ...raw }) : raw)).filter(Boolean).map(convertRow);
+ all.sort((a, b) => cmpRows(a, b, names.map((_, i) => i)));
+ for (const r of all) { h1.update(line(r)); c1++; }
+ }
+
+ const h2 = crypto.createHash('sha256'); let c2 = 0;
+ const colList = sql.join(names.map(sql.ident));
+ const orderBy = (ks) => sql.join(ks.map((k) => (text.has(k) ? sql`${sql.ident(k)} COLLATE "C"` : sql.ident(k))));
+ const PAGE = 5000;
+ if (keys.length === 1) {
+ const k = keys[0]; let last = null;
+ for (;;) {
+ const cond = last === null ? sql`` : (text.has(k) ? sql`WHERE ${sql.ident(k)} COLLATE "C" > ${last}` : sql`WHERE ${sql.ident(k)} > ${last}`);
+ const rows = await db.many(sql`SELECT ${colList} FROM ${sql.ident(t.name)} ${cond} ORDER BY ${orderBy([k])} LIMIT ${PAGE}`);
+ for (const r of rows) { h2.update(line(names.map((n) => r[n]))); c2++; }
+ if (rows.length < PAGE) break;
+ last = rows[rows.length - 1][k];
+ }
+ } else {
+ const ks = keys.length ? keys : names;
+ for (let off = 0; ; off += PAGE) {
+ const rows = await db.many(sql`SELECT ${colList} FROM ${sql.ident(t.name)} ORDER BY ${orderBy(ks)} LIMIT ${PAGE} OFFSET ${off}`);
+ for (const r of rows) { h2.update(line(names.map((n) => r[n]))); c2++; }
+ if (rows.length < PAGE) break;
+ }
+ }
+ return { sqlite: { rows: c1, sum: h1.digest('hex').slice(0, 16) }, pg: { rows: c2, sum: h2.digest('hex').slice(0, 16) } };
+}
+
+/** Order rows the way PostgreSQL does with COLLATE "C" (byte order for text, numeric for numbers). */
+function cmpRows(a, b, idx) {
+ for (const i of idx) {
+ const x = a[i]; const y = b[i];
+ if (x === y) continue;
+ if (x === null) return 1; if (y === null) return -1; // NULLS LAST, PostgreSQL's default for ASC
+ if (typeof x === 'number' && typeof y === 'number') return x - y;
+ const bx = Buffer.from(String(x)); const by = Buffer.from(String(y));
+ const c = Buffer.compare(bx, by);
+ if (c) return c;
+ }
+ return 0;
+}
+
+module.exports = { importSqlite, convert, canonical };
@@ -16,6 +16,11 @@
@@ -16,6 +16,11 @@
* yourself; return null to skip counting (a trusted internal caller). Counters live in this process (a restart
* yourself; return null to skip counting (a trusted internal caller). Counters live in this process (a restart
* forgets them) and are capped (maxActors, oldest forgotten first), so memory stays bounded under a flood.
* forgets them) and are capped (maxActors, oldest forgotten first), so memory stays bounded under a flood.
* Each limit is { minute?, hour?, day? }; a route's own limits replace the defaults window by window.
* Each limit is { minute?, hour?, day? }; a route's own limits replace the defaults window by window.
+ *
+ * Shared counters (ADR-035): pass `store: createValkeyLimitStore(valkey)` and every process and host counts one
+ * actor together (one atomic script per request: check every window, then count). If Valkey errors, that
+ * request is counted by this process's own counters instead, so a Valkey outage never opens the gates nor
+ * blocks the service.
*/
*/
const WINDOWS = { minute: 60, hour: 3600, day: 86400 };
const WINDOWS = { minute: 60, hour: 3600, day: 86400 };
@@ -46,7 +51,7 @@ function problem(res, retryAfter, detail) {
@@ -46,7 +51,7 @@ function problem(res, retryAfter, detail) {
* @param {(e: {actor, name, window, limit}) => void} [opts.onLimited] called on every refusal (metrics, logs)
* @param {(e: {actor, name, window, limit}) => void} [opts.onLimited] called on every refusal (metrics, logs)
* @returns {((name: string, limits?: object) => Function) & { stats(): object, reset(): void }}
* @returns {((name: string, limits?: object) => Function) & { stats(): object, reset(): void }}
*/
*/
-function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maxActors = 50000, now = () => Date.now(), onLimited = null } = {}) {
+function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, maxActors = 50000, now = () => Date.now(), onLimited = null, store = null, log = console } = {}) {
for (const w of Object.keys(limits)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);
for (const w of Object.keys(limits)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);
const counters = new Map(); // `${name}|${actor}` -> { [window]: { start, count } }
const counters = new Map(); // `${name}|${actor}` -> { [window]: { start, count } }
const stats = { allowed: 0, limited: 0, actors: 0 };
const stats = { allowed: 0, limited: 0, actors: 0 };
@@ -66,9 +71,29 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, ma
@@ -66,9 +71,29 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, ma
const merged = { ...limits, ...own };
const merged = { ...limits, ...own };
for (const w of Object.keys(merged)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);
for (const w of Object.keys(merged)) if (!WINDOWS[w]) throw new Error(`limits: unknown window ${w}`);
const windows = Object.entries(merged).filter(([, max]) => Number.isFinite(max) && max >= 0);
const windows = Object.entries(merged).filter(([, max]) => Number.isFinite(max) && max >= 0);
+ const refuse = (res, who, worst) => {
+ stats.limited++;
+ if (onLimited) { try { onLimited({ actor: who, name, window: worst.window, limit: worst.max }); } catch { /* observers never block */ } }
+ return problem(res, Math.max(1, worst.retry), `${name}: at most ${worst.max} per ${worst.window} for each caller`);
+ };
return function actorLimit(req, res, next) {
return function actorLimit(req, res, next) {
const who = actor(req);
const who = actor(req);
if (who == null) return next();
if (who == null) return next();
+ if (store) {
+ const t = Math.floor(now() / 1000);
+ return store.hit(name, who, windows.map(([w, max]) => [w, max, WINDOWS[w]]), t).then((r) => {
+ if (r && r.limited) return refuse(res, who, r);
+ stats.allowed++;
+ return next();
+ }, (err) => {
+ stats.storeErrors = (stats.storeErrors || 0) + 1;
+ if (stats.storeErrors === 1 || stats.storeErrors % 100 === 0) log.warn(`[limits] shared store failed (${err.message}); counting in this process`);
+ return local(req, res, next, who);
+ });
+ }
+ return local(req, res, next, who);
+ };
+ function local(req, res, next, who) {
const t = Math.floor(now() / 1000);
const t = Math.floor(now() / 1000);
const c = touch(`${name}|${who}`);
const c = touch(`${name}|${who}`);
let worst = null;
let worst = null;
@@ -80,19 +105,56 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, ma
@@ -80,19 +105,56 @@ function createActorLimiter({ limits = { minute: 120 }, actor = defaultActor, ma
if (!worst || retry > worst.retry) worst = { window: w, max, retry };
if (!worst || retry > worst.retry) worst = { window: w, max, retry };
}
}
}
}
- if (worst) {
- stats.limited++;
- if (onLimited) { try { onLimited({ actor: who, name, window: worst.window, limit: worst.max }); } catch { /* observers never block */ } }
- return problem(res, Math.max(1, worst.retry), `${name}: at most ${worst.max} per ${worst.window} for each caller`);
- }
+ if (worst) return refuse(res, who, worst);
for (const [w] of windows) c[w].count++;
for (const [w] of windows) c[w].count++;
stats.allowed++;
stats.allowed++;
return next();
return next();
- };
+ }
}
}
middleware.stats = () => ({ ...stats });
middleware.stats = () => ({ ...stats });
middleware.reset = () => { counters.clear(); stats.actors = 0; };
middleware.reset = () => { counters.clear(); stats.actors = 0; };
return middleware;
return middleware;
}
}
-module.exports = { createActorLimiter, defaultActor, WINDOWS };
+// Check every window, then count every window, atomically. KEYS: one counter per window.
+// ARGV: per window max, ttl (seconds), retry (seconds). Returns {0}, or the exceeded window with the longest
+// wait as {index (1-based), retry}, as the in-process counters decide.
+const HIT_SCRIPT = `
+local n = #KEYS
+local worst, wr = 0, -1
+for i = 1, n do
+ local c = tonumber(redis.call('GET', KEYS[i]) or '0')
+ if c >= tonumber(ARGV[(i - 1) * 3 + 1]) then
+ local r = tonumber(ARGV[(i - 1) * 3 + 3])
+ if r > wr then worst = i; wr = r end
+ end
+end
+if worst > 0 then return {worst, wr} end
+for i = 1, n do
+ if redis.call('INCR', KEYS[i]) == 1 then redis.call('EXPIRE', KEYS[i], ARGV[(i - 1) * 3 + 2]) end
+end
+return {0}`;
+
+/** A shared counter store on Valkey for createActorLimiter({ store }). */
+function createValkeyLimitStore(valkey) {
+ if (!valkey) return null;
+ const c = valkey.client;
+ if (typeof c.ovLimitHit !== 'function') c.defineCommand('ovLimitHit', { lua: HIT_SCRIPT });
+ return {
+ async hit(name, actor, windows, t) {
+ const keys = []; const args = [];
+ for (const [w, max, len] of windows) {
+ const start = Math.floor(t / len) * len;
+ keys.push(valkey.key('lim', name, w, String(start), actor));
+ args.push(String(max), String(len + 5), String(start + len - t));
+ }
+ if (!keys.length) return { limited: false };
+ const r = await c.ovLimitHit(keys.length, ...keys, ...args);
+ if (!r || Number(r[0]) === 0) return { limited: false };
+ const [w, max] = windows[Number(r[0]) - 1];
+ return { limited: true, window: w, max, retry: Number(r[1]) };
+ },
+ };
+}
+
+module.exports = { createActorLimiter, createValkeyLimitStore, defaultActor, WINDOWS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,55 @@
@@ -0,0 +1,55 @@
+'use strict';
+/**
+ * openvibe-sdk/pubsub — fan-out between a service's processes and hosts on Valkey (ADR-035), in-process without
+ * Valkey. For realtime delivery (chat rooms, live counters, cache invalidation), never for durable work: a
+ * message published while nobody listens is gone (use openvibe-sdk/queue or OpenVibe.Events for that).
+ *
+ * const ps = createPubSub({ valkey });
+ * const off = await ps.subscribe('room:42', (msg) => broadcast(msg));
+ * await ps.publish('room:42', { text: 'hi' });
+ *
+ * Channels live inside the service's prefix (its Valkey user may use no others). Messages are JSON.
+ */
+const { EventEmitter } = require('events');
+
+function createPubSub({ valkey = null, log = console } = {}) {
+ if (!valkey) {
+ const ee = new EventEmitter(); ee.setMaxListeners(0);
+ return {
+ async publish(channel, msg) { const n = ee.listenerCount(channel); ee.emit(channel, JSON.parse(JSON.stringify(msg))); return n; },
+ async subscribe(channel, fn) { ee.on(channel, fn); return async () => { ee.off(channel, fn); }; },
+ async close() { ee.removeAllListeners(); },
+ };
+ }
+ const handlers = new Map(); // full channel -> Set(fn)
+ let sub = null;
+ const full = (ch) => valkey.key('ps', ch);
+ function subscriber() {
+ if (sub) return sub;
+ sub = valkey.duplicate();
+ sub.on('message', (ch, raw) => {
+ const set = handlers.get(ch);
+ if (!set) return;
+ let msg; try { msg = JSON.parse(raw); } catch { return; }
+ for (const fn of set) { try { fn(msg); } catch (err) { log.warn(`[pubsub] handler on ${ch}: ${err.message}`); } }
+ });
+ return sub;
+ }
+ return {
+ publish: (channel, msg) => valkey.client.publish(full(channel), JSON.stringify(msg)),
+ async subscribe(channel, fn) {
+ const ch = full(channel);
+ if (!handlers.has(ch)) { handlers.set(ch, new Set()); await subscriber().subscribe(ch); }
+ handlers.get(ch).add(fn);
+ return async () => {
+ const set = handlers.get(ch);
+ if (!set) return;
+ set.delete(fn);
+ if (!set.size) { handlers.delete(ch); await subscriber().unsubscribe(ch).catch(() => {}); }
+ };
+ },
+ async close() { handlers.clear(); if (sub) { try { await sub.quit(); } catch { sub.disconnect(); } sub = null; } },
+ };
+}
+
+module.exports = { createPubSub };
new file mode 100644
new file mode 100644
@@ -0,0 +1,187 @@
@@ -0,0 +1,187 @@
+'use strict';
+/**
+ * openvibe-sdk/queue — durable background jobs on Valkey streams (ADR-035), in-process without Valkey (tests).
+ *
+ * const q = createQueue({ valkey, name: 'thumbnails' });
+ * await q.add({ objectId }, { delayMs: 0, attempts: 5, id: `thumb:${objectId}` });
+ * const worker = q.process(async (data, job) => { … }, { concurrency: 4, visibilityMs: 60000 });
+ * await worker.stop(); // finishes running jobs first
+ *
+ * At-least-once: a job leaves the stream only after its handler resolves. A worker that dies mid-job leaves it
+ * pending, and another worker reclaims it after visibilityMs. A failure retries with exponential backoff
+ * (1 s, 2 s, 4 s … capped at 5 min), and after `attempts` tries the job lands in the dead-letter stream with its
+ * last error (`q.dead()`). `id` dedupes adds for 24 hours. Handlers must therefore be idempotent. Any number of
+ * workers on any number of hosts share one queue through its consumer group.
+ */
+const crypto = require('crypto');
+const os = require('os');
+
+const backoff = (attempt) => Math.min(300000, 1000 * 2 ** (attempt - 1));
+const uid = () => crypto.randomBytes(6).toString('hex');
+
+// Move due delayed jobs into the stream. Member: "<uid>|<attempt>|<max>|<json>" (no JSON re-encoding in Lua).
+const PROMOTE = `
+local due = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', ARGV[1], 'LIMIT', 0, 200)
+for _, m in ipairs(due) do
+ local p1 = string.find(m, '|', 1, true)
+ local p2 = string.find(m, '|', p1 + 1, true)
+ local p3 = string.find(m, '|', p2 + 1, true)
+ redis.call('XADD', KEYS[2], 'MAXLEN', '~', ARGV[2], '*', 'd', string.sub(m, p3 + 1), 'a', string.sub(m, p1 + 1, p2 - 1), 'm', string.sub(m, p2 + 1, p3 - 1))
+ redis.call('ZREM', KEYS[1], m)
+end
+return #due`;
+
+function fieldsOf(arr) { const o = {}; for (let i = 0; i < arr.length; i += 2) o[arr[i]] = arr[i + 1]; return o; }
+
+function createQueue({ valkey = null, name, group = 'workers', maxLen = 100000, log = console } = {}) {
+ if (!/^[a-z][a-z0-9_.-]{0,63}$/.test(String(name || ''))) throw new TypeError('queue: name must be a short lowercase identifier');
+ return valkey ? valkeyQueue(valkey, name, group, maxLen, log) : memoryQueue(name, log);
+}
+
+function valkeyQueue(valkey, name, group, maxLen, log) {
+ const c = valkey.client;
+ const key = valkey.key('q', name);
+ const delayed = valkey.key('q', name, 'delayed');
+ const dead = valkey.key('q', name, 'dead');
+ const dedupe = (id) => valkey.key('q', name, 'id', id);
+ if (typeof c.ovQueuePromote !== 'function') c.defineCommand('ovQueuePromote', { numberOfKeys: 2, lua: PROMOTE });
+
+ async function ensureGroup() {
+ try { await c.xgroup('CREATE', key, group, '0', 'MKSTREAM'); } catch (err) { if (!/BUSYGROUP/.test(err.message)) throw err; }
+ }
+
+ async function add(data, { delayMs = 0, attempts = 5, id = null } = {}) {
+ if (id && (await c.set(dedupe(id), '1', 'EX', 86400, 'NX')) !== 'OK') return null;
+ const json = JSON.stringify(data === undefined ? null : data);
+ if (delayMs > 0) {
+ await c.zadd(delayed, Date.now() + delayMs, `${uid()}|1|${attempts}|${json}`);
+ return 'delayed';
+ }
+ return c.xadd(key, 'MAXLEN', '~', maxLen, '*', 'd', json, 'a', '1', 'm', String(attempts));
+ }
+
+ function process(handler, { concurrency = 4, visibilityMs = 60000, blockMs = 5000 } = {}) {
+ const consumer = `${os.hostname()}:${global.process.pid}:${uid()}`;
+ const reader = valkey.duplicate();
+ let stopped = false; let active = 0; let wake = null;
+ const stats = { done: 0, failed: 0, retried: 0, dead: 0 };
+
+ async function run(id, raw) {
+ const f = fieldsOf(raw);
+ const attempt = Number(f.a) || 1; const max = Number(f.m) || 5;
+ active++;
+ try {
+ await handler(JSON.parse(f.d), { id, attempt, queue: name });
+ await c.multi().xack(key, group, id).xdel(key, id).exec();
+ stats.done++;
+ } catch (err) {
+ stats.failed++;
+ const reason = String((err && err.message) || err).slice(0, 500);
+ const m = c.multi();
+ if (attempt >= max) { m.xadd(dead, 'MAXLEN', '~', 10000, '*', 'd', f.d, 'a', String(attempt), 'e', reason, 'at', new Date().toISOString()); stats.dead++; log.warn(`[queue ${name}] job ${id} dead after ${attempt} attempts: ${reason}`); }
+ else { m.zadd(delayed, Date.now() + backoff(attempt), `${uid()}|${attempt + 1}|${max}|${f.d}`); stats.retried++; }
+ await m.xack(key, group, id).xdel(key, id).exec().catch((e) => log.warn(`[queue ${name}] could not requeue ${id}: ${e.message}`));
+ } finally {
+ active--;
+ if (wake) { const w = wake; wake = null; w(); }
+ }
+ }
+
+ async function loop() {
+ await ensureGroup();
+ while (!stopped) {
+ const free = concurrency - active;
+ if (free <= 0) { await new Promise((r) => { wake = r; }); continue; }
+ let res;
+ try { res = await reader.xreadgroup('GROUP', group, consumer, 'COUNT', free, 'BLOCK', blockMs, 'STREAMS', key, '>'); } catch (err) {
+ if (stopped) break;
+ if (/NOGROUP/.test(err.message)) { await ensureGroup().catch(() => {}); continue; }
+ log.warn(`[queue ${name}] read: ${err.message}`);
+ await new Promise((r) => setTimeout(r, 1000));
+ continue;
+ }
+ if (!res) continue;
+ for (const [, entries] of res) for (const [id, raw] of entries) run(id, raw);
+ }
+ }
+
+ const promote = setInterval(() => { c.ovQueuePromote(delayed, key, String(Date.now()), String(maxLen)).catch(() => {}); }, 1000);
+ const reclaim = setInterval(async () => {
+ if (stopped || active >= concurrency) return;
+ try {
+ const [, entries] = await c.xautoclaim(key, group, consumer, visibilityMs, '0-0', 'COUNT', Math.max(1, concurrency - active));
+ for (const [id, raw] of entries || []) if (raw) run(id, raw);
+ } catch { /* the next tick tries again */ }
+ }, Math.max(1000, Math.floor(visibilityMs / 2)));
+ promote.unref(); reclaim.unref();
+ const done = loop().catch((err) => log.error(`[queue ${name}] worker stopped: ${err.message}`));
+
+ return {
+ stats: () => ({ ...stats, active }),
+ async stop(timeoutMs = 30000) {
+ stopped = true;
+ clearInterval(promote); clearInterval(reclaim);
+ reader.disconnect();
+ if (wake) { const w = wake; wake = null; w(); }
+ const until = Date.now() + timeoutMs;
+ while (active > 0 && Date.now() < until) await new Promise((r) => setTimeout(r, 50));
+ await done;
+ },
+ };
+ }
+
+ return {
+ name, add, process,
+ async stats() {
+ const [len, del, dd] = await Promise.all([c.xlen(key).catch(() => 0), c.zcard(delayed), c.xlen(dead).catch(() => 0)]);
+ let pending = 0;
+ try { const p = await c.xpending(key, group); pending = Number(p[0]) || 0; } catch { /* no group yet */ }
+ return { waiting: Math.max(0, len - pending), pending, delayed: del, dead: dd };
+ },
+ async dead(count = 50) {
+ const rows = await c.xrevrange(dead, '+', '-', 'COUNT', count);
+ return rows.map(([id, raw]) => { const f = fieldsOf(raw); return { id, data: JSON.parse(f.d), attempts: Number(f.a), error: f.e, at: f.at }; });
+ },
+ };
+}
+
+// ── In-process queue with the same behaviour (tests, single-process tools) ─────────────────
+function memoryQueue(name, log) {
+ const ready = []; const deadList = []; const seen = new Map(); const workers = new Set();
+ let seq = 0;
+ const kick = () => { for (const w of workers) w(); };
+ function add(data, { delayMs = 0, attempts = 5, id = null } = {}) {
+ if (id) { const t = seen.get(id); if (t && t > Date.now()) return Promise.resolve(null); seen.set(id, Date.now() + 86400000); }
+ const job = { id: `m-${++seq}`, data: JSON.parse(JSON.stringify(data === undefined ? null : data)), attempt: 1, max: attempts };
+ if (delayMs > 0) setTimeout(() => { ready.push(job); kick(); }, delayMs).unref();
+ else { ready.push(job); kick(); }
+ return Promise.resolve(delayMs > 0 ? 'delayed' : job.id);
+ }
+ function process(handler, { concurrency = 4, backoffMs = backoff } = {}) {
+ let active = 0; let stopped = false;
+ const stats = { done: 0, failed: 0, retried: 0, dead: 0 };
+ const pump = () => {
+ while (!stopped && active < concurrency && ready.length) {
+ const job = ready.shift(); active++;
+ Promise.resolve().then(() => handler(job.data, { id: job.id, attempt: job.attempt, queue: name })).then(() => { stats.done++; }, (err) => {
+ stats.failed++;
+ if (job.attempt >= job.max) { stats.dead++; deadList.push({ id: job.id, data: job.data, attempts: job.attempt, error: String((err && err.message) || err), at: new Date().toISOString() }); }
+ else { stats.retried++; const next = { ...job, attempt: job.attempt + 1 }; setTimeout(() => { ready.push(next); kick(); }, backoffMs(job.attempt)).unref(); }
+ }).finally(() => { active--; pump(); });
+ }
+ };
+ workers.add(pump); pump();
+ return {
+ stats: () => ({ ...stats, active }),
+ async stop(timeoutMs = 30000) { stopped = true; workers.delete(pump); const until = Date.now() + timeoutMs; while (active > 0 && Date.now() < until) await new Promise((r) => setTimeout(r, 10)); },
+ };
+ }
+ void log;
+ return {
+ name, add, process,
+ async stats() { return { waiting: ready.length, pending: 0, delayed: 0, dead: deadList.length }; },
+ async dead(count = 50) { return deadList.slice(-count).reverse(); },
+ };
+}
+
+module.exports = { createQueue, backoff };
new file mode 100644
new file mode 100644
@@ -0,0 +1,46 @@
@@ -0,0 +1,46 @@
+'use strict';
+/**
+ * openvibe-sdk/valkey — the shared connection to Valkey (ADR-035): the service's user is confined to its key
+ * prefix (VALKEY_PREFIX, e.g. "ov:wiki:"), so every key and channel this SDK touches goes through key().
+ * Valkey is never authoritative (ADR-007 2026-09-24 rule 4): caches, limits, queues and fan-out only.
+ *
+ * const { createValkey } = require('openvibe-sdk/valkey');
+ * const valkey = createValkey(); // VALKEY_URL + VALKEY_PREFIX, or { url, prefix }
+ * await valkey.client.set(valkey.key('x'), '1', 'EX', 60);
+ *
+ * Commands are auto-pipelined (many callers in one tick share one round trip). A blocking consumer (queues) or a
+ * subscriber gets its own connection through duplicate(). Without a URL, createValkey returns null, and the
+ * cache, limits, queue and pubsub modules fall back to their in-process implementations.
+ */
+
+function createValkey({ url = process.env.VALKEY_URL, prefix = process.env.VALKEY_PREFIX || '', lazyConnect = false, log = console, client = null } = {}) {
+ if (!url && !client) return null;
+ let Valkey;
+ if (!client) {
+ try { Valkey = require('iovalkey'); } catch { throw new Error('openvibe-sdk/valkey: install `iovalkey` in the service (npm i iovalkey)'); }
+ }
+ const opts = { maxRetriesPerRequest: 3, enableAutoPipelining: true, connectTimeout: 5000, lazyConnect };
+ const c = client || new Valkey(url, opts);
+ let lastError = null;
+ c.on('error', (err) => { if (!lastError || lastError.message !== err.message) log.warn(`[valkey] ${err.message}`); lastError = err; });
+ c.on('ready', () => { lastError = null; });
+ const dupes = new Set();
+ return {
+ client: c,
+ prefix,
+ /** The full key (or channel) for a name inside this service's prefix. */
+ key: (...parts) => prefix + parts.join(':'),
+ /** A second connection with the same settings (for BLOCK reads and subscriptions). */
+ duplicate() { const d = c.duplicate({ enableAutoPipelining: false }); d.on('error', () => {}); dupes.add(d); return d; },
+ /** Readiness: a real PING within 1 s. */
+ async ready() {
+ try {
+ const pong = await Promise.race([c.ping(), new Promise((_, rej) => setTimeout(() => rej(new Error('timeout')), 1000))]);
+ return pong === 'PONG' ? { ok: true, detail: { store: 'valkey' } } : { ok: false, error: `unexpected ${pong}` };
+ } catch (err) { return { ok: false, error: `valkey: ${err.message}` }; }
+ },
+ async close() { for (const d of dupes) { try { await d.quit(); } catch { d.disconnect(); } } try { await c.quit(); } catch { c.disconnect(); } },
+ };
+}
+
+module.exports = { createValkey };
@@ -76,7 +76,7 @@ run([
@@ -76,7 +76,7 @@ run([
for (const [re, why] of FORBIDDEN) if (re.test(src)) problems.push(`${path.relative(ROOT, file)}: ${why}`);
for (const [re, why] of FORBIDDEN) if (re.test(src)) problems.push(`${path.relative(ROOT, file)}: ${why}`);
}
}
assert.deepEqual(problems, []);
assert.deepEqual(problems, []);
- assert.deepEqual(blocked.sort(), ['./events', './identity', './limits', './testing', './vip']);
+ assert.deepEqual(blocked.sort(), ['./cache', './db', './events', './identity', './limits', './pubsub', './queue', './testing', './valkey', './vip']);
const rel = [...scanned].map((f) => path.relative(ROOT, f));
const rel = [...scanned].map((f) => path.relative(ROOT, f));
for (const must of ['browser.js', 'src/core/client.js', 'src/auth/browser.js', 'src/realtime.js', 'src/media.js', 'src/community.js', 'src/modules.js', 'src/registry.js', 'src/jobs.js', 'src/tools.js', 'src/core/form.js', 'src/projects.js', 'browser/openvibe-sdk.mjs']) {
for (const must of ['browser.js', 'src/core/client.js', 'src/auth/browser.js', 'src/realtime.js', 'src/media.js', 'src/community.js', 'src/modules.js', 'src/registry.js', 'src/jobs.js', 'src/tools.js', 'src/core/form.js', 'src/projects.js', 'browser/openvibe-sdk.mjs']) {
assert.ok(rel.includes(must), `${must} is scanned`);
assert.ok(rel.includes(must), `${must} is scanned`);
new file mode 100644
new file mode 100644
@@ -0,0 +1,188 @@
@@ -0,0 +1,188 @@
+'use strict';
+/**
+ * db (ADR-035): the sql builder binds every value; both adapters return the same row shapes (int8 → Number,
+ * timestamps → ISO, dates → 'YYYY-MM-DD', numeric → exact text, json parsed); transactions commit, roll back
+ * and nest as savepoints; serializable conflicts retry; migrations run once, in order, refuse edits, and hold a
+ * contract migration inside the N-1 window; importSqlite copies a SQLite file into the schema parents first,
+ * converts by target type, keeps identity values, and verifies counts and checksums. The suite runs on PGlite
+ * always, and again through PgBouncer against PostgreSQL 18 when OV_TEST_PG_URL is set
+ * (scripts/test-services.sh up).
+ */
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { run } = require('./helpers');
+const { createDb, sql, importSqlite } = require('../src/db');
+const { parse } = require('../src/db/migrate');
+
+const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ovsdk-db-'));
+const mdir = (name, files) => { const d = path.join(tmp, name); fs.mkdirSync(d, { recursive: true }); for (const [f, text] of Object.entries(files)) fs.writeFileSync(path.join(d, f), text); return d; };
+const PAGES = mdir('pages', {
+ '0001_pages.sql': `-- phase: expand
+CREATE TABLE spaces (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, name text NOT NULL);
+CREATE TABLE pages (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, space_id bigint REFERENCES spaces(id), slug text NOT NULL UNIQUE,
+ title text NOT NULL, meta jsonb NOT NULL DEFAULT '{}', public boolean NOT NULL DEFAULT true, views bigint NOT NULL DEFAULT 0,
+ price numeric(10,2), tags text[] NOT NULL DEFAULT '{}', created_at timestamptz NOT NULL DEFAULT now(), day date);
+`,
+});
+
+function suite(label, open) {
+ return [
+ [`${label}: sql binds values, fragments nest, identifiers are checked`, async () => {
+ const q = sql`SELECT * FROM t WHERE a = ${1} AND b = ANY(${[2, 3]}) AND ${sql`c = ${'x'}`}`.compile();
+ assert.equal(q.text, 'SELECT * FROM t WHERE a = $1 AND b = ANY($2) AND c = $3');
+ assert.deepEqual(q.values, [1, [2, 3], 'x']);
+ assert.equal(sql`${sql.ident('public.pages')}`.compile().text, '"public"."pages"');
+ assert.throws(() => sql.ident('pages; DROP TABLE x'), /not an identifier/);
+ const ins = sql`INSERT INTO t ${sql.insert([{ a: 1, b: null }, { a: 2, b: 'y' }])}`.compile();
+ assert.equal(ins.text, 'INSERT INTO t ("a", "b") VALUES ($1, $2), ($3, $4)');
+ assert.equal(sql`UPDATE t SET ${sql.set({ a: 1, skip: undefined, b: 2 })}`.compile().text, 'UPDATE t SET "a" = $1, "b" = $2');
+ }],
+ [`${label}: migrations, types, queries, transactions`, async () => {
+ const { db, migrate } = await open();
+ try {
+ const m = await migrate(PAGES);
+ assert.deepEqual(m.applied.map((a) => a.id), ['0001']);
+ assert.equal((await migrate(PAGES)).applied.length, 0, 'applied once');
+ const sp = await db.one(sql`INSERT INTO spaces (name) VALUES (${'main'}) RETURNING id`);
+ await db.exec(sql`INSERT INTO pages ${sql.insert([
+ { space_id: sp.id, slug: 'a', title: 'A', meta: JSON.stringify({ x: 1, list: [] }), price: '12.50', tags: ['x', 'y'], day: '2026-09-28' },
+ { space_id: sp.id, slug: 'b', title: 'B', meta: '{}', public: false, price: null, tags: [], day: null },
+ ])}`);
+ const a = await db.one(sql`SELECT * FROM pages WHERE slug = ${'a'}`);
+ assert.equal(typeof a.id, 'number');
+ assert.deepEqual(a.meta, { x: 1, list: [] });
+ assert.equal(a.price, '12.50', 'numeric stays exact text');
+ assert.deepEqual(a.tags, ['x', 'y']);
+ assert.match(a.created_at, /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/);
+ assert.equal(a.day, '2026-09-28');
+ assert.equal(a.public, true);
+ assert.equal(await db.maybe(sql`SELECT 1 FROM pages WHERE slug = ${'zzz'}`), null);
+ await assert.rejects(db.one(sql`SELECT 1 FROM pages WHERE slug = ${'zzz'}`), /expected one row/);
+ assert.equal(await db.value(sql`SELECT count(*) FROM pages`), 2);
+ assert.deepEqual((await db.many(sql`SELECT slug FROM pages WHERE slug = ANY(${['a', 'b']}) ORDER BY slug`)).map((r) => r.slug), ['a', 'b']);
+ // A transaction commits; an inner savepoint rolls back alone; a thrown error rolls everything back.
+ const title = await db.tx(async (t) => {
+ await t.exec(sql`UPDATE pages SET views = views + 1 WHERE slug = 'a'`);
+ await t.tx(async (s) => { await s.exec(sql`UPDATE pages SET title = 'X'`); throw new Error('inner'); }).catch(() => {});
+ return t.value(sql`SELECT title FROM pages WHERE slug = 'a'`);
+ });
+ assert.equal(title, 'A');
+ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'a'`), 1);
+ await assert.rejects(db.tx(async (t) => { await t.exec(sql`UPDATE pages SET views = 99`); throw new Error('boom'); }), /boom/);
+ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'a'`), 1, 'rolled back');
+ await assert.rejects(db.tx(async () => {}, { isolation: 'nonsense' }), /isolation/);
+ // Errors carry the PostgreSQL code and the statement, not the values.
+ const err = await db.exec(sql`INSERT INTO pages (slug, title) VALUES (${'a'}, ${'dup'})`).catch((e) => e);
+ assert.equal(err.code, '23505');
+ assert.match(err.statement, /INSERT INTO pages/);
+ // Serializable increments from 8 concurrent transactions all land (retries on 40001).
+ await Promise.all(Array.from({ length: 8 }, () => db.tx(async (t) => {
+ const v = await t.value(sql`SELECT views FROM pages WHERE slug = 'b'`);
+ await t.exec(sql`UPDATE pages SET views = ${v + 1} WHERE slug = 'b'`);
+ }, { isolation: 'serializable', retries: 30 })));
+ assert.equal(await db.value(sql`SELECT views FROM pages WHERE slug = 'b'`), 8);
+ const r = await db.ready();
+ assert.equal(r.ok, true);
+ assert.equal(r.detail.store, label === 'pglite' ? 'pglite' : 'postgresql');
+ } finally { await db.close(); }
+ }],
+ ];
+}
+
+async function pgOpen() {
+ // Migrations run as the owner on the direct connection; everything else through PgBouncer, as in production.
+ const owner = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL, service: 'sdk-test' });
+ await owner.query('DROP TABLE IF EXISTS pages, spaces, ov_migrations CASCADE');
+ const db = createDb({ url: process.env.OV_TEST_PG_URL, service: 'sdk-test', max: 4 });
+ const close = db.close;
+ db.close = async () => { await close(); await owner.close(); };
+ return { db, migrate: (dir) => owner.migrate({ dir, log: { log() {} } }) };
+}
+
+const tests = [
+ ...suite('pglite', async () => { const db = createDb({ pglite: true, service: 'sdk-test' }); return { db, migrate: (dir) => db.migrate({ dir, log: { log() {} } }) }; }),
+ ['migration rules: phases, edits refused, order kept, contract held in the N-1 window, no-transaction', async () => {
+ assert.throws(() => parse(mdir('bad1', { 'x.sql': '-- phase: expand\nSELECT 1;' })), /NNNN_description/);
+ assert.throws(() => parse(mdir('bad2', { '0001_a.sql': 'SELECT 1;' })), /phase/);
+ assert.throws(() => parse(mdir('bad3', { '0001_a.sql': '-- phase: contract\nSELECT 1;' })), /must name its expand/);
+ const db = createDb({ pglite: true });
+ const quiet = { log() {} };
+ try {
+ const d = mdir('rules', {
+ '0001_add.sql': '-- phase: expand\nCREATE TABLE t (id int PRIMARY KEY, old text, new text);',
+ '0002_fill.sql': '-- phase: migrate\nUPDATE t SET new = old;',
+ '0003_drop.sql': '-- phase: contract\n-- after: 0001\nALTER TABLE t DROP COLUMN old;',
+ '0004_idx.sql': '-- phase: expand\n-- no-transaction\nCREATE INDEX t_new ON t (new);',
+ });
+ let r = await db.migrate({ dir: d, log: quiet });
+ assert.deepEqual(r.applied.map((a) => a.id), ['0001', '0002']);
+ assert.match(r.held[0].reason, /N-1 window is 7/);
+ r = await db.migrate({ dir: d, log: quiet, now: () => Date.now() + 8 * 86400000 });
+ assert.deepEqual(r.applied.map((a) => a.id), ['0003', '0004'], 'after the window the contract runs, then the rest');
+ fs.writeFileSync(path.join(d, '0002_fill.sql'), '-- phase: migrate\nUPDATE t SET new = upper(old);');
+ await assert.rejects(db.migrate({ dir: d, log: quiet }), /changed after it was applied/);
+ } finally { await db.close(); }
+ const db2 = createDb({ pglite: true });
+ try {
+ const d2 = mdir('order', { '0001_add.sql': '-- phase: expand\nCREATE TABLE u (id int);', '0005_b.sql': '-- phase: expand\nSELECT 1;' });
+ await db2.migrate({ dir: d2, log: quiet });
+ fs.writeFileSync(path.join(d2, '0003_late.sql'), '-- phase: expand\nSELECT 1;');
+ await assert.rejects(db2.migrate({ dir: d2, log: quiet }), /older than applied migration 0005/);
+ } finally { await db2.close(); }
+ }],
+ ['importSqlite: parents first, conversions by target type, identities kept, verified', async () => {
+ const Database = require('better-sqlite3');
+ const file = path.join(tmp, 'legacy.db');
+ const s = new Database(file);
+ s.exec(`CREATE TABLE people (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT NOT NULL, is_admin INTEGER DEFAULT 0, settings TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, legacy_note TEXT);
+ CREATE TABLE notes (id INTEGER PRIMARY KEY, person_id INTEGER REFERENCES people(id), body TEXT, pinned INTEGER, posted INTEGER, tags TEXT);
+ CREATE TABLE loose (k TEXT, v TEXT);`);
+ const ins = s.prepare('INSERT INTO people (id, username, is_admin, settings, created_at, legacy_note) VALUES (?, ?, ?, ?, ?, ?)');
+ ins.run(3, 'ann', 1, '{"theme":"blue","n":[1,2]}', '2026-09-01 10:00:00', 'drop me');
+ ins.run(7, 'bob', 0, null, '2026-09-02T11:30:00.000Z', null);
+ ins.run(12, 'zoë', 0, '{}', '2026-09-03 09:15:00', null);
+ s.prepare('INSERT INTO notes VALUES (?, ?, ?, ?, ?, ?)').run(1, 3, 'hello', 1, 1790000000, '["a","b"]');
+ s.prepare('INSERT INTO notes VALUES (?, ?, ?, ?, ?, ?)').run(2, 7, 'ms time', 0, 1790000000123, '[]');
+ s.prepare('INSERT INTO loose VALUES (?, ?)').run('b', '2');
+ s.prepare('INSERT INTO loose VALUES (?, ?)').run('a', '1');
+ s.close();
+
+ const db = createDb({ pglite: true });
+ try {
+ await db.migrate({ dir: mdir('imp', { '0001_s.sql': `-- phase: expand
+CREATE TABLE people (id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, username text NOT NULL, is_admin boolean NOT NULL DEFAULT false, settings jsonb, created_at timestamptz NOT NULL DEFAULT now());
+CREATE TABLE notes (id bigint PRIMARY KEY, person_id bigint REFERENCES people(id), body text, pinned boolean, posted timestamptz, tags text[] NOT NULL DEFAULT '{}');
+CREATE TABLE loose (k text, v text);` }), log: { log() {} } });
+ const quiet = { log() {} };
+ // A source column with no target column is a problem until it is dropped on purpose.
+ let r = await importSqlite({ sqlite: file, db, log: quiet });
+ assert.equal(r.ok, false);
+ assert.match(r.problems[0].problem, /legacy_note/);
+ r = await importSqlite({ sqlite: file, db, log: quiet, tables: { people: { dropColumns: ['legacy_note'] } } });
+ assert.equal(r.ok, true, JSON.stringify(r.problems));
+ assert.deepEqual(r.tables.map((t) => [t.table, t.rows]), [['loose', 2], ['people', 3], ['notes', 2]], 'parents before children');
+ const ann = await db.one(sql`SELECT * FROM people WHERE id = 3`);
+ assert.deepEqual([ann.username, ann.is_admin, ann.settings, ann.created_at], ['ann', true, { theme: 'blue', n: [1, 2] }, '2026-09-01T10:00:00.000Z']);
+ const n2 = await db.one(sql`SELECT * FROM notes WHERE id = 2`);
+ assert.deepEqual([n2.pinned, n2.posted, n2.tags], [false, new Date(1790000000123).toISOString(), []]);
+ assert.equal((await db.one(sql`SELECT * FROM notes WHERE id = 1`)).posted, new Date(1790000000 * 1000).toISOString(), 'epoch seconds');
+ const next = await db.one(sql`INSERT INTO people (username) VALUES ('new') RETURNING id`);
+ assert.equal(next.id, 13, 'the identity continues after the imported maximum');
+ // A second import onto the same rows fails loudly; truncate makes a rehearsal repeatable.
+ r = await importSqlite({ sqlite: file, db, log: quiet, truncate: true, tables: { people: { dropColumns: ['legacy_note'] } } });
+ assert.equal(r.ok, true);
+ // A transformation that differs between copy and check shows up as a failed verification.
+ let flip = 0;
+ r = await importSqlite({ sqlite: file, db, log: quiet, truncate: true, only: ['loose'], tables: { loose: { map: (row) => ({ ...row, v: String(flip++) }) } } });
+ assert.equal(r.ok, false);
+ assert.match(r.problems[0].problem, /verification failed/);
+ } finally { await db.close(); }
+ }],
+];
+
+if (process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL) tests.push(...suite('postgresql+pgbouncer', pgOpen));
+else console.log('db through PgBouncer: skipped (OV_TEST_PG_URL not set; scripts/test-services.sh up)');
+
+run(tests).then(() => fs.rmSync(tmp, { recursive: true, force: true }));
@@ -66,7 +66,7 @@ run([
@@ -66,7 +66,7 @@ run([
})().catch((err) => { console.error(err); process.exit(1); });`;
})().catch((err) => { console.error(err); process.exit(1); });`;
const res = execFileSync(process.execPath, ['-e', script], { cwd: app, encoding: 'utf8' });
const res = execFileSync(process.execPath, ['-e', script], { cwd: app, encoding: 'utf8' });
assert.match(res, new RegExp(`consumer ok: ${subpaths.length} subpaths \\+ the browser bundle`));
assert.match(res, new RegExp(`consumer ok: ${subpaths.length} subpaths \\+ the browser bundle`));
- assert.equal(subpaths.length, 21);
+ assert.equal(subpaths.length, 26);
assert.ok(!fs.existsSync(path.join(app, 'node_modules/openvibe-contracts')), 'the optional peer is not installed');
assert.ok(!fs.existsSync(path.join(app, 'node_modules/openvibe-contracts')), 'the optional peer is not installed');
} finally {
} finally {
fs.rmSync(tmp, { recursive: true, force: true });
fs.rmSync(tmp, { recursive: true, force: true });
new file mode 100644
new file mode 100644
@@ -0,0 +1,168 @@
@@ -0,0 +1,168 @@
+'use strict';
+/**
+ * valkey, cache, limits store, queue, pubsub (ADR-035): the in-process versions always; against Valkey 9 when
+ * OV_TEST_VALKEY_URL is set (scripts/test-services.sh up). With Valkey: getOrSet loads once across two
+ * connections, tags invalidate, two limiters share one count and fall back to their own when Valkey fails, the
+ * queue retries with backoff, dead-letters, delays, dedupes and reclaims a job a hung worker holds, pub/sub
+ * crosses connections, and every key stays inside the service prefix.
+ */
+const assert = require('node:assert/strict');
+const crypto = require('node:crypto');
+const { run, sleep } = require('./helpers');
+const { createValkey } = require('../src/valkey');
+const { createCache } = require('../src/cache');
+const { createQueue } = require('../src/queue');
+const { createPubSub } = require('../src/pubsub');
+const { createActorLimiter, createValkeyLimitStore } = require('../src/limits');
+
+const quiet = { log() {}, warn() {}, error() {} };
+function call(mw, req) {
+ return new Promise((resolve) => {
+ const res = { statusCode: 200, headers: {}, setHeader(k, v) { this.headers[k.toLowerCase()] = v; }, end: (body) => resolve({ status: res.statusCode, retry: res.headers['retry-after'], body }) };
+ mw(req, res, () => resolve({ status: 200 }));
+ });
+}
+const until = async (fn, ms = 5000) => { const end = Date.now() + ms; while (Date.now() < end) { if (await fn()) return true; await sleep(25); } return false; };
+
+const tests = [
+ ['memory cache: single-flight, TTL, tags', async () => {
+ const cache = createCache({ namespace: 'm', ttlSec: 60, log: quiet });
+ let loads = 0;
+ const vals = await Promise.all(Array.from({ length: 10 }, () => cache.getOrSet('k', 60, async () => { loads++; await sleep(20); return { v: 1 }; })));
+ assert.equal(loads, 1);
+ assert.deepEqual(vals[9], { v: 1 });
+ await cache.set('t1', 'a', 60, { tags: ['space:1'] }); await cache.set('t2', 'b', 60, { tags: ['space:1'] });
+ assert.equal(await cache.invalidateTag('space:1'), 2);
+ assert.equal(await cache.get('t1'), undefined);
+ assert.equal(cache.stats().store, 'memory');
+ }],
+ ['memory queue: runs, retries, dead-letters, dedupes', async () => {
+ const q = createQueue({ name: 'jobs' });
+ const seen = []; let flaky = 0;
+ const w = q.process(async (d) => {
+ if (d.kind === 'flaky' && flaky++ < 1) throw new Error('once');
+ if (d.kind === 'bad') throw new Error('always');
+ seen.push(d.n);
+ }, { concurrency: 2, backoffMs: () => 5 });
+ await q.add({ n: 1 }); await q.add({ n: 2 }); await q.add({ kind: 'flaky', n: 3 });
+ await q.add({ kind: 'bad' }, { attempts: 2 });
+ assert.equal(await q.add({ n: 9 }, { id: 'same' }) !== null, true);
+ assert.equal(await q.add({ n: 9 }, { id: 'same' }), null, 'deduped');
+ assert.ok(await until(async () => seen.length === 4 && (await q.stats()).dead === 1));
+ assert.deepEqual(seen.sort(), [1, 2, 3, 9]);
+ assert.equal((await q.dead())[0].error, 'always');
+ await w.stop();
+ }],
+ ['memory pubsub', async () => {
+ const ps = createPubSub();
+ const got = [];
+ const off = await ps.subscribe('room:1', (m) => got.push(m));
+ await ps.publish('room:1', { t: 'hi' });
+ await off();
+ await ps.publish('room:1', { t: 'gone' });
+ assert.deepEqual(got, [{ t: 'hi' }]);
+ }],
+];
+
+const URL = process.env.OV_TEST_VALKEY_URL;
+if (!URL) console.log('valkey integration: skipped (OV_TEST_VALKEY_URL not set; scripts/test-services.sh up)');
+else {
+ const prefix = `ovsdk:test:${crypto.randomBytes(3).toString('hex')}:`;
+ const open = () => createValkey({ url: URL, prefix, log: quiet });
+ tests.push(
+ ['valkey: ready, cache single-flight across connections, tags', async () => {
+ const a = open(); const b = open();
+ try {
+ await a.client.flushdb();
+ assert.equal((await a.ready()).ok, true);
+ const ca = createCache({ valkey: a, namespace: 'pages', log: quiet }); const cb = createCache({ valkey: b, namespace: 'pages', log: quiet });
+ let loads = 0;
+ const loader = async () => { loads++; await sleep(150); return { title: 'A' }; };
+ const [x, y] = await Promise.all([ca.getOrSet('slug:a', 60, loader), cb.getOrSet('slug:a', 60, loader)]);
+ assert.deepEqual([x, y], [{ title: 'A' }, { title: 'A' }]);
+ assert.equal(loads, 1, 'the second process waited for the first one\'s value');
+ await ca.set('p1', 1, 60, { tags: ['space:7'] });
+ assert.equal(await cb.invalidateTag('space:7'), 1);
+ assert.equal(await ca.get('p1'), undefined);
+ } finally { await a.close(); await b.close(); }
+ }],
+ ['valkey: two limiters share one count; a failing store falls back', async () => {
+ const a = open(); const b = open();
+ try {
+ let t = 1_000_000_020_000;
+ const la = createActorLimiter({ limits: { minute: 3 }, now: () => t, store: createValkeyLimitStore(a), log: quiet });
+ const lb = createActorLimiter({ limits: { minute: 3 }, now: () => t, store: createValkeyLimitStore(b), log: quiet });
+ const req = { user: { subject_id: 'usr_x' } };
+ const out = [];
+ for (const lim of [la, lb, la, lb]) out.push((await call(lim('wiki.page.write'), req)).status);
+ assert.deepEqual(out, [200, 200, 200, 429], 'three per minute across both processes');
+ t += 60_000;
+ assert.equal((await call(lb('wiki.page.write'), req)).status, 200, 'the next window reopens');
+ // Valkey gone: the limiter keeps counting in its own process.
+ const dead = createValkey({ url: 'redis://127.0.0.1:1/0', prefix, log: quiet });
+ dead.client.options.maxRetriesPerRequest = 0; dead.client.options.enableOfflineQueue = false;
+ const lf = createActorLimiter({ limits: { minute: 1 }, now: () => t, store: createValkeyLimitStore(dead), log: quiet });
+ assert.equal((await call(lf('x'), req)).status, 200);
+ assert.equal((await call(lf('x'), req)).status, 429, 'counted locally while the store fails');
+ dead.client.disconnect();
+ } finally { await a.close(); await b.close(); }
+ }],
+ ['valkey queue: processes, retries with backoff, dead-letters, delays, dedupes', async () => {
+ const v = open();
+ try {
+ const q = createQueue({ valkey: v, name: 'thumbs', log: quiet });
+ const done = []; let flaky = 0;
+ const w = q.process(async (d) => {
+ if (d.kind === 'flaky' && flaky++ < 1) throw new Error('once');
+ if (d.kind === 'bad') throw new Error('always');
+ done.push(d.n);
+ }, { concurrency: 3, blockMs: 200 });
+ await q.add({ n: 1 }); await q.add({ n: 2 }); await q.add({ kind: 'flaky', n: 3 });
+ await q.add({ kind: 'bad', n: 4 }, { attempts: 2 });
+ await q.add({ n: 5 }, { delayMs: 300 });
+ assert.notEqual(await q.add({ n: 6 }, { id: 'thumb:6' }), null);
+ assert.equal(await q.add({ n: 6 }, { id: 'thumb:6' }), null, 'deduped');
+ assert.ok(await until(async () => done.length === 5 && (await q.stats()).dead === 1, 8000), `done ${done}`);
+ assert.deepEqual(done.sort(), [1, 2, 3, 5, 6]);
+ const [d] = await q.dead();
+ assert.deepEqual([d.data, d.attempts, d.error], [{ kind: 'bad', n: 4 }, 2, 'always']);
+ const st = await q.stats();
+ assert.deepEqual([st.waiting, st.pending, st.delayed], [0, 0, 0]);
+ await w.stop();
+ } finally { await v.close(); }
+ }],
+ ['valkey queue: a job a hung worker holds is reclaimed after the visibility timeout', async () => {
+ const v1 = open(); const v2 = open();
+ try {
+ const q1 = createQueue({ valkey: v1, name: 'reclaim', log: quiet });
+ const q2 = createQueue({ valkey: v2, name: 'reclaim', log: quiet });
+ let took = false;
+ const hung = q1.process(async () => { took = true; await new Promise(() => {}); }, { concurrency: 1, blockMs: 200, visibilityMs: 60000 });
+ await q1.add({ n: 'x' });
+ assert.ok(await until(async () => took));
+ const got = [];
+ const rescuer = q2.process(async (d, job) => { got.push([d.n, job.attempt]); }, { concurrency: 1, blockMs: 200, visibilityMs: 600 });
+ assert.ok(await until(async () => got.length === 1, 6000), 'reclaimed');
+ assert.deepEqual(got[0], ['x', 1]);
+ await rescuer.stop(); await hung.stop(100);
+ } finally { await v1.close(); await v2.close(); }
+ }],
+ ['valkey pubsub across connections; every key stays inside the prefix', async () => {
+ const a = open(); const b = open();
+ try {
+ const pa = createPubSub({ valkey: a, log: quiet }); const pb = createPubSub({ valkey: b, log: quiet });
+ const got = [];
+ const off = await pb.subscribe('room:1', (m) => got.push(m));
+ assert.ok(await until(async () => (await pa.publish('room:1', { t: 'hi' })) === 1));
+ assert.ok(await until(async () => got.length >= 1));
+ assert.deepEqual(got[0], { t: 'hi' });
+ await off(); await pb.close();
+ const keys = await a.client.keys('*');
+ assert.ok(keys.length > 0);
+ assert.deepEqual(keys.filter((k) => !k.startsWith(prefix)), [], 'no key outside the prefix');
+ } finally { await a.close(); await b.close(); }
+ }],
+ );
+}
+
+run(tests).then(() => process.exit(0));
new file mode 100644
new file mode 100644
@@ -0,0 +1,12 @@
@@ -0,0 +1,12 @@
+/** openvibe-sdk/cache — shared caching on Valkey, in-process without it; never authoritative (ADR-035). */
+import type { Valkey } from './valkey';
+export interface Cache {
+ get<T = unknown>(key: string): Promise<T | undefined>;
+ set(key: string, value: unknown, ttlSec?: number, opts?: { tags?: string[] }): Promise<void>;
+ del(...keys: string[]): Promise<number>;
+ /** One load per key across processes (single-flight plus a short lock). */
+ getOrSet<T>(key: string, ttlSec: number, loader: () => Promise<T>, opts?: { tags?: string[] }): Promise<T>;
+ invalidateTag(tag: string): Promise<number>;
+ stats(): { hits: number; misses: number; loads: number; errors: number; store: 'valkey' | 'memory' };
+}
+export function createCache(opts?: { valkey?: Valkey | null; namespace?: string; ttlSec?: number; lockMs?: number; waitMs?: number; memoryMax?: number; log?: { warn(msg: string): void } }): Cache;
new file mode 100644
new file mode 100644
@@ -0,0 +1,64 @@
@@ -0,0 +1,64 @@
+/** openvibe-sdk/db — the async PostgreSQL data layer (ADR-035). */
+export declare class Sql {
+ readonly parts: string[];
+ readonly values: unknown[];
+ compile(start?: number): { text: string; values: unknown[] };
+}
+export interface SqlTag {
+ (strings: TemplateStringsArray, ...values: unknown[]): Sql;
+ /** "name" or "schema"."name"; throws on anything else. */
+ ident(name: string): Sql;
+ /** Trusted text, inserted as is. Never user input. */
+ raw(text: string): Sql;
+ join(items: Iterable<unknown>, sep?: Sql): Sql;
+ set(obj: Record<string, unknown>): Sql;
+ insert(rows: Record<string, unknown> | Record<string, unknown>[], columns?: string[]): Sql;
+ json(value: unknown): Sql;
+}
+export const sql: SqlTag;
+
+export type Row = Record<string, any>;
+export interface Queryable {
+ query(q: Sql | string, values?: unknown[]): Promise<{ rows: Row[]; rowCount: number }>;
+ many<T = Row>(q: Sql | string, values?: unknown[]): Promise<T[]>;
+ maybe<T = Row>(q: Sql | string, values?: unknown[]): Promise<T | null>;
+ /** Throws DbError (code 'no_rows') when there is none. */
+ one<T = Row>(q: Sql | string, values?: unknown[]): Promise<T>;
+ value<T = unknown>(q: Sql | string, values?: unknown[]): Promise<T | null>;
+ exec(q: Sql | string, values?: unknown[]): Promise<number>;
+}
+export interface Tx extends Queryable {
+ /** A nested savepoint: rolls back alone when fn throws. */
+ tx<T>(fn: (t: Tx) => Promise<T>): Promise<T>;
+}
+export interface TxOptions { isolation?: 'read committed' | 'repeatable read' | 'serializable'; retries?: number; readOnly?: boolean }
+export interface MigrateResult { applied: { id: string; name: string; phase: string; ms: number }[]; pending: { id: string; name: string; phase: string }[]; held: { id: string; reason: string }[] }
+export interface Db extends Queryable {
+ sql: SqlTag;
+ readonly store: 'postgresql' | 'pglite';
+ tx<T>(fn: (t: Tx) => Promise<T>, opts?: TxOptions): Promise<T>;
+ ready(): Promise<{ ok: true; detail: { store: string; pool: { total: number; idle: number; waiting: number } } } | { ok: false; error: string }>;
+ migrate(o: { dir: string; windowDays?: number; dryRun?: boolean; now?: () => number; log?: { log(msg: string): void } }): Promise<MigrateResult>;
+ stats(): { queries: number; errors: number; slow: number; retries: number; tx: number; pool: { total: number; idle: number; waiting: number } };
+ close(): Promise<void>;
+}
+export interface CreateDbOptions {
+ /** Default: DATABASE_URL (through PgBouncer). */
+ url?: string;
+ /** true: in-memory PGlite (tests); a directory: persisted; or a PGlite instance. */
+ pglite?: boolean | string | object;
+ service?: string;
+ max?: number;
+ queryTimeoutMs?: number;
+ slowMs?: number;
+ log?: { warn(msg: string): void; error(msg: string): void };
+ registry?: object;
+}
+export function createDb(opts?: CreateDbOptions): Db;
+export declare class DbError extends Error { code?: string; detail?: string; constraint?: string; table?: string; column?: string; statement?: string; cause?: unknown }
+export interface ImportTableOptions { from?: string; dropColumns?: string[]; where?: string; map?: (row: Row) => Row | null }
+export interface ImportReport { ok: boolean; tables: { table: string; source: string; rows: number; ms: number; checksum?: string }[]; problems: { table: string; problem: string }[] }
+/** The one-time SQLite → PostgreSQL move with verification (needs better-sqlite3). */
+export function importSqlite(o: { sqlite: string | object; db: Db; tables?: Record<string, ImportTableOptions>; only?: string[]; truncate?: boolean; verify?: boolean; log?: { log(msg: string): void } }): Promise<ImportReport>;
+export const PARSERS: Record<number, (v: string) => unknown>;
+export const ISOLATION: Record<string, string>;
@@ -10,7 +10,10 @@ export interface ActorLimiter {
@@ -10,7 +10,10 @@ export interface ActorLimiter {
stats(): { allowed: number; limited: number; actors: number };
stats(): { allowed: number; limited: number; actors: number };
reset(): void;
reset(): void;
}
}
-export function createActorLimiter(opts?: { limits?: WindowLimits; actor?: ActorOf; maxActors?: number; now?: () => number; onLimited?: (e: LimitedEvent) => void }): ActorLimiter;
+export interface LimitStore { hit(name: string, actor: string, windows: [string, number, number][], nowSec: number): Promise<{ limited: false } | { limited: true; window: string; max: number; retry: number }> }
+export function createActorLimiter(opts?: { limits?: WindowLimits; actor?: ActorOf; maxActors?: number; now?: () => number; onLimited?: (e: LimitedEvent) => void; store?: LimitStore | null; log?: { warn(msg: string): void } }): ActorLimiter;
+/** Shared counters on Valkey: every process and host counts an actor together (null without Valkey). */
+export function createValkeyLimitStore(valkey: import('./valkey').Valkey | null): LimitStore | null;
/** req.principal.sub, else the signed-in person (user:<subject>), else ip:<address>. */
/** req.principal.sub, else the signed-in person (user:<subject>), else ip:<address>. */
export const defaultActor: ActorOf;
export const defaultActor: ActorOf;
export const WINDOWS: { minute: 60; hour: 3600; day: 86400 };
export const WINDOWS: { minute: 60; hour: 3600; day: 86400 };
new file mode 100644
new file mode 100644
@@ -0,0 +1,9 @@
@@ -0,0 +1,9 @@
+/** openvibe-sdk/pubsub — fan-out between processes on Valkey, in-process without it (ADR-035). Not durable. */
+import type { Valkey } from './valkey';
+export interface PubSub {
+ publish(channel: string, message: unknown): Promise<number>;
+ /** Resolves to an unsubscribe function. */
+ subscribe(channel: string, handler: (message: any) => void): Promise<() => Promise<void>>;
+ close(): Promise<void>;
+}
+export function createPubSub(opts?: { valkey?: Valkey | null; log?: { warn(msg: string): void } }): PubSub;
new file mode 100644
new file mode 100644
@@ -0,0 +1,14 @@
@@ -0,0 +1,14 @@
+/** openvibe-sdk/queue — durable at-least-once jobs on Valkey streams, in-process without Valkey (ADR-035). */
+import type { Valkey } from './valkey';
+export interface JobInfo { id: string; attempt: number; queue: string }
+export interface Worker { stats(): { done: number; failed: number; retried: number; dead: number; active: number }; stop(timeoutMs?: number): Promise<void> }
+export interface Queue {
+ name: string;
+ /** Returns the job id, 'delayed', or null when `id` was already queued in the last 24 hours. */
+ add(data: unknown, opts?: { delayMs?: number; attempts?: number; id?: string }): Promise<string | null>;
+ process(handler: (data: any, job: JobInfo) => Promise<unknown>, opts?: { concurrency?: number; visibilityMs?: number; blockMs?: number; backoffMs?: (attempt: number) => number }): Worker;
+ stats(): Promise<{ waiting: number; pending: number; delayed: number; dead: number }>;
+ dead(count?: number): Promise<{ id: string; data: any; attempts: number; error: string; at: string }[]>;
+}
+export function createQueue(opts: { valkey?: Valkey | null; name: string; group?: string; maxLen?: number; log?: object }): Queue;
+export function backoff(attempt: number): number;
new file mode 100644
new file mode 100644
@@ -0,0 +1,13 @@
@@ -0,0 +1,13 @@
+/** openvibe-sdk/valkey — the shared Valkey connection, confined to the service prefix (ADR-035). */
+export interface Valkey {
+ /** The iovalkey client (auto-pipelined). */
+ client: any;
+ prefix: string;
+ /** The full key or channel name inside the prefix. */
+ key(...parts: string[]): string;
+ duplicate(): any;
+ ready(): Promise<{ ok: true; detail: { store: 'valkey' } } | { ok: false; error: string }>;
+ close(): Promise<void>;
+}
+/** null when no URL is configured (the cache, limits, queue and pubsub modules then run in-process). */
+export function createValkey(opts?: { url?: string; prefix?: string; lazyConnect?: boolean; log?: { warn(msg: string): void }; client?: any }): Valkey | null;