openvibers/openvibe.events Commits
The SQLite era is gone (on PostgreSQL since 2026-09-28; no compatibility kept): the one-time migrate-to-postgres script, the *_DB_PATH settings (config, .env.example, systemd unit, test helpers), the unused better-sqlite3 dependency and its lockfile entries, and docs, docstrings and STATUS lines about SQLite or its rollback window. Reviewed independently before shipping. npm test 17/17 test files passed.
Changes
@@@ -24,14 +24,12 @@ OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh events: the runtime role through PgBouncer,--# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).++# the owner on a direct connection (migrations), and Valkey (shared limit counters).# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.DATABASE_URL=DATABASE_DIRECT_URL=VALKEY_URL=VALKEY_PREFIX=--# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.--EVENTS_DB_PATH=./data/events.db# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.EVENTS_RETENTION_DAYS=30# Accepted event ids are remembered this long, so late re-publishes stay duplicates.@@@ -20,7 +20,7 @@ npm run dev # http://127.0.0.1:4300npm test # every test/*.test.js, temp databases, no network needed```--Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store `/var/lib/openvibe-events/events.db`, nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream`, health, and the token-guarded `/api/v1/events`, `/api/v1/subscriptions` and `/api/v1/checkpoints` for developer apps; `/api/v1/deliveries` stays host-local; services on the host call `127.0.0.1:4300`).++Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store PostgreSQL (`ov_events` on the host's data role), nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream`, health, and the token-guarded `/api/v1/events`, `/api/v1/subscriptions` and `/api/v1/checkpoints` for developer apps; `/api/v1/deliveries` stays host-local; services on the host call `127.0.0.1:4300`).`GET /api/health` is liveness. `GET /api/ready` (openvibe-shared/ready) is 200 only when every required check passes — `db` (a real query), `network_jwks` (the Network signing key has loaded; it retries every 30 s while Network boots) and `delivery_worker` (when `EVENTS_WORKER` is on) — and 503 otherwise, with `failed: [...]`. The optional `dlq` check fails once more than `EVENTS_DLQ_DEGRADED_AT` (default 100) deliveries are dead: the service stays ready (200) and reports `status: "degraded"`, `degraded: ["dlq"]`. Each check carries `status`, `required`, `latency_ms`, `checked_at` and, for `dlq`, `detail: { depth, threshold }`; `latest_seq`, `deliveries`, `worker` and `realtime_connections` are still in the body. **Shape change (Track O):** `checks` used to be booleans (`{ db, worker, key }`); they are now objects keyed `db`, `network_jwks`, `delivery_worker`, `dlq`.@@@ -273,9 +273,7 @@ Production deploys with `sudo ovhost deploy events` on the host (strategy `git-cfast-forward `/opt/openvibe.events`, install on a lockfile change, restart, wait for `/api/ready`).The unit is `openvibe-events.service` on `127.0.0.1:4300`, the env file `/etc/openvibe/events.env`. The database is`ov_events` on the host's data role (`sudo /opt/openvibe.host/roles/data/add-service.sh events` writes its settings); the--release migrates it at boot. The one-time move from SQLite is `scripts/migrate-to-postgres.js` (openvibe-sdk--`runSqliteMigration`, with a `--pglite` rehearsal mode), run while the service is stopped; the old--`/var/lib/openvibe-events/events.db` stays read-only for 7 days as the rollback. nginx serves `events.openvibe.network` from++release migrates it at boot. nginx serves `events.openvibe.network` from[deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf). ovhost treats openrealtime connections as a report-only drain, so `--wait-idle` waits for them.Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the@@@ -26,10 +26,8 @@ NoNewPrivileges=truePrivateTmp=trueProtectSystem=strictProtectHome=true--# The code tree stays read-only. The event store is PostgreSQL (ov_events); /var/lib/openvibe-events keeps the--# pre-PostgreSQL SQLite file (read-only) for the rollback window.--StateDirectory=openvibe-events--Environment=EVENTS_DB_PATH=/var/lib/openvibe-events/events.db++# The code tree stays read-only. The event store is PostgreSQL (ov_events on the host's data role); the service++# writes nothing to disk.[Install]WantedBy=multi-user.targetddeleted file mode 100755@@@ -1,26 +0,0 @@--#!/usr/bin/env node--'use strict';--/**-- * The one-time move of Events' SQLite database (EVENTS_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is-- * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).-- *-- * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]-- *-- * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts-- * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its-- * name and columns.-- */--require('dotenv').config();--const { runSqliteMigration } = require('openvibe-sdk/db');--const { load } = require('../server/config');--const { MIGRATIONS } = require('../server/store');----const TABLES = {};----if (require.main === module) {-- const config = load();-- runSqliteMigration({ service: 'events', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })-- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });--}----module.exports = { TABLES };@@@ -68,8 +68,6 @@ function load(env = process.env) {db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },// Valkey (ADR-035): per-actor limit counters shared across processes; without it they count in this process.valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:events:' },-- // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.-- dbPath: env.EVENTS_DB_PATH || './data/events.db',retentionDays: int(env.EVENTS_RETENTION_DAYS, 30),receiptRetentionDays: int(env.EVENTS_RECEIPT_RETENTION_DAYS, 90),pruneIntervalMs: int(env.EVENTS_PRUNE_INTERVAL_MS, 60 * 60 * 1000),@@@ -1,9 +1,9 @@'use strict';/**-- * The durable store (SQLite, better-sqlite3). An event is committed, together with one delivery row-- * per matching subscription, before anything is sent anywhere: "event persists before consumer-- * delivery". Global order is `seq`, handed out from a counter that never goes backwards, even after-- * retention has pruned the newest rows.++ * The durable store (PostgreSQL through openvibe-sdk/db). An event is committed, together with one++ * delivery row per matching subscription, before anything is sent anywhere: "event persists before++ * consumer delivery". Global order is `seq`, handed out from a counter that never goes backwards,++ * even after retention has pruned the newest rows.*/const fs = require('fs');const path = require('path');@@@ -48,7 +48,7 @@ function deliveryCode(outcome) {}/**-- * @param {import('better-sqlite3').Database} db the Events database++ * @param {object} db the Events database (an openvibe-sdk/db handle)* @param {object} [o]* @param {{ now(): number }} [o.clock]* @param {boolean} [o.enabled=true] false (EVENTS_USAGE=off): nothing is counted@@@ -73,11 +73,9 @@ function tmpDir() {/** Boot the service. env overrides go through config.load(); `worker: 'manual'` keeps the loop off. */async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchImpl, deliveryFetch, appPost, dnsLookup } = {}) {-- const dir = tmpDir();const config = load({NODE_ENV: 'test',PORT: '0',-- EVENTS_DB_PATH: path.join(dir, 'events.db'),OV_NETWORK_PUBLIC_KEY: publicKey,EVENTS_WORKER: worker === 'manual' ? 'off' : 'on',// test:pg boots with the Valkey container too (limit counters shared across processes), under a prefix of its own.@@@ -90,8 +88,8 @@ async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchIconst h = await start({ config, db: testdb.db, clock, log: silent, fetchImpl, deliveryFetch, appPost, dnsLookup });const base = `http://127.0.0.1:${h.server.address().port}`;return {-- ...h, base, clock, dir,-- async stop() { await h.close(); await testdb.close(); fs.rmSync(dir, { recursive: true, force: true }); },++ ...h, base, clock,++ async stop() { await h.close(); await testdb.close(); },};}@@@ -1,8 +1,9 @@#!/usr/bin/env node/*** Runs every test in test/ — the files named *.test.js — each in its own process, and fails-- * if any of them fails. They use temp SQLite databases, generated RSA keys and stub subscribers-- * on random ports; none of them needs the network or a running OpenVibe.Network.++ * if any of them fails. They use a PostgreSQL database of their own (PGlite, or the containers under++ * test:pg), generated RSA keys and stub subscribers on random ports; none of them needs the network++ * or a running OpenVibe.Network.** npm test # everything* npm test -- publish sse # only files whose name contains one of the words@@@ -36,7 +36,7 @@ t('Events reads no secret from its environment (so there is none to plant a sent}};walk(path.join(__dirname, '..', 'server'));-- assert.ok(names.has('EVENTS_DB_PATH') && names.has('OV_NETWORK_PUBLIC_KEY'), 'the scan sees the config');++ assert.ok(names.has('DATABASE_URL') && names.has('OV_NETWORK_PUBLIC_KEY'), 'the scan sees the config');const secretish = [...names].filter((n) => /SECRET|PASSWORD|TOKEN|PRIVATE|CREDENTIAL|_KEY$/.test(n) && n !== 'OV_NETWORK_PUBLIC_KEY');assert.deepStrictEqual(secretish, [], 'a new secret-looking variable: give it a sentinel in this test');});@@@ -81,7 +81,7 @@ t('crawl: no GET, refusal or error carries a secret, a token or the key', asyncconst values = (name) => ({ id: [subs.live.id, subs.media.id, subs.appA.id, subs.appB.id, ev.public, ev.internal, ev.appA, 'sub_00000000000000000000000000', 'x', '..%2f'] }[name] || ['x']);const queries = ['topic=*', 'topic=live.*', `topic=app.${ids.keyA}.*`, `topic=app.${ids.keyB}.*,live.*`, 'status=dead', `subscription_id=${subs.appB.id}`, 'after_seq=0&limit=1000','topics=live.*,network.notification.*&last_event_id=0', 'debug=1&include=secret'];-- const paths = crawler.pathsFor(w.routes, values, { method: 'get', queries, extra: ['/nope', '/api/nope', '/.env', '/data/events.db', '/package.json', '/api/v1/subscriptions/../../package.json'] });++ const paths = crawler.pathsFor(w.routes, values, { method: 'get', queries, extra: ['/nope', '/api/nope', '/.env', '/data/pglite/PG_VERSION', '/package.json', '/api/v1/subscriptions/../../package.json'] });const gets = await crawler.crawl(w.base, paths, people, needlesFor);const found = [...gets.found];assert.ok(gets.answered === paths.length * Object.keys(people).length, 'every request answered');@@@ -15,22 +15,19 @@* each project and environment, each with a distinct marker in its payload* `secrets` collects every secret value with a label. Not a test itself (no .test.js).*/--const fs = require('fs');--const path = require('path');const nodeHttp = require('http');const { ids } = require('openvibe-contracts');const { load } = require('../server/config');const { start } = require('../server/index');const apps = require('../server/apps');const { createGuardedPost, isPublicAddress } = require('../server/egress');--const { privateKey, publicKey, serviceToken, appToken, userToken, realtimeTicket, manualClock, tmpDir, request, envelope, subscriber } = require('./helpers');++const { privateKey, publicKey, serviceToken, appToken, userToken, realtimeTicket, manualClock, request, envelope, subscriber } = require('./helpers');const crawler = require('./security-crawl');const S = (name) => `sentinel-not-a-secret-${name}`;const ALL = ['events.app.publish', 'events.app.read', 'events.app.subscribe'];async function buildWorld({ env = {} } = {}) {-- const dir = tmpDir();const logged = [];const keep = (...a) => { logged.push(a.map((x) => (x && x.stack) || String(x)).join(' ')); if (process.env.DEBUG) console.error(...a); };const log = { log: keep, info: keep, warn: keep, error: keep };@@@ -62,7 +59,7 @@ async function buildWorld({ env = {} } = {}) {});const clock = manualClock();-- const config = load({ NODE_ENV: 'test', PORT: '0', EVENTS_DB_PATH: path.join(dir, 'events.db'), OV_NETWORK_PUBLIC_KEY: publicKey, EVENTS_WORKER: 'off', ...env });++ const config = load({ NODE_ENV: 'test', PORT: '0', OV_NETWORK_PUBLIC_KEY: publicKey, EVENTS_WORKER: 'off', ...env });const testdb = await require('./db').testDb(); // a database of its own (PGlite, or the containers under test:pg)const h = await start({ config, db: testdb.db, clock, log, appPost, dnsLookup });const base = `http://127.0.0.1:${h.server.address().port}`;@@@ -132,9 +129,9 @@ async function buildWorld({ env = {} } = {}) {appA: 'marker-app-a-sandbox', appA2: 'marker-app-a-production', appB: 'marker-app-b-sandbox',};return {-- h, base, dir, config, clock, routes, logged, secrets, subs, ev, markers, tok, shownOnce, call, fp, appSeen, dnsTable, dnsLookup,++ h, base, config, clock, routes, logged, secrets, subs, ev, markers, tok, shownOnce, call, fp, appSeen, dnsTable, dnsLookup,ids: { PA, PB, A, A2, B, x, y, keyA, keyB }, privateKey, realtimeTicket,-- stop: async () => { await fp.close(); await new Promise((r) => receiver.close(r)); await h.close(); await testdb.close(); fs.rmSync(dir, { recursive: true, force: true }); },++ stop: async () => { await fp.close(); await new Promise((r) => receiver.close(r)); await h.close(); await testdb.close(); },};}
@@ -24,14 +24,12 @@ OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000
@@ -24,14 +24,12 @@ OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000
#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network
#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network
# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh events: the runtime role through PgBouncer,
# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh events: the runtime role through PgBouncer,
-# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).
+# the owner on a direct connection (migrations), and Valkey (shared limit counters).
# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
DATABASE_URL=
DATABASE_URL=
DATABASE_DIRECT_URL=
DATABASE_DIRECT_URL=
VALKEY_URL=
VALKEY_URL=
VALKEY_PREFIX=
VALKEY_PREFIX=
-# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.
-EVENTS_DB_PATH=./data/events.db
# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.
# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.
EVENTS_RETENTION_DAYS=30
EVENTS_RETENTION_DAYS=30
# Accepted event ids are remembered this long, so late re-publishes stay duplicates.
# Accepted event ids are remembered this long, so late re-publishes stay duplicates.
@@ -20,7 +20,7 @@ npm run dev # http://127.0.0.1:4300
@@ -20,7 +20,7 @@ npm run dev # http://127.0.0.1:4300
npm test # every test/*.test.js, temp databases, no network needed
npm test # every test/*.test.js, temp databases, no network needed
```
```
-Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store `/var/lib/openvibe-events/events.db`, nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream`, health, and the token-guarded `/api/v1/events`, `/api/v1/subscriptions` and `/api/v1/checkpoints` for developer apps; `/api/v1/deliveries` stays host-local; services on the host call `127.0.0.1:4300`).
+Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store PostgreSQL (`ov_events` on the host's data role), nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream`, health, and the token-guarded `/api/v1/events`, `/api/v1/subscriptions` and `/api/v1/checkpoints` for developer apps; `/api/v1/deliveries` stays host-local; services on the host call `127.0.0.1:4300`).
`GET /api/health` is liveness. `GET /api/ready` (openvibe-shared/ready) is 200 only when every required check passes — `db` (a real query), `network_jwks` (the Network signing key has loaded; it retries every 30 s while Network boots) and `delivery_worker` (when `EVENTS_WORKER` is on) — and 503 otherwise, with `failed: [...]`. The optional `dlq` check fails once more than `EVENTS_DLQ_DEGRADED_AT` (default 100) deliveries are dead: the service stays ready (200) and reports `status: "degraded"`, `degraded: ["dlq"]`. Each check carries `status`, `required`, `latency_ms`, `checked_at` and, for `dlq`, `detail: { depth, threshold }`; `latest_seq`, `deliveries`, `worker` and `realtime_connections` are still in the body. **Shape change (Track O):** `checks` used to be booleans (`{ db, worker, key }`); they are now objects keyed `db`, `network_jwks`, `delivery_worker`, `dlq`.
`GET /api/health` is liveness. `GET /api/ready` (openvibe-shared/ready) is 200 only when every required check passes — `db` (a real query), `network_jwks` (the Network signing key has loaded; it retries every 30 s while Network boots) and `delivery_worker` (when `EVENTS_WORKER` is on) — and 503 otherwise, with `failed: [...]`. The optional `dlq` check fails once more than `EVENTS_DLQ_DEGRADED_AT` (default 100) deliveries are dead: the service stays ready (200) and reports `status: "degraded"`, `degraded: ["dlq"]`. Each check carries `status`, `required`, `latency_ms`, `checked_at` and, for `dlq`, `detail: { depth, threshold }`; `latest_seq`, `deliveries`, `worker` and `realtime_connections` are still in the body. **Shape change (Track O):** `checks` used to be booleans (`{ db, worker, key }`); they are now objects keyed `db`, `network_jwks`, `delivery_worker`, `dlq`.
@@ -273,9 +273,7 @@ Production deploys with `sudo ovhost deploy events` on the host (strategy `git-c
@@ -273,9 +273,7 @@ Production deploys with `sudo ovhost deploy events` on the host (strategy `git-c
fast-forward `/opt/openvibe.events`, install on a lockfile change, restart, wait for `/api/ready`).
fast-forward `/opt/openvibe.events`, install on a lockfile change, restart, wait for `/api/ready`).
The unit is `openvibe-events.service` on `127.0.0.1:4300`, the env file `/etc/openvibe/events.env`. The database is
The unit is `openvibe-events.service` on `127.0.0.1:4300`, the env file `/etc/openvibe/events.env`. The database is
`ov_events` on the host's data role (`sudo /opt/openvibe.host/roles/data/add-service.sh events` writes its settings); the
`ov_events` on the host's data role (`sudo /opt/openvibe.host/roles/data/add-service.sh events` writes its settings); the
-release migrates it at boot. The one-time move from SQLite is `scripts/migrate-to-postgres.js` (openvibe-sdk
-`runSqliteMigration`, with a `--pglite` rehearsal mode), run while the service is stopped; the old
-`/var/lib/openvibe-events/events.db` stays read-only for 7 days as the rollback. nginx serves `events.openvibe.network` from
+release migrates it at boot. nginx serves `events.openvibe.network` from
[deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf). ovhost treats open
[deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf). ovhost treats open
realtime connections as a report-only drain, so `--wait-idle` waits for them.
realtime connections as a report-only drain, so `--wait-idle` waits for them.
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
@@ -26,10 +26,8 @@ NoNewPrivileges=true
@@ -26,10 +26,8 @@ NoNewPrivileges=true
PrivateTmp=true
PrivateTmp=true
ProtectSystem=strict
ProtectSystem=strict
ProtectHome=true
ProtectHome=true
-# The code tree stays read-only. The event store is PostgreSQL (ov_events); /var/lib/openvibe-events keeps the
-# pre-PostgreSQL SQLite file (read-only) for the rollback window.
-StateDirectory=openvibe-events
-Environment=EVENTS_DB_PATH=/var/lib/openvibe-events/events.db
+# The code tree stays read-only. The event store is PostgreSQL (ov_events on the host's data role); the service
+# writes nothing to disk.
[Install]
[Install]
WantedBy=multi-user.target
WantedBy=multi-user.target
deleted file mode 100755
deleted file mode 100755
@@ -1,26 +0,0 @@
@@ -1,26 +0,0 @@
-#!/usr/bin/env node
-'use strict';
-/**
- * The one-time move of Events' SQLite database (EVENTS_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is
- * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).
- *
- * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
- *
- * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts
- * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its
- * name and columns.
- */
-require('dotenv').config();
-const { runSqliteMigration } = require('openvibe-sdk/db');
-const { load } = require('../server/config');
-const { MIGRATIONS } = require('../server/store');
-
-const TABLES = {};
-
-if (require.main === module) {
- const config = load();
- runSqliteMigration({ service: 'events', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })
- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
-}
-
-module.exports = { TABLES };
@@ -68,8 +68,6 @@ function load(env = process.env) {
@@ -68,8 +68,6 @@ function load(env = process.env) {
db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },
db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },
// Valkey (ADR-035): per-actor limit counters shared across processes; without it they count in this process.
// Valkey (ADR-035): per-actor limit counters shared across processes; without it they count in this process.
valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:events:' },
valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:events:' },
- // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.
- dbPath: env.EVENTS_DB_PATH || './data/events.db',
retentionDays: int(env.EVENTS_RETENTION_DAYS, 30),
retentionDays: int(env.EVENTS_RETENTION_DAYS, 30),
receiptRetentionDays: int(env.EVENTS_RECEIPT_RETENTION_DAYS, 90),
receiptRetentionDays: int(env.EVENTS_RECEIPT_RETENTION_DAYS, 90),
pruneIntervalMs: int(env.EVENTS_PRUNE_INTERVAL_MS, 60 * 60 * 1000),
pruneIntervalMs: int(env.EVENTS_PRUNE_INTERVAL_MS, 60 * 60 * 1000),
@@ -1,9 +1,9 @@
@@ -1,9 +1,9 @@
'use strict';
'use strict';
/**
/**
- * The durable store (SQLite, better-sqlite3). An event is committed, together with one delivery row
- * per matching subscription, before anything is sent anywhere: "event persists before consumer
- * delivery". Global order is `seq`, handed out from a counter that never goes backwards, even after
- * retention has pruned the newest rows.
+ * The durable store (PostgreSQL through openvibe-sdk/db). An event is committed, together with one
+ * delivery row per matching subscription, before anything is sent anywhere: "event persists before
+ * consumer delivery". Global order is `seq`, handed out from a counter that never goes backwards,
+ * even after retention has pruned the newest rows.
*/
*/
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
@@ -48,7 +48,7 @@ function deliveryCode(outcome) {
@@ -48,7 +48,7 @@ function deliveryCode(outcome) {
}
}
/**
/**
- * @param {import('better-sqlite3').Database} db the Events database
+ * @param {object} db the Events database (an openvibe-sdk/db handle)
* @param {object} [o]
* @param {object} [o]
* @param {{ now(): number }} [o.clock]
* @param {{ now(): number }} [o.clock]
* @param {boolean} [o.enabled=true] false (EVENTS_USAGE=off): nothing is counted
* @param {boolean} [o.enabled=true] false (EVENTS_USAGE=off): nothing is counted
@@ -73,11 +73,9 @@ function tmpDir() {
@@ -73,11 +73,9 @@ function tmpDir() {
/** Boot the service. env overrides go through config.load(); `worker: 'manual'` keeps the loop off. */
/** Boot the service. env overrides go through config.load(); `worker: 'manual'` keeps the loop off. */
async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchImpl, deliveryFetch, appPost, dnsLookup } = {}) {
async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchImpl, deliveryFetch, appPost, dnsLookup } = {}) {
- const dir = tmpDir();
const config = load({
const config = load({
NODE_ENV: 'test',
NODE_ENV: 'test',
PORT: '0',
PORT: '0',
- EVENTS_DB_PATH: path.join(dir, 'events.db'),
OV_NETWORK_PUBLIC_KEY: publicKey,
OV_NETWORK_PUBLIC_KEY: publicKey,
EVENTS_WORKER: worker === 'manual' ? 'off' : 'on',
EVENTS_WORKER: worker === 'manual' ? 'off' : 'on',
// test:pg boots with the Valkey container too (limit counters shared across processes), under a prefix of its own.
// test:pg boots with the Valkey container too (limit counters shared across processes), under a prefix of its own.
@@ -90,8 +88,8 @@ async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchI
@@ -90,8 +88,8 @@ async function boot({ env = {}, clock = manualClock(), worker = 'manual', fetchI
const h = await start({ config, db: testdb.db, clock, log: silent, fetchImpl, deliveryFetch, appPost, dnsLookup });
const h = await start({ config, db: testdb.db, clock, log: silent, fetchImpl, deliveryFetch, appPost, dnsLookup });
const base = `http://127.0.0.1:${h.server.address().port}`;
const base = `http://127.0.0.1:${h.server.address().port}`;
return {
return {
- ...h, base, clock, dir,
- async stop() { await h.close(); await testdb.close(); fs.rmSync(dir, { recursive: true, force: true }); },
+ ...h, base, clock,
+ async stop() { await h.close(); await testdb.close(); },
};
};
}
}
@@ -1,8 +1,9 @@
@@ -1,8 +1,9 @@
#!/usr/bin/env node
#!/usr/bin/env node
/**
/**
* Runs every test in test/ — the files named *.test.js — each in its own process, and fails
* Runs every test in test/ — the files named *.test.js — each in its own process, and fails
- * if any of them fails. They use temp SQLite databases, generated RSA keys and stub subscribers
- * on random ports; none of them needs the network or a running OpenVibe.Network.
+ * if any of them fails. They use a PostgreSQL database of their own (PGlite, or the containers under
+ * test:pg), generated RSA keys and stub subscribers on random ports; none of them needs the network
+ * or a running OpenVibe.Network.
*
*
* npm test # everything
* npm test # everything
* npm test -- publish sse # only files whose name contains one of the words
* npm test -- publish sse # only files whose name contains one of the words
@@ -36,7 +36,7 @@ t('Events reads no secret from its environment (so there is none to plant a sent
@@ -36,7 +36,7 @@ t('Events reads no secret from its environment (so there is none to plant a sent
}
}
};
};
walk(path.join(__dirname, '..', 'server'));
walk(path.join(__dirname, '..', 'server'));
- assert.ok(names.has('EVENTS_DB_PATH') && names.has('OV_NETWORK_PUBLIC_KEY'), 'the scan sees the config');
+ assert.ok(names.has('DATABASE_URL') && names.has('OV_NETWORK_PUBLIC_KEY'), 'the scan sees the config');
const secretish = [...names].filter((n) => /SECRET|PASSWORD|TOKEN|PRIVATE|CREDENTIAL|_KEY$/.test(n) && n !== 'OV_NETWORK_PUBLIC_KEY');
const secretish = [...names].filter((n) => /SECRET|PASSWORD|TOKEN|PRIVATE|CREDENTIAL|_KEY$/.test(n) && n !== 'OV_NETWORK_PUBLIC_KEY');
assert.deepStrictEqual(secretish, [], 'a new secret-looking variable: give it a sentinel in this test');
assert.deepStrictEqual(secretish, [], 'a new secret-looking variable: give it a sentinel in this test');
});
});
@@ -81,7 +81,7 @@ t('crawl: no GET, refusal or error carries a secret, a token or the key', async
@@ -81,7 +81,7 @@ t('crawl: no GET, refusal or error carries a secret, a token or the key', async
const values = (name) => ({ id: [subs.live.id, subs.media.id, subs.appA.id, subs.appB.id, ev.public, ev.internal, ev.appA, 'sub_00000000000000000000000000', 'x', '..%2f'] }[name] || ['x']);
const values = (name) => ({ id: [subs.live.id, subs.media.id, subs.appA.id, subs.appB.id, ev.public, ev.internal, ev.appA, 'sub_00000000000000000000000000', 'x', '..%2f'] }[name] || ['x']);
const queries = ['topic=*', 'topic=live.*', `topic=app.${ids.keyA}.*`, `topic=app.${ids.keyB}.*,live.*`, 'status=dead', `subscription_id=${subs.appB.id}`, 'after_seq=0&limit=1000',
const queries = ['topic=*', 'topic=live.*', `topic=app.${ids.keyA}.*`, `topic=app.${ids.keyB}.*,live.*`, 'status=dead', `subscription_id=${subs.appB.id}`, 'after_seq=0&limit=1000',
'topics=live.*,network.notification.*&last_event_id=0', 'debug=1&include=secret'];
'topics=live.*,network.notification.*&last_event_id=0', 'debug=1&include=secret'];
- const paths = crawler.pathsFor(w.routes, values, { method: 'get', queries, extra: ['/nope', '/api/nope', '/.env', '/data/events.db', '/package.json', '/api/v1/subscriptions/../../package.json'] });
+ const paths = crawler.pathsFor(w.routes, values, { method: 'get', queries, extra: ['/nope', '/api/nope', '/.env', '/data/pglite/PG_VERSION', '/package.json', '/api/v1/subscriptions/../../package.json'] });
const gets = await crawler.crawl(w.base, paths, people, needlesFor);
const gets = await crawler.crawl(w.base, paths, people, needlesFor);
const found = [...gets.found];
const found = [...gets.found];
assert.ok(gets.answered === paths.length * Object.keys(people).length, 'every request answered');
assert.ok(gets.answered === paths.length * Object.keys(people).length, 'every request answered');
@@ -15,22 +15,19 @@
@@ -15,22 +15,19 @@
* each project and environment, each with a distinct marker in its payload
* each project and environment, each with a distinct marker in its payload
* `secrets` collects every secret value with a label. Not a test itself (no .test.js).
* `secrets` collects every secret value with a label. Not a test itself (no .test.js).
*/
*/
-const fs = require('fs');
-const path = require('path');
const nodeHttp = require('http');
const nodeHttp = require('http');
const { ids } = require('openvibe-contracts');
const { ids } = require('openvibe-contracts');
const { load } = require('../server/config');
const { load } = require('../server/config');
const { start } = require('../server/index');
const { start } = require('../server/index');
const apps = require('../server/apps');
const apps = require('../server/apps');
const { createGuardedPost, isPublicAddress } = require('../server/egress');
const { createGuardedPost, isPublicAddress } = require('../server/egress');
-const { privateKey, publicKey, serviceToken, appToken, userToken, realtimeTicket, manualClock, tmpDir, request, envelope, subscriber } = require('./helpers');
+const { privateKey, publicKey, serviceToken, appToken, userToken, realtimeTicket, manualClock, request, envelope, subscriber } = require('./helpers');
const crawler = require('./security-crawl');
const crawler = require('./security-crawl');
const S = (name) => `sentinel-not-a-secret-${name}`;
const S = (name) => `sentinel-not-a-secret-${name}`;
const ALL = ['events.app.publish', 'events.app.read', 'events.app.subscribe'];
const ALL = ['events.app.publish', 'events.app.read', 'events.app.subscribe'];
async function buildWorld({ env = {} } = {}) {
async function buildWorld({ env = {} } = {}) {
- const dir = tmpDir();
const logged = [];
const logged = [];
const keep = (...a) => { logged.push(a.map((x) => (x && x.stack) || String(x)).join(' ')); if (process.env.DEBUG) console.error(...a); };
const keep = (...a) => { logged.push(a.map((x) => (x && x.stack) || String(x)).join(' ')); if (process.env.DEBUG) console.error(...a); };
const log = { log: keep, info: keep, warn: keep, error: keep };
const log = { log: keep, info: keep, warn: keep, error: keep };
@@ -62,7 +59,7 @@ async function buildWorld({ env = {} } = {}) {
@@ -62,7 +59,7 @@ async function buildWorld({ env = {} } = {}) {
});
});
const clock = manualClock();
const clock = manualClock();
- const config = load({ NODE_ENV: 'test', PORT: '0', EVENTS_DB_PATH: path.join(dir, 'events.db'), OV_NETWORK_PUBLIC_KEY: publicKey, EVENTS_WORKER: 'off', ...env });
+ const config = load({ NODE_ENV: 'test', PORT: '0', OV_NETWORK_PUBLIC_KEY: publicKey, EVENTS_WORKER: 'off', ...env });
const testdb = await require('./db').testDb(); // a database of its own (PGlite, or the containers under test:pg)
const testdb = await require('./db').testDb(); // a database of its own (PGlite, or the containers under test:pg)
const h = await start({ config, db: testdb.db, clock, log, appPost, dnsLookup });
const h = await start({ config, db: testdb.db, clock, log, appPost, dnsLookup });
const base = `http://127.0.0.1:${h.server.address().port}`;
const base = `http://127.0.0.1:${h.server.address().port}`;
@@ -132,9 +129,9 @@ async function buildWorld({ env = {} } = {}) {
@@ -132,9 +129,9 @@ async function buildWorld({ env = {} } = {}) {
appA: 'marker-app-a-sandbox', appA2: 'marker-app-a-production', appB: 'marker-app-b-sandbox',
appA: 'marker-app-a-sandbox', appA2: 'marker-app-a-production', appB: 'marker-app-b-sandbox',
};
};
return {
return {
- h, base, dir, config, clock, routes, logged, secrets, subs, ev, markers, tok, shownOnce, call, fp, appSeen, dnsTable, dnsLookup,
+ h, base, config, clock, routes, logged, secrets, subs, ev, markers, tok, shownOnce, call, fp, appSeen, dnsTable, dnsLookup,
ids: { PA, PB, A, A2, B, x, y, keyA, keyB }, privateKey, realtimeTicket,
ids: { PA, PB, A, A2, B, x, y, keyA, keyB }, privateKey, realtimeTicket,
- stop: async () => { await fp.close(); await new Promise((r) => receiver.close(r)); await h.close(); await testdb.close(); fs.rmSync(dir, { recursive: true, force: true }); },
+ stop: async () => { await fp.close(); await new Promise((r) => receiver.close(r)); await h.close(); await testdb.close(); },
};
};
}
}