openvibers/openrestream Commits
The SQLite era is gone (on PostgreSQL since 2026-09-28; no compatibility kept): the one-time migrate-to-postgres script, the *_DB_PATH settings (config, .env.example, systemd unit, test helpers), the unused better-sqlite3 dependency and its lockfile entries, and docs, docstrings and STATUS lines about SQLite or its rollback window. Reviewed independently before shipping. npm test 15/18 test files passed, 3 skipped (api-restart.test.js: api-restart: skipped (OpenRe proc.
Changes
@@@ -2,7 +2,7 @@# OpenRe.Stream — Environment Configuration# Copy to .env for development. In production this is /etc/openvibe/openre.env (0600), read by# every unit (openre-api, openre-session-coordinator, openre-rtmp-ingest@, openre-restream-worker@);--# the units set NODE_ENV, PORT, OPENRE_RELEASE and OPENRE_DB_PATH themselves.++# the units set NODE_ENV, PORT and OPENRE_RELEASE themselves.# Names only here: never commit a value.# ═══════════════════════════════════════════════════════════════@@@ -29,15 +29,13 @@ OV_OAUTH_CLIENT_SECRET=# A browser's Network JWT is accepted when its aud contains one of these.#OPENRE_USER_AUDIENCES=openvibe.openre,openvibe.network--# ── Store (SQLite, shared by every OpenRe process; WAL) ───────++# ── Store (PostgreSQL, shared by every OpenRe process) ────────# PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/ at boot;# OpenVibe.Host `roles/data/add-service.sh openre` writes both. Every OpenRe process shares the database. Without# DATABASE_URL, development runs on an embedded PGlite database in data/pglite (OPENRE_PGLITE_DIR): one process only.DATABASE_URL=DATABASE_DIRECT_URL=VALKEY_URL=--# The SQLite file the one-time move reads (scripts/migrate-to-postgres.js).--OPENRE_DB_PATH=./data/openre.db# ── Secrets ───────────────────────────────────────────────────# 32 bytes (64 hex chars or base64): encrypts restream destination keys and SRT passphrases at rest.@@@ -65,7 +65,7 @@ openre-api (4500) openre-session-coordinator openre-rtmp-inges- **Generations.** Each worker process registers as generation N+1 of its kind. When a newer generation is `ready`, the coordinator marks older ones `draining` (deadline `OPENRE_DRAIN_MAX_MS`, default 24 h). A draining RTMP worker closes its public listener (the kernel sends new encoder connections to the new generation through the shared port) and keeps its publishers; a draining restream worker keeps its outputs. Idle → the process exits 0 by itself. At the deadline, remaining sessions are ended (encoders reconnect to the newest generation) and remaining outputs are handed over.- **Leases.** Every heartbeat (2 s) renews the worker and the lease of every session it owns in one transaction. A worker silent for `OPENRE_WORKER_LEASE_MS` (15 s) is `lost`: its sessions fail (`openre.session.failed`), its outputs go back to `pending` for another worker. A lost worker that wakes up drops its transports and exits.- **Isolation.** An output only ever changes its own row. A dead destination fails (`openre.output.failed`, destination cooldown 15 min → 24 h) and the session stays live (`test/rtmp-e2e.test.js`, `test/sessions.test.js`, `test/coordinator.test.js`).--- **Store.** PostgreSQL 18 through PgBouncer (`ov_openre` on the host's data role, ADR-035; schema in [migrations/](migrations/)), shared by every OpenRe process. Every transaction is SERIALIZABLE (`server/store/index.js`): an ingest worker admitting a publish and the coordinator failing a lost worker's session conflict, and PostgreSQL makes one of them retry, as SQLite's one writer ordered them. Admission is async, so the RTMP worker admits a publish before node-media-server's own publish handling runs. Valkey holds the API's per-actor limit counters. The one-time move is `scripts/migrate-to-postgres.js` (run by OpenVibe.Host `roles/data/switch-service.sh` with every OpenRe unit stopped); `/var/lib/openre/openre.db` stays read-only for 7 days as the rollback.++- **Store.** PostgreSQL 18 through PgBouncer (`ov_openre` on the host's data role, ADR-035; schema in [migrations/](migrations/)), shared by every OpenRe process. Every transaction is SERIALIZABLE (`server/store/index.js`): an ingest worker admitting a publish and the coordinator failing a lost worker's session conflict, and PostgreSQL makes one of them retry. Admission is async, so the RTMP worker admits a publish before node-media-server's own publish handling runs. Valkey holds the API's per-actor limit counters.## Running it@@@ -168,7 +168,7 @@ The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_## Deploying--Layout: `/opt/openre.stream/repo` (the git clone releases are made from), `/opt/openre.stream/releases/<sha12>` (full checkouts with their own `node_modules`, owned by ubuntu), `current` → the release the API and coordinator run, env `/etc/openvibe/openre.env` (see [.env.example](.env.example)), store `ov_openre` on PostgreSQL (`sudo /opt/openvibe.host/roles/data/add-service.sh openre` writes its settings; the release migrates it at boot; the old SQLite file is `/var/lib/openre/openre.db`), units in [deploy/systemd/](deploy/systemd/), nginx [deploy/nginx/openre.stream.conf](deploy/nginx/openre.stream.conf), script [deploy/scripts/deploy.sh](deploy/scripts/deploy.sh). The release and API phases run through `ovhost deploy openre` (OpenVibe.Host, strategy `release-layout`; roadmap WS-N task 11); the script is a thin wrapper that keeps the old subcommands:++Layout: `/opt/openre.stream/repo` (the git clone releases are made from), `/opt/openre.stream/releases/<sha12>` (full checkouts with their own `node_modules`, owned by ubuntu), `current` → the release the API and coordinator run, env `/etc/openvibe/openre.env` (see [.env.example](.env.example)), store `ov_openre` on PostgreSQL (`sudo /opt/openvibe.host/roles/data/add-service.sh openre` writes its settings; the release migrates it at boot), units in [deploy/systemd/](deploy/systemd/), nginx [deploy/nginx/openre.stream.conf](deploy/nginx/openre.stream.conf), script [deploy/scripts/deploy.sh](deploy/scripts/deploy.sh). The release and API phases run through `ovhost deploy openre` (OpenVibe.Host, strategy `release-layout`; roadmap WS-N task 11); the script is a thin wrapper that keeps the old subcommands:```bashsudo deploy/scripts/deploy.sh # ovhost deploy openre: release + api in one@@@ -15,7 +15,7 @@## Layout: /opt/openre.stream/releases/<sha>/ (full checkout + node_modules), `current` symlink# for the API and coordinator, one openre-rtmp-ingest@<sha> / openre-restream-worker@<sha>--# instance per worker generation. Env: /etc/openvibe/openre.env. Store: /var/lib/openre/openre.db.++# instance per worker generation. Env: /etc/openvibe/openre.env. Store: ov_openre on PostgreSQL (ADR-035).# ═══════════════════════════════════════════════════════════════set -euo pipefail@@@ -13,8 +13,6 @@ WorkingDirectory=/opt/openre.stream/currentEnvironmentFile=/etc/openvibe/openre.envEnvironment=NODE_ENV=productionEnvironment=PORT=4500--# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).--Environment=OPENRE_DB_PATH=/var/lib/openre/openre.dbEnvironment=PATH=/usr/local/bin:/usr/bin:/binExecStart=/usr/bin/env node server/index.jsRestart=always@@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%iEnvironmentFile=/etc/openvibe/openre.envEnvironment=NODE_ENV=productionEnvironment=OPENRE_RELEASE=%i--# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).--Environment=OPENRE_DB_PATH=/var/lib/openre/openre.dbEnvironment=PATH=/usr/local/bin:/usr/bin:/binExecStart=/usr/bin/env node workers/restream-worker.js# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.@@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%iEnvironmentFile=/etc/openvibe/openre.envEnvironment=NODE_ENV=productionEnvironment=OPENRE_RELEASE=%i--# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).--Environment=OPENRE_DB_PATH=/var/lib/openre/openre.dbEnvironment=PATH=/usr/local/bin:/usr/bin:/binExecStart=/usr/bin/env node workers/rtmp-ingest.js# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.@@@ -11,8 +11,6 @@ Group=ubuntuWorkingDirectory=/opt/openre.stream/currentEnvironmentFile=/etc/openvibe/openre.envEnvironment=NODE_ENV=production--# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).--Environment=OPENRE_DB_PATH=/var/lib/openre/openre.dbEnvironment=PATH=/usr/local/bin:/usr/bin:/binExecStart=/usr/bin/env node workers/coordinator.jsRestart=always@@@ -46,8 +46,8 @@ Everything below runs on the host. The scripts ship with the release (`scripts/`|---|---|---|| `cutover-preflight.js` | read-only checks: `release service env bind dns port db live-env events [slot]` (`--only`, `--skip`, `--slot <id>`, `--json`, `--strict`, `--expect-release <sha>`, `--probe-url`, `--host-ip`) | nothing || `subscribe-live-events.js` | Live's Events subscription `openre.session.*` → Live, **as Live** (reads `/etc/openvibe/live.env`); `--dry-run`, `--disable`, `--enable` | one subscription in Events |--| `migrate-from-live.js` | imports slots and destinations from a Live **snapshot** (dry run unless `--apply`) | `openre.db` |--| `set-definition-state.js` | disables or re-enables one slot's OpenRe definition, **as Live** (per-slot rollback) | `openre.db` via the API |++| `migrate-from-live.js` | imports slots and destinations from a Live **snapshot** (dry run unless `--apply`) | the PostgreSQL store (`ov_openre`) |++| `set-definition-state.js` | disables or re-enables one slot's OpenRe definition, **as Live** (per-slot rollback) | the PostgreSQL store (`ov_openre`) via the API |Shell helpers for the session (paste once per SSH session):@@@ -56,8 +56,7 @@ SHA=<the 12-character release id printed by step A2># read-only preflight from that releasepf() { sudo node /opt/openre.stream/releases/$SHA/scripts/cutover-preflight.js --expect-release "$SHA" "$@"; }# run a command in the current release with OpenRe's production env, AS THE SERVICE USER (ubuntu).--# Never run a script that opens openre.db as root: root-owned -wal/-shm files lock the service out.--openre_as_service() { sudo bash -c "set -a; . /etc/openvibe/openre.env; set +a; export NODE_ENV=production OPENRE_DB_PATH=/var/lib/openre/openre.db OPENRE_ENV_FILE=/nonexistent; cd /opt/openre.stream/current && sudo -E -u ubuntu $1"; }++openre_as_service() { sudo bash -c "set -a; . /etc/openvibe/openre.env; set +a; export NODE_ENV=production OPENRE_ENV_FILE=/nonexistent; cd /opt/openre.stream/current && sudo -E -u ubuntu $1"; }DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh```@@@ -77,7 +76,7 @@ from this commit onward is on GitHub yet. Undo: nothing to undo.```bashsudo /opt/openre.stream/current/deploy/scripts/deploy.sh release origin/main # prints the new <sha>SHA=<that sha>; DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh # and re-paste pf()--pf --only db # openre.db is sound before anything restarts++pf --only db # the PostgreSQL store is sound before anything restartssudo $DEPLOY api $SHA # restarts openre-api + openre-session-coordinator ONLYcurl -s http://127.0.0.1:4500/api/ready # "status":"ready"pf --only release,env,db # current = $SHA, it contains 6dc78a5@@@ -219,11 +218,11 @@ refuses that user's personal RTMP key (`refusesLiveIngest`).`flv.internal_url`).- `await api('/admin/openre/status')` lists the session under `live_sessions`.- Sessions and outputs:-- `sudo sqlite3 -readonly /var/lib/openre/openre.db "SELECT id, state, worker_generation FROM ingest_sessions ORDER BY created_at DESC LIMIT 1; SELECT o.state, o.last_error FROM outputs o ORDER BY o.created_at DESC LIMIT 3; SELECT state, media_vod_id, last_error FROM recordings ORDER BY created_at DESC LIMIT 1;"`++ `sudo -u ubuntu bash -c "set -a; . /etc/openvibe/openre.env; set +a; psql \"\$DATABASE_URL\" -c \"SELECT id, state, worker_generation FROM ingest_sessions ORDER BY created_at DESC LIMIT 1; SELECT o.state, o.last_error FROM outputs o ORDER BY o.created_at DESC LIMIT 3; SELECT state, media_vod_id, last_error FROM recordings ORDER BY created_at DESC LIMIT 1;\""`should show a `live` session, a `live` output and a `recording` recording.- The destination (YouTube) shows the stream.- Deliveries:-- `sudo sqlite3 -readonly /var/lib/openvibe-events/events.db "SELECT d.status, count(*) FROM deliveries d JOIN subscriptions s ON s.id = d.subscription_id WHERE s.consumer = 'live' AND s.topic_pattern = 'openre.session.*' GROUP BY 1"`++ `sudo -u ubuntu bash -c "set -a; . /etc/openvibe/events.env; set +a; psql \"\$DATABASE_URL\" -c \"SELECT d.status, count(*) FROM deliveries d JOIN subscriptions s ON s.id = d.subscription_id WHERE s.consumer = 'live' AND s.topic_pattern = 'openre.session.*' GROUP BY 1\""`should all be `delivered`.5. **During the broadcast (AGENT):**- `sudo $DEPLOY api $SHA`: the stream, the restream and the recording do not drop. OpenRe@@@ -11,8 +11,8 @@* Take the snapshot with `sqlite3 /opt/openvibe.live/data/live.db ".backup /tmp/live-snapshot.db"`* (Live's DB_PATH in production; never point this at the live file). Uses the OpenRe environment* (.env or OPENRE_ENV_FILE) for its own database and OPENRE_SECRETS_KEY; in production set-- * OPENRE_ENV_FILE=/etc/openvibe/openre.env, OPENRE_DB_PATH=/var/lib/openre/openre.db and-- * NODE_ENV=production (the units set the last two themselves) — docs/cutover.md has the command. Prints no secret: old keys are not imported and the new++ * OPENRE_ENV_FILE=/etc/openvibe/openre.env and NODE_ENV=production (a unit sets NODE_ENV itself)++ * — docs/cutover.md has the command. Prints no secret: old keys are not imported and the new* keys are never shown (the broadcaster rotates to get one). Safe to re-run.*/const fs = require('fs');ddeleted file mode 100755@@@ -1,27 +0,0 @@--#!/usr/bin/env node--'use strict';--/**-- * The one-time move of OpenRe's SQLite database (OPENRE_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is-- * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).-- *-- * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]-- *-- * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts-- * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its-- * name and columns; the SDK outbox's rows move into its PostgreSQL outbox (the same columns). Run with every OpenRe-- * process stopped (OpenVibe.Host roles/data/switch-service.sh with SWITCH_UNITS).-- */--require('dotenv').config({ path: process.env.OPENRE_ENV_FILE || require('path').join(process.cwd(), '.env') });--const { runSqliteMigration } = require('openvibe-sdk/db');--const { load } = require('../server/config');--const { MIGRATIONS } = require('../server/db');----const TABLES = {};----if (require.main === module) {-- const config = load();-- runSqliteMigration({ service: 'openre', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })-- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });--}----module.exports = { TABLES };@@@ -76,7 +76,7 @@ function createApp({ rt, auth, keys, log = console, fetchImpl }) {service: 'openre',release: release.release,checks: [-- { name: 'db', required: true, description: 'SQLite answers a query', check: async () => (await db.prepare('SELECT 1 AS ok').get()).ok === 1 },++ { name: 'db', required: true, description: 'PostgreSQL answers a query', check: async () => (await db.prepare('SELECT 1 AS ok').get()).ok === 1 },{ name: 'network_key', required: true, description: 'OpenVibe.Network RS256 key (sign-in and service tokens)', check: () => keys.loaded() || 'Network public key not loaded yet' },{name: 'workers', required: false, description: 'a ready rtmp-ingest and restream worker',@@@ -5,8 +5,8 @@** load(env) is pure so tests (and every worker process) can build a config without touching* process.env. All processes of the service (openre-api, openre-session-coordinator and the-- * transport workers) read the same file and share one SQLite database (ADR-007: one service,-- * one store; the processes are the reason this service is the first candidate for PostgreSQL).++ * transport workers) read the same file and share one PostgreSQL database (ADR-035: one++ * service, one store).*/const int = (v, d) => {@@@ -58,11 +58,9 @@ function load(env = process.env) {// ── Store ───────────────────────────────────────────────// PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/.-- // Without DATABASE_URL, development and tests use an embedded PGlite database (OPENRE_PGLITE_DIR). dbPath: the-- // SQLite file the one-time move reads (scripts/migrate-to-postgres.js).++ // Without DATABASE_URL, development and tests use an embedded PGlite database (OPENRE_PGLITE_DIR).db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '', pgliteDir: env.OPENRE_PGLITE_DIR || '' },valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:openre:' },-- dbPath: env.OPENRE_DB_PATH || './data/openre.db',// ── Secrets ─────────────────────────────────────────────// 32-byte key (hex or base64) that encrypts destination stream keys and SRT passphrases@@@ -7,8 +7,8 @@* openre-rtmp-ingest sessions it admits, their leases and transitions* openre-restream-worker output state, health and logs*-- * Several writer processes were the trigger ADR-007 named for leaving SQLite; every statement lives behind-- * server/store/*, and every transaction there is SERIALIZABLE (store/index.js), as SQLite's one writer made them.++ * Several writer processes change the same rows; every statement lives behind server/store/*, and every++ * transaction there is SERIALIZABLE (store/index.js), so PostgreSQL detects a conflict and the SDK retries.* The schema is migrations/NNNN_*.sql, applied at boot by the owner role. Times are epoch milliseconds.*/const fs = require('fs');@@@ -15,8 +15,8 @@ const { createRecordings } = require('./recordings');/*** The handle every store gets: its transactions are SERIALIZABLE. Several OpenRe processes change the same rows (a-- * publish admitted by ingest while the coordinator fails a lost worker's session), and SQLite's one writer used to-- * order them; PostgreSQL detects the conflict and the SDK runs the loser again (40001 is retried).++ * publish admitted by ingest while the coordinator fails a lost worker's session); PostgreSQL detects the conflict++ * and the SDK runs the loser again (40001 is retried).*/function serializable(db) {if (db.serializable) return db;@@@ -2,7 +2,7 @@/*** Ingest sessions: admission, the state machine, leases and the playback descriptor.*-- * Every transition is one SQLite transaction that also writes the session_transitions audit row++ * Every transition is one serializable PostgreSQL transaction that also writes the session_transitions audit row* and, where the transition is externally visible, the outbox event (so the event exists if and* only if the change committed). Workers and the coordinator can race on the same session (a* publisher leaving while its lease is being expired); whoever commits first wins and the other@@@ -57,7 +57,7 @@ t('the entry points exit before opening the database', async () => {const r = await p.exited;assert.strictEqual(r.code, 1, `${script} exit code`);assert.match(p.output, /does not run in a restore drill/, script);-- assert.ok(!fs.existsSync(path.join(dir, 'openre.db')), `${script} must not create or touch the database`);++ assert.ok(!fs.existsSync(path.join(dir, 'pglite')), `${script} must not create or touch the database`);}});@@@ -80,7 +80,6 @@ async function testEnv(dir, extra = {}) {NODE_ENV: 'test',PORT: '0',...(t.url ? { DATABASE_URL: t.url, DATABASE_DIRECT_URL: t.directUrl } : { OPENRE_PGLITE_DIR: path.join(dir, 'pglite') }),-- OPENRE_DB_PATH: path.join(dir, 'openre.db'),OPENRE_SECRETS_KEY: SECRETS_KEY,OV_NETWORK_PUBLIC_KEY: publicKey,OPENRE_DEST_ALLOW_PRIVATE: '1',@@@ -1,7 +1,7 @@#!/usr/bin/env node/*** Runs every test in test/ — the files named *.test.js — each in its own process, and fails-- * if any of them fails. They use temp SQLite databases, generated RSA keys and stub subscribers++ * if any of them fails. They use temp databases (PGlite in memory), generated RSA keys and stub subscribers* on random ports; none of them needs the network or a running OpenVibe.Network. The RTMP end-to-end test uses* the system ffmpeg and prints `rtmp-e2e: skipped (no ffmpeg on PATH)` when it is missing.*
@@ -2,7 +2,7 @@
@@ -2,7 +2,7 @@
# OpenRe.Stream — Environment Configuration
# OpenRe.Stream — Environment Configuration
# Copy to .env for development. In production this is /etc/openvibe/openre.env (0600), read by
# Copy to .env for development. In production this is /etc/openvibe/openre.env (0600), read by
# every unit (openre-api, openre-session-coordinator, openre-rtmp-ingest@, openre-restream-worker@);
# every unit (openre-api, openre-session-coordinator, openre-rtmp-ingest@, openre-restream-worker@);
-# the units set NODE_ENV, PORT, OPENRE_RELEASE and OPENRE_DB_PATH themselves.
+# the units set NODE_ENV, PORT and OPENRE_RELEASE themselves.
# Names only here: never commit a value.
# Names only here: never commit a value.
# ═══════════════════════════════════════════════════════════════
# ═══════════════════════════════════════════════════════════════
@@ -29,15 +29,13 @@ OV_OAUTH_CLIENT_SECRET=
@@ -29,15 +29,13 @@ OV_OAUTH_CLIENT_SECRET=
# A browser's Network JWT is accepted when its aud contains one of these.
# A browser's Network JWT is accepted when its aud contains one of these.
#OPENRE_USER_AUDIENCES=openvibe.openre,openvibe.network
#OPENRE_USER_AUDIENCES=openvibe.openre,openvibe.network
-# ── Store (SQLite, shared by every OpenRe process; WAL) ───────
+# ── Store (PostgreSQL, shared by every OpenRe process) ────────
# PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/ at boot;
# PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/ at boot;
# OpenVibe.Host `roles/data/add-service.sh openre` writes both. Every OpenRe process shares the database. Without
# OpenVibe.Host `roles/data/add-service.sh openre` writes both. Every OpenRe process shares the database. Without
# DATABASE_URL, development runs on an embedded PGlite database in data/pglite (OPENRE_PGLITE_DIR): one process only.
# DATABASE_URL, development runs on an embedded PGlite database in data/pglite (OPENRE_PGLITE_DIR): one process only.
DATABASE_URL=
DATABASE_URL=
DATABASE_DIRECT_URL=
DATABASE_DIRECT_URL=
VALKEY_URL=
VALKEY_URL=
-# The SQLite file the one-time move reads (scripts/migrate-to-postgres.js).
-OPENRE_DB_PATH=./data/openre.db
# ── Secrets ───────────────────────────────────────────────────
# ── Secrets ───────────────────────────────────────────────────
# 32 bytes (64 hex chars or base64): encrypts restream destination keys and SRT passphrases at rest.
# 32 bytes (64 hex chars or base64): encrypts restream destination keys and SRT passphrases at rest.
@@ -65,7 +65,7 @@ openre-api (4500) openre-session-coordinator openre-rtmp-inges
@@ -65,7 +65,7 @@ openre-api (4500) openre-session-coordinator openre-rtmp-inges
- **Generations.** Each worker process registers as generation N+1 of its kind. When a newer generation is `ready`, the coordinator marks older ones `draining` (deadline `OPENRE_DRAIN_MAX_MS`, default 24 h). A draining RTMP worker closes its public listener (the kernel sends new encoder connections to the new generation through the shared port) and keeps its publishers; a draining restream worker keeps its outputs. Idle → the process exits 0 by itself. At the deadline, remaining sessions are ended (encoders reconnect to the newest generation) and remaining outputs are handed over.
- **Generations.** Each worker process registers as generation N+1 of its kind. When a newer generation is `ready`, the coordinator marks older ones `draining` (deadline `OPENRE_DRAIN_MAX_MS`, default 24 h). A draining RTMP worker closes its public listener (the kernel sends new encoder connections to the new generation through the shared port) and keeps its publishers; a draining restream worker keeps its outputs. Idle → the process exits 0 by itself. At the deadline, remaining sessions are ended (encoders reconnect to the newest generation) and remaining outputs are handed over.
- **Leases.** Every heartbeat (2 s) renews the worker and the lease of every session it owns in one transaction. A worker silent for `OPENRE_WORKER_LEASE_MS` (15 s) is `lost`: its sessions fail (`openre.session.failed`), its outputs go back to `pending` for another worker. A lost worker that wakes up drops its transports and exits.
- **Leases.** Every heartbeat (2 s) renews the worker and the lease of every session it owns in one transaction. A worker silent for `OPENRE_WORKER_LEASE_MS` (15 s) is `lost`: its sessions fail (`openre.session.failed`), its outputs go back to `pending` for another worker. A lost worker that wakes up drops its transports and exits.
- **Isolation.** An output only ever changes its own row. A dead destination fails (`openre.output.failed`, destination cooldown 15 min → 24 h) and the session stays live (`test/rtmp-e2e.test.js`, `test/sessions.test.js`, `test/coordinator.test.js`).
- **Isolation.** An output only ever changes its own row. A dead destination fails (`openre.output.failed`, destination cooldown 15 min → 24 h) and the session stays live (`test/rtmp-e2e.test.js`, `test/sessions.test.js`, `test/coordinator.test.js`).
-- **Store.** PostgreSQL 18 through PgBouncer (`ov_openre` on the host's data role, ADR-035; schema in [migrations/](migrations/)), shared by every OpenRe process. Every transaction is SERIALIZABLE (`server/store/index.js`): an ingest worker admitting a publish and the coordinator failing a lost worker's session conflict, and PostgreSQL makes one of them retry, as SQLite's one writer ordered them. Admission is async, so the RTMP worker admits a publish before node-media-server's own publish handling runs. Valkey holds the API's per-actor limit counters. The one-time move is `scripts/migrate-to-postgres.js` (run by OpenVibe.Host `roles/data/switch-service.sh` with every OpenRe unit stopped); `/var/lib/openre/openre.db` stays read-only for 7 days as the rollback.
+- **Store.** PostgreSQL 18 through PgBouncer (`ov_openre` on the host's data role, ADR-035; schema in [migrations/](migrations/)), shared by every OpenRe process. Every transaction is SERIALIZABLE (`server/store/index.js`): an ingest worker admitting a publish and the coordinator failing a lost worker's session conflict, and PostgreSQL makes one of them retry. Admission is async, so the RTMP worker admits a publish before node-media-server's own publish handling runs. Valkey holds the API's per-actor limit counters.
## Running it
## Running it
@@ -168,7 +168,7 @@ The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_
@@ -168,7 +168,7 @@ The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_
## Deploying
## Deploying
-Layout: `/opt/openre.stream/repo` (the git clone releases are made from), `/opt/openre.stream/releases/<sha12>` (full checkouts with their own `node_modules`, owned by ubuntu), `current` → the release the API and coordinator run, env `/etc/openvibe/openre.env` (see [.env.example](.env.example)), store `ov_openre` on PostgreSQL (`sudo /opt/openvibe.host/roles/data/add-service.sh openre` writes its settings; the release migrates it at boot; the old SQLite file is `/var/lib/openre/openre.db`), units in [deploy/systemd/](deploy/systemd/), nginx [deploy/nginx/openre.stream.conf](deploy/nginx/openre.stream.conf), script [deploy/scripts/deploy.sh](deploy/scripts/deploy.sh). The release and API phases run through `ovhost deploy openre` (OpenVibe.Host, strategy `release-layout`; roadmap WS-N task 11); the script is a thin wrapper that keeps the old subcommands:
+Layout: `/opt/openre.stream/repo` (the git clone releases are made from), `/opt/openre.stream/releases/<sha12>` (full checkouts with their own `node_modules`, owned by ubuntu), `current` → the release the API and coordinator run, env `/etc/openvibe/openre.env` (see [.env.example](.env.example)), store `ov_openre` on PostgreSQL (`sudo /opt/openvibe.host/roles/data/add-service.sh openre` writes its settings; the release migrates it at boot), units in [deploy/systemd/](deploy/systemd/), nginx [deploy/nginx/openre.stream.conf](deploy/nginx/openre.stream.conf), script [deploy/scripts/deploy.sh](deploy/scripts/deploy.sh). The release and API phases run through `ovhost deploy openre` (OpenVibe.Host, strategy `release-layout`; roadmap WS-N task 11); the script is a thin wrapper that keeps the old subcommands:
```bash
```bash
sudo deploy/scripts/deploy.sh # ovhost deploy openre: release + api in one
sudo deploy/scripts/deploy.sh # ovhost deploy openre: release + api in one
@@ -15,7 +15,7 @@
@@ -15,7 +15,7 @@
#
#
# Layout: /opt/openre.stream/releases/<sha>/ (full checkout + node_modules), `current` symlink
# Layout: /opt/openre.stream/releases/<sha>/ (full checkout + node_modules), `current` symlink
# for the API and coordinator, one openre-rtmp-ingest@<sha> / openre-restream-worker@<sha>
# for the API and coordinator, one openre-rtmp-ingest@<sha> / openre-restream-worker@<sha>
-# instance per worker generation. Env: /etc/openvibe/openre.env. Store: /var/lib/openre/openre.db.
+# instance per worker generation. Env: /etc/openvibe/openre.env. Store: ov_openre on PostgreSQL (ADR-035).
# ═══════════════════════════════════════════════════════════════
# ═══════════════════════════════════════════════════════════════
set -euo pipefail
set -euo pipefail
@@ -13,8 +13,6 @@ WorkingDirectory=/opt/openre.stream/current
@@ -13,8 +13,6 @@ WorkingDirectory=/opt/openre.stream/current
EnvironmentFile=/etc/openvibe/openre.env
EnvironmentFile=/etc/openvibe/openre.env
Environment=NODE_ENV=production
Environment=NODE_ENV=production
Environment=PORT=4500
Environment=PORT=4500
-# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).
-Environment=OPENRE_DB_PATH=/var/lib/openre/openre.db
Environment=PATH=/usr/local/bin:/usr/bin:/bin
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node server/index.js
ExecStart=/usr/bin/env node server/index.js
Restart=always
Restart=always
@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%i
@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%i
EnvironmentFile=/etc/openvibe/openre.env
EnvironmentFile=/etc/openvibe/openre.env
Environment=NODE_ENV=production
Environment=NODE_ENV=production
Environment=OPENRE_RELEASE=%i
Environment=OPENRE_RELEASE=%i
-# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).
-Environment=OPENRE_DB_PATH=/var/lib/openre/openre.db
Environment=PATH=/usr/local/bin:/usr/bin:/bin
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node workers/restream-worker.js
ExecStart=/usr/bin/env node workers/restream-worker.js
# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.
# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.
@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%i
@@ -16,8 +16,6 @@ WorkingDirectory=/opt/openre.stream/releases/%i
EnvironmentFile=/etc/openvibe/openre.env
EnvironmentFile=/etc/openvibe/openre.env
Environment=NODE_ENV=production
Environment=NODE_ENV=production
Environment=OPENRE_RELEASE=%i
Environment=OPENRE_RELEASE=%i
-# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).
-Environment=OPENRE_DB_PATH=/var/lib/openre/openre.db
Environment=PATH=/usr/local/bin:/usr/bin:/bin
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node workers/rtmp-ingest.js
ExecStart=/usr/bin/env node workers/rtmp-ingest.js
# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.
# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.
@@ -11,8 +11,6 @@ Group=ubuntu
@@ -11,8 +11,6 @@ Group=ubuntu
WorkingDirectory=/opt/openre.stream/current
WorkingDirectory=/opt/openre.stream/current
EnvironmentFile=/etc/openvibe/openre.env
EnvironmentFile=/etc/openvibe/openre.env
Environment=NODE_ENV=production
Environment=NODE_ENV=production
-# The SQLite file the one-time move read (scripts/migrate-to-postgres.js); the store is ov_openre (ADR-035).
-Environment=OPENRE_DB_PATH=/var/lib/openre/openre.db
Environment=PATH=/usr/local/bin:/usr/bin:/bin
Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node workers/coordinator.js
ExecStart=/usr/bin/env node workers/coordinator.js
Restart=always
Restart=always
@@ -46,8 +46,8 @@ Everything below runs on the host. The scripts ship with the release (`scripts/`
@@ -46,8 +46,8 @@ Everything below runs on the host. The scripts ship with the release (`scripts/`
|---|---|---|
|---|---|---|
| `cutover-preflight.js` | read-only checks: `release service env bind dns port db live-env events [slot]` (`--only`, `--skip`, `--slot <id>`, `--json`, `--strict`, `--expect-release <sha>`, `--probe-url`, `--host-ip`) | nothing |
| `cutover-preflight.js` | read-only checks: `release service env bind dns port db live-env events [slot]` (`--only`, `--skip`, `--slot <id>`, `--json`, `--strict`, `--expect-release <sha>`, `--probe-url`, `--host-ip`) | nothing |
| `subscribe-live-events.js` | Live's Events subscription `openre.session.*` → Live, **as Live** (reads `/etc/openvibe/live.env`); `--dry-run`, `--disable`, `--enable` | one subscription in Events |
| `subscribe-live-events.js` | Live's Events subscription `openre.session.*` → Live, **as Live** (reads `/etc/openvibe/live.env`); `--dry-run`, `--disable`, `--enable` | one subscription in Events |
-| `migrate-from-live.js` | imports slots and destinations from a Live **snapshot** (dry run unless `--apply`) | `openre.db` |
-| `set-definition-state.js` | disables or re-enables one slot's OpenRe definition, **as Live** (per-slot rollback) | `openre.db` via the API |
+| `migrate-from-live.js` | imports slots and destinations from a Live **snapshot** (dry run unless `--apply`) | the PostgreSQL store (`ov_openre`) |
+| `set-definition-state.js` | disables or re-enables one slot's OpenRe definition, **as Live** (per-slot rollback) | the PostgreSQL store (`ov_openre`) via the API |
Shell helpers for the session (paste once per SSH session):
Shell helpers for the session (paste once per SSH session):
@@ -56,8 +56,7 @@ SHA=<the 12-character release id printed by step A2>
@@ -56,8 +56,7 @@ SHA=<the 12-character release id printed by step A2>
# read-only preflight from that release
# read-only preflight from that release
pf() { sudo node /opt/openre.stream/releases/$SHA/scripts/cutover-preflight.js --expect-release "$SHA" "$@"; }
pf() { sudo node /opt/openre.stream/releases/$SHA/scripts/cutover-preflight.js --expect-release "$SHA" "$@"; }
# run a command in the current release with OpenRe's production env, AS THE SERVICE USER (ubuntu).
# run a command in the current release with OpenRe's production env, AS THE SERVICE USER (ubuntu).
-# Never run a script that opens openre.db as root: root-owned -wal/-shm files lock the service out.
-openre_as_service() { sudo bash -c "set -a; . /etc/openvibe/openre.env; set +a; export NODE_ENV=production OPENRE_DB_PATH=/var/lib/openre/openre.db OPENRE_ENV_FILE=/nonexistent; cd /opt/openre.stream/current && sudo -E -u ubuntu $1"; }
+openre_as_service() { sudo bash -c "set -a; . /etc/openvibe/openre.env; set +a; export NODE_ENV=production OPENRE_ENV_FILE=/nonexistent; cd /opt/openre.stream/current && sudo -E -u ubuntu $1"; }
DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh
DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh
```
```
@@ -77,7 +76,7 @@ from this commit onward is on GitHub yet. Undo: nothing to undo.
@@ -77,7 +76,7 @@ from this commit onward is on GitHub yet. Undo: nothing to undo.
```bash
```bash
sudo /opt/openre.stream/current/deploy/scripts/deploy.sh release origin/main # prints the new <sha>
sudo /opt/openre.stream/current/deploy/scripts/deploy.sh release origin/main # prints the new <sha>
SHA=<that sha>; DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh # and re-paste pf()
SHA=<that sha>; DEPLOY=/opt/openre.stream/releases/$SHA/deploy/scripts/deploy.sh # and re-paste pf()
-pf --only db # openre.db is sound before anything restarts
+pf --only db # the PostgreSQL store is sound before anything restarts
sudo $DEPLOY api $SHA # restarts openre-api + openre-session-coordinator ONLY
sudo $DEPLOY api $SHA # restarts openre-api + openre-session-coordinator ONLY
curl -s http://127.0.0.1:4500/api/ready # "status":"ready"
curl -s http://127.0.0.1:4500/api/ready # "status":"ready"
pf --only release,env,db # current = $SHA, it contains 6dc78a5
pf --only release,env,db # current = $SHA, it contains 6dc78a5
@@ -219,11 +218,11 @@ refuses that user's personal RTMP key (`refusesLiveIngest`).
@@ -219,11 +218,11 @@ refuses that user's personal RTMP key (`refusesLiveIngest`).
`flv.internal_url`).
`flv.internal_url`).
- `await api('/admin/openre/status')` lists the session under `live_sessions`.
- `await api('/admin/openre/status')` lists the session under `live_sessions`.
- Sessions and outputs:
- Sessions and outputs:
- `sudo sqlite3 -readonly /var/lib/openre/openre.db "SELECT id, state, worker_generation FROM ingest_sessions ORDER BY created_at DESC LIMIT 1; SELECT o.state, o.last_error FROM outputs o ORDER BY o.created_at DESC LIMIT 3; SELECT state, media_vod_id, last_error FROM recordings ORDER BY created_at DESC LIMIT 1;"`
+ `sudo -u ubuntu bash -c "set -a; . /etc/openvibe/openre.env; set +a; psql \"\$DATABASE_URL\" -c \"SELECT id, state, worker_generation FROM ingest_sessions ORDER BY created_at DESC LIMIT 1; SELECT o.state, o.last_error FROM outputs o ORDER BY o.created_at DESC LIMIT 3; SELECT state, media_vod_id, last_error FROM recordings ORDER BY created_at DESC LIMIT 1;\""`
should show a `live` session, a `live` output and a `recording` recording.
should show a `live` session, a `live` output and a `recording` recording.
- The destination (YouTube) shows the stream.
- The destination (YouTube) shows the stream.
- Deliveries:
- Deliveries:
- `sudo sqlite3 -readonly /var/lib/openvibe-events/events.db "SELECT d.status, count(*) FROM deliveries d JOIN subscriptions s ON s.id = d.subscription_id WHERE s.consumer = 'live' AND s.topic_pattern = 'openre.session.*' GROUP BY 1"`
+ `sudo -u ubuntu bash -c "set -a; . /etc/openvibe/events.env; set +a; psql \"\$DATABASE_URL\" -c \"SELECT d.status, count(*) FROM deliveries d JOIN subscriptions s ON s.id = d.subscription_id WHERE s.consumer = 'live' AND s.topic_pattern = 'openre.session.*' GROUP BY 1\""`
should all be `delivered`.
should all be `delivered`.
5. **During the broadcast (AGENT):**
5. **During the broadcast (AGENT):**
- `sudo $DEPLOY api $SHA`: the stream, the restream and the recording do not drop. OpenRe
- `sudo $DEPLOY api $SHA`: the stream, the restream and the recording do not drop. OpenRe
@@ -11,8 +11,8 @@
@@ -11,8 +11,8 @@
* Take the snapshot with `sqlite3 /opt/openvibe.live/data/live.db ".backup /tmp/live-snapshot.db"`
* Take the snapshot with `sqlite3 /opt/openvibe.live/data/live.db ".backup /tmp/live-snapshot.db"`
* (Live's DB_PATH in production; never point this at the live file). Uses the OpenRe environment
* (Live's DB_PATH in production; never point this at the live file). Uses the OpenRe environment
* (.env or OPENRE_ENV_FILE) for its own database and OPENRE_SECRETS_KEY; in production set
* (.env or OPENRE_ENV_FILE) for its own database and OPENRE_SECRETS_KEY; in production set
- * OPENRE_ENV_FILE=/etc/openvibe/openre.env, OPENRE_DB_PATH=/var/lib/openre/openre.db and
- * NODE_ENV=production (the units set the last two themselves) — docs/cutover.md has the command. Prints no secret: old keys are not imported and the new
+ * OPENRE_ENV_FILE=/etc/openvibe/openre.env and NODE_ENV=production (a unit sets NODE_ENV itself)
+ * — docs/cutover.md has the command. Prints no secret: old keys are not imported and the new
* keys are never shown (the broadcaster rotates to get one). Safe to re-run.
* keys are never shown (the broadcaster rotates to get one). Safe to re-run.
*/
*/
const fs = require('fs');
const fs = require('fs');
deleted file mode 100755
deleted file mode 100755
@@ -1,27 +0,0 @@
@@ -1,27 +0,0 @@
-#!/usr/bin/env node
-'use strict';
-/**
- * The one-time move of OpenRe's SQLite database (OPENRE_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is
- * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).
- *
- * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
- *
- * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts
- * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its
- * name and columns; the SDK outbox's rows move into its PostgreSQL outbox (the same columns). Run with every OpenRe
- * process stopped (OpenVibe.Host roles/data/switch-service.sh with SWITCH_UNITS).
- */
-require('dotenv').config({ path: process.env.OPENRE_ENV_FILE || require('path').join(process.cwd(), '.env') });
-const { runSqliteMigration } = require('openvibe-sdk/db');
-const { load } = require('../server/config');
-const { MIGRATIONS } = require('../server/db');
-
-const TABLES = {};
-
-if (require.main === module) {
- const config = load();
- runSqliteMigration({ service: 'openre', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })
- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
-}
-
-module.exports = { TABLES };
@@ -76,7 +76,7 @@ function createApp({ rt, auth, keys, log = console, fetchImpl }) {
@@ -76,7 +76,7 @@ function createApp({ rt, auth, keys, log = console, fetchImpl }) {
service: 'openre',
service: 'openre',
release: release.release,
release: release.release,
checks: [
checks: [
- { name: 'db', required: true, description: 'SQLite answers a query', check: async () => (await db.prepare('SELECT 1 AS ok').get()).ok === 1 },
+ { name: 'db', required: true, description: 'PostgreSQL answers a query', check: async () => (await db.prepare('SELECT 1 AS ok').get()).ok === 1 },
{ name: 'network_key', required: true, description: 'OpenVibe.Network RS256 key (sign-in and service tokens)', check: () => keys.loaded() || 'Network public key not loaded yet' },
{ name: 'network_key', required: true, description: 'OpenVibe.Network RS256 key (sign-in and service tokens)', check: () => keys.loaded() || 'Network public key not loaded yet' },
{
{
name: 'workers', required: false, description: 'a ready rtmp-ingest and restream worker',
name: 'workers', required: false, description: 'a ready rtmp-ingest and restream worker',
@@ -5,8 +5,8 @@
@@ -5,8 +5,8 @@
*
*
* load(env) is pure so tests (and every worker process) can build a config without touching
* load(env) is pure so tests (and every worker process) can build a config without touching
* process.env. All processes of the service (openre-api, openre-session-coordinator and the
* process.env. All processes of the service (openre-api, openre-session-coordinator and the
- * transport workers) read the same file and share one SQLite database (ADR-007: one service,
- * one store; the processes are the reason this service is the first candidate for PostgreSQL).
+ * transport workers) read the same file and share one PostgreSQL database (ADR-035: one
+ * service, one store).
*/
*/
const int = (v, d) => {
const int = (v, d) => {
@@ -58,11 +58,9 @@ function load(env = process.env) {
@@ -58,11 +58,9 @@ function load(env = process.env) {
// ── Store ───────────────────────────────────────────────
// ── Store ───────────────────────────────────────────────
// PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/.
// PostgreSQL (ADR-035): DATABASE_URL serves through PgBouncer, DATABASE_DIRECT_URL (the owner) runs migrations/.
- // Without DATABASE_URL, development and tests use an embedded PGlite database (OPENRE_PGLITE_DIR). dbPath: the
- // SQLite file the one-time move reads (scripts/migrate-to-postgres.js).
+ // Without DATABASE_URL, development and tests use an embedded PGlite database (OPENRE_PGLITE_DIR).
db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '', pgliteDir: env.OPENRE_PGLITE_DIR || '' },
db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '', pgliteDir: env.OPENRE_PGLITE_DIR || '' },
valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:openre:' },
valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:openre:' },
- dbPath: env.OPENRE_DB_PATH || './data/openre.db',
// ── Secrets ─────────────────────────────────────────────
// ── Secrets ─────────────────────────────────────────────
// 32-byte key (hex or base64) that encrypts destination stream keys and SRT passphrases
// 32-byte key (hex or base64) that encrypts destination stream keys and SRT passphrases
@@ -7,8 +7,8 @@
@@ -7,8 +7,8 @@
* openre-rtmp-ingest sessions it admits, their leases and transitions
* openre-rtmp-ingest sessions it admits, their leases and transitions
* openre-restream-worker output state, health and logs
* openre-restream-worker output state, health and logs
*
*
- * Several writer processes were the trigger ADR-007 named for leaving SQLite; every statement lives behind
- * server/store/*, and every transaction there is SERIALIZABLE (store/index.js), as SQLite's one writer made them.
+ * Several writer processes change the same rows; every statement lives behind server/store/*, and every
+ * transaction there is SERIALIZABLE (store/index.js), so PostgreSQL detects a conflict and the SDK retries.
* The schema is migrations/NNNN_*.sql, applied at boot by the owner role. Times are epoch milliseconds.
* The schema is migrations/NNNN_*.sql, applied at boot by the owner role. Times are epoch milliseconds.
*/
*/
const fs = require('fs');
const fs = require('fs');
@@ -15,8 +15,8 @@ const { createRecordings } = require('./recordings');
@@ -15,8 +15,8 @@ const { createRecordings } = require('./recordings');
/**
/**
* The handle every store gets: its transactions are SERIALIZABLE. Several OpenRe processes change the same rows (a
* The handle every store gets: its transactions are SERIALIZABLE. Several OpenRe processes change the same rows (a
- * publish admitted by ingest while the coordinator fails a lost worker's session), and SQLite's one writer used to
- * order them; PostgreSQL detects the conflict and the SDK runs the loser again (40001 is retried).
+ * publish admitted by ingest while the coordinator fails a lost worker's session); PostgreSQL detects the conflict
+ * and the SDK runs the loser again (40001 is retried).
*/
*/
function serializable(db) {
function serializable(db) {
if (db.serializable) return db;
if (db.serializable) return db;
@@ -2,7 +2,7 @@
@@ -2,7 +2,7 @@
/**
/**
* Ingest sessions: admission, the state machine, leases and the playback descriptor.
* Ingest sessions: admission, the state machine, leases and the playback descriptor.
*
*
- * Every transition is one SQLite transaction that also writes the session_transitions audit row
+ * Every transition is one serializable PostgreSQL transaction that also writes the session_transitions audit row
* and, where the transition is externally visible, the outbox event (so the event exists if and
* and, where the transition is externally visible, the outbox event (so the event exists if and
* only if the change committed). Workers and the coordinator can race on the same session (a
* only if the change committed). Workers and the coordinator can race on the same session (a
* publisher leaving while its lease is being expired); whoever commits first wins and the other
* publisher leaving while its lease is being expired); whoever commits first wins and the other
@@ -57,7 +57,7 @@ t('the entry points exit before opening the database', async () => {
@@ -57,7 +57,7 @@ t('the entry points exit before opening the database', async () => {
const r = await p.exited;
const r = await p.exited;
assert.strictEqual(r.code, 1, `${script} exit code`);
assert.strictEqual(r.code, 1, `${script} exit code`);
assert.match(p.output, /does not run in a restore drill/, script);
assert.match(p.output, /does not run in a restore drill/, script);
- assert.ok(!fs.existsSync(path.join(dir, 'openre.db')), `${script} must not create or touch the database`);
+ assert.ok(!fs.existsSync(path.join(dir, 'pglite')), `${script} must not create or touch the database`);
}
}
});
});
@@ -80,7 +80,6 @@ async function testEnv(dir, extra = {}) {
@@ -80,7 +80,6 @@ async function testEnv(dir, extra = {}) {
NODE_ENV: 'test',
NODE_ENV: 'test',
PORT: '0',
PORT: '0',
...(t.url ? { DATABASE_URL: t.url, DATABASE_DIRECT_URL: t.directUrl } : { OPENRE_PGLITE_DIR: path.join(dir, 'pglite') }),
...(t.url ? { DATABASE_URL: t.url, DATABASE_DIRECT_URL: t.directUrl } : { OPENRE_PGLITE_DIR: path.join(dir, 'pglite') }),
- OPENRE_DB_PATH: path.join(dir, 'openre.db'),
OPENRE_SECRETS_KEY: SECRETS_KEY,
OPENRE_SECRETS_KEY: SECRETS_KEY,
OV_NETWORK_PUBLIC_KEY: publicKey,
OV_NETWORK_PUBLIC_KEY: publicKey,
OPENRE_DEST_ALLOW_PRIVATE: '1',
OPENRE_DEST_ALLOW_PRIVATE: '1',
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
#!/usr/bin/env node
#!/usr/bin/env node
/**
/**
* Runs every test in test/ — the files named *.test.js — each in its own process, and fails
* Runs every test in test/ — the files named *.test.js — each in its own process, and fails
- * if any of them fails. They use temp SQLite databases, generated RSA keys and stub subscribers
+ * if any of them fails. They use temp databases (PGlite in memory), generated RSA keys and stub subscribers
* on random ports; none of them needs the network or a running OpenVibe.Network. The RTMP end-to-end test uses
* on random ports; none of them needs the network or a running OpenVibe.Network. The RTMP end-to-end test uses
* the system ffmpeg and prints `rtmp-e2e: skipped (no ffmpeg on PATH)` when it is missing.
* the system ffmpeg and prints `rtmp-e2e: skipped (no ffmpeg on PATH)` when it is missing.
*
*