openvibers/openvibe.network Commits
T2 cutover: runbook and rehearsal for the PostgreSQL switch (#11)
Changes
nnew file mode 100644@@@ -0,0 +1,370 @@++# T2 cutover: Network from SQLite to PostgreSQL++++The production move of OpenVibe.Network from its SQLite file to PostgreSQL (plan T2, ADR-035). Production++(`openvibe-ovh`) runs **20dd7ff**, the last SQLite release. Every later commit serves from PostgreSQL only:++`DATABASE_URL` is required in production (there is no SQLite path), migrations run as the owner on++`DATABASE_DIRECT_URL`, and the limit counters live on Valkey (`VALKEY_URL`). Two manual deploys of that range++failed readiness and ovhost rolled them back. The most likely cause is that the server refused to boot without++`DATABASE_URL` (`server/db/database.js`), but this is unconfirmed. This runbook covers the data move those++deploys skipped. It was rehearsed with `npm run rehearse-pg-cutover` (`scripts/rehearse-pg-cutover.js`) on++20dd7ff's full schema: the 51 tables `initDb()` makes plus the 26 its modules create at boot. That rehearsal++found two import blockers, fixed in `scripts/migrate-to-postgres.js` with it:++++- `analytics_rate_tracking` (openvibe-shared's old per-IP counters) has no PostgreSQL table, so the import refused++ to start. It is now in `SKIP_SOURCE`, so IPs are never carried over.++- The `users_profile_created` trigger fired on every imported account. That collided with the copied++ `user_profile_changes` rows, and would have published one profile event per account. The import now turns++ that trigger off while it copies (`QUIET_TRIGGERS`).++++Every step ends with a **go / no-go** check. A no-go before step 8 (deploy) leaves production on SQLite with no++data change: restart the old release (`sudo systemctl start openvibe-network`), reopen, and reschedule. A no-go++from step 8 on means [rollback](#rollback).++++Conventions:++++- `$STAGE` is the staging checkout of the target commit (step 1).++- `$COPY` is the read-only SQLite copy (step 4).++- `$T0` is when writes stopped (step 3).++- `$T1` is when the PostgreSQL release started serving (step 8).++- Commands that need the database URLs load `/etc/openvibe/network.env` with `node --env-file`, which never++ prints it, as README.md does for `subscribe-events.js`. Never `cat`, `source -x`, `env` or `printenv` that file,++ and never paste a URL into a terminal, chat or ticket.++++---++++## Prerequisites (owner only, before the window)++++These are done by the owner and never by an agent. Each must be true before the window opens.++++1. **PostgreSQL, PgBouncer and Valkey run on openvibe-ovh.** PostgreSQL 18, with PgBouncer in transaction++ pooling mode in front of it, and Valkey. Network's database and its two roles are created by OpenVibe.Host++ `roles/data/add-service.sh network`:++ - an **owner** role that owns the schema, runs the migrations and does the import;++ - a **runtime** role with DML rights only, which serves requests through PgBouncer.++2. **The three URLs are in `/etc/openvibe/network.env`**, mode `0600`, and are never printed:++ - `DATABASE_URL`: the runtime role, through PgBouncer.++ - `DATABASE_DIRECT_URL`: the owner role, on a **direct** connection (not PgBouncer). Transaction pooling cannot++ hold the import's single transaction or the migration lock.++ - `VALKEY_URL`: the shared limit counters. Without it, each process counts on its own.++++ Check them by name only. The first command must print `3`, the second `600`:++++ ```bash++ sudo grep -cE '^(DATABASE_URL|DATABASE_DIRECT_URL|VALKEY_URL)=.+' /etc/openvibe/network.env++ sudo stat -c '%a' /etc/openvibe/network.env++ ```++3. **The URLs connect.** Run this from `$STAGE` (step 1). It prints three `ok` lines and never prints a URL:++++ ```bash++ cd "$STAGE" && sudo node --env-file=/etc/openvibe/network.env -e '++ const { Client } = require("pg");++ (async () => {++ for (const k of ["DATABASE_URL", "DATABASE_DIRECT_URL"]) {++ const c = new Client({ connectionString: process.env[k] });++ await c.connect();++ await c.query("SELECT 1");++ await c.end();++ console.log(k, "ok");++ }++ const v = new (require("iovalkey"))(process.env.VALKEY_URL, { lazyConnect: true });++ await v.connect();++ console.log("VALKEY_URL", await v.ping() === "PONG" ? "ok" : "FAIL");++ v.disconnect();++ })().catch((e) => { console.error("FAIL", e.code || e.message.replace(/\/\/[^@]*@/, "//***@")); process.exit(1); });'++ ```++4. **A green rehearsal on a fresh production snapshot** (step 2). Only after it passes does the owner write++ `~/openvibe/agents/ds/deploy/rehearsals/network-t2-pg.json` with `{"ok": true}`. ds-finish refuses to deploy++ without that file and without the merged PR's `cutover` block.++5. **No case-fold collisions**, or each one resolved on the SQLite side (step 5 explains how). Check this at++ rehearsal time so the window does not stop on it.++6. **The window is announced.** Post a status-page maintenance (admin → Status, kind `maintenance`) at least a++ day ahead. While Network is stopped, nobody can sign in, run an OAuth authorize, or refresh a token on any++ OpenVibe site. Services verify JWTs they already have offline (cached JWKS), so signed-in pages keep working.++ Calls into Network fail until it is back: notifications, coins, follows and blocks. Expect about 15 minutes++ plus the import time that the snapshot rehearsal printed (`ms`).++7. **The target commit is fixed.** It is the merge commit of the PR carrying the `cutover` block (origin/main at++ merge). Steps 1, 2 and 8 must all use that one SHA.++++---++++## Step 1: stage the target release (T-1 day, no downtime)++++The running checkout `/opt/openvibe.network` stays on 20dd7ff until step 8. The import tools and migrations++come from a separate checkout of the target commit. It is never served from.++++```bash++TARGET=<merge sha>++STAGE=/var/tmp/network-t2-stage++sudo -u ubuntu git -C /opt/openvibe.network fetch origin++sudo -u ubuntu git clone --no-checkout /opt/openvibe.network "$STAGE"++sudo -u ubuntu git -C "$STAGE" checkout --detach "$TARGET"++cd "$STAGE" && sudo -u ubuntu npm ci # dev dependencies too: the rehearsal's PGlite lives there++```++++**Go:** `git -C "$STAGE" rev-parse HEAD` equals `$TARGET`, and++`node -e "require('better-sqlite3'); require('pg'); require('@electric-sql/pglite')"` exits 0 in `$STAGE`.++**No-go:** `npm ci` fails. Fix the host's build tools or network first.++++## Step 2: rehearse on a production snapshot (T-1 day, no downtime)++++Take an online copy (the service keeps running) and rehearse on it, on the host, so the data never leaves it:++++```bash++SNAP=/var/backups/openvibe/network-rehearsal-$(date -u +%Y%m%dT%H%M%SZ).db++sudo -u ubuntu node -e "new (require('$STAGE/node_modules/better-sqlite3'))('/opt/openvibe.network/data/network.db', { readonly: true }).backup('$SNAP').then(() => console.log('ok'))"++cd "$STAGE" && sudo -u ubuntu node scripts/rehearse-pg-cutover.js --sqlite "$SNAP" --store pglite++```++++The live file is `/opt/openvibe.network/data/network.db` unless `DB_PATH` says otherwise. A path is not a++secret, so `sudo grep '^DB_PATH=' /etc/openvibe/network.env` may be used to check it.++++The rehearsal runs, in order:++++1. pg-preflight;++2. `migrate-to-postgres.js --pglite`;++3. the same import into a PGlite directory;++4. the row-count parity check;++5. boots the target server on the result and requires `/api/ready` to answer 200 with `db` and `signing_key` ok.++++It prints one JSON line, with counts only.++++**Go:** exit 0 and `"ok":true`. The owner keeps that line and writes the `network-t2-pg` rehearsal file++(prerequisite 4).++**No-go:** read `problems` in the line.++- `preflight` collisions: see step 5.++- Import problems, for example a source table with no target or an unparseable date: fix them in the migrations++ or in `TABLES`/`SKIP_SOURCE` of `scripts/migrate-to-postgres.js` with a PR, then rehearse again.++++`--store env` (only when given; the default never reads these URLs) runs the same rehearsal against a scratch++PostgreSQL database instead of PGlite. Put that database's own `DATABASE_URL`/`DATABASE_DIRECT_URL` in the++environment. The rehearsal empties its tables and refuses a target where any table holds a row (unless++`--truncate-target`), but **never** point it at the production URLs.++++## Step 3: freeze writes (T0, the window opens)++++```bash++sudo systemctl stop openvibe-network++T0=$(date -u +%Y-%m-%dT%H:%M:%SZ); echo "$T0"++```++++**Go:** all three hold:++- `systemctl is-active openvibe-network` prints `inactive`;++- `curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4000/api/health` prints `000`;++- nothing restarts it within a minute. Check `systemctl is-active` again.++++**No-go:** it comes back on its own (a watchdog or a deploy). Find what restarted it before going on. A write++made after the copy in step 4 would be lost.++++## Step 4: back up, and copy the SQLite file++++```bash++ov access run openvibe-ovh db-backup++COPY=/var/backups/openvibe/network-pre-t2-${T0//:/}.db++sudo -u ubuntu node -e "new (require('$STAGE/node_modules/better-sqlite3'))('/opt/openvibe.network/data/network.db', { readonly: true }).backup('$COPY').then(() => console.log('ok'))"++sudo chmod 0400 "$COPY" && sudo sha256sum "$COPY" | tee "$COPY.sha256"++sudo -u ubuntu node -e "const d = new (require('$STAGE/node_modules/better-sqlite3'))('$COPY', { readonly: true }); console.log(d.pragma('integrity_check', { simple: true }))"++```++++The backup API copies the file together with its WAL. From here on, every step reads `$COPY`. The live++`data/network.db` is not opened again, and it is what a rollback serves.++++**Go:** all three hold:++- `db-backup` exits 0;++- `integrity_check` prints `ok`;++- the user count of `$COPY` equals the live file's. Run this for each file:++ `node -e "…prepare('SELECT COUNT(*) n FROM users').get().n"`.++++**No-go:** the backup fails or the integrity check is not `ok`. Restart the old release and investigate.++++## Step 5: pg-preflight (case-fold collisions)++++```bash++cd "$STAGE" && sudo -u ubuntu npm run pg-preflight -- --sqlite "$COPY"++```++++The script is `scripts/pg-preflight-collisions.js` (`--sqlite <file>`, `--json`). It opens the copy read-only.++++**Go:** exit 0, `no case-fold collisions: safe to import`.++**No-go:**++- **Exit 1 (collisions).** PostgreSQL keeps usernames unique on `lower(username)`, so `Alex` and `alex` cannot++ both be imported. This is a data decision, never made by a script. Restart the old release and close the++ window. For each listed pair:++ - `users.username`: ask the newer or inactive account to rename, or rename it from the old release's admin++ (users → edit username). That records the old name.++ - `verification_keys.target_username` (active keys): revoke the duplicate key in admin.++ - `username_history.old_username`: absent at 20dd7ff. The script reports `table absent`.++++ Run the preflight on a fresh snapshot until it is clean, then schedule a new window.++- **Exit 2.** The file is unreadable or better-sqlite3 is missing. Fix that before going on.++++## Step 6: migrate-to-postgres++++The script reads these and nothing else:++++| Input | Meaning |++|---|---|++| `--sqlite <file>` | The source: `$COPY`. Without it, `DB_PATH`, else `data/network.db` under the checkout. Always pass it. |++| `--pglite` | A dry run into an in-memory PostgreSQL. No URL is needed. |++| `--json` | The report as JSON. |++| `DATABASE_DIRECT_URL` | The owner, on a direct connection. Required without `--pglite`. `DATABASE_URL` is not used. |++| `.env` | Read from the working directory by dotenv. `$STAGE` has none. Keep it that way. |++++```bash++cd "$STAGE"++sudo -u ubuntu node scripts/migrate-to-postgres.js --sqlite "$COPY" --pglite # dry run++sudo node --env-file=/etc/openvibe/network.env scripts/migrate-to-postgres.js --sqlite "$COPY" # the import++```++++The import does four things:++++1. Applies `migrations/0001…0007` as the owner.++2. **Empties** every Network table and copies every table in one transaction, parents first. Ids are kept and++ sequences are moved past them. `users_profile_created` is disabled for the copy and enabled again afterwards,++ even when the copy fails. `analytics_rate_tracking` is left behind.++3. Drops a NUL or an unpaired surrogate, which PostgreSQL refuses, and reports it as `cleaned:`.++4. Verifies each table's row count and a checksum of every row.++++The SQLite copy is opened read-only. Because the import empties first, it can be repeated.++++**Go:** exit 0, first line `… : OK`, and no `problem:` line. A `cleaned:` line is expected and fine. Note it in++the log.++**No-go:** exit 1 or a `problem:` line. Leave the window closed, restart the old release, and fix the problem++with a PR and a new rehearsal. Nothing serves from PostgreSQL yet, so there is nothing to undo. If the process++was killed mid-import, the trigger is still off: run the import again. It ends by enabling the trigger, and++step 7 checks that it did.++++## Step 7: row-count parity, per table++++```bash++cd "$STAGE" && sudo node --env-file=/etc/openvibe/network.env scripts/pg-row-parity.js --sqlite "$COPY"++```++++`scripts/pg-row-parity.js` reads `DATABASE_DIRECT_URL` (else `DATABASE_URL`). It compares every SQLite table:++++- **SQLite** (`$COPY`, read-only): `SELECT COUNT(*) FROM "<table>"` for each table in++ `SELECT name FROM sqlite_master WHERE type = 'table'`. SQLite's own `sqlite_*` tables, FTS5 virtual tables and++ their shadow tables, and `SKIP_SOURCE` (`analytics_rate_tracking`: old per-IP counters, never carried over)++ are left out.++- **PostgreSQL**: `SELECT COUNT(*) FROM "<table>"` for the same name, in++ `SELECT tablename FROM pg_tables WHERE schemaname = current_schema()`.++++PostgreSQL-only tables, new in T2, are listed and not compared. The script also requires the triggers that the++import turned off to be enabled again:++`SELECT tgenabled FROM pg_trigger WHERE tgname = 'users_profile_created'` must be `O`. Run the check **before** step 8. The new++release's boot seeds rows the SQLite file never had: built-in themes, OAuth clients and principal grants.++++**Go:** exit 0, `OK`, and no `✗` row.++**No-go:** any table differs, or the trigger is reported disabled. Do not deploy. Run step 6 again; it empties first. If the same table still differs,++restart the old release and investigate.++++## Step 8: deploy (T1)++++```bash++ov access run openvibe-ovh deploy network++T1=$(date -u +%Y-%m-%dT%H:%M:%SZ); echo "$T1"++```++++ovhost fast-forwards `/opt/openvibe.network` to the target and runs `npm install --omit=dev`. It installs the unit++and starts it with `network.env`, polls `/api/ready`, and rolls back on its own if the release does not come up.++At boot, the migrations find everything applied, and the seed adds only the boot rows that are missing.++++**Go:** the deploy exits 0, and its release is `$TARGET`.++**No-go:**++- **ovhost rolled back (exit 3).** Production is on 20dd7ff and SQLite again, and no write reached PostgreSQL.++ Read `ovhost releases network` and the boot log++ (`sudo journalctl -u openvibe-network --since "$T0" | tail -50`). The log names its failure without printing++ URLs. Then close the window.++- **Any other failure.** Go to [rollback](#rollback).++++## Step 9: checks++++Do these within 30 minutes of T1. All must pass.++++1. **Health.** `ov access run openvibe-ovh health network` reports healthy, on release `$TARGET`.++2. **Readiness.** This must print `true "ok" "ok"`:++++ ```bash++ curl -s https://openvibe.network/api/ready | jq '.ready, .checks.db.status, .checks.signing_key.status'++ ```++++ `status` may be `degraded` only because of optional checks (`registry_poll`, `discord_bot`).++ `failed` must be `[]`.++3. **Sign-in.** Sign in at `https://openvibe.network/login` with an account that existed before T0. That proves++ the imported password hashes. Then `GET /api/auth/me` returns the same username and subject id as before.++ A session from before T0 is still signed in: same RS256 key, imported `user_sessions`.++4. **OAuth authorize.** On a first-party site (openvibe.live, then openvibe.chat), sign out, then use "Sign in++ with OpenVibe". `/oauth/authorize` must redirect back with a code and the site must sign you in. That proves++ `oauth_clients`, the redirect URIs and token issuance. Also refresh a token on a site that was signed in++ before T0.++5. **Notifications.** `GET /api/notifications` lists the notifications from before T0, and++ `GET /api/notifications/unread-count` answers 200. Trigger a new one, for example a follow from a second++ account, and check that it arrives.++6. **OpenCoins wallet.** `GET /api/coins/me` returns the same balance that `$COPY` holds for that user. Check it++ with this, run in `$STAGE`:++++ ```bash++ node -e "const d = new (require('better-sqlite3'))('$COPY', { readonly: true });++ console.log(d.prepare('SELECT w.balance FROM wallets w JOIN users u ON u.id = w.user_id WHERE u.username = ?').get('<you>'))"++ ```++++7. **Logs.** `sudo journalctl -u openvibe-network --since "$T1" -p warning` shows no database, PgBouncer or++ Valkey errors.++++**Go:** all seven pass. Close the status-page maintenance. The cutover is done.++**No-go:** any check fails and cannot be fixed within the window. Go to [rollback](#rollback).++++After a go, keep `$COPY` (0400) and the live `data/network.db` untouched for at least 30 days. They are the++rollback and the audit trail. `$STAGE` may be deleted.++++---++++## Rollback++++```bash++ov access run openvibe-ovh rollback network++```++++This brings back the previous release, **20dd7ff**, which serves from `data/network.db` again. Nothing in this++runbook writes to that file: the import read `$COPY`, and the PostgreSQL release never opens `DB_PATH`. The file++is exactly as it was at T0. The old release ignores `DATABASE_URL`, `DATABASE_DIRECT_URL` and `VALKEY_URL`, so++they stay in `network.env`.++++**Go (rollback worked):** `ov access run openvibe-ovh health network` is healthy on 20dd7ff, and `/api/ready`++answers 200. Sign-in works.++++**What is lost:** every write that reached PostgreSQL between T1 and the rollback. That includes:++- new accounts, and profile, username and password changes;++- sessions and refresh tokens issued in that time (those people sign in again);++- notifications, coin transactions and wallet balances;++- follows and blocks;++- preferences and developer projects.++++Events that the PostgreSQL release already published stay published. Events treats a repeated `event_id` as a++duplicate.++++**Replay or accept.** Decide before anything else is attempted. A later cutover empties PostgreSQL and imports++from a new SQLite copy, which discards these rows for good.++++- **Accept** when the rollback came within the window and the list below is empty or trivial. Say so on the++ status page: "changes made between T1 and the rollback were not kept; sign in again if asked".++- **Replay** money and accounts. On PostgreSQL, as the owner (`DATABASE_DIRECT_URL`, from `$STAGE` with++ `node --env-file`, or `psql` on the host without echoing the URL), list them:++++ ```sql++ SELECT id, user_id, app_id, delta, reason, ref, idempotency_key, created_at++ FROM coin_transactions WHERE created_at >= '<T1>' ORDER BY id;++ SELECT id, username, subject_id, created_at FROM users WHERE created_at >= '<T1>' ORDER BY id;++ SELECT 'notifications', COUNT(*) FROM notifications WHERE created_at >= '<T1>'++ UNION ALL SELECT 'user_follows', COUNT(*) FROM user_follows WHERE updated_at >= '<T1>'++ UNION ALL SELECT 'user_blocks', COUNT(*) FROM user_blocks WHERE updated_at >= '<T1>';++ ```++++ - Re-apply each coin transaction through the old release's coins API with its **same `idempotency_key`**, so a++ second replay is a no-op. A row without a key is checked by hand against the wallet first.++ - New accounts are told to sign up again. Their PostgreSQL subject id does not exist on SQLite.++ - Follows and blocks that the person made again on the old release need nothing more.++++ Keep the query output with the incident notes. It holds usernames, so it is not posted publicly.++++After the rollback, read the boot log and `ovhost releases network`, fix the cause with a PR, and rehearse again++on a fresh snapshot (step 2) before a new window.@@@ -13,7 +13,8 @@"migrate-to-postgres": "node scripts/migrate-to-postgres.js","pg-preflight": "node scripts/pg-preflight-collisions.js","follows-import": "node scripts/follows-import.js",-- "follows-preflight": "node scripts/follows-preflight.js"++ "follows-preflight": "node scripts/follows-preflight.js",++ "rehearse-pg-cutover": "node scripts/rehearse-pg-cutover.js"},"license": "MIT","dependencies": {nnew file mode 100644@@@ -0,0 +1,1063 @@++-- OpenVibe.Network SQLite schema at commit 20dd7ff (the last SQLite release, in production before the T2 cutover).++-- Generated from that commit's server booted once on an empty file (initDb() plus the tables its modules create++-- lazily at startup: analytics, account merges/exports/deletions, history, frame cache, …): sqlite_master in++-- creation order. Read by scripts/rehearse-pg-cutover.js to build the seeded rehearsal database. Do not edit by++-- hand: it is history.++++CREATE TABLE users (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ username TEXT UNIQUE NOT NULL,++ email TEXT UNIQUE,++ password_hash TEXT NOT NULL,++ display_name TEXT,++ avatar_url TEXT,++ bio TEXT DEFAULT '',++ role TEXT DEFAULT 'user' CHECK(role IN ('user','streamer','global_mod','admin')),++ profile_color TEXT DEFAULT '#8b5cf6',++ is_banned INTEGER DEFAULT 0,++ ban_reason TEXT,++ token_valid_after TEXT DEFAULT NULL,++ legacy_source TEXT, -- 'live' for migrated accounts++ legacy_id INTEGER, -- original user ID in source platform++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP++ , is_anon INTEGER DEFAULT 0, anon_number INTEGER, name_effect TEXT, particle_effect TEXT, email_verified INTEGER DEFAULT 0, email_verified_at DATETIME, email_bounced_at DATETIME, email_bounce_reason TEXT, history_paused INTEGER DEFAULT 0, subject_id TEXT, merged_into INTEGER, deleted_at TEXT, profile_revision INTEGER NOT NULL DEFAULT 0);++++CREATE TABLE oauth_clients (++ client_id TEXT PRIMARY KEY,++ client_secret TEXT NOT NULL,++ name TEXT NOT NULL,++ redirect_uris TEXT NOT NULL, -- JSON array of allowed redirect URIs++ is_first_party INTEGER DEFAULT 0,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP++ );++++CREATE TABLE oauth_codes (++ code TEXT PRIMARY KEY,++ client_id TEXT NOT NULL,++ user_id INTEGER NOT NULL,++ redirect_uri TEXT NOT NULL,++ scope TEXT DEFAULT 'profile theme',++ expires_at DATETIME NOT NULL,++ used INTEGER DEFAULT 0,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP, code_challenge TEXT, code_challenge_method TEXT, nonce TEXT,++ FOREIGN KEY (client_id) REFERENCES oauth_clients(client_id),++ FOREIGN KEY (user_id) REFERENCES users(id)++ );++++CREATE TABLE oauth_tokens (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ token TEXT UNIQUE NOT NULL,++ client_id TEXT NOT NULL,++ user_id INTEGER NOT NULL,++ scope TEXT DEFAULT 'profile theme',++ expires_at DATETIME NOT NULL,++ revoked INTEGER DEFAULT 0,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP, family_id TEXT, generation INTEGER NOT NULL DEFAULT 0, revoked_reason TEXT, revoked_at DATETIME,++ FOREIGN KEY (client_id) REFERENCES oauth_clients(client_id),++ FOREIGN KEY (user_id) REFERENCES users(id)++ );++++CREATE TABLE user_preferences (++ user_id INTEGER PRIMARY KEY,++ theme_id TEXT DEFAULT 'vibe',++ custom_theme_variables TEXT, -- JSON: custom CSS var overrides++ language TEXT DEFAULT 'en',++ notifications_enabled INTEGER DEFAULT 1,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP, display_prefs TEXT,++ FOREIGN KEY (user_id) REFERENCES users(id)++ );++++CREATE TABLE themes (++ id TEXT PRIMARY KEY,++ name TEXT NOT NULL,++ slug TEXT UNIQUE NOT NULL,++ author_id INTEGER,++ description TEXT DEFAULT '',++ mode TEXT DEFAULT 'dark' CHECK(mode IN ('dark','light')),++ variables TEXT NOT NULL, -- JSON: CSS variable map++ preview_colors TEXT, -- JSON: preview color swatches++ is_builtin INTEGER DEFAULT 0,++ is_public INTEGER DEFAULT 1,++ downloads INTEGER DEFAULT 0,++ rating_sum INTEGER DEFAULT 0,++ rating_count INTEGER DEFAULT 0,++ tags TEXT DEFAULT '[]', -- JSON array++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (author_id) REFERENCES users(id)++ );++++CREATE TABLE linked_accounts (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ service TEXT NOT NULL, -- 'live', 'games', etc.++ service_user_id TEXT NOT NULL,++ linked_at DATETIME DEFAULT CURRENT_TIMESTAMP, service_username TEXT, last_used_at DATETIME,++ FOREIGN KEY (user_id) REFERENCES users(id),++ UNIQUE(service, service_user_id)++ );++++CREATE TABLE audit_log (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER,++ action TEXT NOT NULL,++ details TEXT, -- JSON context++ ip TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP++ );++++CREATE TABLE ip_log (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER,++ ip TEXT NOT NULL,++ action TEXT DEFAULT 'login',++ country TEXT,++ region TEXT,++ city TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id)++ );++++CREATE TABLE site_settings (++ key TEXT PRIMARY KEY,++ value TEXT,++ type TEXT DEFAULT 'string'++ );++++CREATE TABLE url_registry (++ key TEXT PRIMARY KEY,++ label TEXT NOT NULL,++ category TEXT NOT NULL,++ service TEXT NOT NULL,++ scope TEXT NOT NULL,++ type TEXT NOT NULL,++ value TEXT,++ description TEXT,++ updated_by INTEGER,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP++ , source TEXT NOT NULL DEFAULT 'admin');++++CREATE TABLE notifications (++ id TEXT PRIMARY KEY, -- UUID++ user_id INTEGER NOT NULL,++ type TEXT NOT NULL, -- from TYPES enum (FOLLOW, MENTION, etc.)++ category TEXT NOT NULL, -- social, chat, game, stream, economy, etc.++ priority TEXT NOT NULL DEFAULT 'normal', -- low, normal, high, critical++ title TEXT NOT NULL,++ message TEXT,++ icon TEXT,++ sender_id INTEGER, -- who triggered this (nullable)++ sender_name TEXT, -- denormalized for display++ sender_avatar TEXT,++ service TEXT, -- originating service (openvibelive, etc.)++ url TEXT, -- click-through destination++ rich_content TEXT, -- JSON: images, actions, embeds++ is_read INTEGER DEFAULT 0,++ is_dismissed INTEGER DEFAULT 0,++ is_emailed INTEGER DEFAULT 0,++ expires_at DATETIME,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_notif_user_read ON notifications(user_id, is_read, created_at DESC);++++CREATE INDEX idx_notif_user_cat ON notifications(user_id, category);++++CREATE INDEX idx_notif_expires ON notifications(expires_at) WHERE expires_at IS NOT NULL;++++CREATE TABLE notification_preferences (++ user_id INTEGER NOT NULL,++ category TEXT NOT NULL, -- or '*' for global++ enabled INTEGER DEFAULT 1,++ sound INTEGER DEFAULT 1,++ toasts INTEGER DEFAULT 1,++ email INTEGER DEFAULT 0,++ PRIMARY KEY (user_id, category),++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE TABLE email_delivery_log (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ email_type TEXT NOT NULL,++ recipient TEXT NOT NULL,++ subject TEXT,++ status TEXT NOT NULL CHECK(status IN ('sent', 'failed')),++ error_message TEXT,++ user_id INTEGER,++ notification_id TEXT,++ metadata TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE SET NULL,++ FOREIGN KEY (notification_id) REFERENCES notifications(id) ON DELETE SET NULL++ );++++CREATE INDEX idx_email_delivery_created ON email_delivery_log(created_at DESC);++++CREATE INDEX idx_email_delivery_status ON email_delivery_log(status, created_at DESC);++++CREATE INDEX idx_email_delivery_type ON email_delivery_log(email_type, created_at DESC);++++CREATE TABLE password_reset_tokens (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ token_hash TEXT UNIQUE NOT NULL,++ expires_at DATETIME NOT NULL,++ used_at DATETIME,++ requested_ip TEXT,++ requested_user_agent TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_password_reset_user ON password_reset_tokens(user_id, used_at, expires_at);++++CREATE INDEX idx_password_reset_expires ON password_reset_tokens(expires_at);++++CREATE TABLE anon_users (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ anon_number INTEGER UNIQUE NOT NULL,++ fingerprint TEXT, -- browser fingerprint hash (optional)++ session_token TEXT UNIQUE NOT NULL,++ display_name TEXT,++ preferences TEXT DEFAULT '{}', -- JSON++ total_messages INTEGER DEFAULT 0,++ total_commands INTEGER DEFAULT 0,++ first_seen DATETIME DEFAULT CURRENT_TIMESTAMP,++ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP++ , ip TEXT, subject_id TEXT);++++CREATE TABLE user_sessions (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ session_token TEXT UNIQUE NOT NULL,++ device_name TEXT,++ ip TEXT,++ user_agent TEXT,++ is_active INTEGER DEFAULT 1,++ last_used DATETIME DEFAULT CURRENT_TIMESTAMP,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ expires_at DATETIME NOT NULL,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_sessions_user ON user_sessions(user_id, is_active);++++CREATE INDEX idx_sessions_token ON user_sessions(session_token);++++CREATE TABLE user_effects (++ user_id INTEGER NOT NULL,++ effect_type TEXT NOT NULL, -- 'name' or 'particle'++ effect_id TEXT NOT NULL, -- e.g. 'rainbow', 'fire', 'neon'++ is_active INTEGER DEFAULT 0,++ acquired_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ PRIMARY KEY (user_id, effect_type, effect_id),++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE TABLE follows (++ follower_id INTEGER NOT NULL,++ followed_id INTEGER NOT NULL,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ PRIMARY KEY (follower_id, followed_id),++ FOREIGN KEY (follower_id) REFERENCES users(id) ON DELETE CASCADE,++ FOREIGN KEY (followed_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE TABLE wallets (++ user_id INTEGER PRIMARY KEY,++ balance INTEGER NOT NULL DEFAULT 0,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE TABLE coin_transactions (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ app_id TEXT,++ delta INTEGER NOT NULL,++ reason TEXT,++ ref TEXT,++ idempotency_key TEXT UNIQUE,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_coin_tx_user ON coin_transactions(user_id, created_at DESC);++++CREATE INDEX idx_coin_tx_app ON coin_transactions(app_id, created_at DESC);++++CREATE TABLE verification_keys (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ key TEXT UNIQUE NOT NULL,++ target_username TEXT NOT NULL,++ note TEXT DEFAULT '',++ created_by INTEGER NOT NULL,++ used_by INTEGER,++ status TEXT DEFAULT 'active' CHECK(status IN ('active', 'used', 'revoked')),++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ used_at DATETIME,++ FOREIGN KEY (created_by) REFERENCES users(id) ON DELETE CASCADE,++ FOREIGN KEY (used_by) REFERENCES users(id) ON DELETE SET NULL++ );++++CREATE INDEX idx_vkeys_key ON verification_keys(key);++++CREATE INDEX idx_vkeys_target ON verification_keys(target_username);++++CREATE INDEX idx_vkeys_status ON verification_keys(status);++++CREATE TABLE anon_ip_log (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ anon_id INTEGER NOT NULL,++ ip TEXT NOT NULL,++ first_seen DATETIME DEFAULT CURRENT_TIMESTAMP,++ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (anon_id) REFERENCES anon_users(id) ON DELETE CASCADE,++ UNIQUE(anon_id, ip)++ );++++CREATE INDEX idx_anon_ip_log_ip ON anon_ip_log(ip);++++CREATE INDEX idx_anon_ip_log_anon ON anon_ip_log(anon_id);++++CREATE TABLE push_subscriptions (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ endpoint TEXT NOT NULL UNIQUE,++ keys_p256dh TEXT NOT NULL,++ keys_auth TEXT NOT NULL,++ user_agent TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_push_sub_user ON push_subscriptions(user_id);++++CREATE INDEX idx_push_sub_endpoint ON push_subscriptions(endpoint);++++CREATE INDEX idx_oauth_tokens_family ON oauth_tokens(family_id);++++CREATE TABLE email_verification_tokens (++ token_hash TEXT PRIMARY KEY,++ user_id INTEGER NOT NULL,++ email TEXT NOT NULL,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ expires_at DATETIME NOT NULL,++ used_at DATETIME,++ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE++ );++++CREATE INDEX idx_email_verify_user ON email_verification_tokens(user_id, created_at DESC);++++CREATE UNIQUE INDEX idx_linked_user_service ON linked_accounts(user_id, service);++++CREATE TABLE identity_legacy_map (++ source_system TEXT NOT NULL,++ source_type TEXT NOT NULL,++ source_id TEXT NOT NULL,++ subject_id TEXT NOT NULL,++ first_seen_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ verified_at DATETIME,++ metadata TEXT,++ PRIMARY KEY (source_system, source_type, source_id)++ );++++CREATE INDEX idx_identity_legacy_subject ON identity_legacy_map(subject_id);++++CREATE UNIQUE INDEX idx_users_subject_id ON users(subject_id);++++CREATE UNIQUE INDEX idx_anon_users_subject_id ON anon_users(subject_id);++++CREATE TABLE principal_grants (++ client_id TEXT NOT NULL,++ capability TEXT NOT NULL,++ audience TEXT NOT NULL,++ namespaces TEXT NOT NULL DEFAULT '[]',++ granted_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ granted_by TEXT,++ revoked_at DATETIME, expires_at DATETIME, reason TEXT, revoked_by TEXT,++ PRIMARY KEY (client_id, capability, audience)++ );++++CREATE TABLE principal_usage (++ principal TEXT NOT NULL,++ route TEXT NOT NULL,++ capability TEXT NOT NULL,++ auth TEXT NOT NULL,++ allowed INTEGER NOT NULL,++ code TEXT NOT NULL DEFAULT '',++ count INTEGER NOT NULL DEFAULT 0,++ first_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ last_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ PRIMARY KEY (principal, route, auth, allowed, code)++ );++++CREATE TABLE principal_grant_changes (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ client_id TEXT NOT NULL,++ capability TEXT NOT NULL,++ audience TEXT NOT NULL,++ change TEXT NOT NULL CHECK (change IN ('granted', 'updated', 'revoked', 'expired')),++ namespaces TEXT NOT NULL DEFAULT '[]',++ expires_at TEXT,++ reason TEXT,++ actor TEXT,++ event_id TEXT,++ at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))++ );++++CREATE INDEX idx_grant_changes_client ON principal_grant_changes(client_id, id);++++CREATE TABLE user_modules (++ subject_id TEXT NOT NULL,++ namespace TEXT NOT NULL,++ version INTEGER NOT NULL,++ revision INTEGER NOT NULL DEFAULT 0,++ data TEXT NOT NULL,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ updated_by TEXT,++ PRIMARY KEY (subject_id, namespace)++ );++++CREATE TABLE user_module_revisions (++ subject_id TEXT NOT NULL,++ namespace TEXT NOT NULL,++ revision INTEGER NOT NULL,++ PRIMARY KEY (subject_id, namespace)++ );++++CREATE TABLE user_module_retirements (++ namespace TEXT PRIMARY KEY,++ owner TEXT NOT NULL,++ retired_seen_at INTEGER NOT NULL++ );++++CREATE TABLE network_event_outbox (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ event_id TEXT NOT NULL UNIQUE,++ envelope TEXT NOT NULL,++ traceparent TEXT,++ created_at INTEGER NOT NULL,++ attempts INTEGER NOT NULL DEFAULT 0,++ next_attempt_at INTEGER NOT NULL DEFAULT 0,++ sent_at INTEGER,++ seq INTEGER,++ rejected_at INTEGER,++ last_error TEXT++ );++++CREATE INDEX idx_network_event_outbox_due ON network_event_outbox(sent_at, rejected_at, next_attempt_at);++++CREATE TRIGGER user_modules_guard_users_delete BEFORE DELETE ON users++ WHEN OLD.subject_id IS NOT NULL AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)++ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectRemoved or onSubjectMerged first'); END;++++CREATE TRIGGER user_modules_guard_users_rekey BEFORE UPDATE OF subject_id ON users++ WHEN OLD.subject_id IS NOT NULL AND NEW.subject_id IS NOT OLD.subject_id++ AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)++ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectMerged first'); END;++++CREATE TRIGGER user_modules_guard_anon_users_delete BEFORE DELETE ON anon_users++ WHEN OLD.subject_id IS NOT NULL AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)++ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectRemoved or onSubjectMerged first'); END;++++CREATE TRIGGER user_modules_guard_anon_users_rekey BEFORE UPDATE OF subject_id ON anon_users++ WHEN OLD.subject_id IS NOT NULL AND NEW.subject_id IS NOT OLD.subject_id++ AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)++ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectMerged first'); END;++++CREATE TABLE dev_projects (++ id TEXT PRIMARY KEY,++ owner_subject TEXT NOT NULL,++ name TEXT NOT NULL,++ environment_policy TEXT NOT NULL DEFAULT 'sandbox' CHECK (environment_policy IN ('sandbox', 'sandbox+production')),++ allowance TEXT NOT NULL DEFAULT '[]',++ created_at TEXT NOT NULL,++ created_by TEXT NOT NULL,++ archived_at TEXT,++ archived_by TEXT++ );++++CREATE INDEX idx_dev_projects_owner ON dev_projects(owner_subject);++++CREATE TABLE dev_project_members (++ project_id TEXT NOT NULL REFERENCES dev_projects(id),++ subject_id TEXT NOT NULL,++ role TEXT NOT NULL CHECK (role IN ('owner', 'admin', 'developer', 'viewer')),++ added_at TEXT NOT NULL,++ added_by TEXT NOT NULL,++ PRIMARY KEY (project_id, subject_id)++ );++++CREATE INDEX idx_dev_members_subject ON dev_project_members(subject_id);++++CREATE TABLE dev_apps (++ id TEXT PRIMARY KEY,++ project_id TEXT NOT NULL REFERENCES dev_projects(id),++ name TEXT NOT NULL,++ environment TEXT NOT NULL CHECK (environment IN ('sandbox', 'production')),++ oauth_client_id TEXT NOT NULL UNIQUE,++ client_type TEXT NOT NULL CHECK (client_type IN ('confidential', 'public')),++ redirect_uris TEXT NOT NULL DEFAULT '[]',++ created_at TEXT NOT NULL,++ created_by TEXT NOT NULL,++ revoked_at TEXT,++ revoked_by TEXT++ );++++CREATE INDEX idx_dev_apps_project ON dev_apps(project_id);++++CREATE TABLE dev_credentials (++ id TEXT PRIMARY KEY,++ app_id TEXT NOT NULL REFERENCES dev_apps(id),++ secret_hash TEXT NOT NULL UNIQUE,++ hint TEXT NOT NULL,++ created_at TEXT NOT NULL,++ created_by TEXT NOT NULL,++ expires_at TEXT,++ revoked_at TEXT,++ revoked_by TEXT,++ last_used_at TEXT++ );++++CREATE INDEX idx_dev_credentials_app ON dev_credentials(app_id);++++CREATE TABLE dev_grants (++ app_id TEXT NOT NULL REFERENCES dev_apps(id),++ capability TEXT NOT NULL,++ audience TEXT NOT NULL,++ status TEXT NOT NULL CHECK (status IN ('requested', 'approved', 'denied', 'revoked')),++ requested_by TEXT NOT NULL,++ requested_at TEXT NOT NULL,++ decided_by TEXT,++ decided_at TEXT,++ PRIMARY KEY (app_id, capability)++ );++++CREATE TABLE dev_quotas (++ project_id TEXT NOT NULL REFERENCES dev_projects(id),++ capability TEXT NOT NULL,++ limit_value INTEGER NOT NULL CHECK (limit_value >= 0),++ quota_window TEXT NOT NULL CHECK (quota_window IN ('minute', 'hour', 'day', 'month', 'total')),++ unit TEXT NOT NULL DEFAULT 'requests',++ updated_at TEXT NOT NULL,++ updated_by TEXT NOT NULL,++ PRIMARY KEY (project_id, capability)++ );++++CREATE TABLE dev_auth_codes (++ code_hash TEXT PRIMARY KEY,++ app_id TEXT NOT NULL REFERENCES dev_apps(id),++ user_subject TEXT NOT NULL,++ redirect_uri TEXT NOT NULL,++ scope TEXT NOT NULL DEFAULT '',++ code_challenge TEXT NOT NULL,++ expires_at TEXT NOT NULL,++ used INTEGER NOT NULL DEFAULT 0++ );++++CREATE TABLE dev_audit (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ at TEXT NOT NULL,++ project_id TEXT,++ actor TEXT NOT NULL,++ action TEXT NOT NULL,++ target TEXT,++ detail TEXT NOT NULL DEFAULT '{}',++ request_id TEXT,++ event_type TEXT,++ event TEXT++ );++++CREATE INDEX idx_dev_audit_project ON dev_audit(project_id, id);++++CREATE TRIGGER dev_audit_append_only_update BEFORE UPDATE ON dev_audit++ BEGIN SELECT RAISE(ABORT, 'dev_audit is append-only'); END;++++CREATE TRIGGER dev_audit_append_only_delete BEFORE DELETE ON dev_audit++ BEGIN SELECT RAISE(ABORT, 'dev_audit is append-only'); END;++++CREATE TABLE dev_usage_windows (++ project_id TEXT NOT NULL,++ env TEXT NOT NULL CHECK (env IN ('sandbox', 'production')),++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ dimension TEXT NOT NULL DEFAULT '',++ unit TEXT NOT NULL,++ window_start TEXT NOT NULL,++ window_end TEXT NOT NULL,++ quantity INTEGER NOT NULL,++ errors INTEGER NOT NULL,++ error_codes TEXT NOT NULL DEFAULT '{}',++ revision INTEGER NOT NULL DEFAULT 1,++ event_id TEXT NOT NULL,++ recorded_at TEXT NOT NULL,++ PRIMARY KEY (project_id, env, service, capability, dimension, unit, window_start)++ );++++CREATE INDEX idx_dev_usage_windows_when ON dev_usage_windows(window_start);++++CREATE INDEX idx_dev_usage_windows_project ON dev_usage_windows(project_id, recorded_at);++++CREATE TABLE dev_usage_daily (++ project_id TEXT NOT NULL,++ env TEXT NOT NULL,++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ dimension TEXT NOT NULL DEFAULT '',++ unit TEXT NOT NULL,++ day TEXT NOT NULL,++ quantity INTEGER NOT NULL DEFAULT 0,++ errors INTEGER NOT NULL DEFAULT 0,++ error_codes TEXT NOT NULL DEFAULT '{}',++ updated_at TEXT NOT NULL,++ PRIMARY KEY (project_id, env, service, capability, dimension, unit, day)++ );++++CREATE INDEX idx_dev_usage_daily_day ON dev_usage_daily(project_id, day);++++CREATE TABLE dev_usage_errors (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ project_id TEXT NOT NULL,++ env TEXT NOT NULL,++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ at TEXT NOT NULL,++ code TEXT NOT NULL,++ status INTEGER,++ trace_id TEXT,++ ref TEXT NOT NULL DEFAULT '',++ event_id TEXT NOT NULL++ );++++CREATE UNIQUE INDEX idx_dev_usage_errors_once ON dev_usage_errors(project_id, env, service, capability, at, code, ref);++++CREATE INDEX idx_dev_usage_errors_recent ON dev_usage_errors(project_id, at);++++CREATE TABLE user_blocks (++ blocker_subject TEXT NOT NULL,++ blocked_subject TEXT NOT NULL,++ active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),++ revision INTEGER NOT NULL DEFAULT 1 CHECK (revision >= 1),++ created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),++ updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),++ PRIMARY KEY (blocker_subject, blocked_subject),++ CHECK (blocker_subject <> blocked_subject)++ );++++CREATE INDEX idx_user_blocks_blocked ON user_blocks(blocked_subject, active);++++CREATE TABLE user_follows (++ follower_subject TEXT NOT NULL,++ target_type TEXT NOT NULL CHECK (target_type IN ('channel')),++ target_id TEXT NOT NULL,++ active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),++ notify_email INTEGER NOT NULL DEFAULT 1 CHECK (notify_email IN (0, 1)),++ notify_push INTEGER NOT NULL DEFAULT 1 CHECK (notify_push IN (0, 1)),++ revision INTEGER NOT NULL DEFAULT 1 CHECK (revision >= 1),++ source TEXT NOT NULL DEFAULT 'network',++ created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),++ updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),++ PRIMARY KEY (follower_subject, target_type, target_id),++ CHECK (follower_subject <> target_id)++ );++++CREATE INDEX idx_user_follows_target ON user_follows(target_type, target_id, active, created_at);++++CREATE TABLE follow_import_holds (++ source TEXT NOT NULL,++ follower_ref TEXT NOT NULL,++ target_ref TEXT NOT NULL,++ reason TEXT NOT NULL,++ seen_at TEXT NOT NULL,++ PRIMARY KEY (source, follower_ref, target_ref)++ );++++CREATE TABLE status_incidents (++ id TEXT PRIMARY KEY,++ kind TEXT NOT NULL CHECK (kind IN ('incident', 'maintenance')),++ title TEXT NOT NULL,++ severity TEXT CHECK (severity IS NULL OR severity IN ('minor', 'major', 'critical')),++ state TEXT NOT NULL,++ services TEXT NOT NULL DEFAULT '[]',++ starts_at TEXT NOT NULL,++ ends_at TEXT,++ created_by TEXT NOT NULL,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++ );++++CREATE TABLE status_incident_updates (++ incident_id TEXT NOT NULL REFERENCES status_incidents(id),++ at TEXT NOT NULL,++ state TEXT NOT NULL,++ message TEXT NOT NULL,++ author TEXT NOT NULL++ );++++CREATE INDEX idx_status_incident_updates ON status_incident_updates(incident_id, at);++++CREATE TABLE creator_streams (++ stream_id INTEGER PRIMARY KEY,++ creator_subject TEXT NOT NULL,++ title TEXT,++ category TEXT,++ started_at TEXT NOT NULL,++ ended_at TEXT NOT NULL,++ duration_seconds INTEGER NOT NULL DEFAULT 0,++ peak_viewers INTEGER,++ avg_viewers REAL,++ unique_chatters INTEGER,++ messages INTEGER,++ watch_minutes INTEGER,++ received_at TEXT NOT NULL++ );++++CREATE INDEX idx_creator_streams_creator ON creator_streams(creator_subject, started_at);++++CREATE TABLE platform_nodes (++ id TEXT PRIMARY KEY,++ source TEXT NOT NULL,++ doc TEXT NOT NULL,++ status TEXT NOT NULL,++ reported_at TEXT NOT NULL++ );++++CREATE TABLE analytics_events (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ service TEXT NOT NULL,++ event_type TEXT NOT NULL DEFAULT 'pageview',++ path TEXT,++ method TEXT DEFAULT 'GET',++ status_code INTEGER,++ response_time_ms INTEGER,++ user_id INTEGER,++ session_id TEXT,++ ip TEXT,++ country TEXT,++ city TEXT,++ user_agent TEXT,++ referer TEXT,++ is_bot INTEGER DEFAULT 0,++ bot_type TEXT,++ device_type TEXT,++ browser TEXT,++ os TEXT,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP++ , authenticated INTEGER NOT NULL DEFAULT 0);++++CREATE TABLE analytics_hourly (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ service TEXT NOT NULL,++ hour TEXT NOT NULL,++ pageviews INTEGER DEFAULT 0,++ api_calls INTEGER DEFAULT 0,++ unique_visitors INTEGER DEFAULT 0,++ unique_users INTEGER DEFAULT 0,++ bot_hits INTEGER DEFAULT 0,++ avg_response_ms INTEGER DEFAULT 0,++ error_count INTEGER DEFAULT 0,++ bandwidth_bytes INTEGER DEFAULT 0,++ top_paths TEXT,++ top_referers TEXT,++ UNIQUE(service, hour)++ );++++CREATE TABLE analytics_daily (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ service TEXT NOT NULL,++ date TEXT NOT NULL,++ pageviews INTEGER DEFAULT 0,++ api_calls INTEGER DEFAULT 0,++ unique_visitors INTEGER DEFAULT 0,++ unique_users INTEGER DEFAULT 0,++ new_users INTEGER DEFAULT 0,++ bot_hits INTEGER DEFAULT 0,++ avg_response_ms INTEGER DEFAULT 0,++ error_count INTEGER DEFAULT 0,++ top_paths TEXT,++ top_referers TEXT,++ top_countries TEXT,++ device_breakdown TEXT,++ browser_breakdown TEXT,++ UNIQUE(service, date)++ );++++CREATE TABLE analytics_rate_tracking (++ ip TEXT NOT NULL,++ window_start INTEGER NOT NULL,++ hit_count INTEGER DEFAULT 1,++ PRIMARY KEY (ip, window_start)++ );++++CREATE TABLE analytics_visitor_days (++ service TEXT NOT NULL,++ day TEXT NOT NULL,++ hour TEXT NOT NULL,++ vhash TEXT NOT NULL,++ authenticated INTEGER NOT NULL DEFAULT 0,++ PRIMARY KEY (service, hour, vhash)++ );++++CREATE TABLE analytics_day_salts (++ service TEXT NOT NULL,++ day TEXT NOT NULL,++ salt TEXT NOT NULL,++ PRIMARY KEY (service, day)++ );++++CREATE INDEX idx_analytics_events_created ON analytics_events(created_at);++++CREATE INDEX idx_analytics_events_service ON analytics_events(service, created_at);++++CREATE INDEX idx_analytics_events_path ON analytics_events(service, path, created_at);++++CREATE INDEX idx_analytics_events_bot ON analytics_events(is_bot, created_at);++++CREATE INDEX idx_analytics_hourly_lookup ON analytics_hourly(service, hour);++++CREATE INDEX idx_analytics_daily_lookup ON analytics_daily(service, date);++++CREATE INDEX idx_analytics_visitor_days_day ON analytics_visitor_days(service, day);++++CREATE TABLE moderation_audit (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ event_id TEXT NOT NULL UNIQUE,++ service TEXT NOT NULL,++ action TEXT NOT NULL,++ actor_subject TEXT,++ target_type TEXT,++ target_id TEXT,++ target_subject TEXT,++ scope TEXT,++ reason TEXT,++ details TEXT NOT NULL DEFAULT '{}',++ occurred_at TEXT NOT NULL,++ recorded_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))++ );++++CREATE INDEX idx_modaudit_when ON moderation_audit(occurred_at);++++CREATE INDEX idx_modaudit_service ON moderation_audit(service, id);++++CREATE INDEX idx_modaudit_actor ON moderation_audit(actor_subject, id);++++CREATE TABLE network_event_inbox (++ consumer TEXT NOT NULL,++ event_id TEXT NOT NULL,++ processed_at INTEGER NOT NULL,++ PRIMARY KEY (consumer, event_id)++ );++++CREATE TABLE subject_aliases (++ alias_id TEXT PRIMARY KEY,++ subject_id TEXT NOT NULL,++ merge_id TEXT NOT NULL,++ merged_at TEXT NOT NULL,++ merged_by TEXT NOT NULL++ );++++CREATE INDEX idx_subject_aliases_subject ON subject_aliases(subject_id);++++CREATE TABLE account_merges (++ id TEXT PRIMARY KEY,++ from_subject TEXT NOT NULL UNIQUE,++ into_subject TEXT NOT NULL,++ from_user_id INTEGER NOT NULL,++ into_user_id INTEGER NOT NULL,++ initiated_by TEXT NOT NULL CHECK (initiated_by IN ('person', 'staff')),++ actor_subject TEXT NOT NULL,++ reason TEXT,++ moved TEXT NOT NULL,++ pre_state TEXT,++ merged_at TEXT NOT NULL,++ split_until TEXT NOT NULL,++ reduced_at TEXT++ );++++CREATE TABLE account_merge_intents (++ id TEXT PRIMARY KEY,++ into_user_id INTEGER NOT NULL,++ created_at INTEGER NOT NULL,++ expires_at INTEGER NOT NULL,++ used_at INTEGER++ );++++CREATE TABLE mod_principals (++ mod_id TEXT PRIMARY KEY,++ owner TEXT NOT NULL,++ runtime TEXT NOT NULL,++ name TEXT,++ version TEXT,++ publisher TEXT,++ manifest TEXT NOT NULL,++ status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'revoked')),++ revision INTEGER NOT NULL DEFAULT 1,++ created_at TEXT NOT NULL,++ created_by TEXT,++ updated_at TEXT NOT NULL,++ revoked_at TEXT,++ revoked_by TEXT++ );++++CREATE TABLE mod_principal_grants (++ mod_id TEXT NOT NULL,++ capability TEXT NOT NULL,++ state TEXT NOT NULL CHECK (state IN ('pending', 'approved', 'revoked')),++ changed_at TEXT NOT NULL,++ changed_by TEXT,++ PRIMARY KEY (mod_id, capability)++ );++++CREATE TABLE account_exports (++ id TEXT PRIMARY KEY,++ user_id INTEGER NOT NULL,++ subject TEXT NOT NULL,++ status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'ready', 'partial', 'expired', 'failed')),++ expected TEXT NOT NULL DEFAULT '[]',++ requested_at TEXT NOT NULL,++ deadline TEXT NOT NULL,++ ready_at TEXT,++ expires_at TEXT,++ size_bytes INTEGER,++ services TEXT,++ error TEXT++ );++++CREATE INDEX idx_account_exports_user ON account_exports(user_id, requested_at);++++CREATE TABLE account_export_parts (++ export_id TEXT NOT NULL,++ service TEXT NOT NULL,++ received_at TEXT NOT NULL,++ files INTEGER NOT NULL,++ bytes INTEGER NOT NULL,++ body TEXT,++ PRIMARY KEY (export_id, service)++ );++++CREATE TABLE account_deletions (++ id TEXT PRIMARY KEY,++ user_id INTEGER NOT NULL,++ subject TEXT NOT NULL,++ status TEXT NOT NULL DEFAULT 'scheduled' CHECK (status IN ('scheduled', 'cancelled', 'deleted')),++ requested_at TEXT NOT NULL,++ delete_after TEXT NOT NULL,++ cancelled_at TEXT,++ cancelled_by TEXT,++ deleted_at TEXT,++ expected TEXT,++ erased TEXT++ );++++CREATE UNIQUE INDEX idx_account_deletions_open ON account_deletions(user_id) WHERE status = 'scheduled';++++CREATE TABLE account_deletion_confirmations (++ deletion_id TEXT NOT NULL,++ service TEXT NOT NULL,++ received_at TEXT NOT NULL,++ completed_at TEXT NOT NULL,++ erased TEXT NOT NULL,++ retained TEXT NOT NULL,++ PRIMARY KEY (deletion_id, service)++ );++++CREATE TABLE user_profile_changes (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ changed TEXT NOT NULL,++ at INTEGER NOT NULL++ );++++CREATE INDEX idx_user_profile_changes_user ON user_profile_changes(user_id);++++CREATE TRIGGER users_profile_changed AFTER UPDATE OF username, display_name, avatar_url, profile_color, role, is_banned ON users++ WHEN COALESCE(NEW.is_anon, 0) = 0 AND (OLD.username IS NOT NEW.username OR OLD.display_name IS NOT NEW.display_name OR OLD.avatar_url IS NOT NEW.avatar_url OR OLD.profile_color IS NOT NEW.profile_color OR OLD.role IS NOT NEW.role OR OLD.is_banned IS NOT NEW.is_banned)++ BEGIN INSERT INTO user_profile_changes (user_id, changed, at) VALUES (NEW.id, trim((CASE WHEN OLD.username IS NOT NEW.username THEN 'username ' ELSE '' END) || (CASE WHEN OLD.display_name IS NOT NEW.display_name THEN 'display_name ' ELSE '' END) || (CASE WHEN OLD.avatar_url IS NOT NEW.avatar_url THEN 'avatar_url ' ELSE '' END) || (CASE WHEN OLD.profile_color IS NOT NEW.profile_color THEN 'profile_color ' ELSE '' END) || (CASE WHEN OLD.role IS NOT NEW.role THEN 'role ' ELSE '' END) || (CASE WHEN OLD.is_banned IS NOT NEW.is_banned THEN 'banned ' ELSE '' END)), CAST(strftime('%s', 'now') AS INTEGER) * 1000); END;++++CREATE TRIGGER users_profile_created AFTER INSERT ON users++ WHEN COALESCE(NEW.is_anon, 0) = 0++ BEGIN INSERT INTO user_profile_changes (user_id, changed, at) VALUES (NEW.id, 'created', CAST(strftime('%s', 'now') AS INTEGER) * 1000); END;++++CREATE TABLE user_history (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ service TEXT,++ sub TEXT,++ type TEXT NOT NULL DEFAULT 'page',++ title TEXT NOT NULL,++ url TEXT NOT NULL,++ icon TEXT,++ meta TEXT,++ hits INTEGER NOT NULL DEFAULT 1,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ FOREIGN KEY (user_id) REFERENCES users(id)++ );++++CREATE INDEX idx_user_history_user_time ON user_history(user_id, updated_at DESC);++++CREATE INDEX idx_user_history_user_url ON user_history(user_id, url);++++CREATE TABLE frame_cache (key TEXT PRIMARY KEY, value TEXT NOT NULL, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);++++CREATE TABLE frame_hits (day TEXT NOT NULL, host TEXT NOT NULL, hits INTEGER NOT NULL DEFAULT 0, PRIMARY KEY (day, host));++++CREATE TABLE tool_domains (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ tool_id TEXT NOT NULL,++ host TEXT NOT NULL UNIQUE,++ role TEXT NOT NULL DEFAULT 'alias' CHECK (role IN ('canonical', 'short', 'alias', 'mirror')),++ enabled INTEGER NOT NULL DEFAULT 1,++ note TEXT,++ created_by INTEGER,++ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,++ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP++ );++++CREATE INDEX idx_tool_domains_tool ON tool_domains(tool_id, role, enabled);++++CREATE TABLE operator_alerts (++ source TEXT NOT NULL,++ fingerprint TEXT NOT NULL,++ name TEXT NOT NULL,++ severity TEXT NOT NULL,++ summary TEXT NOT NULL,++ description TEXT,++ service TEXT,++ state TEXT NOT NULL CHECK (state IN ('firing', 'resolved')),++ started_at TEXT NOT NULL,++ opened_at INTEGER NOT NULL,++ last_seen_at INTEGER NOT NULL,++ notified_at INTEGER,++ resolved_at INTEGER, notices TEXT, reopened INTEGER NOT NULL DEFAULT 0,++ PRIMARY KEY (source, fingerprint)++ );++++CREATE TABLE stream_live_announcements (++ id INTEGER PRIMARY KEY AUTOINCREMENT, streamer_key TEXT NOT NULL, stream_id TEXT, sent_at DATETIME DEFAULT CURRENT_TIMESTAMP);++++CREATE INDEX idx_sla_key ON stream_live_announcements(streamer_key, sent_at DESC);++++CREATE TABLE username_history (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ user_id INTEGER NOT NULL,++ old_username TEXT NOT NULL,++ new_username TEXT NOT NULL,++ changed_by INTEGER,++ changed_at DATETIME DEFAULT CURRENT_TIMESTAMP++ );++++CREATE INDEX idx_username_history_old ON username_history(old_username COLLATE NOCASE);++++CREATE INDEX idx_username_history_user ON username_history(user_id);++++CREATE TABLE admin_rate_limits (++ user_id INTEGER NOT NULL,++ action TEXT NOT NULL,++ last_at INTEGER NOT NULL,++ PRIMARY KEY (user_id, action)++ );@@@ -2,7 +2,7 @@'use strict';/*** The one-time move of Network's SQLite database (DB_PATH) into its PostgreSQL schema (plan T2, ADR-035; the-- * procedure is openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by runSqliteMigration).++ * procedure is openvibe-sdk docs/migrating-to-postgresql.md, section 6, the steps of openvibe-sdk's runSqliteMigration).* Opus runs it at the cutover, after scripts/pg-preflight-collisions.js.** node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]@@@ -19,10 +19,16 @@** The legacy integer-keyed `follows` table is imported like every other: server/auth/routes.js and* server/internal/routes.js still read it (the subject-keyed user_follows is the forward model).++ *++ * The steps are runSqliteMigration's (migrate, cleaning options, importSqlite with truncate, report), run here++ * so QUIET_TRIGGERS can be off while the rows are copied. A run that dies between the two ALTERs leaves them++ * off: run the import again (it re-enables them at the end), never boot on such a database.*/require('dotenv').config();const path = require('path');--const { runSqliteMigration } = require('openvibe-sdk/db');++const { createDb, importSqlite } = require('openvibe-sdk/db');++// cleaningOptions is sqlite-cli.js's export, beside openvibe-sdk/db's entry (the package exports map does not list it).++const { cleaningOptions } = require(path.join(path.dirname(require.resolve('openvibe-sdk/db')), 'sqlite-cli.js'));const config = require('../server/config');const MIGRATIONS = path.join(__dirname, '..', 'migrations');@@@ -30,16 +36,62 @@ const MIGRATIONS = path.join(__dirname, '..', 'migrations');// importSqlite's per-table options. Empty: every PostgreSQL column is fed by the same-named SQLite one.const TABLES = {};--// SQLite tables the import leaves behind. Empty: even the SDK's SQLite outbox/inbox rows are carried over--// (unsent events keep their place in the PostgreSQL outbox).--const SKIP_SOURCE = [];++// SQLite tables the import leaves behind. The SDK's SQLite outbox/inbox rows are carried over (unsent events++// keep their place in the PostgreSQL outbox). Left behind:++// analytics_rate_tracking openvibe-shared's old per-IP bot-rate counters. The rate check keeps them in++// memory only now and PostgreSQL has no such table: IPs are never carried over.++const SKIP_SOURCE = ['analytics_rate_tracking'];++++// Triggers that derive rows from an insert. The import copies those rows from SQLite as they are, so these are++// off while it runs: users_profile_created would add a 'created' change for every imported account (a key++// collision with the copied user_profile_changes rows, and one profile event per account at the first drain).++const QUIET_TRIGGERS = [['users', 'users_profile_created']];// The retired SQLite file: --sqlite, else DB_PATH, else data/network.db (relative to the repo root).const DEFAULT_SQLITE = path.resolve(__dirname, '..', process.env.DB_PATH || path.join('data', 'network.db'));++/** Migrate `owner` (a createDb() handle, the owner role) and copy `sqlite` into it. → importSqlite's report + cleaned */++async function importInto(owner, sqlite, { log = { log() {}, warn: console.warn, error: console.error } } = {}) {++ await owner.migrate({ dir: MIGRATIONS, log });++ const options = await cleaningOptions(owner, TABLES);++ for (const [table, trigger] of QUIET_TRIGGERS) await owner.exec(`ALTER TABLE ${table} DISABLE TRIGGER ${trigger}`);++ try {++ const report = await importSqlite({ sqlite, db: owner, truncate: true, tables: options.tables, skipSource: SKIP_SOURCE, log });++ return { ...report, cleaned: options.cleaned() };++ } finally {++ for (const [table, trigger] of QUIET_TRIGGERS) await owner.exec(`ALTER TABLE ${table} ENABLE TRIGGER ${trigger}`);++ }++}++++async function main(argv = process.argv.slice(2), out = console.log) {++ const opt = (name) => { const i = argv.indexOf(`--${name}`); return i >= 0 ? argv[i + 1] : null; };++ const flag = (name) => argv.includes(`--${name}`);++ const file = path.resolve(opt('sqlite') || DEFAULT_SQLITE);++ const quiet = { log() {}, warn: console.warn, error: console.error };++ let owner;++ if (flag('pglite')) owner = createDb({ pglite: true, service: 'network-import', log: quiet });++ else {++ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: the import runs as the owner on a direct connection');++ owner = createDb({ url: config.db.directUrl, service: 'network-import', max: 2, log: quiet });++ }++ try {++ const t0 = Date.now();++ const report = await importInto(owner, file, { log: quiet });++ if (flag('json')) out(JSON.stringify({ sqlite: file, into: owner.store, ...report }, null, 2));++ else {++ out(`import ${file} → ${owner.store} (${Date.now() - t0} ms): ${report.ok ? 'OK' : 'PROBLEMS'}`);++ for (const t of report.tables) out(` ${t.table.padEnd(30)} ${String(t.rows).padStart(8)} rows ${t.checksum || '-'}${t.source !== t.table ? ` (from ${t.source})` : ''}`);++ for (const c of report.cleaned) out(` cleaned: ${c.column}: ${c.values} value(s) with a NUL or an unpaired surrogate`);++ for (const p of report.problems) out(` problem: ${p.table}: ${p.problem}`);++ }++ return report.ok ? 0 : 1;++ } finally {++ await owner.close();++ }++}++if (require.main === module) {-- runSqliteMigration({ service: 'network', sqlite: DEFAULT_SQLITE, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES, skipSource: SKIP_SOURCE })-- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });++ main().then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });}--module.exports = { TABLES, SKIP_SOURCE, MIGRATIONS, DEFAULT_SQLITE };++module.exports = { TABLES, SKIP_SOURCE, QUIET_TRIGGERS, MIGRATIONS, DEFAULT_SQLITE, importInto };nnew file mode 100644@@@ -0,0 +1,99 @@++#!/usr/bin/env node++'use strict';++/**++ * The T2 cutover's row-count parity check (docs/cutover-t2-postgres.md, step 6): after++ * scripts/migrate-to-postgres.js, every SQLite table must hold exactly as many rows in PostgreSQL. It is++ * independent of the import's own verification (openvibe-sdk importSqlite counts and checksums what it++ * copied): this one starts from the SQLite side, so a table the import never touched shows up as a mismatch.++ *++ * node scripts/pg-row-parity.js --sqlite <file> [--json] DATABASE_DIRECT_URL, else DATABASE_URL++ * node scripts/pg-row-parity.js --sqlite <file> --pglite <dir> a PGlite directory (the rehearsal)++ *++ * Per table: SQLite `SELECT COUNT(*) FROM "<t>"` against PostgreSQL `SELECT COUNT(*) FROM "<t>"` (both++ * sides read-only). Compared: every SQLite table except SQLite's own, FTS5 virtual tables and their shadow++ * tables, and migrate-to-postgres.js's SKIP_SOURCE. PostgreSQL-only tables (new in T2) are listed, not++ * compared. The triggers the import turns off (QUIET_TRIGGERS) must be on again. Run it before the new++ * release boots: its boot seeds rows (OAuth clients, themes, grants) that the SQLite file never had. Prints counts only, never a row. Exit 0 = parity, 1 = a mismatch, 2 = unusable input.++ */++const fs = require('fs');++const path = require('path');++const { parseArgs } = require('./lib/db-ops');++const { TABLES, SKIP_SOURCE, QUIET_TRIGGERS } = require('./migrate-to-postgres');++++const NAME_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;++const quote = (t) => { if (!NAME_RE.test(t)) throw new Error(`unexpected table name ${JSON.stringify(t)}`); return `"${t}"`; };++++/** The SQLite tables the import must carry over (importSqlite's own rule), each with its PostgreSQL name. */++function sourceTables(src) {++ const all = src.prepare("SELECT name, sql FROM sqlite_master WHERE type = 'table' ORDER BY name").all();++ const virtuals = all.filter((t) => /^CREATE VIRTUAL TABLE/i.test(t.sql || '')).map((t) => t.name);++ const renamed = Object.fromEntries(Object.entries(TABLES).filter(([, o]) => o && o.from).map(([to, o]) => [o.from, to]));++ return all.map((t) => t.name)++ .filter((n) => !n.startsWith('sqlite_') && !SKIP_SOURCE.includes(n) && !virtuals.some((v) => n === v || n.startsWith(`${v}_`)))++ .map((n) => ({ source: n, target: renamed[n] || n }));++}++++/**++ * Compare row counts. `src` is an open better-sqlite3 Database, `db` a createDb() handle on the target.++ * → { ok, tables: { [target]: { sqlite, pg } }, pg_only: { [table]: rows }, problems: [] }++ */++async function rowParity(src, db) {++ const report = { ok: true, tables: {}, pg_only: {}, problems: [] };++ const pgTables = new Set((await db.many(`SELECT tablename FROM pg_tables++ WHERE schemaname = current_schema() AND tablename <> 'ov_migrations'`)).map((r) => r.tablename));++ const compared = new Set();++ for (const { source, target } of sourceTables(src)) {++ const sqlite = src.prepare(`SELECT COUNT(*) AS n FROM ${quote(source)}`).get().n;++ const pg = pgTables.has(target) ? Number(await db.value(`SELECT COUNT(*) FROM ${quote(target)}`)) : null;++ report.tables[target] = { sqlite, pg };++ compared.add(target);++ if (pg === null) { report.ok = false; report.problems.push({ table: target, problem: 'no PostgreSQL table' }); }++ else if (pg !== sqlite) { report.ok = false; report.problems.push({ table: target, problem: `row count differs: SQLite ${sqlite}, PostgreSQL ${pg}` }); }++ }++ for (const t of [...pgTables].sort()) if (!compared.has(t)) report.pg_only[t] = Number(await db.value(`SELECT COUNT(*) FROM ${quote(t)}`));++ // The import turns these off while it copies; an import that died half-way leaves them off.++ for (const [table, trigger] of QUIET_TRIGGERS) {++ if (!pgTables.has(table)) continue;++ const t = quote(trigger);++ // quote() validates the name and identifier-quotes it; tgname is a text column, so the++ // comparison takes that quoted value as a string literal (the name itself holds no quotes).++ const state = await db.value(++ `SELECT tgenabled::text FROM pg_trigger WHERE tgname = ${t.replace(/"/g, "'")} AND tgrelid = '${quote(table)}'::regclass`,++ );++ if (state !== 'O') { report.ok = false; report.problems.push({ table, problem: `trigger ${trigger} is ${state ? 'disabled' : 'missing'}: run migrate-to-postgres.js again` }); }++ }++ return report;++}++++async function main(argv = process.argv.slice(2)) {++ let args;++ try { args = parseArgs(argv, { flags: ['json'], values: ['sqlite', 'pglite'] }); } catch (e) { console.error(e.message); return 2; }++ if (args.help || !args.sqlite) { console.error('usage: node scripts/pg-row-parity.js --sqlite <file> [--pglite <dir>] [--json]'); return 2; }++ const file = path.resolve(args.sqlite);++ if (!fs.existsSync(file)) { console.error(`no such SQLite database: ${file}`); return 2; }++ const url = process.env.DATABASE_DIRECT_URL || process.env.DATABASE_URL;++ if (!args.pglite && !url) { console.error('DATABASE_DIRECT_URL (or DATABASE_URL) is not set, and no --pglite <dir>'); return 2; }++ const { createDb } = require('openvibe-sdk/db');++ const src = new (require('better-sqlite3'))(file, { readonly: true, fileMustExist: true });++ const quiet = { log() {}, warn: console.warn, error: console.error };++ const db = args.pglite ? createDb({ pglite: path.resolve(args.pglite), service: 'network-parity', log: quiet })++ : createDb({ url, service: 'network-parity', max: 1, log: quiet });++ try {++ const report = await rowParity(src, db);++ if (args.json) console.log(JSON.stringify({ sqlite: file, ...report }, null, 2));++ else {++ console.log(`row parity ${file} → ${db.store}: ${report.ok ? 'OK' : 'MISMATCH'}`);++ for (const [t, c] of Object.entries(report.tables)) console.log(` ${t.padEnd(30)} ${String(c.sqlite).padStart(8)} ${String(c.pg ?? '-').padStart(8)}${c.sqlite === c.pg ? '' : ' ✗'}`);++ for (const [t, n] of Object.entries(report.pg_only)) console.log(` ${t.padEnd(30)} ${'-'.padStart(8)} ${String(n).padStart(8)} (PostgreSQL only)`);++ for (const p of report.problems) console.log(` problem: ${p.table}: ${p.problem}`);++ }++ return report.ok ? 0 : 1;++ } finally {++ src.close();++ await db.close();++ }++}++++if (require.main === module) main().then((code) => process.exit(code), (err) => { console.error(`pg-row-parity failed: ${err.message}`); process.exit(2); });++++module.exports = { rowParity, sourceTables };nnew file mode 100644@@@ -0,0 +1,474 @@++#!/usr/bin/env node++'use strict';++/**++ * A repeatable rehearsal of the T2 PostgreSQL cutover (docs/cutover-t2-postgres.md), end to end on scratch++ * stores: the SQLite database → pg-preflight → migrate-to-postgres → row-count parity → the server booted on++ * the result, /api/ready green. Nothing outside a temporary directory (and, on the containers, a database of++ * its own, dropped at the end) is written; the SQLite file is only ever opened read-only.++ *++ * npm run rehearse-pg-cutover a seeded database with commit 20dd7ff's schema++ * npm run rehearse-pg-cutover -- --sqlite /path/to/copy.db a real snapshot (a copy, never the live file)++ * node scripts/rehearse-pg-cutover.js --write-fixture <file> only write the seeded SQLite database, then exit++ *++ * --store auto|pglite|containers|env where the PostgreSQL side lives (default auto: containers, else pglite;++ * auto never uses inherited DATABASE_URLs)++ * env only when given explicitly. DATABASE_DIRECT_URL + DATABASE_URL: a scratch database you made for++ * this. Both must reach that same database (a probe table made through one is read through the++ * other) and it must be empty through both. The import empties every Network table there, so++ * a target where any table already holds a row is refused unless --truncate-target is given.++ * Never point it at the production database.++ * containers openvibe-sdk's test services (eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)"):++ * OV_TEST_PG_DIRECT_URL. A throwaway database network_rehearsal_* with roles shaped like++ * production's: the owner migrates and imports, a DML-only role (statement_timeout 15 s) serves.++ * A database, not a schema in the shared one: an import reads information_schema, which the++ * shared test database's thousands of test schemas slow past the 15 s query cap. The container++ * PgBouncer routes only that shared database, so here the server connects directly (npm run++ * test:pg covers serving through PgBouncer). OV_TEST_VALKEY_URL, when set, backs the limit++ * counters under a prefix of its own.++ * pglite an embedded PGlite directory: nothing to set up.++ * --keep keep the temporary directory (its path is in the JSON line as "dir")++ *++ * On PostgreSQL the import is the real `node scripts/migrate-to-postgres.js --sqlite <file>` with the target's++ * DATABASE_DIRECT_URL. PGlite has no URL, so there the script runs as its own --pglite dry run and the same++ * import (its importInto: migrations, TABLES, SKIP_SOURCE, QUIET_TRIGGERS) fills the PGlite directory the++ * server then boots on. On PostgreSQL the server boots with NODE_ENV=production (DATABASE_URL required, the++ * RS256 signing key required); on PGlite with NODE_ENV=test (production refuses PGlite). Either way /api/ready++ * must answer 200 with `db` and `signing_key` ok.++ *++ * Progress goes to stderr. The last line on stdout is one JSON object, counts only, never a row:++ * {"ok":true,"tables":{"users":{"sqlite":6,"pg":6},…},"ready":{"ready":true,"status":…,"db":"ok","signing_key":"ok"},…}++ * Exit 0 only when ok: preflight clean, import verified, every table's count equal, and ready.++ */++const crypto = require('crypto');++const fs = require('fs');++const net = require('net');++const os = require('os');++const path = require('path');++const { spawn } = require('child_process');++const { parseArgs, ROOT } = require('./lib/db-ops');++++const FIXTURE_SCHEMA = path.join(__dirname, 'fixtures', 'network-sqlite-20dd7ff.sql');++const MIGRATIONS = path.join(ROOT, 'migrations');++const log = (...a) => console.error('[rehearse]', ...a);++++// ── The seeded database ──────────────────────────────────────++// Commit 20dd7ff's schema (scripts/fixtures/network-sqlite-20dd7ff.sql) with rows shaped like production's:++// accounts (mixed-case names, an anonymous and a banned one), OAuth clients/codes/token families, notifications++// (JSON rich content, one message holding a NUL that PostgreSQL refuses), the wallet ledger, legacy and++// subject follows, blocks, sessions, an unsent outbox event, the profile changes 20dd7ff's users_profile_created++// trigger writes, an old per-IP rate counter. SQLite CURRENT_TIMESTAMP text throughout.++const SUBJECT = (n) => `usr_rehearsal${String(n).padStart(4, '0')}`;++function seedFixture(file) {++ const Database = require('better-sqlite3');++ const db = new Database(file);++ db.pragma('foreign_keys = ON');++ db.exec(fs.readFileSync(FIXTURE_SCHEMA, 'utf8'));++ const ins = (table, rows) => {++ const cols = Object.keys(rows[0]);++ const st = db.prepare(`INSERT INTO ${table} (${cols.join(', ')}) VALUES (${cols.map((c) => `@${c}`).join(', ')})`);++ for (const r of rows) st.run(r);++ };++ const at = (d, h = 12) => `2026-0${d}-1${d} ${String(h).padStart(2, '0')}:0${d}:3${d}`;++ const hash = (n) => `$2a$10$rehearsalfixturehashnotarealpasswordhash${String(n).padStart(13, '0')}`;++ // Opaque token strings, generated per run: the fixture holds row shapes, not credentials.++ const rtToken = (family, gen) => `rt-family-${family}-gen${gen}-${crypto.randomBytes(8).toString('hex')}`;++ db.transaction(() => {++ ins('users', [++ { id: 1, username: 'alex', email: '[email protected]', password_hash: hash(1), display_name: 'Alex', role: 'admin', email_verified: 1, email_verified_at: at(1), subject_id: SUBJECT(1), created_at: at(1), updated_at: at(2), last_seen: at(3), is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },++ { id: 2, username: 'Bea_Streams', email: '[email protected]', password_hash: hash(2), display_name: 'Bea ✨ Streams', role: 'streamer', email_verified: 1, email_verified_at: at(1), subject_id: SUBJECT(2), created_at: at(1), updated_at: at(1), last_seen: at(4), is_anon: 0, history_paused: 1, is_banned: 0, ban_reason: null, anon_number: null },++ { id: 3, username: 'carol', email: null, password_hash: hash(3), display_name: null, role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(3), created_at: at(2), updated_at: at(2), last_seen: null, is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },++ { id: 4, username: 'DanTheMod', email: '[email protected]', password_hash: hash(4), display_name: 'Dan', role: 'global_mod', email_verified: 1, email_verified_at: at(2), subject_id: SUBJECT(4), created_at: at(2), updated_at: at(3), last_seen: at(5), is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },++ { id: 5, username: 'spam_account', email: '[email protected]', password_hash: hash(5), display_name: 'Spam', role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(5), created_at: at(3), updated_at: at(3), last_seen: at(3), is_anon: 0, history_paused: 0, is_banned: 1, ban_reason: 'spam', anon_number: null },++ { id: 6, username: 'anon_7', email: null, password_hash: hash(6), display_name: 'Anon #7', role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(6), created_at: at(4), updated_at: at(4), last_seen: at(4), is_anon: 1, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: 7 },++ ]);++ ins('oauth_clients', [++ { client_id: 'live', client_secret: 'rehearsal-client-secret-live', name: 'OpenVibe.Live', redirect_uris: '["https://openvibe.live/auth/callback"]', is_first_party: 1, created_at: at(1) },++ { client_id: 'chat', client_secret: 'rehearsal-client-secret-chat', name: 'OpenVibe.Chat', redirect_uris: '["https://openvibe.chat/auth/callback"]', is_first_party: 1, created_at: at(1) },++ ]);++ ins('oauth_codes', [++ { code: 'code-used-1', client_id: 'live', user_id: 2, redirect_uri: 'https://openvibe.live/auth/callback', scope: 'openid profile', expires_at: at(4, 13), used: 1, code_challenge: 'E9Melhoa2OwvFrEMTJguCHaoeK1t8URWbuGJSstw-cM', code_challenge_method: 'S256', nonce: 'n-1', created_at: at(4) },++ { code: 'code-open-2', client_id: 'chat', user_id: 3, redirect_uri: 'https://openvibe.chat/auth/callback', scope: 'openid', expires_at: at(5, 13), used: 0, code_challenge: null, code_challenge_method: null, nonce: null, created_at: at(5) },++ ]);++ ins('oauth_tokens', [++ { id: 1, token: rtToken('a', 1), client_id: 'live', user_id: 2, scope: 'openid profile', expires_at: at(9), revoked: 1, family_id: 'fam-a', generation: 1, revoked_reason: 'rotated', revoked_at: at(5), created_at: at(4) },++ { id: 2, token: rtToken('a', 2), client_id: 'live', user_id: 2, scope: 'openid profile', expires_at: at(9), revoked: 0, family_id: 'fam-a', generation: 2, revoked_reason: null, revoked_at: null, created_at: at(5) },++ { id: 3, token: rtToken('b', 1), client_id: 'chat', user_id: 1, scope: 'openid', expires_at: at(9), revoked: 0, family_id: 'fam-b', generation: 1, revoked_reason: null, revoked_at: null, created_at: at(5) },++ ]);++ ins('user_preferences', [++ { user_id: 1, theme_id: 'midnight', custom_theme_variables: '{"--accent":"#8b5cf6"}', language: 'en', notifications_enabled: 1, display_prefs: '{"compact":true,"motion":"reduce"}', updated_at: at(2) },++ { user_id: 2, theme_id: null, custom_theme_variables: null, language: 'fr', notifications_enabled: 0, display_prefs: null, updated_at: at(3) },++ ]);++ ins('notifications', [++ { id: 'ntf-0001', user_id: 1, type: 'follow', category: 'social', priority: 'normal', title: 'Bea followed you', message: 'Say hi!', sender_id: 2, sender_name: 'Bea', service: 'network', url: '/u/Bea_Streams', rich_content: '{"actor":{"subject":"usr_rehearsal0002"}}', is_read: 1, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(5) },++ { id: 'ntf-0002', user_id: 1, type: 'live', category: 'streams', priority: 'high', title: 'Bea is live', message: 'Speedrun night', sender_id: 2, sender_name: 'Bea', service: 'live', url: 'https://openvibe.live/Bea_Streams', rich_content: null, is_read: 0, is_dismissed: 0, is_emailed: 1, expires_at: at(9), created_at: at(6) },++ { id: 'ntf-0003', user_id: 2, type: 'system', category: 'system', priority: 'normal', title: 'Welcome', message: 'legacy text with a NUL here:\u0000end', sender_id: null, sender_name: null, service: 'network', url: null, rich_content: '{"steps":["profile","theme"]}', is_read: 0, is_dismissed: 1, is_emailed: 0, expires_at: null, created_at: at(4) },++ { id: 'ntf-0004', user_id: 3, type: 'coins', category: 'wallet', priority: 'low', title: 'You received 50 OpenCoins', message: null, sender_id: null, sender_name: null, service: 'network', url: '/wallet', rich_content: null, is_read: 0, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(6) },++ { id: 'ntf-0005', user_id: 4, type: 'moderation', category: 'moderation', priority: 'high', title: 'Report resolved', message: 'Thanks', sender_id: null, sender_name: null, service: 'community', url: null, rich_content: null, is_read: 1, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(7) },++ ]);++ ins('notification_preferences', [++ { user_id: 1, category: 'social', enabled: 1, sound: 0, toasts: 1, email: 0 },++ { user_id: 1, category: 'streams', enabled: 1, sound: 1, toasts: 1, email: 1 },++ { user_id: 2, category: 'social', enabled: 0, sound: 0, toasts: 0, email: 0 },++ ]);++ ins('user_sessions', [++ { id: 1, user_id: 1, session_token: 'sess-rehearsal-1', device_name: 'Firefox on Linux', ip: null, user_agent: null, is_active: 1, last_used: at(6), created_at: at(5), expires_at: at(9) },++ { id: 2, user_id: 2, session_token: 'sess-rehearsal-2', device_name: 'Phone', ip: null, user_agent: null, is_active: 0, last_used: at(5), created_at: at(4), expires_at: at(8) },++ ]);++ ins('linked_accounts', [{ id: 1, user_id: 2, service: 'twitch', service_user_id: '424242', service_username: 'bea_streams', linked_at: at(3), last_used_at: at(5) }]);++ ins('verification_keys', [++ { id: 1, key: 'vk-rehearsal-1', target_username: 'ReservedName', note: 'creator reservation', created_by: 1, used_by: null, status: 'active', created_at: at(3), used_at: null },++ { id: 2, key: 'vk-rehearsal-2', target_username: 'Bea_Streams', note: null, created_by: 1, used_by: 2, status: 'used', created_at: at(1), used_at: at(1) },++ ]);++ ins('anon_users', [{ id: 1, anon_number: 7, fingerprint: 'fp-rehearsal', session_token: 'anon-sess-7', display_name: 'Anon #7', preferences: '{"color":"#ff0"}', total_messages: 12, total_commands: 1, first_seen: at(4), last_seen: at(5), ip: null, subject_id: 'anon_rehearsal0007' }]);++ // The wallet ledger: balances are the sum of each account's transactions.++ ins('coin_transactions', [++ { id: 1, user_id: 1, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:1', created_at: at(1) },++ { id: 2, user_id: 2, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:2', created_at: at(1) },++ { id: 3, user_id: 3, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:3', created_at: at(2) },++ { id: 4, user_id: 2, app_id: 'live', delta: -50, reason: 'tip', ref: 'tip:ntf-0004', idempotency_key: 'tip:2:3:1', created_at: at(6) },++ { id: 5, user_id: 3, app_id: 'live', delta: 50, reason: 'tip_received', ref: 'tip:ntf-0004', idempotency_key: 'tip:2:3:1:recv', created_at: at(6) },++ { id: 6, user_id: 1, app_id: 'games', delta: -25, reason: 'purchase', ref: 'item:hat', idempotency_key: null, created_at: at(7) },++ ]);++ ins('wallets', [++ { user_id: 1, balance: 75, updated_at: at(7) },++ { user_id: 2, balance: 50, updated_at: at(6) },++ { user_id: 3, balance: 150, updated_at: at(6) },++ ]);++ ins('follows', [++ { follower_id: 1, followed_id: 2, created_at: at(4) },++ { follower_id: 3, followed_id: 2, created_at: at(5) },++ { follower_id: 2, followed_id: 1, created_at: at(5) },++ ]);++ ins('user_follows', [++ { follower_subject: SUBJECT(1), target_type: 'channel', target_id: SUBJECT(2), active: 1, notify_email: 1, notify_push: 0, revision: 1, source: 'live', created_at: '2026-04-14T12:00:00.000Z', updated_at: '2026-04-14T12:00:00.000Z' },++ { follower_subject: SUBJECT(3), target_type: 'channel', target_id: SUBJECT(2), active: 1, notify_email: 0, notify_push: 1, revision: 2, source: 'network', created_at: '2026-05-15T12:00:00.000Z', updated_at: '2026-05-16T08:00:00.000Z' },++ { follower_subject: SUBJECT(4), target_type: 'channel', target_id: SUBJECT(2), active: 0, notify_email: 1, notify_push: 1, revision: 3, source: 'network', created_at: '2026-05-15T12:00:00.000Z', updated_at: '2026-06-01T08:00:00.000Z' },++ ]);++ ins('user_blocks', [++ { blocker_subject: SUBJECT(2), blocked_subject: SUBJECT(5), active: 1, revision: 1, created_at: '2026-05-01T10:00:00.000Z', updated_at: '2026-05-01T10:00:00.000Z' },++ { blocker_subject: SUBJECT(1), blocked_subject: SUBJECT(5), active: 0, revision: 2, created_at: '2026-05-01T10:00:00.000Z', updated_at: '2026-05-02T10:00:00.000Z' },++ ]);++ ins('audit_log', [{ id: 1, user_id: 1, action: 'user.ban', details: '{"target":5,"reason":"spam"}', ip: null, created_at: at(3) }]);++ ins('site_settings', [{ key: 'registration_open', value: 'true', type: 'boolean' }]);++ // Left behind by the import (SKIP_SOURCE): per-IP counters, never carried over.++ ins('analytics_rate_tracking', [{ ip: '198.51.100.7', window_start: 1767268800, hit_count: 3 }]);++ ins('network_event_outbox', [{ id: 1, event_id: 'evt-rehearsal-1', envelope: '{"type":"network.user.updated","subject":"usr_rehearsal0002"}', traceparent: null, created_at: 1767268800000, attempts: 2, next_attempt_at: 1767268860000, sent_at: null, seq: 1, rejected_at: null, last_error: 'events unreachable' }]);++ })();++ db.close();++}++++// ── Helpers ──────────────────────────────────────────────────++function freePort() {++ return new Promise((resolve, reject) => {++ const s = net.createServer();++ s.once('error', reject);++ s.listen(0, '127.0.0.1', () => { const { port } = s.address(); s.close(() => resolve(port)); });++ });++}++++/** Run a node script from scripts/ in `cwd` (a directory with no .env) with exactly `env`. → { code, stdout, stderr } */++function runNode(args, { cwd, env }) {++ return new Promise((resolve) => {++ const p = spawn(process.execPath, args, { cwd, env, stdio: ['ignore', 'pipe', 'pipe'] });++ let stdout = '', stderr = '';++ p.stdout.on('data', (d) => { stdout += d; });++ p.stderr.on('data', (d) => { stderr += d; });++ p.on('close', (code) => resolve({ code, stdout, stderr }));++ });++}++++/** The first JSON object a script printed (they pretty-print it), or null. */++function jsonOf(text) {++ const i = text.indexOf('{');++ if (i < 0) return null;++ try { return JSON.parse(text.slice(i)); } catch { return null; }++}++++/** A URL with its password masked, for the log. */++const masked = (url) => { try { const u = new URL(url); if (u.password) u.password = '***'; return u.toString(); } catch { return '(unparseable url)'; } };++++/**++ * --store env's guard, before anything is imported. `direct` (DATABASE_DIRECT_URL, the import's connection) and++ * `pooled` (DATABASE_URL, the server's) must reach the same database; DATABASE_URL may be a pooler, so host and port++ * prove nothing. The target must be empty through `direct` first (read-only, so a live database is refused before++ * anything is written). Then a probe table with a random name, made through `direct` with a random token as its++ * comment, must show that token through `pooled` (catalogs only, so a DML-only role can read it); the probe is++ * dropped either way. Then the target must be empty through `pooled` too. Throws with a message naming no URL.++ */++async function checkScratchTarget(direct, pooled, { truncateTarget = false } = {}) {++ const assertEmpty = async (name, db) => {++ if (truncateTarget) return;++ // A scratch database holds no rows anywhere (ov_migrations aside): any row means it is not one.++ const tables = (await db.many(`SELECT tablename FROM pg_tables++ WHERE schemaname = current_schema() AND tablename <> 'ov_migrations'`)).map((r) => r.tablename);++ const held = [];++ for (const t of tables) if (await db.value(`SELECT EXISTS (SELECT 1 FROM "${t.replace(/"/g, '""')}")`)) held.push(t);++ if (held.length) throw new Error(`the target is not empty through ${name} (rows in ${held.length} table(s): ${held.slice(0, 5).join(', ')}${held.length > 5 ? ', …' : ''}); the import empties it. Give --truncate-target if it really is a scratch database`);++ };++ await assertEmpty('DATABASE_DIRECT_URL', direct);++ const probe = `ov_rehearsal_probe_${crypto.randomBytes(6).toString('hex')}`;++ const token = crypto.randomBytes(16).toString('hex');++ let seen = null;++ try {++ await direct.exec(`CREATE TABLE ${probe} (id int)`);++ await direct.exec(`COMMENT ON TABLE ${probe} IS '${token}'`);++ seen = await pooled.value(`SELECT obj_description(c.oid, 'pg_class') FROM pg_class c WHERE c.relname = '${probe}'`)++ .catch((e) => { throw new Error(`reading the probe through DATABASE_URL failed: ${e.message}`); });++ } finally {++ await direct.exec(`DROP TABLE IF EXISTS ${probe}`).catch((e) => log(`probe cleanup: ${e.message}`));++ }++ if (seen !== token) throw new Error('DATABASE_URL does not reach the database DATABASE_DIRECT_URL imports into (a probe table made through DATABASE_DIRECT_URL is not visible through DATABASE_URL); --store env needs both on one scratch database');++ await assertEmpty('DATABASE_URL', pooled);++}++++/** The PostgreSQL side: { store, directUrl?, url?, pgliteDir?, valkeyUrl?, close() }. */++async function openTarget(store, dir, truncateTarget) {++ if (store === 'env') {++ const directUrl = (process.env.DATABASE_DIRECT_URL || '').trim(), url = (process.env.DATABASE_URL || '').trim();++ if (!directUrl || !url) throw new Error(`--store env needs DATABASE_DIRECT_URL and DATABASE_URL, both on one scratch database (${directUrl ? 'DATABASE_URL' : 'DATABASE_DIRECT_URL'} is unset or empty)`);++ const { createDb } = require('openvibe-sdk/db');++ const silent = { log() {}, warn() {}, error() {} };++ const direct = createDb({ url: directUrl, service: 'network-rehearsal', max: 1, log: silent });++ const pooled = createDb({ url, service: 'network-rehearsal-pooled', max: 1, log: silent });++ try { await checkScratchTarget(direct, pooled, { truncateTarget }); }++ finally { await Promise.allSettled([direct.close(), pooled.close()]); }++ log(`target: DATABASE_DIRECT_URL ${masked(directUrl)}, DATABASE_URL ${masked(url)} (the same database)`);++ return { store: 'postgresql', directUrl, url, valkeyUrl: process.env.VALKEY_URL || null, close: async () => {} };++ }++ if (store === 'containers') {++ if (!process.env.OV_TEST_PG_DIRECT_URL) throw new Error('--store containers needs OV_TEST_PG_DIRECT_URL (eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)")');++ return scratchDatabase(process.env.OV_TEST_PG_DIRECT_URL);++ }++ const pgliteDir = path.join(dir, 'pglite');++ log(`target: PGlite ${pgliteDir}`);++ return { store: 'pglite', pgliteDir, valkeyUrl: null, close: async () => {} };++}++++/**++ * A database of its own on the containers' PostgreSQL (`superUrl`, a superuser on a direct connection), shaped as++ * OpenVibe.Host's roles/data/add-service.sh makes production's: an owner role owning the database and its public++ * schema (migrations, the import) and a runtime role with DML only. close() drops the database and both roles.++ */++async function scratchDatabase(superUrl) {++ const { createDb } = require('openvibe-sdk/db');++ const silent = { log() {}, warn() {}, error() {} };++ const name = `network_rehearsal_${process.pid}_${crypto.randomBytes(4).toString('hex')}`;++ const owner = `${name}_owner`;++ const pw = crypto.randomBytes(16).toString('hex');++ const as = (user, db = name) => { const u = new URL(superUrl); u.username = user; u.password = pw; u.pathname = `/${db}`; return u.toString(); };++ // DROP DATABASE … WITH (FORCE) on the shared, busy container can outlast the 15 s default cap.++ const su = createDb({ url: superUrl, service: 'network-rehearsal-admin', max: 1, queryTimeoutMs: 120000, log: silent });++ const drop = async () => {++ await su.exec(`DROP DATABASE IF EXISTS ${name} WITH (FORCE)`);++ for (const r of [name, owner]) await su.exec(`DROP ROLE IF EXISTS ${r}`);++ };++ try {++ for (const stmt of [++ `CREATE ROLE ${owner} LOGIN PASSWORD '${pw}'`,++ `CREATE ROLE ${name} LOGIN PASSWORD '${pw}'`,++ `ALTER ROLE ${name} SET statement_timeout = '15s'`,++ `ALTER ROLE ${name} SET lock_timeout = '5s'`,++ `CREATE DATABASE ${name} OWNER ${owner}`,++ `REVOKE ALL ON DATABASE ${name} FROM PUBLIC`,++ `GRANT CONNECT ON DATABASE ${name} TO ${owner}, ${name}`,++ ]) await su.exec(stmt);++ const o = createDb({ url: as(owner), service: 'network-rehearsal-owner', max: 1, log: silent });++ try {++ for (const stmt of [++ `GRANT USAGE ON SCHEMA public TO ${name}`,++ `ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ${name}`,++ `ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO ${name}`,++ ]) await o.exec(stmt);++ } finally { await o.close(); }++ } catch (e) { await drop().catch(() => {}); await su.close(); throw e; }++ log(`target: test-services container, database ${name}`);++ return {++ store: 'postgresql', directUrl: as(owner), url: as(name), valkeyUrl: process.env.OV_TEST_VALKEY_URL || null,++ close: async () => { try { await drop(); } finally { await su.close(); } },++ };++}++++/** PGlite: migrate-to-postgres.js's import, into the directory the server then boots on. */++async function importIntoPglite(sqlite, pgliteDir) {++ const { createDb } = require('openvibe-sdk/db');++ const { importInto } = require('./migrate-to-postgres');++ const owner = createDb({ pglite: pgliteDir, service: 'network-import', log: { log() {}, warn: console.error, error: console.error } });++ try { return await importInto(owner, sqlite); } finally { await owner.close(); }++}++++/**++ * GET `${base}/api/ready` until it answers 200 with ready, stop() is true, or timeoutMs passes. Each request, body++ * included, is aborted after min(the time left, requestMs), so a server that accepts and never answers cannot hold++ * the rehearsal past its deadline. → { status, body } of the last answer read (null when none was).++ */++async function pollReady(base, { timeoutMs, stop = () => false, requestMs = 5000 }) {++ const t0 = Date.now();++ let status = null, body = null;++ while (!stop()) {++ const left = timeoutMs - (Date.now() - t0);++ if (left <= 0) break;++ try {++ const r = await fetch(`${base}/api/ready`, { signal: AbortSignal.timeout(Math.max(1, Math.min(left, requestMs))) });++ status = r.status;++ body = await r.json();++ if (r.status === 200 && body.ready) break;++ } catch { /* not listening yet, aborted at the deadline, or not JSON */ }++ await new Promise((r) => setTimeout(r, 250));++ }++ return { status, body };++}++++/** Boot server/index.js on the target and read /api/ready until it is ready (or the deadline). */++async function bootAndCheck(target, dir, { timeoutMs = 120000 } = {}) {++ const port = await freePort();++ const keys = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });++ fs.mkdirSync(path.join(dir, 'keys'), { recursive: true });++ fs.writeFileSync(path.join(dir, 'keys', 'private.pem'), keys.privateKey, { mode: 0o600 });++ fs.writeFileSync(path.join(dir, 'keys', 'public.pem'), keys.publicKey);++ const env = {++ PATH: process.env.PATH, HOME: dir, PORT: String(port), HOST: '127.0.0.1',++ NODE_ENV: target.store === 'pglite' ? 'test' : 'production',++ JWT_PRIVATE_KEY: path.join(dir, 'keys', 'private.pem'), JWT_PUBLIC_KEY: path.join(dir, 'keys', 'public.pem'),++ AVATAR_PATH: path.join(dir, 'avatars'), INTERNAL_API_KEY: crypto.randomBytes(24).toString('hex'),++ OV_NETWORK_INTERNAL_URL: `http://127.0.0.1:${port}`,++ OV_LIVE_INTERNAL_URL: 'http://127.0.0.1:9', OV_TOOLS_INTERNAL_URL: 'http://127.0.0.1:9', OV_GAMES_INTERNAL_URL: 'http://127.0.0.1:9',++ OV_MEDIA_INTERNAL_URL: 'http://127.0.0.1:9', OV_AI_INTERNAL_URL: 'http://127.0.0.1:9',++ ...(target.store === 'pglite' ? { PGLITE_DIR: target.pgliteDir } : { DATABASE_URL: target.url, DATABASE_DIRECT_URL: target.directUrl }),++ ...(target.valkeyUrl ? { VALKEY_URL: target.valkeyUrl, VALKEY_PREFIX: `ov:network-rehearsal-${crypto.randomBytes(4).toString('hex')}:` } : {}),++ };++ const proc = spawn(process.execPath, [path.join(ROOT, 'server', 'index.js')], { cwd: dir, env, stdio: ['ignore', 'pipe', 'pipe'] });++ let out = '';++ proc.stdout.on('data', (d) => { out += d; });++ proc.stderr.on('data', (d) => { out += d; });++ let exited = null;++ proc.on('exit', (code, sig) => { exited = { code, sig }; });++ const base = `http://127.0.0.1:${port}`;++ const t0 = Date.now();++ let body = null, status = null;++ try {++ ({ status, body } = await pollReady(base, { timeoutMs, stop: () => !!exited }));++ } finally {++ if (!exited) {++ await new Promise((resolve) => {++ const timer = setTimeout(resolve, 5000);++ proc.once('exit', () => { clearTimeout(timer); resolve(); });++ proc.kill('SIGTERM');++ });++ if (!exited) {++ try { proc.kill('SIGKILL'); } catch { /* gone */ }++ await new Promise((resolve) => { if (exited) resolve(); else proc.once('exit', resolve); });++ }++ }++ }++ const checks = (body && body.checks) || {};++ const ready = {++ http: status, ready: !!(body && body.ready), status: body ? body.status : null,++ db: checks.db ? checks.db.status : null, signing_key: checks.signing_key ? checks.signing_key.status : null,++ failed: body ? body.failed : null, degraded: body ? body.degraded : null, node_env: env.NODE_ENV, valkey: !!target.valkeyUrl, ms: Date.now() - t0,++ };++ ready.ok = status === 200 && ready.ready && ready.db === 'ok' && ready.signing_key === 'ok';++ if (!ready.ok) {++ ready.exited = exited;++ // The boot log's tail, for the operator; the server never logs secrets or rows.++ log(`server did not become ready (${JSON.stringify(exited)}); last log lines:\n${out.split('\n').slice(-25).join('\n')}`);++ }++ return ready;++}++++// ── The rehearsal ────────────────────────────────────────────++async function rehearse(opts) {++ const t0 = Date.now();++ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-network-rehearsal-'));++ const result = { ok: false, source: opts.sqlite ? 'snapshot' : 'seeded-20dd7ff', store: null, preflight: null, import: null, tables: {}, pg_only: {}, ready: null, problems: [], dir: opts.keep ? dir : undefined };++ // Children get exactly this environment, in a directory with no .env, so nothing else is picked up.++ const childEnv = { PATH: process.env.PATH, HOME: dir };++ let target = null;++ try {++ const sqlite = opts.sqlite ? path.resolve(opts.sqlite) : path.join(dir, 'network.db');++ if (opts.sqlite) { if (!fs.existsSync(sqlite)) throw new Error(`no such SQLite database: ${sqlite}`); }++ else seedFixture(sqlite);++ log(`source: ${sqlite}`);++++ // 1. pg-preflight: case-fold collisions block the cutover.++ const pre = await runNode([path.join(__dirname, 'pg-preflight-collisions.js'), '--sqlite', sqlite, '--json'], { cwd: dir, env: childEnv });++ const preReport = jsonOf(pre.stdout);++ result.preflight = { ok: pre.code === 0, exit: pre.code, collisions: preReport ? preReport.collisions.map((c) => ({ table: c.table, column: c.column, spellings: c.spellings })) : null };++ if (pre.code !== 0) { result.problems.push(`pg-preflight exited ${pre.code}${pre.stderr ? `: ${pre.stderr.trim().split('\n').pop()}` : ''}`); return result; }++ log('preflight: no case-fold collisions');++++ // 2. migrate-to-postgres into the scratch target.++ const store = opts.store === 'auto'++ ? (process.env.OV_TEST_PG_DIRECT_URL ? 'containers' : 'pglite')++ : opts.store;++ target = await openTarget(store, dir, opts['truncate-target']);++ result.store = target.store;++ let imp;++ if (target.store === 'pglite') {++ const dry = await runNode([path.join(__dirname, 'migrate-to-postgres.js'), '--sqlite', sqlite, '--pglite', '--json'], { cwd: dir, env: childEnv });++ const dryReport = jsonOf(dry.stdout);++ if (dry.code !== 0) result.problems.push(`migrate-to-postgres --pglite exited ${dry.code}: ${dryReport ? JSON.stringify(dryReport.problems) : dry.stderr.trim().split('\n').pop()}`);++ imp = await importIntoPglite(sqlite, target.pgliteDir);++ imp.script = { exit: dry.code };++ } else {++ const run = await runNode([path.join(__dirname, 'migrate-to-postgres.js'), '--sqlite', sqlite, '--json'], { cwd: dir, env: { ...childEnv, DATABASE_DIRECT_URL: target.directUrl } });++ imp = jsonOf(run.stdout) || { ok: false, tables: [], problems: [{ table: '-', problem: run.stderr.trim().split('\n').pop() || `exit ${run.code}` }] };++ imp.script = { exit: run.code };++ }++ result.import = { ok: imp.ok && imp.script.exit === 0, tables: imp.tables.length, rows: imp.tables.reduce((n, t) => n + t.rows, 0), cleaned: imp.cleaned || [], problems: imp.problems };++ if (!result.import.ok) { result.problems.push('migrate-to-postgres did not verify'); return result; }++ log(`import: ${result.import.tables} tables, ${result.import.rows} rows verified`);++++ // 3. Row-count parity, before the server's boot seeds anything.++ const { rowParity } = require('./pg-row-parity');++ const { createDb } = require('openvibe-sdk/db');++ const quiet = { log() {}, warn: console.error, error: console.error };++ const src = new (require('better-sqlite3'))(sqlite, { readonly: true, fileMustExist: true });++ const db = target.store === 'pglite' ? createDb({ pglite: target.pgliteDir, service: 'network-parity', log: quiet })++ : createDb({ url: target.directUrl, service: 'network-parity', max: 1, log: quiet });++ let parity;++ try { parity = await rowParity(src, db); } finally { src.close(); await db.close(); }++ result.tables = parity.tables;++ result.pg_only = parity.pg_only;++ for (const p of parity.problems) result.problems.push(`${p.table}: ${p.problem}`);++ if (!parity.ok) return result;++ log(`parity: ${Object.keys(parity.tables).length} tables equal`);++++ // 4. The server on the imported database: /api/ready with db and signing_key ok.++ result.ready = await bootAndCheck(target, dir, { timeoutMs: opts.timeoutMs });++ if (!result.ready.ok) result.problems.push(`/api/ready: ${result.ready.http} ${result.ready.status} (failed: ${JSON.stringify(result.ready.failed)})`);++ else log(`ready: ${result.ready.status} in ${result.ready.ms} ms`);++ result.ok = result.preflight.ok && result.import.ok && parity.ok && result.ready.ok && result.problems.length === 0;++ return result;++ } catch (err) {++ result.problems.push(err.message);++ return result;++ } finally {++ if (target) await target.close().catch((e) => log(`cleanup: ${e.message}`));++ if (!opts.keep) fs.rmSync(dir, { recursive: true, force: true });++ result.ms = Date.now() - t0;++ }++}++++async function main(argv = process.argv.slice(2)) {++ let opts;++ try { opts = parseArgs(argv, { flags: ['keep', 'truncate-target'], values: ['sqlite', 'store', 'write-fixture', 'timeout'] }); } catch (e) { console.error(e.message); return 2; }++ if (opts.help) { console.error('usage: node scripts/rehearse-pg-cutover.js [--sqlite <copy.db>] [--store auto|pglite|containers|env] [--truncate-target] [--keep] [--write-fixture <file>]'); return 2; }++ if (opts['write-fixture']) {++ const file = path.resolve(opts['write-fixture']);++ if (fs.existsSync(file)) { console.error(`${file} exists; not overwriting it`); return 2; }++ seedFixture(file);++ console.error(`wrote ${file}`);++ return 0;++ }++ opts.store = opts.store || 'auto';++ if (!['auto', 'pglite', 'containers', 'env'].includes(opts.store)) { console.error(`--store ${opts.store}: auto, pglite, containers or env`); return 2; }++ opts.timeoutMs = opts.timeout ? Number(opts.timeout) * 1000 : 120000;++ const result = await rehearse(opts);++ if (result.dir === undefined) delete result.dir;++ console.log(JSON.stringify(result));++ return result.ok ? 0 : 1;++}++++if (require.main === module) main().then((code) => process.exit(code), (err) => { console.log(JSON.stringify({ ok: false, problems: [err.message] })); process.exit(1); });++++module.exports = { rehearse, seedFixture, FIXTURE_SCHEMA, checkScratchTarget, pollReady };nnew file mode 100644@@@ -0,0 +1,153 @@++'use strict';++// The T2 cutover rehearsal (scripts/rehearse-pg-cutover.js, docs/cutover-t2-postgres.md) on its seeded fixture:++// commit 20dd7ff's SQLite schema with representative rows → pg-preflight → migrate-to-postgres → row-count++// parity → the server booted on the result with /api/ready green. On PGlite by default, on a scratch database++// in the containers under npm run test:pg. A case-fold collision stops it at the preflight, and the parity++// check reports a target that lacks the rows or whose users_profile_created trigger the import left off.++// node test/pg-cutover-rehearsal.test.js++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { spawn } = require('child_process');++const { rowParity } = require('../scripts/pg-row-parity');++++const SCRIPT = path.join(__dirname, '..', 'scripts', 'rehearse-pg-cutover.js');++// Never a real target: the default store ignores DATABASE_URL, and the child runs without it anyway.++const env = { ...process.env };++delete env.DATABASE_URL;++delete env.DATABASE_DIRECT_URL;++delete env.NODE_OPTIONS;++++function rehearse(args, extraEnv = {}) {++ return new Promise((resolve) => {++ const p = spawn(process.execPath, [SCRIPT, ...args], { env: { ...env, ...extraEnv }, stdio: ['ignore', 'pipe', 'pipe'] });++ let stdout = '', stderr = '';++ p.stdout.on('data', (d) => { stdout += d; });++ p.stderr.on('data', (d) => { stderr += d; });++ p.on('close', (code) => {++ const lines = stdout.trim().split('\n');++ let json = null;++ try { json = JSON.parse(lines[lines.length - 1]); } catch { /* asserted below */ }++ resolve({ code, json, lines, stderr });++ });++ });++}++++(async () => {++ // 1. The seeded fixture, end to end.++ const r = await rehearse([]);++ assert.ok(r.json, `one JSON line last on stdout:\n${r.lines.join('\n')}\n${r.stderr}`);++ assert.strictEqual(r.lines.length, 1, 'stdout is the JSON line alone');++ assert.strictEqual(r.json.ok, true, `rehearsal ok: ${JSON.stringify(r.json.problems)}\n${r.stderr}`);++ assert.strictEqual(r.code, 0);++ assert.strictEqual(r.json.source, 'seeded-20dd7ff');++ assert.strictEqual(r.json.store, process.env.OV_TEST_PG_DIRECT_URL ? 'postgresql' : 'pglite');++ assert.deepStrictEqual(r.json.preflight, { ok: true, exit: 0, collisions: [] });++ const expected = { users: 6, oauth_clients: 2, oauth_codes: 2, oauth_tokens: 3, notifications: 5, notification_preferences: 3, user_sessions: 2,++ wallets: 3, coin_transactions: 6, follows: 3, user_follows: 3, user_blocks: 2, verification_keys: 2, network_event_outbox: 1, themes: 0,++ // 20dd7ff's trigger wrote one per non-anonymous account; PostgreSQL's must not add its own during the import.++ user_profile_changes: 5 };++ for (const [t, n] of Object.entries(expected)) assert.deepStrictEqual(r.json.tables[t], { sqlite: n, pg: n }, `${t}: ${n} rows on both sides`);++ for (const [t, c] of Object.entries(r.json.tables)) assert.strictEqual(c.sqlite, c.pg, `${t}: parity`);++ // 77 tables at 20dd7ff (initDb's and those its modules create at boot), less the one left behind.++ assert.strictEqual(Object.keys(r.json.tables).length, 76, 'every table of the 20dd7ff schema is compared');++ assert.ok(!('analytics_rate_tracking' in r.json.tables) && !('analytics_rate_tracking' in r.json.pg_only), 'per-IP counters are never carried over');++ assert.deepStrictEqual(r.json.import.cleaned, [{ column: 'notifications.message', values: 1 }], 'the NUL PostgreSQL refuses is dropped and reported');++ assert.strictEqual(r.json.ready.ok, true);++ assert.strictEqual(r.json.ready.http, 200);++ assert.strictEqual(r.json.ready.db, 'ok');++ assert.strictEqual(r.json.ready.signing_key, 'ok');++ assert.strictEqual(r.json.ready.node_env, process.env.OV_TEST_PG_DIRECT_URL ? 'production' : 'test');++ assert.ok(!/\$2a\$10\$|rehearsal-client-secret|rt-family/.test(r.lines[0]), 'counts only: no row value in the output');++++ // 2. A case-fold collision (alex / ALEX) stops the cutover at the preflight, before any import.++ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-rehearsal-test-'));++ const file = path.join(dir, 'collide.db');++ const w = await rehearse(['--write-fixture', file]);++ assert.strictEqual(w.code, 0, w.stderr);++ const Database = require('better-sqlite3');++ const db = new Database(file);++ db.prepare("INSERT INTO users (id, username, password_hash, subject_id) VALUES (7, 'ALEX', 'x', 'usr_rehearsal0007')").run();++ db.close();++ const c = await rehearse(['--sqlite', file, '--store', 'pglite']);++ assert.strictEqual(c.code, 1);++ assert.strictEqual(c.json.ok, false);++ assert.strictEqual(c.json.source, 'snapshot');++ assert.strictEqual(c.json.preflight.ok, false);++ assert.deepStrictEqual(c.json.preflight.collisions, [{ table: 'users', column: 'username', spellings: 2 }]);++ assert.strictEqual(c.json.import, null, 'nothing imported');++ assert.strictEqual(c.json.ready, null, 'nothing booted');++ assert.ok(fs.existsSync(file), 'the --sqlite file is left in place');++++ // 3. Parity against a database that lacks the rows (this process's migrated, empty one) is a mismatch per table.++ const src = new Database(file, { readonly: true });++ const report = await rowParity(src, globalThis.__ovNetworkTestDb);++ src.close();++ assert.strictEqual(report.ok, false);++ assert.deepStrictEqual(report.tables.users, { sqlite: 7, pg: 0 });++ assert.ok(report.problems.some((p) => p.table === 'coin_transactions' && /SQLite 6, PostgreSQL 0/.test(p.problem)));++ assert.ok(!report.problems.some((p) => p.table === 'dev_projects'), 'equal counts (0 and 0) are not problems');++ assert.ok(!report.problems.some((p) => /trigger/.test(p.problem)), 'users_profile_created is enabled');++++ // 4. An import killed between its ALTERs leaves users_profile_created off: parity refuses that database. On a++ // PGlite of its own: the test database's role (DML only under test:pg, as production's runtime) cannot ALTER.++ const { createDb } = require('openvibe-sdk/db');++ const { MIGRATIONS } = require('../scripts/migrate-to-postgres');++ const owner = createDb({ pglite: true, service: 'network-rehearsal-test', log: { log() {}, warn() {}, error: console.error } });++ let off;++ try {++ await owner.migrate({ dir: MIGRATIONS, log: { log() {}, warn() {}, error: console.error } });++ await owner.exec('ALTER TABLE users DISABLE TRIGGER users_profile_created');++ const src2 = new Database(file, { readonly: true });++ try { off = await rowParity(src2, owner); } finally { src2.close(); }++ } finally { await owner.close(); }++ assert.ok(off.problems.some((p) => p.table === 'users' && /trigger users_profile_created is disabled/.test(p.problem)), JSON.stringify(off.problems));++ fs.rmSync(dir, { recursive: true, force: true });++++ // 5. --store env refuses an unset or empty DATABASE_URL before it connects anywhere; the message names no URL.++ for (const extra of [{}, { DATABASE_URL: '' }, { DATABASE_URL: ' ' }]) {++ const e = await rehearse(['--store', 'env'], { DATABASE_DIRECT_URL: 'postgres://[email protected]:9/none', ...extra });++ assert.strictEqual(e.code, 1, e.stderr);++ assert.strictEqual(e.json.ok, false);++ assert.ok(e.json.problems.some((p) => /needs DATABASE_DIRECT_URL and DATABASE_URL.*DATABASE_URL is unset or empty/.test(p)), JSON.stringify(e.json.problems));++ assert.strictEqual(e.json.import, null, 'nothing imported');++ assert.ok(!/127\.0\.0\.1:9|postgres:\/\//.test(e.lines.join('\n')), 'no URL in the output');++ }++++ // 6. The same-database guard: two separate databases are refused; one empty database is accepted; rows refuse it++ // unless --truncate-target; the probe table never stays behind.++ const { checkScratchTarget, pollReady } = require('../scripts/rehearse-pg-cutover');++ const quietLog = { log() {}, warn() {}, error: console.error };++ const dbA = createDb({ pglite: true, service: 'network-rehearsal-test-a', log: quietLog });++ const dbB = createDb({ pglite: true, service: 'network-rehearsal-test-b', log: quietLog });++ try {++ await assert.rejects(checkScratchTarget(dbA, dbB), (e) => /does not reach the database/.test(e.message) && !/postgres:\/\//.test(e.message));++ await checkScratchTarget(dbA, dbA);++ const probes = `SELECT count(*)::int FROM pg_class WHERE relname LIKE 'ov_rehearsal_probe_%'`;++ assert.strictEqual(Number(await dbA.value(probes)), 0, 'the probe is dropped');++ assert.strictEqual(Number(await dbB.value(probes)), 0);++ await dbA.exec('CREATE TABLE rehearsal_rows (id int)');++ await dbA.exec('INSERT INTO rehearsal_rows VALUES (1)');++ await assert.rejects(checkScratchTarget(dbA, dbA), /not empty through DATABASE_DIRECT_URL/);++ await checkScratchTarget(dbA, dbA, { truncateTarget: true });++ assert.strictEqual(Number(await dbA.value(probes)), 0);++ } finally { await dbA.close(); await dbB.close(); }++++ // 7. A server that accepts and never answers (or stalls mid-body) cannot hold the readiness poll past its deadline.++ const http = require('http');++ for (const handler of [() => {}, (req, res) => { res.writeHead(200, { 'content-type': 'application/json' }); res.write('{"ready":'); }]) {++ const stalled = http.createServer(handler);++ await new Promise((r) => stalled.listen(0, '127.0.0.1', r));++ const t = Date.now();++ try {++ const p = await pollReady(`http://127.0.0.1:${stalled.address().port}`, { timeoutMs: 3000, requestMs: 1000 });++ const took = Date.now() - t;++ assert.strictEqual(p.body, null, 'no answer read');++ assert.ok(took >= 2500 && took < 6000, `returned in ${took} ms (deadline 3000)`);++ } finally { stalled.closeAllConnections(); await new Promise((r) => stalled.close(r)); }++ }++++ console.log(`pg cutover rehearsal: all checks passed (${r.json.store}, ${Object.keys(r.json.tables).length} tables, ready ${r.json.ready.status})`);++ process.exit(0);++})().catch((err) => { console.error(err); process.exit(1); });
new file mode 100644
new file mode 100644
@@ -0,0 +1,370 @@
@@ -0,0 +1,370 @@
+# T2 cutover: Network from SQLite to PostgreSQL
+
+The production move of OpenVibe.Network from its SQLite file to PostgreSQL (plan T2, ADR-035). Production
+(`openvibe-ovh`) runs **20dd7ff**, the last SQLite release. Every later commit serves from PostgreSQL only:
+`DATABASE_URL` is required in production (there is no SQLite path), migrations run as the owner on
+`DATABASE_DIRECT_URL`, and the limit counters live on Valkey (`VALKEY_URL`). Two manual deploys of that range
+failed readiness and ovhost rolled them back. The most likely cause is that the server refused to boot without
+`DATABASE_URL` (`server/db/database.js`), but this is unconfirmed. This runbook covers the data move those
+deploys skipped. It was rehearsed with `npm run rehearse-pg-cutover` (`scripts/rehearse-pg-cutover.js`) on
+20dd7ff's full schema: the 51 tables `initDb()` makes plus the 26 its modules create at boot. That rehearsal
+found two import blockers, fixed in `scripts/migrate-to-postgres.js` with it:
+
+- `analytics_rate_tracking` (openvibe-shared's old per-IP counters) has no PostgreSQL table, so the import refused
+ to start. It is now in `SKIP_SOURCE`, so IPs are never carried over.
+- The `users_profile_created` trigger fired on every imported account. That collided with the copied
+ `user_profile_changes` rows, and would have published one profile event per account. The import now turns
+ that trigger off while it copies (`QUIET_TRIGGERS`).
+
+Every step ends with a **go / no-go** check. A no-go before step 8 (deploy) leaves production on SQLite with no
+data change: restart the old release (`sudo systemctl start openvibe-network`), reopen, and reschedule. A no-go
+from step 8 on means [rollback](#rollback).
+
+Conventions:
+
+- `$STAGE` is the staging checkout of the target commit (step 1).
+- `$COPY` is the read-only SQLite copy (step 4).
+- `$T0` is when writes stopped (step 3).
+- `$T1` is when the PostgreSQL release started serving (step 8).
+- Commands that need the database URLs load `/etc/openvibe/network.env` with `node --env-file`, which never
+ prints it, as README.md does for `subscribe-events.js`. Never `cat`, `source -x`, `env` or `printenv` that file,
+ and never paste a URL into a terminal, chat or ticket.
+
+---
+
+## Prerequisites (owner only, before the window)
+
+These are done by the owner and never by an agent. Each must be true before the window opens.
+
+1. **PostgreSQL, PgBouncer and Valkey run on openvibe-ovh.** PostgreSQL 18, with PgBouncer in transaction
+ pooling mode in front of it, and Valkey. Network's database and its two roles are created by OpenVibe.Host
+ `roles/data/add-service.sh network`:
+ - an **owner** role that owns the schema, runs the migrations and does the import;
+ - a **runtime** role with DML rights only, which serves requests through PgBouncer.
+2. **The three URLs are in `/etc/openvibe/network.env`**, mode `0600`, and are never printed:
+ - `DATABASE_URL`: the runtime role, through PgBouncer.
+ - `DATABASE_DIRECT_URL`: the owner role, on a **direct** connection (not PgBouncer). Transaction pooling cannot
+ hold the import's single transaction or the migration lock.
+ - `VALKEY_URL`: the shared limit counters. Without it, each process counts on its own.
+
+ Check them by name only. The first command must print `3`, the second `600`:
+
+ ```bash
+ sudo grep -cE '^(DATABASE_URL|DATABASE_DIRECT_URL|VALKEY_URL)=.+' /etc/openvibe/network.env
+ sudo stat -c '%a' /etc/openvibe/network.env
+ ```
+3. **The URLs connect.** Run this from `$STAGE` (step 1). It prints three `ok` lines and never prints a URL:
+
+ ```bash
+ cd "$STAGE" && sudo node --env-file=/etc/openvibe/network.env -e '
+ const { Client } = require("pg");
+ (async () => {
+ for (const k of ["DATABASE_URL", "DATABASE_DIRECT_URL"]) {
+ const c = new Client({ connectionString: process.env[k] });
+ await c.connect();
+ await c.query("SELECT 1");
+ await c.end();
+ console.log(k, "ok");
+ }
+ const v = new (require("iovalkey"))(process.env.VALKEY_URL, { lazyConnect: true });
+ await v.connect();
+ console.log("VALKEY_URL", await v.ping() === "PONG" ? "ok" : "FAIL");
+ v.disconnect();
+ })().catch((e) => { console.error("FAIL", e.code || e.message.replace(/\/\/[^@]*@/, "//***@")); process.exit(1); });'
+ ```
+4. **A green rehearsal on a fresh production snapshot** (step 2). Only after it passes does the owner write
+ `~/openvibe/agents/ds/deploy/rehearsals/network-t2-pg.json` with `{"ok": true}`. ds-finish refuses to deploy
+ without that file and without the merged PR's `cutover` block.
+5. **No case-fold collisions**, or each one resolved on the SQLite side (step 5 explains how). Check this at
+ rehearsal time so the window does not stop on it.
+6. **The window is announced.** Post a status-page maintenance (admin → Status, kind `maintenance`) at least a
+ day ahead. While Network is stopped, nobody can sign in, run an OAuth authorize, or refresh a token on any
+ OpenVibe site. Services verify JWTs they already have offline (cached JWKS), so signed-in pages keep working.
+ Calls into Network fail until it is back: notifications, coins, follows and blocks. Expect about 15 minutes
+ plus the import time that the snapshot rehearsal printed (`ms`).
+7. **The target commit is fixed.** It is the merge commit of the PR carrying the `cutover` block (origin/main at
+ merge). Steps 1, 2 and 8 must all use that one SHA.
+
+---
+
+## Step 1: stage the target release (T-1 day, no downtime)
+
+The running checkout `/opt/openvibe.network` stays on 20dd7ff until step 8. The import tools and migrations
+come from a separate checkout of the target commit. It is never served from.
+
+```bash
+TARGET=<merge sha>
+STAGE=/var/tmp/network-t2-stage
+sudo -u ubuntu git -C /opt/openvibe.network fetch origin
+sudo -u ubuntu git clone --no-checkout /opt/openvibe.network "$STAGE"
+sudo -u ubuntu git -C "$STAGE" checkout --detach "$TARGET"
+cd "$STAGE" && sudo -u ubuntu npm ci # dev dependencies too: the rehearsal's PGlite lives there
+```
+
+**Go:** `git -C "$STAGE" rev-parse HEAD` equals `$TARGET`, and
+`node -e "require('better-sqlite3'); require('pg'); require('@electric-sql/pglite')"` exits 0 in `$STAGE`.
+**No-go:** `npm ci` fails. Fix the host's build tools or network first.
+
+## Step 2: rehearse on a production snapshot (T-1 day, no downtime)
+
+Take an online copy (the service keeps running) and rehearse on it, on the host, so the data never leaves it:
+
+```bash
+SNAP=/var/backups/openvibe/network-rehearsal-$(date -u +%Y%m%dT%H%M%SZ).db
+sudo -u ubuntu node -e "new (require('$STAGE/node_modules/better-sqlite3'))('/opt/openvibe.network/data/network.db', { readonly: true }).backup('$SNAP').then(() => console.log('ok'))"
+cd "$STAGE" && sudo -u ubuntu node scripts/rehearse-pg-cutover.js --sqlite "$SNAP" --store pglite
+```
+
+The live file is `/opt/openvibe.network/data/network.db` unless `DB_PATH` says otherwise. A path is not a
+secret, so `sudo grep '^DB_PATH=' /etc/openvibe/network.env` may be used to check it.
+
+The rehearsal runs, in order:
+
+1. pg-preflight;
+2. `migrate-to-postgres.js --pglite`;
+3. the same import into a PGlite directory;
+4. the row-count parity check;
+5. boots the target server on the result and requires `/api/ready` to answer 200 with `db` and `signing_key` ok.
+
+It prints one JSON line, with counts only.
+
+**Go:** exit 0 and `"ok":true`. The owner keeps that line and writes the `network-t2-pg` rehearsal file
+(prerequisite 4).
+**No-go:** read `problems` in the line.
+- `preflight` collisions: see step 5.
+- Import problems, for example a source table with no target or an unparseable date: fix them in the migrations
+ or in `TABLES`/`SKIP_SOURCE` of `scripts/migrate-to-postgres.js` with a PR, then rehearse again.
+
+`--store env` (only when given; the default never reads these URLs) runs the same rehearsal against a scratch
+PostgreSQL database instead of PGlite. Put that database's own `DATABASE_URL`/`DATABASE_DIRECT_URL` in the
+environment. The rehearsal empties its tables and refuses a target where any table holds a row (unless
+`--truncate-target`), but **never** point it at the production URLs.
+
+## Step 3: freeze writes (T0, the window opens)
+
+```bash
+sudo systemctl stop openvibe-network
+T0=$(date -u +%Y-%m-%dT%H:%M:%SZ); echo "$T0"
+```
+
+**Go:** all three hold:
+- `systemctl is-active openvibe-network` prints `inactive`;
+- `curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:4000/api/health` prints `000`;
+- nothing restarts it within a minute. Check `systemctl is-active` again.
+
+**No-go:** it comes back on its own (a watchdog or a deploy). Find what restarted it before going on. A write
+made after the copy in step 4 would be lost.
+
+## Step 4: back up, and copy the SQLite file
+
+```bash
+ov access run openvibe-ovh db-backup
+COPY=/var/backups/openvibe/network-pre-t2-${T0//:/}.db
+sudo -u ubuntu node -e "new (require('$STAGE/node_modules/better-sqlite3'))('/opt/openvibe.network/data/network.db', { readonly: true }).backup('$COPY').then(() => console.log('ok'))"
+sudo chmod 0400 "$COPY" && sudo sha256sum "$COPY" | tee "$COPY.sha256"
+sudo -u ubuntu node -e "const d = new (require('$STAGE/node_modules/better-sqlite3'))('$COPY', { readonly: true }); console.log(d.pragma('integrity_check', { simple: true }))"
+```
+
+The backup API copies the file together with its WAL. From here on, every step reads `$COPY`. The live
+`data/network.db` is not opened again, and it is what a rollback serves.
+
+**Go:** all three hold:
+- `db-backup` exits 0;
+- `integrity_check` prints `ok`;
+- the user count of `$COPY` equals the live file's. Run this for each file:
+ `node -e "…prepare('SELECT COUNT(*) n FROM users').get().n"`.
+
+**No-go:** the backup fails or the integrity check is not `ok`. Restart the old release and investigate.
+
+## Step 5: pg-preflight (case-fold collisions)
+
+```bash
+cd "$STAGE" && sudo -u ubuntu npm run pg-preflight -- --sqlite "$COPY"
+```
+
+The script is `scripts/pg-preflight-collisions.js` (`--sqlite <file>`, `--json`). It opens the copy read-only.
+
+**Go:** exit 0, `no case-fold collisions: safe to import`.
+**No-go:**
+- **Exit 1 (collisions).** PostgreSQL keeps usernames unique on `lower(username)`, so `Alex` and `alex` cannot
+ both be imported. This is a data decision, never made by a script. Restart the old release and close the
+ window. For each listed pair:
+ - `users.username`: ask the newer or inactive account to rename, or rename it from the old release's admin
+ (users → edit username). That records the old name.
+ - `verification_keys.target_username` (active keys): revoke the duplicate key in admin.
+ - `username_history.old_username`: absent at 20dd7ff. The script reports `table absent`.
+
+ Run the preflight on a fresh snapshot until it is clean, then schedule a new window.
+- **Exit 2.** The file is unreadable or better-sqlite3 is missing. Fix that before going on.
+
+## Step 6: migrate-to-postgres
+
+The script reads these and nothing else:
+
+| Input | Meaning |
+|---|---|
+| `--sqlite <file>` | The source: `$COPY`. Without it, `DB_PATH`, else `data/network.db` under the checkout. Always pass it. |
+| `--pglite` | A dry run into an in-memory PostgreSQL. No URL is needed. |
+| `--json` | The report as JSON. |
+| `DATABASE_DIRECT_URL` | The owner, on a direct connection. Required without `--pglite`. `DATABASE_URL` is not used. |
+| `.env` | Read from the working directory by dotenv. `$STAGE` has none. Keep it that way. |
+
+```bash
+cd "$STAGE"
+sudo -u ubuntu node scripts/migrate-to-postgres.js --sqlite "$COPY" --pglite # dry run
+sudo node --env-file=/etc/openvibe/network.env scripts/migrate-to-postgres.js --sqlite "$COPY" # the import
+```
+
+The import does four things:
+
+1. Applies `migrations/0001…0007` as the owner.
+2. **Empties** every Network table and copies every table in one transaction, parents first. Ids are kept and
+ sequences are moved past them. `users_profile_created` is disabled for the copy and enabled again afterwards,
+ even when the copy fails. `analytics_rate_tracking` is left behind.
+3. Drops a NUL or an unpaired surrogate, which PostgreSQL refuses, and reports it as `cleaned:`.
+4. Verifies each table's row count and a checksum of every row.
+
+The SQLite copy is opened read-only. Because the import empties first, it can be repeated.
+
+**Go:** exit 0, first line `… : OK`, and no `problem:` line. A `cleaned:` line is expected and fine. Note it in
+the log.
+**No-go:** exit 1 or a `problem:` line. Leave the window closed, restart the old release, and fix the problem
+with a PR and a new rehearsal. Nothing serves from PostgreSQL yet, so there is nothing to undo. If the process
+was killed mid-import, the trigger is still off: run the import again. It ends by enabling the trigger, and
+step 7 checks that it did.
+
+## Step 7: row-count parity, per table
+
+```bash
+cd "$STAGE" && sudo node --env-file=/etc/openvibe/network.env scripts/pg-row-parity.js --sqlite "$COPY"
+```
+
+`scripts/pg-row-parity.js` reads `DATABASE_DIRECT_URL` (else `DATABASE_URL`). It compares every SQLite table:
+
+- **SQLite** (`$COPY`, read-only): `SELECT COUNT(*) FROM "<table>"` for each table in
+ `SELECT name FROM sqlite_master WHERE type = 'table'`. SQLite's own `sqlite_*` tables, FTS5 virtual tables and
+ their shadow tables, and `SKIP_SOURCE` (`analytics_rate_tracking`: old per-IP counters, never carried over)
+ are left out.
+- **PostgreSQL**: `SELECT COUNT(*) FROM "<table>"` for the same name, in
+ `SELECT tablename FROM pg_tables WHERE schemaname = current_schema()`.
+
+PostgreSQL-only tables, new in T2, are listed and not compared. The script also requires the triggers that the
+import turned off to be enabled again:
+`SELECT tgenabled FROM pg_trigger WHERE tgname = 'users_profile_created'` must be `O`. Run the check **before** step 8. The new
+release's boot seeds rows the SQLite file never had: built-in themes, OAuth clients and principal grants.
+
+**Go:** exit 0, `OK`, and no `✗` row.
+**No-go:** any table differs, or the trigger is reported disabled. Do not deploy. Run step 6 again; it empties first. If the same table still differs,
+restart the old release and investigate.
+
+## Step 8: deploy (T1)
+
+```bash
+ov access run openvibe-ovh deploy network
+T1=$(date -u +%Y-%m-%dT%H:%M:%SZ); echo "$T1"
+```
+
+ovhost fast-forwards `/opt/openvibe.network` to the target and runs `npm install --omit=dev`. It installs the unit
+and starts it with `network.env`, polls `/api/ready`, and rolls back on its own if the release does not come up.
+At boot, the migrations find everything applied, and the seed adds only the boot rows that are missing.
+
+**Go:** the deploy exits 0, and its release is `$TARGET`.
+**No-go:**
+- **ovhost rolled back (exit 3).** Production is on 20dd7ff and SQLite again, and no write reached PostgreSQL.
+ Read `ovhost releases network` and the boot log
+ (`sudo journalctl -u openvibe-network --since "$T0" | tail -50`). The log names its failure without printing
+ URLs. Then close the window.
+- **Any other failure.** Go to [rollback](#rollback).
+
+## Step 9: checks
+
+Do these within 30 minutes of T1. All must pass.
+
+1. **Health.** `ov access run openvibe-ovh health network` reports healthy, on release `$TARGET`.
+2. **Readiness.** This must print `true "ok" "ok"`:
+
+ ```bash
+ curl -s https://openvibe.network/api/ready | jq '.ready, .checks.db.status, .checks.signing_key.status'
+ ```
+
+ `status` may be `degraded` only because of optional checks (`registry_poll`, `discord_bot`).
+ `failed` must be `[]`.
+3. **Sign-in.** Sign in at `https://openvibe.network/login` with an account that existed before T0. That proves
+ the imported password hashes. Then `GET /api/auth/me` returns the same username and subject id as before.
+ A session from before T0 is still signed in: same RS256 key, imported `user_sessions`.
+4. **OAuth authorize.** On a first-party site (openvibe.live, then openvibe.chat), sign out, then use "Sign in
+ with OpenVibe". `/oauth/authorize` must redirect back with a code and the site must sign you in. That proves
+ `oauth_clients`, the redirect URIs and token issuance. Also refresh a token on a site that was signed in
+ before T0.
+5. **Notifications.** `GET /api/notifications` lists the notifications from before T0, and
+ `GET /api/notifications/unread-count` answers 200. Trigger a new one, for example a follow from a second
+ account, and check that it arrives.
+6. **OpenCoins wallet.** `GET /api/coins/me` returns the same balance that `$COPY` holds for that user. Check it
+ with this, run in `$STAGE`:
+
+ ```bash
+ node -e "const d = new (require('better-sqlite3'))('$COPY', { readonly: true });
+ console.log(d.prepare('SELECT w.balance FROM wallets w JOIN users u ON u.id = w.user_id WHERE u.username = ?').get('<you>'))"
+ ```
+
+7. **Logs.** `sudo journalctl -u openvibe-network --since "$T1" -p warning` shows no database, PgBouncer or
+ Valkey errors.
+
+**Go:** all seven pass. Close the status-page maintenance. The cutover is done.
+**No-go:** any check fails and cannot be fixed within the window. Go to [rollback](#rollback).
+
+After a go, keep `$COPY` (0400) and the live `data/network.db` untouched for at least 30 days. They are the
+rollback and the audit trail. `$STAGE` may be deleted.
+
+---
+
+## Rollback
+
+```bash
+ov access run openvibe-ovh rollback network
+```
+
+This brings back the previous release, **20dd7ff**, which serves from `data/network.db` again. Nothing in this
+runbook writes to that file: the import read `$COPY`, and the PostgreSQL release never opens `DB_PATH`. The file
+is exactly as it was at T0. The old release ignores `DATABASE_URL`, `DATABASE_DIRECT_URL` and `VALKEY_URL`, so
+they stay in `network.env`.
+
+**Go (rollback worked):** `ov access run openvibe-ovh health network` is healthy on 20dd7ff, and `/api/ready`
+answers 200. Sign-in works.
+
+**What is lost:** every write that reached PostgreSQL between T1 and the rollback. That includes:
+- new accounts, and profile, username and password changes;
+- sessions and refresh tokens issued in that time (those people sign in again);
+- notifications, coin transactions and wallet balances;
+- follows and blocks;
+- preferences and developer projects.
+
+Events that the PostgreSQL release already published stay published. Events treats a repeated `event_id` as a
+duplicate.
+
+**Replay or accept.** Decide before anything else is attempted. A later cutover empties PostgreSQL and imports
+from a new SQLite copy, which discards these rows for good.
+
+- **Accept** when the rollback came within the window and the list below is empty or trivial. Say so on the
+ status page: "changes made between T1 and the rollback were not kept; sign in again if asked".
+- **Replay** money and accounts. On PostgreSQL, as the owner (`DATABASE_DIRECT_URL`, from `$STAGE` with
+ `node --env-file`, or `psql` on the host without echoing the URL), list them:
+
+ ```sql
+ SELECT id, user_id, app_id, delta, reason, ref, idempotency_key, created_at
+ FROM coin_transactions WHERE created_at >= '<T1>' ORDER BY id;
+ SELECT id, username, subject_id, created_at FROM users WHERE created_at >= '<T1>' ORDER BY id;
+ SELECT 'notifications', COUNT(*) FROM notifications WHERE created_at >= '<T1>'
+ UNION ALL SELECT 'user_follows', COUNT(*) FROM user_follows WHERE updated_at >= '<T1>'
+ UNION ALL SELECT 'user_blocks', COUNT(*) FROM user_blocks WHERE updated_at >= '<T1>';
+ ```
+
+ - Re-apply each coin transaction through the old release's coins API with its **same `idempotency_key`**, so a
+ second replay is a no-op. A row without a key is checked by hand against the wallet first.
+ - New accounts are told to sign up again. Their PostgreSQL subject id does not exist on SQLite.
+ - Follows and blocks that the person made again on the old release need nothing more.
+
+ Keep the query output with the incident notes. It holds usernames, so it is not posted publicly.
+
+After the rollback, read the boot log and `ovhost releases network`, fix the cause with a PR, and rehearse again
+on a fresh snapshot (step 2) before a new window.
@@ -13,7 +13,8 @@
@@ -13,7 +13,8 @@
"migrate-to-postgres": "node scripts/migrate-to-postgres.js",
"migrate-to-postgres": "node scripts/migrate-to-postgres.js",
"pg-preflight": "node scripts/pg-preflight-collisions.js",
"pg-preflight": "node scripts/pg-preflight-collisions.js",
"follows-import": "node scripts/follows-import.js",
"follows-import": "node scripts/follows-import.js",
- "follows-preflight": "node scripts/follows-preflight.js"
+ "follows-preflight": "node scripts/follows-preflight.js",
+ "rehearse-pg-cutover": "node scripts/rehearse-pg-cutover.js"
},
},
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
new file mode 100644
new file mode 100644
@@ -0,0 +1,1063 @@
@@ -0,0 +1,1063 @@
+-- OpenVibe.Network SQLite schema at commit 20dd7ff (the last SQLite release, in production before the T2 cutover).
+-- Generated from that commit's server booted once on an empty file (initDb() plus the tables its modules create
+-- lazily at startup: analytics, account merges/exports/deletions, history, frame cache, …): sqlite_master in
+-- creation order. Read by scripts/rehearse-pg-cutover.js to build the seeded rehearsal database. Do not edit by
+-- hand: it is history.
+
+CREATE TABLE users (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ username TEXT UNIQUE NOT NULL,
+ email TEXT UNIQUE,
+ password_hash TEXT NOT NULL,
+ display_name TEXT,
+ avatar_url TEXT,
+ bio TEXT DEFAULT '',
+ role TEXT DEFAULT 'user' CHECK(role IN ('user','streamer','global_mod','admin')),
+ profile_color TEXT DEFAULT '#8b5cf6',
+ is_banned INTEGER DEFAULT 0,
+ ban_reason TEXT,
+ token_valid_after TEXT DEFAULT NULL,
+ legacy_source TEXT, -- 'live' for migrated accounts
+ legacy_id INTEGER, -- original user ID in source platform
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP
+ , is_anon INTEGER DEFAULT 0, anon_number INTEGER, name_effect TEXT, particle_effect TEXT, email_verified INTEGER DEFAULT 0, email_verified_at DATETIME, email_bounced_at DATETIME, email_bounce_reason TEXT, history_paused INTEGER DEFAULT 0, subject_id TEXT, merged_into INTEGER, deleted_at TEXT, profile_revision INTEGER NOT NULL DEFAULT 0);
+
+CREATE TABLE oauth_clients (
+ client_id TEXT PRIMARY KEY,
+ client_secret TEXT NOT NULL,
+ name TEXT NOT NULL,
+ redirect_uris TEXT NOT NULL, -- JSON array of allowed redirect URIs
+ is_first_party INTEGER DEFAULT 0,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ );
+
+CREATE TABLE oauth_codes (
+ code TEXT PRIMARY KEY,
+ client_id TEXT NOT NULL,
+ user_id INTEGER NOT NULL,
+ redirect_uri TEXT NOT NULL,
+ scope TEXT DEFAULT 'profile theme',
+ expires_at DATETIME NOT NULL,
+ used INTEGER DEFAULT 0,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP, code_challenge TEXT, code_challenge_method TEXT, nonce TEXT,
+ FOREIGN KEY (client_id) REFERENCES oauth_clients(client_id),
+ FOREIGN KEY (user_id) REFERENCES users(id)
+ );
+
+CREATE TABLE oauth_tokens (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ token TEXT UNIQUE NOT NULL,
+ client_id TEXT NOT NULL,
+ user_id INTEGER NOT NULL,
+ scope TEXT DEFAULT 'profile theme',
+ expires_at DATETIME NOT NULL,
+ revoked INTEGER DEFAULT 0,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP, family_id TEXT, generation INTEGER NOT NULL DEFAULT 0, revoked_reason TEXT, revoked_at DATETIME,
+ FOREIGN KEY (client_id) REFERENCES oauth_clients(client_id),
+ FOREIGN KEY (user_id) REFERENCES users(id)
+ );
+
+CREATE TABLE user_preferences (
+ user_id INTEGER PRIMARY KEY,
+ theme_id TEXT DEFAULT 'vibe',
+ custom_theme_variables TEXT, -- JSON: custom CSS var overrides
+ language TEXT DEFAULT 'en',
+ notifications_enabled INTEGER DEFAULT 1,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP, display_prefs TEXT,
+ FOREIGN KEY (user_id) REFERENCES users(id)
+ );
+
+CREATE TABLE themes (
+ id TEXT PRIMARY KEY,
+ name TEXT NOT NULL,
+ slug TEXT UNIQUE NOT NULL,
+ author_id INTEGER,
+ description TEXT DEFAULT '',
+ mode TEXT DEFAULT 'dark' CHECK(mode IN ('dark','light')),
+ variables TEXT NOT NULL, -- JSON: CSS variable map
+ preview_colors TEXT, -- JSON: preview color swatches
+ is_builtin INTEGER DEFAULT 0,
+ is_public INTEGER DEFAULT 1,
+ downloads INTEGER DEFAULT 0,
+ rating_sum INTEGER DEFAULT 0,
+ rating_count INTEGER DEFAULT 0,
+ tags TEXT DEFAULT '[]', -- JSON array
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (author_id) REFERENCES users(id)
+ );
+
+CREATE TABLE linked_accounts (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ service TEXT NOT NULL, -- 'live', 'games', etc.
+ service_user_id TEXT NOT NULL,
+ linked_at DATETIME DEFAULT CURRENT_TIMESTAMP, service_username TEXT, last_used_at DATETIME,
+ FOREIGN KEY (user_id) REFERENCES users(id),
+ UNIQUE(service, service_user_id)
+ );
+
+CREATE TABLE audit_log (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER,
+ action TEXT NOT NULL,
+ details TEXT, -- JSON context
+ ip TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ );
+
+CREATE TABLE ip_log (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER,
+ ip TEXT NOT NULL,
+ action TEXT DEFAULT 'login',
+ country TEXT,
+ region TEXT,
+ city TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id)
+ );
+
+CREATE TABLE site_settings (
+ key TEXT PRIMARY KEY,
+ value TEXT,
+ type TEXT DEFAULT 'string'
+ );
+
+CREATE TABLE url_registry (
+ key TEXT PRIMARY KEY,
+ label TEXT NOT NULL,
+ category TEXT NOT NULL,
+ service TEXT NOT NULL,
+ scope TEXT NOT NULL,
+ type TEXT NOT NULL,
+ value TEXT,
+ description TEXT,
+ updated_by INTEGER,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ , source TEXT NOT NULL DEFAULT 'admin');
+
+CREATE TABLE notifications (
+ id TEXT PRIMARY KEY, -- UUID
+ user_id INTEGER NOT NULL,
+ type TEXT NOT NULL, -- from TYPES enum (FOLLOW, MENTION, etc.)
+ category TEXT NOT NULL, -- social, chat, game, stream, economy, etc.
+ priority TEXT NOT NULL DEFAULT 'normal', -- low, normal, high, critical
+ title TEXT NOT NULL,
+ message TEXT,
+ icon TEXT,
+ sender_id INTEGER, -- who triggered this (nullable)
+ sender_name TEXT, -- denormalized for display
+ sender_avatar TEXT,
+ service TEXT, -- originating service (openvibelive, etc.)
+ url TEXT, -- click-through destination
+ rich_content TEXT, -- JSON: images, actions, embeds
+ is_read INTEGER DEFAULT 0,
+ is_dismissed INTEGER DEFAULT 0,
+ is_emailed INTEGER DEFAULT 0,
+ expires_at DATETIME,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_notif_user_read ON notifications(user_id, is_read, created_at DESC);
+
+CREATE INDEX idx_notif_user_cat ON notifications(user_id, category);
+
+CREATE INDEX idx_notif_expires ON notifications(expires_at) WHERE expires_at IS NOT NULL;
+
+CREATE TABLE notification_preferences (
+ user_id INTEGER NOT NULL,
+ category TEXT NOT NULL, -- or '*' for global
+ enabled INTEGER DEFAULT 1,
+ sound INTEGER DEFAULT 1,
+ toasts INTEGER DEFAULT 1,
+ email INTEGER DEFAULT 0,
+ PRIMARY KEY (user_id, category),
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE TABLE email_delivery_log (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ email_type TEXT NOT NULL,
+ recipient TEXT NOT NULL,
+ subject TEXT,
+ status TEXT NOT NULL CHECK(status IN ('sent', 'failed')),
+ error_message TEXT,
+ user_id INTEGER,
+ notification_id TEXT,
+ metadata TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE SET NULL,
+ FOREIGN KEY (notification_id) REFERENCES notifications(id) ON DELETE SET NULL
+ );
+
+CREATE INDEX idx_email_delivery_created ON email_delivery_log(created_at DESC);
+
+CREATE INDEX idx_email_delivery_status ON email_delivery_log(status, created_at DESC);
+
+CREATE INDEX idx_email_delivery_type ON email_delivery_log(email_type, created_at DESC);
+
+CREATE TABLE password_reset_tokens (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ token_hash TEXT UNIQUE NOT NULL,
+ expires_at DATETIME NOT NULL,
+ used_at DATETIME,
+ requested_ip TEXT,
+ requested_user_agent TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_password_reset_user ON password_reset_tokens(user_id, used_at, expires_at);
+
+CREATE INDEX idx_password_reset_expires ON password_reset_tokens(expires_at);
+
+CREATE TABLE anon_users (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ anon_number INTEGER UNIQUE NOT NULL,
+ fingerprint TEXT, -- browser fingerprint hash (optional)
+ session_token TEXT UNIQUE NOT NULL,
+ display_name TEXT,
+ preferences TEXT DEFAULT '{}', -- JSON
+ total_messages INTEGER DEFAULT 0,
+ total_commands INTEGER DEFAULT 0,
+ first_seen DATETIME DEFAULT CURRENT_TIMESTAMP,
+ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP
+ , ip TEXT, subject_id TEXT);
+
+CREATE TABLE user_sessions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ session_token TEXT UNIQUE NOT NULL,
+ device_name TEXT,
+ ip TEXT,
+ user_agent TEXT,
+ is_active INTEGER DEFAULT 1,
+ last_used DATETIME DEFAULT CURRENT_TIMESTAMP,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ expires_at DATETIME NOT NULL,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_sessions_user ON user_sessions(user_id, is_active);
+
+CREATE INDEX idx_sessions_token ON user_sessions(session_token);
+
+CREATE TABLE user_effects (
+ user_id INTEGER NOT NULL,
+ effect_type TEXT NOT NULL, -- 'name' or 'particle'
+ effect_id TEXT NOT NULL, -- e.g. 'rainbow', 'fire', 'neon'
+ is_active INTEGER DEFAULT 0,
+ acquired_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ PRIMARY KEY (user_id, effect_type, effect_id),
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE TABLE follows (
+ follower_id INTEGER NOT NULL,
+ followed_id INTEGER NOT NULL,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ PRIMARY KEY (follower_id, followed_id),
+ FOREIGN KEY (follower_id) REFERENCES users(id) ON DELETE CASCADE,
+ FOREIGN KEY (followed_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE TABLE wallets (
+ user_id INTEGER PRIMARY KEY,
+ balance INTEGER NOT NULL DEFAULT 0,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE TABLE coin_transactions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ app_id TEXT,
+ delta INTEGER NOT NULL,
+ reason TEXT,
+ ref TEXT,
+ idempotency_key TEXT UNIQUE,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_coin_tx_user ON coin_transactions(user_id, created_at DESC);
+
+CREATE INDEX idx_coin_tx_app ON coin_transactions(app_id, created_at DESC);
+
+CREATE TABLE verification_keys (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ key TEXT UNIQUE NOT NULL,
+ target_username TEXT NOT NULL,
+ note TEXT DEFAULT '',
+ created_by INTEGER NOT NULL,
+ used_by INTEGER,
+ status TEXT DEFAULT 'active' CHECK(status IN ('active', 'used', 'revoked')),
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ used_at DATETIME,
+ FOREIGN KEY (created_by) REFERENCES users(id) ON DELETE CASCADE,
+ FOREIGN KEY (used_by) REFERENCES users(id) ON DELETE SET NULL
+ );
+
+CREATE INDEX idx_vkeys_key ON verification_keys(key);
+
+CREATE INDEX idx_vkeys_target ON verification_keys(target_username);
+
+CREATE INDEX idx_vkeys_status ON verification_keys(status);
+
+CREATE TABLE anon_ip_log (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ anon_id INTEGER NOT NULL,
+ ip TEXT NOT NULL,
+ first_seen DATETIME DEFAULT CURRENT_TIMESTAMP,
+ last_seen DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (anon_id) REFERENCES anon_users(id) ON DELETE CASCADE,
+ UNIQUE(anon_id, ip)
+ );
+
+CREATE INDEX idx_anon_ip_log_ip ON anon_ip_log(ip);
+
+CREATE INDEX idx_anon_ip_log_anon ON anon_ip_log(anon_id);
+
+CREATE TABLE push_subscriptions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ endpoint TEXT NOT NULL UNIQUE,
+ keys_p256dh TEXT NOT NULL,
+ keys_auth TEXT NOT NULL,
+ user_agent TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_push_sub_user ON push_subscriptions(user_id);
+
+CREATE INDEX idx_push_sub_endpoint ON push_subscriptions(endpoint);
+
+CREATE INDEX idx_oauth_tokens_family ON oauth_tokens(family_id);
+
+CREATE TABLE email_verification_tokens (
+ token_hash TEXT PRIMARY KEY,
+ user_id INTEGER NOT NULL,
+ email TEXT NOT NULL,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ expires_at DATETIME NOT NULL,
+ used_at DATETIME,
+ FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
+ );
+
+CREATE INDEX idx_email_verify_user ON email_verification_tokens(user_id, created_at DESC);
+
+CREATE UNIQUE INDEX idx_linked_user_service ON linked_accounts(user_id, service);
+
+CREATE TABLE identity_legacy_map (
+ source_system TEXT NOT NULL,
+ source_type TEXT NOT NULL,
+ source_id TEXT NOT NULL,
+ subject_id TEXT NOT NULL,
+ first_seen_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ verified_at DATETIME,
+ metadata TEXT,
+ PRIMARY KEY (source_system, source_type, source_id)
+ );
+
+CREATE INDEX idx_identity_legacy_subject ON identity_legacy_map(subject_id);
+
+CREATE UNIQUE INDEX idx_users_subject_id ON users(subject_id);
+
+CREATE UNIQUE INDEX idx_anon_users_subject_id ON anon_users(subject_id);
+
+CREATE TABLE principal_grants (
+ client_id TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ audience TEXT NOT NULL,
+ namespaces TEXT NOT NULL DEFAULT '[]',
+ granted_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ granted_by TEXT,
+ revoked_at DATETIME, expires_at DATETIME, reason TEXT, revoked_by TEXT,
+ PRIMARY KEY (client_id, capability, audience)
+ );
+
+CREATE TABLE principal_usage (
+ principal TEXT NOT NULL,
+ route TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ auth TEXT NOT NULL,
+ allowed INTEGER NOT NULL,
+ code TEXT NOT NULL DEFAULT '',
+ count INTEGER NOT NULL DEFAULT 0,
+ first_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ last_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ PRIMARY KEY (principal, route, auth, allowed, code)
+ );
+
+CREATE TABLE principal_grant_changes (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ client_id TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ audience TEXT NOT NULL,
+ change TEXT NOT NULL CHECK (change IN ('granted', 'updated', 'revoked', 'expired')),
+ namespaces TEXT NOT NULL DEFAULT '[]',
+ expires_at TEXT,
+ reason TEXT,
+ actor TEXT,
+ event_id TEXT,
+ at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
+ );
+
+CREATE INDEX idx_grant_changes_client ON principal_grant_changes(client_id, id);
+
+CREATE TABLE user_modules (
+ subject_id TEXT NOT NULL,
+ namespace TEXT NOT NULL,
+ version INTEGER NOT NULL,
+ revision INTEGER NOT NULL DEFAULT 0,
+ data TEXT NOT NULL,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ updated_by TEXT,
+ PRIMARY KEY (subject_id, namespace)
+ );
+
+CREATE TABLE user_module_revisions (
+ subject_id TEXT NOT NULL,
+ namespace TEXT NOT NULL,
+ revision INTEGER NOT NULL,
+ PRIMARY KEY (subject_id, namespace)
+ );
+
+CREATE TABLE user_module_retirements (
+ namespace TEXT PRIMARY KEY,
+ owner TEXT NOT NULL,
+ retired_seen_at INTEGER NOT NULL
+ );
+
+CREATE TABLE network_event_outbox (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ event_id TEXT NOT NULL UNIQUE,
+ envelope TEXT NOT NULL,
+ traceparent TEXT,
+ created_at INTEGER NOT NULL,
+ attempts INTEGER NOT NULL DEFAULT 0,
+ next_attempt_at INTEGER NOT NULL DEFAULT 0,
+ sent_at INTEGER,
+ seq INTEGER,
+ rejected_at INTEGER,
+ last_error TEXT
+ );
+
+CREATE INDEX idx_network_event_outbox_due ON network_event_outbox(sent_at, rejected_at, next_attempt_at);
+
+CREATE TRIGGER user_modules_guard_users_delete BEFORE DELETE ON users
+ WHEN OLD.subject_id IS NOT NULL AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)
+ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectRemoved or onSubjectMerged first'); END;
+
+CREATE TRIGGER user_modules_guard_users_rekey BEFORE UPDATE OF subject_id ON users
+ WHEN OLD.subject_id IS NOT NULL AND NEW.subject_id IS NOT OLD.subject_id
+ AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)
+ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectMerged first'); END;
+
+CREATE TRIGGER user_modules_guard_anon_users_delete BEFORE DELETE ON anon_users
+ WHEN OLD.subject_id IS NOT NULL AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)
+ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectRemoved or onSubjectMerged first'); END;
+
+CREATE TRIGGER user_modules_guard_anon_users_rekey BEFORE UPDATE OF subject_id ON anon_users
+ WHEN OLD.subject_id IS NOT NULL AND NEW.subject_id IS NOT OLD.subject_id
+ AND EXISTS (SELECT 1 FROM user_modules WHERE subject_id = OLD.subject_id)
+ BEGIN SELECT RAISE(ABORT, 'user_modules rows remain for this subject: call modules.onSubjectMerged first'); END;
+
+CREATE TABLE dev_projects (
+ id TEXT PRIMARY KEY,
+ owner_subject TEXT NOT NULL,
+ name TEXT NOT NULL,
+ environment_policy TEXT NOT NULL DEFAULT 'sandbox' CHECK (environment_policy IN ('sandbox', 'sandbox+production')),
+ allowance TEXT NOT NULL DEFAULT '[]',
+ created_at TEXT NOT NULL,
+ created_by TEXT NOT NULL,
+ archived_at TEXT,
+ archived_by TEXT
+ );
+
+CREATE INDEX idx_dev_projects_owner ON dev_projects(owner_subject);
+
+CREATE TABLE dev_project_members (
+ project_id TEXT NOT NULL REFERENCES dev_projects(id),
+ subject_id TEXT NOT NULL,
+ role TEXT NOT NULL CHECK (role IN ('owner', 'admin', 'developer', 'viewer')),
+ added_at TEXT NOT NULL,
+ added_by TEXT NOT NULL,
+ PRIMARY KEY (project_id, subject_id)
+ );
+
+CREATE INDEX idx_dev_members_subject ON dev_project_members(subject_id);
+
+CREATE TABLE dev_apps (
+ id TEXT PRIMARY KEY,
+ project_id TEXT NOT NULL REFERENCES dev_projects(id),
+ name TEXT NOT NULL,
+ environment TEXT NOT NULL CHECK (environment IN ('sandbox', 'production')),
+ oauth_client_id TEXT NOT NULL UNIQUE,
+ client_type TEXT NOT NULL CHECK (client_type IN ('confidential', 'public')),
+ redirect_uris TEXT NOT NULL DEFAULT '[]',
+ created_at TEXT NOT NULL,
+ created_by TEXT NOT NULL,
+ revoked_at TEXT,
+ revoked_by TEXT
+ );
+
+CREATE INDEX idx_dev_apps_project ON dev_apps(project_id);
+
+CREATE TABLE dev_credentials (
+ id TEXT PRIMARY KEY,
+ app_id TEXT NOT NULL REFERENCES dev_apps(id),
+ secret_hash TEXT NOT NULL UNIQUE,
+ hint TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ created_by TEXT NOT NULL,
+ expires_at TEXT,
+ revoked_at TEXT,
+ revoked_by TEXT,
+ last_used_at TEXT
+ );
+
+CREATE INDEX idx_dev_credentials_app ON dev_credentials(app_id);
+
+CREATE TABLE dev_grants (
+ app_id TEXT NOT NULL REFERENCES dev_apps(id),
+ capability TEXT NOT NULL,
+ audience TEXT NOT NULL,
+ status TEXT NOT NULL CHECK (status IN ('requested', 'approved', 'denied', 'revoked')),
+ requested_by TEXT NOT NULL,
+ requested_at TEXT NOT NULL,
+ decided_by TEXT,
+ decided_at TEXT,
+ PRIMARY KEY (app_id, capability)
+ );
+
+CREATE TABLE dev_quotas (
+ project_id TEXT NOT NULL REFERENCES dev_projects(id),
+ capability TEXT NOT NULL,
+ limit_value INTEGER NOT NULL CHECK (limit_value >= 0),
+ quota_window TEXT NOT NULL CHECK (quota_window IN ('minute', 'hour', 'day', 'month', 'total')),
+ unit TEXT NOT NULL DEFAULT 'requests',
+ updated_at TEXT NOT NULL,
+ updated_by TEXT NOT NULL,
+ PRIMARY KEY (project_id, capability)
+ );
+
+CREATE TABLE dev_auth_codes (
+ code_hash TEXT PRIMARY KEY,
+ app_id TEXT NOT NULL REFERENCES dev_apps(id),
+ user_subject TEXT NOT NULL,
+ redirect_uri TEXT NOT NULL,
+ scope TEXT NOT NULL DEFAULT '',
+ code_challenge TEXT NOT NULL,
+ expires_at TEXT NOT NULL,
+ used INTEGER NOT NULL DEFAULT 0
+ );
+
+CREATE TABLE dev_audit (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ at TEXT NOT NULL,
+ project_id TEXT,
+ actor TEXT NOT NULL,
+ action TEXT NOT NULL,
+ target TEXT,
+ detail TEXT NOT NULL DEFAULT '{}',
+ request_id TEXT,
+ event_type TEXT,
+ event TEXT
+ );
+
+CREATE INDEX idx_dev_audit_project ON dev_audit(project_id, id);
+
+CREATE TRIGGER dev_audit_append_only_update BEFORE UPDATE ON dev_audit
+ BEGIN SELECT RAISE(ABORT, 'dev_audit is append-only'); END;
+
+CREATE TRIGGER dev_audit_append_only_delete BEFORE DELETE ON dev_audit
+ BEGIN SELECT RAISE(ABORT, 'dev_audit is append-only'); END;
+
+CREATE TABLE dev_usage_windows (
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL CHECK (env IN ('sandbox', 'production')),
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ dimension TEXT NOT NULL DEFAULT '',
+ unit TEXT NOT NULL,
+ window_start TEXT NOT NULL,
+ window_end TEXT NOT NULL,
+ quantity INTEGER NOT NULL,
+ errors INTEGER NOT NULL,
+ error_codes TEXT NOT NULL DEFAULT '{}',
+ revision INTEGER NOT NULL DEFAULT 1,
+ event_id TEXT NOT NULL,
+ recorded_at TEXT NOT NULL,
+ PRIMARY KEY (project_id, env, service, capability, dimension, unit, window_start)
+ );
+
+CREATE INDEX idx_dev_usage_windows_when ON dev_usage_windows(window_start);
+
+CREATE INDEX idx_dev_usage_windows_project ON dev_usage_windows(project_id, recorded_at);
+
+CREATE TABLE dev_usage_daily (
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL,
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ dimension TEXT NOT NULL DEFAULT '',
+ unit TEXT NOT NULL,
+ day TEXT NOT NULL,
+ quantity INTEGER NOT NULL DEFAULT 0,
+ errors INTEGER NOT NULL DEFAULT 0,
+ error_codes TEXT NOT NULL DEFAULT '{}',
+ updated_at TEXT NOT NULL,
+ PRIMARY KEY (project_id, env, service, capability, dimension, unit, day)
+ );
+
+CREATE INDEX idx_dev_usage_daily_day ON dev_usage_daily(project_id, day);
+
+CREATE TABLE dev_usage_errors (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL,
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ at TEXT NOT NULL,
+ code TEXT NOT NULL,
+ status INTEGER,
+ trace_id TEXT,
+ ref TEXT NOT NULL DEFAULT '',
+ event_id TEXT NOT NULL
+ );
+
+CREATE UNIQUE INDEX idx_dev_usage_errors_once ON dev_usage_errors(project_id, env, service, capability, at, code, ref);
+
+CREATE INDEX idx_dev_usage_errors_recent ON dev_usage_errors(project_id, at);
+
+CREATE TABLE user_blocks (
+ blocker_subject TEXT NOT NULL,
+ blocked_subject TEXT NOT NULL,
+ active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),
+ revision INTEGER NOT NULL DEFAULT 1 CHECK (revision >= 1),
+ created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
+ updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
+ PRIMARY KEY (blocker_subject, blocked_subject),
+ CHECK (blocker_subject <> blocked_subject)
+ );
+
+CREATE INDEX idx_user_blocks_blocked ON user_blocks(blocked_subject, active);
+
+CREATE TABLE user_follows (
+ follower_subject TEXT NOT NULL,
+ target_type TEXT NOT NULL CHECK (target_type IN ('channel')),
+ target_id TEXT NOT NULL,
+ active INTEGER NOT NULL DEFAULT 1 CHECK (active IN (0, 1)),
+ notify_email INTEGER NOT NULL DEFAULT 1 CHECK (notify_email IN (0, 1)),
+ notify_push INTEGER NOT NULL DEFAULT 1 CHECK (notify_push IN (0, 1)),
+ revision INTEGER NOT NULL DEFAULT 1 CHECK (revision >= 1),
+ source TEXT NOT NULL DEFAULT 'network',
+ created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
+ updated_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
+ PRIMARY KEY (follower_subject, target_type, target_id),
+ CHECK (follower_subject <> target_id)
+ );
+
+CREATE INDEX idx_user_follows_target ON user_follows(target_type, target_id, active, created_at);
+
+CREATE TABLE follow_import_holds (
+ source TEXT NOT NULL,
+ follower_ref TEXT NOT NULL,
+ target_ref TEXT NOT NULL,
+ reason TEXT NOT NULL,
+ seen_at TEXT NOT NULL,
+ PRIMARY KEY (source, follower_ref, target_ref)
+ );
+
+CREATE TABLE status_incidents (
+ id TEXT PRIMARY KEY,
+ kind TEXT NOT NULL CHECK (kind IN ('incident', 'maintenance')),
+ title TEXT NOT NULL,
+ severity TEXT CHECK (severity IS NULL OR severity IN ('minor', 'major', 'critical')),
+ state TEXT NOT NULL,
+ services TEXT NOT NULL DEFAULT '[]',
+ starts_at TEXT NOT NULL,
+ ends_at TEXT,
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+ );
+
+CREATE TABLE status_incident_updates (
+ incident_id TEXT NOT NULL REFERENCES status_incidents(id),
+ at TEXT NOT NULL,
+ state TEXT NOT NULL,
+ message TEXT NOT NULL,
+ author TEXT NOT NULL
+ );
+
+CREATE INDEX idx_status_incident_updates ON status_incident_updates(incident_id, at);
+
+CREATE TABLE creator_streams (
+ stream_id INTEGER PRIMARY KEY,
+ creator_subject TEXT NOT NULL,
+ title TEXT,
+ category TEXT,
+ started_at TEXT NOT NULL,
+ ended_at TEXT NOT NULL,
+ duration_seconds INTEGER NOT NULL DEFAULT 0,
+ peak_viewers INTEGER,
+ avg_viewers REAL,
+ unique_chatters INTEGER,
+ messages INTEGER,
+ watch_minutes INTEGER,
+ received_at TEXT NOT NULL
+ );
+
+CREATE INDEX idx_creator_streams_creator ON creator_streams(creator_subject, started_at);
+
+CREATE TABLE platform_nodes (
+ id TEXT PRIMARY KEY,
+ source TEXT NOT NULL,
+ doc TEXT NOT NULL,
+ status TEXT NOT NULL,
+ reported_at TEXT NOT NULL
+ );
+
+CREATE TABLE analytics_events (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ service TEXT NOT NULL,
+ event_type TEXT NOT NULL DEFAULT 'pageview',
+ path TEXT,
+ method TEXT DEFAULT 'GET',
+ status_code INTEGER,
+ response_time_ms INTEGER,
+ user_id INTEGER,
+ session_id TEXT,
+ ip TEXT,
+ country TEXT,
+ city TEXT,
+ user_agent TEXT,
+ referer TEXT,
+ is_bot INTEGER DEFAULT 0,
+ bot_type TEXT,
+ device_type TEXT,
+ browser TEXT,
+ os TEXT,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ , authenticated INTEGER NOT NULL DEFAULT 0);
+
+CREATE TABLE analytics_hourly (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ service TEXT NOT NULL,
+ hour TEXT NOT NULL,
+ pageviews INTEGER DEFAULT 0,
+ api_calls INTEGER DEFAULT 0,
+ unique_visitors INTEGER DEFAULT 0,
+ unique_users INTEGER DEFAULT 0,
+ bot_hits INTEGER DEFAULT 0,
+ avg_response_ms INTEGER DEFAULT 0,
+ error_count INTEGER DEFAULT 0,
+ bandwidth_bytes INTEGER DEFAULT 0,
+ top_paths TEXT,
+ top_referers TEXT,
+ UNIQUE(service, hour)
+ );
+
+CREATE TABLE analytics_daily (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ service TEXT NOT NULL,
+ date TEXT NOT NULL,
+ pageviews INTEGER DEFAULT 0,
+ api_calls INTEGER DEFAULT 0,
+ unique_visitors INTEGER DEFAULT 0,
+ unique_users INTEGER DEFAULT 0,
+ new_users INTEGER DEFAULT 0,
+ bot_hits INTEGER DEFAULT 0,
+ avg_response_ms INTEGER DEFAULT 0,
+ error_count INTEGER DEFAULT 0,
+ top_paths TEXT,
+ top_referers TEXT,
+ top_countries TEXT,
+ device_breakdown TEXT,
+ browser_breakdown TEXT,
+ UNIQUE(service, date)
+ );
+
+CREATE TABLE analytics_rate_tracking (
+ ip TEXT NOT NULL,
+ window_start INTEGER NOT NULL,
+ hit_count INTEGER DEFAULT 1,
+ PRIMARY KEY (ip, window_start)
+ );
+
+CREATE TABLE analytics_visitor_days (
+ service TEXT NOT NULL,
+ day TEXT NOT NULL,
+ hour TEXT NOT NULL,
+ vhash TEXT NOT NULL,
+ authenticated INTEGER NOT NULL DEFAULT 0,
+ PRIMARY KEY (service, hour, vhash)
+ );
+
+CREATE TABLE analytics_day_salts (
+ service TEXT NOT NULL,
+ day TEXT NOT NULL,
+ salt TEXT NOT NULL,
+ PRIMARY KEY (service, day)
+ );
+
+CREATE INDEX idx_analytics_events_created ON analytics_events(created_at);
+
+CREATE INDEX idx_analytics_events_service ON analytics_events(service, created_at);
+
+CREATE INDEX idx_analytics_events_path ON analytics_events(service, path, created_at);
+
+CREATE INDEX idx_analytics_events_bot ON analytics_events(is_bot, created_at);
+
+CREATE INDEX idx_analytics_hourly_lookup ON analytics_hourly(service, hour);
+
+CREATE INDEX idx_analytics_daily_lookup ON analytics_daily(service, date);
+
+CREATE INDEX idx_analytics_visitor_days_day ON analytics_visitor_days(service, day);
+
+CREATE TABLE moderation_audit (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ event_id TEXT NOT NULL UNIQUE,
+ service TEXT NOT NULL,
+ action TEXT NOT NULL,
+ actor_subject TEXT,
+ target_type TEXT,
+ target_id TEXT,
+ target_subject TEXT,
+ scope TEXT,
+ reason TEXT,
+ details TEXT NOT NULL DEFAULT '{}',
+ occurred_at TEXT NOT NULL,
+ recorded_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now'))
+ );
+
+CREATE INDEX idx_modaudit_when ON moderation_audit(occurred_at);
+
+CREATE INDEX idx_modaudit_service ON moderation_audit(service, id);
+
+CREATE INDEX idx_modaudit_actor ON moderation_audit(actor_subject, id);
+
+CREATE TABLE network_event_inbox (
+ consumer TEXT NOT NULL,
+ event_id TEXT NOT NULL,
+ processed_at INTEGER NOT NULL,
+ PRIMARY KEY (consumer, event_id)
+ );
+
+CREATE TABLE subject_aliases (
+ alias_id TEXT PRIMARY KEY,
+ subject_id TEXT NOT NULL,
+ merge_id TEXT NOT NULL,
+ merged_at TEXT NOT NULL,
+ merged_by TEXT NOT NULL
+ );
+
+CREATE INDEX idx_subject_aliases_subject ON subject_aliases(subject_id);
+
+CREATE TABLE account_merges (
+ id TEXT PRIMARY KEY,
+ from_subject TEXT NOT NULL UNIQUE,
+ into_subject TEXT NOT NULL,
+ from_user_id INTEGER NOT NULL,
+ into_user_id INTEGER NOT NULL,
+ initiated_by TEXT NOT NULL CHECK (initiated_by IN ('person', 'staff')),
+ actor_subject TEXT NOT NULL,
+ reason TEXT,
+ moved TEXT NOT NULL,
+ pre_state TEXT,
+ merged_at TEXT NOT NULL,
+ split_until TEXT NOT NULL,
+ reduced_at TEXT
+ );
+
+CREATE TABLE account_merge_intents (
+ id TEXT PRIMARY KEY,
+ into_user_id INTEGER NOT NULL,
+ created_at INTEGER NOT NULL,
+ expires_at INTEGER NOT NULL,
+ used_at INTEGER
+ );
+
+CREATE TABLE mod_principals (
+ mod_id TEXT PRIMARY KEY,
+ owner TEXT NOT NULL,
+ runtime TEXT NOT NULL,
+ name TEXT,
+ version TEXT,
+ publisher TEXT,
+ manifest TEXT NOT NULL,
+ status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'revoked')),
+ revision INTEGER NOT NULL DEFAULT 1,
+ created_at TEXT NOT NULL,
+ created_by TEXT,
+ updated_at TEXT NOT NULL,
+ revoked_at TEXT,
+ revoked_by TEXT
+ );
+
+CREATE TABLE mod_principal_grants (
+ mod_id TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ state TEXT NOT NULL CHECK (state IN ('pending', 'approved', 'revoked')),
+ changed_at TEXT NOT NULL,
+ changed_by TEXT,
+ PRIMARY KEY (mod_id, capability)
+ );
+
+CREATE TABLE account_exports (
+ id TEXT PRIMARY KEY,
+ user_id INTEGER NOT NULL,
+ subject TEXT NOT NULL,
+ status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'ready', 'partial', 'expired', 'failed')),
+ expected TEXT NOT NULL DEFAULT '[]',
+ requested_at TEXT NOT NULL,
+ deadline TEXT NOT NULL,
+ ready_at TEXT,
+ expires_at TEXT,
+ size_bytes INTEGER,
+ services TEXT,
+ error TEXT
+ );
+
+CREATE INDEX idx_account_exports_user ON account_exports(user_id, requested_at);
+
+CREATE TABLE account_export_parts (
+ export_id TEXT NOT NULL,
+ service TEXT NOT NULL,
+ received_at TEXT NOT NULL,
+ files INTEGER NOT NULL,
+ bytes INTEGER NOT NULL,
+ body TEXT,
+ PRIMARY KEY (export_id, service)
+ );
+
+CREATE TABLE account_deletions (
+ id TEXT PRIMARY KEY,
+ user_id INTEGER NOT NULL,
+ subject TEXT NOT NULL,
+ status TEXT NOT NULL DEFAULT 'scheduled' CHECK (status IN ('scheduled', 'cancelled', 'deleted')),
+ requested_at TEXT NOT NULL,
+ delete_after TEXT NOT NULL,
+ cancelled_at TEXT,
+ cancelled_by TEXT,
+ deleted_at TEXT,
+ expected TEXT,
+ erased TEXT
+ );
+
+CREATE UNIQUE INDEX idx_account_deletions_open ON account_deletions(user_id) WHERE status = 'scheduled';
+
+CREATE TABLE account_deletion_confirmations (
+ deletion_id TEXT NOT NULL,
+ service TEXT NOT NULL,
+ received_at TEXT NOT NULL,
+ completed_at TEXT NOT NULL,
+ erased TEXT NOT NULL,
+ retained TEXT NOT NULL,
+ PRIMARY KEY (deletion_id, service)
+ );
+
+CREATE TABLE user_profile_changes (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ changed TEXT NOT NULL,
+ at INTEGER NOT NULL
+ );
+
+CREATE INDEX idx_user_profile_changes_user ON user_profile_changes(user_id);
+
+CREATE TRIGGER users_profile_changed AFTER UPDATE OF username, display_name, avatar_url, profile_color, role, is_banned ON users
+ WHEN COALESCE(NEW.is_anon, 0) = 0 AND (OLD.username IS NOT NEW.username OR OLD.display_name IS NOT NEW.display_name OR OLD.avatar_url IS NOT NEW.avatar_url OR OLD.profile_color IS NOT NEW.profile_color OR OLD.role IS NOT NEW.role OR OLD.is_banned IS NOT NEW.is_banned)
+ BEGIN INSERT INTO user_profile_changes (user_id, changed, at) VALUES (NEW.id, trim((CASE WHEN OLD.username IS NOT NEW.username THEN 'username ' ELSE '' END) || (CASE WHEN OLD.display_name IS NOT NEW.display_name THEN 'display_name ' ELSE '' END) || (CASE WHEN OLD.avatar_url IS NOT NEW.avatar_url THEN 'avatar_url ' ELSE '' END) || (CASE WHEN OLD.profile_color IS NOT NEW.profile_color THEN 'profile_color ' ELSE '' END) || (CASE WHEN OLD.role IS NOT NEW.role THEN 'role ' ELSE '' END) || (CASE WHEN OLD.is_banned IS NOT NEW.is_banned THEN 'banned ' ELSE '' END)), CAST(strftime('%s', 'now') AS INTEGER) * 1000); END;
+
+CREATE TRIGGER users_profile_created AFTER INSERT ON users
+ WHEN COALESCE(NEW.is_anon, 0) = 0
+ BEGIN INSERT INTO user_profile_changes (user_id, changed, at) VALUES (NEW.id, 'created', CAST(strftime('%s', 'now') AS INTEGER) * 1000); END;
+
+CREATE TABLE user_history (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ service TEXT,
+ sub TEXT,
+ type TEXT NOT NULL DEFAULT 'page',
+ title TEXT NOT NULL,
+ url TEXT NOT NULL,
+ icon TEXT,
+ meta TEXT,
+ hits INTEGER NOT NULL DEFAULT 1,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ FOREIGN KEY (user_id) REFERENCES users(id)
+ );
+
+CREATE INDEX idx_user_history_user_time ON user_history(user_id, updated_at DESC);
+
+CREATE INDEX idx_user_history_user_url ON user_history(user_id, url);
+
+CREATE TABLE frame_cache (key TEXT PRIMARY KEY, value TEXT NOT NULL, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);
+
+CREATE TABLE frame_hits (day TEXT NOT NULL, host TEXT NOT NULL, hits INTEGER NOT NULL DEFAULT 0, PRIMARY KEY (day, host));
+
+CREATE TABLE tool_domains (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ tool_id TEXT NOT NULL,
+ host TEXT NOT NULL UNIQUE,
+ role TEXT NOT NULL DEFAULT 'alias' CHECK (role IN ('canonical', 'short', 'alias', 'mirror')),
+ enabled INTEGER NOT NULL DEFAULT 1,
+ note TEXT,
+ created_by INTEGER,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ );
+
+CREATE INDEX idx_tool_domains_tool ON tool_domains(tool_id, role, enabled);
+
+CREATE TABLE operator_alerts (
+ source TEXT NOT NULL,
+ fingerprint TEXT NOT NULL,
+ name TEXT NOT NULL,
+ severity TEXT NOT NULL,
+ summary TEXT NOT NULL,
+ description TEXT,
+ service TEXT,
+ state TEXT NOT NULL CHECK (state IN ('firing', 'resolved')),
+ started_at TEXT NOT NULL,
+ opened_at INTEGER NOT NULL,
+ last_seen_at INTEGER NOT NULL,
+ notified_at INTEGER,
+ resolved_at INTEGER, notices TEXT, reopened INTEGER NOT NULL DEFAULT 0,
+ PRIMARY KEY (source, fingerprint)
+ );
+
+CREATE TABLE stream_live_announcements (
+ id INTEGER PRIMARY KEY AUTOINCREMENT, streamer_key TEXT NOT NULL, stream_id TEXT, sent_at DATETIME DEFAULT CURRENT_TIMESTAMP);
+
+CREATE INDEX idx_sla_key ON stream_live_announcements(streamer_key, sent_at DESC);
+
+CREATE TABLE username_history (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ user_id INTEGER NOT NULL,
+ old_username TEXT NOT NULL,
+ new_username TEXT NOT NULL,
+ changed_by INTEGER,
+ changed_at DATETIME DEFAULT CURRENT_TIMESTAMP
+ );
+
+CREATE INDEX idx_username_history_old ON username_history(old_username COLLATE NOCASE);
+
+CREATE INDEX idx_username_history_user ON username_history(user_id);
+
+CREATE TABLE admin_rate_limits (
+ user_id INTEGER NOT NULL,
+ action TEXT NOT NULL,
+ last_at INTEGER NOT NULL,
+ PRIMARY KEY (user_id, action)
+ );
@@ -2,7 +2,7 @@
@@ -2,7 +2,7 @@
'use strict';
'use strict';
/**
/**
* The one-time move of Network's SQLite database (DB_PATH) into its PostgreSQL schema (plan T2, ADR-035; the
* The one-time move of Network's SQLite database (DB_PATH) into its PostgreSQL schema (plan T2, ADR-035; the
- * procedure is openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by runSqliteMigration).
+ * procedure is openvibe-sdk docs/migrating-to-postgresql.md, section 6, the steps of openvibe-sdk's runSqliteMigration).
* Opus runs it at the cutover, after scripts/pg-preflight-collisions.js.
* Opus runs it at the cutover, after scripts/pg-preflight-collisions.js.
*
*
* node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
* node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
@@ -19,10 +19,16 @@
@@ -19,10 +19,16 @@
*
*
* The legacy integer-keyed `follows` table is imported like every other: server/auth/routes.js and
* The legacy integer-keyed `follows` table is imported like every other: server/auth/routes.js and
* server/internal/routes.js still read it (the subject-keyed user_follows is the forward model).
* server/internal/routes.js still read it (the subject-keyed user_follows is the forward model).
+ *
+ * The steps are runSqliteMigration's (migrate, cleaning options, importSqlite with truncate, report), run here
+ * so QUIET_TRIGGERS can be off while the rows are copied. A run that dies between the two ALTERs leaves them
+ * off: run the import again (it re-enables them at the end), never boot on such a database.
*/
*/
require('dotenv').config();
require('dotenv').config();
const path = require('path');
const path = require('path');
-const { runSqliteMigration } = require('openvibe-sdk/db');
+const { createDb, importSqlite } = require('openvibe-sdk/db');
+// cleaningOptions is sqlite-cli.js's export, beside openvibe-sdk/db's entry (the package exports map does not list it).
+const { cleaningOptions } = require(path.join(path.dirname(require.resolve('openvibe-sdk/db')), 'sqlite-cli.js'));
const config = require('../server/config');
const config = require('../server/config');
const MIGRATIONS = path.join(__dirname, '..', 'migrations');
const MIGRATIONS = path.join(__dirname, '..', 'migrations');
@@ -30,16 +36,62 @@ const MIGRATIONS = path.join(__dirname, '..', 'migrations');
@@ -30,16 +36,62 @@ const MIGRATIONS = path.join(__dirname, '..', 'migrations');
// importSqlite's per-table options. Empty: every PostgreSQL column is fed by the same-named SQLite one.
// importSqlite's per-table options. Empty: every PostgreSQL column is fed by the same-named SQLite one.
const TABLES = {};
const TABLES = {};
-// SQLite tables the import leaves behind. Empty: even the SDK's SQLite outbox/inbox rows are carried over
-// (unsent events keep their place in the PostgreSQL outbox).
-const SKIP_SOURCE = [];
+// SQLite tables the import leaves behind. The SDK's SQLite outbox/inbox rows are carried over (unsent events
+// keep their place in the PostgreSQL outbox). Left behind:
+// analytics_rate_tracking openvibe-shared's old per-IP bot-rate counters. The rate check keeps them in
+// memory only now and PostgreSQL has no such table: IPs are never carried over.
+const SKIP_SOURCE = ['analytics_rate_tracking'];
+
+// Triggers that derive rows from an insert. The import copies those rows from SQLite as they are, so these are
+// off while it runs: users_profile_created would add a 'created' change for every imported account (a key
+// collision with the copied user_profile_changes rows, and one profile event per account at the first drain).
+const QUIET_TRIGGERS = [['users', 'users_profile_created']];
// The retired SQLite file: --sqlite, else DB_PATH, else data/network.db (relative to the repo root).
// The retired SQLite file: --sqlite, else DB_PATH, else data/network.db (relative to the repo root).
const DEFAULT_SQLITE = path.resolve(__dirname, '..', process.env.DB_PATH || path.join('data', 'network.db'));
const DEFAULT_SQLITE = path.resolve(__dirname, '..', process.env.DB_PATH || path.join('data', 'network.db'));
+/** Migrate `owner` (a createDb() handle, the owner role) and copy `sqlite` into it. → importSqlite's report + cleaned */
+async function importInto(owner, sqlite, { log = { log() {}, warn: console.warn, error: console.error } } = {}) {
+ await owner.migrate({ dir: MIGRATIONS, log });
+ const options = await cleaningOptions(owner, TABLES);
+ for (const [table, trigger] of QUIET_TRIGGERS) await owner.exec(`ALTER TABLE ${table} DISABLE TRIGGER ${trigger}`);
+ try {
+ const report = await importSqlite({ sqlite, db: owner, truncate: true, tables: options.tables, skipSource: SKIP_SOURCE, log });
+ return { ...report, cleaned: options.cleaned() };
+ } finally {
+ for (const [table, trigger] of QUIET_TRIGGERS) await owner.exec(`ALTER TABLE ${table} ENABLE TRIGGER ${trigger}`);
+ }
+}
+
+async function main(argv = process.argv.slice(2), out = console.log) {
+ const opt = (name) => { const i = argv.indexOf(`--${name}`); return i >= 0 ? argv[i + 1] : null; };
+ const flag = (name) => argv.includes(`--${name}`);
+ const file = path.resolve(opt('sqlite') || DEFAULT_SQLITE);
+ const quiet = { log() {}, warn: console.warn, error: console.error };
+ let owner;
+ if (flag('pglite')) owner = createDb({ pglite: true, service: 'network-import', log: quiet });
+ else {
+ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: the import runs as the owner on a direct connection');
+ owner = createDb({ url: config.db.directUrl, service: 'network-import', max: 2, log: quiet });
+ }
+ try {
+ const t0 = Date.now();
+ const report = await importInto(owner, file, { log: quiet });
+ if (flag('json')) out(JSON.stringify({ sqlite: file, into: owner.store, ...report }, null, 2));
+ else {
+ out(`import ${file} → ${owner.store} (${Date.now() - t0} ms): ${report.ok ? 'OK' : 'PROBLEMS'}`);
+ for (const t of report.tables) out(` ${t.table.padEnd(30)} ${String(t.rows).padStart(8)} rows ${t.checksum || '-'}${t.source !== t.table ? ` (from ${t.source})` : ''}`);
+ for (const c of report.cleaned) out(` cleaned: ${c.column}: ${c.values} value(s) with a NUL or an unpaired surrogate`);
+ for (const p of report.problems) out(` problem: ${p.table}: ${p.problem}`);
+ }
+ return report.ok ? 0 : 1;
+ } finally {
+ await owner.close();
+ }
+}
+
if (require.main === module) {
if (require.main === module) {
- runSqliteMigration({ service: 'network', sqlite: DEFAULT_SQLITE, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES, skipSource: SKIP_SOURCE })
- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
+ main().then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
}
}
-module.exports = { TABLES, SKIP_SOURCE, MIGRATIONS, DEFAULT_SQLITE };
+module.exports = { TABLES, SKIP_SOURCE, QUIET_TRIGGERS, MIGRATIONS, DEFAULT_SQLITE, importInto };
new file mode 100644
new file mode 100644
@@ -0,0 +1,99 @@
@@ -0,0 +1,99 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * The T2 cutover's row-count parity check (docs/cutover-t2-postgres.md, step 6): after
+ * scripts/migrate-to-postgres.js, every SQLite table must hold exactly as many rows in PostgreSQL. It is
+ * independent of the import's own verification (openvibe-sdk importSqlite counts and checksums what it
+ * copied): this one starts from the SQLite side, so a table the import never touched shows up as a mismatch.
+ *
+ * node scripts/pg-row-parity.js --sqlite <file> [--json] DATABASE_DIRECT_URL, else DATABASE_URL
+ * node scripts/pg-row-parity.js --sqlite <file> --pglite <dir> a PGlite directory (the rehearsal)
+ *
+ * Per table: SQLite `SELECT COUNT(*) FROM "<t>"` against PostgreSQL `SELECT COUNT(*) FROM "<t>"` (both
+ * sides read-only). Compared: every SQLite table except SQLite's own, FTS5 virtual tables and their shadow
+ * tables, and migrate-to-postgres.js's SKIP_SOURCE. PostgreSQL-only tables (new in T2) are listed, not
+ * compared. The triggers the import turns off (QUIET_TRIGGERS) must be on again. Run it before the new
+ * release boots: its boot seeds rows (OAuth clients, themes, grants) that the SQLite file never had. Prints counts only, never a row. Exit 0 = parity, 1 = a mismatch, 2 = unusable input.
+ */
+const fs = require('fs');
+const path = require('path');
+const { parseArgs } = require('./lib/db-ops');
+const { TABLES, SKIP_SOURCE, QUIET_TRIGGERS } = require('./migrate-to-postgres');
+
+const NAME_RE = /^[A-Za-z_][A-Za-z0-9_]*$/;
+const quote = (t) => { if (!NAME_RE.test(t)) throw new Error(`unexpected table name ${JSON.stringify(t)}`); return `"${t}"`; };
+
+/** The SQLite tables the import must carry over (importSqlite's own rule), each with its PostgreSQL name. */
+function sourceTables(src) {
+ const all = src.prepare("SELECT name, sql FROM sqlite_master WHERE type = 'table' ORDER BY name").all();
+ const virtuals = all.filter((t) => /^CREATE VIRTUAL TABLE/i.test(t.sql || '')).map((t) => t.name);
+ const renamed = Object.fromEntries(Object.entries(TABLES).filter(([, o]) => o && o.from).map(([to, o]) => [o.from, to]));
+ return all.map((t) => t.name)
+ .filter((n) => !n.startsWith('sqlite_') && !SKIP_SOURCE.includes(n) && !virtuals.some((v) => n === v || n.startsWith(`${v}_`)))
+ .map((n) => ({ source: n, target: renamed[n] || n }));
+}
+
+/**
+ * Compare row counts. `src` is an open better-sqlite3 Database, `db` a createDb() handle on the target.
+ * → { ok, tables: { [target]: { sqlite, pg } }, pg_only: { [table]: rows }, problems: [] }
+ */
+async function rowParity(src, db) {
+ const report = { ok: true, tables: {}, pg_only: {}, problems: [] };
+ const pgTables = new Set((await db.many(`SELECT tablename FROM pg_tables
+ WHERE schemaname = current_schema() AND tablename <> 'ov_migrations'`)).map((r) => r.tablename));
+ const compared = new Set();
+ for (const { source, target } of sourceTables(src)) {
+ const sqlite = src.prepare(`SELECT COUNT(*) AS n FROM ${quote(source)}`).get().n;
+ const pg = pgTables.has(target) ? Number(await db.value(`SELECT COUNT(*) FROM ${quote(target)}`)) : null;
+ report.tables[target] = { sqlite, pg };
+ compared.add(target);
+ if (pg === null) { report.ok = false; report.problems.push({ table: target, problem: 'no PostgreSQL table' }); }
+ else if (pg !== sqlite) { report.ok = false; report.problems.push({ table: target, problem: `row count differs: SQLite ${sqlite}, PostgreSQL ${pg}` }); }
+ }
+ for (const t of [...pgTables].sort()) if (!compared.has(t)) report.pg_only[t] = Number(await db.value(`SELECT COUNT(*) FROM ${quote(t)}`));
+ // The import turns these off while it copies; an import that died half-way leaves them off.
+ for (const [table, trigger] of QUIET_TRIGGERS) {
+ if (!pgTables.has(table)) continue;
+ const t = quote(trigger);
+ // quote() validates the name and identifier-quotes it; tgname is a text column, so the
+ // comparison takes that quoted value as a string literal (the name itself holds no quotes).
+ const state = await db.value(
+ `SELECT tgenabled::text FROM pg_trigger WHERE tgname = ${t.replace(/"/g, "'")} AND tgrelid = '${quote(table)}'::regclass`,
+ );
+ if (state !== 'O') { report.ok = false; report.problems.push({ table, problem: `trigger ${trigger} is ${state ? 'disabled' : 'missing'}: run migrate-to-postgres.js again` }); }
+ }
+ return report;
+}
+
+async function main(argv = process.argv.slice(2)) {
+ let args;
+ try { args = parseArgs(argv, { flags: ['json'], values: ['sqlite', 'pglite'] }); } catch (e) { console.error(e.message); return 2; }
+ if (args.help || !args.sqlite) { console.error('usage: node scripts/pg-row-parity.js --sqlite <file> [--pglite <dir>] [--json]'); return 2; }
+ const file = path.resolve(args.sqlite);
+ if (!fs.existsSync(file)) { console.error(`no such SQLite database: ${file}`); return 2; }
+ const url = process.env.DATABASE_DIRECT_URL || process.env.DATABASE_URL;
+ if (!args.pglite && !url) { console.error('DATABASE_DIRECT_URL (or DATABASE_URL) is not set, and no --pglite <dir>'); return 2; }
+ const { createDb } = require('openvibe-sdk/db');
+ const src = new (require('better-sqlite3'))(file, { readonly: true, fileMustExist: true });
+ const quiet = { log() {}, warn: console.warn, error: console.error };
+ const db = args.pglite ? createDb({ pglite: path.resolve(args.pglite), service: 'network-parity', log: quiet })
+ : createDb({ url, service: 'network-parity', max: 1, log: quiet });
+ try {
+ const report = await rowParity(src, db);
+ if (args.json) console.log(JSON.stringify({ sqlite: file, ...report }, null, 2));
+ else {
+ console.log(`row parity ${file} → ${db.store}: ${report.ok ? 'OK' : 'MISMATCH'}`);
+ for (const [t, c] of Object.entries(report.tables)) console.log(` ${t.padEnd(30)} ${String(c.sqlite).padStart(8)} ${String(c.pg ?? '-').padStart(8)}${c.sqlite === c.pg ? '' : ' ✗'}`);
+ for (const [t, n] of Object.entries(report.pg_only)) console.log(` ${t.padEnd(30)} ${'-'.padStart(8)} ${String(n).padStart(8)} (PostgreSQL only)`);
+ for (const p of report.problems) console.log(` problem: ${p.table}: ${p.problem}`);
+ }
+ return report.ok ? 0 : 1;
+ } finally {
+ src.close();
+ await db.close();
+ }
+}
+
+if (require.main === module) main().then((code) => process.exit(code), (err) => { console.error(`pg-row-parity failed: ${err.message}`); process.exit(2); });
+
+module.exports = { rowParity, sourceTables };
new file mode 100644
new file mode 100644
@@ -0,0 +1,474 @@
@@ -0,0 +1,474 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * A repeatable rehearsal of the T2 PostgreSQL cutover (docs/cutover-t2-postgres.md), end to end on scratch
+ * stores: the SQLite database → pg-preflight → migrate-to-postgres → row-count parity → the server booted on
+ * the result, /api/ready green. Nothing outside a temporary directory (and, on the containers, a database of
+ * its own, dropped at the end) is written; the SQLite file is only ever opened read-only.
+ *
+ * npm run rehearse-pg-cutover a seeded database with commit 20dd7ff's schema
+ * npm run rehearse-pg-cutover -- --sqlite /path/to/copy.db a real snapshot (a copy, never the live file)
+ * node scripts/rehearse-pg-cutover.js --write-fixture <file> only write the seeded SQLite database, then exit
+ *
+ * --store auto|pglite|containers|env where the PostgreSQL side lives (default auto: containers, else pglite;
+ * auto never uses inherited DATABASE_URLs)
+ * env only when given explicitly. DATABASE_DIRECT_URL + DATABASE_URL: a scratch database you made for
+ * this. Both must reach that same database (a probe table made through one is read through the
+ * other) and it must be empty through both. The import empties every Network table there, so
+ * a target where any table already holds a row is refused unless --truncate-target is given.
+ * Never point it at the production database.
+ * containers openvibe-sdk's test services (eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)"):
+ * OV_TEST_PG_DIRECT_URL. A throwaway database network_rehearsal_* with roles shaped like
+ * production's: the owner migrates and imports, a DML-only role (statement_timeout 15 s) serves.
+ * A database, not a schema in the shared one: an import reads information_schema, which the
+ * shared test database's thousands of test schemas slow past the 15 s query cap. The container
+ * PgBouncer routes only that shared database, so here the server connects directly (npm run
+ * test:pg covers serving through PgBouncer). OV_TEST_VALKEY_URL, when set, backs the limit
+ * counters under a prefix of its own.
+ * pglite an embedded PGlite directory: nothing to set up.
+ * --keep keep the temporary directory (its path is in the JSON line as "dir")
+ *
+ * On PostgreSQL the import is the real `node scripts/migrate-to-postgres.js --sqlite <file>` with the target's
+ * DATABASE_DIRECT_URL. PGlite has no URL, so there the script runs as its own --pglite dry run and the same
+ * import (its importInto: migrations, TABLES, SKIP_SOURCE, QUIET_TRIGGERS) fills the PGlite directory the
+ * server then boots on. On PostgreSQL the server boots with NODE_ENV=production (DATABASE_URL required, the
+ * RS256 signing key required); on PGlite with NODE_ENV=test (production refuses PGlite). Either way /api/ready
+ * must answer 200 with `db` and `signing_key` ok.
+ *
+ * Progress goes to stderr. The last line on stdout is one JSON object, counts only, never a row:
+ * {"ok":true,"tables":{"users":{"sqlite":6,"pg":6},…},"ready":{"ready":true,"status":…,"db":"ok","signing_key":"ok"},…}
+ * Exit 0 only when ok: preflight clean, import verified, every table's count equal, and ready.
+ */
+const crypto = require('crypto');
+const fs = require('fs');
+const net = require('net');
+const os = require('os');
+const path = require('path');
+const { spawn } = require('child_process');
+const { parseArgs, ROOT } = require('./lib/db-ops');
+
+const FIXTURE_SCHEMA = path.join(__dirname, 'fixtures', 'network-sqlite-20dd7ff.sql');
+const MIGRATIONS = path.join(ROOT, 'migrations');
+const log = (...a) => console.error('[rehearse]', ...a);
+
+// ── The seeded database ──────────────────────────────────────
+// Commit 20dd7ff's schema (scripts/fixtures/network-sqlite-20dd7ff.sql) with rows shaped like production's:
+// accounts (mixed-case names, an anonymous and a banned one), OAuth clients/codes/token families, notifications
+// (JSON rich content, one message holding a NUL that PostgreSQL refuses), the wallet ledger, legacy and
+// subject follows, blocks, sessions, an unsent outbox event, the profile changes 20dd7ff's users_profile_created
+// trigger writes, an old per-IP rate counter. SQLite CURRENT_TIMESTAMP text throughout.
+const SUBJECT = (n) => `usr_rehearsal${String(n).padStart(4, '0')}`;
+function seedFixture(file) {
+ const Database = require('better-sqlite3');
+ const db = new Database(file);
+ db.pragma('foreign_keys = ON');
+ db.exec(fs.readFileSync(FIXTURE_SCHEMA, 'utf8'));
+ const ins = (table, rows) => {
+ const cols = Object.keys(rows[0]);
+ const st = db.prepare(`INSERT INTO ${table} (${cols.join(', ')}) VALUES (${cols.map((c) => `@${c}`).join(', ')})`);
+ for (const r of rows) st.run(r);
+ };
+ const at = (d, h = 12) => `2026-0${d}-1${d} ${String(h).padStart(2, '0')}:0${d}:3${d}`;
+ const hash = (n) => `$2a$10$rehearsalfixturehashnotarealpasswordhash${String(n).padStart(13, '0')}`;
+ // Opaque token strings, generated per run: the fixture holds row shapes, not credentials.
+ const rtToken = (family, gen) => `rt-family-${family}-gen${gen}-${crypto.randomBytes(8).toString('hex')}`;
+ db.transaction(() => {
+ ins('users', [
+ { id: 1, username: 'alex', email: '[email protected]', password_hash: hash(1), display_name: 'Alex', role: 'admin', email_verified: 1, email_verified_at: at(1), subject_id: SUBJECT(1), created_at: at(1), updated_at: at(2), last_seen: at(3), is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },
+ { id: 2, username: 'Bea_Streams', email: '[email protected]', password_hash: hash(2), display_name: 'Bea ✨ Streams', role: 'streamer', email_verified: 1, email_verified_at: at(1), subject_id: SUBJECT(2), created_at: at(1), updated_at: at(1), last_seen: at(4), is_anon: 0, history_paused: 1, is_banned: 0, ban_reason: null, anon_number: null },
+ { id: 3, username: 'carol', email: null, password_hash: hash(3), display_name: null, role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(3), created_at: at(2), updated_at: at(2), last_seen: null, is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },
+ { id: 4, username: 'DanTheMod', email: '[email protected]', password_hash: hash(4), display_name: 'Dan', role: 'global_mod', email_verified: 1, email_verified_at: at(2), subject_id: SUBJECT(4), created_at: at(2), updated_at: at(3), last_seen: at(5), is_anon: 0, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: null },
+ { id: 5, username: 'spam_account', email: '[email protected]', password_hash: hash(5), display_name: 'Spam', role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(5), created_at: at(3), updated_at: at(3), last_seen: at(3), is_anon: 0, history_paused: 0, is_banned: 1, ban_reason: 'spam', anon_number: null },
+ { id: 6, username: 'anon_7', email: null, password_hash: hash(6), display_name: 'Anon #7', role: 'user', email_verified: 0, email_verified_at: null, subject_id: SUBJECT(6), created_at: at(4), updated_at: at(4), last_seen: at(4), is_anon: 1, history_paused: 0, is_banned: 0, ban_reason: null, anon_number: 7 },
+ ]);
+ ins('oauth_clients', [
+ { client_id: 'live', client_secret: 'rehearsal-client-secret-live', name: 'OpenVibe.Live', redirect_uris: '["https://openvibe.live/auth/callback"]', is_first_party: 1, created_at: at(1) },
+ { client_id: 'chat', client_secret: 'rehearsal-client-secret-chat', name: 'OpenVibe.Chat', redirect_uris: '["https://openvibe.chat/auth/callback"]', is_first_party: 1, created_at: at(1) },
+ ]);
+ ins('oauth_codes', [
+ { code: 'code-used-1', client_id: 'live', user_id: 2, redirect_uri: 'https://openvibe.live/auth/callback', scope: 'openid profile', expires_at: at(4, 13), used: 1, code_challenge: 'E9Melhoa2OwvFrEMTJguCHaoeK1t8URWbuGJSstw-cM', code_challenge_method: 'S256', nonce: 'n-1', created_at: at(4) },
+ { code: 'code-open-2', client_id: 'chat', user_id: 3, redirect_uri: 'https://openvibe.chat/auth/callback', scope: 'openid', expires_at: at(5, 13), used: 0, code_challenge: null, code_challenge_method: null, nonce: null, created_at: at(5) },
+ ]);
+ ins('oauth_tokens', [
+ { id: 1, token: rtToken('a', 1), client_id: 'live', user_id: 2, scope: 'openid profile', expires_at: at(9), revoked: 1, family_id: 'fam-a', generation: 1, revoked_reason: 'rotated', revoked_at: at(5), created_at: at(4) },
+ { id: 2, token: rtToken('a', 2), client_id: 'live', user_id: 2, scope: 'openid profile', expires_at: at(9), revoked: 0, family_id: 'fam-a', generation: 2, revoked_reason: null, revoked_at: null, created_at: at(5) },
+ { id: 3, token: rtToken('b', 1), client_id: 'chat', user_id: 1, scope: 'openid', expires_at: at(9), revoked: 0, family_id: 'fam-b', generation: 1, revoked_reason: null, revoked_at: null, created_at: at(5) },
+ ]);
+ ins('user_preferences', [
+ { user_id: 1, theme_id: 'midnight', custom_theme_variables: '{"--accent":"#8b5cf6"}', language: 'en', notifications_enabled: 1, display_prefs: '{"compact":true,"motion":"reduce"}', updated_at: at(2) },
+ { user_id: 2, theme_id: null, custom_theme_variables: null, language: 'fr', notifications_enabled: 0, display_prefs: null, updated_at: at(3) },
+ ]);
+ ins('notifications', [
+ { id: 'ntf-0001', user_id: 1, type: 'follow', category: 'social', priority: 'normal', title: 'Bea followed you', message: 'Say hi!', sender_id: 2, sender_name: 'Bea', service: 'network', url: '/u/Bea_Streams', rich_content: '{"actor":{"subject":"usr_rehearsal0002"}}', is_read: 1, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(5) },
+ { id: 'ntf-0002', user_id: 1, type: 'live', category: 'streams', priority: 'high', title: 'Bea is live', message: 'Speedrun night', sender_id: 2, sender_name: 'Bea', service: 'live', url: 'https://openvibe.live/Bea_Streams', rich_content: null, is_read: 0, is_dismissed: 0, is_emailed: 1, expires_at: at(9), created_at: at(6) },
+ { id: 'ntf-0003', user_id: 2, type: 'system', category: 'system', priority: 'normal', title: 'Welcome', message: 'legacy text with a NUL here:\u0000end', sender_id: null, sender_name: null, service: 'network', url: null, rich_content: '{"steps":["profile","theme"]}', is_read: 0, is_dismissed: 1, is_emailed: 0, expires_at: null, created_at: at(4) },
+ { id: 'ntf-0004', user_id: 3, type: 'coins', category: 'wallet', priority: 'low', title: 'You received 50 OpenCoins', message: null, sender_id: null, sender_name: null, service: 'network', url: '/wallet', rich_content: null, is_read: 0, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(6) },
+ { id: 'ntf-0005', user_id: 4, type: 'moderation', category: 'moderation', priority: 'high', title: 'Report resolved', message: 'Thanks', sender_id: null, sender_name: null, service: 'community', url: null, rich_content: null, is_read: 1, is_dismissed: 0, is_emailed: 0, expires_at: null, created_at: at(7) },
+ ]);
+ ins('notification_preferences', [
+ { user_id: 1, category: 'social', enabled: 1, sound: 0, toasts: 1, email: 0 },
+ { user_id: 1, category: 'streams', enabled: 1, sound: 1, toasts: 1, email: 1 },
+ { user_id: 2, category: 'social', enabled: 0, sound: 0, toasts: 0, email: 0 },
+ ]);
+ ins('user_sessions', [
+ { id: 1, user_id: 1, session_token: 'sess-rehearsal-1', device_name: 'Firefox on Linux', ip: null, user_agent: null, is_active: 1, last_used: at(6), created_at: at(5), expires_at: at(9) },
+ { id: 2, user_id: 2, session_token: 'sess-rehearsal-2', device_name: 'Phone', ip: null, user_agent: null, is_active: 0, last_used: at(5), created_at: at(4), expires_at: at(8) },
+ ]);
+ ins('linked_accounts', [{ id: 1, user_id: 2, service: 'twitch', service_user_id: '424242', service_username: 'bea_streams', linked_at: at(3), last_used_at: at(5) }]);
+ ins('verification_keys', [
+ { id: 1, key: 'vk-rehearsal-1', target_username: 'ReservedName', note: 'creator reservation', created_by: 1, used_by: null, status: 'active', created_at: at(3), used_at: null },
+ { id: 2, key: 'vk-rehearsal-2', target_username: 'Bea_Streams', note: null, created_by: 1, used_by: 2, status: 'used', created_at: at(1), used_at: at(1) },
+ ]);
+ ins('anon_users', [{ id: 1, anon_number: 7, fingerprint: 'fp-rehearsal', session_token: 'anon-sess-7', display_name: 'Anon #7', preferences: '{"color":"#ff0"}', total_messages: 12, total_commands: 1, first_seen: at(4), last_seen: at(5), ip: null, subject_id: 'anon_rehearsal0007' }]);
+ // The wallet ledger: balances are the sum of each account's transactions.
+ ins('coin_transactions', [
+ { id: 1, user_id: 1, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:1', created_at: at(1) },
+ { id: 2, user_id: 2, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:2', created_at: at(1) },
+ { id: 3, user_id: 3, app_id: 'network', delta: 100, reason: 'signup_bonus', ref: null, idempotency_key: 'signup:3', created_at: at(2) },
+ { id: 4, user_id: 2, app_id: 'live', delta: -50, reason: 'tip', ref: 'tip:ntf-0004', idempotency_key: 'tip:2:3:1', created_at: at(6) },
+ { id: 5, user_id: 3, app_id: 'live', delta: 50, reason: 'tip_received', ref: 'tip:ntf-0004', idempotency_key: 'tip:2:3:1:recv', created_at: at(6) },
+ { id: 6, user_id: 1, app_id: 'games', delta: -25, reason: 'purchase', ref: 'item:hat', idempotency_key: null, created_at: at(7) },
+ ]);
+ ins('wallets', [
+ { user_id: 1, balance: 75, updated_at: at(7) },
+ { user_id: 2, balance: 50, updated_at: at(6) },
+ { user_id: 3, balance: 150, updated_at: at(6) },
+ ]);
+ ins('follows', [
+ { follower_id: 1, followed_id: 2, created_at: at(4) },
+ { follower_id: 3, followed_id: 2, created_at: at(5) },
+ { follower_id: 2, followed_id: 1, created_at: at(5) },
+ ]);
+ ins('user_follows', [
+ { follower_subject: SUBJECT(1), target_type: 'channel', target_id: SUBJECT(2), active: 1, notify_email: 1, notify_push: 0, revision: 1, source: 'live', created_at: '2026-04-14T12:00:00.000Z', updated_at: '2026-04-14T12:00:00.000Z' },
+ { follower_subject: SUBJECT(3), target_type: 'channel', target_id: SUBJECT(2), active: 1, notify_email: 0, notify_push: 1, revision: 2, source: 'network', created_at: '2026-05-15T12:00:00.000Z', updated_at: '2026-05-16T08:00:00.000Z' },
+ { follower_subject: SUBJECT(4), target_type: 'channel', target_id: SUBJECT(2), active: 0, notify_email: 1, notify_push: 1, revision: 3, source: 'network', created_at: '2026-05-15T12:00:00.000Z', updated_at: '2026-06-01T08:00:00.000Z' },
+ ]);
+ ins('user_blocks', [
+ { blocker_subject: SUBJECT(2), blocked_subject: SUBJECT(5), active: 1, revision: 1, created_at: '2026-05-01T10:00:00.000Z', updated_at: '2026-05-01T10:00:00.000Z' },
+ { blocker_subject: SUBJECT(1), blocked_subject: SUBJECT(5), active: 0, revision: 2, created_at: '2026-05-01T10:00:00.000Z', updated_at: '2026-05-02T10:00:00.000Z' },
+ ]);
+ ins('audit_log', [{ id: 1, user_id: 1, action: 'user.ban', details: '{"target":5,"reason":"spam"}', ip: null, created_at: at(3) }]);
+ ins('site_settings', [{ key: 'registration_open', value: 'true', type: 'boolean' }]);
+ // Left behind by the import (SKIP_SOURCE): per-IP counters, never carried over.
+ ins('analytics_rate_tracking', [{ ip: '198.51.100.7', window_start: 1767268800, hit_count: 3 }]);
+ ins('network_event_outbox', [{ id: 1, event_id: 'evt-rehearsal-1', envelope: '{"type":"network.user.updated","subject":"usr_rehearsal0002"}', traceparent: null, created_at: 1767268800000, attempts: 2, next_attempt_at: 1767268860000, sent_at: null, seq: 1, rejected_at: null, last_error: 'events unreachable' }]);
+ })();
+ db.close();
+}
+
+// ── Helpers ──────────────────────────────────────────────────
+function freePort() {
+ return new Promise((resolve, reject) => {
+ const s = net.createServer();
+ s.once('error', reject);
+ s.listen(0, '127.0.0.1', () => { const { port } = s.address(); s.close(() => resolve(port)); });
+ });
+}
+
+/** Run a node script from scripts/ in `cwd` (a directory with no .env) with exactly `env`. → { code, stdout, stderr } */
+function runNode(args, { cwd, env }) {
+ return new Promise((resolve) => {
+ const p = spawn(process.execPath, args, { cwd, env, stdio: ['ignore', 'pipe', 'pipe'] });
+ let stdout = '', stderr = '';
+ p.stdout.on('data', (d) => { stdout += d; });
+ p.stderr.on('data', (d) => { stderr += d; });
+ p.on('close', (code) => resolve({ code, stdout, stderr }));
+ });
+}
+
+/** The first JSON object a script printed (they pretty-print it), or null. */
+function jsonOf(text) {
+ const i = text.indexOf('{');
+ if (i < 0) return null;
+ try { return JSON.parse(text.slice(i)); } catch { return null; }
+}
+
+/** A URL with its password masked, for the log. */
+const masked = (url) => { try { const u = new URL(url); if (u.password) u.password = '***'; return u.toString(); } catch { return '(unparseable url)'; } };
+
+/**
+ * --store env's guard, before anything is imported. `direct` (DATABASE_DIRECT_URL, the import's connection) and
+ * `pooled` (DATABASE_URL, the server's) must reach the same database; DATABASE_URL may be a pooler, so host and port
+ * prove nothing. The target must be empty through `direct` first (read-only, so a live database is refused before
+ * anything is written). Then a probe table with a random name, made through `direct` with a random token as its
+ * comment, must show that token through `pooled` (catalogs only, so a DML-only role can read it); the probe is
+ * dropped either way. Then the target must be empty through `pooled` too. Throws with a message naming no URL.
+ */
+async function checkScratchTarget(direct, pooled, { truncateTarget = false } = {}) {
+ const assertEmpty = async (name, db) => {
+ if (truncateTarget) return;
+ // A scratch database holds no rows anywhere (ov_migrations aside): any row means it is not one.
+ const tables = (await db.many(`SELECT tablename FROM pg_tables
+ WHERE schemaname = current_schema() AND tablename <> 'ov_migrations'`)).map((r) => r.tablename);
+ const held = [];
+ for (const t of tables) if (await db.value(`SELECT EXISTS (SELECT 1 FROM "${t.replace(/"/g, '""')}")`)) held.push(t);
+ if (held.length) throw new Error(`the target is not empty through ${name} (rows in ${held.length} table(s): ${held.slice(0, 5).join(', ')}${held.length > 5 ? ', …' : ''}); the import empties it. Give --truncate-target if it really is a scratch database`);
+ };
+ await assertEmpty('DATABASE_DIRECT_URL', direct);
+ const probe = `ov_rehearsal_probe_${crypto.randomBytes(6).toString('hex')}`;
+ const token = crypto.randomBytes(16).toString('hex');
+ let seen = null;
+ try {
+ await direct.exec(`CREATE TABLE ${probe} (id int)`);
+ await direct.exec(`COMMENT ON TABLE ${probe} IS '${token}'`);
+ seen = await pooled.value(`SELECT obj_description(c.oid, 'pg_class') FROM pg_class c WHERE c.relname = '${probe}'`)
+ .catch((e) => { throw new Error(`reading the probe through DATABASE_URL failed: ${e.message}`); });
+ } finally {
+ await direct.exec(`DROP TABLE IF EXISTS ${probe}`).catch((e) => log(`probe cleanup: ${e.message}`));
+ }
+ if (seen !== token) throw new Error('DATABASE_URL does not reach the database DATABASE_DIRECT_URL imports into (a probe table made through DATABASE_DIRECT_URL is not visible through DATABASE_URL); --store env needs both on one scratch database');
+ await assertEmpty('DATABASE_URL', pooled);
+}
+
+/** The PostgreSQL side: { store, directUrl?, url?, pgliteDir?, valkeyUrl?, close() }. */
+async function openTarget(store, dir, truncateTarget) {
+ if (store === 'env') {
+ const directUrl = (process.env.DATABASE_DIRECT_URL || '').trim(), url = (process.env.DATABASE_URL || '').trim();
+ if (!directUrl || !url) throw new Error(`--store env needs DATABASE_DIRECT_URL and DATABASE_URL, both on one scratch database (${directUrl ? 'DATABASE_URL' : 'DATABASE_DIRECT_URL'} is unset or empty)`);
+ const { createDb } = require('openvibe-sdk/db');
+ const silent = { log() {}, warn() {}, error() {} };
+ const direct = createDb({ url: directUrl, service: 'network-rehearsal', max: 1, log: silent });
+ const pooled = createDb({ url, service: 'network-rehearsal-pooled', max: 1, log: silent });
+ try { await checkScratchTarget(direct, pooled, { truncateTarget }); }
+ finally { await Promise.allSettled([direct.close(), pooled.close()]); }
+ log(`target: DATABASE_DIRECT_URL ${masked(directUrl)}, DATABASE_URL ${masked(url)} (the same database)`);
+ return { store: 'postgresql', directUrl, url, valkeyUrl: process.env.VALKEY_URL || null, close: async () => {} };
+ }
+ if (store === 'containers') {
+ if (!process.env.OV_TEST_PG_DIRECT_URL) throw new Error('--store containers needs OV_TEST_PG_DIRECT_URL (eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)")');
+ return scratchDatabase(process.env.OV_TEST_PG_DIRECT_URL);
+ }
+ const pgliteDir = path.join(dir, 'pglite');
+ log(`target: PGlite ${pgliteDir}`);
+ return { store: 'pglite', pgliteDir, valkeyUrl: null, close: async () => {} };
+}
+
+/**
+ * A database of its own on the containers' PostgreSQL (`superUrl`, a superuser on a direct connection), shaped as
+ * OpenVibe.Host's roles/data/add-service.sh makes production's: an owner role owning the database and its public
+ * schema (migrations, the import) and a runtime role with DML only. close() drops the database and both roles.
+ */
+async function scratchDatabase(superUrl) {
+ const { createDb } = require('openvibe-sdk/db');
+ const silent = { log() {}, warn() {}, error() {} };
+ const name = `network_rehearsal_${process.pid}_${crypto.randomBytes(4).toString('hex')}`;
+ const owner = `${name}_owner`;
+ const pw = crypto.randomBytes(16).toString('hex');
+ const as = (user, db = name) => { const u = new URL(superUrl); u.username = user; u.password = pw; u.pathname = `/${db}`; return u.toString(); };
+ // DROP DATABASE … WITH (FORCE) on the shared, busy container can outlast the 15 s default cap.
+ const su = createDb({ url: superUrl, service: 'network-rehearsal-admin', max: 1, queryTimeoutMs: 120000, log: silent });
+ const drop = async () => {
+ await su.exec(`DROP DATABASE IF EXISTS ${name} WITH (FORCE)`);
+ for (const r of [name, owner]) await su.exec(`DROP ROLE IF EXISTS ${r}`);
+ };
+ try {
+ for (const stmt of [
+ `CREATE ROLE ${owner} LOGIN PASSWORD '${pw}'`,
+ `CREATE ROLE ${name} LOGIN PASSWORD '${pw}'`,
+ `ALTER ROLE ${name} SET statement_timeout = '15s'`,
+ `ALTER ROLE ${name} SET lock_timeout = '5s'`,
+ `CREATE DATABASE ${name} OWNER ${owner}`,
+ `REVOKE ALL ON DATABASE ${name} FROM PUBLIC`,
+ `GRANT CONNECT ON DATABASE ${name} TO ${owner}, ${name}`,
+ ]) await su.exec(stmt);
+ const o = createDb({ url: as(owner), service: 'network-rehearsal-owner', max: 1, log: silent });
+ try {
+ for (const stmt of [
+ `GRANT USAGE ON SCHEMA public TO ${name}`,
+ `ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ${name}`,
+ `ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO ${name}`,
+ ]) await o.exec(stmt);
+ } finally { await o.close(); }
+ } catch (e) { await drop().catch(() => {}); await su.close(); throw e; }
+ log(`target: test-services container, database ${name}`);
+ return {
+ store: 'postgresql', directUrl: as(owner), url: as(name), valkeyUrl: process.env.OV_TEST_VALKEY_URL || null,
+ close: async () => { try { await drop(); } finally { await su.close(); } },
+ };
+}
+
+/** PGlite: migrate-to-postgres.js's import, into the directory the server then boots on. */
+async function importIntoPglite(sqlite, pgliteDir) {
+ const { createDb } = require('openvibe-sdk/db');
+ const { importInto } = require('./migrate-to-postgres');
+ const owner = createDb({ pglite: pgliteDir, service: 'network-import', log: { log() {}, warn: console.error, error: console.error } });
+ try { return await importInto(owner, sqlite); } finally { await owner.close(); }
+}
+
+/**
+ * GET `${base}/api/ready` until it answers 200 with ready, stop() is true, or timeoutMs passes. Each request, body
+ * included, is aborted after min(the time left, requestMs), so a server that accepts and never answers cannot hold
+ * the rehearsal past its deadline. → { status, body } of the last answer read (null when none was).
+ */
+async function pollReady(base, { timeoutMs, stop = () => false, requestMs = 5000 }) {
+ const t0 = Date.now();
+ let status = null, body = null;
+ while (!stop()) {
+ const left = timeoutMs - (Date.now() - t0);
+ if (left <= 0) break;
+ try {
+ const r = await fetch(`${base}/api/ready`, { signal: AbortSignal.timeout(Math.max(1, Math.min(left, requestMs))) });
+ status = r.status;
+ body = await r.json();
+ if (r.status === 200 && body.ready) break;
+ } catch { /* not listening yet, aborted at the deadline, or not JSON */ }
+ await new Promise((r) => setTimeout(r, 250));
+ }
+ return { status, body };
+}
+
+/** Boot server/index.js on the target and read /api/ready until it is ready (or the deadline). */
+async function bootAndCheck(target, dir, { timeoutMs = 120000 } = {}) {
+ const port = await freePort();
+ const keys = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });
+ fs.mkdirSync(path.join(dir, 'keys'), { recursive: true });
+ fs.writeFileSync(path.join(dir, 'keys', 'private.pem'), keys.privateKey, { mode: 0o600 });
+ fs.writeFileSync(path.join(dir, 'keys', 'public.pem'), keys.publicKey);
+ const env = {
+ PATH: process.env.PATH, HOME: dir, PORT: String(port), HOST: '127.0.0.1',
+ NODE_ENV: target.store === 'pglite' ? 'test' : 'production',
+ JWT_PRIVATE_KEY: path.join(dir, 'keys', 'private.pem'), JWT_PUBLIC_KEY: path.join(dir, 'keys', 'public.pem'),
+ AVATAR_PATH: path.join(dir, 'avatars'), INTERNAL_API_KEY: crypto.randomBytes(24).toString('hex'),
+ OV_NETWORK_INTERNAL_URL: `http://127.0.0.1:${port}`,
+ OV_LIVE_INTERNAL_URL: 'http://127.0.0.1:9', OV_TOOLS_INTERNAL_URL: 'http://127.0.0.1:9', OV_GAMES_INTERNAL_URL: 'http://127.0.0.1:9',
+ OV_MEDIA_INTERNAL_URL: 'http://127.0.0.1:9', OV_AI_INTERNAL_URL: 'http://127.0.0.1:9',
+ ...(target.store === 'pglite' ? { PGLITE_DIR: target.pgliteDir } : { DATABASE_URL: target.url, DATABASE_DIRECT_URL: target.directUrl }),
+ ...(target.valkeyUrl ? { VALKEY_URL: target.valkeyUrl, VALKEY_PREFIX: `ov:network-rehearsal-${crypto.randomBytes(4).toString('hex')}:` } : {}),
+ };
+ const proc = spawn(process.execPath, [path.join(ROOT, 'server', 'index.js')], { cwd: dir, env, stdio: ['ignore', 'pipe', 'pipe'] });
+ let out = '';
+ proc.stdout.on('data', (d) => { out += d; });
+ proc.stderr.on('data', (d) => { out += d; });
+ let exited = null;
+ proc.on('exit', (code, sig) => { exited = { code, sig }; });
+ const base = `http://127.0.0.1:${port}`;
+ const t0 = Date.now();
+ let body = null, status = null;
+ try {
+ ({ status, body } = await pollReady(base, { timeoutMs, stop: () => !!exited }));
+ } finally {
+ if (!exited) {
+ await new Promise((resolve) => {
+ const timer = setTimeout(resolve, 5000);
+ proc.once('exit', () => { clearTimeout(timer); resolve(); });
+ proc.kill('SIGTERM');
+ });
+ if (!exited) {
+ try { proc.kill('SIGKILL'); } catch { /* gone */ }
+ await new Promise((resolve) => { if (exited) resolve(); else proc.once('exit', resolve); });
+ }
+ }
+ }
+ const checks = (body && body.checks) || {};
+ const ready = {
+ http: status, ready: !!(body && body.ready), status: body ? body.status : null,
+ db: checks.db ? checks.db.status : null, signing_key: checks.signing_key ? checks.signing_key.status : null,
+ failed: body ? body.failed : null, degraded: body ? body.degraded : null, node_env: env.NODE_ENV, valkey: !!target.valkeyUrl, ms: Date.now() - t0,
+ };
+ ready.ok = status === 200 && ready.ready && ready.db === 'ok' && ready.signing_key === 'ok';
+ if (!ready.ok) {
+ ready.exited = exited;
+ // The boot log's tail, for the operator; the server never logs secrets or rows.
+ log(`server did not become ready (${JSON.stringify(exited)}); last log lines:\n${out.split('\n').slice(-25).join('\n')}`);
+ }
+ return ready;
+}
+
+// ── The rehearsal ────────────────────────────────────────────
+async function rehearse(opts) {
+ const t0 = Date.now();
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-network-rehearsal-'));
+ const result = { ok: false, source: opts.sqlite ? 'snapshot' : 'seeded-20dd7ff', store: null, preflight: null, import: null, tables: {}, pg_only: {}, ready: null, problems: [], dir: opts.keep ? dir : undefined };
+ // Children get exactly this environment, in a directory with no .env, so nothing else is picked up.
+ const childEnv = { PATH: process.env.PATH, HOME: dir };
+ let target = null;
+ try {
+ const sqlite = opts.sqlite ? path.resolve(opts.sqlite) : path.join(dir, 'network.db');
+ if (opts.sqlite) { if (!fs.existsSync(sqlite)) throw new Error(`no such SQLite database: ${sqlite}`); }
+ else seedFixture(sqlite);
+ log(`source: ${sqlite}`);
+
+ // 1. pg-preflight: case-fold collisions block the cutover.
+ const pre = await runNode([path.join(__dirname, 'pg-preflight-collisions.js'), '--sqlite', sqlite, '--json'], { cwd: dir, env: childEnv });
+ const preReport = jsonOf(pre.stdout);
+ result.preflight = { ok: pre.code === 0, exit: pre.code, collisions: preReport ? preReport.collisions.map((c) => ({ table: c.table, column: c.column, spellings: c.spellings })) : null };
+ if (pre.code !== 0) { result.problems.push(`pg-preflight exited ${pre.code}${pre.stderr ? `: ${pre.stderr.trim().split('\n').pop()}` : ''}`); return result; }
+ log('preflight: no case-fold collisions');
+
+ // 2. migrate-to-postgres into the scratch target.
+ const store = opts.store === 'auto'
+ ? (process.env.OV_TEST_PG_DIRECT_URL ? 'containers' : 'pglite')
+ : opts.store;
+ target = await openTarget(store, dir, opts['truncate-target']);
+ result.store = target.store;
+ let imp;
+ if (target.store === 'pglite') {
+ const dry = await runNode([path.join(__dirname, 'migrate-to-postgres.js'), '--sqlite', sqlite, '--pglite', '--json'], { cwd: dir, env: childEnv });
+ const dryReport = jsonOf(dry.stdout);
+ if (dry.code !== 0) result.problems.push(`migrate-to-postgres --pglite exited ${dry.code}: ${dryReport ? JSON.stringify(dryReport.problems) : dry.stderr.trim().split('\n').pop()}`);
+ imp = await importIntoPglite(sqlite, target.pgliteDir);
+ imp.script = { exit: dry.code };
+ } else {
+ const run = await runNode([path.join(__dirname, 'migrate-to-postgres.js'), '--sqlite', sqlite, '--json'], { cwd: dir, env: { ...childEnv, DATABASE_DIRECT_URL: target.directUrl } });
+ imp = jsonOf(run.stdout) || { ok: false, tables: [], problems: [{ table: '-', problem: run.stderr.trim().split('\n').pop() || `exit ${run.code}` }] };
+ imp.script = { exit: run.code };
+ }
+ result.import = { ok: imp.ok && imp.script.exit === 0, tables: imp.tables.length, rows: imp.tables.reduce((n, t) => n + t.rows, 0), cleaned: imp.cleaned || [], problems: imp.problems };
+ if (!result.import.ok) { result.problems.push('migrate-to-postgres did not verify'); return result; }
+ log(`import: ${result.import.tables} tables, ${result.import.rows} rows verified`);
+
+ // 3. Row-count parity, before the server's boot seeds anything.
+ const { rowParity } = require('./pg-row-parity');
+ const { createDb } = require('openvibe-sdk/db');
+ const quiet = { log() {}, warn: console.error, error: console.error };
+ const src = new (require('better-sqlite3'))(sqlite, { readonly: true, fileMustExist: true });
+ const db = target.store === 'pglite' ? createDb({ pglite: target.pgliteDir, service: 'network-parity', log: quiet })
+ : createDb({ url: target.directUrl, service: 'network-parity', max: 1, log: quiet });
+ let parity;
+ try { parity = await rowParity(src, db); } finally { src.close(); await db.close(); }
+ result.tables = parity.tables;
+ result.pg_only = parity.pg_only;
+ for (const p of parity.problems) result.problems.push(`${p.table}: ${p.problem}`);
+ if (!parity.ok) return result;
+ log(`parity: ${Object.keys(parity.tables).length} tables equal`);
+
+ // 4. The server on the imported database: /api/ready with db and signing_key ok.
+ result.ready = await bootAndCheck(target, dir, { timeoutMs: opts.timeoutMs });
+ if (!result.ready.ok) result.problems.push(`/api/ready: ${result.ready.http} ${result.ready.status} (failed: ${JSON.stringify(result.ready.failed)})`);
+ else log(`ready: ${result.ready.status} in ${result.ready.ms} ms`);
+ result.ok = result.preflight.ok && result.import.ok && parity.ok && result.ready.ok && result.problems.length === 0;
+ return result;
+ } catch (err) {
+ result.problems.push(err.message);
+ return result;
+ } finally {
+ if (target) await target.close().catch((e) => log(`cleanup: ${e.message}`));
+ if (!opts.keep) fs.rmSync(dir, { recursive: true, force: true });
+ result.ms = Date.now() - t0;
+ }
+}
+
+async function main(argv = process.argv.slice(2)) {
+ let opts;
+ try { opts = parseArgs(argv, { flags: ['keep', 'truncate-target'], values: ['sqlite', 'store', 'write-fixture', 'timeout'] }); } catch (e) { console.error(e.message); return 2; }
+ if (opts.help) { console.error('usage: node scripts/rehearse-pg-cutover.js [--sqlite <copy.db>] [--store auto|pglite|containers|env] [--truncate-target] [--keep] [--write-fixture <file>]'); return 2; }
+ if (opts['write-fixture']) {
+ const file = path.resolve(opts['write-fixture']);
+ if (fs.existsSync(file)) { console.error(`${file} exists; not overwriting it`); return 2; }
+ seedFixture(file);
+ console.error(`wrote ${file}`);
+ return 0;
+ }
+ opts.store = opts.store || 'auto';
+ if (!['auto', 'pglite', 'containers', 'env'].includes(opts.store)) { console.error(`--store ${opts.store}: auto, pglite, containers or env`); return 2; }
+ opts.timeoutMs = opts.timeout ? Number(opts.timeout) * 1000 : 120000;
+ const result = await rehearse(opts);
+ if (result.dir === undefined) delete result.dir;
+ console.log(JSON.stringify(result));
+ return result.ok ? 0 : 1;
+}
+
+if (require.main === module) main().then((code) => process.exit(code), (err) => { console.log(JSON.stringify({ ok: false, problems: [err.message] })); process.exit(1); });
+
+module.exports = { rehearse, seedFixture, FIXTURE_SCHEMA, checkScratchTarget, pollReady };
new file mode 100644
new file mode 100644
@@ -0,0 +1,153 @@
@@ -0,0 +1,153 @@
+'use strict';
+// The T2 cutover rehearsal (scripts/rehearse-pg-cutover.js, docs/cutover-t2-postgres.md) on its seeded fixture:
+// commit 20dd7ff's SQLite schema with representative rows → pg-preflight → migrate-to-postgres → row-count
+// parity → the server booted on the result with /api/ready green. On PGlite by default, on a scratch database
+// in the containers under npm run test:pg. A case-fold collision stops it at the preflight, and the parity
+// check reports a target that lacks the rows or whose users_profile_created trigger the import left off.
+// node test/pg-cutover-rehearsal.test.js
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { spawn } = require('child_process');
+const { rowParity } = require('../scripts/pg-row-parity');
+
+const SCRIPT = path.join(__dirname, '..', 'scripts', 'rehearse-pg-cutover.js');
+// Never a real target: the default store ignores DATABASE_URL, and the child runs without it anyway.
+const env = { ...process.env };
+delete env.DATABASE_URL;
+delete env.DATABASE_DIRECT_URL;
+delete env.NODE_OPTIONS;
+
+function rehearse(args, extraEnv = {}) {
+ return new Promise((resolve) => {
+ const p = spawn(process.execPath, [SCRIPT, ...args], { env: { ...env, ...extraEnv }, stdio: ['ignore', 'pipe', 'pipe'] });
+ let stdout = '', stderr = '';
+ p.stdout.on('data', (d) => { stdout += d; });
+ p.stderr.on('data', (d) => { stderr += d; });
+ p.on('close', (code) => {
+ const lines = stdout.trim().split('\n');
+ let json = null;
+ try { json = JSON.parse(lines[lines.length - 1]); } catch { /* asserted below */ }
+ resolve({ code, json, lines, stderr });
+ });
+ });
+}
+
+(async () => {
+ // 1. The seeded fixture, end to end.
+ const r = await rehearse([]);
+ assert.ok(r.json, `one JSON line last on stdout:\n${r.lines.join('\n')}\n${r.stderr}`);
+ assert.strictEqual(r.lines.length, 1, 'stdout is the JSON line alone');
+ assert.strictEqual(r.json.ok, true, `rehearsal ok: ${JSON.stringify(r.json.problems)}\n${r.stderr}`);
+ assert.strictEqual(r.code, 0);
+ assert.strictEqual(r.json.source, 'seeded-20dd7ff');
+ assert.strictEqual(r.json.store, process.env.OV_TEST_PG_DIRECT_URL ? 'postgresql' : 'pglite');
+ assert.deepStrictEqual(r.json.preflight, { ok: true, exit: 0, collisions: [] });
+ const expected = { users: 6, oauth_clients: 2, oauth_codes: 2, oauth_tokens: 3, notifications: 5, notification_preferences: 3, user_sessions: 2,
+ wallets: 3, coin_transactions: 6, follows: 3, user_follows: 3, user_blocks: 2, verification_keys: 2, network_event_outbox: 1, themes: 0,
+ // 20dd7ff's trigger wrote one per non-anonymous account; PostgreSQL's must not add its own during the import.
+ user_profile_changes: 5 };
+ for (const [t, n] of Object.entries(expected)) assert.deepStrictEqual(r.json.tables[t], { sqlite: n, pg: n }, `${t}: ${n} rows on both sides`);
+ for (const [t, c] of Object.entries(r.json.tables)) assert.strictEqual(c.sqlite, c.pg, `${t}: parity`);
+ // 77 tables at 20dd7ff (initDb's and those its modules create at boot), less the one left behind.
+ assert.strictEqual(Object.keys(r.json.tables).length, 76, 'every table of the 20dd7ff schema is compared');
+ assert.ok(!('analytics_rate_tracking' in r.json.tables) && !('analytics_rate_tracking' in r.json.pg_only), 'per-IP counters are never carried over');
+ assert.deepStrictEqual(r.json.import.cleaned, [{ column: 'notifications.message', values: 1 }], 'the NUL PostgreSQL refuses is dropped and reported');
+ assert.strictEqual(r.json.ready.ok, true);
+ assert.strictEqual(r.json.ready.http, 200);
+ assert.strictEqual(r.json.ready.db, 'ok');
+ assert.strictEqual(r.json.ready.signing_key, 'ok');
+ assert.strictEqual(r.json.ready.node_env, process.env.OV_TEST_PG_DIRECT_URL ? 'production' : 'test');
+ assert.ok(!/\$2a\$10\$|rehearsal-client-secret|rt-family/.test(r.lines[0]), 'counts only: no row value in the output');
+
+ // 2. A case-fold collision (alex / ALEX) stops the cutover at the preflight, before any import.
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-rehearsal-test-'));
+ const file = path.join(dir, 'collide.db');
+ const w = await rehearse(['--write-fixture', file]);
+ assert.strictEqual(w.code, 0, w.stderr);
+ const Database = require('better-sqlite3');
+ const db = new Database(file);
+ db.prepare("INSERT INTO users (id, username, password_hash, subject_id) VALUES (7, 'ALEX', 'x', 'usr_rehearsal0007')").run();
+ db.close();
+ const c = await rehearse(['--sqlite', file, '--store', 'pglite']);
+ assert.strictEqual(c.code, 1);
+ assert.strictEqual(c.json.ok, false);
+ assert.strictEqual(c.json.source, 'snapshot');
+ assert.strictEqual(c.json.preflight.ok, false);
+ assert.deepStrictEqual(c.json.preflight.collisions, [{ table: 'users', column: 'username', spellings: 2 }]);
+ assert.strictEqual(c.json.import, null, 'nothing imported');
+ assert.strictEqual(c.json.ready, null, 'nothing booted');
+ assert.ok(fs.existsSync(file), 'the --sqlite file is left in place');
+
+ // 3. Parity against a database that lacks the rows (this process's migrated, empty one) is a mismatch per table.
+ const src = new Database(file, { readonly: true });
+ const report = await rowParity(src, globalThis.__ovNetworkTestDb);
+ src.close();
+ assert.strictEqual(report.ok, false);
+ assert.deepStrictEqual(report.tables.users, { sqlite: 7, pg: 0 });
+ assert.ok(report.problems.some((p) => p.table === 'coin_transactions' && /SQLite 6, PostgreSQL 0/.test(p.problem)));
+ assert.ok(!report.problems.some((p) => p.table === 'dev_projects'), 'equal counts (0 and 0) are not problems');
+ assert.ok(!report.problems.some((p) => /trigger/.test(p.problem)), 'users_profile_created is enabled');
+
+ // 4. An import killed between its ALTERs leaves users_profile_created off: parity refuses that database. On a
+ // PGlite of its own: the test database's role (DML only under test:pg, as production's runtime) cannot ALTER.
+ const { createDb } = require('openvibe-sdk/db');
+ const { MIGRATIONS } = require('../scripts/migrate-to-postgres');
+ const owner = createDb({ pglite: true, service: 'network-rehearsal-test', log: { log() {}, warn() {}, error: console.error } });
+ let off;
+ try {
+ await owner.migrate({ dir: MIGRATIONS, log: { log() {}, warn() {}, error: console.error } });
+ await owner.exec('ALTER TABLE users DISABLE TRIGGER users_profile_created');
+ const src2 = new Database(file, { readonly: true });
+ try { off = await rowParity(src2, owner); } finally { src2.close(); }
+ } finally { await owner.close(); }
+ assert.ok(off.problems.some((p) => p.table === 'users' && /trigger users_profile_created is disabled/.test(p.problem)), JSON.stringify(off.problems));
+ fs.rmSync(dir, { recursive: true, force: true });
+
+ // 5. --store env refuses an unset or empty DATABASE_URL before it connects anywhere; the message names no URL.
+ for (const extra of [{}, { DATABASE_URL: '' }, { DATABASE_URL: ' ' }]) {
+ const e = await rehearse(['--store', 'env'], { DATABASE_DIRECT_URL: 'postgres://[email protected]:9/none', ...extra });
+ assert.strictEqual(e.code, 1, e.stderr);
+ assert.strictEqual(e.json.ok, false);
+ assert.ok(e.json.problems.some((p) => /needs DATABASE_DIRECT_URL and DATABASE_URL.*DATABASE_URL is unset or empty/.test(p)), JSON.stringify(e.json.problems));
+ assert.strictEqual(e.json.import, null, 'nothing imported');
+ assert.ok(!/127\.0\.0\.1:9|postgres:\/\//.test(e.lines.join('\n')), 'no URL in the output');
+ }
+
+ // 6. The same-database guard: two separate databases are refused; one empty database is accepted; rows refuse it
+ // unless --truncate-target; the probe table never stays behind.
+ const { checkScratchTarget, pollReady } = require('../scripts/rehearse-pg-cutover');
+ const quietLog = { log() {}, warn() {}, error: console.error };
+ const dbA = createDb({ pglite: true, service: 'network-rehearsal-test-a', log: quietLog });
+ const dbB = createDb({ pglite: true, service: 'network-rehearsal-test-b', log: quietLog });
+ try {
+ await assert.rejects(checkScratchTarget(dbA, dbB), (e) => /does not reach the database/.test(e.message) && !/postgres:\/\//.test(e.message));
+ await checkScratchTarget(dbA, dbA);
+ const probes = `SELECT count(*)::int FROM pg_class WHERE relname LIKE 'ov_rehearsal_probe_%'`;
+ assert.strictEqual(Number(await dbA.value(probes)), 0, 'the probe is dropped');
+ assert.strictEqual(Number(await dbB.value(probes)), 0);
+ await dbA.exec('CREATE TABLE rehearsal_rows (id int)');
+ await dbA.exec('INSERT INTO rehearsal_rows VALUES (1)');
+ await assert.rejects(checkScratchTarget(dbA, dbA), /not empty through DATABASE_DIRECT_URL/);
+ await checkScratchTarget(dbA, dbA, { truncateTarget: true });
+ assert.strictEqual(Number(await dbA.value(probes)), 0);
+ } finally { await dbA.close(); await dbB.close(); }
+
+ // 7. A server that accepts and never answers (or stalls mid-body) cannot hold the readiness poll past its deadline.
+ const http = require('http');
+ for (const handler of [() => {}, (req, res) => { res.writeHead(200, { 'content-type': 'application/json' }); res.write('{"ready":'); }]) {
+ const stalled = http.createServer(handler);
+ await new Promise((r) => stalled.listen(0, '127.0.0.1', r));
+ const t = Date.now();
+ try {
+ const p = await pollReady(`http://127.0.0.1:${stalled.address().port}`, { timeoutMs: 3000, requestMs: 1000 });
+ const took = Date.now() - t;
+ assert.strictEqual(p.body, null, 'no answer read');
+ assert.ok(took >= 2500 && took < 6000, `returned in ${took} ms (deadline 3000)`);
+ } finally { stalled.closeAllConnections(); await new Promise((r) => stalled.close(r)); }
+ }
+
+ console.log(`pg cutover rehearsal: all checks passed (${r.json.store}, ${Object.keys(r.json.tables).length} tables, ready ${r.json.ready.status})`);
+ process.exit(0);
+})().catch((err) => { console.error(err); process.exit(1); });