openvibers/openvibe.billing Commits
Billing: the Live importer reads Live's PostgreSQL; the cutover runbook follows (C-30) (#47)
Changes
@@@ -4,7 +4,7 @@**Status:** alpha — runtime built and tested (Wave 8). Deployed on the host in **shadow** modesince 2026-09-23 (`/opt/openvibe.billing`, API on `127.0.0.1:4600`): no provider secret is configured,--it holds a reconciled import of a Live snapshot, and OpenVibe.Live stays the money authority++it holds a reconciled import of Live's money data, and OpenVibe.Live stays the money authority(`BILLING_AUTHORITY` is not set in Live) until the cutover below is run. The cutover has not been run.**Domain:** `billing.openvibe.network` — the staff console (Network SSO, staff only), `/webhooks/*` and`/api/health`; the service API `/api/v1` is reachable only on loopback (`127.0.0.1:4600`)@@@ -89,8 +89,8 @@ npm run dev # http://localhost:4600npm test # 13 test files: stub Network/Events/providers, temp DBs, random portsnpm run reconcile # reconciliation report (exit 1 on failure)npm run freeze -- on "reason" # economy freeze from the host (status | on "<reason>" | off)--node scripts/import-live.js --live-db <snapshot> [--dry-run] [--json]--node scripts/import-live.js --live-db <snapshot> --accounts-only # only PowerChat account → creator mappings++LIVE_DATABASE_URL=… node scripts/import-live.js [--dry-run] [--json] # reads Live's PostgreSQL (or --live-env <Live's env file>)++LIVE_DATABASE_URL=… node scripts/import-live.js --accounts-only # only PowerChat account → creator mappings```Production: `/opt/openvibe.billing`, env `/etc/openvibe/billing.env`, unit@@@ -426,12 +426,12 @@ The exact production sequence, rollback, grants and the Live behaviour the switcswitch unset. None of the steps below has been executed yet; step 3 needs the owner (PowerChatdashboard). In short:--1. **Shadow import** as often as needed (`node scripts/import-live.js --live-db <snapshot> [--dry-run]`);++1. **Shadow import** as often as needed (`billing_live 'node scripts/import-live.js --dry-run'`, docs/live-cutover.md);map held users in the Network and explain every adjustment until reconciliation is clean.2. **Freeze Live money writes**, wait for in-flight PowerChat checkouts (an hour), back up and **freeze Billing**,set `BILLING_AUTHORITY=billing` in billing.env (EXTERNAL tips are then announced for Tips).3. **Re-point the PowerChat webhook** to `https://billing.openvibe.network/webhooks/powerchat` (Billing holds-- the deliveries while frozen), then take the **final snapshot and import**; reconcile.++ the deliveries while frozen), then run the **final import** (Live's money writes frozen); reconcile.4. **Switch Live** (`BILLING_AUTHORITY=billing`, deploy with `--wait-idle`), **unfreeze Billing** (helddeliveries settle; ones Live already credited are rejected for review, never credited twice).5. **Verify** the reads, **unfreeze Live**, verify with a small real PowerChat purchase.@@@ -127,13 +127,12 @@ celebrated exactly once.Shell helpers used below (on the host):```bash--# billing '<command>' runs one command with Billing's environment, as Billing's service user (ubuntu)--# so the Live snapshot files keep their owner — the same way the shadow imports were run.++# billing '<command>' runs one command with Billing's environment, as Billing's service user (ubuntu).billing() { sudo bash -c "set -a; . /etc/openvibe/billing.env; set +a; export NODE_ENV=production; cd /opt/openvibe.billing && sudo -E -u ubuntu $1"; }--LIVE_DB=/opt/openvibe.live/data/live.db # production runs Live's legacy layout (a git checkout); there is no shared/--# /var/backups/openvibe is root-only (0700), so Billing's user cannot read a snapshot there: the import reads--# its copy from Billing's own state directory, and a second copy stays in /var/backups/openvibe for rollback.--SNAPDIR=/var/lib/openvibe-billing++# billing_live '<command>': the same, plus LIVE_DATABASE_URL = Live's own DATABASE_URL. Live runs on PostgreSQL++# (ov_live, since 2026-10-08) and the import reads it directly, in one REPEATABLE READ, READ ONLY transaction.++# Root reads /etc/openvibe/live.env (Billing's user cannot), and the URL never appears on a command line.++billing_live() { sudo bash -c "set -a; . /etc/openvibe/billing.env; LIVE_DATABASE_URL=\$(sed -n 's/^DATABASE_URL=//p' /etc/openvibe/live.env); set +a; export NODE_ENV=production; cd /opt/openvibe.billing && sudo -E -u ubuntu $1"; }```Live owner calls are made from the browser console on openvibe.live, signed in as the owner:@@@ -148,7 +147,7 @@ Live owner calls are made from the browser console on openvibe.live, signed in a2. **Let in-flight checkouts land** on Live: wait at least **60 minutes** after step 1 (minted PowerChatcheckout links live one hour), then run the backfill once: `api('/powerchat/reconcile', { method: 'POST' })`.Look at what is still open:-- `sudo sqlite3 -readonly "$LIVE_DB" "SELECT id, provider, kind, amount_cents, status, created_at FROM payment_orders WHERE status IN ('pending','paid') AND created_at >= datetime('now','-3 days')"`.++ `sudo -u postgres psql -d ov_live -c "SELECT id, provider, kind, amount_cents, status, created_at FROM payment_orders WHERE status IN ('pending','paid') AND created_at >= datetime('now','-3 days')"`.A `paid` row is an anomaly to fix first; a `pending` one is carried into Billing as an intent andsettles there if it is ever paid (canonical, un-minted links do not expire).3. **Back up and freeze Billing** so it holds PowerChat deliveries until the final import is in:@@@ -164,14 +163,14 @@ Live owner calls are made from the browser console on openvibe.live, signed in a`https://openvibe.live/api/powerchat/webhook` → **`https://billing.openvibe.network/webhooks/powerchat`**,same signing secret. From this moment every delivery is stored (202) and held by Billing; Live gets none.`billing 'node scripts/freeze.js status'` shows `held_webhooks` growing only if tips arrive.--5. **Final snapshot and import** (after step 4, so every delivery either reached Live before the snapshot-- or is held by Billing):++5. **Final import** (after step 4, so every delivery either reached Live before this read or is held by++ Billing). Live's money writes are frozen since step 1, and the import reads every Live table in one++ read-only, repeatable-read transaction, so it sees one moment of Live. Dump Live first and keep the dump:```bash-- SNAP=$SNAPDIR/live-cutover-$(date +%F-%H%M).db-- sudo sqlite3 "$LIVE_DB" ".backup $SNAP" && sudo chown ubuntu:ubuntu "$SNAP" && sudo chmod 600 "$SNAP"-- sudo cp -p "$SNAP" /var/backups/openvibe/ # keep this copy (rollback)-- billing "node scripts/import-live.js --live-db $SNAP --dry-run"-- billing "node scripts/import-live.js --live-db $SNAP"++ DUMP=/var/backups/openvibe/live-cutover-$(date +%F-%H%M).sql.gz++ sudo -u postgres pg_dump -d ov_live | gzip | sudo tee "$DUMP" > /dev/null && sudo chmod 600 "$DUMP" # rollback copy++ billing_live 'node scripts/import-live.js --dry-run'++ billing_live 'node scripts/import-live.js'billing 'node scripts/reconcile.js'```Read the report: `holds` (must be empty for anyone with money), `adjustments`, `anomalies`,@@@ -185,9 +184,9 @@ Live owner calls are made from the browser console on openvibe.live, signed in aOV_BILLING_INTERNAL_URL=http://127.0.0.1:4600OV_BILLING_PUBLIC_URL=https://billing.openvibe.network```-- then `sudo /opt/openvibe.live/deploy/scripts/deploy.sh --restart --wait-idle`. `--restart` is-- required: with no new commit, deploy.sh changes nothing unless told to restart, and only a restart-- reads the new env. The systemd socket keeps accepting while Live boots. Wait for `GET /api/ready`.++ then restart Live so it reads the new env: check `curl -s https://openvibe.live/api/streams` for anyone live++ over RTMP (a restart drops them), then `sudo systemctl restart openvibe-live.service`. The systemd socket keeps++ accepting while Live boots. Wait for `GET /api/ready`.`api('/admin/money')` → `authority: "billing"`, `billing.reachable: true`, `billing.rates.drift: []`.Then let Tips post chat lines: set `TIPS_CHAT_ADAPTER=live-chat` in `/etc/openvibe/tips.env` and run`sudo systemctl restart openvibe-tips`. `curl -s http://127.0.0.1:4610/api/health` →@@@ -221,7 +220,7 @@ exactly one of the two: Live before the snapshot, or Billing's held queue after## Rollback--Keep the step-5 snapshot. The Billing side never needs undoing to roll Live back; the question is only++Keep the step-5 dump. The Billing side never needs undoing to roll Live back; the question is onlywhich money moved where.- **Before step 6** (Live still on its own columns): re-point the PowerChat webhook back to@@@ -1,14 +1,14 @@# OpenVibe.Live → OpenVibe.Billing: what maps whereWhere every piece of Live's money state goes in Billing, as the importer--([server/importer/live.js](../server/importer/live.js), `node scripts/import-live.js --live-db <snapshot>`) and++([server/importer/live.js](../server/importer/live.js), `node scripts/import-live.js` with `LIVE_DATABASE_URL`) andthe cutover ([live-cutover.md](live-cutover.md)) implement it. The audit found this was never written down,in particular for **payouts, refunds and plans**. **Live has no table for any of those three.** They liveinside other rows, and this page says which ones.--Counts are from production `/opt/openvibe.live/data/live.db`, read on 2026-09-23. The importer opens a--**snapshot copy** read-only and never writes Live. Every write it makes is keyed, so a re-run over the same--snapshot changes nothing.++Counts are from production Live, read on 2026-09-23 (then its SQLite live.db; Live has run on PostgreSQL,++`ov_live`, since 2026-10-08). The importer reads Live's database in one read-only, repeatable-read transaction and++never writes Live. Every write it makes is keyed, so a re-run over the same data changes nothing.## Balances: `users`@@@ -9,7 +9,6 @@"version": "0.1.0","license": "AGPL-3.0-only","dependencies": {-- "better-sqlite3": "^11.10.0","dotenv": "^16.4.7","express": "^4.21.2","iovalkey": "^0.4.0",@@@ -90,57 +89,6 @@"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==","license": "MIT"},-- "node_modules/base64-js": {-- "version": "1.5.1",-- "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",-- "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT"-- },-- "node_modules/better-sqlite3": {-- "version": "11.10.0",-- "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",-- "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",-- "hasInstallScript": true,-- "license": "MIT",-- "dependencies": {-- "bindings": "^1.5.0",-- "prebuild-install": "^7.1.1"-- }-- },-- "node_modules/bindings": {-- "version": "1.5.0",-- "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",-- "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",-- "license": "MIT",-- "dependencies": {-- "file-uri-to-path": "1.0.0"-- }-- },-- "node_modules/bl": {-- "version": "4.1.0",-- "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",-- "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",-- "license": "MIT",-- "dependencies": {-- "buffer": "^5.5.0",-- "inherits": "^2.0.4",-- "readable-stream": "^3.4.0"-- }-- },"node_modules/body-parser": {"version": "1.20.8","resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.8.tgz",@@@ -165,30 +113,6 @@"npm": "1.2.8000 || >= 1.4.16"}},-- "node_modules/buffer": {-- "version": "5.7.1",-- "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",-- "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT",-- "dependencies": {-- "base64-js": "^1.3.1",-- "ieee754": "^1.1.13"-- }-- },"node_modules/buffer-equal-constant-time": {"version": "1.0.1","resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",@@@ -233,12 +157,6 @@"url": "https://github.com/sponsors/ljharb"}},-- "node_modules/chownr": {-- "version": "1.1.4",-- "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",-- "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",-- "license": "ISC"-- },"node_modules/cluster-key-slot": {"version": "1.1.2","resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",@@@ -293,30 +211,6 @@"ms": "2.0.0"}},-- "node_modules/decompress-response": {-- "version": "6.0.0",-- "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",-- "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",-- "license": "MIT",-- "dependencies": {-- "mimic-response": "^3.1.0"-- },-- "engines": {-- "node": ">=10"-- },-- "funding": {-- "url": "https://github.com/sponsors/sindresorhus"-- }-- },-- "node_modules/deep-extend": {-- "version": "0.6.0",-- "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",-- "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",-- "license": "MIT",-- "engines": {-- "node": ">=4.0.0"-- }-- },"node_modules/denque": {"version": "2.1.0","resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",@@@ -345,15 +239,6 @@"npm": "1.2.8000 || >= 1.4.16"}},-- "node_modules/detect-libc": {-- "version": "2.1.2",-- "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",-- "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",-- "license": "Apache-2.0",-- "engines": {-- "node": ">=8"-- }-- },"node_modules/dotenv": {"version": "16.6.1","resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",@@@ -404,15 +289,6 @@"node": ">= 0.8"}},-- "node_modules/end-of-stream": {-- "version": "1.4.5",-- "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",-- "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",-- "license": "MIT",-- "dependencies": {-- "once": "^1.4.0"-- }-- },"node_modules/es-define-property": {"version": "1.0.1","resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",@@@ -458,15 +334,6 @@"node": ">= 0.6"}},-- "node_modules/expand-template": {-- "version": "2.0.3",-- "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",-- "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",-- "license": "(MIT OR WTFPL)",-- "engines": {-- "node": ">=6"-- }-- },"node_modules/express": {"version": "4.22.3","resolved": "https://registry.npmjs.org/express/-/express-4.22.3.tgz",@@@ -535,12 +402,6 @@],"license": "BSD-3-Clause"},-- "node_modules/file-uri-to-path": {-- "version": "1.0.0",-- "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",-- "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",-- "license": "MIT"-- },"node_modules/finalhandler": {"version": "1.3.2","resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",@@@ -577,12 +438,6 @@"node": ">= 0.6"}},-- "node_modules/fs-constants": {-- "version": "1.0.0",-- "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",-- "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",-- "license": "MIT"-- },"node_modules/function-bind": {"version": "1.1.2","resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",@@@ -629,12 +484,6 @@"node": ">= 0.4"}},-- "node_modules/github-from-package": {-- "version": "0.0.0",-- "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",-- "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",-- "license": "MIT"-- },"node_modules/gopd": {"version": "1.2.0","resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",@@@ -703,38 +552,12 @@"node": ">=0.10.0"}},-- "node_modules/ieee754": {-- "version": "1.2.1",-- "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",-- "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "BSD-3-Clause"-- },"node_modules/inherits": {"version": "2.0.4","resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz","integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==","license": "ISC"},-- "node_modules/ini": {-- "version": "1.3.8",-- "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",-- "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",-- "license": "ISC"-- },"node_modules/iovalkey": {"version": "0.4.0","resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",@@@ -965,45 +788,12 @@"node": ">= 0.6"}},-- "node_modules/mimic-response": {-- "version": "3.1.0",-- "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",-- "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",-- "license": "MIT",-- "engines": {-- "node": ">=10"-- },-- "funding": {-- "url": "https://github.com/sponsors/sindresorhus"-- }-- },-- "node_modules/minimist": {-- "version": "1.2.8",-- "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",-- "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",-- "license": "MIT",-- "funding": {-- "url": "https://github.com/sponsors/ljharb"-- }-- },-- "node_modules/mkdirp-classic": {-- "version": "0.5.3",-- "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",-- "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",-- "license": "MIT"-- },"node_modules/ms": {"version": "2.0.0","resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz","integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==","license": "MIT"},-- "node_modules/napi-build-utils": {-- "version": "2.0.0",-- "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",-- "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",-- "license": "MIT"-- },"node_modules/negotiator": {"version": "0.6.3","resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",@@@ -1013,18 +803,6 @@"node": ">= 0.6"}},-- "node_modules/node-abi": {-- "version": "3.96.0",-- "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",-- "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",-- "license": "MIT",-- "dependencies": {-- "semver": "^7.3.5"-- },-- "engines": {-- "node": ">=10"-- }-- },"node_modules/object-inspect": {"version": "1.13.4","resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz",@@@ -1049,15 +827,6 @@"node": ">= 0.8"}},-- "node_modules/once": {-- "version": "1.4.0",-- "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",-- "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",-- "license": "ISC",-- "dependencies": {-- "wrappy": "1"-- }-- },"node_modules/openvibe-contracts": {"version": "0.129.0","resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.129.0",@@@ -1254,33 +1023,6 @@"node": ">=0.10.0"}},-- "node_modules/prebuild-install": {-- "version": "7.1.3",-- "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",-- "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",-- "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",-- "license": "MIT",-- "dependencies": {-- "detect-libc": "^2.0.0",-- "expand-template": "^2.0.3",-- "github-from-package": "0.0.0",-- "minimist": "^1.2.3",-- "mkdirp-classic": "^0.5.3",-- "napi-build-utils": "^2.0.0",-- "node-abi": "^3.3.0",-- "pump": "^3.0.0",-- "rc": "^1.2.7",-- "simple-get": "^4.0.0",-- "tar-fs": "^2.0.0",-- "tunnel-agent": "^0.6.0"-- },-- "bin": {-- "prebuild-install": "bin.js"-- },-- "engines": {-- "node": ">=10"-- }-- },"node_modules/proxy-addr": {"version": "2.0.8","resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.8.tgz",@@@ -1298,16 +1040,6 @@"url": "https://opencollective.com/express"}},-- "node_modules/pump": {-- "version": "3.0.4",-- "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",-- "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",-- "license": "MIT",-- "dependencies": {-- "end-of-stream": "^1.1.0",-- "once": "^1.3.1"-- }-- },"node_modules/qs": {"version": "6.16.0","resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",@@@ -1348,35 +1080,6 @@"node": ">= 0.8"}},-- "node_modules/rc": {-- "version": "1.2.8",-- "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",-- "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",-- "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",-- "dependencies": {-- "deep-extend": "^0.6.0",-- "ini": "~1.3.0",-- "minimist": "^1.2.0",-- "strip-json-comments": "~2.0.1"-- },-- "bin": {-- "rc": "cli.js"-- }-- },-- "node_modules/readable-stream": {-- "version": "3.6.2",-- "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",-- "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",-- "license": "MIT",-- "dependencies": {-- "inherits": "^2.0.3",-- "string_decoder": "^1.1.1",-- "util-deprecate": "^1.0.1"-- },-- "engines": {-- "node": ">= 6"-- }-- },"node_modules/redis-errors": {"version": "1.2.0","resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",@@@ -1568,51 +1271,6 @@"url": "https://github.com/sponsors/ljharb"}},-- "node_modules/simple-concat": {-- "version": "1.0.1",-- "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",-- "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT"-- },-- "node_modules/simple-get": {-- "version": "4.0.1",-- "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",-- "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT",-- "dependencies": {-- "decompress-response": "^6.0.0",-- "once": "^1.3.1",-- "simple-concat": "^1.0.0"-- }-- },"node_modules/split2": {"version": "4.2.0","resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",@@@ -1637,52 +1295,6 @@"node": ">= 0.8"}},-- "node_modules/string_decoder": {-- "version": "1.3.0",-- "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",-- "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",-- "license": "MIT",-- "dependencies": {-- "safe-buffer": "~5.2.0"-- }-- },-- "node_modules/strip-json-comments": {-- "version": "2.0.1",-- "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",-- "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",-- "license": "MIT",-- "engines": {-- "node": ">=0.10.0"-- }-- },-- "node_modules/tar-fs": {-- "version": "2.1.5",-- "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",-- "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",-- "license": "MIT",-- "dependencies": {-- "chownr": "^1.1.1",-- "mkdirp-classic": "^0.5.2",-- "pump": "^3.0.0",-- "tar-stream": "^2.1.4"-- }-- },-- "node_modules/tar-stream": {-- "version": "2.2.0",-- "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",-- "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",-- "license": "MIT",-- "dependencies": {-- "bl": "^4.0.3",-- "end-of-stream": "^1.4.1",-- "fs-constants": "^1.0.0",-- "inherits": "^2.0.3",-- "readable-stream": "^3.1.1"-- },-- "engines": {-- "node": ">=6"-- }-- },"node_modules/toidentifier": {"version": "1.0.1","resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",@@@ -1692,18 +1304,6 @@"node": ">=0.6"}},-- "node_modules/tunnel-agent": {-- "version": "0.6.0",-- "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",-- "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",-- "license": "Apache-2.0",-- "dependencies": {-- "safe-buffer": "^5.0.1"-- },-- "engines": {-- "node": "*"-- }-- },"node_modules/type-is": {"version": "1.6.18","resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",@@@ -1726,12 +1326,6 @@"node": ">= 0.8"}},-- "node_modules/util-deprecate": {-- "version": "1.0.2",-- "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",-- "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",-- "license": "MIT"-- },"node_modules/utils-merge": {"version": "1.0.1","resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",@@@ -1750,12 +1344,6 @@"node": ">= 0.8"}},-- "node_modules/wrappy": {-- "version": "1.0.2",-- "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",-- "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",-- "license": "ISC"-- },"node_modules/xtend": {"version": "4.0.2","resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",@@@ -23,7 +23,6 @@"author": "OpenVibers","license": "AGPL-3.0-only","dependencies": {-- "better-sqlite3": "^11.10.0","dotenv": "^16.4.7","express": "^4.21.2","iovalkey": "^0.4.0",@@@ -3,35 +3,52 @@/*** Import OpenVibe.Live's money state into Billing (ADR-012 migration).*-- * node scripts/import-live.js --live-db /path/to/live-snapshot.db [--dry-run] [--json]-- * node scripts/import-live.js --live-db /path/to/live-snapshot.db --accounts-only [--dry-run] [--json]++ * LIVE_DATABASE_URL=… node scripts/import-live.js [--dry-run] [--json]++ * LIVE_DATABASE_URL=… node scripts/import-live.js --accounts-only [--dry-run] [--json]++ * (or --live-env <Live's env file> where this user can read it; docs/live-cutover.md has the billing_live helper)** --accounts-only refreshes only provider_accounts from Live's powerchat_connections (who an* EXTERNAL PowerChat tip belongs to); no money, no reconciliation. Safe to run daily after cutover.*-- * --live-db must be a COPY of live.db (e.g. `sqlite3 live.db ".backup live-snapshot.db"`); it is-- * opened read-only. Live user ids are resolved to Network subjects through resolve-batch with this++ * Live's database is OpenVibe.Live's PostgreSQL (`ov_live`). Its URL is LIVE_DATABASE_URL, or the DATABASE_URL line++ * of the env file --live-env names (Live's own, /etc/openvibe/live.env); it is never printed and never passed on a++ * command line. Every read runs in one REPEATABLE READ, READ ONLY transaction (server/importer/live-source.js), so++ * the import sees one moment of Live and cannot write there. Live user ids are resolved to Network subjects through resolve-batch with this* service's client credentials (OV_OAUTH_CLIENT_ID / OV_OAUTH_CLIENT_SECRET, capability* identity.subject.resolve). The report is printed and stored in import_runs; a reconciliation run* follows every real import. Safe to re-run: a second run over the same snapshot changes nothing.*/--const Database = require('better-sqlite3');++const fs = require('fs');const { loadConfig } = require('../server/config');const { openDb } = require('../server/db');const { createRates } = require('../server/rates');const { createIdentity } = require('../server/network');const { importLive, importProviderAccounts } = require('../server/importer/live');++const { openLiveSource } = require('../server/importer/live-source');const { reconcile } = require('../server/reconcile');const args = process.argv.slice(2);const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };const flag = (name) => args.includes(`--${name}`);++/** Live's database URL: LIVE_DATABASE_URL, else the DATABASE_URL line of the --live-env file. Never printed. */++function liveUrl() {++ if (process.env.LIVE_DATABASE_URL) return process.env.LIVE_DATABASE_URL;++ const file = opt('live-env');++ if (!file) return null;++ for (const line of fs.readFileSync(file, 'utf8').split('\n')) {++ const m = /^\s*DATABASE_URL\s*=\s*(.*)$/.exec(line);++ if (m) return m[1].trim().replace(/^(['"])(.*)\1$/, '$2') || null;++ }++ return null;++}++async function main() {-- const livePath = opt('live-db');-- if (!livePath) { console.error('usage: import-live.js --live-db <snapshot path> [--dry-run] [--json]'); process.exit(2); }++ if (args.includes('--live-db')) { console.error('--live-db is gone: Live runs on PostgreSQL; use --live-env /etc/openvibe/live.env (or LIVE_DATABASE_URL)'); process.exit(2); }++ const url = liveUrl();++ if (!url) { console.error('usage: import-live.js --live-env <Live env file> [--dry-run] [--json] [--accounts-only] (or set LIVE_DATABASE_URL)'); process.exit(2); }const config = loadConfig();-- const live = new Database(livePath, { readonly: true, fileMustExist: true });++ const live = await openLiveSource({ url });const db = await openDb(config);const ctx = { db, config, rates: createRates(config.rates), now: () => Date.now(), log: console };const identity = createIdentity(config);@@@ -43,7 +60,7 @@ async function main() {console.log(`provider accounts ${r.run_id}${r.dry_run ? ' (DRY RUN — nothing kept)' : ''}: ${a.mapped} mapped, ${a.unchanged} unchanged, ${a.kept_admin.length} kept (set by an operator)`);for (const u of a.unmapped) console.log(` unmapped: live user ${u.live_user_id} (${u.username}): ${u.reason}`);}-- live.close();++ await live.close();db.close();return;}@@@ -52,7 +69,7 @@ async function main() {if (flag('json')) console.log(JSON.stringify({ import: report, reconciliation: rec }, null, 2));else {console.log(`import ${report.run_id}${report.dry_run ? ' (DRY RUN — nothing kept)' : ''}`);-- console.log(` live snapshot: ${JSON.stringify(report.counts.live)}; identities ${JSON.stringify(report.counts.identities)}`);++ console.log(` live database: ${JSON.stringify(report.counts.live)}; identities ${JSON.stringify(report.counts.identities)}`);console.log(` new: ${report.new_transactions} transactions, ${report.counts.intents_new} intents, ${report.counts.receipts_new} receipts; subscriptions ${report.subscriptions.imported} imported / ${report.subscriptions.updated} updated`);console.log(` opening adjustments (${report.adjustments.length}):`);for (const a of report.adjustments) console.log(` live user ${a.live_user_id} (${a.username}) ${a.account}: history ${a.replayed_history} → column ${a.live_column} (adjust ${a.adjustment > 0 ? '+' : ''}${a.adjustment})`);@@@ -64,7 +81,7 @@ async function main() {if (pa) console.log(` PowerChat accounts: ${pa.mapped} mapped, ${pa.unchanged} unchanged, ${pa.unmapped.length} unmapped, ${pa.kept_admin.length} kept (operator)`);if (rec) console.log(` reconciliation ${rec.id}: ${rec.ok ? 'OK' : 'FAILED'} (${rec.checks.filter((c) => !c.ok).map((c) => c.id).join(', ') || 'all checks pass'})`);}-- live.close();++ await live.close();db.close();if (rec && !rec.ok) process.exit(1);}nnew file mode 100644@@@ -0,0 +1,63 @@++'use strict';++++/**++ * The OpenVibe.Live database the importer reads (./live.js): Live's PostgreSQL database (`ov_live`, since++ * 2026-10-08; the SQLite live.db is gone).++ *++ * openLiveSource({ url }) a pg connection to Live's database. Every read runs in ONE transaction,++ * REPEATABLE READ and READ ONLY, so all the tables come from the same moment++ * (at cutover Live's money writes are frozen first, so that moment is the++ * final state) and nothing can be written through it. close() ends it.++ * liveSourceFromQuery(query) the same interface over any (text, params) → rows function (the tests'++ * PGlite).++ *++ * The interface is the one the importer was written against: live.prepare(sql).all(...params) and .get(...),++ * with `?` placeholders. Values arrive as openvibe-sdk/db hands them to Live itself (its PARSERS): ids and other++ * int8 columns as numbers, timestamps as ISO strings. Live's money columns are double precision and its++ * dates are text, so they come through unchanged.++ */++const { PARSERS } = require('openvibe-sdk/db');++++/** `?` placeholders → `$1, $2, …` (the importer's Live queries put no `?` inside a string literal). */++function toPg(sql) {++ let n = 0;++ return String(sql).replace(/\?/g, () => `$${++n}`);++}++++function liveSourceFromQuery(query, close = async () => {}) {++ return {++ prepare(sql) {++ const text = toPg(sql);++ return {++ all: async (...params) => await query(text, params),++ get: async (...params) => (await query(text, params))[0],++ };++ },++ close,++ };++}++++async function openLiveSource({ url, applicationName = 'billing-import-live' }) {++ if (!url) throw new Error('no Live database URL');++ const pg = require('pg');++ const client = new pg.Client({++ connectionString: url,++ application_name: applicationName,++ types: { getTypeParser: (oid, format) => (format === 'text' && PARSERS[oid]) || pg.types.getTypeParser(oid, format) },++ });++ await client.connect();++ try {++ await client.query('BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY');++ } catch (err) {++ await client.end().catch(() => {});++ throw err;++ }++ let closed = false;++ return liveSourceFromQuery(async (text, params) => (await client.query(text, params)).rows, async () => {++ if (closed) return;++ closed = true;++ try { await client.query('ROLLBACK'); } finally { await client.end(); }++ });++}++++module.exports = { openLiveSource, liveSourceFromQuery, toPg };@@@ -1,7 +1,8 @@'use strict';/**-- * Import from a SNAPSHOT COPY of OpenVibe.Live's live.db (opened read-only; never the live file).++ * Import from OpenVibe.Live's PostgreSQL database (`ov_live`), read in one REPEATABLE READ, READ ONLY++ * transaction (./live-source.js), so every table comes from the same moment and nothing is written there.** 1. identities every Live user id that holds a balance or appears in a money row is resolved to a* Network subject (resolve-batch). Unmapped users are never dropped: their accounts@@@ -42,14 +43,20 @@ const ROUTE_MARKER = /^(direct|site)(:|$)/;const liveTs = (s) => {if (!s) return null;++ if (s instanceof Date) return Number.isFinite(s.getTime()) ? s.toISOString() : null;const str = String(s);const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);return Number.isFinite(t) ? new Date(t).toISOString() : null;};--// Live's own database, still SQLite (read-only here): its catalog.--function tableExists(live, name) { return !!live.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name); }--function columns(live, table) { return tableExists(live, table) ? live.prepare(`PRAGMA table_info(${table})`).all().map((c) => c.name) : []; }++// Live's own database (PostgreSQL, read-only here): its catalog, in the schema the connection uses.++async function tableExists(live, name) {++ return !!await live.prepare('SELECT 1 FROM information_schema.tables WHERE table_schema = current_schema() AND table_name = ?').get(name);++}++async function columns(live, table) {++ const rows = await live.prepare('SELECT column_name FROM information_schema.columns WHERE table_schema = current_schema() AND table_name = ? ORDER BY ordinal_position').all(table);++ return rows.map((c) => c.column_name);++}class DryRun extends Error {}@@@ -84,12 +91,12 @@ async function applyProviderAccounts(ctx, rows, report) {return out;}--/** Step 6 alone: refresh provider_accounts from a Live snapshot (no money is touched). */++/** Step 6 alone: refresh provider_accounts from Live's database (no money is touched). */async function importProviderAccounts(ctx, { live, resolveLiveUsers, dryRun = false, log = console }) {const { db } = ctx;const started = ctx.now();const runId = prefixedId('imp', started);-- const report = { run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live snapshot (provider accounts only)' };++ const report = { run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live database (provider accounts only)' };const rows = await readProviderAccounts(live, resolveLiveUsers);try {await money(db, async () => {@@@ -110,12 +117,12 @@ async function importLive(ctx, { live, resolveLiveUsers, dryRun = false, log = cconst started = ctx.now();const runId = prefixedId('imp', started);const report = {-- run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live snapshot',++ run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live database',counts: {}, holds: [], duplicate_provider_refs: [], route_markers: 0, adjustments: [], unreplayable: [],anomalies: [], subscriptions: { imported: 0, updated: 0, skipped: [] }, new_transactions: 0,};-- // ── Read the snapshot ────────────────────────────────────++ // ── Read Live (one read-only snapshot) ───────────────────const userCols = await columns(live, 'users');const hasCashout = userCols.includes('openvibe_bucks_cashout_balance');const users = await live.prepare(`SELECT id, username, COALESCE(openvibe_bucks_balance, 0) AS credit${hasCashout ? ', COALESCE(openvibe_bucks_cashout_balance, 0) AS payable' : ', 0 AS payable'} FROM users`).all();@@@ -9,36 +9,25 @@* import run never books an opening-balance adjustment against what Billing settled itself.*/const assert = require('assert');--const path = require('path');--const Database = require('better-sqlite3');++const { fakeLive } = require('./helpers/live-pg');const { boot, check, done } = require('./helpers/app');--async function fabricateLive(file) {-- const live = new Database(file);++async function fabricateLive() {++ const live = await fakeLive();const recent = new Date(Date.now() - 30 * 60_000).toISOString().replace('T', ' ').slice(0, 19);-- live.exec(`-- CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, openvibe_bucks_balance REAL DEFAULT 0.00,-- openvibe_bucks_cashout_balance REAL DEFAULT 0.00);-- CREATE TABLE transactions (id INTEGER PRIMARY KEY AUTOINCREMENT, from_user_id INTEGER, to_user_id INTEGER, stream_id INTEGER,-- amount INTEGER NOT NULL, type TEXT NOT NULL, status TEXT DEFAULT 'completed', message TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP);-- CREATE TABLE payment_orders (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, provider TEXT NOT NULL, provider_ref TEXT,-- kind TEXT NOT NULL DEFAULT 'bucks', amount_cents INTEGER NOT NULL DEFAULT 0, currency TEXT DEFAULT 'usd', bucks INTEGER DEFAULT 0,-- streamer_id INTEGER, status TEXT DEFAULT 'pending', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);-- CREATE TABLE subscriptions (id INTEGER PRIMARY KEY AUTOINCREMENT, subscriber_id INTEGER NOT NULL, streamer_id INTEGER NOT NULL, tier INTEGER DEFAULT 1,-- provider TEXT, provider_ref TEXT, price_cents INTEGER DEFAULT 0, status TEXT DEFAULT 'active', auto_renew INTEGER DEFAULT 0,-- current_period_end DATETIME, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);++ await live.exec(`INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance) VALUES (1, 'alice', 500, 0), (2, 'bob', 0, 349), (3, 'carol', 0, 0);INSERT INTO transactions (id, from_user_id, to_user_id, amount, type, status, created_at) VALUES (1, NULL, 1, 500, 'purchase', 'completed', '${recent}');`);-- await live.prepare(`INSERT INTO payment_orders (id, user_id, provider, provider_ref, kind, amount_cents, bucks, streamer_id, status, created_at) VALUES-- (1, 1, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'credited', ?),-- (2, 3, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'pending', ?),-- (3, 1, 'powerchat', 'site:fee=50', 'subscription', 549, 0, 2, 'credited', ?),-- (4, 3, 'powerchat', NULL, 'bucks', 1300, 1000, NULL, 'pending', ?)`).run(recent, recent, recent, recent);++ await live.query(`INSERT INTO payment_orders (id, user_id, provider, provider_ref, kind, amount_cents, bucks, streamer_id, status, created_at) VALUES++ (1, 1, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'credited', $1),++ (2, 3, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'pending', $1),++ (3, 1, 'powerchat', 'site:fee=50', 'subscription', 549, 0, 2, 'credited', $1),++ (4, 3, 'powerchat', NULL, 'bucks', 1300, 1000, NULL, 'pending', $1)`, [recent]);const end = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);-- await live.prepare(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, current_period_end, created_at)-- VALUES (1, 1, 2, 'powerchat', 549, 'active', ?, ?)`).run(end, recent);-- live.close();++ await live.query(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, current_period_end, created_at)++ VALUES (1, 1, 2, 'powerchat', 549, 'active', $1, $2)`, [end, recent]);++ return live;}(async () => {@@@ -46,14 +35,12 @@ async function fabricateLive(file) {const alice = t.network.addUser(1);const bob = t.network.addUser(2);const carol = t.network.addUser(3);-- const file = path.join(t.dir, 'live-snapshot.db');-- await fabricateLive(file);++ const fake = await fabricateLive();const { importLive } = require('../server/importer/live');const { createIdentity } = require('../server/network');const identity = createIdentity(t.config);const run = async () => {-- const live = new Database(file, { readonly: true, fileMustExist: true });-- try { return await importLive(t.ctx, { live, resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} } }); } finally { live.close(); }++ return await importLive(t.ctx, { live: fake.source(), resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} } });};const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };const intent = async (order) => await t.db.prepare('SELECT * FROM payment_intents WHERE legacy_order_id = ?').get(order);@@@ -96,11 +83,9 @@ async function fabricateLive(file) {await check('an order Live credited AFTER the shadow import is settled-in-Live after the final import', async () => {assert.strictEqual((await intent(4)).status, 'created');-- const live = new Database(file);-- live.exec(`UPDATE payment_orders SET status = 'credited' WHERE id = 4;++ await fake.exec(`UPDATE payment_orders SET status = 'credited' WHERE id = 4;UPDATE users SET openvibe_bucks_balance = 1000 WHERE id = 3;-- INSERT INTO transactions (from_user_id, to_user_id, amount, type, status) VALUES (NULL, 3, 1000, 'purchase', 'completed');`);-- live.close();++ INSERT INTO transactions (id, from_user_id, to_user_id, amount, type, status) VALUES (2, NULL, 3, 1000, 'purchase', 'completed');`);const report = await run();assert.ok(report.counts.intents_updated >= 1, JSON.stringify(report.counts));assert.strictEqual((await intent(4)).status, 'settled');@@@ -133,9 +118,7 @@ async function fabricateLive(file) {assert.strictEqual((await cli('on', 'Live cutover')).frozen, true);const w = await t.call('POST', '/api/v1/transfers', { body: { from: { type: 'user', id: carol }, to: { type: 'user', id: bob }, amount: 5 } });assert.strictEqual(w.status, 503); assert.strictEqual(w.json.code, 'billing.frozen');-- const live = new Database(file);-- live.exec("INSERT INTO payment_orders (id, user_id, provider, kind, amount_cents, bucks, status) VALUES (5, 3, 'powerchat', 'bucks', 700, 500, 'pending')");-- live.close();++ await fake.exec("INSERT INTO payment_orders (id, user_id, provider, kind, amount_cents, bucks, status) VALUES (5, 3, 'powerchat', 'bucks', 700, 500, 'pending')");await run(); // the final import, while Billing is frozenconst held = await tip('pcorder:5', 700);assert.strictEqual(held.status, 202, JSON.stringify(held.json));@@@ -7,8 +7,7 @@* never celebrated twice. Accounts come from Live's powerchat_connections (import) or an operator.*/const assert = require('assert');--const path = require('path');--const Database = require('better-sqlite3');++const { fakeLive } = require('./helpers/live-pg');const { validate } = require('openvibe-contracts');const { boot, check, done } = require('./helpers/app');const { startEvents } = require('./helpers/stubs');@@@ -166,18 +165,16 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS event});await check('the importer maps Live\'s powerchat_connections (never over an operator\'s mapping); --accounts-only alone', async () => {-- const file = path.join(t.dir, 'live-accounts.db');-- const live = new Database(file);-- live.exec(`CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT, openvibe_bucks_balance REAL DEFAULT 0, openvibe_bucks_cashout_balance REAL DEFAULT 0);-- CREATE TABLE powerchat_connections (user_id INTEGER PRIMARY KEY, powerchat_username TEXT, powerchat_user_id TEXT);-- INSERT INTO users (id, username) VALUES (50, 'newbie'), (51, 'ghost'), (52, 'clash');++ // A Live database with users and PowerChat connections only (the other money tables absent).++ const fake = await fakeLive({ tables: `CREATE TABLE users (id bigint PRIMARY KEY, username text, openvibe_bucks_balance double precision DEFAULT 0, openvibe_bucks_cashout_balance double precision DEFAULT 0);++ CREATE TABLE powerchat_connections (user_id bigint PRIMARY KEY, powerchat_username text, powerchat_user_id text);` });++ await fake.exec(`INSERT INTO users (id, username) VALUES (50, 'newbie'), (51, 'ghost'), (52, 'clash');INSERT INTO powerchat_connections VALUES (50, 'NewbiePC', 'pc-50'), (51, 'GhostPC', 'pc-51'), (52, 'StreamerPC', 'pc-77'), (53, 'bad name!', NULL);`);-- live.close();const newbie = t.network.addUser(50);t.network.addUser(52);t.network.addUser(53);const { importProviderAccounts, importLive } = require('../server/importer/live');-- const ro = new Database(file, { readonly: true });++ const ro = fake.source();const dry = await importProviderAccounts(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, dryRun: true, log: {} });assert.strictEqual(dry.provider_accounts.mapped, 1);assert.strictEqual((await t.db.prepare("SELECT COUNT(*) AS n FROM provider_accounts WHERE username = 'newbiepc'").get()).n, 0, 'dry run keeps nothing');@@@ -191,7 +188,8 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS eventassert.strictEqual((await t.db.prepare("SELECT subject FROM provider_accounts WHERE username = 'streamerpc'").get()).subject, streamer.id);const full = await importLive(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, log: {} });assert.strictEqual(full.provider_accounts.unchanged, 1, 'the full import runs the same step; a re-run changes nothing');-- ro.close();++ await ro.close();++ await fake.close();const r = await t.powerchat(donation({ eventId: 'ext-newbie', amountUsdCents: 100 }, { id: 'pc-50', username: 'NewbiePC' }));assert.strictEqual(r.json.result.streamer, newbie);await t.assertReconciled('after the accounts import');nnew file mode 100644@@@ -0,0 +1,49 @@++'use strict';++++/**++ * A fabricated OpenVibe.Live database for the importer tests: PGlite (PostgreSQL, as Live's ov_live is) with Live's++ * money tables in the column types its migrations give them (bigint ids and amounts, double precision balances,++ * text dates). Values come back through openvibe-sdk/db's PARSERS, as they do from the real database.++ *++ * const live = await fakeLive(); // the tables, empty++ * await live.exec('INSERT …'); // set it up, or change it between two imports++ * importLive(ctx, { live: live.source(), … })++ */++const { PGlite } = require('@electric-sql/pglite');++const { PARSERS } = require('openvibe-sdk/db');++const { liveSourceFromQuery } = require('../../server/importer/live-source');++++// The columns the importer reads, as OpenVibe.Live's migrations/0002_live.sql creates them.++const LIVE_MONEY_TABLES = `++ CREATE TABLE users (id bigint PRIMARY KEY, username text UNIQUE NOT NULL, openvibe_bucks_balance double precision DEFAULT 0.00,++ openvibe_coins_balance bigint DEFAULT 0, openvibe_bucks_cashout_balance double precision DEFAULT 0.00);++ CREATE TABLE transactions (id bigint PRIMARY KEY, from_user_id bigint, to_user_id bigint, stream_id bigint,++ amount bigint NOT NULL, type text NOT NULL CHECK(type IN ('donation', 'purchase', 'subscription', 'cashout', 'refund', 'bonus')),++ status text DEFAULT 'completed' CHECK(status IN ('pending', 'completed', 'failed', 'escrow', 'refunded')), message text,++ paypal_transaction_id text, created_at text);++ CREATE TABLE payment_orders (id bigint PRIMARY KEY, user_id bigint NOT NULL, provider text NOT NULL, provider_ref text,++ kind text NOT NULL DEFAULT 'bucks', amount_cents bigint NOT NULL DEFAULT 0, currency text DEFAULT 'usd', bucks bigint DEFAULT 0,++ streamer_id bigint, status text DEFAULT 'pending', created_at text, updated_at text);++ CREATE TABLE subscriptions (id bigint PRIMARY KEY, subscriber_id bigint NOT NULL, streamer_id bigint NOT NULL, tier bigint DEFAULT 1,++ is_active bigint DEFAULT 1, started_at text, expires_at text, provider text, provider_ref text,++ price_cents bigint DEFAULT 0, currency text DEFAULT 'usd', status text DEFAULT 'active', cancel_at_period_end bigint DEFAULT 0,++ auto_renew bigint DEFAULT 0, current_period_end text, created_at text, updated_at text);++ CREATE TABLE site_settings (key text PRIMARY KEY, value text NOT NULL DEFAULT '', description text DEFAULT '', type text DEFAULT 'string', updated_at text);++ CREATE TABLE powerchat_connections (user_id bigint PRIMARY KEY, powerchat_username text, powerchat_user_id text, updated_at text);++`;++++async function fakeLive({ tables = LIVE_MONEY_TABLES } = {}) {++ const parsers = {};++ for (const [oid, fn] of Object.entries(PARSERS)) parsers[oid] = (v) => fn(v);++ const pg = new PGlite(undefined, { parsers });++ await pg.waitReady;++ if (tables) await pg.exec(tables);++ return {++ exec: async (sql) => { await pg.exec(sql); },++ query: async (text, params = []) => (await pg.query(text, params, { parsers })).rows,++ source: () => liveSourceFromQuery(async (text, params) => (await pg.query(text, params, { parsers })).rows),++ close: async () => { await pg.close(); },++ };++}++++module.exports = { fakeLive, LIVE_MONEY_TABLES };@@@ -1,32 +1,17 @@'use strict';/**-- * Import from a fabricated live.db snapshot (Live's own table shapes): opening balances equal the++ * Import from a fabricated Live database (PostgreSQL in Live's own column types, test/helpers/live-pg.js): opening balances equal the* Live columns, differences against the replayed history are adjustments and listed, unmapped users* are held (never dropped) and released once mapped, test-era rows are flagged, duplicate provider* refs and route markers are handled, dry runs keep nothing, and a re-run changes nothing.*/const assert = require('assert');--const path = require('path');--const Database = require('better-sqlite3');++const { fakeLive } = require('./helpers/live-pg');const { boot, check, done } = require('./helpers/app');--async function fabricateLive(file) {-- const live = new Database(file);-- live.exec(`-- CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, openvibe_bucks_balance REAL DEFAULT 0.00,-- openvibe_coins_balance INTEGER DEFAULT 0, openvibe_bucks_cashout_balance REAL DEFAULT 0.00);-- CREATE TABLE transactions (id INTEGER PRIMARY KEY AUTOINCREMENT, from_user_id INTEGER, to_user_id INTEGER, stream_id INTEGER,-- amount INTEGER NOT NULL, type TEXT NOT NULL CHECK(type IN ('donation', 'purchase', 'subscription', 'cashout', 'refund', 'bonus')),-- status TEXT DEFAULT 'completed' CHECK(status IN ('pending', 'completed', 'failed', 'escrow', 'refunded')), message TEXT,-- paypal_transaction_id TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP);-- CREATE TABLE payment_orders (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, provider TEXT NOT NULL, provider_ref TEXT,-- kind TEXT NOT NULL DEFAULT 'bucks', amount_cents INTEGER NOT NULL DEFAULT 0, currency TEXT DEFAULT 'usd', bucks INTEGER DEFAULT 0,-- streamer_id INTEGER, status TEXT DEFAULT 'pending', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);-- CREATE TABLE subscriptions (id INTEGER PRIMARY KEY AUTOINCREMENT, subscriber_id INTEGER NOT NULL, streamer_id INTEGER NOT NULL, tier INTEGER DEFAULT 1,-- is_active INTEGER DEFAULT 1, started_at DATETIME DEFAULT CURRENT_TIMESTAMP, expires_at DATETIME, provider TEXT, provider_ref TEXT,-- price_cents INTEGER DEFAULT 0, currency TEXT DEFAULT 'usd', status TEXT DEFAULT 'active', cancel_at_period_end INTEGER DEFAULT 0,-- auto_renew INTEGER DEFAULT 0, current_period_end DATETIME, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);-- CREATE TABLE site_settings (key TEXT PRIMARY KEY, value TEXT NOT NULL DEFAULT '', description TEXT DEFAULT '', type TEXT DEFAULT 'string', updated_at DATETIME);++async function fabricateLive() {++ const live = await fakeLive();++ await live.exec(`INSERT INTO site_settings (key, value) VALUES ('stats_vibes_reset_at', '2026-06-01T00:00:00Z');INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance, openvibe_coins_balance) VALUES(1, 'alice', 1500, 0, 999), (2, 'bob', 0, 700, 0), (3, 'carol', 250, 0, 0), (4, 'dave', 100.5, 0, 0), (5, 'erin', 0, 0, 0);@@@ -48,10 +33,9 @@ async function fabricateLive(file) {(5, 3, 'powerchat', NULL, 'bucks', 325, 250, NULL, 'paid', '2026-07-06 09:00:00');`);const future = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);-- await live.prepare(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, auto_renew, current_period_end, created_at)-- VALUES (1, 1, 2, 'powerchat', 549, 'active', 1, ?, '2026-07-08 09:00:00'), (2, 4, 2, 'bucks', 499, 'expired', 0, '2026-07-01 00:00:00', '2026-06-01 00:00:00')`).run(future);-- live.close();-- return new Database(file, { readonly: true, fileMustExist: true });++ await live.query(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, auto_renew, current_period_end, created_at)++ VALUES (1, 1, 2, 'powerchat', 549, 'active', 1, $1, '2026-07-08 09:00:00'), (2, 4, 2, 'bucks', 499, 'expired', 0, '2026-07-01 00:00:00', '2026-06-01 00:00:00')`, [future]);++ return live;}(async () => {@@@ -60,14 +44,15 @@ async function fabricateLive(file) {const bob = t.network.addUser(2);const dave = t.network.addUser(4);t.network.addUser(5);-- const live = await fabricateLive(path.join(t.dir, 'live-snapshot.db'));++ const fake = await fabricateLive();++ const live = fake.source();const { importLive } = require('../server/importer/live');const { createIdentity } = require('../server/network');const identity = createIdentity(t.config);const run = async (opts = {}) => await importLive(t.ctx, { live, resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} }, ...opts });const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };const count = async (table) => (await t.db.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get()).n;-- console.log('import from live.db');++ console.log('import from Live\'s database');await check('a dry run reports everything and keeps nothing but its report', async () => {const r = await run({ dryRun: true });@@@ -159,7 +144,7 @@ async function fabricateLive(file) {assert.strictEqual((await run()).new_transactions, 0);});-- live.close();++ await fake.close();await t.close();done();})();
@@ -4,7 +4,7 @@
@@ -4,7 +4,7 @@
**Status:** alpha — runtime built and tested (Wave 8). Deployed on the host in **shadow** mode
**Status:** alpha — runtime built and tested (Wave 8). Deployed on the host in **shadow** mode
since 2026-09-23 (`/opt/openvibe.billing`, API on `127.0.0.1:4600`): no provider secret is configured,
since 2026-09-23 (`/opt/openvibe.billing`, API on `127.0.0.1:4600`): no provider secret is configured,
-it holds a reconciled import of a Live snapshot, and OpenVibe.Live stays the money authority
+it holds a reconciled import of Live's money data, and OpenVibe.Live stays the money authority
(`BILLING_AUTHORITY` is not set in Live) until the cutover below is run. The cutover has not been run.
(`BILLING_AUTHORITY` is not set in Live) until the cutover below is run. The cutover has not been run.
**Domain:** `billing.openvibe.network` — the staff console (Network SSO, staff only), `/webhooks/*` and
**Domain:** `billing.openvibe.network` — the staff console (Network SSO, staff only), `/webhooks/*` and
`/api/health`; the service API `/api/v1` is reachable only on loopback (`127.0.0.1:4600`)
`/api/health`; the service API `/api/v1` is reachable only on loopback (`127.0.0.1:4600`)
@@ -89,8 +89,8 @@ npm run dev # http://localhost:4600
@@ -89,8 +89,8 @@ npm run dev # http://localhost:4600
npm test # 13 test files: stub Network/Events/providers, temp DBs, random ports
npm test # 13 test files: stub Network/Events/providers, temp DBs, random ports
npm run reconcile # reconciliation report (exit 1 on failure)
npm run reconcile # reconciliation report (exit 1 on failure)
npm run freeze -- on "reason" # economy freeze from the host (status | on "<reason>" | off)
npm run freeze -- on "reason" # economy freeze from the host (status | on "<reason>" | off)
-node scripts/import-live.js --live-db <snapshot> [--dry-run] [--json]
-node scripts/import-live.js --live-db <snapshot> --accounts-only # only PowerChat account → creator mappings
+LIVE_DATABASE_URL=… node scripts/import-live.js [--dry-run] [--json] # reads Live's PostgreSQL (or --live-env <Live's env file>)
+LIVE_DATABASE_URL=… node scripts/import-live.js --accounts-only # only PowerChat account → creator mappings
```
```
Production: `/opt/openvibe.billing`, env `/etc/openvibe/billing.env`, unit
Production: `/opt/openvibe.billing`, env `/etc/openvibe/billing.env`, unit
@@ -426,12 +426,12 @@ The exact production sequence, rollback, grants and the Live behaviour the switc
@@ -426,12 +426,12 @@ The exact production sequence, rollback, grants and the Live behaviour the switc
switch unset. None of the steps below has been executed yet; step 3 needs the owner (PowerChat
switch unset. None of the steps below has been executed yet; step 3 needs the owner (PowerChat
dashboard). In short:
dashboard). In short:
-1. **Shadow import** as often as needed (`node scripts/import-live.js --live-db <snapshot> [--dry-run]`);
+1. **Shadow import** as often as needed (`billing_live 'node scripts/import-live.js --dry-run'`, docs/live-cutover.md);
map held users in the Network and explain every adjustment until reconciliation is clean.
map held users in the Network and explain every adjustment until reconciliation is clean.
2. **Freeze Live money writes**, wait for in-flight PowerChat checkouts (an hour), back up and **freeze Billing**,
2. **Freeze Live money writes**, wait for in-flight PowerChat checkouts (an hour), back up and **freeze Billing**,
set `BILLING_AUTHORITY=billing` in billing.env (EXTERNAL tips are then announced for Tips).
set `BILLING_AUTHORITY=billing` in billing.env (EXTERNAL tips are then announced for Tips).
3. **Re-point the PowerChat webhook** to `https://billing.openvibe.network/webhooks/powerchat` (Billing holds
3. **Re-point the PowerChat webhook** to `https://billing.openvibe.network/webhooks/powerchat` (Billing holds
- the deliveries while frozen), then take the **final snapshot and import**; reconcile.
+ the deliveries while frozen), then run the **final import** (Live's money writes frozen); reconcile.
4. **Switch Live** (`BILLING_AUTHORITY=billing`, deploy with `--wait-idle`), **unfreeze Billing** (held
4. **Switch Live** (`BILLING_AUTHORITY=billing`, deploy with `--wait-idle`), **unfreeze Billing** (held
deliveries settle; ones Live already credited are rejected for review, never credited twice).
deliveries settle; ones Live already credited are rejected for review, never credited twice).
5. **Verify** the reads, **unfreeze Live**, verify with a small real PowerChat purchase.
5. **Verify** the reads, **unfreeze Live**, verify with a small real PowerChat purchase.
@@ -127,13 +127,12 @@ celebrated exactly once.
@@ -127,13 +127,12 @@ celebrated exactly once.
Shell helpers used below (on the host):
Shell helpers used below (on the host):
```bash
```bash
-# billing '<command>' runs one command with Billing's environment, as Billing's service user (ubuntu)
-# so the Live snapshot files keep their owner — the same way the shadow imports were run.
+# billing '<command>' runs one command with Billing's environment, as Billing's service user (ubuntu).
billing() { sudo bash -c "set -a; . /etc/openvibe/billing.env; set +a; export NODE_ENV=production; cd /opt/openvibe.billing && sudo -E -u ubuntu $1"; }
billing() { sudo bash -c "set -a; . /etc/openvibe/billing.env; set +a; export NODE_ENV=production; cd /opt/openvibe.billing && sudo -E -u ubuntu $1"; }
-LIVE_DB=/opt/openvibe.live/data/live.db # production runs Live's legacy layout (a git checkout); there is no shared/
-# /var/backups/openvibe is root-only (0700), so Billing's user cannot read a snapshot there: the import reads
-# its copy from Billing's own state directory, and a second copy stays in /var/backups/openvibe for rollback.
-SNAPDIR=/var/lib/openvibe-billing
+# billing_live '<command>': the same, plus LIVE_DATABASE_URL = Live's own DATABASE_URL. Live runs on PostgreSQL
+# (ov_live, since 2026-10-08) and the import reads it directly, in one REPEATABLE READ, READ ONLY transaction.
+# Root reads /etc/openvibe/live.env (Billing's user cannot), and the URL never appears on a command line.
+billing_live() { sudo bash -c "set -a; . /etc/openvibe/billing.env; LIVE_DATABASE_URL=\$(sed -n 's/^DATABASE_URL=//p' /etc/openvibe/live.env); set +a; export NODE_ENV=production; cd /opt/openvibe.billing && sudo -E -u ubuntu $1"; }
```
```
Live owner calls are made from the browser console on openvibe.live, signed in as the owner:
Live owner calls are made from the browser console on openvibe.live, signed in as the owner:
@@ -148,7 +147,7 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
@@ -148,7 +147,7 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
2. **Let in-flight checkouts land** on Live: wait at least **60 minutes** after step 1 (minted PowerChat
2. **Let in-flight checkouts land** on Live: wait at least **60 minutes** after step 1 (minted PowerChat
checkout links live one hour), then run the backfill once: `api('/powerchat/reconcile', { method: 'POST' })`.
checkout links live one hour), then run the backfill once: `api('/powerchat/reconcile', { method: 'POST' })`.
Look at what is still open:
Look at what is still open:
- `sudo sqlite3 -readonly "$LIVE_DB" "SELECT id, provider, kind, amount_cents, status, created_at FROM payment_orders WHERE status IN ('pending','paid') AND created_at >= datetime('now','-3 days')"`.
+ `sudo -u postgres psql -d ov_live -c "SELECT id, provider, kind, amount_cents, status, created_at FROM payment_orders WHERE status IN ('pending','paid') AND created_at >= datetime('now','-3 days')"`.
A `paid` row is an anomaly to fix first; a `pending` one is carried into Billing as an intent and
A `paid` row is an anomaly to fix first; a `pending` one is carried into Billing as an intent and
settles there if it is ever paid (canonical, un-minted links do not expire).
settles there if it is ever paid (canonical, un-minted links do not expire).
3. **Back up and freeze Billing** so it holds PowerChat deliveries until the final import is in:
3. **Back up and freeze Billing** so it holds PowerChat deliveries until the final import is in:
@@ -164,14 +163,14 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
@@ -164,14 +163,14 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
`https://openvibe.live/api/powerchat/webhook` → **`https://billing.openvibe.network/webhooks/powerchat`**,
`https://openvibe.live/api/powerchat/webhook` → **`https://billing.openvibe.network/webhooks/powerchat`**,
same signing secret. From this moment every delivery is stored (202) and held by Billing; Live gets none.
same signing secret. From this moment every delivery is stored (202) and held by Billing; Live gets none.
`billing 'node scripts/freeze.js status'` shows `held_webhooks` growing only if tips arrive.
`billing 'node scripts/freeze.js status'` shows `held_webhooks` growing only if tips arrive.
-5. **Final snapshot and import** (after step 4, so every delivery either reached Live before the snapshot
- or is held by Billing):
+5. **Final import** (after step 4, so every delivery either reached Live before this read or is held by
+ Billing). Live's money writes are frozen since step 1, and the import reads every Live table in one
+ read-only, repeatable-read transaction, so it sees one moment of Live. Dump Live first and keep the dump:
```bash
```bash
- SNAP=$SNAPDIR/live-cutover-$(date +%F-%H%M).db
- sudo sqlite3 "$LIVE_DB" ".backup $SNAP" && sudo chown ubuntu:ubuntu "$SNAP" && sudo chmod 600 "$SNAP"
- sudo cp -p "$SNAP" /var/backups/openvibe/ # keep this copy (rollback)
- billing "node scripts/import-live.js --live-db $SNAP --dry-run"
- billing "node scripts/import-live.js --live-db $SNAP"
+ DUMP=/var/backups/openvibe/live-cutover-$(date +%F-%H%M).sql.gz
+ sudo -u postgres pg_dump -d ov_live | gzip | sudo tee "$DUMP" > /dev/null && sudo chmod 600 "$DUMP" # rollback copy
+ billing_live 'node scripts/import-live.js --dry-run'
+ billing_live 'node scripts/import-live.js'
billing 'node scripts/reconcile.js'
billing 'node scripts/reconcile.js'
```
```
Read the report: `holds` (must be empty for anyone with money), `adjustments`, `anomalies`,
Read the report: `holds` (must be empty for anyone with money), `adjustments`, `anomalies`,
@@ -185,9 +184,9 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
@@ -185,9 +184,9 @@ Live owner calls are made from the browser console on openvibe.live, signed in a
OV_BILLING_INTERNAL_URL=http://127.0.0.1:4600
OV_BILLING_INTERNAL_URL=http://127.0.0.1:4600
OV_BILLING_PUBLIC_URL=https://billing.openvibe.network
OV_BILLING_PUBLIC_URL=https://billing.openvibe.network
```
```
- then `sudo /opt/openvibe.live/deploy/scripts/deploy.sh --restart --wait-idle`. `--restart` is
- required: with no new commit, deploy.sh changes nothing unless told to restart, and only a restart
- reads the new env. The systemd socket keeps accepting while Live boots. Wait for `GET /api/ready`.
+ then restart Live so it reads the new env: check `curl -s https://openvibe.live/api/streams` for anyone live
+ over RTMP (a restart drops them), then `sudo systemctl restart openvibe-live.service`. The systemd socket keeps
+ accepting while Live boots. Wait for `GET /api/ready`.
`api('/admin/money')` → `authority: "billing"`, `billing.reachable: true`, `billing.rates.drift: []`.
`api('/admin/money')` → `authority: "billing"`, `billing.reachable: true`, `billing.rates.drift: []`.
Then let Tips post chat lines: set `TIPS_CHAT_ADAPTER=live-chat` in `/etc/openvibe/tips.env` and run
Then let Tips post chat lines: set `TIPS_CHAT_ADAPTER=live-chat` in `/etc/openvibe/tips.env` and run
`sudo systemctl restart openvibe-tips`. `curl -s http://127.0.0.1:4610/api/health` →
`sudo systemctl restart openvibe-tips`. `curl -s http://127.0.0.1:4610/api/health` →
@@ -221,7 +220,7 @@ exactly one of the two: Live before the snapshot, or Billing's held queue after
@@ -221,7 +220,7 @@ exactly one of the two: Live before the snapshot, or Billing's held queue after
## Rollback
## Rollback
-Keep the step-5 snapshot. The Billing side never needs undoing to roll Live back; the question is only
+Keep the step-5 dump. The Billing side never needs undoing to roll Live back; the question is only
which money moved where.
which money moved where.
- **Before step 6** (Live still on its own columns): re-point the PowerChat webhook back to
- **Before step 6** (Live still on its own columns): re-point the PowerChat webhook back to
@@ -1,14 +1,14 @@
@@ -1,14 +1,14 @@
# OpenVibe.Live → OpenVibe.Billing: what maps where
# OpenVibe.Live → OpenVibe.Billing: what maps where
Where every piece of Live's money state goes in Billing, as the importer
Where every piece of Live's money state goes in Billing, as the importer
-([server/importer/live.js](../server/importer/live.js), `node scripts/import-live.js --live-db <snapshot>`) and
+([server/importer/live.js](../server/importer/live.js), `node scripts/import-live.js` with `LIVE_DATABASE_URL`) and
the cutover ([live-cutover.md](live-cutover.md)) implement it. The audit found this was never written down,
the cutover ([live-cutover.md](live-cutover.md)) implement it. The audit found this was never written down,
in particular for **payouts, refunds and plans**. **Live has no table for any of those three.** They live
in particular for **payouts, refunds and plans**. **Live has no table for any of those three.** They live
inside other rows, and this page says which ones.
inside other rows, and this page says which ones.
-Counts are from production `/opt/openvibe.live/data/live.db`, read on 2026-09-23. The importer opens a
-**snapshot copy** read-only and never writes Live. Every write it makes is keyed, so a re-run over the same
-snapshot changes nothing.
+Counts are from production Live, read on 2026-09-23 (then its SQLite live.db; Live has run on PostgreSQL,
+`ov_live`, since 2026-10-08). The importer reads Live's database in one read-only, repeatable-read transaction and
+never writes Live. Every write it makes is keyed, so a re-run over the same data changes nothing.
## Balances: `users`
## Balances: `users`
@@ -9,7 +9,6 @@
@@ -9,7 +9,6 @@
"version": "0.1.0",
"version": "0.1.0",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
- "better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
@@ -90,57 +89,6 @@
@@ -90,57 +89,6 @@
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
"license": "MIT"
"license": "MIT"
},
},
- "node_modules/base64-js": {
- "version": "1.5.1",
- "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
- "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/better-sqlite3": {
- "version": "11.10.0",
- "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",
- "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",
- "hasInstallScript": true,
- "license": "MIT",
- "dependencies": {
- "bindings": "^1.5.0",
- "prebuild-install": "^7.1.1"
- }
- },
- "node_modules/bindings": {
- "version": "1.5.0",
- "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
- "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
- "license": "MIT",
- "dependencies": {
- "file-uri-to-path": "1.0.0"
- }
- },
- "node_modules/bl": {
- "version": "4.1.0",
- "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
- "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
- "license": "MIT",
- "dependencies": {
- "buffer": "^5.5.0",
- "inherits": "^2.0.4",
- "readable-stream": "^3.4.0"
- }
- },
"node_modules/body-parser": {
"node_modules/body-parser": {
"version": "1.20.8",
"version": "1.20.8",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.8.tgz",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.8.tgz",
@@ -165,30 +113,6 @@
@@ -165,30 +113,6 @@
"npm": "1.2.8000 || >= 1.4.16"
"npm": "1.2.8000 || >= 1.4.16"
}
}
},
},
- "node_modules/buffer": {
- "version": "5.7.1",
- "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
- "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT",
- "dependencies": {
- "base64-js": "^1.3.1",
- "ieee754": "^1.1.13"
- }
- },
"node_modules/buffer-equal-constant-time": {
"node_modules/buffer-equal-constant-time": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
@@ -233,12 +157,6 @@
@@ -233,12 +157,6 @@
"url": "https://github.com/sponsors/ljharb"
"url": "https://github.com/sponsors/ljharb"
}
}
},
},
- "node_modules/chownr": {
- "version": "1.1.4",
- "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
- "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
- "license": "ISC"
- },
"node_modules/cluster-key-slot": {
"node_modules/cluster-key-slot": {
"version": "1.1.2",
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
"resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
@@ -293,30 +211,6 @@
@@ -293,30 +211,6 @@
"ms": "2.0.0"
"ms": "2.0.0"
}
}
},
},
- "node_modules/decompress-response": {
- "version": "6.0.0",
- "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
- "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
- "license": "MIT",
- "dependencies": {
- "mimic-response": "^3.1.0"
- },
- "engines": {
- "node": ">=10"
- },
- "funding": {
- "url": "https://github.com/sponsors/sindresorhus"
- }
- },
- "node_modules/deep-extend": {
- "version": "0.6.0",
- "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
- "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
- "license": "MIT",
- "engines": {
- "node": ">=4.0.0"
- }
- },
"node_modules/denque": {
"node_modules/denque": {
"version": "2.1.0",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
@@ -345,15 +239,6 @@
@@ -345,15 +239,6 @@
"npm": "1.2.8000 || >= 1.4.16"
"npm": "1.2.8000 || >= 1.4.16"
}
}
},
},
- "node_modules/detect-libc": {
- "version": "2.1.2",
- "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
- "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=8"
- }
- },
"node_modules/dotenv": {
"node_modules/dotenv": {
"version": "16.6.1",
"version": "16.6.1",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
@@ -404,15 +289,6 @@
@@ -404,15 +289,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/end-of-stream": {
- "version": "1.4.5",
- "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
- "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
- "license": "MIT",
- "dependencies": {
- "once": "^1.4.0"
- }
- },
"node_modules/es-define-property": {
"node_modules/es-define-property": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
@@ -458,15 +334,6 @@
@@ -458,15 +334,6 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
- "node_modules/expand-template": {
- "version": "2.0.3",
- "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
- "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
- "license": "(MIT OR WTFPL)",
- "engines": {
- "node": ">=6"
- }
- },
"node_modules/express": {
"node_modules/express": {
"version": "4.22.3",
"version": "4.22.3",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.3.tgz",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.3.tgz",
@@ -535,12 +402,6 @@
@@ -535,12 +402,6 @@
],
],
"license": "BSD-3-Clause"
"license": "BSD-3-Clause"
},
},
- "node_modules/file-uri-to-path": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
- "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
- "license": "MIT"
- },
"node_modules/finalhandler": {
"node_modules/finalhandler": {
"version": "1.3.2",
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
@@ -577,12 +438,6 @@
@@ -577,12 +438,6 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
- "node_modules/fs-constants": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
- "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
- "license": "MIT"
- },
"node_modules/function-bind": {
"node_modules/function-bind": {
"version": "1.1.2",
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
@@ -629,12 +484,6 @@
@@ -629,12 +484,6 @@
"node": ">= 0.4"
"node": ">= 0.4"
}
}
},
},
- "node_modules/github-from-package": {
- "version": "0.0.0",
- "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
- "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
- "license": "MIT"
- },
"node_modules/gopd": {
"node_modules/gopd": {
"version": "1.2.0",
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
@@ -703,38 +552,12 @@
@@ -703,38 +552,12 @@
"node": ">=0.10.0"
"node": ">=0.10.0"
}
}
},
},
- "node_modules/ieee754": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
- "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "BSD-3-Clause"
- },
"node_modules/inherits": {
"node_modules/inherits": {
"version": "2.0.4",
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
"license": "ISC"
},
},
- "node_modules/ini": {
- "version": "1.3.8",
- "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
- "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
- "license": "ISC"
- },
"node_modules/iovalkey": {
"node_modules/iovalkey": {
"version": "0.4.0",
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
"resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
@@ -965,45 +788,12 @@
@@ -965,45 +788,12 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
- "node_modules/mimic-response": {
- "version": "3.1.0",
- "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
- "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
- "license": "MIT",
- "engines": {
- "node": ">=10"
- },
- "funding": {
- "url": "https://github.com/sponsors/sindresorhus"
- }
- },
- "node_modules/minimist": {
- "version": "1.2.8",
- "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
- "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
- "license": "MIT",
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/mkdirp-classic": {
- "version": "0.5.3",
- "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
- "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
- "license": "MIT"
- },
"node_modules/ms": {
"node_modules/ms": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
"license": "MIT"
},
},
- "node_modules/napi-build-utils": {
- "version": "2.0.0",
- "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
- "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
- "license": "MIT"
- },
"node_modules/negotiator": {
"node_modules/negotiator": {
"version": "0.6.3",
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
@@ -1013,18 +803,6 @@
@@ -1013,18 +803,6 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
- "node_modules/node-abi": {
- "version": "3.96.0",
- "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",
- "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",
- "license": "MIT",
- "dependencies": {
- "semver": "^7.3.5"
- },
- "engines": {
- "node": ">=10"
- }
- },
"node_modules/object-inspect": {
"node_modules/object-inspect": {
"version": "1.13.4",
"version": "1.13.4",
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz",
"resolved": "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz",
@@ -1049,15 +827,6 @@
@@ -1049,15 +827,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/once": {
- "version": "1.4.0",
- "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
- "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
- "license": "ISC",
- "dependencies": {
- "wrappy": "1"
- }
- },
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
"version": "0.129.0",
"version": "0.129.0",
"resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.129.0",
"resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.129.0",
@@ -1254,33 +1023,6 @@
@@ -1254,33 +1023,6 @@
"node": ">=0.10.0"
"node": ">=0.10.0"
}
}
},
},
- "node_modules/prebuild-install": {
- "version": "7.1.3",
- "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
- "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
- "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
- "license": "MIT",
- "dependencies": {
- "detect-libc": "^2.0.0",
- "expand-template": "^2.0.3",
- "github-from-package": "0.0.0",
- "minimist": "^1.2.3",
- "mkdirp-classic": "^0.5.3",
- "napi-build-utils": "^2.0.0",
- "node-abi": "^3.3.0",
- "pump": "^3.0.0",
- "rc": "^1.2.7",
- "simple-get": "^4.0.0",
- "tar-fs": "^2.0.0",
- "tunnel-agent": "^0.6.0"
- },
- "bin": {
- "prebuild-install": "bin.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
"node_modules/proxy-addr": {
"node_modules/proxy-addr": {
"version": "2.0.8",
"version": "2.0.8",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.8.tgz",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.8.tgz",
@@ -1298,16 +1040,6 @@
@@ -1298,16 +1040,6 @@
"url": "https://opencollective.com/express"
"url": "https://opencollective.com/express"
}
}
},
},
- "node_modules/pump": {
- "version": "3.0.4",
- "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
- "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
- "license": "MIT",
- "dependencies": {
- "end-of-stream": "^1.1.0",
- "once": "^1.3.1"
- }
- },
"node_modules/qs": {
"node_modules/qs": {
"version": "6.16.0",
"version": "6.16.0",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",
@@ -1348,35 +1080,6 @@
@@ -1348,35 +1080,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/rc": {
- "version": "1.2.8",
- "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
- "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
- "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
- "dependencies": {
- "deep-extend": "^0.6.0",
- "ini": "~1.3.0",
- "minimist": "^1.2.0",
- "strip-json-comments": "~2.0.1"
- },
- "bin": {
- "rc": "cli.js"
- }
- },
- "node_modules/readable-stream": {
- "version": "3.6.2",
- "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
- "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
- "license": "MIT",
- "dependencies": {
- "inherits": "^2.0.3",
- "string_decoder": "^1.1.1",
- "util-deprecate": "^1.0.1"
- },
- "engines": {
- "node": ">= 6"
- }
- },
"node_modules/redis-errors": {
"node_modules/redis-errors": {
"version": "1.2.0",
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
"resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
@@ -1568,51 +1271,6 @@
@@ -1568,51 +1271,6 @@
"url": "https://github.com/sponsors/ljharb"
"url": "https://github.com/sponsors/ljharb"
}
}
},
},
- "node_modules/simple-concat": {
- "version": "1.0.1",
- "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
- "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/simple-get": {
- "version": "4.0.1",
- "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
- "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT",
- "dependencies": {
- "decompress-response": "^6.0.0",
- "once": "^1.3.1",
- "simple-concat": "^1.0.0"
- }
- },
"node_modules/split2": {
"node_modules/split2": {
"version": "4.2.0",
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
@@ -1637,52 +1295,6 @@
@@ -1637,52 +1295,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/string_decoder": {
- "version": "1.3.0",
- "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
- "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
- "license": "MIT",
- "dependencies": {
- "safe-buffer": "~5.2.0"
- }
- },
- "node_modules/strip-json-comments": {
- "version": "2.0.1",
- "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
- "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
- "license": "MIT",
- "engines": {
- "node": ">=0.10.0"
- }
- },
- "node_modules/tar-fs": {
- "version": "2.1.5",
- "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",
- "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",
- "license": "MIT",
- "dependencies": {
- "chownr": "^1.1.1",
- "mkdirp-classic": "^0.5.2",
- "pump": "^3.0.0",
- "tar-stream": "^2.1.4"
- }
- },
- "node_modules/tar-stream": {
- "version": "2.2.0",
- "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
- "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
- "license": "MIT",
- "dependencies": {
- "bl": "^4.0.3",
- "end-of-stream": "^1.4.1",
- "fs-constants": "^1.0.0",
- "inherits": "^2.0.3",
- "readable-stream": "^3.1.1"
- },
- "engines": {
- "node": ">=6"
- }
- },
"node_modules/toidentifier": {
"node_modules/toidentifier": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
@@ -1692,18 +1304,6 @@
@@ -1692,18 +1304,6 @@
"node": ">=0.6"
"node": ">=0.6"
}
}
},
},
- "node_modules/tunnel-agent": {
- "version": "0.6.0",
- "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
- "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
- "license": "Apache-2.0",
- "dependencies": {
- "safe-buffer": "^5.0.1"
- },
- "engines": {
- "node": "*"
- }
- },
"node_modules/type-is": {
"node_modules/type-is": {
"version": "1.6.18",
"version": "1.6.18",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
@@ -1726,12 +1326,6 @@
@@ -1726,12 +1326,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/util-deprecate": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
- "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
- "license": "MIT"
- },
"node_modules/utils-merge": {
"node_modules/utils-merge": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
@@ -1750,12 +1344,6 @@
@@ -1750,12 +1344,6 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
- "node_modules/wrappy": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
- "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
- "license": "ISC"
- },
"node_modules/xtend": {
"node_modules/xtend": {
"version": "4.0.2",
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
@@ -23,7 +23,6 @@
@@ -23,7 +23,6 @@
"author": "OpenVibers",
"author": "OpenVibers",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
- "better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
@@ -3,35 +3,52 @@
@@ -3,35 +3,52 @@
/**
/**
* Import OpenVibe.Live's money state into Billing (ADR-012 migration).
* Import OpenVibe.Live's money state into Billing (ADR-012 migration).
*
*
- * node scripts/import-live.js --live-db /path/to/live-snapshot.db [--dry-run] [--json]
- * node scripts/import-live.js --live-db /path/to/live-snapshot.db --accounts-only [--dry-run] [--json]
+ * LIVE_DATABASE_URL=… node scripts/import-live.js [--dry-run] [--json]
+ * LIVE_DATABASE_URL=… node scripts/import-live.js --accounts-only [--dry-run] [--json]
+ * (or --live-env <Live's env file> where this user can read it; docs/live-cutover.md has the billing_live helper)
*
*
* --accounts-only refreshes only provider_accounts from Live's powerchat_connections (who an
* --accounts-only refreshes only provider_accounts from Live's powerchat_connections (who an
* EXTERNAL PowerChat tip belongs to); no money, no reconciliation. Safe to run daily after cutover.
* EXTERNAL PowerChat tip belongs to); no money, no reconciliation. Safe to run daily after cutover.
*
*
- * --live-db must be a COPY of live.db (e.g. `sqlite3 live.db ".backup live-snapshot.db"`); it is
- * opened read-only. Live user ids are resolved to Network subjects through resolve-batch with this
+ * Live's database is OpenVibe.Live's PostgreSQL (`ov_live`). Its URL is LIVE_DATABASE_URL, or the DATABASE_URL line
+ * of the env file --live-env names (Live's own, /etc/openvibe/live.env); it is never printed and never passed on a
+ * command line. Every read runs in one REPEATABLE READ, READ ONLY transaction (server/importer/live-source.js), so
+ * the import sees one moment of Live and cannot write there. Live user ids are resolved to Network subjects through resolve-batch with this
* service's client credentials (OV_OAUTH_CLIENT_ID / OV_OAUTH_CLIENT_SECRET, capability
* service's client credentials (OV_OAUTH_CLIENT_ID / OV_OAUTH_CLIENT_SECRET, capability
* identity.subject.resolve). The report is printed and stored in import_runs; a reconciliation run
* identity.subject.resolve). The report is printed and stored in import_runs; a reconciliation run
* follows every real import. Safe to re-run: a second run over the same snapshot changes nothing.
* follows every real import. Safe to re-run: a second run over the same snapshot changes nothing.
*/
*/
-const Database = require('better-sqlite3');
+const fs = require('fs');
const { loadConfig } = require('../server/config');
const { loadConfig } = require('../server/config');
const { openDb } = require('../server/db');
const { openDb } = require('../server/db');
const { createRates } = require('../server/rates');
const { createRates } = require('../server/rates');
const { createIdentity } = require('../server/network');
const { createIdentity } = require('../server/network');
const { importLive, importProviderAccounts } = require('../server/importer/live');
const { importLive, importProviderAccounts } = require('../server/importer/live');
+const { openLiveSource } = require('../server/importer/live-source');
const { reconcile } = require('../server/reconcile');
const { reconcile } = require('../server/reconcile');
const args = process.argv.slice(2);
const args = process.argv.slice(2);
const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };
const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };
const flag = (name) => args.includes(`--${name}`);
const flag = (name) => args.includes(`--${name}`);
+/** Live's database URL: LIVE_DATABASE_URL, else the DATABASE_URL line of the --live-env file. Never printed. */
+function liveUrl() {
+ if (process.env.LIVE_DATABASE_URL) return process.env.LIVE_DATABASE_URL;
+ const file = opt('live-env');
+ if (!file) return null;
+ for (const line of fs.readFileSync(file, 'utf8').split('\n')) {
+ const m = /^\s*DATABASE_URL\s*=\s*(.*)$/.exec(line);
+ if (m) return m[1].trim().replace(/^(['"])(.*)\1$/, '$2') || null;
+ }
+ return null;
+}
+
async function main() {
async function main() {
- const livePath = opt('live-db');
- if (!livePath) { console.error('usage: import-live.js --live-db <snapshot path> [--dry-run] [--json]'); process.exit(2); }
+ if (args.includes('--live-db')) { console.error('--live-db is gone: Live runs on PostgreSQL; use --live-env /etc/openvibe/live.env (or LIVE_DATABASE_URL)'); process.exit(2); }
+ const url = liveUrl();
+ if (!url) { console.error('usage: import-live.js --live-env <Live env file> [--dry-run] [--json] [--accounts-only] (or set LIVE_DATABASE_URL)'); process.exit(2); }
const config = loadConfig();
const config = loadConfig();
- const live = new Database(livePath, { readonly: true, fileMustExist: true });
+ const live = await openLiveSource({ url });
const db = await openDb(config);
const db = await openDb(config);
const ctx = { db, config, rates: createRates(config.rates), now: () => Date.now(), log: console };
const ctx = { db, config, rates: createRates(config.rates), now: () => Date.now(), log: console };
const identity = createIdentity(config);
const identity = createIdentity(config);
@@ -43,7 +60,7 @@ async function main() {
@@ -43,7 +60,7 @@ async function main() {
console.log(`provider accounts ${r.run_id}${r.dry_run ? ' (DRY RUN — nothing kept)' : ''}: ${a.mapped} mapped, ${a.unchanged} unchanged, ${a.kept_admin.length} kept (set by an operator)`);
console.log(`provider accounts ${r.run_id}${r.dry_run ? ' (DRY RUN — nothing kept)' : ''}: ${a.mapped} mapped, ${a.unchanged} unchanged, ${a.kept_admin.length} kept (set by an operator)`);
for (const u of a.unmapped) console.log(` unmapped: live user ${u.live_user_id} (${u.username}): ${u.reason}`);
for (const u of a.unmapped) console.log(` unmapped: live user ${u.live_user_id} (${u.username}): ${u.reason}`);
}
}
- live.close();
+ await live.close();
db.close();
db.close();
return;
return;
}
}
@@ -52,7 +69,7 @@ async function main() {
@@ -52,7 +69,7 @@ async function main() {
if (flag('json')) console.log(JSON.stringify({ import: report, reconciliation: rec }, null, 2));
if (flag('json')) console.log(JSON.stringify({ import: report, reconciliation: rec }, null, 2));
else {
else {
console.log(`import ${report.run_id}${report.dry_run ? ' (DRY RUN — nothing kept)' : ''}`);
console.log(`import ${report.run_id}${report.dry_run ? ' (DRY RUN — nothing kept)' : ''}`);
- console.log(` live snapshot: ${JSON.stringify(report.counts.live)}; identities ${JSON.stringify(report.counts.identities)}`);
+ console.log(` live database: ${JSON.stringify(report.counts.live)}; identities ${JSON.stringify(report.counts.identities)}`);
console.log(` new: ${report.new_transactions} transactions, ${report.counts.intents_new} intents, ${report.counts.receipts_new} receipts; subscriptions ${report.subscriptions.imported} imported / ${report.subscriptions.updated} updated`);
console.log(` new: ${report.new_transactions} transactions, ${report.counts.intents_new} intents, ${report.counts.receipts_new} receipts; subscriptions ${report.subscriptions.imported} imported / ${report.subscriptions.updated} updated`);
console.log(` opening adjustments (${report.adjustments.length}):`);
console.log(` opening adjustments (${report.adjustments.length}):`);
for (const a of report.adjustments) console.log(` live user ${a.live_user_id} (${a.username}) ${a.account}: history ${a.replayed_history} → column ${a.live_column} (adjust ${a.adjustment > 0 ? '+' : ''}${a.adjustment})`);
for (const a of report.adjustments) console.log(` live user ${a.live_user_id} (${a.username}) ${a.account}: history ${a.replayed_history} → column ${a.live_column} (adjust ${a.adjustment > 0 ? '+' : ''}${a.adjustment})`);
@@ -64,7 +81,7 @@ async function main() {
@@ -64,7 +81,7 @@ async function main() {
if (pa) console.log(` PowerChat accounts: ${pa.mapped} mapped, ${pa.unchanged} unchanged, ${pa.unmapped.length} unmapped, ${pa.kept_admin.length} kept (operator)`);
if (pa) console.log(` PowerChat accounts: ${pa.mapped} mapped, ${pa.unchanged} unchanged, ${pa.unmapped.length} unmapped, ${pa.kept_admin.length} kept (operator)`);
if (rec) console.log(` reconciliation ${rec.id}: ${rec.ok ? 'OK' : 'FAILED'} (${rec.checks.filter((c) => !c.ok).map((c) => c.id).join(', ') || 'all checks pass'})`);
if (rec) console.log(` reconciliation ${rec.id}: ${rec.ok ? 'OK' : 'FAILED'} (${rec.checks.filter((c) => !c.ok).map((c) => c.id).join(', ') || 'all checks pass'})`);
}
}
- live.close();
+ await live.close();
db.close();
db.close();
if (rec && !rec.ok) process.exit(1);
if (rec && !rec.ok) process.exit(1);
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,63 @@
@@ -0,0 +1,63 @@
+'use strict';
+
+/**
+ * The OpenVibe.Live database the importer reads (./live.js): Live's PostgreSQL database (`ov_live`, since
+ * 2026-10-08; the SQLite live.db is gone).
+ *
+ * openLiveSource({ url }) a pg connection to Live's database. Every read runs in ONE transaction,
+ * REPEATABLE READ and READ ONLY, so all the tables come from the same moment
+ * (at cutover Live's money writes are frozen first, so that moment is the
+ * final state) and nothing can be written through it. close() ends it.
+ * liveSourceFromQuery(query) the same interface over any (text, params) → rows function (the tests'
+ * PGlite).
+ *
+ * The interface is the one the importer was written against: live.prepare(sql).all(...params) and .get(...),
+ * with `?` placeholders. Values arrive as openvibe-sdk/db hands them to Live itself (its PARSERS): ids and other
+ * int8 columns as numbers, timestamps as ISO strings. Live's money columns are double precision and its
+ * dates are text, so they come through unchanged.
+ */
+const { PARSERS } = require('openvibe-sdk/db');
+
+/** `?` placeholders → `$1, $2, …` (the importer's Live queries put no `?` inside a string literal). */
+function toPg(sql) {
+ let n = 0;
+ return String(sql).replace(/\?/g, () => `$${++n}`);
+}
+
+function liveSourceFromQuery(query, close = async () => {}) {
+ return {
+ prepare(sql) {
+ const text = toPg(sql);
+ return {
+ all: async (...params) => await query(text, params),
+ get: async (...params) => (await query(text, params))[0],
+ };
+ },
+ close,
+ };
+}
+
+async function openLiveSource({ url, applicationName = 'billing-import-live' }) {
+ if (!url) throw new Error('no Live database URL');
+ const pg = require('pg');
+ const client = new pg.Client({
+ connectionString: url,
+ application_name: applicationName,
+ types: { getTypeParser: (oid, format) => (format === 'text' && PARSERS[oid]) || pg.types.getTypeParser(oid, format) },
+ });
+ await client.connect();
+ try {
+ await client.query('BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY');
+ } catch (err) {
+ await client.end().catch(() => {});
+ throw err;
+ }
+ let closed = false;
+ return liveSourceFromQuery(async (text, params) => (await client.query(text, params)).rows, async () => {
+ if (closed) return;
+ closed = true;
+ try { await client.query('ROLLBACK'); } finally { await client.end(); }
+ });
+}
+
+module.exports = { openLiveSource, liveSourceFromQuery, toPg };
@@ -1,7 +1,8 @@
@@ -1,7 +1,8 @@
'use strict';
'use strict';
/**
/**
- * Import from a SNAPSHOT COPY of OpenVibe.Live's live.db (opened read-only; never the live file).
+ * Import from OpenVibe.Live's PostgreSQL database (`ov_live`), read in one REPEATABLE READ, READ ONLY
+ * transaction (./live-source.js), so every table comes from the same moment and nothing is written there.
*
*
* 1. identities every Live user id that holds a balance or appears in a money row is resolved to a
* 1. identities every Live user id that holds a balance or appears in a money row is resolved to a
* Network subject (resolve-batch). Unmapped users are never dropped: their accounts
* Network subject (resolve-batch). Unmapped users are never dropped: their accounts
@@ -42,14 +43,20 @@ const ROUTE_MARKER = /^(direct|site)(:|$)/;
@@ -42,14 +43,20 @@ const ROUTE_MARKER = /^(direct|site)(:|$)/;
const liveTs = (s) => {
const liveTs = (s) => {
if (!s) return null;
if (!s) return null;
+ if (s instanceof Date) return Number.isFinite(s.getTime()) ? s.toISOString() : null;
const str = String(s);
const str = String(s);
const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);
const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);
return Number.isFinite(t) ? new Date(t).toISOString() : null;
return Number.isFinite(t) ? new Date(t).toISOString() : null;
};
};
-// Live's own database, still SQLite (read-only here): its catalog.
-function tableExists(live, name) { return !!live.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name); }
-function columns(live, table) { return tableExists(live, table) ? live.prepare(`PRAGMA table_info(${table})`).all().map((c) => c.name) : []; }
+// Live's own database (PostgreSQL, read-only here): its catalog, in the schema the connection uses.
+async function tableExists(live, name) {
+ return !!await live.prepare('SELECT 1 FROM information_schema.tables WHERE table_schema = current_schema() AND table_name = ?').get(name);
+}
+async function columns(live, table) {
+ const rows = await live.prepare('SELECT column_name FROM information_schema.columns WHERE table_schema = current_schema() AND table_name = ? ORDER BY ordinal_position').all(table);
+ return rows.map((c) => c.column_name);
+}
class DryRun extends Error {}
class DryRun extends Error {}
@@ -84,12 +91,12 @@ async function applyProviderAccounts(ctx, rows, report) {
@@ -84,12 +91,12 @@ async function applyProviderAccounts(ctx, rows, report) {
return out;
return out;
}
}
-/** Step 6 alone: refresh provider_accounts from a Live snapshot (no money is touched). */
+/** Step 6 alone: refresh provider_accounts from Live's database (no money is touched). */
async function importProviderAccounts(ctx, { live, resolveLiveUsers, dryRun = false, log = console }) {
async function importProviderAccounts(ctx, { live, resolveLiveUsers, dryRun = false, log = console }) {
const { db } = ctx;
const { db } = ctx;
const started = ctx.now();
const started = ctx.now();
const runId = prefixedId('imp', started);
const runId = prefixedId('imp', started);
- const report = { run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live snapshot (provider accounts only)' };
+ const report = { run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live database (provider accounts only)' };
const rows = await readProviderAccounts(live, resolveLiveUsers);
const rows = await readProviderAccounts(live, resolveLiveUsers);
try {
try {
await money(db, async () => {
await money(db, async () => {
@@ -110,12 +117,12 @@ async function importLive(ctx, { live, resolveLiveUsers, dryRun = false, log = c
@@ -110,12 +117,12 @@ async function importLive(ctx, { live, resolveLiveUsers, dryRun = false, log = c
const started = ctx.now();
const started = ctx.now();
const runId = prefixedId('imp', started);
const runId = prefixedId('imp', started);
const report = {
const report = {
- run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live snapshot',
+ run_id: runId, dry_run: !!dryRun, started_at: iso(started), source: 'openvibe-live database',
counts: {}, holds: [], duplicate_provider_refs: [], route_markers: 0, adjustments: [], unreplayable: [],
counts: {}, holds: [], duplicate_provider_refs: [], route_markers: 0, adjustments: [], unreplayable: [],
anomalies: [], subscriptions: { imported: 0, updated: 0, skipped: [] }, new_transactions: 0,
anomalies: [], subscriptions: { imported: 0, updated: 0, skipped: [] }, new_transactions: 0,
};
};
- // ── Read the snapshot ────────────────────────────────────
+ // ── Read Live (one read-only snapshot) ───────────────────
const userCols = await columns(live, 'users');
const userCols = await columns(live, 'users');
const hasCashout = userCols.includes('openvibe_bucks_cashout_balance');
const hasCashout = userCols.includes('openvibe_bucks_cashout_balance');
const users = await live.prepare(`SELECT id, username, COALESCE(openvibe_bucks_balance, 0) AS credit${hasCashout ? ', COALESCE(openvibe_bucks_cashout_balance, 0) AS payable' : ', 0 AS payable'} FROM users`).all();
const users = await live.prepare(`SELECT id, username, COALESCE(openvibe_bucks_balance, 0) AS credit${hasCashout ? ', COALESCE(openvibe_bucks_cashout_balance, 0) AS payable' : ', 0 AS payable'} FROM users`).all();
@@ -9,36 +9,25 @@
@@ -9,36 +9,25 @@
* import run never books an opening-balance adjustment against what Billing settled itself.
* import run never books an opening-balance adjustment against what Billing settled itself.
*/
*/
const assert = require('assert');
const assert = require('assert');
-const path = require('path');
-const Database = require('better-sqlite3');
+const { fakeLive } = require('./helpers/live-pg');
const { boot, check, done } = require('./helpers/app');
const { boot, check, done } = require('./helpers/app');
-async function fabricateLive(file) {
- const live = new Database(file);
+async function fabricateLive() {
+ const live = await fakeLive();
const recent = new Date(Date.now() - 30 * 60_000).toISOString().replace('T', ' ').slice(0, 19);
const recent = new Date(Date.now() - 30 * 60_000).toISOString().replace('T', ' ').slice(0, 19);
- live.exec(`
- CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, openvibe_bucks_balance REAL DEFAULT 0.00,
- openvibe_bucks_cashout_balance REAL DEFAULT 0.00);
- CREATE TABLE transactions (id INTEGER PRIMARY KEY AUTOINCREMENT, from_user_id INTEGER, to_user_id INTEGER, stream_id INTEGER,
- amount INTEGER NOT NULL, type TEXT NOT NULL, status TEXT DEFAULT 'completed', message TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP);
- CREATE TABLE payment_orders (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, provider TEXT NOT NULL, provider_ref TEXT,
- kind TEXT NOT NULL DEFAULT 'bucks', amount_cents INTEGER NOT NULL DEFAULT 0, currency TEXT DEFAULT 'usd', bucks INTEGER DEFAULT 0,
- streamer_id INTEGER, status TEXT DEFAULT 'pending', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);
- CREATE TABLE subscriptions (id INTEGER PRIMARY KEY AUTOINCREMENT, subscriber_id INTEGER NOT NULL, streamer_id INTEGER NOT NULL, tier INTEGER DEFAULT 1,
- provider TEXT, provider_ref TEXT, price_cents INTEGER DEFAULT 0, status TEXT DEFAULT 'active', auto_renew INTEGER DEFAULT 0,
- current_period_end DATETIME, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);
+ await live.exec(`
INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance) VALUES (1, 'alice', 500, 0), (2, 'bob', 0, 349), (3, 'carol', 0, 0);
INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance) VALUES (1, 'alice', 500, 0), (2, 'bob', 0, 349), (3, 'carol', 0, 0);
INSERT INTO transactions (id, from_user_id, to_user_id, amount, type, status, created_at) VALUES (1, NULL, 1, 500, 'purchase', 'completed', '${recent}');
INSERT INTO transactions (id, from_user_id, to_user_id, amount, type, status, created_at) VALUES (1, NULL, 1, 500, 'purchase', 'completed', '${recent}');
`);
`);
- await live.prepare(`INSERT INTO payment_orders (id, user_id, provider, provider_ref, kind, amount_cents, bucks, streamer_id, status, created_at) VALUES
- (1, 1, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'credited', ?),
- (2, 3, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'pending', ?),
- (3, 1, 'powerchat', 'site:fee=50', 'subscription', 549, 0, 2, 'credited', ?),
- (4, 3, 'powerchat', NULL, 'bucks', 1300, 1000, NULL, 'pending', ?)`).run(recent, recent, recent, recent);
+ await live.query(`INSERT INTO payment_orders (id, user_id, provider, provider_ref, kind, amount_cents, bucks, streamer_id, status, created_at) VALUES
+ (1, 1, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'credited', $1),
+ (2, 3, 'powerchat', NULL, 'bucks', 700, 500, NULL, 'pending', $1),
+ (3, 1, 'powerchat', 'site:fee=50', 'subscription', 549, 0, 2, 'credited', $1),
+ (4, 3, 'powerchat', NULL, 'bucks', 1300, 1000, NULL, 'pending', $1)`, [recent]);
const end = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);
const end = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);
- await live.prepare(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, current_period_end, created_at)
- VALUES (1, 1, 2, 'powerchat', 549, 'active', ?, ?)`).run(end, recent);
- live.close();
+ await live.query(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, current_period_end, created_at)
+ VALUES (1, 1, 2, 'powerchat', 549, 'active', $1, $2)`, [end, recent]);
+ return live;
}
}
(async () => {
(async () => {
@@ -46,14 +35,12 @@ async function fabricateLive(file) {
@@ -46,14 +35,12 @@ async function fabricateLive(file) {
const alice = t.network.addUser(1);
const alice = t.network.addUser(1);
const bob = t.network.addUser(2);
const bob = t.network.addUser(2);
const carol = t.network.addUser(3);
const carol = t.network.addUser(3);
- const file = path.join(t.dir, 'live-snapshot.db');
- await fabricateLive(file);
+ const fake = await fabricateLive();
const { importLive } = require('../server/importer/live');
const { importLive } = require('../server/importer/live');
const { createIdentity } = require('../server/network');
const { createIdentity } = require('../server/network');
const identity = createIdentity(t.config);
const identity = createIdentity(t.config);
const run = async () => {
const run = async () => {
- const live = new Database(file, { readonly: true, fileMustExist: true });
- try { return await importLive(t.ctx, { live, resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} } }); } finally { live.close(); }
+ return await importLive(t.ctx, { live: fake.source(), resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} } });
};
};
const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };
const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };
const intent = async (order) => await t.db.prepare('SELECT * FROM payment_intents WHERE legacy_order_id = ?').get(order);
const intent = async (order) => await t.db.prepare('SELECT * FROM payment_intents WHERE legacy_order_id = ?').get(order);
@@ -96,11 +83,9 @@ async function fabricateLive(file) {
@@ -96,11 +83,9 @@ async function fabricateLive(file) {
await check('an order Live credited AFTER the shadow import is settled-in-Live after the final import', async () => {
await check('an order Live credited AFTER the shadow import is settled-in-Live after the final import', async () => {
assert.strictEqual((await intent(4)).status, 'created');
assert.strictEqual((await intent(4)).status, 'created');
- const live = new Database(file);
- live.exec(`UPDATE payment_orders SET status = 'credited' WHERE id = 4;
+ await fake.exec(`UPDATE payment_orders SET status = 'credited' WHERE id = 4;
UPDATE users SET openvibe_bucks_balance = 1000 WHERE id = 3;
UPDATE users SET openvibe_bucks_balance = 1000 WHERE id = 3;
- INSERT INTO transactions (from_user_id, to_user_id, amount, type, status) VALUES (NULL, 3, 1000, 'purchase', 'completed');`);
- live.close();
+ INSERT INTO transactions (id, from_user_id, to_user_id, amount, type, status) VALUES (2, NULL, 3, 1000, 'purchase', 'completed');`);
const report = await run();
const report = await run();
assert.ok(report.counts.intents_updated >= 1, JSON.stringify(report.counts));
assert.ok(report.counts.intents_updated >= 1, JSON.stringify(report.counts));
assert.strictEqual((await intent(4)).status, 'settled');
assert.strictEqual((await intent(4)).status, 'settled');
@@ -133,9 +118,7 @@ async function fabricateLive(file) {
@@ -133,9 +118,7 @@ async function fabricateLive(file) {
assert.strictEqual((await cli('on', 'Live cutover')).frozen, true);
assert.strictEqual((await cli('on', 'Live cutover')).frozen, true);
const w = await t.call('POST', '/api/v1/transfers', { body: { from: { type: 'user', id: carol }, to: { type: 'user', id: bob }, amount: 5 } });
const w = await t.call('POST', '/api/v1/transfers', { body: { from: { type: 'user', id: carol }, to: { type: 'user', id: bob }, amount: 5 } });
assert.strictEqual(w.status, 503); assert.strictEqual(w.json.code, 'billing.frozen');
assert.strictEqual(w.status, 503); assert.strictEqual(w.json.code, 'billing.frozen');
- const live = new Database(file);
- live.exec("INSERT INTO payment_orders (id, user_id, provider, kind, amount_cents, bucks, status) VALUES (5, 3, 'powerchat', 'bucks', 700, 500, 'pending')");
- live.close();
+ await fake.exec("INSERT INTO payment_orders (id, user_id, provider, kind, amount_cents, bucks, status) VALUES (5, 3, 'powerchat', 'bucks', 700, 500, 'pending')");
await run(); // the final import, while Billing is frozen
await run(); // the final import, while Billing is frozen
const held = await tip('pcorder:5', 700);
const held = await tip('pcorder:5', 700);
assert.strictEqual(held.status, 202, JSON.stringify(held.json));
assert.strictEqual(held.status, 202, JSON.stringify(held.json));
@@ -7,8 +7,7 @@
@@ -7,8 +7,7 @@
* never celebrated twice. Accounts come from Live's powerchat_connections (import) or an operator.
* never celebrated twice. Accounts come from Live's powerchat_connections (import) or an operator.
*/
*/
const assert = require('assert');
const assert = require('assert');
-const path = require('path');
-const Database = require('better-sqlite3');
+const { fakeLive } = require('./helpers/live-pg');
const { validate } = require('openvibe-contracts');
const { validate } = require('openvibe-contracts');
const { boot, check, done } = require('./helpers/app');
const { boot, check, done } = require('./helpers/app');
const { startEvents } = require('./helpers/stubs');
const { startEvents } = require('./helpers/stubs');
@@ -166,18 +165,16 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS event
@@ -166,18 +165,16 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS event
});
});
await check('the importer maps Live\'s powerchat_connections (never over an operator\'s mapping); --accounts-only alone', async () => {
await check('the importer maps Live\'s powerchat_connections (never over an operator\'s mapping); --accounts-only alone', async () => {
- const file = path.join(t.dir, 'live-accounts.db');
- const live = new Database(file);
- live.exec(`CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT, openvibe_bucks_balance REAL DEFAULT 0, openvibe_bucks_cashout_balance REAL DEFAULT 0);
- CREATE TABLE powerchat_connections (user_id INTEGER PRIMARY KEY, powerchat_username TEXT, powerchat_user_id TEXT);
- INSERT INTO users (id, username) VALUES (50, 'newbie'), (51, 'ghost'), (52, 'clash');
+ // A Live database with users and PowerChat connections only (the other money tables absent).
+ const fake = await fakeLive({ tables: `CREATE TABLE users (id bigint PRIMARY KEY, username text, openvibe_bucks_balance double precision DEFAULT 0, openvibe_bucks_cashout_balance double precision DEFAULT 0);
+ CREATE TABLE powerchat_connections (user_id bigint PRIMARY KEY, powerchat_username text, powerchat_user_id text);` });
+ await fake.exec(`INSERT INTO users (id, username) VALUES (50, 'newbie'), (51, 'ghost'), (52, 'clash');
INSERT INTO powerchat_connections VALUES (50, 'NewbiePC', 'pc-50'), (51, 'GhostPC', 'pc-51'), (52, 'StreamerPC', 'pc-77'), (53, 'bad name!', NULL);`);
INSERT INTO powerchat_connections VALUES (50, 'NewbiePC', 'pc-50'), (51, 'GhostPC', 'pc-51'), (52, 'StreamerPC', 'pc-77'), (53, 'bad name!', NULL);`);
- live.close();
const newbie = t.network.addUser(50);
const newbie = t.network.addUser(50);
t.network.addUser(52);
t.network.addUser(52);
t.network.addUser(53);
t.network.addUser(53);
const { importProviderAccounts, importLive } = require('../server/importer/live');
const { importProviderAccounts, importLive } = require('../server/importer/live');
- const ro = new Database(file, { readonly: true });
+ const ro = fake.source();
const dry = await importProviderAccounts(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, dryRun: true, log: {} });
const dry = await importProviderAccounts(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, dryRun: true, log: {} });
assert.strictEqual(dry.provider_accounts.mapped, 1);
assert.strictEqual(dry.provider_accounts.mapped, 1);
assert.strictEqual((await t.db.prepare("SELECT COUNT(*) AS n FROM provider_accounts WHERE username = 'newbiepc'").get()).n, 0, 'dry run keeps nothing');
assert.strictEqual((await t.db.prepare("SELECT COUNT(*) AS n FROM provider_accounts WHERE username = 'newbiepc'").get()).n, 0, 'dry run keeps nothing');
@@ -191,7 +188,8 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS event
@@ -191,7 +188,8 @@ const externalEvents = async (db) => (await db.prepare("SELECT envelope AS event
assert.strictEqual((await t.db.prepare("SELECT subject FROM provider_accounts WHERE username = 'streamerpc'").get()).subject, streamer.id);
assert.strictEqual((await t.db.prepare("SELECT subject FROM provider_accounts WHERE username = 'streamerpc'").get()).subject, streamer.id);
const full = await importLive(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, log: {} });
const full = await importLive(t.ctx, { live: ro, resolveLiveUsers: t.ctx.network.resolveLiveUsers, log: {} });
assert.strictEqual(full.provider_accounts.unchanged, 1, 'the full import runs the same step; a re-run changes nothing');
assert.strictEqual(full.provider_accounts.unchanged, 1, 'the full import runs the same step; a re-run changes nothing');
- ro.close();
+ await ro.close();
+ await fake.close();
const r = await t.powerchat(donation({ eventId: 'ext-newbie', amountUsdCents: 100 }, { id: 'pc-50', username: 'NewbiePC' }));
const r = await t.powerchat(donation({ eventId: 'ext-newbie', amountUsdCents: 100 }, { id: 'pc-50', username: 'NewbiePC' }));
assert.strictEqual(r.json.result.streamer, newbie);
assert.strictEqual(r.json.result.streamer, newbie);
await t.assertReconciled('after the accounts import');
await t.assertReconciled('after the accounts import');
new file mode 100644
new file mode 100644
@@ -0,0 +1,49 @@
@@ -0,0 +1,49 @@
+'use strict';
+
+/**
+ * A fabricated OpenVibe.Live database for the importer tests: PGlite (PostgreSQL, as Live's ov_live is) with Live's
+ * money tables in the column types its migrations give them (bigint ids and amounts, double precision balances,
+ * text dates). Values come back through openvibe-sdk/db's PARSERS, as they do from the real database.
+ *
+ * const live = await fakeLive(); // the tables, empty
+ * await live.exec('INSERT …'); // set it up, or change it between two imports
+ * importLive(ctx, { live: live.source(), … })
+ */
+const { PGlite } = require('@electric-sql/pglite');
+const { PARSERS } = require('openvibe-sdk/db');
+const { liveSourceFromQuery } = require('../../server/importer/live-source');
+
+// The columns the importer reads, as OpenVibe.Live's migrations/0002_live.sql creates them.
+const LIVE_MONEY_TABLES = `
+ CREATE TABLE users (id bigint PRIMARY KEY, username text UNIQUE NOT NULL, openvibe_bucks_balance double precision DEFAULT 0.00,
+ openvibe_coins_balance bigint DEFAULT 0, openvibe_bucks_cashout_balance double precision DEFAULT 0.00);
+ CREATE TABLE transactions (id bigint PRIMARY KEY, from_user_id bigint, to_user_id bigint, stream_id bigint,
+ amount bigint NOT NULL, type text NOT NULL CHECK(type IN ('donation', 'purchase', 'subscription', 'cashout', 'refund', 'bonus')),
+ status text DEFAULT 'completed' CHECK(status IN ('pending', 'completed', 'failed', 'escrow', 'refunded')), message text,
+ paypal_transaction_id text, created_at text);
+ CREATE TABLE payment_orders (id bigint PRIMARY KEY, user_id bigint NOT NULL, provider text NOT NULL, provider_ref text,
+ kind text NOT NULL DEFAULT 'bucks', amount_cents bigint NOT NULL DEFAULT 0, currency text DEFAULT 'usd', bucks bigint DEFAULT 0,
+ streamer_id bigint, status text DEFAULT 'pending', created_at text, updated_at text);
+ CREATE TABLE subscriptions (id bigint PRIMARY KEY, subscriber_id bigint NOT NULL, streamer_id bigint NOT NULL, tier bigint DEFAULT 1,
+ is_active bigint DEFAULT 1, started_at text, expires_at text, provider text, provider_ref text,
+ price_cents bigint DEFAULT 0, currency text DEFAULT 'usd', status text DEFAULT 'active', cancel_at_period_end bigint DEFAULT 0,
+ auto_renew bigint DEFAULT 0, current_period_end text, created_at text, updated_at text);
+ CREATE TABLE site_settings (key text PRIMARY KEY, value text NOT NULL DEFAULT '', description text DEFAULT '', type text DEFAULT 'string', updated_at text);
+ CREATE TABLE powerchat_connections (user_id bigint PRIMARY KEY, powerchat_username text, powerchat_user_id text, updated_at text);
+`;
+
+async function fakeLive({ tables = LIVE_MONEY_TABLES } = {}) {
+ const parsers = {};
+ for (const [oid, fn] of Object.entries(PARSERS)) parsers[oid] = (v) => fn(v);
+ const pg = new PGlite(undefined, { parsers });
+ await pg.waitReady;
+ if (tables) await pg.exec(tables);
+ return {
+ exec: async (sql) => { await pg.exec(sql); },
+ query: async (text, params = []) => (await pg.query(text, params, { parsers })).rows,
+ source: () => liveSourceFromQuery(async (text, params) => (await pg.query(text, params, { parsers })).rows),
+ close: async () => { await pg.close(); },
+ };
+}
+
+module.exports = { fakeLive, LIVE_MONEY_TABLES };
@@ -1,32 +1,17 @@
@@ -1,32 +1,17 @@
'use strict';
'use strict';
/**
/**
- * Import from a fabricated live.db snapshot (Live's own table shapes): opening balances equal the
+ * Import from a fabricated Live database (PostgreSQL in Live's own column types, test/helpers/live-pg.js): opening balances equal the
* Live columns, differences against the replayed history are adjustments and listed, unmapped users
* Live columns, differences against the replayed history are adjustments and listed, unmapped users
* are held (never dropped) and released once mapped, test-era rows are flagged, duplicate provider
* are held (never dropped) and released once mapped, test-era rows are flagged, duplicate provider
* refs and route markers are handled, dry runs keep nothing, and a re-run changes nothing.
* refs and route markers are handled, dry runs keep nothing, and a re-run changes nothing.
*/
*/
const assert = require('assert');
const assert = require('assert');
-const path = require('path');
-const Database = require('better-sqlite3');
+const { fakeLive } = require('./helpers/live-pg');
const { boot, check, done } = require('./helpers/app');
const { boot, check, done } = require('./helpers/app');
-async function fabricateLive(file) {
- const live = new Database(file);
- live.exec(`
- CREATE TABLE users (id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT UNIQUE NOT NULL, openvibe_bucks_balance REAL DEFAULT 0.00,
- openvibe_coins_balance INTEGER DEFAULT 0, openvibe_bucks_cashout_balance REAL DEFAULT 0.00);
- CREATE TABLE transactions (id INTEGER PRIMARY KEY AUTOINCREMENT, from_user_id INTEGER, to_user_id INTEGER, stream_id INTEGER,
- amount INTEGER NOT NULL, type TEXT NOT NULL CHECK(type IN ('donation', 'purchase', 'subscription', 'cashout', 'refund', 'bonus')),
- status TEXT DEFAULT 'completed' CHECK(status IN ('pending', 'completed', 'failed', 'escrow', 'refunded')), message TEXT,
- paypal_transaction_id TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP);
- CREATE TABLE payment_orders (id INTEGER PRIMARY KEY AUTOINCREMENT, user_id INTEGER NOT NULL, provider TEXT NOT NULL, provider_ref TEXT,
- kind TEXT NOT NULL DEFAULT 'bucks', amount_cents INTEGER NOT NULL DEFAULT 0, currency TEXT DEFAULT 'usd', bucks INTEGER DEFAULT 0,
- streamer_id INTEGER, status TEXT DEFAULT 'pending', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);
- CREATE TABLE subscriptions (id INTEGER PRIMARY KEY AUTOINCREMENT, subscriber_id INTEGER NOT NULL, streamer_id INTEGER NOT NULL, tier INTEGER DEFAULT 1,
- is_active INTEGER DEFAULT 1, started_at DATETIME DEFAULT CURRENT_TIMESTAMP, expires_at DATETIME, provider TEXT, provider_ref TEXT,
- price_cents INTEGER DEFAULT 0, currency TEXT DEFAULT 'usd', status TEXT DEFAULT 'active', cancel_at_period_end INTEGER DEFAULT 0,
- auto_renew INTEGER DEFAULT 0, current_period_end DATETIME, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP);
- CREATE TABLE site_settings (key TEXT PRIMARY KEY, value TEXT NOT NULL DEFAULT '', description TEXT DEFAULT '', type TEXT DEFAULT 'string', updated_at DATETIME);
+async function fabricateLive() {
+ const live = await fakeLive();
+ await live.exec(`
INSERT INTO site_settings (key, value) VALUES ('stats_vibes_reset_at', '2026-06-01T00:00:00Z');
INSERT INTO site_settings (key, value) VALUES ('stats_vibes_reset_at', '2026-06-01T00:00:00Z');
INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance, openvibe_coins_balance) VALUES
INSERT INTO users (id, username, openvibe_bucks_balance, openvibe_bucks_cashout_balance, openvibe_coins_balance) VALUES
(1, 'alice', 1500, 0, 999), (2, 'bob', 0, 700, 0), (3, 'carol', 250, 0, 0), (4, 'dave', 100.5, 0, 0), (5, 'erin', 0, 0, 0);
(1, 'alice', 1500, 0, 999), (2, 'bob', 0, 700, 0), (3, 'carol', 250, 0, 0), (4, 'dave', 100.5, 0, 0), (5, 'erin', 0, 0, 0);
@@ -48,10 +33,9 @@ async function fabricateLive(file) {
@@ -48,10 +33,9 @@ async function fabricateLive(file) {
(5, 3, 'powerchat', NULL, 'bucks', 325, 250, NULL, 'paid', '2026-07-06 09:00:00');
(5, 3, 'powerchat', NULL, 'bucks', 325, 250, NULL, 'paid', '2026-07-06 09:00:00');
`);
`);
const future = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);
const future = new Date(Date.now() + 20 * 86_400_000).toISOString().replace('T', ' ').slice(0, 19);
- await live.prepare(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, auto_renew, current_period_end, created_at)
- VALUES (1, 1, 2, 'powerchat', 549, 'active', 1, ?, '2026-07-08 09:00:00'), (2, 4, 2, 'bucks', 499, 'expired', 0, '2026-07-01 00:00:00', '2026-06-01 00:00:00')`).run(future);
- live.close();
- return new Database(file, { readonly: true, fileMustExist: true });
+ await live.query(`INSERT INTO subscriptions (id, subscriber_id, streamer_id, provider, price_cents, status, auto_renew, current_period_end, created_at)
+ VALUES (1, 1, 2, 'powerchat', 549, 'active', 1, $1, '2026-07-08 09:00:00'), (2, 4, 2, 'bucks', 499, 'expired', 0, '2026-07-01 00:00:00', '2026-06-01 00:00:00')`, [future]);
+ return live;
}
}
(async () => {
(async () => {
@@ -60,14 +44,15 @@ async function fabricateLive(file) {
@@ -60,14 +44,15 @@ async function fabricateLive(file) {
const bob = t.network.addUser(2);
const bob = t.network.addUser(2);
const dave = t.network.addUser(4);
const dave = t.network.addUser(4);
t.network.addUser(5);
t.network.addUser(5);
- const live = await fabricateLive(path.join(t.dir, 'live-snapshot.db'));
+ const fake = await fabricateLive();
+ const live = fake.source();
const { importLive } = require('../server/importer/live');
const { importLive } = require('../server/importer/live');
const { createIdentity } = require('../server/network');
const { createIdentity } = require('../server/network');
const identity = createIdentity(t.config);
const identity = createIdentity(t.config);
const run = async (opts = {}) => await importLive(t.ctx, { live, resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} }, ...opts });
const run = async (opts = {}) => await importLive(t.ctx, { live, resolveLiveUsers: identity.resolveLiveUsers, log: { log() {} }, ...opts });
const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };
const bal = async (kind, owner) => { const { balance } = require('../server/ledger'); return await balance(t.db, { kind, owner, currency: 'vibes-bits' }); };
const count = async (table) => (await t.db.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get()).n;
const count = async (table) => (await t.db.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get()).n;
- console.log('import from live.db');
+ console.log('import from Live\'s database');
await check('a dry run reports everything and keeps nothing but its report', async () => {
await check('a dry run reports everything and keeps nothing but its report', async () => {
const r = await run({ dryRun: true });
const r = await run({ dryRun: true });
@@ -159,7 +144,7 @@ async function fabricateLive(file) {
@@ -159,7 +144,7 @@ async function fabricateLive(file) {
assert.strictEqual((await run()).new_transactions, 0);
assert.strictEqual((await run()).new_transactions, 0);
});
});
- live.close();
+ await fake.close();
await t.close();
await t.close();
done();
done();
})();
})();