openvibers/openvibe.tips Commits

Tips moves to PostgreSQL and Valkey (ADR-035 and its amendment, roadmap WS-X2, following openvibe-sdk's docs/migrating-to-postgresql.md). Everything is async on openvibe-sdk/db v0.15.0: node-postgres through PgBouncer in production, PGlite in tests and in development without DATABASE_URL. migrations/0001_initial.sql (expand) is the SQLite v3 schema in PostgreSQL types: ISO text times become timestamptz, epoch-ms stays bigint, 0/1 becomes boolean, JSON text becomes jsonb, amounts become bigint with their sums cast back, and the AUTOINCREMENT ids become identities. Two exceptions: the stored API answers stay text so a replay is byte for byte, and the SQLite settings row is not carried (ov_migrations records the schema). The SDK outbox and inbox are tips_event_outbox and tips_event_inbox, and every query shape has an index. Boot migrates with DATABASE_DIRECT_URL and then serves on DATABASE_URL; /api/ready reports db.ready() (the store that answered, and the pool) and valkey.ready(). A function that touches the database takes the handle first (db, or the transaction's t), after-commit work hangs on the transaction (t.after, per attempt), and Express handlers are async with their errors reaching the error handler.

OP OpenVibers committed 28 Sept 2026 · fe05335d27 · 53 files, +4076 −1703

Money paths are SERIALIZABLE with a fixed lock order (a creator's profile row, the interaction, its goals by id, the creator's overlay stream, the rest): a request and its unpaid-checkout limit, settlement, payment failure, funding, reversal, the Billing inbox, moderation's hide and restore, the effects worker's writes, and the Live import. Settings changes and per-creator limits lock the profile row. The Idempotency-Key is claimed before the handler runs: the same key sent twice at once runs once, and the other answer is 409 idempotency.in_progress with Retry-After. A creator's overlay deliveries are inserted under a transaction advisory lock, so their seqs commit in order and no stream skips one. Background work runs in every process and claims its rows: due effects and transfers with a lease (FOR UPDATE SKIP LOCKED), the overlay window sweep with SKIP LOCKED, and the outbox relay with the SDK's lease.

Per-process state goes to Valkey: per-actor limits (createValkeyLimitStore), overlay fan-out on pub/sub (a stream in one process hears the tips, config changes, retractions and revocations made in another; server/domain/overlay-hub.js, with local delivery when Valkey fails), and overlay stream slots (a sorted set per creator, refreshed by heartbeats). Without TIPS_FORM_SECRET, the form secret is derived from the client secret. Lists page by keyset and cost a fixed number of queries. Text PostgreSQL cannot store (NUL, an unpaired surrogate) is cleaned on the way in, and a database refusal of input answers 422.

scripts/migrate-to-postgres.js is the one-time import. It migrates as the owner, runs importSqlite with truncate, and prints the report, exiting 1 unless it is ok; the SQLite file is untouched. event_outbox and idempotency_receipts are renamed, JSON text 'null' in the reference columns becomes NULL, and a NUL or an unpaired surrogate is cleaned and reported. It was rehearsed on the 14 SQLite databases the previous release's own suite makes, into PGlite and into PostgreSQL 18: report.ok for every file, 906 rows over 18 tables. Only that script and scripts/import-live.js (which reads Live's and Billing's snapshot files) load better-sqlite3.

Tests: every file runs on PGlite. New: test/migrations.test.js (the files parse and apply twice without change, no SQLite column is left behind, index plans cover 31 query shapes, no runtime better-sqlite3), test/sqlite-import.test.js, and test/query-budget.test.js. test/integration.test.js runs two app instances on PostgreSQL behind PgBouncer and one Valkey: settlement, reversal and Idempotency-Key races count once, a creator's deliveries commit in seq order, the unpaid-checkout limit holds, leased work runs once, limits, fan-out and stream slots are shared, and flushing Valkey changes no money answer. npm run test:pg runs every file through PgBouncer and Valkey, and CI runs both on the SDK's containers. README (Depends on, Data, API, Deploy: the switch procedure), STATUS.json (database), .env.example (DATABASE_URL, DATABASE_DIRECT_URL, VALKEY_URL, VALKEY_PREFIX) and the unit are updated to match. npm test (Node 22): 17/18 passed and 1 skipped without the containers, 18/18 with them; npm run test:pg: 18/18.

Changes

ddiff --git a/.env.example b/.env.example
iindex b4ec922..a3b1b3c 100644
---- a/.env.example
++++ b/.env.example
@@@ -11,8 +11,22 @@ NODE_ENV=development
BASE_URL=http://localhost:4610
TRUST_PROXY=1
--# SQLite (one database per service). The systemd unit sets /var/lib/openvibe-tips/tips.db.
--TIPS_DB_PATH=./data/tips.db
++# ── Data: PostgreSQL + Valkey (ADR-035) ──────────────────────
++# In production OpenVibe.Host writes all four (sudo /opt/openvibe.host/roles/data/add-service.sh tips).
++# DATABASE_URL: the runtime role through PgBouncer (transaction mode; DML only). Unset in development:
++# an embedded PGlite database in data/pglite (one process). Tests need neither (PGlite, or the containers
++# of node_modules/openvibe-sdk/scripts/test-services.sh).
++DATABASE_URL=
++# DATABASE_DIRECT_URL: the owner role on a direct connection, for the migrations at boot and
++# scripts/migrate-to-postgres.js.
++DATABASE_DIRECT_URL=
++# VALKEY_URL: shared, never authoritative state (per-actor limits, overlay fan-out and stream slots).
++# Unset: kept in each process, right for one process only.
++VALKEY_URL=
++VALKEY_PREFIX=ov:tips:
++# The SQLite file of the release before PostgreSQL: read (never written) only by
++# scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.
++#TIPS_DB_PATH=./data/tips.db
# ── Identity: OpenVibe.Network ──────────────────────────────
OV_NETWORK_URL=https://openvibe.network
@@@ -28,7 +42,8 @@ OV_OAUTH_CLIENT_ID=tips
OV_OAUTH_CLIENT_SECRET=
#OV_OAUTH_REDIRECT_URI=https://openvibe.tips/auth/callback
#COOKIE_SECURE=true
--# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32).
++# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32). Every process must
++# share it; unset, it is derived from OV_OAUTH_CLIENT_SECRET (and random per process without either).
TIPS_FORM_SECRET=
# ── OpenVibe.Billing (money moves only there) ───────────────
ddiff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
iindex 72123e0..0a36680 100644
---- a/.github/workflows/ci.yml
++++ b/.github/workflows/ci.yml
@@@ -9,6 +9,10 @@ jobs:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
with:
contracts-service: tips
++ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers
++ # (openvibe-sdk scripts/test-services.sh). npm test runs every file on PGlite and
++ # test/integration.test.js on the containers; test:pg runs the whole suite through PgBouncer and Valkey.
++ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
security:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
ddiff --git a/README.md b/README.md
iindex d9b8577..6f53ebd 100644
---- a/README.md
++++ b/README.md
@@@ -4,7 +4,9 @@
**Status:** alpha — runtime built and tested against stubs (roadmap Wave 9). Deployed internally, not
launched: running on the host since 2026-09-23 on `127.0.0.1:4610` only, holding the Live import and no
--creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).
++creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).
++Data: PostgreSQL through PgBouncer, and Valkey for shared state ([ADR-035](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-035-postgresql-and-valkey.md));
++production runs the SQLite release until the switch in *Deploy*.
**Domain:** `openvibe.tips` (keeps its OpenVibe.Sites placeholder until the launch rule below holds)
**Port / service id:** 4610 / `tips`
**Decision:** [ADR-012](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-012-economic-classification.md) — Tips never moves money; Billing does.
@@@ -21,8 +23,8 @@ from *interaction delivered*.
- `tip_interactions`, `creator_tip_profiles`, `tip_goals`, `tip_goal_contributions`, `paid_messages`,
`paid_media_requests`, `overlay_configs`, `overlay_deliveries`, `interaction_effects`, `migration_maps`
-- (its own SQLite, `server/db.js`) — interaction state and **references** to Billing transaction ids,
-- never a mutable cash balance
++ (its own PostgreSQL database `ov_tips`, schema in [migrations/](migrations/), access in `server/db.js`) —
++ interaction state and **references** to Billing transaction ids, never a mutable cash balance
- revocable scoped overlay tokens (`overlay_tokens`, hashed at rest; never creator cookies)
## Does not own
@@@ -34,6 +36,10 @@ from *interaction delivered*.
## Depends on
++- **PostgreSQL 18 through PgBouncer (transaction mode) and Valkey 9** — OpenVibe.Host's data role
++ (`roles/data/add-service.sh tips`: database `ov_tips`, owner and runtime roles, a Valkey user on
++ `ov:tips:*`). PostgreSQL is required (the system of record); Valkey holds only what the processes share
++ (per-actor limit counters, overlay fan-out, overlay stream slots) and its loss changes no answer about money
- **OpenVibe.Billing** — `POST /api/v1/intents` (checkout), `POST /api/v1/transfers` (a tip from credit),
`billing.transaction.settled|reversed` events, and `billing.receipt.external` (a tip on a creator's own
PowerChat, once Billing receives the PowerChat webhook)
@@@ -42,8 +48,9 @@ from *interaction delivered*.
- **OpenVibe.Network** — SSO for people, service tokens, JWKS, identity resolve (importer)
- **OpenVibe.Live** — chat delivery through `/internal/tips/deliveries` (deployed in Live since `f11f809`; [docs/live-patch.diff](docs/live-patch.diff) is the original patch) until OpenVibe.Chat exposes paid messages; overlay consumer
- **OpenVibe.Shared** v1.25.0 (app icon, SSR footer, noscript nav, release manifest, legal pages) and
-- the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.12.0 (service tokens,
-- outbox and inbox, per-actor limits), pinned by release tarball
++ the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.15.0 (service tokens,
++ the async data layer `openvibe-sdk/db`, the PostgreSQL outbox and inbox, Valkey, pub/sub and the shared
++ limit store), pinned by release tarball; `pg` and `iovalkey` (drivers), PGlite for tests
## Capabilities
@@@ -66,10 +73,14 @@ Called elsewhere, as the service principal `tips`:
```bash
fnm exec --using=22.22.1 npm install
cp .env.example .env # OV_OAUTH_CLIENT_SECRET, TIPS_FORM_SECRET, TIPS_EVENTS_SECRET, …
--npm run dev # http://localhost:4610
--npm test # every test/*.test.js: stub Network/Billing/Events/Live, temp DBs, random ports
++npm run dev # http://localhost:4610 (without DATABASE_URL: an embedded PGlite database in data/pglite)
++npm test # every test/*.test.js: stub Network/Billing/Events/Live, PGlite, random ports
++eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" # PostgreSQL 18 + PgBouncer + Valkey 9 containers
++npm test # … now also test/integration.test.js on the containers
++npm run test:pg # every test file through PgBouncer and Valkey
npm run subscribe # create the billing.transaction.* and billing.receipt.* subscriptions in OpenVibe.Events
node scripts/import-live.js --live-db <live snapshot> --billing-db <billing snapshot> [--dry-run] [--json]
++npm run migrate-to-postgres -- [--sqlite <copy>] [--pglite] [--json] # the one-time SQLite import (Deploy)
```
Production (deployed, loopback only): `/opt/openvibe.tips`, env `/etc/openvibe/tips.env`, unit
@@@ -101,7 +112,7 @@ delivery record; a reversal before delivery cancels the queued effects (`tips.in
| `external` | nothing — a tip on the creator's own PowerChat (ADR-012 EXTERNAL) | `POST /api/v1/interactions/external`, or Billing's `billing.receipt.external` (origin `billing-external`: Tips posts the chat line — `… tipped N Vibes: msg (PowerChat)` — plus overlay alert and goal, as Live's webhook did); once per (provider, provider_ref); never in Billing totals |
| `none` (simulation) | nothing | at once, `test = 1`: full effect path, no goal contribution, no durable event, excluded from totals |
--**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same SQLite
++**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same serializable
transaction as the change, so a redelivered event does nothing; `billing_txn_id` is UNIQUE, so the same
Billing transaction under another event id (republish, replay) is still one interaction; API calls carry
`Idempotency-Key` (stored responses + the interaction's own unique key); the no-JS form carries a nonce.
@@@ -117,9 +128,35 @@ from Live (whose `current_amount` cannot be traced to individual donations).
minus what Billing took back) — the number that must equal Billing's books — with `external` and
`simulations` reported apart.
++**Data** (ADR-035; `server/db.js`, [migrations/](migrations/)):
++- Everything is async on `openvibe-sdk/db`: a function that touches the database takes the query handle
++ first (`db`, or the transaction's `t`), and after-commit work (overlay pushes, worker kicks) hangs on
++ the transaction (`t.after`). The schema is `migrations/NNNN_*.sql` (expand/migrate/contract), applied
++ at boot with the owner role; the service then serves through PgBouncer with no session state.
++- **Money paths are SERIALIZABLE with row locks** (ADR-007 amendment 2026-09-24, rule 3): a request and
++ its unpaid-checkout limit, settlement (paid message, media request, goal contribution), payment failure,
++ funding, reversal, the Billing inbox, and what cancels or releases what a payment bought (moderation's
++ hide and restore, the effects worker's writes). Locks are taken in one order: a creator's profile row
++ (settings changes and per-creator limits), the interaction row, its goals by id, the creator's overlay
++ stream (an advisory lock, so a creator's deliveries commit in seq order), then the rest. Serialization
++ failures and deadlocks are retried by the data layer.
++- Text is made storable on the way in: PostgreSQL keeps no NUL character, and jsonb no unpaired surrogate
++ (SQLite kept both), so they are dropped or replaced (U+FFFD) in what people and Billing's events send.
++- **Background work runs in every process** and claims its rows: due effects and due transfers with a
++ lease (`FOR UPDATE SKIP LOCKED`, the lease in `next_attempt_at` / `next_transfer_at`), the overlay
++ window sweep with `SKIP LOCKED`, the outbox relay by the SDK's lease; prunes are idempotent deletes.
++- **Shared state in Valkey** (none of it authoritative): per-actor limit counters (the SDK's shared limit
++ store), overlay fan-out (`overlay:<creator>` on pub/sub: a stream in one process hears a tip, a config
++ change, a retraction or a revocation made in another), overlay stream slots (a sorted set per creator,
++ refreshed by heartbeats). Without `VALKEY_URL` each process keeps its own, which is right for one process.
++- Every query shape has an index (`test/migrations.test.js` checks the plans), lists page by keyset, and a
++ list costs a fixed number of queries whatever its size (`test/query-budget.test.js`).
++
## API
--`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH. Services present a Network
++`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH (claimed before the handler runs:
++the same key twice at once runs once, the other answered `409 idempotency.in_progress` with `Retry-After`
++until the first answer is stored and replayed). Services present a Network
service token (audience `openvibe.tips`, one capability per route); people present their Network user
token as a Bearer and act on their own things only. The API never reads cookies.
@@@ -218,8 +255,8 @@ toward its goal and the creator's totals.
## Events
--Produced through the openvibe-sdk outbox (table `event_outbox`, source `tips`, same transaction as the
--change; relayed when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,
++Produced through the openvibe-sdk outbox (table `tips_event_outbox`, source `tips`, same transaction as
++the change; relayed by any process, rows claimed with a lease, when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,
`tips.interaction.failed`, `tips.interaction.cancelled`, `tips.goal.updated`, `tips.overlay.delivered`,
`tips.overlay.failed`, and two whose schemas were proposed in [docs/events-proposal/](docs/events-proposal/)
and are in openvibe-contracts since v0.32.0: `tips.interaction.erased` (a supporter's erasure; `{ interaction_id,
@@@ -319,6 +356,7 @@ service counts as its principal (`svc:live`), a person as `user:usr_…`, anyone
Never limited: `/api/health`, `/api/ready`, `/release.json`, `/metrics`, the overlays (bounded by
`TIPS_OVERLAY_MAX_STREAMS`) and the signed Events deliveries at `/internal/events`. `test/actor-limits.test.js`.
++With `VALKEY_URL` every process counts one caller together (`test/integration.test.js`).
## Import and reconciliation
@@@ -347,6 +385,8 @@ a copy of Billing's database for the links and totals:
| simulation never counted; token revocation immediate; goals from settled only; reversal keeps the delivery record | `test/overlays.test.js`, `test/api.test.js`, `test/settlement.test.js` |
| an EXTERNAL PowerChat tip Billing announced is celebrated once: Live chat line, overlay, goal; never Billing money | `test/external.test.js` (redelivery and republish, same key as `POST /interactions/external`, Tips and Live goal ids, anonymous, test receipts, malformed payloads) |
| paid messages are filtered before they are shown or read; the creator and their moderators hide and show them without touching the money | `test/moderation.test.js` (mask, hold and release, invisible/full-width evasion, TTS text, invitation links, hide cancels queued chat/TTS and retracts the overlay live/replay/state, restore, pending payments, who may moderate, event payloads) |
++| the suite passes on PostgreSQL behind PgBouncer in transaction mode; one settlement or reversal raced across two processes counts once; the unpaid-checkout limit holds under concurrency; leased work runs once; flushing Valkey changes no money answer (ADR-007 2026-09-24, ADR-035) | `npm run test:pg`; `test/integration.test.js` (two app instances on one database and one Valkey) |
++| the SQLite data moves whole: every table and column, verified by counts and checksums | `test/sqlite-import.test.js`, `test/migrations.test.js` (no column left behind) |
| a supporter's privacy holds everywhere; the creator chooses what public pages show; export and erasure keep the books reconciled | `test/privacy.test.js` (anonymous / hidden amount / private message across API, overlays, chat jobs, pages and events; goal and supporters page settings; export; erasure scrubs rows, overlay payloads, stored answers and unsent events, emits `tips.interaction.erased`, totals still equal Billing) |
## What waits
@@@ -375,8 +415,25 @@ The unit is `openvibe-tips.service` on `127.0.0.1:4610`, the env file `/etc/open
`/var/lib/openvibe-tips`. The vhost [deploy/nginx/openvibe.tips.conf](deploy/nginx/openvibe.tips.conf)
waits for the launch: `openvibe.tips` serves the Sites placeholder.
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
--restart; afterwards `sudo ovhost rollback tips --to <sha>`. Nothing blocks a rollback: the schema
--code only adds tables and columns.
++restart; afterwards `sudo ovhost rollback tips --to <sha>`. Migrations are expand-only until a
++`contract` migration, which waits out the 7-day N-1 window.
++
++**The switch from SQLite to PostgreSQL** (once; the procedure is the SDK's
++[migrating-to-postgresql.md](https://github.com/OpenVibers/OpenVibe.SDK/blob/main/docs/migrating-to-postgresql.md), section 6):
++1. `sudo /opt/openvibe.host/roles/data/add-service.sh tips` — database `ov_tips`, its roles and Valkey
++ user; it writes `DATABASE_URL`, `DATABASE_DIRECT_URL`, `VALKEY_URL`, `VALKEY_PREFIX` into
++ `/etc/openvibe/tips.env`.
++2. Rehearse on a copy of `/var/lib/openvibe-tips/tips.db` (`sqlite3 … ".backup …"` as the service user):
++ `npm run migrate-to-postgres -- --sqlite <copy> --pglite` must end OK.
++3. Merge the PostgreSQL release; do not deploy it yet.
++4. `sudo systemctl stop openvibe-tips` (the write freeze), then back up `tips.db`.
++5. From the new release's directory, as the service user with the service's environment:
++ `node scripts/migrate-to-postgres.js` — it migrates as the owner, imports every table with
++ `truncate`, verifies counts and checksums, changes nothing in the SQLite file, and exits 1 unless OK.
++6. `sudo ovhost deploy tips`; check `/api/ready` shows `db` `store: postgresql` and `valkey` ok, the
++ dashboard, a receipt and an overlay, and a flat error rate.
++7. Keep `tips.db` read-only for 7 days (the rollback: the previous release reads it, losing the writes
++ made since the switch unless they are replayed), then `ovhost archive push` it and delete it.
## Launch rule
@@@ -385,8 +442,8 @@ This repository does not make the product real, and the domain keeps its placeho
exist here (plan §12.12):
1. an owning runtime with health/readiness endpoints and observability — ✔ `/api/health`, `/api/ready`
-- (openvibe-shared/ready: the database required; Network key, Billing and Events optional, so
-- their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals by
++ (openvibe-shared/ready: the database required and reporting the store that answered; Valkey, the
++ Network key, Billing and Events optional, so their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals by
route template, pending effects and overlay deliveries, outbox backlog);
2. canonical identity/auth integration (Network subjects, scoped service principals) — ✔; the `tips`
principal is provisioned on the host;
ddiff --git a/STATUS.json b/STATUS.json
iindex c79075e..debac6f 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -8,6 +8,7 @@
"authoritative": false,
"deployed": true,
"deployment": "Deployed internally, not launched: deployed with `sudo ovhost deploy tips`, /opt/openvibe.tips, unit openvibe-tips, 127.0.0.1:4610 loopback only; https://openvibe.tips still serves the OpenVibe.Sites placeholder. Principal tips; Events subscriptions billing.transaction.* and billing.receipt.* -> /internal/events (secret rotated 2026-09-24); TIPS_CHAT_ADAPTER is empty (no chat side effects).",
++ "database": "PostgreSQL 18 through PgBouncer (transaction mode) as the system of record, Valkey 9 for shared state (per-actor limits, overlay fan-out, overlay stream slots), ADR-035: database ov_tips from OpenVibe.Host's data role, schema in migrations/ (0001_initial, expand), async on openvibe-sdk/db v0.15.0, money paths serializable with row locks, PostgreSQL outbox and inbox. The deployed release still runs on SQLite (/var/lib/openvibe-tips/tips.db) until the switch in README Deploy: scripts/migrate-to-postgres.js imports it verified by counts and checksums (rehearsed on the SQLite databases the previous release makes: all tables and columns, report.ok), and the file stays read-only for the 7-day N-1 window.",
"contracts": "openvibe-contracts v0.49.0",
"notes": "Runtime, API, SSR pages, overlays, the Billing settlement consumer, events outbox, Live import and reconciliation are built, tested and deployed, together with /metrics (loopback only), truthful /api/ready (Billing and Events checks), supporter privacy (anonymous, hidden amount, private message), goal and supporter-page settings, export and erasure (tips.interaction.erased), moderation (word filter, hide/show, invited moderators, tips.interaction.moderated) and per-actor limits. Capabilities (18 tips.*) and payload schemas are in openvibe-contracts. On 2026-09-24 production held the Live import (3 import runs, the last 2026-09-23 18:31 UTC: 139 interactions, 4 goals, 143 migration maps); 0 creator profiles and 0 overlay tokens (nobody had set up the product). No billing.* event had arrived: Billing runs in shadow. A written internal threat review (docs/threat-review.md) with its fixes and test/security.test.js. Not done: an independent security review; the openvibe.tips launch. Live stays the place tips happen until the Billing cutover (blocked on the owner re-pointing the PowerChat webhook) and the launch rule in README.md holds.",
"updated": "2026-09-28",
@@@ -22,7 +23,8 @@
"Live import and reconciliation against Billing",
"server-rendered creator pages useful without JavaScript",
"/api/v1 by capability",
-- "SDK outbox and inbox",
-- "per-actor limits (openvibe-sdk/limits)"
++ "per-actor limits (openvibe-sdk/limits)",
++ "PostgreSQL + Valkey data tier (ADR-035): serializable money paths, leased background work, shared limits and overlay fan-out",
++ "SDK outbox and inbox on PostgreSQL"
]
}
ddiff --git a/deploy/systemd/openvibe-tips.service b/deploy/systemd/openvibe-tips.service
iindex f1f33e1..06cb639 100644
---- a/deploy/systemd/openvibe-tips.service
++++ b/deploy/systemd/openvibe-tips.service
@@@ -3,8 +3,11 @@ Description=OpenVibe.Tips — creator support: tips, goals, paid messages, TTS a
# Network signs the tokens Tips verifies (its key is fetched at boot and retried); Billing moves
# the money; Events carries Billing's settlements in and Tips' events out. Tips starts without any
# of them: pages and overlays keep working, payments wait.
--After=network.target
--Wants=openvibe-network.service openvibe-billing.service
++# Its own data (ADR-035, OpenVibe.Host's data role): PostgreSQL (migrations at boot on the direct
++# connection, then PgBouncer) is required, the process exits and is restarted until it answers;
++# Valkey is shared state only (without it everything shared is counted per process).
++After=network.target postgresql.service pgbouncer.service valkey-server.service
++Wants=openvibe-network.service openvibe-billing.service postgresql.service pgbouncer.service valkey-server.service
[Service]
Type=simple
@@@ -19,7 +22,9 @@ Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node /opt/openvibe.tips/server/index.js
Restart=always
RestartSec=5
--# Tips' own database (ADR-007: one database per service); the code tree stays read-only.
++# DATABASE_URL, DATABASE_DIRECT_URL, VALKEY_URL and VALKEY_PREFIX come from the env file
++# (roles/data/add-service.sh tips). The state directory keeps the SQLite file of the release before
++# PostgreSQL: read-only rollback for the 7-day N-1 window, read by scripts/migrate-to-postgres.js.
StateDirectory=openvibe-tips
StateDirectoryMode=0700
Environment=TIPS_DB_PATH=/var/lib/openvibe-tips/tips.db
ddiff --git a/migrations/0001_initial.sql b/migrations/0001_initial.sql
nnew file mode 100644
iindex 0000000..d49512c
---- /dev/null
++++ b/migrations/0001_initial.sql
@@@ -0,0 +1,365 @@
++-- phase: expand
++--
++-- OpenVibe.Tips on PostgreSQL (ADR-035): the SQLite schema of the last SQLite release (server/db.js,
++-- schema v3 with its added columns; the SDK outbox and inbox) in PostgreSQL types, so
++-- scripts/migrate-to-postgres.js can import the SQLite file as it is.
++--
++-- ISO-8601 text times -> timestamptz (the data layer returns the same ISO strings the API sent)
++-- epoch-ms integers -> bigint (next_transfer_at, next_attempt_at, the outbox's times)
++-- 0/1 integers -> boolean
++-- JSON text -> jsonb, except api_idempotency.response (below)
++-- amounts (bits/cents) -> bigint; SUM() of a bigint is numeric, so queries cast sums back ::bigint
++-- AUTOINCREMENT -> bigint GENERATED ALWAYS AS IDENTITY (the importer keeps the values)
++--
++-- Not carried over: the SQLite `settings` table (one row, schema_version 3): ov_migrations records the
++-- schema now, and readiness asks the database itself (db.ready()).
++--
++-- Every query shape the code runs has an index; the comment above each index names the queries.
++-- Locks are always taken in one order: a creator's profile row, tip_interactions, tip_goals by id, the
++-- creator's overlay stream (an advisory lock), then the rest (server/domain/index.js).
++
++-- ── Creator profiles ────────────────────────────────────────────────────────────────────────────
++CREATE TABLE creator_tip_profiles (
++ creator_subject text PRIMARY KEY, -- usr_…
++ handle text NOT NULL UNIQUE, -- lower-case Network username (projection)
++ display_name text NOT NULL,
++ avatar_url text,
++ headline text,
++ page_enabled boolean NOT NULL DEFAULT false, -- public page + indexing
++ accepting boolean NOT NULL DEFAULT true, -- new tips accepted
++ min_amount bigint NOT NULL DEFAULT 1,
++ paid_message_min bigint NOT NULL DEFAULT 100,
++ tts_enabled boolean NOT NULL DEFAULT false,
++ tts_min_amount bigint NOT NULL DEFAULT 100,
++ tts_max_chars integer NOT NULL DEFAULT 200,
++ tts_voice text NOT NULL DEFAULT 'gary',
++ media_requests_enabled boolean NOT NULL DEFAULT false,
++ media_request_min bigint NOT NULL DEFAULT 25,
++ media_max_seconds integer NOT NULL DEFAULT 600,
++ revision integer NOT NULL DEFAULT 1,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL,
++ page_settings jsonb NOT NULL DEFAULT '{}', -- what the public goal / supporters pages show
++ filter_words jsonb NOT NULL DEFAULT '[]', -- the creator's blocklist (filter.js)
++ filter_action text NOT NULL DEFAULT 'mask', -- mask | hold
++ filter_links boolean NOT NULL DEFAULT true -- links in paid messages shown as [link]
++);
++-- Home page and sitemap: switched-on pages, newest first / by handle.
++CREATE INDEX creator_tip_profiles_enabled ON creator_tip_profiles (updated_at DESC) WHERE page_enabled;
++
++-- ── Interactions ────────────────────────────────────────────────────────────────────────────────
++CREATE TABLE tip_interactions (
++ id text PRIMARY KEY, -- tint_<ULID>
++ creator_subject text NOT NULL,
++ supporter_subject text, -- null: external/anonymous/unmapped/erased
++ supporter_name text, -- display projection at the time
++ kind text NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
++ amount bigint NOT NULL CHECK (amount > 0),
++ currency text NOT NULL DEFAULT 'vibes-bits',
++ amount_cents bigint, -- money that arrived (provider/external)
++ message text,
++ request jsonb NOT NULL DEFAULT '{}', -- tts text/voice, media url, goal_id, highlight
++ funding text NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
++ settlement text NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
++ payment_state text NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
++ delivery_state text NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
++ test boolean NOT NULL DEFAULT false, -- simulation or Billing test money: never counted
++ billing_txn_id text UNIQUE, -- the Billing transaction that settled it
++ billing_intent_id text UNIQUE, -- checkout: the Billing payment intent
++ funding_txn_id text UNIQUE, -- checkout: the purchase that funded it
++ reversal_txn_ids jsonb NOT NULL DEFAULT '[]',
++ reversed_bits bigint NOT NULL DEFAULT 0, -- bits Billing took back from the creator
++ checkout_url text,
++ checkout_ref text,
++ provider text,
++ provider_ref text,
++ legacy_source text, -- 'live:transactions:<id>' etc.
++ idempotency_key text UNIQUE,
++ failure text,
++ origin text NOT NULL DEFAULT 'tips', -- tips | billing | external | billing-external | import
++ transfer_due boolean NOT NULL DEFAULT false, -- a Billing transfer still has to be (re)tried
++ transfer_attempts integer NOT NULL DEFAULT 0,
++ next_transfer_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the lease of the transfer job
++ target jsonb, -- EntityRef (e.g. a Live stream)
++ created_at timestamptz NOT NULL,
++ settled_at timestamptz,
++ reversed_at timestamptz,
++ delivered_at timestamptz,
++ updated_at timestamptz NOT NULL,
++ anonymous boolean NOT NULL DEFAULT false, -- "Anonymous" to everyone but the supporter
++ hide_amount boolean NOT NULL DEFAULT false, -- amount left out of overlays, chat, public pages
++ private_message boolean NOT NULL DEFAULT false, -- a tip's message is for the creator only
++ erased_at timestamptz, -- the supporter erased their data from it
++ moderation text NOT NULL DEFAULT 'visible', -- visible | held | hidden
++ moderated_at timestamptz,
++ moderated_by text, -- 'filter', usr_… or svc:…
++ filtered boolean NOT NULL DEFAULT false, -- the word filter matched it at settlement
++ UNIQUE (provider, provider_ref),
++ UNIQUE (legacy_source)
++);
++-- A creator's interactions, newest first, keyset-paged (list, dashboard, moderation queue "all"); totals.
++CREATE INDEX tip_interactions_creator ON tip_interactions (creator_subject, created_at, id);
++-- A supporter's receipts, newest first, keyset-paged; export; erasure.
++CREATE INDEX tip_interactions_supporter ON tip_interactions (supporter_subject, created_at, id) WHERE supporter_subject IS NOT NULL;
++-- The moderation queue by state, keyset-paged; the dashboard's held count.
++CREATE INDEX tip_interactions_moderation ON tip_interactions (creator_subject, moderation, created_at, id);
++-- Unpaid checkouts one supporter holds (the pending-checkout limit).
++CREATE INDEX tip_interactions_open_checkouts ON tip_interactions (supporter_subject, created_at)
++ WHERE funding = 'checkout' AND payment_state = 'pending';
++-- Due Billing transfers, claimed with a lease (FOR UPDATE SKIP LOCKED).
++CREATE INDEX tip_interactions_transfer_due ON tip_interactions (next_transfer_at) WHERE payment_state = 'pending' AND transfer_due;
++-- The public supporters page: the leaderboard (settled or reversed, public name and amount).
++CREATE INDEX tip_interactions_leaderboard ON tip_interactions (creator_subject, supporter_subject)
++ WHERE supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed') AND NOT anonymous
++ AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible';
++-- The public supporters page: recent public messages.
++CREATE INDEX tip_interactions_recent_public ON tip_interactions (creator_subject, settled_at, id)
++ WHERE payment_state = 'settled' AND message IS NOT NULL;
++
++-- ── Goals ───────────────────────────────────────────────────────────────────────────────────────
++CREATE TABLE tip_goals (
++ id text PRIMARY KEY, -- tgoal_<ULID>
++ creator_subject text NOT NULL,
++ title text NOT NULL,
++ description text,
++ target_amount bigint NOT NULL CHECK (target_amount > 0),
++ currency text NOT NULL DEFAULT 'vibes-bits',
++ image_url text,
++ status text NOT NULL CHECK (status IN ('active', 'closed')),
++ sort_order integer NOT NULL DEFAULT 0,
++ reached_at timestamptz,
++ closed_at timestamptz,
++ opening_amount bigint NOT NULL DEFAULT 0, -- carried over from Live's goal column
++ legacy_source text UNIQUE,
++ revision integer NOT NULL DEFAULT 1,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL
++);
++-- A creator's goals (by status), in the creator's order; active count; the goal a tip counts toward.
++CREATE INDEX tip_goals_creator ON tip_goals (creator_subject, status, sort_order, created_at);
++
++CREATE TABLE tip_goal_contributions (
++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
++ goal_id text NOT NULL REFERENCES tip_goals(id),
++ interaction_id text NOT NULL REFERENCES tip_interactions(id),
++ amount bigint NOT NULL CHECK (amount > 0),
++ reversed_amount bigint NOT NULL DEFAULT 0,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL,
++ UNIQUE (goal_id, interaction_id)
++);
++-- Goal totals (goal_id = ANY), the latest contributions of a goal (goal_id, id DESC).
++CREATE INDEX tip_goal_contributions_goal ON tip_goal_contributions (goal_id, id);
++-- A reversal's contributions; a supporter's export.
++CREATE INDEX tip_goal_contributions_interaction ON tip_goal_contributions (interaction_id);
++
++-- ── What settlement creates ─────────────────────────────────────────────────────────────────────
++CREATE TABLE paid_messages (
++ id text PRIMARY KEY, -- tpm_<ULID>
++ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),
++ creator_subject text NOT NULL,
++ kind text NOT NULL CHECK (kind IN ('paid_message', 'tts')),
++ text text NOT NULL,
++ voice text,
++ highlight_seconds integer NOT NULL DEFAULT 0,
++ status text NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
++ chat_ref jsonb, -- the reference the chat adapter returned
++ test boolean NOT NULL DEFAULT false,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL
++);
++
++CREATE TABLE paid_media_requests (
++ id text PRIMARY KEY, -- tmr_<ULID>
++ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),
++ creator_subject text NOT NULL,
++ url text NOT NULL,
++ provider text,
++ status text NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
++ queue_ref jsonb, -- the reference returned by the queue owner
++ test boolean NOT NULL DEFAULT false,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL
++);
++
++-- ── Overlays ────────────────────────────────────────────────────────────────────────────────────
++CREATE TABLE overlay_configs (
++ id text PRIMARY KEY, -- tovc_<ULID>
++ creator_subject text NOT NULL,
++ kind text NOT NULL CHECK (kind IN ('alerts', 'goal')),
++ name text NOT NULL,
++ settings jsonb NOT NULL DEFAULT '{}',
++ goal_id text,
++ revision integer NOT NULL DEFAULT 1,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL
++);
++-- A creator's configs in creation order.
++CREATE INDEX overlay_configs_creator ON overlay_configs (creator_subject, created_at);
++
++CREATE TABLE overlay_tokens (
++ id text PRIMARY KEY, -- tovt_<ULID>
++ creator_subject text NOT NULL,
++ token_hash text NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
++ scopes jsonb NOT NULL, -- ["alerts","goals"]
++ config_id text,
++ label text,
++ created_by text NOT NULL,
++ created_at timestamptz NOT NULL,
++ last_used_at timestamptz,
++ revoked_at timestamptz
++);
++-- A creator's tokens (listing, the active count).
++CREATE INDEX overlay_tokens_creator ON overlay_tokens (creator_subject, created_at);
++
++CREATE TABLE overlay_deliveries (
++ seq bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, -- the SSE event id (monotonic)
++ id text NOT NULL UNIQUE, -- tovd_<ULID>
++ creator_subject text NOT NULL,
++ kind text NOT NULL CHECK (kind IN ('alert', 'goal')),
++ interaction_id text,
++ goal_id text,
++ payload jsonb NOT NULL,
++ test boolean NOT NULL DEFAULT false,
++ status text NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
++ delivered_at timestamptz,
++ failed_at timestamptz,
++ sends integer NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
++ dedupe_key text NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
++ created_at timestamptz NOT NULL,
++ hidden boolean NOT NULL DEFAULT false -- retracted: never sent or replayed again
++);
++-- An overlay's stream: a creator's rows after a seq (push, replay, pending on connect, /state, max seq).
++CREATE INDEX overlay_deliveries_creator ON overlay_deliveries (creator_subject, seq);
++-- The delivery window sweep and the pending gauge.
++CREATE INDEX overlay_deliveries_pending ON overlay_deliveries (seq) WHERE status = 'pending';
++-- An interaction's deliveries (erasure rewrite, moderation retract/restore).
++CREATE INDEX overlay_deliveries_interaction ON overlay_deliveries (interaction_id) WHERE interaction_id IS NOT NULL;
++
++-- ── Delivery effects (the effects worker claims due rows with a lease) ──────────────────────────
++CREATE TABLE interaction_effects (
++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
++ interaction_id text NOT NULL REFERENCES tip_interactions(id),
++ effect text NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
++ adapter text NOT NULL,
++ state text NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
++ attempts integer NOT NULL DEFAULT 0,
++ next_attempt_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the worker's lease
++ last_error text,
++ result jsonb,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL,
++ UNIQUE (interaction_id, effect)
++);
++-- Due effects (the worker's claim) and the pending/due gauges.
++CREATE INDEX interaction_effects_due ON interaction_effects (next_attempt_at, id) WHERE state = 'queued';
++
++-- ── Live import bookkeeping ─────────────────────────────────────────────────────────────────────
++CREATE TABLE migration_maps (
++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
++ source text NOT NULL, -- 'live'
++ source_table text NOT NULL,
++ source_id text NOT NULL,
++ target_type text, -- 'interaction' | 'goal' | 'refund'
++ target_id text,
++ status text NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
++ reason text,
++ billing_txn_id text,
++ run_id text NOT NULL,
++ created_at timestamptz NOT NULL,
++ updated_at timestamptz NOT NULL,
++ UNIQUE (source, source_table, source_id)
++);
++
++CREATE TABLE import_runs (
++ id text PRIMARY KEY,
++ source text NOT NULL,
++ dry_run boolean NOT NULL,
++ started_at timestamptz NOT NULL,
++ finished_at timestamptz,
++ report jsonb
++);
++
++-- ── Moderation ──────────────────────────────────────────────────────────────────────────────────
++CREATE TABLE tip_moderators (
++ creator_subject text NOT NULL,
++ moderator_subject text NOT NULL,
++ name text,
++ added_by text NOT NULL, -- the creator, a service, or 'invite:<id>'
++ created_at timestamptz NOT NULL,
++ removed_at timestamptz,
++ PRIMARY KEY (creator_subject, moderator_subject)
++);
++-- The creators a person moderates.
++CREATE INDEX tip_moderators_moderator ON tip_moderators (moderator_subject);
++
++CREATE TABLE tip_moderator_invites (
++ id text PRIMARY KEY, -- tmin_<ULID>
++ creator_subject text NOT NULL,
++ token_hash text NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
++ created_by text NOT NULL,
++ created_at timestamptz NOT NULL,
++ expires_at timestamptz NOT NULL,
++ used_at timestamptz,
++ used_by text,
++ revoked_at timestamptz
++);
++-- A creator's open invitations (listing and the open count).
++CREATE INDEX tip_moderator_invites_creator ON tip_moderator_invites (creator_subject, created_at);
++
++CREATE TABLE tip_moderation_log (
++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
++ creator_subject text NOT NULL,
++ interaction_id text NOT NULL,
++ action text NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
++ by_role text NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
++ actor text, -- usr_… / svc:… ; null for the filter
++ reason text,
++ created_at timestamptz NOT NULL
++);
++-- A creator's log, newest first.
++CREATE INDEX tip_moderation_log_creator ON tip_moderation_log (creator_subject, id);
++
++-- ── Stored API answers (Idempotency-Key) ────────────────────────────────────────────────────────
++-- response stays text: a replay returns the stored answer byte for byte, and jsonb would reorder keys.
++-- A supporter's erasure deletes theirs by key prefix or by their subject in the answer: a scan of at
++-- most a week of answers (pruned daily), for a rare action.
++CREATE TABLE api_idempotency (
++ key text PRIMARY KEY, -- <principal>:<Idempotency-Key>
++ request_hash text NOT NULL,
++ method text NOT NULL,
++ path text NOT NULL,
++ status integer NOT NULL,
++ response text NOT NULL,
++ created_at timestamptz NOT NULL
++);
++-- The weekly prune.
++CREATE INDEX api_idempotency_created ON api_idempotency (created_at);
++
++-- ── Events: the openvibe-sdk outbox and inbox (outboxSchema('tips_event_outbox'), inboxSchema('tips_event_inbox')) ──
++CREATE TABLE IF NOT EXISTS tips_event_outbox (
++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
++ event_id text NOT NULL UNIQUE,
++ envelope jsonb NOT NULL,
++ traceparent text,
++ created_at bigint NOT NULL,
++ attempts integer NOT NULL DEFAULT 0,
++ next_attempt_at bigint NOT NULL DEFAULT 0,
++ sent_at bigint,
++ seq bigint,
++ rejected_at bigint,
++ last_error text
++);
++CREATE INDEX IF NOT EXISTS tips_event_outbox_due ON tips_event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;
++CREATE INDEX IF NOT EXISTS tips_event_outbox_sent ON tips_event_outbox (sent_at) WHERE sent_at IS NOT NULL;
++-- Tips' own queries on it: the rejected count (/api/ready), and an erasure scrubbing the local copies
++-- of an interaction's events.
++CREATE INDEX tips_event_outbox_rejected ON tips_event_outbox (id) WHERE rejected_at IS NOT NULL;
++CREATE INDEX tips_event_outbox_interaction ON tips_event_outbox ((envelope->'subject'->>'id')) WHERE envelope->'subject'->>'type' = 'interaction';
++
++CREATE TABLE IF NOT EXISTS tips_event_inbox (
++ consumer text NOT NULL,
++ event_id text NOT NULL,
++ processed_at bigint NOT NULL,
++ PRIMARY KEY (consumer, event_id)
++);
ddiff --git a/package-lock.json b/package-lock.json
iindex f982215..941e0b3 100644
---- a/package-lock.json
++++ b/package-lock.json
@@@ -13,15 +13,33 @@
"cookie-parser": "^1.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
++ "iovalkey": "^0.4.0",
"jsonwebtoken": "^9.0.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
++ "pg": "^8.23.0"
++ },
++ "devDependencies": {
++ "@electric-sql/pglite": "^0.5.8"
},
"engines": {
"node": ">=22.0.0"
}
},
++ "node_modules/@electric-sql/pglite": {
++ "version": "0.5.8",
++ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",
++ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",
++ "devOptional": true,
++ "license": "Apache-2.0"
++ },
++ "node_modules/@iovalkey/commands": {
++ "version": "0.1.0",
++ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",
++ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",
++ "license": "MIT"
++ },
"node_modules/accepts": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
@@@ -223,6 +241,15 @@
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
},
++ "node_modules/cluster-key-slot": {
++ "version": "1.1.2",
++ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
++ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
++ "license": "Apache-2.0",
++ "engines": {
++ "node": ">=0.10.0"
++ }
++ },
"node_modules/content-disposition": {
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@@ -305,6 +332,15 @@
"node": ">=4.0.0"
}
},
++ "node_modules/denque": {
++ "version": "2.1.0",
++ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
++ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
++ "license": "Apache-2.0",
++ "engines": {
++ "node": ">=0.10"
++ }
++ },
"node_modules/depd": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@@ -714,6 +750,49 @@
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
},
++ "node_modules/iovalkey": {
++ "version": "0.4.0",
++ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
++ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",
++ "license": "MIT",
++ "dependencies": {
++ "@iovalkey/commands": "^0.1.0",
++ "cluster-key-slot": "^1.1.0",
++ "debug": "^4.3.4",
++ "denque": "^2.1.0",
++ "lodash.defaults": "^4.2.0",
++ "lodash.isarguments": "^3.1.0",
++ "redis-errors": "^1.2.0",
++ "redis-parser": "^3.0.0",
++ "standard-as-callback": "^2.1.0"
++ },
++ "engines": {
++ "node": ">=18.12.0"
++ }
++ },
++ "node_modules/iovalkey/node_modules/debug": {
++ "version": "4.4.3",
++ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
++ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
++ "license": "MIT",
++ "dependencies": {
++ "ms": "^2.1.3"
++ },
++ "engines": {
++ "node": ">=6.0"
++ },
++ "peerDependenciesMeta": {
++ "supports-color": {
++ "optional": true
++ }
++ }
++ },
++ "node_modules/iovalkey/node_modules/ms": {
++ "version": "2.1.3",
++ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
++ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
++ "license": "MIT"
++ },
"node_modules/ipaddr.js": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@@ -778,12 +857,24 @@
"safe-buffer": "^5.0.1"
}
},
++ "node_modules/lodash.defaults": {
++ "version": "4.2.0",
++ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
++ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
++ "license": "MIT"
++ },
"node_modules/lodash.includes": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
"license": "MIT"
},
++ "node_modules/lodash.isarguments": {
++ "version": "3.1.0",
++ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
++ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
++ "license": "MIT"
++ },
"node_modules/lodash.isboolean": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
@@@ -999,12 +1090,32 @@
}
},
"node_modules/openvibe-sdk": {
-- "version": "0.12.0",
-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
-- "integrity": "sha512-iG1h905O1f7y/PnP8xLA6GCVzXEBDfSaBj2tc1ucNreWg2sT/AJWosb8NW0WRMK5WS4sIQ6tMsprmDLj/fjjeQ==",
++ "version": "0.15.0",
++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
++ "integrity": "sha512-5ZcqEZxRQjT15erqtEdJ4js3roSijUFiQZPbGIHpaOVFRJSI45ZCEgTBb9ByKNRdUNBl4ub9Vb1dF2CcCVJjrg==",
"license": "MIT",
"engines": {
"node": ">=20"
++ },
++ "peerDependencies": {
++ "@electric-sql/pglite": "^0.5.8",
++ "better-sqlite3": ">=11",
++ "iovalkey": "^0.4.0",
++ "pg": "^8.23.0"
++ },
++ "peerDependenciesMeta": {
++ "@electric-sql/pglite": {
++ "optional": true
++ },
++ "better-sqlite3": {
++ "optional": true
++ },
++ "iovalkey": {
++ "optional": true
++ },
++ "pg": {
++ "optional": true
++ }
}
},
"node_modules/openvibe-shared": {
@@@ -1031,6 +1142,134 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
},
++ "node_modules/pg": {
++ "version": "8.23.0",
++ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
++ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
++ "license": "MIT",
++ "dependencies": {
++ "pg-connection-string": "^2.14.0",
++ "pg-pool": "^3.14.0",
++ "pg-protocol": "^1.16.0",
++ "pg-types": "2.2.0",
++ "pgpass": "1.0.5"
++ },
++ "engines": {
++ "node": ">= 16.0.0"
++ },
++ "optionalDependencies": {
++ "pg-cloudflare": "^1.4.0"
++ },
++ "peerDependencies": {
++ "pg-native": ">=3.0.1"
++ },
++ "peerDependenciesMeta": {
++ "pg-native": {
++ "optional": true
++ }
++ }
++ },
++ "node_modules/pg-cloudflare": {
++ "version": "1.4.0",
++ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
++ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
++ "license": "MIT",
++ "optional": true
++ },
++ "node_modules/pg-connection-string": {
++ "version": "2.14.0",
++ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
++ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
++ "license": "MIT"
++ },
++ "node_modules/pg-int8": {
++ "version": "1.0.1",
++ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
++ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
++ "license": "ISC",
++ "engines": {
++ "node": ">=4.0.0"
++ }
++ },
++ "node_modules/pg-pool": {
++ "version": "3.14.0",
++ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
++ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
++ "license": "MIT",
++ "peerDependencies": {
++ "pg": ">=8.0"
++ }
++ },
++ "node_modules/pg-protocol": {
++ "version": "1.16.0",
++ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
++ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
++ "license": "MIT"
++ },
++ "node_modules/pg-types": {
++ "version": "2.2.0",
++ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
++ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
++ "license": "MIT",
++ "dependencies": {
++ "pg-int8": "1.0.1",
++ "postgres-array": "~2.0.0",
++ "postgres-bytea": "~1.0.0",
++ "postgres-date": "~1.0.4",
++ "postgres-interval": "^1.1.0"
++ },
++ "engines": {
++ "node": ">=4"
++ }
++ },
++ "node_modules/pgpass": {
++ "version": "1.0.5",
++ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
++ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
++ "license": "MIT",
++ "dependencies": {
++ "split2": "^4.1.0"
++ }
++ },
++ "node_modules/postgres-array": {
++ "version": "2.0.0",
++ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
++ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
++ "license": "MIT",
++ "engines": {
++ "node": ">=4"
++ }
++ },
++ "node_modules/postgres-bytea": {
++ "version": "1.0.1",
++ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
++ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
++ "license": "MIT",
++ "engines": {
++ "node": ">=0.10.0"
++ }
++ },
++ "node_modules/postgres-date": {
++ "version": "1.0.7",
++ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
++ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
++ "license": "MIT",
++ "engines": {
++ "node": ">=0.10.0"
++ }
++ },
++ "node_modules/postgres-interval": {
++ "version": "1.2.0",
++ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
++ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
++ "license": "MIT",
++ "dependencies": {
++ "xtend": "^4.0.0"
++ },
++ "engines": {
++ "node": ">=0.10.0"
++ }
++ },
"node_modules/prebuild-install": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@@ -1154,6 +1393,27 @@
"node": ">= 6"
}
},
++ "node_modules/redis-errors": {
++ "version": "1.2.0",
++ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
++ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
++ "license": "MIT",
++ "engines": {
++ "node": ">=4"
++ }
++ },
++ "node_modules/redis-parser": {
++ "version": "3.0.0",
++ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
++ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
++ "license": "MIT",
++ "dependencies": {
++ "redis-errors": "^1.0.0"
++ },
++ "engines": {
++ "node": ">=4"
++ }
++ },
"node_modules/require-from-string": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@@ -1369,6 +1629,21 @@
"simple-concat": "^1.0.0"
}
},
++ "node_modules/split2": {
++ "version": "4.2.0",
++ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
++ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
++ "license": "ISC",
++ "engines": {
++ "node": ">= 10.x"
++ }
++ },
++ "node_modules/standard-as-callback": {
++ "version": "2.1.0",
++ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
++ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
++ "license": "MIT"
++ },
"node_modules/statuses": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@@ -1496,6 +1771,15 @@
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
++ },
++ "node_modules/xtend": {
++ "version": "4.0.2",
++ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
++ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
++ "license": "MIT",
++ "engines": {
++ "node": ">=0.4"
++ }
}
}
}
ddiff --git a/package.json b/package.json
iindex 3941d11..d5cfe31 100644
---- a/package.json
++++ b/package.json
@@@ -8,10 +8,17 @@
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"test": "node test/run.js",
++ "test:pg": "TIPS_TEST_STORE=pg node test/run.js",
"subscribe": "node scripts/subscribe.js",
-- "import-live": "node scripts/import-live.js"
++ "import-live": "node scripts/import-live.js",
++ "migrate-to-postgres": "node scripts/migrate-to-postgres.js"
},
-- "keywords": ["openvibe", "tips", "overlays", "creator"],
++ "keywords": [
++ "openvibe",
++ "tips",
++ "overlays",
++ "creator"
++ ],
"author": "OpenVibers",
"license": "AGPL-3.0-only",
"repository": "github:OpenVibers/OpenVibe.Tips",
@@@ -20,12 +27,17 @@
"cookie-parser": "^1.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
++ "iovalkey": "^0.4.0",
"jsonwebtoken": "^9.0.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
++ "pg": "^8.23.0"
},
"engines": {
"node": ">=22.0.0"
++ },
++ "devDependencies": {
++ "@electric-sql/pglite": "^0.5.8"
}
}
ddiff --git a/scripts/import-live.js b/scripts/import-live.js
iindex 09a3e55..62d96a3 100644
---- a/scripts/import-live.js
++++ b/scripts/import-live.js
@@@ -12,11 +12,15 @@
*
* With --billing-db the report ends with the reconciliation: per creator, Tips' settled total must
* equal Billing's; the exit code is 1 when it does not. Safe to re-run.
++ *
++ * Tips' side is its PostgreSQL database (DATABASE_URL; migrations applied with DATABASE_DIRECT_URL
++ * first). The two snapshots are other services' SQLite files, which is why this operator tool, and
++ * scripts/migrate-to-postgres.js, are the only code here that loads better-sqlite3.
*/
const path = require('path');
const Database = require('better-sqlite3');
const { loadConfig } = require('../server/config');
--const { openDb } = require('../server/db');
++const { openDb, migrate } = require('../server/db');
const { createIdentity } = require('../server/network');
const { createTipsOutbox } = require('../server/events/outbox');
const { createDomain } = require('../server/domain');
@@@ -30,10 +34,11 @@ async function main() {
const livePath = opt('live-db');
if (!livePath) { console.error('usage: import-live.js --live-db <snapshot> [--billing-db <snapshot>] [--dry-run] [--json]'); process.exit(2); }
const config = loadConfig();
-- for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.dbPath)) throw new Error(`${p} is the Tips database`);
++ for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.sqlitePath)) throw new Error(`${p} is Tips' own (pre-PostgreSQL) database`);
const live = new Database(livePath, { readonly: true, fileMustExist: true });
const bdb = opt('billing-db') ? new Database(opt('billing-db'), { readonly: true, fileMustExist: true }) : null;
-- const db = openDb(config.dbPath);
++ const db = openDb(config);
++ await migrate(config, { serving: db });
const outbox = createTipsOutbox({ db, config });
const domain = createDomain({ db, config, outbox, billing: null, adapters: {}, log: console });
const identity = createIdentity(config);
@@@ -55,7 +60,7 @@ async function main() {
}
live.close();
if (bdb) bdb.close();
-- db.close();
++ await db.close();
if (report.reconciliation && !report.reconciliation.ok) process.exit(1);
}
ddiff --git a/scripts/migrate-to-postgres.js b/scripts/migrate-to-postgres.js
nnew file mode 100644
iindex 0000000..0a15448
---- /dev/null
++++ b/scripts/migrate-to-postgres.js
@@@ -0,0 +1,142 @@
++#!/usr/bin/env node
++'use strict';
++/**
++ * The one-time move of Tips' SQLite database into its PostgreSQL schema (ADR-035; the procedure is
++ * openvibe-sdk docs/migrating-to-postgresql.md, section 6).
++ *
++ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
++ *
++ * 1. applies migrations/ as the owner (DATABASE_DIRECT_URL, a direct connection);
++ * 2. copies every table with openvibe-sdk's importSqlite (batched, parents first, identities kept and
++ * their sequences advanced) into emptied tables (truncate: a rehearsal can be repeated);
++ * 3. verifies each table's row count and a checksum of every row on both sides, prints the report,
++ * and exits 1 unless report.ok.
++ *
++ * The SQLite file (TIPS_DB_PATH, config.sqlitePath; --sqlite for a copy) is opened read-only: nothing in
++ * it changes. --pglite imports into an in-memory PostgreSQL instead (a rehearsal with nothing to set up;
++ * the report is all it leaves). --json prints the whole report.
++ *
++ * In production it runs once, from the new release's directory, as the service user with the service's
++ * environment, while the service is stopped (the write freeze), before the PostgreSQL release starts.
++ *
++ * What does not map one to one:
++ * settings not carried: the SQLite bookkeeping row (schema_version 3); ov_migrations
++ * records the schema now (migrations/0001_initial.sql)
++ * event_outbox → tips_event_outbox (the SDK outbox, outboxSchema('tips_event_outbox'))
++ * idempotency_receipts → tips_event_inbox (the SDK inbox, inboxSchema('tips_event_inbox'))
++ * JSON text 'null' → SQL NULL in the reference columns (interaction_effects.result,
++ * paid_messages.chat_ref, paid_media_requests.queue_ref): the SQLite code
++ * wrote JSON.stringify(null) when an adapter returned no reference, the
++ * PostgreSQL code writes NULL, and both mean "no reference"
++ * NUL, unpaired surrogate PostgreSQL stores no NUL character in text or jsonb and no unpaired surrogate
++ * in jsonb (SQLite took both; the code strips them now): a NUL is dropped and an
++ * unpaired surrogate becomes U+FFFD, in text and inside JSON values alike. The
++ * report lists every column where that happened and how many values changed
++ * No column is dropped.
++ */
++const path = require('path');
++const { createDb, importSqlite } = require('openvibe-sdk/db');
++const { loadConfig } = require('../server/config');
++const { migrate, MIGRATIONS } = require('../server/db');
++
++const args = process.argv.slice(2);
++const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };
++const flag = (name) => args.includes(`--${name}`);
++
++/** A row map turning the JSON text 'null' into SQL NULL for these columns. */
++const jsonNullToSql = (...cols) => (row) => {
++ for (const c of cols) if (row[c] === 'null') row[c] = null;
++ return row;
++};
++
++const TABLES = {
++ tips_event_outbox: { from: 'event_outbox' },
++ tips_event_inbox: { from: 'idempotency_receipts' },
++ interaction_effects: { map: jsonNullToSql('result') },
++ paid_messages: { map: jsonNullToSql('chat_ref') },
++ paid_media_requests: { map: jsonNullToSql('queue_ref') },
++};
++
++const cleanText = (v) => v.toWellFormed().replace(/\u0000/g, '');
++function cleanJson(v) {
++ if (typeof v === 'string') return cleanText(v);
++ if (Array.isArray(v)) return v.map(cleanJson);
++ if (v && typeof v === 'object') return Object.fromEntries(Object.entries(v).map(([k, x]) => [cleanText(k), cleanJson(x)]));
++ return v;
++}
++
++/**
++ * importSqlite's `tables` for this schema: TABLES, and for every table a map that makes its text and
++ * JSON values storable (see above). `cleaned` counts the distinct values changed per column (the
++ * verification sees the same mapped values on both sides, so this is the record of what changed).
++ */
++async function importOptions(db) {
++ const cols = await db.many(`SELECT table_name, column_name, udt_name FROM information_schema.columns
++ WHERE table_schema = current_schema() AND table_name <> 'ov_migrations' AND udt_name IN ('text', 'varchar', 'json', 'jsonb')`);
++ const byTable = new Map();
++ for (const c of cols) { if (!byTable.has(c.table_name)) byTable.set(c.table_name, []); byTable.get(c.table_name).push(c); }
++ const cleaned = new Map(); // "table.column" → Set of the original values changed
++ const note = (t, c, v) => { const k = `${t}.${c}`; if (!cleaned.has(k)) cleaned.set(k, new Set()); cleaned.get(k).add(v); };
++ const tables = {};
++ for (const [table, list] of byTable) {
++ const base = TABLES[table] || {};
++ tables[table] = {
++ ...base,
++ map(row) {
++ const r = base.map ? base.map(row) : row;
++ if (!r) return r;
++ for (const { column_name: c, udt_name: udt } of list) {
++ const v = r[c];
++ if (typeof v !== 'string') continue;
++ if (udt === 'json' || udt === 'jsonb') {
++ let parsed;
++ try { parsed = JSON.parse(v); } catch { continue; } // left for the importer to report
++ const fixed = JSON.stringify(cleanJson(parsed));
++ if (fixed !== JSON.stringify(parsed)) { note(table, c, v); r[c] = fixed; }
++ } else {
++ const fixed = cleanText(v);
++ if (fixed !== v) { note(table, c, v); r[c] = fixed; }
++ }
++ }
++ return r;
++ },
++ };
++ }
++ return { tables, cleaned: () => [...cleaned].map(([column, values]) => ({ column, values: values.size })) };
++}
++
++async function main() {
++ const config = loadConfig();
++ const sqlite = path.resolve(opt('sqlite') || config.sqlitePath);
++ const quiet = { log() {}, warn: console.warn, error: console.error };
++ let owner;
++ if (flag('pglite')) {
++ owner = createDb({ pglite: true, service: 'tips-import' });
++ await owner.migrate({ dir: MIGRATIONS, log: quiet });
++ } else {
++ await migrate(config, { log: quiet });
++ owner = createDb({ url: config.db.directUrl, service: 'tips-import', max: 2 });
++ }
++ try {
++ const t0 = Date.now();
++ const options = await importOptions(owner);
++ const report = await importSqlite({ sqlite, db: owner, truncate: true, tables: options.tables, log: quiet });
++ const cleaned = options.cleaned();
++ if (flag('json')) console.log(JSON.stringify({ sqlite, into: owner.store, ...report, cleaned }, null, 2));
++ else {
++ console.log(`import ${sqlite} → ${owner.store} (${Date.now() - t0} ms): ${report.ok ? 'OK' : 'PROBLEMS'}`);
++ for (const t of report.tables) console.log(` ${t.table.padEnd(24)} ${String(t.rows).padStart(8)} rows ${t.checksum || '-'}${t.source !== t.table ? ` (from ${t.source})` : ''}`);
++ for (const c of cleaned) console.log(` cleaned: ${c.column}: ${c.values} value(s) with a NUL or an unpaired surrogate`);
++ for (const p of report.problems) console.log(` problem: ${p.table}: ${p.problem}`);
++ }
++ return report.ok ? 0 : 1;
++ } finally {
++ await owner.close();
++ }
++}
++
++if (require.main === module) {
++ main().then((code) => process.exit(code), (e) => { console.error(`migrate-to-postgres failed: ${e.message}`); process.exit(1); });
++}
++
++module.exports = { TABLES, importOptions };
ddiff --git a/server/api/actor-limits.js b/server/api/actor-limits.js
iindex 12b1258..9ee4213 100644
---- a/server/api/actor-limits.js
++++ b/server/api/actor-limits.js
@@@ -8,13 +8,16 @@
* problem+json `rate_limited` with Retry-After before it does any work (before the idempotency store,
* Billing or chat delivery); the refusal is logged once and counted in tips_rate_limited_total{limit,window}.
* Reads get TIPS_LIMITS_MINUTE / TIPS_LIMITS_HOUR (120 and 3000); writes and payments set their own,
-- * tighter numbers in api/v1.js. Counters live in this process: a restart forgets them.
++ * tighter numbers in api/v1.js. With VALKEY_URL the counters are shared by every process and host
++ * (openvibe-sdk createValkeyLimitStore: one atomic script per request); if Valkey fails, a request is
++ * counted by its process's own counters instead, so an outage neither opens nor closes the gates.
++ * Without Valkey they live in this process (one process only).
*
* Never limited: /api/health, /api/ready, /release.json, /metrics, the overlays (a scoped token each,
* capped by TIPS_OVERLAY_MAX_STREAMS) and the signed Events deliveries at /internal/events (Billing's
* settlements: Events pushes at its own pace, and a 429 would only make it retry and fall behind).
*/
--const { createActorLimiter, defaultActor } = require('openvibe-sdk/limits');
++const { createActorLimiter, createValkeyLimitStore, defaultActor } = require('openvibe-sdk/limits');
function actor(req) {
const p = req.principal;
@@@ -24,7 +27,7 @@ function actor(req) {
}
/** limits(name, own) middleware for one app, plus limits.reads(name): the defaults on GET/HEAD. */
--function createActorLimits({ config, now = () => Date.now(), registry = null, log = console }) {
++function createActorLimits({ config, valkey = null, now = () => Date.now(), registry = null, log = console }) {
const refused = registry
? registry.counter({ name: 'tips_rate_limited_total', help: 'Requests refused 429 by a per-actor limit, by limit name and window', labelNames: ['limit', 'window'] })
: null;
@@@ -32,6 +35,8 @@ function createActorLimits({ config, now = () => Date.now(), registry = null, lo
limits: { minute: config.actorLimits.minute, hour: config.actorLimits.hour },
actor,
now,
++ store: createValkeyLimitStore(valkey),
++ log,
onLimited(e) {
// The actor is a principal, a subject id or an address, never a token.
log.warn(`[Tips] limit ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);
ddiff --git a/server/api/idempotency.js b/server/api/idempotency.js
iindex 4aa171b..95a446e 100644
---- a/server/api/idempotency.js
++++ b/server/api/idempotency.js
@@@ -6,12 +6,21 @@
* same key and request returns it verbatim (Idempotent-Replayed: true) and does nothing; the same
* key with another request is 422 idempotency.key_reused. Refusals are not stored.
*
++ * The key is claimed before the handler runs (a row with status 0), so the same key sent twice at once,
++ * to one process or two, runs the handler once: the other request is answered 409
++ * idempotency.in_progress with Retry-After, and a retry after the first finished gets the stored answer.
++ * A 2xx answer is written into the claim before it is sent; any other outcome releases the claim. A
++ * claim older than a minute (its process died mid-request) may be taken over.
++ *
* The scoped key also becomes the interaction's idempotency_key (UNIQUE), the second line of defence.
*/
const crypto = require('crypto');
const { http } = require('openvibe-contracts');
++const { sql } = require('openvibe-sdk/db');
const KEY_RE = /^[A-Za-z0-9._:-]{8,200}$/;
++const CLAIM_MS = 60_000;
++const PENDING = 0; // api_idempotency.status of a claim whose answer is not in yet
function stable(v) {
if (Array.isArray(v)) return `[${v.map(stable).join(',')}]`;
@@@ -21,31 +30,62 @@ function stable(v) {
const principalKey = (p) => (p.kind === 'service' ? p.sub : p.kind === 'user' ? p.subject : 'anonymous');
--function idempotent(db, now) {
-- return function idempotencyKey(req, res, next) {
++function idempotent(db, now, log = console) {
++ const iso = (ms) => new Date(ms).toISOString();
++ return async function idempotencyKey(req, res, next) {
const key = req.get('Idempotency-Key');
if (!key || !KEY_RE.test(key)) {
return http.sendProblem(res, 400, 'idempotency.key_required', { detail: 'mutating calls need an Idempotency-Key header (8-200 of A-Z a-z 0-9 . _ : -)', ctx: req.ov });
}
const scoped = `${principalKey(req.principal)}:${key}`;
const hash = crypto.createHash('sha256').update(`${req.method} ${req.baseUrl}${req.path}\n${stable(req.body || {})}`).digest('hex');
-- const row = db.prepare('SELECT * FROM api_idempotency WHERE key = ?').get(scoped);
-- if (row) {
-- if (row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });
++ const path = `${req.baseUrl}${req.path}`;
++ let claimed;
++ let row = null;
++ try {
++ const at = now();
++ claimed = await db.maybe(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
++ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${PENDING}, '', ${iso(at)})
++ ON CONFLICT (key) DO UPDATE SET request_hash = excluded.request_hash, method = excluded.method, path = excluded.path, created_at = excluded.created_at
++ WHERE api_idempotency.status = ${PENDING} AND api_idempotency.created_at < ${iso(at - CLAIM_MS)}
++ RETURNING key`);
++ if (!claimed) row = await db.maybe(sql`SELECT request_hash, status, response FROM api_idempotency WHERE key = ${scoped}`);
++ } catch (e) { return next(e); }
++ if (!claimed) {
++ if (row && row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });
++ if (!row || row.status === PENDING) {
++ res.setHeader('Retry-After', '1');
++ return http.sendProblem(res, 409, 'idempotency.in_progress', { detail: 'a request with this Idempotency-Key is still being handled; retry shortly for its answer', ctx: req.ov });
++ }
res.setHeader('Idempotent-Replayed', 'true');
-- return res.status(row.status).json(JSON.parse(row.response));
++ // The stored text is the answer as first sent (jsonb would reorder its keys).
++ return res.status(row.status).type('application/json').send(row.response);
}
req.idempotencyKey = `api:${scoped}`;
++ let answered = false;
++ const release = () => {
++ if (answered) return;
++ answered = true;
++ db.exec(sql`DELETE FROM api_idempotency WHERE key = ${scoped} AND status = ${PENDING} AND request_hash = ${hash}`)
++ .catch((e) => log.warn(`[Tips] Idempotency-Key claim not released: ${e.message}`));
++ };
++ res.on('finish', release);
++ res.on('close', release);
const json = res.json.bind(res);
res.json = (body) => {
-- if (res.statusCode >= 200 && res.statusCode < 300) {
-- // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the
-- // replay gets that instead, so no secret is kept in plain text here.
-- const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;
-- db.prepare('INSERT OR IGNORE INTO api_idempotency (key, request_hash, method, path, status, response, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')
-- .run(scoped, hash, req.method, `${req.baseUrl}${req.path}`, res.statusCode, JSON.stringify(stored), new Date(now()).toISOString());
-- }
-- return json(body);
++ if (answered || res.statusCode < 200 || res.statusCode >= 300) return json(body);
++ answered = true;
++ // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the
++ // replay gets that instead, so no secret is kept in plain text here. An upsert: an erasure
++ // during the request may have removed the claim.
++ const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;
++ db.exec(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
++ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${res.statusCode}, ${JSON.stringify(stored)}, ${iso(now())})
++ ON CONFLICT (key) DO UPDATE SET status = excluded.status, response = excluded.response, created_at = excluded.created_at
++ WHERE api_idempotency.request_hash = excluded.request_hash`)
++ .catch((e) => log.warn(`[Tips] stored answer for an Idempotency-Key not kept: ${e.message}`))
++ .then(() => json(body));
++ return res;
};
return next();
};
@@@ -57,7 +97,7 @@ function idempotent(db, now) {
*/
const RETAIN_MS = 7 * 24 * 3600 * 1000;
function pruneAnswers(db, now = Date.now(), retainMs = RETAIN_MS) {
-- return db.prepare('DELETE FROM api_idempotency WHERE created_at < ?').run(new Date(now - retainMs).toISOString()).changes;
++ return db.exec(sql`DELETE FROM api_idempotency WHERE created_at < ${new Date(now - retainMs).toISOString()}`);
}
module.exports = { idempotent, stable, principalKey, pruneAnswers, RETAIN_MS };
ddiff --git a/server/api/v1.js b/server/api/v1.js
iindex 5c20e17..9812354 100644
---- a/server/api/v1.js
++++ b/server/api/v1.js
@@@ -40,7 +40,7 @@
*/
const express = require('express');
const { http } = require('openvibe-contracts');
--const { TipsError, fail, userSubject, displayName } = require('../util');
++const { TipsError, fail, userSubject, displayName, inputError } = require('../util');
const { idempotent } = require('./idempotency');
const CAP = {
@@@ -67,7 +67,7 @@ const CAP = {
function v1Router({ domain, apiAuth, limits }) {
const r = express.Router();
const { db, profiles, goals, interactions, overlays } = domain;
-- const idem = idempotent(db, domain.now);
++ const idem = idempotent(db, domain.now, domain.log);
// Per-actor limits (api/actor-limits.js), after the credential and before any work: every read takes
// the defaults; each write below names its own, tighter numbers.
@@@ -86,40 +86,40 @@ function v1Router({ domain, apiAuth, limits }) {
return null;
}
/** The creator a call is about: body/query `creator` (subject or handle), or the signed-in user. */
-- function creatorOf(req, value, { mustExist = true } = {}) {
++ async function creatorOf(req, value, { mustExist = true } = {}) {
const v = value != null && value !== '' ? value : (req.principal.kind === 'user' ? req.principal.subject : null);
if (!v) fail(422, 'tips.invalid_input', 'creator is required');
const ref = typeof v === 'object' ? userSubject(v, 'creator') : String(v);
-- const profile = ref === 'me' && req.principal.kind === 'user' ? profiles.bySubject(req.principal.subject) : profiles.resolve(ref);
++ const profile = ref === 'me' && req.principal.kind === 'user' ? await profiles.bySubject(db, req.principal.subject) : await profiles.resolve(db, ref);
if (!profile && mustExist) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
return profile;
}
const write = (handler) => [idem, wrap(handler)];
// ── Profiles ─────────────────────────────────────────────
-- r.get('/profiles/:creator', wrap((req, res) => {
-- const p = creatorOf(req, req.params.creator);
++ r.get('/profiles/:creator', wrap(async (req, res) => {
++ const p = await creatorOf(req, req.params.creator);
const who = req.principal;
const mine = who.kind === 'user' && who.subject === p.creator_subject;
const svc = who.kind === 'service' && apiAuth.granted(who, CAP.profileGet);
// A page that is switched off does not exist for anyone but its creator and granted services.
if (!p.page_enabled && !mine && !svc) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
const owner = mine || svc;
-- const active = goals.list(p.creator_subject, { status: 'active' });
-- res.json({ profile: profiles.present(p, { owner }), goals: active.map((g) => (owner ? goals.present(g) : goals.publicGoal(g, p.page))) });
++ const active = await goals.list(db, p.creator_subject, { status: 'active' });
++ res.json({ profile: profiles.present(p, { owner }), goals: owner ? await goals.presentMany(db, active) : await goals.publicGoals(db, active, p) });
}));
/** The supporters page: top supporters and recent public messages, as the creator chose to show them. */
-- r.get('/profiles/:creator/supporters', wrap((req, res) => {
-- const p = creatorOf(req, req.params.creator);
++ r.get('/profiles/:creator/supporters', wrap(async (req, res) => {
++ const p = await creatorOf(req, req.params.creator);
const who = req.principal;
const owner = (who.kind === 'user' && who.subject === p.creator_subject) || (who.kind === 'service' && apiAuth.granted(who, CAP.profileGet));
if (!owner && (!p.page_enabled || !p.page.supporters_page)) fail(404, 'tips.supporters_not_public', 'this creator does not show their supporters');
const page = p.page;
res.json({
creator: { type: 'user', id: p.creator_subject }, page,
-- leaderboard: interactions.leaderboard(p.creator_subject).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),
-- recent: page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,
++ leaderboard: (await interactions.leaderboard(db, p)).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),
++ recent: page.supporters_messages ? (await interactions.recentPublic(db, p)).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,
});
}));
@@@ -128,34 +128,34 @@ function v1Router({ domain, apiAuth, limits }) {
if (req.principal.kind !== 'user') fail(req.principal.kind === 'anonymous' ? 401 : 403, req.principal.kind === 'anonymous' ? 'token.missing' : 'tips.people_only', 'only the person themselves (a Network user token) can do this');
return req.principal.subject;
};
-- r.get('/me/export', wrap((req, res) => res.json(interactions.exportFor(person(req)))));
++ r.get('/me/export', wrap(async (req, res) => res.json(await interactions.exportFor(person(req)))));
// A person erasing their own tips does it once; the network's account deletion arrives through
// /internal/events, which is never limited.
-- r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write((req, res) => res.json(interactions.erase(person(req)))));
++ r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write(async (req, res) => res.json(await interactions.erase(person(req)))));
-- r.patch('/profiles/:creator', edit, ...write((req, res) => {
++ r.patch('/profiles/:creator', edit, ...write(async (req, res) => {
const b = req.body || {};
let p;
if (req.params.creator === 'me') {
if (req.principal.kind !== 'user') fail(401, 'token.missing', '`me` needs a user token');
-- p = profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });
-- } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !profiles.bySubject(req.params.creator)) {
++ p = await profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });
++ } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !(await profiles.bySubject(db, req.params.creator))) {
allow(req, CAP.profileUpdate);
// A service opening a profile for a creator names the account projection it knows.
if (!b.handle) fail(422, 'tips.invalid_input', 'handle is required to create a profile');
-- p = profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });
++ p = await profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });
} else {
-- p = creatorOf(req, req.params.creator);
++ p = await creatorOf(req, req.params.creator);
}
allow(req, CAP.profileUpdate, p.creator_subject);
-- const out = profiles.update(p.creator_subject, b, { expectedRevision: b.revision });
++ const out = await profiles.update(p.creator_subject, b, { expectedRevision: b.revision });
res.json({ profile: profiles.present(out, { owner: true }) });
}));
-- r.get('/profiles/:creator/totals', wrap((req, res) => {
-- const p = creatorOf(req, req.params.creator);
++ r.get('/profiles/:creator/totals', wrap(async (req, res) => {
++ const p = await creatorOf(req, req.params.creator);
allow(req, CAP.interactionList, p.creator_subject);
-- res.json({ creator: { type: 'user', id: p.creator_subject }, totals: interactions.totals(p.creator_subject) });
++ res.json({ creator: { type: 'user', id: p.creator_subject }, totals: await interactions.totals(db, p.creator_subject) });
}));
// ── Paid requests (checkout / credit) ────────────────────
@@@ -174,28 +174,27 @@ function v1Router({ domain, apiAuth, limits }) {
supporter = p.subject;
supporterName = displayName(b.supporter_name) || p.name || p.username;
}
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
const funding = b.pay_with || 'credit';
const tp = req.ov && req.ov.traceparent;
-- const { interaction, replay } = interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });
++ const { interaction, replay } = await interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });
let i = interaction;
-- let checkout = null;
++ const viewer = p.kind === 'service' ? 'service' : 'supporter';
if (!replay) {
if (funding === 'credit') {
const out = await interactions.transfer(i, { traceparent: tp });
-- i = interactions.get(i.id);
++ i = await interactions.get(db, i.id);
if (out.refused) {
-- fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: interactions.present(i, { viewer: 'supporter' }) });
++ fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });
}
} else {
-- checkout = await interactions.startCheckout(i, { provider: b.provider, traceparent: tp });
-- i = checkout.interaction;
++ i = (await interactions.startCheckout(i, { provider: b.provider, traceparent: tp })).interaction;
}
} else if (i.payment_state === 'failed') {
-- fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: interactions.present(i, { viewer: 'supporter' }) });
++ fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });
}
res.status(replay ? 200 : 201).json({
-- interaction: interactions.present(i, { viewer: p.kind === 'service' ? 'service' : 'supporter' }),
++ interaction: await interactions.present(db, i, { viewer }),
checkout_url: i.checkout_url || null, checkout_ref: i.checkout_ref || null,
});
});
@@@ -212,27 +211,29 @@ function v1Router({ domain, apiAuth, limits }) {
r.post('/media-requests', pays, ...paid('media_request', CAP.media));
// ── Interactions ─────────────────────────────────────────
-- r.get('/interactions', wrap((req, res) => {
++ // The hottest list: a keyset page of any size costs three queries (the page, the creators' filters,
++ // the effects) — test/query-budget.test.js holds it there.
++ r.get('/interactions', wrap(async (req, res) => {
const p = req.principal;
const q = req.query;
let filter;
if (p.kind === 'service') {
allow(req, CAP.interactionList);
filter = {
-- creator: q.creator ? (creatorOf(req, q.creator).creator_subject) : null,
++ creator: q.creator ? (await creatorOf(req, q.creator)).creator_subject : null,
supporter: q.supporter ? userSubject(String(q.supporter), 'supporter') : null,
};
if (!filter.creator && !filter.supporter) fail(422, 'tips.invalid_input', 'creator or supporter is required');
} else if (p.kind === 'user') {
filter = q.as === 'creator' ? { creator: p.subject } : { supporter: p.subject };
} else fail(401, 'token.missing', 'sign in to see receipts');
-- const out = interactions.list({ ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });
++ const out = await interactions.list(db, { ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });
const viewer = p.kind === 'service' ? 'service' : (filter.creator ? 'owner' : 'supporter');
-- res.json({ interactions: out.rows.map((i) => interactions.present(i, { viewer })), next_cursor: out.next_cursor });
++ res.json({ interactions: await interactions.presentMany(db, out.rows, { viewer }), next_cursor: out.next_cursor });
}));
-- r.get('/interactions/:id', wrap((req, res) => {
-- const i = interactions.get(req.params.id);
++ r.get('/interactions/:id', wrap(async (req, res) => {
++ const i = await interactions.get(db, req.params.id);
const p = req.principal;
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
let viewer;
@@@ -241,164 +242,160 @@ function v1Router({ domain, apiAuth, limits }) {
else if (p.kind === 'user' && p.subject === i.supporter_subject) viewer = 'supporter';
else if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in to see this receipt');
else fail(404, 'tips.interaction_not_found', 'no such interaction');
-- res.json({ interaction: interactions.present(i, { viewer }) });
++ res.json({ interaction: await interactions.present(db, i, { viewer }) });
}));
// A service importing tips made elsewhere records one per tip, at most one a second.
-- r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write((req, res) => {
++ r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write(async (req, res) => {
allow(req, CAP.interactionRecord);
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
-- const out = interactions.recordExternal(profile, b);
-- res.status(out.replay ? 200 : 201).json({ interaction: interactions.present(out.interaction, { viewer: 'service' }), duplicate: out.replay });
++ const profile = await creatorOf(req, b.creator);
++ const out = await interactions.recordExternal(profile, b);
++ res.status(out.replay ? 200 : 201).json({ interaction: await interactions.present(db, out.interaction, { viewer: 'service' }), duplicate: out.replay });
}));
// ── Goals ────────────────────────────────────────────────
-- function goalVisible(req, g) {
-- const profile = profiles.bySubject(g.creator_subject);
-- if (profile && profile.page_enabled) return true;
-- const p = req.principal;
-- return (p.kind === 'user' && p.subject === g.creator_subject) || (p.kind === 'service' && apiAuth.granted(p, CAP.goalUpdate));
-- }
-- const goalOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));
-- r.get('/goals', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
-- const sample = { creator_subject: profile.creator_subject };
-- if (!goalVisible(req, sample)) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
++ const isOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));
++ r.get('/goals', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
++ const full = isOwner(req, profile.creator_subject);
++ if (!profile.page_enabled && !full) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
const status = ['active', 'closed'].includes(req.query.status) ? req.query.status : undefined;
-- const full = goalOwner(req, profile.creator_subject);
-- res.json({ goals: goals.list(profile.creator_subject, { status }).map((g) => (full ? goals.present(g) : goals.publicGoal(g, profile.page))) });
++ const list = await goals.list(db, profile.creator_subject, { status });
++ res.json({ goals: full ? await goals.presentMany(db, list) : await goals.publicGoals(db, list, profile) });
}));
-- r.get('/goals/:id', wrap((req, res) => {
-- const g = goals.get(req.params.id);
-- if (!g || !goalVisible(req, g)) fail(404, 'tips.goal_not_found', 'no such goal');
-- if (!goalOwner(req, g.creator_subject)) return res.json({ goal: goals.publicGoal(g, profiles.bySubject(g.creator_subject).page) });
-- res.json({ goal: goals.present(g, { contributions: true }) });
++ r.get('/goals/:id', wrap(async (req, res) => {
++ const g = await goals.get(db, req.params.id);
++ const profile = g && await profiles.bySubject(db, g.creator_subject);
++ const full = g && isOwner(req, g.creator_subject);
++ if (!g || (!(profile && profile.page_enabled) && !full)) fail(404, 'tips.goal_not_found', 'no such goal');
++ if (!full) return res.json({ goal: await goals.publicGoal(db, g, profile) });
++ res.json({ goal: await goals.present(db, g, { contributions: true }) });
}));
-- r.post('/goals', edit, ...write((req, res) => {
++ r.post('/goals', edit, ...write(async (req, res) => {
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
allow(req, CAP.goalCreate, profile.creator_subject);
-- res.status(201).json({ goal: goals.create(profile.creator_subject, b) });
++ res.status(201).json({ goal: await goals.create(profile.creator_subject, b) });
}));
-- const ownGoal = (req, cap) => {
-- const g = goals.get(req.params.id);
++ const ownGoal = async (req, cap) => {
++ const g = await goals.get(db, req.params.id);
if (!g) fail(404, 'tips.goal_not_found', 'no such goal');
allow(req, cap, g.creator_subject);
return g;
};
-- r.patch('/goals/:id', edit, ...write((req, res) => res.json({ goal: goals.update(ownGoal(req, CAP.goalUpdate), req.body || {}) })));
-- r.post('/goals/:id/close', edit, ...write((req, res) => res.json({ goal: goals.close(ownGoal(req, CAP.goalClose)) })));
++ r.patch('/goals/:id', edit, ...write(async (req, res) => res.json({ goal: await goals.update(await ownGoal(req, CAP.goalUpdate), req.body || {}) })));
++ r.post('/goals/:id/close', edit, ...write(async (req, res) => res.json({ goal: await goals.close(await ownGoal(req, CAP.goalClose)) })));
// ── Overlay tokens and configs ───────────────────────────
-- r.get('/overlay-tokens', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
++ r.get('/overlay-tokens', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.tokenCreate, profile.creator_subject);
-- res.json({ tokens: overlays.listTokens(profile.creator_subject).map(overlays.presentToken) });
++ res.json({ tokens: (await overlays.listTokens(db, profile.creator_subject)).map(overlays.presentToken) });
}));
// A token opens an overlay's event stream: a creator mints a few for their scenes.
-- r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write((req, res) => {
++ r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write(async (req, res) => {
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
allow(req, CAP.tokenCreate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
-- const out = overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });
++ const out = await overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });
// Shown once: only its hash is stored, and the stored answer for an Idempotency-Key replay has no
// secret either (a caller who lost the answer revokes the token and creates another).
res.locals.storedBody = { token: out.token, secret: null, overlay_url: null, events_url: null, note: 'the secret is shown once; revoke this token and create another' };
res.status(201).json({ token: out.token, secret: out.secret, overlay_url: out.overlay_url, events_url: out.events_url });
}));
-- r.post('/overlay-tokens/:id/revoke', edit, ...write((req, res) => {
-- const t = overlays.getToken(req.params.id);
++ r.post('/overlay-tokens/:id/revoke', edit, ...write(async (req, res) => {
++ const t = await overlays.getToken(db, req.params.id);
if (!t) fail(404, 'tips.overlay_token_not_found', 'no such overlay token');
allow(req, CAP.tokenRevoke, t.creator_subject);
-- res.json({ token: overlays.revokeToken(t) });
++ res.json({ token: await overlays.revokeToken(t) });
}));
-- r.get('/overlay-configs', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
++ r.get('/overlay-configs', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.configGet, profile.creator_subject);
-- res.json({ configs: overlays.listConfigs(profile.creator_subject).map(overlays.presentConfig) });
++ res.json({ configs: (await overlays.listConfigs(db, profile.creator_subject)).map(overlays.presentConfig) });
}));
-- r.get('/overlay-configs/:id', wrap((req, res) => {
-- const c = overlays.getConfig(req.params.id);
++ r.get('/overlay-configs/:id', wrap(async (req, res) => {
++ const c = await overlays.getConfig(db, req.params.id);
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
allow(req, CAP.configGet, c.creator_subject);
res.json({ config: overlays.presentConfig(c) });
}));
-- r.post('/overlay-configs', edit, ...write((req, res) => {
++ r.post('/overlay-configs', edit, ...write(async (req, res) => {
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
allow(req, CAP.configUpdate, profile.creator_subject);
-- res.status(201).json({ config: overlays.createConfig(profile.creator_subject, b) });
++ res.status(201).json({ config: await overlays.createConfig(profile.creator_subject, b) });
}));
-- r.patch('/overlay-configs/:id', edit, ...write((req, res) => {
-- const c = overlays.getConfig(req.params.id);
++ r.patch('/overlay-configs/:id', edit, ...write(async (req, res) => {
++ const c = await overlays.getConfig(db, req.params.id);
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
allow(req, CAP.configUpdate, c.creator_subject);
-- res.json({ config: overlays.updateConfig(c, req.body || {}) });
++ res.json({ config: await overlays.updateConfig(c, req.body || {}) });
}));
// ── Moderation (server/domain/moderation.js) ─────────────
const { moderation } = domain;
/** The creator, one of their moderators, or a service with tips.interaction.moderate. */
-- function moderatorOf(req, creator, { hideExistence = false } = {}) {
++ async function moderatorOf(req, creator, { hideExistence = false } = {}) {
const p = req.principal;
if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in (a Network user token) or present a service token');
-- const a = moderation.actorFor(p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));
++ const a = await moderation.actorFor(db, p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));
if (a) return a;
if (p.kind === 'service') fail(403, 'capability.denied', `${CAP.moderate} not granted`);
if (hideExistence) fail(404, 'tips.interaction_not_found', 'no such interaction');
fail(403, 'tips.forbidden', 'only the creator and their moderators moderate this page');
return null;
}
-- r.get('/moderation', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
-- moderatorOf(req, profile.creator_subject);
-- const out = moderation.queue(profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });
++ r.get('/moderation', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
++ await moderatorOf(req, profile.creator_subject);
++ const out = await moderation.queue(db, profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });
res.json({ creator: { type: 'user', id: profile.creator_subject }, interactions: out.rows, next_cursor: out.next_cursor });
}));
-- r.get('/moderation/log', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
++ r.get('/moderation/log', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.moderate, profile.creator_subject);
-- res.json({ log: moderation.log(profile.creator_subject) });
++ res.json({ log: await moderation.log(db, profile.creator_subject) });
}));
for (const action of ['hide', 'restore']) {
-- r.post(`/interactions/:id/${action}`, moderate, ...write((req, res) => {
-- const i = interactions.get(req.params.id);
++ r.post(`/interactions/:id/${action}`, moderate, ...write(async (req, res) => {
++ const i = await interactions.get(db, req.params.id);
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
-- const actor = moderatorOf(req, i.creator_subject, { hideExistence: true });
-- const out = moderation[action](i, { ...actor, reason: (req.body || {}).reason });
-- res.json({ interaction: moderation.present(out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });
++ const actor = await moderatorOf(req, i.creator_subject, { hideExistence: true });
++ const out = await moderation[action](i, { ...actor, reason: (req.body || {}).reason });
++ res.json({ interaction: await moderation.present(db, out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });
}));
}
-- r.get('/moderators', wrap((req, res) => {
-- const profile = creatorOf(req, req.query.creator);
++ r.get('/moderators', wrap(async (req, res) => {
++ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
-- res.json({ moderators: moderation.listModerators(profile.creator_subject).map(moderation.presentModerator) });
++ res.json({ moderators: (await moderation.listModerators(db, profile.creator_subject)).map(moderation.presentModerator) });
}));
-- r.post('/moderators', staffing, ...write((req, res) => {
++ r.post('/moderators', staffing, ...write(async (req, res) => {
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
-- res.status(201).json({ moderator: moderation.addModerator(profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });
++ res.status(201).json({ moderator: await moderation.addModerator(db, profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });
}));
-- r.post('/moderators/:subject/remove', staffing, ...write((req, res) => {
-- const profile = creatorOf(req, (req.body || {}).creator);
++ r.post('/moderators/:subject/remove', staffing, ...write(async (req, res) => {
++ const profile = await creatorOf(req, (req.body || {}).creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
-- res.json(moderation.removeModerator(profile.creator_subject, req.params.subject));
++ res.json(await moderation.removeModerator(profile.creator_subject, req.params.subject));
}));
// ── Simulation ───────────────────────────────────────────
// A simulated tip plays on the creator's overlays and chat like a real one.
-- r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write((req, res) => {
++ r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write(async (req, res) => {
const b = req.body || {};
-- const profile = creatorOf(req, b.creator);
++ const profile = await creatorOf(req, b.creator);
allow(req, CAP.simulate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
-- const i = interactions.simulate(profile, b, { by });
-- res.status(201).json({ interaction: interactions.present(i, { viewer: 'owner' }) });
++ const i = await interactions.simulate(profile, b, { by });
++ res.status(201).json({ interaction: await interactions.present(db, i, { viewer: 'owner' }) });
}));
return r;
@@@ -416,6 +413,7 @@ function wrap(fn) {
function sendError(req, res, e) {
if (res.headersSent) return;
++ e = inputError(e) || e;
if (e instanceof TipsError) {
return http.sendProblem(res, e.status, e.code, { detail: e.detail || e.message, ctx: req.ov, extra: e.extra && e.status < 500 ? { details: e.extra } : undefined });
}
ddiff --git a/server/app.js b/server/app.js
iindex 948be00..4a9715b 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -11,12 +11,16 @@
* /overlay/:token[/events|/state] overlays (scoped revocable tokens)
* /, /dashboard, /receipts, /:handle … server-rendered pages
*
-- * createApp({ config, db, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.
++ * createApp({ config, db, valkey, registry, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.
++ * `db` is an openvibe-sdk/db handle with the schema migrated (server/index.js migrates at boot; tests
++ * pass a PGlite one); `valkey` an openvibe-sdk/valkey handle or null (everything shared stays in this
++ * process).
*/
const path = require('path');
const express = require('express');
const cookieParser = require('cookie-parser');
const { http } = require('openvibe-contracts');
++const { isLoopbackDirect } = require('openvibe-shared/metrics');
const { loadConfig } = require('./config');
const { openDb } = require('./db');
const { createKeyProvider, createUserAuth } = require('./network');
@@@ -32,6 +36,7 @@ const { createSessionRoutes } = require('./web/session');
const { createWebRoutes } = require('./web/routes');
const { createLayout, assetVersion } = require('./web/layout');
const pages = require('./web/pages');
++const { inputError } = require('./util');
const { createTipsReadiness, registerTipsGauges } = require('./observability');
const VERSION = require('../package.json').version;
@@@ -39,7 +44,8 @@ const PUBLIC_DIR = path.join(__dirname, '..', 'public');
function createApp(opts = {}) {
const config = opts.config || loadConfig();
-- const db = opts.db || openDb(config.dbPath);
++ const db = opts.db || openDb(config);
++ const valkey = opts.valkey || null;
const fetchImpl = opts.fetchImpl || globalThis.fetch;
const log = opts.log || console;
const now = opts.now || (() => Date.now());
@@@ -48,7 +54,7 @@ function createApp(opts = {}) {
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const adapters = opts.adapters || createAdapters(config, { fetchImpl });
const outbox = opts.outbox || createTipsOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
-- const domain = createDomain({ db, config, outbox, billing, adapters, now, log });
++ const domain = createDomain({ db, config, outbox, billing, adapters, valkey, now, log });
const apiAuth = createApiAuth({ config, keys, userAuth });
const release = require('openvibe-shared/release').createRelease({ service: 'tips', root: path.join(__dirname, '..') });
const layout = createLayout({ config, release });
@@@ -58,10 +64,13 @@ function createApp(opts = {}) {
app.set('trust proxy', config.trustProxy);
// HTTP golden signals by route template, process metrics, release_info; GET /metrics answers
// direct loopback callers only (Track O). An overlay's SSE stream is a session, not a request.
++ // The Tips gauges are read from the database just before a direct scrape renders (one query).
++ const registry = opts.registry || require('openvibe-shared/metrics').createRegistry();
++ const gauges = registerTipsGauges(registry, { db, now, log });
++ app.get('/metrics', (req, res, next) => (isLoopbackDirect(req) ? gauges.refresh().then(() => next(), () => next()) : next()));
const metrics = require('openvibe-shared/metrics').instrument(app, {
-- service: 'tips', release: release.release, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),
++ service: 'tips', release: release.release, registry, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),
});
-- registerTipsGauges(metrics.registry, { db, outbox, now });
app.use(http.middleware());
app.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
@@@ -77,12 +86,12 @@ function createApp(opts = {}) {
});
app.use(cookieParser());
-- app.get('/api/health', (req, res) => res.json({
-- ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events: outbox.status(),
-- }));
-- // Readiness (openvibe-shared/ready): 503 only when the database fails; Network key, Billing and
-- // Events are optional and degrade it (see observability.js).
-- const readiness = createTipsReadiness({ db, keys, config, outbox, release: release.release, fetchImpl });
++ app.get('/api/health', (req, res, next) => outbox.status().then((events) => res.json({
++ ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events,
++ }), next));
++ // Readiness (openvibe-shared/ready): 503 only when the database fails; Valkey, the Network key,
++ // Billing and Events are optional and degrade it (see observability.js).
++ const readiness = createTipsReadiness({ db, valkey, keys, config, outbox, release: release.release, fetchImpl });
app.get('/api/ready', readiness.handler);
// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports into /metrics.
release.mount(app, { registry: metrics.registry });
@@@ -94,7 +103,7 @@ function createApp(opts = {}) {
? http.sendProblem(res, 404, 'not_found', { ctx: req.ov }) : next()), consumer.router);
// Per-actor limits on /api/v1 (api/actor-limits.js), counted once apiAuth resolved the caller.
// opts.limitsNow: the limiter's clock (tests).
-- const limits = createActorLimits({ config, now: opts.limitsNow || now, registry: metrics.registry, log });
++ const limits = createActorLimits({ config, valkey, now: opts.limitsNow || now, registry: metrics.registry, log });
app.use('/api/v1', express.json({ limit: '64kb' }), (req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); }, apiAuth.middleware, v1Router({ domain, apiAuth, limits }));
app.use('/auth', createSessionRoutes(config, userAuth, { fetchImpl }));
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'tips', service: 'tips', host: 'openvibe.tips', name: 'OpenVibe.Tips', profile: 'ugc' })); }
@@@ -119,13 +128,18 @@ function createApp(opts = {}) {
app.use((err, req, res, next) => {
if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { ctx: req.ov });
if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });
++ const refused = inputError(err);
++ if (refused && !res.headersSent) {
++ if (req.path.startsWith('/api/')) return http.sendProblem(res, 422, refused.code, { detail: refused.detail, ctx: req.ov });
++ return res.status(400).type('html').send(layout.page({ title: 'Not accepted', robots: 'noindex', body: pages.errorPage({ status: 400, title: 'That could not be saved', message: refused.detail }) }));
++ }
log.error('[Tips] unhandled error:', err);
if (res.headersSent) return undefined;
if (req.path.startsWith('/api/')) return http.sendProblem(res, 500, 'tips.internal', { ctx: req.ov });
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
});
-- Object.assign(app.locals, { config, db, domain, keys, outbox, adapters, billing, consumer, metrics });
++ Object.assign(app.locals, { config, db, valkey, domain, keys, outbox, adapters, billing, consumer, metrics });
return app;
}
ddiff --git a/server/config.js b/server/config.js
iindex 397e104..7058f64 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -8,6 +8,7 @@
* never prices or rates. Prices and the value of a bit are Billing's.
*/
require('dotenv').config();
++const crypto = require('crypto');
const int = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) ? n : d; };
const bool = (v, d = false) => (v == null || v === '' ? d : ['1', 'true', 'yes', 'on'].includes(String(v).toLowerCase()));
@@@ -28,7 +29,23 @@ function loadConfig(env = process.env) {
host: env.HOST || '127.0.0.1',
baseUrl,
trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1,
-- dbPath: env.TIPS_DB_PATH || './data/tips.db',
++
++ // PostgreSQL (ADR-035; OpenVibe.Host's data role writes all four into /etc/openvibe/tips.env):
++ // DATABASE_URL is the pooled runtime role through PgBouncer (transaction mode), DATABASE_DIRECT_URL
++ // the owner role on a direct connection, for migrations at boot and the one-time import.
++ db: {
++ url: env.DATABASE_URL || '',
++ directUrl: env.DATABASE_DIRECT_URL || '',
++ },
++ // Valkey: shared, never authoritative (per-actor limit counters, overlay fan-out between
++ // processes, overlay stream slots). Unset: all of it stays in this process (one process only).
++ valkey: {
++ url: env.VALKEY_URL || '',
++ prefix: env.VALKEY_PREFIX || 'ov:tips:',
++ },
++ // The SQLite file of the release before PostgreSQL: read (never written) only by
++ // scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.
++ sqlitePath: env.TIPS_DB_PATH || './data/tips.db',
// Identity: service tokens and user tokens are RS256 JWTs signed by OpenVibe.Network.
network: {
@@@ -48,8 +65,11 @@ function loadConfig(env = process.env) {
scope: env.OV_OAUTH_SCOPE || 'profile',
},
cookies: { secure: env.COOKIE_SECURE != null ? bool(env.COOKIE_SECURE) : isProduction },
-- // Signs the anti-forgery tokens of the server-rendered forms.
-- formSecret: env.TIPS_FORM_SECRET || '',
++ // Signs the anti-forgery tokens of the server-rendered forms. Every process must use the same
++ // one (a form rendered by one process is posted to another), so without TIPS_FORM_SECRET it is
++ // derived from the client secret; only with neither is it per process (development).
++ formSecret: env.TIPS_FORM_SECRET
++ || (env.OV_OAUTH_CLIENT_SECRET ? crypto.createHmac('sha256', env.OV_OAUTH_CLIENT_SECRET).update('openvibe.tips form secret').digest('hex') : ''),
billing: {
url: trim(env.BILLING_URL || 'http://127.0.0.1:4600'),
ddiff --git a/server/db.js b/server/db.js
iindex c5ad3f7..4264da9 100644
---- a/server/db.js
++++ b/server/db.js
@@@ -1,346 +1,62 @@
'use strict';
/**
-- * Tips' own SQLite database (WAL), created on boot, idempotent.
++ * Tips' own PostgreSQL database (ADR-035), through openvibe-sdk/db: async, pooled, one dialect.
*
-- * The ten product tables (roadmap §15.11) store interaction state and REFERENCES to Billing
-- * transactions — never a mutable cash balance. Amounts are integer vibes-bits as Billing counts
-- * them; a creator's totals are always derived from settled interactions.
++ * The schema is migrations/NNNN_*.sql (expand/migrate/contract, ADR-028), applied at boot by
++ * migrate() on the owner's direct connection (DATABASE_DIRECT_URL); the service then serves on the
++ * pooled runtime role (DATABASE_URL, PgBouncer in transaction mode: no session state, 10 connections
++ * per process at most).
++ *
++ * The product tables (roadmap §15.11) store interaction state and REFERENCES to Billing transactions,
++ * never a mutable cash balance. Amounts are integer vibes-bits as Billing counts them; a creator's
++ * totals are always derived from settled interactions.
*
* creator_tip_profiles one per creator subject: page switch, minimums, TTS/media settings
* tip_interactions one logical interaction; payment_state (mirrored from Billing, keyed by
* Billing txn id) is separate from delivery_state (Tips' own effects)
* tip_goals creator goals
* tip_goal_contributions settled interactions counted toward a goal (never tests, never pending)
-- * paid_messages highlighted paid chat messages and TTS requests — created on settlement
-- * paid_media_requests paid media requests — created on settlement
++ * paid_messages highlighted paid chat messages and TTS requests, created on settlement
++ * paid_media_requests paid media requests, created on settlement
* overlay_configs alert / goal overlay settings
* overlay_deliveries what overlays were sent (monotonic seq per creator, replayable)
* interaction_effects the retryable delivery work of an interaction (chat line, TTS, media, overlay)
* migration_maps legacy (Live) row → Tips entity, with held/excluded reasons
*
-- * Supporting: overlay_tokens (hashed, scoped, revocable), api_idempotency, import_runs,
-- * event_outbox + idempotency_receipts (openvibe-sdk outbox and inbox), settings. Columns added after
-- * v1 are in COLUMNS below.
++ * Supporting: overlay_tokens (hashed, scoped, revocable), tip_moderators / _invites / _moderation_log,
++ * api_idempotency, import_runs, tips_event_outbox + tips_event_inbox (openvibe-sdk outbox and inbox).
++ *
++ * openDb(config) the serving handle: DATABASE_URL; in development without it, an embedded
++ * PGlite database in data/pglite (one process, nothing to install)
++ * migrate(config, {serving}) apply migrations/ with the owner role (DATABASE_DIRECT_URL), then close
++ * that connection; the embedded database migrates on its serving handle
*/
const fs = require('fs');
const path = require('path');
--const Database = require('better-sqlite3');
--
--const SCHEMA = `
--CREATE TABLE IF NOT EXISTS settings (
-- id INTEGER PRIMARY KEY CHECK (id = 1),
-- schema_version INTEGER NOT NULL,
-- created_at TEXT NOT NULL
--);
--
--CREATE TABLE IF NOT EXISTS creator_tip_profiles (
-- creator_subject TEXT PRIMARY KEY, -- usr_…
-- handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)
-- display_name TEXT NOT NULL,
-- avatar_url TEXT,
-- headline TEXT,
-- page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing
-- accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted
-- min_amount INTEGER NOT NULL DEFAULT 1,
-- paid_message_min INTEGER NOT NULL DEFAULT 100,
-- tts_enabled INTEGER NOT NULL DEFAULT 0,
-- tts_min_amount INTEGER NOT NULL DEFAULT 100,
-- tts_max_chars INTEGER NOT NULL DEFAULT 200,
-- tts_voice TEXT NOT NULL DEFAULT 'gary',
-- media_requests_enabled INTEGER NOT NULL DEFAULT 0,
-- media_request_min INTEGER NOT NULL DEFAULT 25,
-- media_max_seconds INTEGER NOT NULL DEFAULT 600,
-- revision INTEGER NOT NULL DEFAULT 1,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL
--);
--
--CREATE TABLE IF NOT EXISTS tip_interactions (
-- id TEXT PRIMARY KEY, -- tint_<ULID>
-- creator_subject TEXT NOT NULL,
-- supporter_subject TEXT, -- null: external/anonymous/unmapped
-- supporter_name TEXT, -- display projection at the time
-- kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
-- amount INTEGER NOT NULL CHECK (amount > 0),
-- currency TEXT NOT NULL DEFAULT 'vibes-bits',
-- amount_cents INTEGER, -- money that arrived (provider/external)
-- message TEXT,
-- request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target
-- funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
-- settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
-- payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
-- delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
-- test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted
-- billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it
-- billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent
-- funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it
-- reversal_txn_ids TEXT NOT NULL DEFAULT '[]',
-- reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator
-- checkout_url TEXT,
-- checkout_ref TEXT,
-- provider TEXT,
-- provider_ref TEXT,
-- legacy_source TEXT, -- 'live:transactions:<id>' etc.
-- idempotency_key TEXT UNIQUE,
-- failure TEXT,
-- origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import
-- transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried
-- transfer_attempts INTEGER NOT NULL DEFAULT 0,
-- next_transfer_at INTEGER NOT NULL DEFAULT 0,
-- target TEXT, -- JSON EntityRef (e.g. a Live stream)
-- created_at TEXT NOT NULL,
-- settled_at TEXT,
-- reversed_at TEXT,
-- delivered_at TEXT,
-- updated_at TEXT NOT NULL,
-- UNIQUE (provider, provider_ref),
-- UNIQUE (legacy_source)
--);
--CREATE INDEX IF NOT EXISTS idx_tint_creator ON tip_interactions (creator_subject, created_at);
--CREATE INDEX IF NOT EXISTS idx_tint_supporter ON tip_interactions (supporter_subject, created_at);
--CREATE INDEX IF NOT EXISTS idx_tint_pending ON tip_interactions (payment_state, funding);
--
--CREATE TABLE IF NOT EXISTS tip_goals (
-- id TEXT PRIMARY KEY, -- tgoal_<ULID>
-- creator_subject TEXT NOT NULL,
-- title TEXT NOT NULL,
-- description TEXT,
-- target_amount INTEGER NOT NULL CHECK (target_amount > 0),
-- currency TEXT NOT NULL DEFAULT 'vibes-bits',
-- image_url TEXT,
-- status TEXT NOT NULL CHECK (status IN ('active', 'closed')),
-- sort_order INTEGER NOT NULL DEFAULT 0,
-- reached_at TEXT,
-- closed_at TEXT,
-- opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)
-- legacy_source TEXT UNIQUE,
-- revision INTEGER NOT NULL DEFAULT 1,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL
--);
--CREATE INDEX IF NOT EXISTS idx_goal_creator ON tip_goals (creator_subject, status);
--
--CREATE TABLE IF NOT EXISTS tip_goal_contributions (
-- id INTEGER PRIMARY KEY AUTOINCREMENT,
-- goal_id TEXT NOT NULL REFERENCES tip_goals(id),
-- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
-- amount INTEGER NOT NULL CHECK (amount > 0),
-- reversed_amount INTEGER NOT NULL DEFAULT 0,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL,
-- UNIQUE (goal_id, interaction_id)
--);
--
--CREATE TABLE IF NOT EXISTS paid_messages (
-- id TEXT PRIMARY KEY, -- tpm_<ULID>
-- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
-- creator_subject TEXT NOT NULL,
-- kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),
-- text TEXT NOT NULL,
-- voice TEXT,
-- highlight_seconds INTEGER NOT NULL DEFAULT 0,
-- status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
-- chat_ref TEXT, -- JSON reference the chat adapter returned
-- test INTEGER NOT NULL DEFAULT 0,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL
--);
--
--CREATE TABLE IF NOT EXISTS paid_media_requests (
-- id TEXT PRIMARY KEY, -- tmr_<ULID>
-- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
-- creator_subject TEXT NOT NULL,
-- url TEXT NOT NULL,
-- provider TEXT,
-- status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
-- queue_ref TEXT, -- JSON reference returned by the queue owner
-- test INTEGER NOT NULL DEFAULT 0,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL
--);
--
--CREATE TABLE IF NOT EXISTS overlay_configs (
-- id TEXT PRIMARY KEY, -- tovc_<ULID>
-- creator_subject TEXT NOT NULL,
-- kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),
-- name TEXT NOT NULL,
-- settings TEXT NOT NULL DEFAULT '{}',
-- goal_id TEXT,
-- revision INTEGER NOT NULL DEFAULT 1,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL
--);
--CREATE INDEX IF NOT EXISTS idx_ovc_creator ON overlay_configs (creator_subject);
--
--CREATE TABLE IF NOT EXISTS overlay_tokens (
-- id TEXT PRIMARY KEY, -- tovt_<ULID>
-- creator_subject TEXT NOT NULL,
-- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
-- scopes TEXT NOT NULL, -- JSON ["alerts","goals"]
-- config_id TEXT,
-- label TEXT,
-- created_by TEXT NOT NULL,
-- created_at TEXT NOT NULL,
-- last_used_at TEXT,
-- revoked_at TEXT
--);
--CREATE INDEX IF NOT EXISTS idx_ovt_creator ON overlay_tokens (creator_subject);
--
--CREATE TABLE IF NOT EXISTS overlay_deliveries (
-- seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)
-- id TEXT NOT NULL UNIQUE, -- tovd_<ULID>
-- creator_subject TEXT NOT NULL,
-- kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),
-- interaction_id TEXT,
-- goal_id TEXT,
-- payload TEXT NOT NULL,
-- test INTEGER NOT NULL DEFAULT 0,
-- status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
-- delivered_at TEXT,
-- failed_at TEXT,
-- sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
-- dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
-- created_at TEXT NOT NULL
--);
--CREATE INDEX IF NOT EXISTS idx_ovd_creator ON overlay_deliveries (creator_subject, seq);
--CREATE INDEX IF NOT EXISTS idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';
--
--CREATE TABLE IF NOT EXISTS interaction_effects (
-- id INTEGER PRIMARY KEY AUTOINCREMENT,
-- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
-- effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
-- adapter TEXT NOT NULL,
-- state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
-- attempts INTEGER NOT NULL DEFAULT 0,
-- next_attempt_at INTEGER NOT NULL DEFAULT 0,
-- last_error TEXT,
-- result TEXT,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL,
-- UNIQUE (interaction_id, effect)
--);
--CREATE INDEX IF NOT EXISTS idx_effects_due ON interaction_effects (state, next_attempt_at);
++const { createDb } = require('openvibe-sdk/db');
--CREATE TABLE IF NOT EXISTS migration_maps (
-- id INTEGER PRIMARY KEY AUTOINCREMENT,
-- source TEXT NOT NULL, -- 'live'
-- source_table TEXT NOT NULL,
-- source_id TEXT NOT NULL,
-- target_type TEXT, -- 'interaction' | 'goal'
-- target_id TEXT,
-- status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
-- reason TEXT,
-- billing_txn_id TEXT,
-- run_id TEXT NOT NULL,
-- created_at TEXT NOT NULL,
-- updated_at TEXT NOT NULL,
-- UNIQUE (source, source_table, source_id)
--);
++const MIGRATIONS = path.join(__dirname, '..', 'migrations');
++const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');
--CREATE TABLE IF NOT EXISTS import_runs (
-- id TEXT PRIMARY KEY,
-- source TEXT NOT NULL,
-- dry_run INTEGER NOT NULL,
-- started_at TEXT NOT NULL,
-- finished_at TEXT,
-- report TEXT
--);
--
---- Moderation (server/domain/moderation.js): the creator's moderators, invitations to become one
---- (show-once links, hashed like overlay tokens) and the record of every moderation outcome.
--CREATE TABLE IF NOT EXISTS tip_moderators (
-- creator_subject TEXT NOT NULL,
-- moderator_subject TEXT NOT NULL,
-- name TEXT, -- display name when they joined (for the creator's list)
-- added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'
-- created_at TEXT NOT NULL,
-- removed_at TEXT,
-- PRIMARY KEY (creator_subject, moderator_subject)
--);
--CREATE INDEX IF NOT EXISTS idx_tmod_moderator ON tip_moderators (moderator_subject);
--
--CREATE TABLE IF NOT EXISTS tip_moderator_invites (
-- id TEXT PRIMARY KEY, -- tmin_<ULID>
-- creator_subject TEXT NOT NULL,
-- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
-- created_by TEXT NOT NULL,
-- created_at TEXT NOT NULL,
-- expires_at TEXT NOT NULL,
-- used_at TEXT,
-- used_by TEXT,
-- revoked_at TEXT
--);
--
--CREATE TABLE IF NOT EXISTS tip_moderation_log (
-- id INTEGER PRIMARY KEY AUTOINCREMENT,
-- creator_subject TEXT NOT NULL,
-- interaction_id TEXT NOT NULL,
-- action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
-- by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
-- actor TEXT, -- usr_… / svc:… ; null for the filter
-- reason TEXT,
-- created_at TEXT NOT NULL
--);
--CREATE INDEX IF NOT EXISTS idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);
--
--CREATE TABLE IF NOT EXISTS api_idempotency (
-- key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>
-- request_hash TEXT NOT NULL,
-- method TEXT NOT NULL,
-- path TEXT NOT NULL,
-- status INTEGER NOT NULL,
-- response TEXT NOT NULL,
-- created_at TEXT NOT NULL
--);
--`;
--
--/**
-- * Columns added after schema v1, for new and existing databases alike (ALTER TABLE … ADD COLUMN when
-- * missing; idempotent, never rewrites a row).
-- *
-- * v2 privacy: what the supporter lets the public see of an interaction (server/domain/privacy.js),
-- * their erasure, and what the creator's public goal and supporters pages show
-- * v3 moderation (server/domain/moderation.js): an interaction's moderation state, the creator's
-- * word filter, overlay alerts retracted by a moderator
-- */
--const COLUMNS = [
-- ['tip_interactions', 'anonymous', 'INTEGER NOT NULL DEFAULT 0'], // name shown as "Anonymous" to everyone but the supporter
-- ['tip_interactions', 'hide_amount', 'INTEGER NOT NULL DEFAULT 0'], // amount left out of overlays, chat lines, public pages
-- ['tip_interactions', 'private_message', 'INTEGER NOT NULL DEFAULT 0'], // a tip's message is for the creator only
-- ['tip_interactions', 'erased_at', 'TEXT'], // the supporter erased their data from it
-- ['creator_tip_profiles', 'page_settings', "TEXT NOT NULL DEFAULT '{}'"], // JSON: what the public goal / supporters pages show
-- ['tip_interactions', 'moderation', "TEXT NOT NULL DEFAULT 'visible'"], // visible | held (the filter, awaiting review) | hidden (a moderator)
-- ['tip_interactions', 'moderated_at', 'TEXT'],
-- ['tip_interactions', 'moderated_by', 'TEXT'], // 'filter', usr_… or svc:…
-- ['tip_interactions', 'filtered', 'INTEGER NOT NULL DEFAULT 0'], // the word filter matched it at settlement
-- ['creator_tip_profiles', 'filter_words', "TEXT NOT NULL DEFAULT '[]'"], // JSON: the creator's blocklist (filter.js)
-- ['creator_tip_profiles', 'filter_action', "TEXT NOT NULL DEFAULT 'mask'"],// mask | hold
-- ['creator_tip_profiles', 'filter_links', 'INTEGER NOT NULL DEFAULT 1'], // links in paid messages shown as [link]
-- ['overlay_deliveries', 'hidden', 'INTEGER NOT NULL DEFAULT 0'], // retracted: never sent or replayed again
--];
--
--const SCHEMA_VERSION = 3;
--
--function addColumns(db) {
-- const have = new Map();
-- for (const [table, name, def] of COLUMNS) {
-- if (!have.has(table)) have.set(table, new Set(db.prepare(`PRAGMA table_info(${table})`).all().map((c) => c.name)));
-- if (!have.get(table).has(name)) { db.exec(`ALTER TABLE ${table} ADD COLUMN ${name} ${def}`); have.get(table).add(name); }
++function openDb(config, { registry, log = console } = {}) {
++ if (!config.db.url && !config.isProduction) {
++ log.warn(`[Tips] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);
++ fs.mkdirSync(DEV_PGLITE, { recursive: true });
++ return createDb({ pglite: DEV_PGLITE, service: 'tips', registry, log });
}
++ return createDb({ url: config.db.url, service: 'tips', registry, log });
}
--function openDb(file) {
-- if (file !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(file)), { recursive: true });
-- const db = new Database(file);
-- db.pragma('journal_mode = WAL');
-- db.pragma('foreign_keys = ON');
-- db.pragma('busy_timeout = 5000');
-- db.exec(SCHEMA);
-- db.transaction(() => addColumns(db))();
-- db.prepare('INSERT OR IGNORE INTO settings (id, schema_version, created_at) VALUES (1, ?, ?)').run(SCHEMA_VERSION, new Date().toISOString());
-- db.prepare('UPDATE settings SET schema_version = ? WHERE id = 1 AND schema_version < ?').run(SCHEMA_VERSION, SCHEMA_VERSION);
-- return db;
++/**
++ * Apply pending migrations. Several processes starting together are safe: the SDK serialises runs
++ * with an advisory lock held on the owner's one direct session.
++ */
++async function migrate(config, { serving = null, log = console } = {}) {
++ if (serving && serving.store === 'pglite') return serving.migrate({ dir: MIGRATIONS, log });
++ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');
++ const owner = createDb({ url: config.db.directUrl, service: 'tips-migrate', max: 1, log });
++ try { return await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }
}
--module.exports = { openDb, SCHEMA_VERSION, COLUMNS };
++module.exports = { openDb, migrate, MIGRATIONS };
ddiff --git a/server/domain/effects.js b/server/domain/effects.js
iindex 9867ca4..99a84d0 100644
---- a/server/domain/effects.js
++++ b/server/domain/effects.js
@@@ -14,9 +14,21 @@
* is not sent, and the creator's word filter has starred blocked words out of the line and dropped
* them from the TTS text. `privacy` tells the receiving product what was withheld. An effect whose
* interaction a moderator hid (or whose effect was cancelled meanwhile) is never delivered.
++ *
++ * Writes to an effect and what it delivers (paid message, media request, the interaction's delivery
++ * state) run SERIALIZABLE with the interaction row locked first, as the money paths do.
++ *
++ * Every process runs the worker; each batch of due effects is claimed with a lease (next_attempt_at
++ * moved past the batch's worst case, FOR UPDATE SKIP LOCKED), so no two processes deliver one effect
++ * at once, and a process that dies mid-batch leaves its effects to the others when the lease ends.
*/
--const { iso, json } = require('../util');
--const { privacyOf, isAnonymous } = require('./privacy');
++const { sql } = require('openvibe-sdk/db');
++const { iso } = require('../util');
++const { privacyOf, isAnonymous, publicView } = require('./privacy');
++
++// A delivery takes up to 10 s (twice with a token refresh): 20 of them finish well inside the lease.
++const BATCH = 20;
++const LEASE_MS = 5 * 60_000;
function createEffects(ctx) {
const { db, config } = ctx;
@@@ -24,8 +36,8 @@ function createEffects(ctx) {
let kickTimer = null;
function jobFor(effect, i, profile) {
-- const req = json(i.request, {});
-- const pv = ctx.view(i);
++ const req = i.request || {};
++ const pv = publicView(i, { filter: profile ? profile.filter : null });
const name = pv.supporter_name;
const amount = pv.amount != null ? `${pv.amount.toLocaleString('en-US')} Vibes` : null;
const msg = pv.message ? `: ${pv.message}` : '';
@@@ -39,7 +51,7 @@ function createEffects(ctx) {
privacy: privacyOf(i),
// A tip on the creator's own PowerChat reads as Live's webhook wrote it: "… (PowerChat)".
text: `${name} ${amount ? `tipped ${amount}` : 'sent a tip'}${msg}${i.settlement === 'external' && i.provider === 'powerchat' ? ' (PowerChat)' : ''}`,
-- target: json(i.target, null),
++ target: i.target || null,
};
if (effect.effect === 'paid_message') job.highlight_seconds = req.highlight_seconds || 0;
if (effect.effect === 'tts') job.tts = pv.tts;
@@@ -48,78 +60,92 @@ function createEffects(ctx) {
}
function markDone(effect, result) {
-- ctx.tx(() => {
++ return ctx.tx(async (t) => {
++ await ctx.interactions.lock(t, effect.interaction_id);
const at = iso(ctx.now());
-- const r = db.prepare("UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ?, updated_at = ? WHERE id = ? AND state = 'queued'")
-- .run(JSON.stringify(result || null), at, effect.id);
-- if (!r.changes) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side
-- const ref = JSON.stringify((result && result.ref) || null);
++ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ${sql.json(result || null)},
++ updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);
++ if (!n) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side
++ const ref = sql.json((result && result.ref) || null);
if (effect.effect === 'paid_message' || effect.effect === 'tts') {
-- db.prepare("UPDATE paid_messages SET status = 'delivered', chat_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);
++ await t.exec(sql`UPDATE paid_messages SET status = 'delivered', chat_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
}
if (effect.effect === 'media_request') {
-- db.prepare("UPDATE paid_media_requests SET status = 'accepted', queue_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);
++ await t.exec(sql`UPDATE paid_media_requests SET status = 'accepted', queue_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
}
-- ctx.interactions.recomputeDelivery(effect.interaction_id);
-- });
++ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);
++ }, ctx.MONEY);
}
-- function markFailed(effect, err) {
-- ctx.tx(() => {
++ async function markFailed(effect, err) {
++ await ctx.tx(async (t) => {
++ await ctx.interactions.lock(t, effect.interaction_id);
const at = iso(ctx.now());
const attempts = effect.attempts + 1;
const final = err.permanent || attempts >= config.chat.maxAttempts;
const backoff = config.chat.backoffMs[Math.min(attempts - 1, config.chat.backoffMs.length - 1)] || 60000;
-- db.prepare('UPDATE interaction_effects SET state = ?, attempts = ?, next_attempt_at = ?, last_error = ?, updated_at = ? WHERE id = ? AND state = \'queued\'')
-- .run(final ? 'failed' : 'queued', attempts, ctx.now() + backoff, String(err.message || err).slice(0, 500), at, effect.id);
++ await t.exec(sql`UPDATE interaction_effects SET state = ${final ? 'failed' : 'queued'}, attempts = ${attempts}, next_attempt_at = ${ctx.now() + backoff},
++ last_error = ${String(err.message || err).slice(0, 500)}, updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);
if (final) {
-- if (effect.effect === 'paid_message' || effect.effect === 'tts') db.prepare("UPDATE paid_messages SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);
-- if (effect.effect === 'media_request') db.prepare("UPDATE paid_media_requests SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);
-- ctx.interactions.recomputeDelivery(effect.interaction_id);
++ if (effect.effect === 'paid_message' || effect.effect === 'tts') await t.exec(sql`UPDATE paid_messages SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
++ if (effect.effect === 'media_request') await t.exec(sql`UPDATE paid_media_requests SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
++ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);
}
-- });
++ }, ctx.MONEY);
ctx.outboxKick();
}
async function runOne(effect) {
-- // Re-read: the batch was selected before earlier deliveries awaited; a hide or a reversal may
++ // Re-read: the batch was claimed before earlier deliveries awaited; a hide or a reversal may
// have cancelled this one since.
-- const current = db.prepare('SELECT state FROM interaction_effects WHERE id = ?').get(effect.id);
++ const current = await db.maybe(sql`SELECT state FROM interaction_effects WHERE id = ${effect.id}`);
if (!current || current.state !== 'queued') return 'cancelled';
-- const i = ctx.interactions.get(effect.interaction_id);
++ const i = await ctx.interactions.get(db, effect.interaction_id);
if (!i || i.payment_state !== 'settled') {
// Reversed or failed before its turn: nothing to deliver.
-- ctx.tx(() => ctx.interactions.cancelQueued(i, 'payment_not_settled'));
++ await ctx.tx(async (t) => { const locked = await ctx.interactions.lock(t, effect.interaction_id); if (locked) await ctx.interactions.cancelQueued(t, locked, 'payment_not_settled'); }, ctx.MONEY);
return 'cancelled';
}
if (i.moderation !== 'visible') {
// Hidden or held: moderation.js cancels or releases these; never deliver one meanwhile.
-- db.prepare("UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE id = ? AND state = 'queued'").run(iso(ctx.now()), effect.id);
++ await db.exec(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${iso(ctx.now())}
++ WHERE id = ${effect.id} AND state = 'queued'`);
return 'cancelled';
}
const adapter = ctx.adapters[effect.adapter];
-- if (!adapter) { markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }
++ if (!adapter) { await markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }
++ let result;
try {
-- const result = await adapter.deliver(jobFor(effect, i, ctx.profiles.bySubject(i.creator_subject)));
-- markDone(effect, result);
-- return 'delivered';
++ result = await adapter.deliver(jobFor(effect, i, await ctx.profiles.bySubject(db, i.creator_subject)));
} catch (e) {
-- markFailed(effect, e);
++ await markFailed(effect, e);
return 'retry';
}
++ await markDone(effect, result);
++ return 'delivered';
++ }
++
++ /** Claim a batch of due effects (a lease each), oldest first. */
++ async function claim() {
++ const now = ctx.now();
++ const rows = await db.many(sql`UPDATE interaction_effects SET next_attempt_at = ${now + LEASE_MS}
++ WHERE id IN (SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now}
++ ORDER BY id LIMIT ${BATCH} FOR UPDATE SKIP LOCKED)
++ RETURNING *`);
++ return rows.sort((a, b) => a.id - b.id);
}
-- /** Deliver every due effect once. */
++ /** Deliver every due effect once (this process's share of them). */
async function drain() {
if (running) return { busy: true };
running = true;
const out = { delivered: 0, retry: 0, failed: 0, cancelled: 0 };
try {
for (;;) {
-- const due = db.prepare("SELECT * FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ? ORDER BY id LIMIT 50").all(ctx.now());
++ const due = await claim();
if (!due.length) break;
for (const e of due) { const r = await runOne(e); out[r] = (out[r] || 0) + 1; }
-- if (due.length < 50) break;
++ if (due.length < BATCH) break;
}
} finally { running = false; }
return out;
ddiff --git a/server/domain/filter.js b/server/domain/filter.js
iindex 2d65c8b..79bc984 100644
---- a/server/domain/filter.js
++++ b/server/domain/filter.js
@@@ -25,9 +25,9 @@ const INVISIBLE = /[­​-‏‪-‮⁠-⁤⁦-⁩]/g;
const COMBINING_RUN = /(\p{M}{2})\p{M}+/gu;
const URL_RE = /\b(?:https?:\/\/|www\.)[^\s<>"']+/gi;
--/** Invisible characters and combining-mark runs out, NFKC. */
++/** Invisible and NUL characters and combining-mark runs out, unpaired surrogates replaced, NFKC. */
function clean(s) {
-- return String(s == null ? '' : s).replace(INVISIBLE, '').normalize('NFKC').replace(COMBINING_RUN, '$1');
++ return String(s == null ? '' : s).toWellFormed().replace(INVISIBLE, '').replace(/\u0000/g, '').normalize('NFKC').replace(COMBINING_RUN, '$1');
}
/** The creator's input (a list, or text with one entry per line or comma) → a clean, bounded list. */
ddiff --git a/server/domain/goals.js b/server/domain/goals.js
iindex c01bb65..7c6a332 100644
---- a/server/domain/goals.js
++++ b/server/domain/goals.js
@@@ -16,75 +16,97 @@
* when goal_supporters is on, each supporter's own privacy applying (an anonymous one reads
* "Anonymous", a hidden amount is left out).
*/
++const { sql } = require('openvibe-sdk/db');
const { fail, iso, prefixedId, text, positiveInt } = require('../util');
const { safeUrl } = require('./profiles');
--const { publicName, shownName } = require('./privacy');
++const { publicName, shownName, publicView } = require('./privacy');
function createGoals(ctx) {
const { db } = ctx;
-- const get = (id) => db.prepare('SELECT * FROM tip_goals WHERE id = ?').get(String(id || '')) || null;
-- /** Settled contributions, plus the opening amount a goal imported from Live carried over. */
-- const totalOf = (goalId) => {
-- const r = db.prepare('SELECT COALESCE(SUM(amount - reversed_amount), 0) AS n, COUNT(*) AS c FROM tip_goal_contributions WHERE goal_id = ?').get(goalId);
-- const g = db.prepare('SELECT opening_amount FROM tip_goals WHERE id = ?').get(goalId);
-- return { n: r.n + ((g && g.opening_amount) || 0), c: r.c, contributions: r.n, opening: (g && g.opening_amount) || 0 };
-- };
--
-- function list(creator, { status } = {}) {
-- const rows = status
-- ? db.prepare('SELECT * FROM tip_goals WHERE creator_subject = ? AND status = ? ORDER BY sort_order, created_at').all(creator, status)
-- : db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? ORDER BY status = 'active' DESC, sort_order, created_at DESC").all(creator);
-- return rows;
++ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_goals WHERE id = ${String(id || '')}`);
++
++ /** Settled contributions per goal, in one query: Map(goal id → { n: sum, c: count }). */
++ async function totalsOf(q, goalIds) {
++ const out = new Map(goalIds.map((id) => [id, { n: 0, c: 0 }]));
++ if (!goalIds.length) return out;
++ const rows = await q.many(sql`SELECT goal_id, COALESCE(SUM(amount - reversed_amount), 0)::bigint AS n, count(*) AS c
++ FROM tip_goal_contributions WHERE goal_id = ANY(${goalIds}) GROUP BY goal_id`);
++ for (const r of rows) out.set(r.goal_id, { n: r.n, c: r.c });
++ return out;
++ }
++
++ function list(q, creator, { status } = {}) {
++ return status
++ ? q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = ${status} ORDER BY sort_order, created_at`)
++ : q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} ORDER BY status = 'active' DESC, sort_order, created_at DESC`);
++ }
++
++ /** The full view of each goal (the creator's, granted services', overlays'): one query for all totals. */
++ async function presentMany(q, goals) {
++ const totals = await totalsOf(q, goals.map((g) => g.id));
++ return goals.map((g) => {
++ const t = totals.get(g.id);
++ const n = t.n + (g.opening_amount || 0);
++ return {
++ id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,
++ target_amount: g.target_amount, current_amount: n, currency: g.currency, supporters_count: t.c,
++ carried_over_amount: g.opening_amount || 0,
++ percent: Math.min(100, Math.floor((n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,
++ image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,
++ created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,
++ };
++ });
}
-- function present(g, { contributions = false } = {}) {
++ async function present(q, g, { contributions = false } = {}) {
if (!g) return null;
-- const t = totalOf(g.id);
-- const out = {
-- id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,
-- target_amount: g.target_amount, current_amount: t.n, currency: g.currency, supporters_count: t.c,
-- carried_over_amount: t.opening,
-- percent: Math.min(100, Math.floor((t.n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,
-- image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,
-- created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,
-- };
++ const [out] = await presentMany(q, [g]);
if (contributions) {
// The creator's view: amounts always (their money), names as the supporter allowed.
-- out.contributions = db.prepare(`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at
-- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? ORDER BY c.id DESC LIMIT 200`).all(g.id)
++ out.contributions = (await q.many(sql`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at
++ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ${g.id} ORDER BY c.id DESC LIMIT 200`))
.map(({ anonymous, erased_at: erasedAt, ...c }) => ({ ...c, supporter_name: publicName({ ...c, anonymous, erased_at: erasedAt }) }));
}
return out;
}
-- /** The public shape of a goal under the creator's page settings (`page`, profiles.js). */
-- function publicGoal(g, page) {
-- const out = present(g);
-- if (!out) return null;
-- if (!page.goal_amounts) Object.assign(out, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });
-- if (page.goal_supporters) {
++ /**
++ * The public shape of goals under the creator's page settings (`profile.page`, profiles.js), with
++ * the creator's word filter (`profile.filter`) on supporter names: totals in one query, and the
++ * latest supporters of every goal in one more when the page shows them.
++ */
++ async function publicGoals(q, goals, profile) {
++ const page = profile.page;
++ const views = await presentMany(q, goals);
++ if (!page.goal_amounts) for (const v of views) Object.assign(v, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });
++ if (page.goal_supporters && goals.length) {
// Moderated (held or hidden) interactions are left out; names go through the creator's filter.
-- const settings = ctx.profiles.filterOf(g.creator_subject);
-- out.supporters = db.prepare(`SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at
-- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? AND c.amount > c.reversed_amount
-- AND i.moderation = 'visible' ORDER BY c.id DESC LIMIT 10`).all(g.id)
-- .map((c) => ({ name: shownName(c, settings), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));
++ const rows = await q.many(sql`SELECT gid AS goal_id, s.amount, s.created_at, s.supporter_name, s.anonymous, s.hide_amount, s.erased_at
++ FROM unnest(${goals.map((g) => g.id)}::text[]) AS gid
++ CROSS JOIN LATERAL (
++ SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at
++ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id
++ WHERE c.goal_id = gid AND c.amount > c.reversed_amount AND i.moderation = 'visible'
++ ORDER BY c.id DESC LIMIT 10) s`);
++ for (const v of views) {
++ v.supporters = rows.filter((r) => r.goal_id === v.id)
++ .map((c) => ({ name: shownName(c, profile.filter), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));
++ }
}
-- return out;
++ return views;
}
++ const publicGoal = async (q, g, profile) => (g ? (await publicGoals(q, [g], profile))[0] : null);
/** Inside a transaction: bump revision, emit the event, queue the overlay delivery. */
-- function changed(goalId, reason, { interactionId = null, by = null } = {}) {
-- const at = iso(ctx.now());
-- db.prepare('UPDATE tip_goals SET revision = revision + 1, updated_at = ? WHERE id = ?').run(at, goalId);
-- const g = get(goalId);
-- const view = present(g);
-- ctx.outbox.emit('tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {
++ async function changed(t, goalId, reason, { interactionId = null, by = null } = {}) {
++ const g = await t.one(sql`UPDATE tip_goals SET revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${goalId} RETURNING *`);
++ const [view] = await presentMany(t, [g]);
++ await ctx.outbox.emit(t, 'tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {
goal_id: g.id, creator: view.creator, title: g.title, target_amount: g.target_amount, current_amount: view.current_amount,
currency: g.currency, status: g.status, reached: view.reached, reason, interaction_id: interactionId,
});
-- ctx.overlays.addGoalDelivery(g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });
++ await ctx.overlays.addGoalDelivery(t, g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });
return view;
}
@@@ -109,88 +131,103 @@ function createGoals(ctx) {
return out;
}
-- function create(creator, input = {}) {
++ async function create(creator, input = {}) {
const f = fields(input, false);
-- const count = db.prepare("SELECT COUNT(*) AS n FROM tip_goals WHERE creator_subject = ? AND status = 'active'").get(creator).n;
-- if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');
-- return ctx.tx(() => {
++ return ctx.tx(async (t) => {
++ await ctx.lockCreator(t, creator);
++ const count = await t.value(sql`SELECT count(*) FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active'`);
++ if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');
const at = iso(ctx.now());
const id = prefixedId('tgoal', ctx.now());
-- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)
-- VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?, ?)`).run(id, creator, f.title, f.description || null, f.target_amount, f.image_url || null, f.sort_order || 0, at, at);
-- return changed(id, 'created');
++ await t.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)
++ VALUES (${id}, ${creator}, ${f.title}, ${f.description || null}, ${f.target_amount}, ${f.image_url || null}, 'active', ${f.sort_order || 0}, ${at}, ${at})`);
++ return changed(t, id, 'created');
});
}
-- function update(goal, input = {}) {
++ /** An edit by the creator. The goal is locked, and its revision checked against the locked row. */
++ async function update(goal, input = {}) {
if (goal.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');
if (input.revision != null && Number(input.revision) !== goal.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${goal.revision}`);
const f = fields(input, true);
-- const keys = Object.keys(f);
-- if (!keys.length) return present(goal);
-- return ctx.tx(() => {
-- db.prepare(`UPDATE tip_goals SET ${keys.map((k) => `${k} = @${k}`).join(', ')} WHERE id = @id`).run({ ...f, id: goal.id });
-- markReached(goal.id);
-- return changed(goal.id, 'updated');
++ if (!Object.keys(f).length) return present(db, goal);
++ return ctx.tx(async (t) => {
++ const cur = await t.one(sql`SELECT * FROM tip_goals WHERE id = ${goal.id} FOR UPDATE`);
++ if (cur.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');
++ if (input.revision != null && Number(input.revision) !== cur.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${cur.revision}`);
++ await t.exec(sql`UPDATE tip_goals SET ${sql.set(f)} WHERE id = ${goal.id}`);
++ await markReached(t, goal.id);
++ return changed(t, goal.id, 'updated');
});
}
-- function close(goal) {
-- if (goal.status === 'closed') return present(goal);
-- return ctx.tx(() => {
-- db.prepare("UPDATE tip_goals SET status = 'closed', closed_at = ? WHERE id = ?").run(iso(ctx.now()), goal.id);
-- return changed(goal.id, 'closed');
++ async function close(goal) {
++ if (goal.status === 'closed') return present(db, goal);
++ return ctx.tx(async (t) => {
++ const n = await t.exec(sql`UPDATE tip_goals SET status = 'closed', closed_at = ${iso(ctx.now())} WHERE id = ${goal.id} AND status = 'active'`);
++ if (!n) return present(t, await get(t, goal.id));
++ return changed(t, goal.id, 'closed');
});
}
-- function markReached(goalId) {
-- const g = get(goalId);
-- const reached = totalOf(goalId).n >= g.target_amount;
-- if (reached && !g.reached_at) { db.prepare('UPDATE tip_goals SET reached_at = ? WHERE id = ?').run(iso(ctx.now()), goalId); return true; }
-- if (!reached && g.reached_at) db.prepare('UPDATE tip_goals SET reached_at = NULL WHERE id = ?').run(goalId);
++ /** Inside a transaction: set or clear reached_at from the goal's total. True when reached just now. */
++ async function markReached(t, goalId) {
++ const g = await t.one(sql`SELECT g.target_amount, g.reached_at,
++ g.opening_amount + COALESCE((SELECT SUM(c.amount - c.reversed_amount) FROM tip_goal_contributions c WHERE c.goal_id = g.id), 0)::bigint AS total
++ FROM tip_goals g WHERE g.id = ${goalId}`);
++ const reached = g.total >= g.target_amount;
++ if (reached && !g.reached_at) { await t.exec(sql`UPDATE tip_goals SET reached_at = ${iso(ctx.now())} WHERE id = ${goalId}`); return true; }
++ if (!reached && g.reached_at) await t.exec(sql`UPDATE tip_goals SET reached_at = NULL WHERE id = ${goalId}`);
return false;
}
/** The goal an interaction counts toward, or null. */
-- function pick(creator, requestedGoalId) {
++ async function pick(q, creator, requestedGoalId) {
if (requestedGoalId) {
-- const g = get(requestedGoalId);
++ const g = await get(q, requestedGoalId);
if (g && g.creator_subject === creator && g.status === 'active') return g;
}
-- const active = db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? AND status = 'active' LIMIT 2").all(creator);
++ const active = await q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active' LIMIT 2`);
return active.length === 1 ? active[0] : null;
}
-- /** Inside the settlement transaction. Simulations never get here. */
-- function contribute(interaction, requestedGoalId) {
-- const g = pick(interaction.creator_subject, requestedGoalId);
-- if (!g) return null;
++ /**
++ * Inside the settlement transaction (the interaction row is locked already; the goal is locked
++ * next, keeping the lock order). Simulations never get here. `filter`: the creator's word filter.
++ */
++ async function contribute(t, interaction, requestedGoalId, filter) {
++ const picked = await pick(t, interaction.creator_subject, requestedGoalId);
++ if (!picked) return null;
++ const g = await t.one(sql`SELECT id, status FROM tip_goals WHERE id = ${picked.id} FOR UPDATE`);
++ if (g.status !== 'active') return null;
const at = iso(ctx.now());
-- const r = db.prepare(`INSERT OR IGNORE INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at) VALUES (?, ?, ?, ?, ?)`)
-- .run(g.id, interaction.id, interaction.amount, at, at);
-- if (!r.changes) return null;
-- const reachedNow = markReached(g.id);
++ const n = await t.exec(sql`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)
++ VALUES (${g.id}, ${interaction.id}, ${interaction.amount}, ${at}, ${at}) ON CONFLICT DO NOTHING`);
++ if (!n) return null;
++ const reachedNow = await markReached(t, g.id);
// The overlay's goal update names the supporter as the alert does; not when the amount is hidden
// (the bar's jump would otherwise pin the amount on them), nor while moderation holds it.
-- const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : ctx.view(interaction).supporter_name;
-- return changed(g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });
++ const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : publicView(interaction, { filter }).supporter_name;
++ return changed(t, g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });
}
/** Inside the reversal transaction: take `bits` back off every goal the interaction counted toward. */
-- function reverse(interaction, bits) {
-- const rows = db.prepare('SELECT * FROM tip_goal_contributions WHERE interaction_id = ?').all(interaction.id);
++ async function reverse(t, interaction, bits) {
++ const rows = await t.many(sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = ${interaction.id} ORDER BY goal_id`);
++ if (!rows.length) return [];
++ await t.many(sql`SELECT id FROM tip_goals WHERE id = ANY(${rows.map((c) => c.goal_id)}) ORDER BY id FOR UPDATE`);
const out = [];
for (const c of rows) {
const take = Math.min(bits, c.amount - c.reversed_amount);
if (take <= 0) continue;
-- db.prepare('UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ?, updated_at = ? WHERE id = ?').run(take, iso(ctx.now()), c.id);
-- markReached(c.goal_id);
-- out.push(changed(c.goal_id, 'reversal', { interactionId: interaction.id }));
++ await t.exec(sql`UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ${take}, updated_at = ${iso(ctx.now())} WHERE id = ${c.id}`);
++ await markReached(t, c.goal_id);
++ out.push(await changed(t, c.goal_id, 'reversal', { interactionId: interaction.id }));
}
return out;
}
-- return { get, list, present, publicGoal, create, update, close, contribute, reverse, pick, totalOf };
++ return { get, list, present, presentMany, publicGoal, publicGoals, create, update, close, contribute, reverse, pick, totalsOf };
}
module.exports = { createGoals };
ddiff --git a/server/domain/index.js b/server/domain/index.js
iindex e6368da..61a0cd9 100644
---- a/server/domain/index.js
++++ b/server/domain/index.js
@@@ -3,12 +3,30 @@
/**
* The Tips domain: one context shared by profiles, goals, interactions, overlays and effects.
*
-- * ctx.tx(fn) run fn in one SQLite transaction; ctx.afterCommit(hook) queues work (overlay
-- * pushes, worker kicks) that must only happen once the transaction committed
-- * ctx.outbox.emit() durable event, inside the transaction (openvibe-sdk outbox)
-- * ctx.view(i) the interaction's public view (privacy.js) through its creator's word filter
++ * Every function that touches the database takes the query handle first: `db` outside a transaction,
++ * or the transaction's `t` inside one (never `db` inside a transaction: that would be another
++ * connection, outside it). Operations that open their own transaction take no handle.
++ *
++ * ctx.tx(fn, { isolation }) run fn(t) in one transaction. t.after(hook) queues work (overlay pushes,
++ * worker and relay kicks) for after the commit: a retried attempt starts
++ * with no hooks, a rolled-back one runs none. Anything that changes money
++ * state (payment state, reversed bits, goal contributions, what a payment
++ * entitles: paid messages, media requests) runs { isolation: 'serializable' }
++ * (ADR-007 amendment 2026-09-24, rule 3); the SDK retries serialization
++ * failures and deadlocks.
++ * ctx.within(q, fn, opts) fn(q) when q is a transaction handle, else fn(t) in a new transaction
++ * ctx.afterCommit(q, hook) t.after(hook) inside a transaction; at once on `db` (already committed)
++ * ctx.lockCreator(t, s) lock a creator's profile row: settings changes and per-creator limits
++ * (20 active goals, 25 tokens, 50 moderators, 10 invitations) take it, so two
++ * requests at once count and write one after the other
++ * ctx.outbox.emit(t, …) a durable event, inside the transaction (openvibe-sdk outbox)
++ *
++ * Lock order (so two transactions never wait on each other in a cycle): a creator's profile row, then
++ * tip_interactions rows (SELECT … FOR UPDATE, several by id), then tip_goals rows by id, then the
++ * creator's overlay stream (an advisory lock taken before a delivery is inserted: overlays.js), then
++ * everything else (effects, paid messages and media requests, overlay deliveries, the outbox).
*/
--const { publicView } = require('./privacy');
++const { sql } = require('openvibe-sdk/db');
const { createProfiles } = require('./profiles');
const { createGoals } = require('./goals');
const { createOverlays } = require('./overlays');
@@@ -16,23 +34,30 @@ const { createInteractions } = require('./interactions');
const { createEffects } = require('./effects');
const { createModeration } = require('./moderation');
--function createDomain({ db, config, outbox, billing, adapters, now = () => Date.now(), log = console }) {
-- const ctx = { db, config, now, log, outbox, billing, adapters, _after: null };
++const MONEY = { isolation: 'serializable' };
++
++function createDomain({ db, config, outbox, billing, adapters, valkey = null, now = () => Date.now(), log = console }) {
++ const ctx = { db, config, now, log, outbox, billing, adapters, valkey, MONEY };
-- ctx.tx = (fn) => {
-- if (ctx._after) return db.transaction(fn)();
-- const hooks = [];
-- ctx._after = hooks;
-- let result;
-- try { result = db.transaction(fn)(); } finally { ctx._after = null; }
-- for (const h of hooks) { try { h(); } catch (e) { log.warn('[Tips] after-commit:', e.message); } }
-- return result;
++ const runHook = (h) => { Promise.resolve().then(h).catch((e) => log.warn('[Tips] after-commit:', e.message)); };
++ ctx.tx = async (fn, opts = {}) => {
++ let hooks = [];
++ const out = await db.tx(async (t) => {
++ const mine = []; // this attempt's hooks only: a retried or rolled-back attempt's never run
++ hooks = mine;
++ t.after = (hook) => { mine.push(hook); };
++ return fn(t);
++ }, opts);
++ hooks.forEach(runHook);
++ return out;
};
-- ctx.afterCommit = (hook) => { if (ctx._after) ctx._after.push(hook); else hook(); };
++ const inTx = (q) => !!(q && typeof q.after === 'function');
++ ctx.within = (q, fn, opts) => (inTx(q) ? fn(q) : ctx.tx(fn, opts));
++ ctx.afterCommit = (q, hook) => { if (inTx(q)) q.after(hook); else runHook(hook); };
++ ctx.lockCreator = (t, subject) => t.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE creator_subject = ${subject} FOR UPDATE`);
ctx.outboxKick = () => { try { outbox.kick(); } catch { /* relay off */ } };
ctx.profiles = createProfiles(ctx);
-- ctx.view = (i, opts = {}) => publicView(i, { ...opts, filter: ctx.profiles.filterOf(i.creator_subject) });
ctx.goals = createGoals(ctx);
ctx.overlays = createOverlays(ctx);
ctx.interactions = createInteractions(ctx);
@@@ -41,4 +66,4 @@ function createDomain({ db, config, outbox, billing, adapters, now = () => Date.
return ctx;
}
--module.exports = { createDomain };
++module.exports = { createDomain, MONEY };
ddiff --git a/server/domain/interactions.js b/server/domain/interactions.js
iindex 09853b0..0f40022 100644
---- a/server/domain/interactions.js
++++ b/server/domain/interactions.js
@@@ -32,15 +32,20 @@
* contribution, the overlay alert, the delivery effects and tips.interaction.ready. Delivery then
* runs in the effects worker; its failure never touches payment_state.
*
++ * Every transaction that changes money state (a request and its pending-checkout limit, settlement,
++ * failure, funding, reversal) is SERIALIZABLE (ctx.MONEY) and locks the interaction row first
++ * (SELECT … FOR UPDATE), then the goals it counts toward: ADR-007 amendment 2026-09-24, rule 3.
++ *
* Privacy (privacy.js): the supporter's choices are stored with the interaction (anonymous,
* hide_amount, private_message); everything that leaves Tips for the public is publicView(), and
* events and answers to anyone but the supporter never name an anonymous supporter. A supporter can
* export their tips and erase their data from them (erase(): the money record stays, the person
* goes; tips.interaction.erased redacts the earlier events in OpenVibe.Events).
*/
--const { fail, iso, prefixedId, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { fail, iso, prefixedId, positiveInt, text, storable, userSubject, entityRef, displayName, readCursor, cursorOf, INVISIBLE } = require('../util');
const { VOICES } = require('./profiles');
--const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous } = require('./privacy');
++const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous, publicView, ANONYMOUS } = require('./privacy');
const { BillingCallError } = require('../billing-client');
const KINDS = ['tip', 'paid_message', 'tts', 'media_request'];
@@@ -53,18 +58,28 @@ const MAX_EXTERNAL_CENTS = 100_000_000; // Billing's per-receipt maximum
const HIGHLIGHT = [[5000, 300], [1000, 120], [500, 60], [100, 30]];
const highlightFor = (bits) => (HIGHLIGHT.find(([min]) => bits >= min) || [0, 0])[1];
const TRANSFER_BACKOFF = [2000, 10000, 30000, 60000, 300000, 600000];
++// Due transfers are claimed in small batches with a lease: a Billing call takes up to 10 s (twice
++// with a token refresh), so a batch finishes well inside the lease, and a process that dies with a
++// claimed batch leaves it to the others once the lease ends.
++const TRANSFER_BATCH = 10;
++const TRANSFER_LEASE_MS = 5 * 60_000;
++const COLUMNS = ['id', 'creator_subject', 'supporter_subject', 'supporter_name', 'kind', 'amount', 'amount_cents', 'message', 'request',
++ 'funding', 'settlement', 'payment_state', 'delivery_state', 'test', 'billing_txn_id', 'billing_intent_id', 'funding_txn_id', 'provider', 'provider_ref',
++ 'legacy_source', 'idempotency_key', 'origin', 'target', 'anonymous', 'hide_amount', 'private_message', 'created_at', 'updated_at'];
function createInteractions(ctx) {
-- const { db, config } = ctx;
++ const { db, config, MONEY } = ctx;
-- const get = (id) => db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(String(id || '')) || null;
-- const byBillingTxn = (txnId) => db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').get(txnId) || null;
++ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')}`);
++ /** Inside a transaction: the interaction row, locked (the first lock any transaction takes). */
++ const lock = (t, id) => t.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')} FOR UPDATE`);
++ const byBillingTxn = (q, txnId) => q.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${txnId}`);
// ── Input rules (the creator's settings) ─────────────────
// Plain words only: no control or invisible characters, no links, no markup a speech engine could
// take as SSML.
function cleanTts(s) {
-- return String(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ')
++ return storable(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ')
.replace(/\s+/g, ' ').trim();
}
@@@ -89,7 +104,7 @@ function createInteractions(ctx) {
* Validate a request against the creator's profile. Returns the normalised fields:
* { kind, amount, message, request: { goal_id, tts, media, highlight_seconds } }.
*/
-- function validate(profile, input, { simulation = false } = {}) {
++ async function validate(q, profile, input, { simulation = false } = {}) {
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip page');
if (!profile.accepting && !simulation) fail(409, 'tips.not_accepting', `${profile.display_name} is not accepting tips right now`);
const kind = input.kind || 'tip';
@@@ -121,7 +136,7 @@ function createInteractions(ctx) {
request.media = { ...mediaUrl(input.media && input.media.url), max_seconds: profile.media_max_seconds };
}
if (input.goal_id) {
-- const g = ctx.goals.get(input.goal_id);
++ const g = await ctx.goals.get(q, input.goal_id);
if (!g || g.creator_subject !== profile.creator_subject || g.status !== 'active') fail(422, 'tips.goal_not_found', 'that goal is not open on this page');
request.goal_id = g.id;
}
@@@ -129,29 +144,27 @@ function createInteractions(ctx) {
}
// ── Creation ─────────────────────────────────────────────
-- function insert(row) {
++ async function insert(q, row) {
const at = iso(ctx.now());
const r = {
-- supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: 0, billing_txn_id: null,
++ supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: false, billing_txn_id: null,
billing_intent_id: null, funding_txn_id: null, provider: null, provider_ref: null, legacy_source: null, idempotency_key: null,
-- target: null, origin: 'tips', anonymous: 0, hide_amount: 0, private_message: 0, created_at: at, ...row,
++ target: null, origin: 'tips', anonymous: false, hide_amount: false, private_message: false, created_at: at, ...row,
++ };
++ const values = {
++ ...r, request: sql.json(r.request || {}), target: r.target ? sql.json(r.target) : null, test: !!r.test,
++ anonymous: !!r.anonymous, hide_amount: !!r.hide_amount, private_message: !!r.private_message, updated_at: r.created_at,
};
-- db.prepare(`INSERT INTO tip_interactions (id, creator_subject, supporter_subject, supporter_name, kind, amount, amount_cents, message, request,
-- funding, settlement, payment_state, delivery_state, test, billing_txn_id, billing_intent_id, funding_txn_id, provider, provider_ref,
-- legacy_source, idempotency_key, origin, target, anonymous, hide_amount, private_message, created_at, updated_at)
-- VALUES (@id, @creator_subject, @supporter_subject, @supporter_name, @kind, @amount, @amount_cents, @message, @request,
-- @funding, @settlement, @payment_state, @delivery_state, @test, @billing_txn_id, @billing_intent_id, @funding_txn_id, @provider, @provider_ref,
-- @legacy_source, @idempotency_key, @origin, @target, @anonymous, @hide_amount, @private_message, @created_at, @created_at)`)
-- .run({ ...r, request: JSON.stringify(r.request || {}), target: r.target ? JSON.stringify(r.target) : null, anonymous: r.anonymous ? 1 : 0, hide_amount: r.hide_amount ? 1 : 0, private_message: r.private_message ? 1 : 0 });
-- return get(r.id);
++ return q.one(sql`INSERT INTO tip_interactions ${sql.insert([values], COLUMNS)} RETURNING *`);
}
/**
* A supporter's request (checkout or credit). Idempotent by `idempotencyKey` (the caller's key,
-- * scoped to the caller): a repeat returns the interaction it created the first time.
++ * scoped to the caller): a repeat returns the interaction it created the first time. Serializable:
++ * the unpaid-checkout count and the new row are one decision.
*/
-- function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {
-- const v = validate(profile, input);
++ async function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {
++ const v = await validate(db, profile, input);
const from = userSubject(supporter, 'supporter');
if (from === profile.creator_subject) fail(422, 'tips.self_dealing', 'you cannot tip yourself');
if (!['credit', 'checkout'].includes(funding)) fail(422, 'tips.invalid_input', "pay_with must be 'credit' or 'checkout'");
@@@ -161,25 +174,26 @@ function createInteractions(ctx) {
const target = entityRef(input.target);
const privacy = parsePrivacy(input.privacy, v.kind);
if (!privacy.anonymous) checkName(profile, displayName(supporterName));
-- return ctx.tx(() => {
++ return ctx.tx(async (t) => {
if (idempotencyKey) {
-- const prev = db.prepare('SELECT * FROM tip_interactions WHERE idempotency_key = ?').get(idempotencyKey);
++ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE idempotency_key = ${idempotencyKey}`);
if (prev) return { interaction: prev, replay: true };
}
if (funding === 'checkout') {
// Each unpaid checkout is a Billing intent and a row here: bounded per supporter.
const since = iso(ctx.now() - 24 * 3600 * 1000);
-- const open = db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ? AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= ?").get(from, since).n;
++ const open = await t.value(sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = ${from} AND funding = 'checkout'
++ AND payment_state = 'pending' AND created_at >= ${since}`);
if (open >= config.limits.pendingCheckouts) fail(429, 'tips.too_many_pending', `you have ${open} unpaid checkouts; finish or wait for them before starting another`);
}
-- const i = insert({
++ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: from, supporter_name: displayName(supporterName) || 'Someone',
kind: v.kind, amount: v.amount, message: v.message, request: v.request, funding, settlement: 'billing', payment_state: 'pending',
delivery_state: 'awaiting_payment', idempotency_key: idempotencyKey || null, target,
...privacy, private_message: privacy.private_message && !!v.message,
});
return { interaction: i, replay: false };
-- });
++ }, MONEY);
}
// ── Paying through Billing ───────────────────────────────
@@@ -194,17 +208,18 @@ function createInteractions(ctx) {
key: `tips:transfer:${i.id}`, traceparent,
});
const txn = out.transaction;
-- ctx.tx(() => settleByTransaction(get(i.id), { id: txn.id, test: !!txn.test }));
++ await ctx.tx(async (t) => settleByTransaction(t, await lock(t, i.id), { id: txn.id, test: !!txn.test }), MONEY);
return { settled: true };
} catch (e) {
if (e instanceof BillingCallError && !e.retryable) {
-- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));
++ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);
return { settled: false, refused: e.code, detail: e.body && e.body.detail };
}
-- const cur = get(i.id);
-- const n = cur.transfer_attempts + 1;
-- db.prepare('UPDATE tip_interactions SET transfer_due = 1, transfer_attempts = ?, next_transfer_at = ?, failure = ?, updated_at = ? WHERE id = ? AND payment_state = \'pending\'')
-- .run(n, ctx.now() + TRANSFER_BACKOFF[Math.min(n - 1, TRANSFER_BACKOFF.length - 1)], String(e.message).slice(0, 300), iso(ctx.now()), i.id);
++ // Due again after the attempt's backoff (TRANSFER_BACKOFF[attempts - 1], the last one repeating).
++ await db.exec(sql`UPDATE tip_interactions SET transfer_due = true, transfer_attempts = transfer_attempts + 1,
++ next_transfer_at = ${ctx.now()} + (${TRANSFER_BACKOFF}::bigint[])[LEAST(transfer_attempts + 1, ${TRANSFER_BACKOFF.length})],
++ failure = ${String(e.message).slice(0, 300)}, updated_at = ${iso(ctx.now())}
++ WHERE id = ${i.id} AND payment_state = 'pending'`);
ctx.log.warn(`[Tips] transfer for ${i.id} will be retried: ${e.message}`);
return { settled: false, retrying: true };
}
@@@ -222,7 +237,7 @@ function createInteractions(ctx) {
});
} catch (e) {
if (e instanceof BillingCallError && !e.retryable) {
-- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));
++ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);
fail(e.status === 409 ? 409 : 422, e.code || 'billing.refused', (e.body && e.body.detail) || 'Billing refused the checkout');
}
fail(502, 'tips.billing_unavailable', 'checkout is unavailable right now; nothing was charged — try again shortly');
@@@ -235,63 +250,72 @@ function createInteractions(ctx) {
}
// Pages link and redirect to it: only an https URL is ever kept.
if (url && !/^https:\/\/[^\s"'<>]+$/.test(url)) url = null;
-- db.prepare('UPDATE tip_interactions SET billing_intent_id = ?, checkout_url = ?, checkout_ref = ?, provider = ?, updated_at = ? WHERE id = ?')
-- .run(intent.id || null, url, ref, p, iso(ctx.now()), i.id);
-- return { interaction: get(i.id), checkout_url: url, checkout_ref: ref, intent };
++ const row = await db.one(sql`UPDATE tip_interactions SET billing_intent_id = ${intent.id || null}, checkout_url = ${url}, checkout_ref = ${ref}, provider = ${p},
++ updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);
++ return { interaction: row, checkout_url: url, checkout_ref: ref, intent };
}
-- /** Due transfers (credit retries and funded checkouts). Called by the jobs loop. */
++ /**
++ * Due transfers (credit retries and funded checkouts). Called by the jobs loop in every process:
++ * each batch is claimed with a lease (FOR UPDATE SKIP LOCKED), so no two processes try one at once.
++ */
async function processDueTransfers() {
-- const rows = db.prepare("SELECT * FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due = 1 AND next_transfer_at <= ? ORDER BY next_transfer_at LIMIT 20").all(ctx.now());
++ const now = ctx.now();
++ const rows = await db.many(sql`UPDATE tip_interactions SET next_transfer_at = ${now + TRANSFER_LEASE_MS}
++ WHERE id IN (SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= ${now}
++ ORDER BY next_transfer_at LIMIT ${TRANSFER_BATCH} FOR UPDATE SKIP LOCKED)
++ RETURNING *`);
++ rows.sort((a, b) => (a.created_at < b.created_at ? -1 : 1));
let settled = 0;
for (const i of rows) { const r = await transfer(i); if (r.settled) settled++; }
return { attempted: rows.length, settled };
}
// ── Settlement ───────────────────────────────────────────
-- /** Inside a transaction: settle `i` by Billing transaction `txn` ({ id, test }). Idempotent. */
-- function settleByTransaction(i, txn) {
++ /** Inside a MONEY transaction, `i` locked: settle it by Billing transaction `txn` ({ id, test }). Idempotent. */
++ async function settleByTransaction(t, i, txn) {
if (!i) return null;
if (i.billing_txn_id === txn.id || i.payment_state === 'settled' || i.payment_state === 'reversed') return i;
-- const other = byBillingTxn(txn.id);
++ const other = await byBillingTxn(t, txn.id);
if (other && other.id !== i.id) return other;
-- db.prepare('UPDATE tip_interactions SET billing_txn_id = ?, test = MAX(test, ?), transfer_due = 0, failure = NULL WHERE id = ?').run(txn.id, txn.test ? 1 : 0, i.id);
-- return settle(get(i.id));
++ const row = await t.one(sql`UPDATE tip_interactions SET billing_txn_id = ${txn.id}, test = test OR ${!!txn.test}, transfer_due = false, failure = NULL
++ WHERE id = ${i.id} RETURNING *`);
++ return settle(t, row);
}
-- /** Inside a transaction: pending/failed → settled, and the whole effect path. */
-- function settle(i) {
++ /** Inside a MONEY transaction, `i` locked (or created by it): pending/failed → settled, and the whole effect path. */
++ async function settle(t, i) {
const at = iso(ctx.now());
const test = !!i.test;
-- const req = json(i.request, {});
++ const req = i.request || {};
// A tip hidden by a moderator while its payment was pending will deliver nothing.
-- db.prepare(`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,
-- settled_at = ?, updated_at = ? WHERE id = ?`).run(at, at, i.id);
-- i = get(i.id);
++ i = await t.one(sql`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,
++ settled_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
if (i.kind === 'paid_message' || i.kind === 'tts') {
-- db.prepare(`INSERT OR IGNORE INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)
-- VALUES (?, ?, ?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tpm', ctx.now()), i.id, i.creator_subject, i.kind,
-- i.kind === 'tts' ? req.tts.text : (i.message || ''), i.kind === 'tts' ? req.tts.voice : null, req.highlight_seconds || 0, test ? 1 : 0, at, at);
++ await t.exec(sql`INSERT INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)
++ VALUES (${prefixedId('tpm', ctx.now())}, ${i.id}, ${i.creator_subject}, ${i.kind}, ${i.kind === 'tts' ? req.tts.text : (i.message || '')},
++ ${i.kind === 'tts' ? req.tts.voice : null}, ${req.highlight_seconds || 0}, 'queued', ${test}, ${at}, ${at}) ON CONFLICT DO NOTHING`);
}
if (i.kind === 'media_request' && req.media) {
-- db.prepare(`INSERT OR IGNORE INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)
-- VALUES (?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tmr', ctx.now()), i.id, i.creator_subject, req.media.url, req.media.provider, test ? 1 : 0, at, at);
++ await t.exec(sql`INSERT INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)
++ VALUES (${prefixedId('tmr', ctx.now())}, ${i.id}, ${i.creator_subject}, ${req.media.url}, ${req.media.provider}, 'queued', ${test}, ${at}, ${at})
++ ON CONFLICT DO NOTHING`);
}
// The creator's word filter runs before anything is shown, posted or read (moderation.js): a
// match is masked, or held for review when the creator chose that. History is not screened.
-- const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i) : { hit: false };
-- if (verdict.hit) ctx.moderation.markScreened(i, verdict);
-- i = get(i.id);
-- if (!test && i.origin !== 'import') ctx.goals.contribute(i, req.goal_id);
-- if (i.origin !== 'import' && i.moderation === 'visible') release(i);
-- recomputeDelivery(i.id);
++ const filter = await ctx.profiles.filterOf(t, i.creator_subject);
++ const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i, filter) : { hit: false };
++ if (verdict.hit) i = await ctx.moderation.markScreened(t, i, verdict);
++ if (!test && i.origin !== 'import') await ctx.goals.contribute(t, i, req.goal_id, filter);
++ if (i.origin !== 'import' && i.moderation === 'visible') await release(t, i, filter);
++ i = await recomputeDelivery(t, i.id);
if (!test && i.origin !== 'import') {
-- ctx.outbox.emit('tips.interaction.ready', { type: 'interaction', id: i.id }, summary(get(i.id)));
++ await ctx.outbox.emit(t, 'tips.interaction.ready', { type: 'interaction', id: i.id }, summary(i));
}
-- if (verdict.hit) ctx.moderation.recordScreened(get(i.id), verdict);
-- ctx.afterCommit(() => { ctx.effects.kick(); ctx.outboxKick(); });
-- return get(i.id);
++ if (verdict.hit) await ctx.moderation.recordScreened(t, i, verdict);
++ ctx.afterCommit(t, () => { ctx.effects.kick(); ctx.outboxKick(); });
++ return i;
}
/**
@@@ -299,97 +323,90 @@ function createInteractions(ctx) {
* deliveries. Settlement calls it, and moderation's restore() for one the filter held. Idempotent:
* nothing that exists (delivered or cancelled) is created again.
*/
-- function release(i) {
++ async function release(t, i, filter) {
const test = !!i.test;
-- ctx.overlays.addAlert(i);
-- addEffect(i, 'overlay_alert', 'overlay', 'delivered');
++ await ctx.overlays.addAlert(t, i, filter);
++ const at = iso(ctx.now());
++ const effects = [{ effect: 'overlay_alert', adapter: 'overlay', state: 'delivered' }];
// Chat effects: never for imports (history), never twice for a donation another product already
// announced (origin billing), never to a real chat room for a simulation.
const adapter = test ? 'test' : config.chat.adapter;
if (ANNOUNCED_HERE.includes(i.origin) && adapter !== 'none') {
-- if (i.kind === 'tip') addEffect(i, 'chat_line', adapter);
-- if (i.kind === 'paid_message') addEffect(i, 'paid_message', adapter);
-- if (i.kind === 'tts') { addEffect(i, 'chat_line', adapter); addEffect(i, 'tts', adapter); }
-- if (i.kind === 'media_request') addEffect(i, 'media_request', adapter);
++ const chat = { tip: ['chat_line'], paid_message: ['paid_message'], tts: ['chat_line', 'tts'], media_request: ['media_request'] }[i.kind] || [];
++ for (const effect of chat) effects.push({ effect, adapter, state: 'queued' });
}
-- recomputeDelivery(i.id);
-- ctx.afterCommit(() => ctx.effects.kick());
++ await t.exec(sql`INSERT INTO interaction_effects ${sql.insert(effects.map((e) => ({ interaction_id: i.id, ...e, created_at: at, updated_at: at })))}
++ ON CONFLICT DO NOTHING`);
++ await recomputeDelivery(t, i.id);
++ ctx.afterCommit(t, () => ctx.effects.kick());
}
-- function addEffect(i, effect, adapter, state = 'queued') {
-- const at = iso(ctx.now());
-- db.prepare(`INSERT OR IGNORE INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?)`)
-- .run(i.id, effect, adapter, state, at, at);
-- }
--
-- /** Inside a transaction: pending → failed (nothing was delivered; nothing moved in Billing). */
-- function failPayment(i, code, detail) {
++ /** Inside a MONEY transaction, `i` locked: pending → failed (nothing was delivered; nothing moved in Billing). */
++ async function failPayment(t, i, code, detail) {
if (!i || i.payment_state !== 'pending') return i;
-- const at = iso(ctx.now());
-- db.prepare("UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = 0, failure = ?, updated_at = ? WHERE id = ?")
-- .run(`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`, at, i.id);
-- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_failed', code });
-- return get(i.id);
++ const row = await t.one(sql`UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = false,
++ failure = ${`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`}, updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);
++ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_failed', code });
++ return row;
}
-- /** Billing settled a transaction. Inside the inbox transaction. */
-- function onBillingSettled(p) {
++ /** Billing settled a transaction. Inside the inbox's MONEY transaction. */
++ async function onBillingSettled(t, p) {
const meta = p.metadata || {};
const txn = { id: p.transaction_id, test: !!p.test };
if (!txn.id) return 'ignored:no_transaction';
if (p.type === 'donation') {
-- const known = byBillingTxn(txn.id);
-- if (known) return 'duplicate_transaction';
++ if (await byBillingTxn(t, txn.id)) return 'duplicate_transaction';
const target = meta.target;
if (target && target.service === 'tips' && target.type === 'interaction') {
-- const i = get(target.id);
++ const i = await lock(t, target.id);
if (!i) return 'ignored:unknown_interaction';
// Only the transfer Tips asked for settles the interaction: same amount, same creator, same
// supporter. Another service's donation that merely names it (a 1-bit transfer claiming a
// 10,000-bit paid message) is recorded as what it is, a donation of its own amount.
const matches = Number(meta.amount_bits) === i.amount && p.to_subject === i.creator_subject && (!i.supporter_subject || p.from_subject === i.supporter_subject);
-- if (matches) { settleByTransaction(i, txn); return 'settled'; }
++ if (matches) { await settleByTransaction(t, i, txn); return 'settled'; }
ctx.log.warn(`[Tips] donation ${txn.id} names interaction ${i.id} but does not match it (amount, creator or supporter); recorded on its own`);
}
// A donation Tips did not start: record it once, by its transaction id.
if (!p.to_subject) return 'ignored:no_recipient';
const amount = Number(meta.amount_bits) || 0;
if (amount <= 0) return 'ignored:no_amount';
-- const i = insert({
++ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: p.to_subject, supporter_subject: p.from_subject || null,
supporter_name: displayName(meta.donor_name) || null, anonymous: meta.anonymous === true, kind: meta.kind === 'paid_interaction' ? 'media_request' : 'tip', amount, amount_cents: meta.paid_cents || null,
-- message: meta.message ? String(meta.message).slice(0, config.limits.messageChars) : null,
++ message: meta.message ? storable(String(meta.message).slice(0, config.limits.messageChars)) : null,
funding: p.provider ? 'provider' : 'credit', settlement: 'billing', payment_state: 'pending', delivery_state: 'awaiting_payment',
-- test: txn.test ? 1 : 0, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',
++ test: txn.test, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',
target: meta.target && meta.target.service ? meta.target : null,
});
-- settle(i);
++ await settle(t, i);
return 'recorded';
}
if (p.type === 'purchase' && meta.intent_id) {
-- const i = db.prepare('SELECT * FROM tip_interactions WHERE billing_intent_id = ?').get(meta.intent_id);
++ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_intent_id = ${meta.intent_id} FOR UPDATE`);
if (!i) return 'ignored:not_a_tips_checkout';
if (i.funding_txn_id) return 'duplicate_funding';
-- db.prepare("UPDATE tip_interactions SET funding_txn_id = ?, test = MAX(test, ?), transfer_due = CASE WHEN payment_state = 'pending' THEN 1 ELSE 0 END, next_transfer_at = 0, updated_at = ? WHERE id = ?")
-- .run(txn.id, txn.test ? 1 : 0, iso(ctx.now()), i.id);
-- ctx.afterCommit(() => { processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)); });
++ await t.exec(sql`UPDATE tip_interactions SET funding_txn_id = ${txn.id}, test = test OR ${txn.test}, transfer_due = (payment_state = 'pending'),
++ next_transfer_at = 0, updated_at = ${iso(ctx.now())} WHERE id = ${i.id}`);
++ ctx.afterCommit(t, () => processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)));
return 'funded';
}
return 'ignored';
}
-- /** Billing reversed a transaction. Inside the inbox transaction. */
-- function onBillingReversed(p) {
++ /** Billing reversed a transaction. Inside the inbox's MONEY transaction. */
++ async function onBillingReversed(t, p) {
const meta = p.metadata || {};
const orig = p.reverses_txn;
if (!orig) return 'ignored:no_original';
-- const i = byBillingTxn(orig);
++ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${orig} FOR UPDATE`);
if (!i) {
-- const funded = db.prepare('SELECT * FROM tip_interactions WHERE funding_txn_id = ?').get(orig);
-- if (funded && funded.payment_state === 'pending') { failPayment(funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }
++ const funded = await t.maybe(sql`SELECT * FROM tip_interactions WHERE funding_txn_id = ${orig} FOR UPDATE`);
++ if (funded && funded.payment_state === 'pending') { await failPayment(t, funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }
return 'ignored';
}
-- const ids = json(i.reversal_txn_ids, []);
++ const ids = Array.isArray(i.reversal_txn_ids) ? [...i.reversal_txn_ids] : [];
if (ids.includes(p.transaction_id)) return 'duplicate_reversal';
ids.push(p.transaction_id);
// Bits Billing took back from the creator: a transfer refund moves them out of the payable. A
@@@ -397,52 +414,54 @@ function createInteractions(ctx) {
const clawed = p.type === 'refund' && p.from_subject === i.creator_subject ? Math.max(0, Number(meta.amount_bits) || 0) : 0;
const take = Math.min(clawed, i.amount - i.reversed_bits);
const at = iso(ctx.now());
-- db.prepare("UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ?, reversed_bits = reversed_bits + ?, reversed_at = ?, updated_at = ? WHERE id = ?")
-- .run(JSON.stringify(ids), take, at, at, i.id);
-- if (take > 0 && !i.test) ctx.goals.reverse(i, take);
++ const reversed = await t.one(sql`UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ${sql.json(ids)}, reversed_bits = reversed_bits + ${take},
++ reversed_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
++ if (take > 0 && !i.test) await ctx.goals.reverse(t, reversed, take);
// Undelivered effects are cancelled; a delivery that already happened stays on record.
-- let cancelled = cancelQueued(get(i.id), 'payment_reversed');
-- const now = get(i.id);
++ let cancelled = await cancelQueued(t, reversed, 'payment_reversed');
++ const now = await get(t, i.id);
if (!cancelled && now.moderation === 'held' && now.delivery_state === 'queued') {
// Held by the filter and never shown: it will not be released now.
-- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);
-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
-- if (!now.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_reversed' });
++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
++ if (!now.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_reversed' });
cancelled = true;
}
return cancelled ? 'reversed_cancelled' : 'reversed';
}
-- /** Inside a transaction: queued effects → cancelled. True when anything was cancelled. */
-- function cancelQueued(i, reason) {
++ /** Inside a transaction, the interaction locked: queued effects → cancelled. True when anything was cancelled. */
++ async function cancelQueued(t, i, reason) {
const at = iso(ctx.now());
-- const r = db.prepare("UPDATE interaction_effects SET state = 'cancelled', updated_at = ? WHERE interaction_id = ? AND state = 'queued'").run(at, i.id);
-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
-- if (!r.changes) return false;
-- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);
-- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason });
++ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND state = 'queued'`);
++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
++ if (!n) return false;
++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
++ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason });
return true;
}
-- /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. */
-- function recomputeDelivery(id) {
-- const i = get(id);
++ /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. Returns the row. */
++ async function recomputeDelivery(t, id) {
++ const i = await get(t, id);
if (!i || i.delivery_state === 'cancelled' || i.delivery_state === 'awaiting_payment') return i;
-- const effects = db.prepare('SELECT state FROM interaction_effects WHERE interaction_id = ?').all(id).map((e) => e.state);
-- if (i.moderation === 'held' && !effects.length) return i; // waiting for a moderator: still to deliver
++ const effects = await t.many(sql`SELECT effect, adapter, state, last_error FROM interaction_effects WHERE interaction_id = ${id} ORDER BY id`);
++ const states = effects.map((e) => e.state);
++ if (i.moderation === 'held' && !states.length) return i; // waiting for a moderator: still to deliver
let next = 'queued';
-- if (effects.includes('failed') && !effects.includes('queued')) next = 'failed';
-- else if (!effects.includes('queued')) next = 'delivered';
++ if (states.includes('failed') && !states.includes('queued')) next = 'failed';
++ else if (!states.includes('queued')) next = 'delivered';
if (next === i.delivery_state) return i;
const at = iso(ctx.now());
-- db.prepare(`UPDATE tip_interactions SET delivery_state = ?, delivered_at = CASE WHEN ? = 'delivered' THEN ? ELSE delivered_at END, updated_at = ? WHERE id = ?`).run(next, next, at, at, id);
++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = ${next},
++ delivered_at = CASE WHEN ${next === 'delivered'} THEN ${at}::timestamptz ELSE delivered_at END, updated_at = ${at} WHERE id = ${id} RETURNING *`);
if (next === 'failed' && !i.test) {
-- const failed = db.prepare("SELECT effect, adapter, last_error FROM interaction_effects WHERE interaction_id = ? AND state = 'failed'").all(id);
-- ctx.outbox.emit('tips.interaction.failed', { type: 'interaction', id }, { ...summary(get(id)), failed_effects: failed });
++ const failed = effects.filter((e) => e.state === 'failed').map(({ effect, adapter, last_error: lastError }) => ({ effect, adapter, last_error: lastError }));
++ await ctx.outbox.emit(t, 'tips.interaction.failed', { type: 'interaction', id }, { ...summary(row), failed_effects: failed });
}
-- return get(id);
++ return row;
}
/**
@@@ -450,12 +469,12 @@ function createInteractions(ctx) {
* or a Live donation_goals id (links minted by Live), which the Live import mapped to a Tips goal.
* goals.contribute() still checks it is the creator's and active, else falls back to Live's rule.
*/
-- function goalFromPurpose(...refs) {
++ async function goalFromPurpose(q, ...refs) {
for (const v of refs) {
-- const m = String(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);
++ const m = storable(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);
if (!m) continue;
if (/^\d+$/.test(m[1])) {
-- const map = db.prepare("SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ? AND status = 'imported'").get(m[1]);
++ const map = await q.maybe(sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ${m[1]} AND status = 'imported'`);
if (map && map.target_id) return map.target_id;
continue;
}
@@@ -466,84 +485,84 @@ function createInteractions(ctx) {
/**
* Billing announced an EXTERNAL receipt (billing.receipt.external): a tip paid on the creator's own
-- * PowerChat. Inside the inbox transaction. Recorded once by (provider, provider event id) — the same
-- * key POST /interactions/external uses — settled at once (the provider already confirmed it), and
-- * announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.
++ * PowerChat. Inside the inbox's MONEY transaction. Recorded once by (provider, provider event id) —
++ * the same key POST /interactions/external uses — settled at once (the provider already confirmed
++ * it), and announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.
*/
-- function onBillingExternal(p) {
++ async function onBillingExternal(t, p) {
const provider = String(p.provider || '').toLowerCase();
-- const ref = String(p.provider_event_id || '').trim();
++ const ref = storable(p.provider_event_id || '').trim();
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider) || !ref || ref.length > 200) return 'ignored:no_reference';
const creator = p.streamer && p.streamer.type === 'user' && SUBJECT_RE.test(String(p.streamer.id || '')) ? p.streamer.id : null;
if (!creator) return 'ignored:no_streamer';
const cents = Number(p.amount_cents);
if (!Number.isSafeInteger(cents) || cents < 1 || cents > MAX_EXTERNAL_CENTS) return 'ignored:no_amount';
-- if (db.prepare('SELECT 1 FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) return 'duplicate_receipt';
++ if (await t.maybe(sql`SELECT 1 FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`)) return 'duplicate_receipt';
// Goals and the chat line count Vibes: the value Billing put on the money (1 bit = 1 cent today).
const bits = Number(p.value_bits);
const amount = Math.min(Number.isSafeInteger(bits) && bits > 0 ? bits : cents, config.limits.maxBits);
-- const message = p.message ? String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars) || null : null;
-- const goalId = goalFromPurpose(p.app_purpose, p.app_ref);
-- const i = insert({
++ const message = p.message ? storable(String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars)) || null : null;
++ const goalId = await goalFromPurpose(t, p.app_purpose, p.app_ref);
++ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: creator, supporter_subject: null,
supporter_name: p.anonymous ? 'Anonymous' : (displayName(p.donor_name) || 'Someone'), anonymous: !!p.anonymous, kind: 'tip', amount, amount_cents: cents,
message, request: goalId ? { goal_id: goalId } : {}, funding: 'external', settlement: 'external', payment_state: 'pending',
-- delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: p.test ? 1 : 0,
++ delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: !!p.test,
});
-- settle(i);
++ await settle(t, i);
return 'recorded_external';
}
// ── External and simulated ───────────────────────────────
/** A tip on the creator's own PowerChat (EXTERNAL, ADR-012): recorded once by provider ref. */
-- function recordExternal(profile, input) {
++ async function recordExternal(profile, input) {
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip profile');
const provider = String(input.provider || '').toLowerCase();
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider)) fail(422, 'tips.invalid_input', 'provider is required');
-- const ref = String(input.provider_ref || '').trim();
++ const ref = storable(input.provider_ref || '').trim();
if (!ref || ref.length > 200) fail(422, 'tips.invalid_input', 'provider_ref (the provider event id) is required');
const cents = positiveInt(input.amount_cents, 'amount_cents', 100_000_000);
const privacy = parsePrivacy(input.privacy, 'tip');
-- return ctx.tx(() => {
-- const prev = db.prepare('SELECT * FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref);
++ return ctx.tx(async (t) => {
++ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`);
if (prev) return { interaction: prev, replay: true };
-- const i = insert({
++ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: input.supporter ? userSubject(input.supporter, 'supporter') : null,
supporter_name: displayName(input.supporter_name) || 'Someone', kind: 'tip', amount: cents, amount_cents: cents,
message: text(input.message, 'message', 500), request: input.goal_id ? { goal_id: String(input.goal_id) } : {},
funding: 'external', settlement: 'external', payment_state: 'pending', delivery_state: 'awaiting_payment',
// origin external: the provider (or Live's webhook) already announced it in chat; with
// announce: true Tips delivers the chat line itself.
-- provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: input.test ? 1 : 0,
++ provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: !!input.test,
...privacy, private_message: privacy.private_message && !!input.message,
});
-- return { interaction: settle(i), replay: false };
-- });
++ return { interaction: await settle(t, i), replay: false };
++ }, MONEY);
}
-- /** The full effect path with test = 1 and no Billing call. */
-- function simulate(profile, input, { by }) {
-- const v = validate(profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });
++ /** The full effect path with test = true and no Billing call. */
++ async function simulate(profile, input, { by }) {
++ const v = await validate(db, profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });
const privacy = parsePrivacy(input.privacy, v.kind);
-- return ctx.tx(() => {
-- const i = insert({
++ return ctx.tx(async (t) => {
++ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: null,
supporter_name: displayName(input.supporter_name) || 'Test supporter', kind: v.kind, amount: v.amount, message: v.message,
request: { ...v.request, simulated_by: by }, funding: 'none', settlement: 'simulated', payment_state: 'pending',
-- delivery_state: 'awaiting_payment', test: 1, ...privacy, private_message: privacy.private_message && !!v.message,
++ delivery_state: 'awaiting_payment', test: true, ...privacy, private_message: privacy.private_message && !!v.message,
});
-- const out = settle(i);
++ const out = await settle(t, i);
// Show the goal widget moving without counting anything: a test goal delivery with the
// would-be total. No contribution row is written.
-- const g = ctx.goals.pick(profile.creator_subject, v.request.goal_id);
++ const g = await ctx.goals.pick(t, profile.creator_subject, v.request.goal_id);
if (g) {
-- const view = ctx.goals.present(g);
++ const [view] = await ctx.goals.presentMany(t, [g]);
const would = { ...view, current_amount: view.current_amount + v.amount, percent: Math.min(100, Math.floor(((view.current_amount + v.amount) * 100) / view.target_amount)) };
-- const shown = get(i.id);
-- ctx.overlays.addGoalDelivery(profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: shown.hide_amount || shown.moderation !== 'visible' ? null : ctx.view(shown).supporter_name, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });
++ const by2 = out.hide_amount || out.moderation !== 'visible' ? null : publicView(out, { filter: profile.filter }).supporter_name;
++ await ctx.overlays.addGoalDelivery(t, profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: by2, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });
}
return out;
-- });
++ }, MONEY);
}
// ── Reads ────────────────────────────────────────────────
@@@ -557,14 +576,8 @@ function createInteractions(ctx) {
};
}
-- /**
-- * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or
-- * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;
-- * `public` is what may be shown to the public, `privacy` the supporter's choices.
-- */
-- function present(i, { viewer = 'owner' } = {}) {
-- if (!i) return null;
-- const req = json(i.request, {});
++ function shape(i, viewer, filter, effects) {
++ const req = i.request || {};
const own = viewer === 'supporter';
const named = own || !isAnonymous(i);
const out = {
@@@ -572,15 +585,15 @@ function createInteractions(ctx) {
supporter_name: named ? (i.supporter_name || null) : publicName(i), kind: i.kind, amount: i.amount, currency: i.currency, message: i.message || null,
tts: req.tts || null, media: req.media ? { url: req.media.url, provider: req.media.provider } : null, goal_id: req.goal_id || null,
funding: i.funding, settlement: i.settlement, origin: i.origin, test: !!i.test,
-- privacy: privacyOf(i), public: ctx.view(i), erased_at: i.erased_at || null,
++ privacy: privacyOf(i), public: publicView(i, { filter }), erased_at: i.erased_at || null,
moderation: { state: i.moderation, filtered: !!i.filtered, at: i.moderated_at || null },
-- payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: json(i.reversal_txn_ids, []), reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },
++ payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: i.reversal_txn_ids || [], reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },
delivery: { state: i.delivery_state, delivered_at: i.delivered_at || null },
created_at: i.created_at, updated_at: i.updated_at,
};
if (i.funding === 'checkout') out.checkout = { url: i.checkout_url || null, ref: i.checkout_ref || null, provider: i.provider || null, intent_id: i.billing_intent_id || null };
if (viewer === 'owner' || viewer === 'service') {
-- out.effects = db.prepare('SELECT effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);
++ out.effects = effects;
out.provider = i.provider || null;
out.provider_ref = i.provider_ref || null;
out.legacy_source = i.legacy_source || null;
@@@ -588,55 +601,83 @@ function createInteractions(ctx) {
return out;
}
-- /** Cursor-paged list, newest first. filter: { creator?, supporter?, includeTest? } */
-- function list({ creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {
++ /**
++ * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or
++ * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;
++ * `public` is what may be shown to the public, `privacy` the supporter's choices. A page of any
++ * size costs two queries: the creators' filters, and (owner, service) every row's effects.
++ */
++ async function presentMany(q, rows, { viewer = 'owner' } = {}) {
++ if (!rows.length) return [];
++ const filters = await ctx.profiles.filtersOf(q, rows.map((i) => i.creator_subject));
++ const effects = new Map();
++ if (viewer === 'owner' || viewer === 'service') {
++ const list = await q.many(sql`SELECT interaction_id, effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects
++ WHERE interaction_id = ANY(${rows.map((i) => i.id)}) ORDER BY interaction_id, id`);
++ for (const { interaction_id: id, ...e } of list) { if (!effects.has(id)) effects.set(id, []); effects.get(id).push(e); }
++ }
++ return rows.map((i) => shape(i, viewer, filters.get(i.creator_subject), effects.get(i.id) || []));
++ }
++ const present = async (q, i, opts) => (i ? (await presentMany(q, [i], opts))[0] : null);
++
++ /** A keyset page, newest first. filter: { creator?, supporter?, includeTest? } (one of the first two). */
++ async function list(q, { creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {
const n = Math.min(200, Math.max(1, Number(limit) || 50));
-- let cur = null;
-- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }
-- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE (@c IS NULL OR creator_subject = @c) AND (@s IS NULL OR supporter_subject = @s)
-- AND (@t = 1 OR test = 0) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))
-- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator || null, s: supporter || null, t: includeTest ? 1 : 0, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });
++ const cur = cursor ? readCursor(cursor) : null;
++ const where = [];
++ if (creator) where.push(sql`creator_subject = ${creator}`);
++ if (supporter) where.push(sql`supporter_subject = ${supporter}`);
++ if (!where.length) fail(422, 'tips.invalid_input', 'creator or supporter is required');
++ if (!includeTest) where.push(sql`NOT test`);
++ if (cur) where.push(sql`(created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)`);
++ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE ${sql.join(where, sql` AND `)} ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);
const page = rows.slice(0, n);
-- const next = rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null;
-- return { rows: page, next_cursor: next };
++ return { rows: page, next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };
}
// ── The public supporters page ───────────────────────────
-- // Only settled, non-test interactions of supporters who kept both their name and their amount
-- // public count toward the leaderboard (a hidden amount must not be inferable from a rank).
-- const leaderboardWhere = (t) => `${t}.creator_subject = @c AND ${t}.supporter_subject IS NOT NULL AND ${t}.test = 0
-- AND ${t}.payment_state IN ('settled', 'reversed') AND ${t}.anonymous = 0 AND ${t}.hide_amount = 0 AND ${t}.erased_at IS NULL
-- AND ${t}.moderation = 'visible'`;
--
-- /** Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). */
-- function leaderboard(creator, { limit = 20 } = {}) {
-- const rows = db.prepare(`SELECT t.supporter_subject AS s, SUM(t.amount - t.reversed_bits) AS total, COUNT(*) AS tips, MIN(t.created_at) AS first_at,
-- (SELECT j.supporter_name FROM tip_interactions j WHERE ${leaderboardWhere('j')} AND j.supporter_subject = t.supporter_subject
-- ORDER BY j.created_at DESC LIMIT 1) AS name
-- FROM tip_interactions t WHERE ${leaderboardWhere('t')}
-- GROUP BY t.supporter_subject HAVING total > 0 ORDER BY total DESC, first_at LIMIT @n`).all({ c: creator, n: Math.min(100, Math.max(1, limit)) });
-- const settings = ctx.profiles.filterOf(creator);
-- return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, settings), total: r.total, tips: r.tips }));
++ /**
++ * Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). Only
++ * settled, non-test interactions of supporters who kept both their name and their amount public
++ * count (a hidden amount must not be inferable from a rank).
++ */
++ async function leaderboard(q, profile, { limit = 20 } = {}) {
++ const rows = await q.many(sql`SELECT supporter_subject AS s, SUM(amount - reversed_bits)::bigint AS total, count(*) AS tips, MIN(created_at) AS first_at,
++ (array_agg(supporter_name ORDER BY created_at DESC))[1] AS name
++ FROM tip_interactions
++ WHERE creator_subject = ${profile.creator_subject} AND supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed')
++ AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible'
++ GROUP BY supporter_subject HAVING SUM(amount - reversed_bits) > 0
++ ORDER BY total DESC, first_at LIMIT ${Math.min(100, Math.max(1, limit))}`);
++ return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, profile.filter), total: r.total, tips: r.tips }));
}
/** Recent settled interactions with a public message, as publicView(). */
-- function recentPublic(creator, { limit = 20 } = {}) {
-- return db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND payment_state = 'settled' AND private_message = 0
-- AND erased_at IS NULL AND moderation = 'visible' AND message IS NOT NULL ORDER BY settled_at DESC, id DESC LIMIT ?`).all(creator, Math.min(100, Math.max(1, limit)))
-- .map((i) => ctx.view(i));
++ async function recentPublic(q, profile, { limit = 20 } = {}) {
++ return (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${profile.creator_subject} AND payment_state = 'settled' AND message IS NOT NULL
++ AND NOT test AND NOT private_message AND erased_at IS NULL AND moderation = 'visible' ORDER BY settled_at DESC, id DESC LIMIT ${Math.min(100, Math.max(1, limit))}`))
++ .map((i) => publicView(i, { filter: profile.filter }));
}
// ── A supporter's own data ───────────────────────────────
-- /** Everything Tips holds about a supporter's tips (their own view), for a download. */
-- function exportFor(subject) {
-- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ? ORDER BY created_at').all(subject);
++ /** Everything Tips holds about a supporter's tips (their own view), for a download: five queries. */
++ async function exportFor(subject) {
++ const rows = await db.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY created_at`);
++ const ids = rows.map((i) => i.id);
++ const views = await presentMany(db, rows, { viewer: 'supporter' });
++ const group = (list) => { const m = new Map(); for (const { interaction_id: id, ...r } of list) { if (!m.has(id)) m.set(id, []); m.get(id).push(r); } return m; };
++ const [contributions, paid, media] = ids.length ? await Promise.all([
++ db.many(sql`SELECT interaction_id, goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ANY(${ids}) ORDER BY id`),
++ db.many(sql`SELECT interaction_id, kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ANY(${ids})`),
++ db.many(sql`SELECT interaction_id, url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ANY(${ids})`),
++ ]).then((r) => r.map(group)) : [new Map(), new Map(), new Map()];
return {
service: 'tips', subject: { type: 'user', id: subject }, exported_at: iso(ctx.now()),
-- interactions: rows.map((i) => ({
-- ...present(i, { viewer: 'supporter' }),
-- goal_contributions: db.prepare('SELECT goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ?').all(i.id),
-- paid_message: db.prepare('SELECT kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ?').get(i.id) || null,
-- media_request: db.prepare('SELECT url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ?').get(i.id) || null,
++ interactions: views.map((v) => ({
++ ...v,
++ goal_contributions: contributions.get(v.id) || [],
++ paid_message: (paid.get(v.id) || [])[0] || null,
++ media_request: (media.get(v.id) || [])[0] || null,
})),
not_included: 'Payments, balances and refunds are OpenVibe.Billing\'s records; ask Billing for them.',
};
@@@ -649,74 +690,67 @@ function createInteractions(ctx) {
* answers, overlay payloads and the unsent events that named them. Each erased interaction emits
* tips.interaction.erased, whose `redacts` has OpenVibe.Events tombstone the earlier events about
* it. Interactions still waiting for their payment are kept (the payment needs the supporter) and
-- * counted in kept_pending.
++ * counted in kept_pending. The supporter's rows are locked in id order first; the writes are one
++ * statement per table.
*/
function erase(subject) {
-- return ctx.tx(() => {
-- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ?').all(subject);
++ return ctx.tx(async (t) => {
++ const rows = await t.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY id FOR UPDATE`);
const at = iso(ctx.now());
-- let erased = 0;
-- for (const i of rows) {
-- if (i.payment_state === 'pending') continue;
-- const req = json(i.request, {});
-- const keep = {};
-- if (req.goal_id) keep.goal_id = req.goal_id;
-- if (req.highlight_seconds) keep.highlight_seconds = req.highlight_seconds;
-- db.prepare(`UPDATE tip_interactions SET supporter_subject = NULL, supporter_name = NULL, anonymous = 1, message = NULL, request = ?,
-- idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ?, updated_at = ? WHERE id = ?`)
-- .run(JSON.stringify(keep), at, at, i.id);
-- db.prepare("UPDATE paid_messages SET text = '', voice = NULL, updated_at = ? WHERE interaction_id = ?").run(at, i.id);
-- db.prepare("UPDATE paid_media_requests SET url = '', updated_at = ? WHERE interaction_id = ?").run(at, i.id);
-- const now = get(i.id);
-- ctx.overlays.refreshInteraction(now);
-- scrubOutbox(now);
-- if (!i.test) {
-- ctx.outbox.emit('tips.interaction.erased', { type: 'interaction', id: i.id }, {
++ const done = rows.filter((i) => i.payment_state !== 'pending');
++ if (done.length) {
++ const keep = done.map((i) => {
++ const req = i.request || {};
++ const k = {};
++ if (req.goal_id) k.goal_id = req.goal_id;
++ if (req.highlight_seconds) k.highlight_seconds = req.highlight_seconds;
++ return { id: i.id, request: k };
++ });
++ const ids = done.map((i) => i.id);
++ const erased = await t.many(sql`UPDATE tip_interactions x SET supporter_subject = NULL, supporter_name = NULL, anonymous = true, message = NULL,
++ request = v.request, idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ${at}, updated_at = ${at}
++ FROM jsonb_to_recordset(${sql.json(keep)}) AS v(id text, request jsonb) WHERE x.id = v.id RETURNING x.*`);
++ await t.exec(sql`UPDATE paid_messages SET text = '', voice = NULL, updated_at = ${at} WHERE interaction_id = ANY(${ids})`);
++ await t.exec(sql`UPDATE paid_media_requests SET url = '', updated_at = ${at} WHERE interaction_id = ANY(${ids})`);
++ await ctx.overlays.refreshInteractions(t, erased, await ctx.profiles.filtersOf(t, erased.map((i) => i.creator_subject)));
++ // Events about them still in the local outbox lose the supporter's name.
++ await t.exec(sql`UPDATE tips_event_outbox
++ SET envelope = jsonb_set(jsonb_set(envelope, '{payload,supporter}', 'null'::jsonb), '{payload,supporter_name}', ${sql.json(ANONYMOUS)})
++ WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY(${ids}) AND envelope->'payload' ? 'supporter'`);
++ for (const i of done) {
++ if (i.test) continue;
++ await ctx.outbox.emit(t, 'tips.interaction.erased', { type: 'interaction', id: i.id }, {
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, erased_at: at,
redacts: { subject_type: 'interaction', subject_ids: [i.id] },
});
}
-- erased++;
++ t.after(ctx.outboxKick);
}
// Stored API answers (Idempotency-Key replays) carry messages and the subject.
-- const prefix = `${subject}:`;
-- const answers = db.prepare('DELETE FROM api_idempotency WHERE substr(key, 1, ?) = ? OR instr(response, ?) > 0').run(prefix.length, prefix, subject).changes;
-- if (erased) ctx.afterCommit(() => ctx.outboxKick());
-- return { erased, kept_pending: rows.length - erased, stored_answers_removed: answers };
++ const answers = await t.exec(sql`DELETE FROM api_idempotency WHERE starts_with(key, ${`${subject}:`}) OR strpos(response, ${subject}) > 0`);
++ return { erased: done.length, kept_pending: rows.length - done.length, stored_answers_removed: answers };
});
}
-- /** Events about an erased interaction that are still in the local outbox lose the supporter's name. */
-- function scrubOutbox(i) {
-- const rows = db.prepare("SELECT id, envelope FROM event_outbox WHERE json_extract(envelope, '$.subject.type') = 'interaction' AND json_extract(envelope, '$.subject.id') = ?").all(i.id);
-- for (const r of rows) {
-- const env = JSON.parse(r.envelope);
-- if (!env.payload || !('supporter' in env.payload)) continue;
-- env.payload.supporter = null;
-- env.payload.supporter_name = publicName(i);
-- db.prepare('UPDATE event_outbox SET envelope = ? WHERE id = ?').run(JSON.stringify(env), r.id);
-- }
-- }
--
/**
* A creator's totals, derived from settled interactions. `billing` is what must equal Billing's
* books (settled through Billing or imported from Live's ledger, minus what Billing took back);
* external and test interactions are reported apart and never mixed in.
*/
-- function totals(creator) {
-- const r = db.prepare(`SELECT
-- COALESCE(SUM(CASE WHEN test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') THEN amount - reversed_bits END), 0) AS billing,
-- COALESCE(SUM(CASE WHEN test = 0 AND settlement = 'external' AND payment_state = 'settled' THEN amount END), 0) AS external,
-- COALESCE(SUM(CASE WHEN test = 0 AND payment_state IN ('settled', 'reversed') THEN 1 END), 0) AS count,
-- COALESCE(SUM(CASE WHEN test = 1 THEN 1 END), 0) AS test_count,
-- COALESCE(SUM(CASE WHEN payment_state = 'pending' THEN 1 END), 0) AS pending_count
-- FROM tip_interactions WHERE creator_subject = ?`).get(creator);
++ async function totals(q, creator) {
++ const r = await q.one(sql`SELECT
++ COALESCE(SUM(amount - reversed_bits) FILTER (WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed')), 0)::bigint AS billing,
++ COALESCE(SUM(amount) FILTER (WHERE NOT test AND settlement = 'external' AND payment_state = 'settled'), 0)::bigint AS external,
++ count(*) FILTER (WHERE NOT test AND payment_state IN ('settled', 'reversed')) AS count,
++ count(*) FILTER (WHERE test) AS test_count,
++ count(*) FILTER (WHERE payment_state = 'pending') AS pending_count
++ FROM tip_interactions WHERE creator_subject = ${creator}`);
return { currency: 'vibes-bits', settled_via_billing: r.billing, external: r.external, interactions: r.count, simulations: r.test_count, pending: r.pending_count };
}
return {
-- get, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,
-- onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, list, totals, insert, KINDS,
++ get, lock, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,
++ onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, presentMany, list, totals, insert, KINDS,
leaderboard, recentPublic, exportFor, erase, release,
};
}
ddiff --git a/server/domain/moderation.js b/server/domain/moderation.js
iindex 073d288..96e50d3 100644
---- a/server/domain/moderation.js
++++ b/server/domain/moderation.js
@@@ -18,9 +18,15 @@
* tips.interaction.moderate. Every outcome is recorded in tip_moderation_log and published as
* tips.interaction.moderated (never for simulations): { interaction_id, creator, action, by,
* moderation_state, cancelled_effects }. The payload names no supporter, message or moderator.
++ *
++ * hide() and restore() cancel or release what a payment bought (paid messages, TTS, media requests), so
++ * they run SERIALIZABLE and lock the interaction row first (the domain's lock order): two moderators
++ * acting at once apply one after the other.
*/
const crypto = require('crypto');
--const { fail, iso, prefixedId, sha256, userSubject, displayName, text } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { fail, iso, prefixedId, sha256, userSubject, displayName, text, readCursor, cursorOf } = require('../util');
++const { publicView } = require('./privacy');
const filter = require('./filter');
const INVITE_RE = /^tmin_[A-Za-z0-9_-]{43}$/;
@@@ -32,139 +38,150 @@ function createModeration(ctx) {
const { db } = ctx;
// ── Moderators ───────────────────────────────────────────
-- const isModerator = (creator, subject) => !!(subject && db.prepare('SELECT 1 FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').get(creator, subject));
-- const listModerators = (creator) => db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL ORDER BY created_at').all(creator);
++ const isModerator = async (q, creator, subject) => !!(subject && await q.maybe(sql`SELECT 1 FROM tip_moderators
++ WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`));
++ const listModerators = (q, creator) => q.many(sql`SELECT * FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL ORDER BY created_at`);
/** Creators whose interactions this person moderates (with their handles). */
-- const moderatedBy = (subject) => db.prepare(`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m
-- JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ? AND m.removed_at IS NULL ORDER BY p.handle`).all(subject);
++ const moderatedBy = (q, subject) => q.many(sql`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m
++ JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ${subject} AND m.removed_at IS NULL ORDER BY p.handle`);
-- function addModerator(creator, moderator, { name, addedBy }) {
++ /** In q's transaction, or a new one: the creator's row is locked while the limit is counted. */
++ function addModerator(q, creator, moderator, { name, addedBy }) {
const subject = userSubject(moderator, 'moderator');
if (subject === creator) fail(422, 'tips.invalid_input', 'the creator moderates their own page already');
-- const count = db.prepare('SELECT COUNT(*) AS n FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL').get(creator).n;
-- if (!isModerator(creator, subject) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);
-- db.prepare(`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at) VALUES (?, ?, ?, ?, ?)
-- ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, name), added_by = excluded.added_by`)
-- .run(creator, subject, displayName(name), addedBy, iso(ctx.now()));
-- return presentModerator(db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ?').get(creator, subject));
++ return ctx.within(q, async (t) => {
++ await ctx.lockCreator(t, creator);
++ const count = await t.value(sql`SELECT count(*) FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL`);
++ if (!(await isModerator(t, creator, subject)) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);
++ return presentModerator(await t.one(sql`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at)
++ VALUES (${creator}, ${subject}, ${displayName(name)}, ${addedBy}, ${iso(ctx.now())})
++ ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, tip_moderators.name), added_by = excluded.added_by
++ RETURNING *`));
++ });
}
-- function removeModerator(creator, moderator) {
++ async function removeModerator(creator, moderator) {
const subject = userSubject(moderator, 'moderator');
-- const r = db.prepare('UPDATE tip_moderators SET removed_at = ? WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').run(iso(ctx.now()), creator, subject);
-- return { removed: r.changes > 0 };
++ const n = await db.exec(sql`UPDATE tip_moderators SET removed_at = ${iso(ctx.now())} WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`);
++ return { removed: n > 0 };
}
const presentModerator = (m) => (m ? { moderator: { type: 'user', id: m.moderator_subject }, name: m.name || null, added_at: m.created_at } : null);
// ── Invitations (a show-once link the creator hands to a moderator) ──
-- function createInvite(creator, { createdBy }) {
-- const open = db.prepare('SELECT COUNT(*) AS n FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ?').get(creator, iso(ctx.now())).n;
-- if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');
++ async function createInvite(creator, { createdBy }) {
const secret = `tmin_${crypto.randomBytes(32).toString('base64url')}`;
const id = prefixedId('tmin', ctx.now());
const at = ctx.now();
-- db.prepare('INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)')
-- .run(id, creator, sha256(secret), createdBy, iso(at), iso(at + INVITE_TTL_MS));
++ await ctx.tx(async (t) => {
++ await ctx.lockCreator(t, creator);
++ const open = await t.value(sql`SELECT count(*) FROM tip_moderator_invites WHERE creator_subject = ${creator} AND used_at IS NULL AND revoked_at IS NULL
++ AND expires_at > ${iso(at)}`);
++ if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');
++ await t.exec(sql`INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at)
++ VALUES (${id}, ${creator}, ${sha256(secret)}, ${createdBy}, ${iso(at)}, ${iso(at + INVITE_TTL_MS)})`);
++ });
return { id, url: `${ctx.config.baseUrl}/moderate/invite/${secret}`, expires_at: iso(at + INVITE_TTL_MS) };
}
/** The open invitation for a presented secret, or null (unknown, used, revoked or expired). */
-- function findInvite(secret) {
++ async function findInvite(q, secret) {
if (!INVITE_RE.test(String(secret || ''))) return null;
-- const row = db.prepare('SELECT * FROM tip_moderator_invites WHERE token_hash = ?').get(sha256(secret));
-- if (!row || row.used_at || row.revoked_at || row.expires_at <= iso(ctx.now())) return null;
++ const row = await q.maybe(sql`SELECT * FROM tip_moderator_invites WHERE token_hash = ${sha256(secret)}`);
++ if (!row || row.used_at || row.revoked_at || Date.parse(row.expires_at) <= ctx.now()) return null;
return row;
}
++ /** Accept once: the invitation is taken by a conditional update, so two people at once cannot both use it. */
function acceptInvite(secret, { subject, name }) {
-- return ctx.tx(() => {
-- const inv = findInvite(secret);
++ return ctx.tx(async (t) => {
++ const inv = await findInvite(t, secret);
if (!inv) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');
if (inv.creator_subject === subject) fail(422, 'tips.invalid_input', 'this is your own page');
-- db.prepare('UPDATE tip_moderator_invites SET used_at = ?, used_by = ? WHERE id = ? AND used_at IS NULL').run(iso(ctx.now()), subject, inv.id);
-- return { creator: inv.creator_subject, moderator: addModerator(inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };
++ const n = await t.exec(sql`UPDATE tip_moderator_invites SET used_at = ${iso(ctx.now())}, used_by = ${subject} WHERE id = ${inv.id} AND used_at IS NULL AND revoked_at IS NULL`);
++ if (!n) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');
++ return { creator: inv.creator_subject, moderator: await addModerator(t, inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };
});
}
-- const openInvites = (creator) => db.prepare('SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ? ORDER BY created_at').all(creator, iso(ctx.now()));
-- function revokeInvite(creator, id) {
-- return { revoked: db.prepare('UPDATE tip_moderator_invites SET revoked_at = ? WHERE id = ? AND creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL').run(iso(ctx.now()), String(id || ''), creator).changes > 0 };
++ const openInvites = (q, creator) => q.many(sql`SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ${creator}
++ AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ${iso(ctx.now())} ORDER BY created_at`);
++ async function revokeInvite(creator, id) {
++ const n = await db.exec(sql`UPDATE tip_moderator_invites SET revoked_at = ${iso(ctx.now())} WHERE id = ${String(id || '')} AND creator_subject = ${creator}
++ AND used_at IS NULL AND revoked_at IS NULL`);
++ return { revoked: n > 0 };
}
// ── The filter at settlement ─────────────────────────────
/**
-- * Inside the settlement transaction, before anything is shown: does the creator's filter match the
-- * name or what will be shown or read? Returns { hit, hold }.
++ * Before anything is shown or read: does the creator's filter (`settings`, profiles.filterOf) match
++ * the name or what will be shown or read? Returns { hit, hold }.
*/
-- function screen(i) {
-- const settings = ctx.profiles.filterOf(i.creator_subject);
++ function screen(i, settings) {
if (!settings || !settings.words.length) return { hit: false, hold: false };
-- const req = JSON.parse(i.request || '{}');
++ const req = i.request || {};
const texts = [i.anonymous ? null : i.supporter_name, i.private_message ? null : i.message, req.tts && req.tts.text];
const hit = filter.hits(settings, ...texts);
return { hit, hold: hit && settings.action === 'hold' };
}
// ── Outcomes ─────────────────────────────────────────────
-- function record(i, action, { role, actor = null, reason = null, cancelled = [] }) {
-- db.prepare('INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')
-- .run(i.creator_subject, i.id, action, role, actor, reason, iso(ctx.now()));
++ async function record(t, i, action, { role, actor = null, reason = null, cancelled = [] }) {
++ await t.exec(sql`INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at)
++ VALUES (${i.creator_subject}, ${i.id}, ${action}, ${role}, ${actor}, ${reason}, ${iso(ctx.now())})`);
if (!i.test) {
-- const now = ctx.interactions.get(i.id);
-- ctx.outbox.emit('tips.interaction.moderated', { type: 'interaction', id: i.id }, {
++ await ctx.outbox.emit(t, 'tips.interaction.moderated', { type: 'interaction', id: i.id }, {
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, action, by: role,
-- moderation_state: now.moderation, cancelled_effects: cancelled,
++ moderation_state: i.moderation, cancelled_effects: cancelled,
});
-- ctx.afterCommit(() => ctx.outboxKick());
++ ctx.afterCommit(t, () => ctx.outboxKick());
}
}
/**
* Settlement found a filter match (inside its transaction). markScreened() sets the state before
-- * anything is shown (held, or only filtered = masked); recordScreened() logs and publishes it once
-- * the settlement's own event is out.
++ * anything is shown (held, or only filtered = masked) and returns the row; recordScreened() logs and
++ * publishes it once the settlement's own event is out.
*/
-- function markScreened(i, verdict) {
++ function markScreened(t, i, verdict) {
const at = iso(ctx.now());
-- if (verdict.hold) db.prepare("UPDATE tip_interactions SET filtered = 1, moderation = 'held', moderated_at = ?, moderated_by = 'filter', updated_at = ? WHERE id = ?").run(at, at, i.id);
-- else db.prepare('UPDATE tip_interactions SET filtered = 1, updated_at = ? WHERE id = ?').run(at, i.id);
++ return verdict.hold
++ ? t.one(sql`UPDATE tip_interactions SET filtered = true, moderation = 'held', moderated_at = ${at}, moderated_by = 'filter', updated_at = ${at} WHERE id = ${i.id} RETURNING *`)
++ : t.one(sql`UPDATE tip_interactions SET filtered = true, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
}
-- function recordScreened(i, verdict) {
-- record(i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });
++ function recordScreened(t, i, verdict) {
++ return record(t, i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });
}
-- /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else 403. */
-- function actorFor(principal, creator, granted) {
++ /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else null. */
++ async function actorFor(q, principal, creator, granted) {
if (principal.kind === 'service' && granted) return { role: 'service', actor: principal.sub };
if (principal.kind === 'user' && principal.subject === creator) return { role: 'creator', actor: principal.subject };
-- if (principal.kind === 'user' && isModerator(creator, principal.subject)) return { role: 'moderator', actor: principal.subject };
++ if (principal.kind === 'user' && await isModerator(q, creator, principal.subject)) return { role: 'moderator', actor: principal.subject };
return null;
}
/** Hide from overlays, chat/TTS still to come and public pages. The money is untouched. */
function hide(i, { role, actor, reason }) {
const note = text(reason, 'reason', 200);
-- return ctx.tx(() => {
-- const cur = ctx.interactions.get(i.id);
++ return ctx.tx(async (t) => {
++ const cur = await ctx.interactions.lock(t, i.id);
if (cur.moderation === 'hidden') return { interaction: cur, changed: false };
const at = iso(ctx.now());
-- const cancelled = db.prepare(`SELECT effect FROM interaction_effects WHERE interaction_id = ? AND state = 'queued' AND effect IN (${CHAT_EFFECTS.map(() => '?').join(',')})`)
-- .all(cur.id, ...CHAT_EFFECTS).map((e) => e.effect);
++ const cancelled = (await t.many(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${at}
++ WHERE interaction_id = ${cur.id} AND state = 'queued' RETURNING effect`)).map((e) => e.effect).filter((e) => CHAT_EFFECTS.includes(e));
if (cancelled.length) {
-- db.prepare(`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE interaction_id = ? AND state = 'queued'`).run(at, cur.id);
-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);
-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);
++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);
++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);
}
const undelivered = cancelled.length > 0 || cur.moderation === 'held' || cur.payment_state === 'pending';
-- db.prepare(`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ?, moderated_by = ?,
-- delivery_state = CASE WHEN ? AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END, updated_at = ? WHERE id = ?`)
-- .run(at, actor || role, undelivered ? 1 : 0, at, cur.id);
-- const now = ctx.interactions.get(cur.id);
-- ctx.overlays.retract(now);
-- record(now, 'hidden', { role, actor, reason: note, cancelled });
-- return { interaction: ctx.interactions.get(cur.id), changed: true, cancelled_effects: cancelled };
-- });
++ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ${at}, moderated_by = ${actor || role},
++ delivery_state = CASE WHEN ${undelivered} AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END,
++ updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);
++ await ctx.overlays.retract(t, now, await ctx.profiles.filterOf(t, now.creator_subject));
++ await record(t, now, 'hidden', { role, actor, reason: note, cancelled });
++ return { interaction: now, changed: true, cancelled_effects: cancelled };
++ }, ctx.MONEY);
}
/**
@@@ -174,26 +191,26 @@ function createModeration(ctx) {
*/
function restore(i, { role, actor, reason }) {
const note = text(reason, 'reason', 200);
-- return ctx.tx(() => {
-- const cur = ctx.interactions.get(i.id);
++ return ctx.tx(async (t) => {
++ const cur = await ctx.interactions.lock(t, i.id);
if (cur.moderation === 'visible') return { interaction: cur, changed: false };
const at = iso(ctx.now());
// Never shown (held, or hidden before it was): released now, its delivery starting over.
-- const neverShown = !db.prepare("SELECT 1 FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").get(cur.id);
++ const neverShown = !(await t.maybe(sql`SELECT 1 FROM overlay_deliveries WHERE interaction_id = ${cur.id} AND kind = 'alert'`));
const release = cur.payment_state === 'settled' && cur.origin !== 'import' && neverShown;
-- db.prepare(`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ?, moderated_by = ?,
-- delivery_state = CASE WHEN ? THEN 'queued' ELSE delivery_state END, updated_at = ? WHERE id = ?`).run(at, actor || role, release ? 1 : 0, at, cur.id);
-- const now = ctx.interactions.get(cur.id);
-- ctx.overlays.unretract(now);
-- if (release) ctx.interactions.release(now);
-- record(now, 'restored', { role, actor, reason: note });
-- return { interaction: ctx.interactions.get(cur.id), changed: true };
-- });
++ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ${at}, moderated_by = ${actor || role},
++ delivery_state = CASE WHEN ${release} THEN 'queued' ELSE delivery_state END, updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);
++ const settings = await ctx.profiles.filterOf(t, now.creator_subject);
++ await ctx.overlays.unretract(t, now, settings);
++ if (release) await ctx.interactions.release(t, now, settings);
++ await record(t, now, 'restored', { role, actor, reason: note });
++ return { interaction: await ctx.interactions.get(t, cur.id), changed: true };
++ }, ctx.MONEY);
}
/** The moderators' view: what was written (not private things), and what the public sees. */
-- function present(i) {
-- const req = JSON.parse(i.request || '{}');
++ function shape(i, settings) {
++ const req = i.request || {};
return {
id: i.id, kind: i.kind, created_at: i.created_at, settled_at: i.settled_at || null, payment_state: i.payment_state, test: !!i.test,
moderation: { state: i.moderation, at: i.moderated_at || null, filtered: !!i.filtered },
@@@ -202,29 +219,34 @@ function createModeration(ctx) {
message: i.private_message || i.erased_at ? null : (i.message || null),
tts_text: req.tts && !i.erased_at ? req.tts.text : null,
media_url: req.media && !i.erased_at ? req.media.url : null,
-- public: ctx.view(i),
++ public: publicView(i, { filter: settings }),
};
}
++ const present = async (q, i) => shape(i, await ctx.profiles.filterOf(q, i.creator_subject));
-- /** A creator's interactions for review, newest first. state: held | hidden | visible | all. */
-- function queue(creator, { state = 'all', cursor, limit = 50 } = {}) {
++ /** A creator's interactions for review, newest first, keyset-paged. state: held | hidden | visible | all. */
++ async function queue(q, creator, { state = 'all', cursor, limit = 50 } = {}) {
if (!['held', 'hidden', 'visible', 'all'].includes(state)) fail(422, 'tips.invalid_input', 'state must be held, hidden, visible or all');
const n = Math.min(200, Math.max(1, Number(limit) || 50));
-- let cur = null;
-- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }
-- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = @c AND payment_state IN ('settled', 'reversed', 'pending')
-- AND (@s = 'all' OR moderation = @s) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))
-- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator, s: state, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });
++ const cur = cursor ? readCursor(cursor) : null;
++ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND payment_state IN ('settled', 'reversed', 'pending')
++ ${state === 'all' ? sql`` : sql`AND moderation = ${state}`} ${cur ? sql`AND (created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)` : sql``}
++ ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);
const page = rows.slice(0, n);
-- return { rows: page.map(present), next_cursor: rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null };
++ const settings = await ctx.profiles.filterOf(q, creator);
++ return { rows: page.map((i) => shape(i, settings)), next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };
}
-- const log = (creator, limit = 100) => db.prepare('SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log WHERE creator_subject = ? ORDER BY id DESC LIMIT ?').all(creator, limit);
++ const log = (q, creator, limit = 100) => q.many(sql`SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log
++ WHERE creator_subject = ${creator} ORDER BY id DESC LIMIT ${limit}`);
++
++ /** Held paid messages waiting for review (the dashboard's count). */
++ const heldCount = (q, creator) => q.value(sql`SELECT count(*) FROM tip_interactions WHERE creator_subject = ${creator} AND moderation = 'held' AND payment_state = 'settled'`);
return {
isModerator, listModerators, moderatedBy, addModerator, removeModerator, presentModerator,
createInvite, findInvite, acceptInvite, openInvites, revokeInvite,
-- screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log,
++ screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log, heldCount,
};
}
ddiff --git a/server/domain/overlay-hub.js b/server/domain/overlay-hub.js
nnew file mode 100644
iindex 0000000..79abfb8
---- /dev/null
++++ b/server/domain/overlay-hub.js
@@@ -0,0 +1,223 @@
++'use strict';
++
++/**
++ * The overlays' live side: open SSE streams, fan-out between processes, and the per-token stream cap.
++ *
++ * Streams are sockets of one process; what they must hear can happen in any process (a tip settled by
++ * another process, a moderator's hide, a config change, a revocation). Each process subscribes, per
++ * creator with an open stream here, to `overlay:<creator>` on openvibe-sdk/pubsub (Valkey, or in-process
++ * without it) and acts on what arrives:
++ *
++ * { t: 'new' } push the creator's new deliveries (read from the database)
++ * { t: 'retract', interaction_id, rows } tell alert streams to drop an alert a moderator hid
++ * { t: 'config', config } a changed overlay config, to the streams using it
++ * { t: 'revoke', token_id } close the token's streams at once
++ *
++ * Nothing authoritative travels here: a lost message costs a push that the next one (or a reconnect's
++ * replay from the database) makes up.
++ *
++ * Stream slots (TIPS_OVERLAY_MAX_STREAMS per token) are counted across processes in a Valkey sorted set
++ * per creator (member <token>|<stream>, score = expiry); a stream refreshes its slot on every heartbeat,
++ * so a dead process's slots lapse after three missed heartbeats. `connected(creator)` counts the live
++ * slots. Without Valkey, or when it fails, the slots are counted in this process.
++ */
++const crypto = require('crypto');
++const { createPubSub } = require('openvibe-sdk/pubsub');
++
++// Drop lapsed slots, count the token's, and take one when under the cap, atomically.
++// KEYS[1] the creator's set; ARGV: now, '<token>|', max, expiry, member, ttl (ms).
++const ACQUIRE = `
++redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])
++local n = 0
++for _, m in ipairs(redis.call('ZRANGE', KEYS[1], 0, -1)) do
++ if string.sub(m, 1, #ARGV[2]) == ARGV[2] then n = n + 1 end
++end
++if n >= tonumber(ARGV[3]) then return 0 end
++redis.call('ZADD', KEYS[1], ARGV[4], ARGV[5])
++redis.call('PEXPIRE', KEYS[1], ARGV[6])
++return 1`;
++
++function memorySlots(now) {
++ const byCreator = new Map(); // creator → Map(member → expiry)
++ const live = (creator) => {
++ const m = byCreator.get(creator);
++ if (!m) return new Map();
++ for (const [k, exp] of m) if (exp <= now()) m.delete(k);
++ if (!m.size) byCreator.delete(creator);
++ return m;
++ };
++ return {
++ async acquire(creator, tokenId, member, max, ttlMs) {
++ const m = live(creator);
++ if ([...m.keys()].filter((k) => k.startsWith(`${tokenId}|`)).length >= max) return false;
++ if (!byCreator.has(creator)) byCreator.set(creator, m);
++ m.set(member, now() + ttlMs);
++ return true;
++ },
++ async refresh(creator, member, ttlMs) { const m = byCreator.get(creator); if (m && m.has(member)) m.set(member, now() + ttlMs); },
++ async release(creator, member) { const m = byCreator.get(creator); if (m) { m.delete(member); if (!m.size) byCreator.delete(creator); } },
++ async count(creator) { return live(creator).size; },
++ };
++}
++
++function valkeySlots(valkey, now) {
++ const c = valkey.client;
++ if (typeof c.tipsOverlayAcquire !== 'function') c.defineCommand('tipsOverlayAcquire', { numberOfKeys: 1, lua: ACQUIRE });
++ const key = (creator) => valkey.key('overlay', 'streams', creator);
++ return {
++ async acquire(creator, tokenId, member, max, ttlMs) {
++ const t = now();
++ return (await c.tipsOverlayAcquire(key(creator), String(t), `${tokenId}|`, String(max), String(t + ttlMs), member, String(ttlMs))) === 1;
++ },
++ async refresh(creator, member, ttlMs) { await c.multi().zadd(key(creator), 'XX', now() + ttlMs, member).pexpire(key(creator), ttlMs).exec(); },
++ async release(creator, member) { await c.zrem(key(creator), member); },
++ async count(creator) { return c.zcount(key(creator), now(), '+inf'); },
++ };
++}
++
++/**
++ * @param {object} o
++ * @param {object|null} o.valkey openvibe-sdk/valkey handle, or null (in-process)
++ * @param {(client) => Promise<void>} o.pushNew read and write a client's new deliveries
++ * @param {number} o.heartbeatMs
++ */
++function createOverlayHub({ valkey = null, pushNew, heartbeatMs, now = () => Date.now(), log = console }) {
++ const ps = createPubSub({ valkey, log });
++ const local = memorySlots(now);
++ const shared = valkey ? valkeySlots(valkey, now) : null;
++ const ttlMs = heartbeatMs * 3;
++ let warned = false;
++ /** The shared slot store; on a Valkey error this call counts in the process instead. */
++ const slots = async (op, ...args) => {
++ if (shared) {
++ try { return await shared[op](...args); } catch (e) {
++ if (!warned) { warned = true; log.warn(`[Tips] overlay stream slots: Valkey failed (${e.message}); counting in this process`); }
++ }
++ }
++ return local[op](...args);
++ };
++ const creators = new Map(); // creator → { clients: Set, off: Promise<() => Promise> }
++ let nextClient = 1;
++
++ function write(client, event, id, data) {
++ if (client.closed) return false;
++ try {
++ client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
++ return true;
++ } catch { return false; }
++ }
++
++ /** Run fn for a client after whatever it is already doing (its pushes never interleave). */
++ function queue(client, fn) {
++ client.chain = client.chain.then(() => (client.closed ? null : fn())).catch((e) => log.warn('[Tips] overlay push:', e.message));
++ return client.chain;
++ }
++ /** New deliveries for this client: coalesced, so a burst of notifications reads once. */
++ function schedulePush(client) {
++ if (client.pushQueued) return;
++ client.pushQueued = true;
++ queue(client, () => { client.pushQueued = false; return pushNew(client); });
++ }
++
++ function onMessage(creator, msg) {
++ const entry = creators.get(creator);
++ if (!entry || !msg) return;
++ for (const c of [...entry.clients]) {
++ if (msg.t === 'new') schedulePush(c);
++ else if (msg.t === 'retract' && c.scopes.includes('alerts')) {
++ queue(c, () => { for (const r of msg.rows || []) write(c, 'retract', null, { interaction_id: msg.interaction_id, delivery_id: r.id, seq: r.seq }); });
++ } else if (msg.t === 'config' && msg.config && c.configId === msg.config.id) {
++ queue(c, () => { write(c, 'config', null, { config: msg.config }); });
++ } else if (msg.t === 'revoke' && c.tokenId === msg.token_id) {
++ write(c, 'revoked', null, { reason: 'this overlay token was revoked' });
++ detach(c);
++ }
++ }
++ }
++
++ // Through Valkey every process hears it, this one included. If publishing fails, this process's own
++ // streams still hear it (the others catch up from the database on their next push or reconnect).
++ const publish = (creator, msg) => ps.publish(`overlay:${creator}`, msg).then(() => undefined, (e) => {
++ log.warn('[Tips] overlay fan-out:', e.message);
++ onMessage(creator, JSON.parse(JSON.stringify(msg)));
++ });
++
++ /** Subscribe for a creator's messages; a failure is retried while the creator has streams here. */
++ function subscribe(creator, entry) {
++ entry.off = ps.subscribe(`overlay:${creator}`, (msg) => onMessage(creator, msg)).catch((e) => {
++ log.warn('[Tips] overlay subscribe:', e.message);
++ const retry = setTimeout(() => { if (creators.get(creator) === entry) subscribe(creator, entry); }, 5000);
++ if (retry.unref) retry.unref();
++ return async () => {};
++ });
++ }
++
++ /** A stream place for this token, or null past the cap (the route answers 429). */
++ async function reserve(token, max) {
++ const slot = { creator: token.creator_subject, member: `${token.id}|${crypto.randomBytes(8).toString('hex')}` };
++ return (await slots('acquire', slot.creator, token.id, slot.member, max, ttlMs)) ? slot : null;
++ }
++
++ /**
++ * Register a stream (its slot reserved) and run `start(client)` (hello, replay) before any push.
++ * The subscription is in place before start reads, so nothing committed meanwhile is missed.
++ */
++ function attach(token, res, slot, { minAmount = 0, configId = null }, start) {
++ const client = {
++ id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId,
++ minAmount, res, lastSeq: 0, closed: false, slot, chain: Promise.resolve(), pushQueued: false,
++ };
++ let entry = creators.get(client.creator);
++ if (!entry) {
++ entry = { clients: new Set(), off: null };
++ creators.set(client.creator, entry);
++ subscribe(client.creator, entry);
++ }
++ entry.clients.add(client);
++ res.on('close', () => detach(client));
++ if (res.destroyed) { detach(client); return client; } // gone before it was attached
++ client.beat = setInterval(() => {
++ if (!write(client, 'ping', null, { t: Date.now() })) { detach(client); return; }
++ slots('refresh', slot.creator, slot.member, ttlMs).catch(() => {});
++ }, heartbeatMs);
++ if (client.beat.unref) client.beat.unref();
++ queue(client, async () => { await entry.off; await start(client); });
++ schedulePush(client);
++ return client;
++ }
++
++ function detach(client) {
++ if (client.closed) return;
++ client.closed = true;
++ clearInterval(client.beat);
++ slots('release', client.slot.creator, client.slot.member).catch(() => {});
++ const entry = creators.get(client.creator);
++ if (entry) {
++ entry.clients.delete(client);
++ if (!entry.clients.size) {
++ creators.delete(client.creator);
++ entry.off.then((off) => off()).catch(() => {});
++ }
++ }
++ try { client.res.end(); } catch { /* already gone */ }
++ }
++
++ return {
++ write,
++ attach,
++ reserve,
++ /** Give a reserved slot back (the stream never started). */
++ release: (slot) => slots('release', slot.creator, slot.member).catch(() => {}),
++ /** Tell every process's streams of this creator: new deliveries are in the database. */
++ notify: (creator) => publish(creator, { t: 'new' }),
++ retract: (creator, interactionId, rows) => publish(creator, { t: 'retract', interaction_id: interactionId, rows }),
++ config: (creator, config) => publish(creator, { t: 'config', config }),
++ revoke: (creator, tokenId) => publish(creator, { t: 'revoke', token_id: tokenId }),
++ /** Open streams of this creator's overlays, in every process. */
++ connected: (creator) => slots('count', creator),
++ closeAll() { for (const entry of [...creators.values()]) for (const c of [...entry.clients]) detach(c); },
++ async close() { this.closeAll(); await ps.close(); },
++ };
++}
++
++module.exports = { createOverlayHub };
ddiff --git a/server/domain/overlays.js b/server/domain/overlays.js
iindex 295b494..4a7b892 100644
---- a/server/domain/overlays.js
++++ b/server/domain/overlays.js
@@@ -1,16 +1,17 @@
'use strict';
/**
-- * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE hub.
++ * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE streams (overlay-hub.js).
*
* Tokens. `tovl_<43 base64url chars>` shown ONCE at creation; only its SHA-256 is stored. A token
* carries scopes (alerts, goals), optionally a config, and is never a creator cookie: an OBS
* browser source holds it in its URL and nothing else. Revocation is immediate: the row is marked
-- * and every open stream of that token is closed in the same call.
++ * and every open stream of that token, in any process, is told and closed.
*
* Deliveries. Every alert (a settled interaction) and goal change becomes one overlay_deliveries
* row with a monotonic seq (the SSE event id). The first time a row is written to any overlay it
-- * becomes `delivered` and emits tips.overlay.delivered; a row no overlay received within the
++ * becomes `delivered` and emits tips.overlay.delivered (recorded just before the bytes are written);
++ * a row no overlay received within the
* delivery window becomes `failed` (tips.overlay.failed). A reconnect with Last-Event-ID
* REPLAYS rows after that id: replay only writes bytes to a socket — it never charges, never
* re-counts a goal, never emits another event. Simulated (test) rows are flagged and emit nothing.
@@@ -22,62 +23,70 @@
* sent, replayed or listed again — and tells connected overlays to drop it (`retract` event).
*/
const crypto = require('crypto');
--const { fail, iso, prefixedId, sha256, json, text } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { fail, iso, prefixedId, sha256, text } = require('../util');
const { safeUrl } = require('./profiles');
--const { isHidden } = require('./privacy');
++const { isHidden, publicView } = require('./privacy');
++const { createOverlayHub } = require('./overlay-hub');
const SCOPES = ['alerts', 'goals'];
++const STREAM_LOCK = 4610; // the advisory-lock namespace of a creator's overlay stream (Tips' port)
const TOKEN_RE = /^tovl_[A-Za-z0-9_-]{43}$/;
++// A delivery row with the interaction's real amount (an overlay's minimum applies to a hidden amount too).
++const DELIVERY_COLUMNS = sql`d.*, i.amount AS interaction_amount`;
++const DELIVERY_FROM = sql`overlay_deliveries d LEFT JOIN tip_interactions i ON i.id = d.interaction_id`;
function createOverlays(ctx) {
const { db } = ctx;
-- const clients = new Map(); // creator subject → Set<client>
-- let nextClient = 1;
++ const hub = createOverlayHub({ valkey: ctx.valkey, pushNew, heartbeatMs: ctx.config.overlays.heartbeatMs, log: ctx.log });
// ── Tokens ───────────────────────────────────────────────
-- function createToken(creator, { scopes, label, configId, createdBy }) {
++ async function createToken(creator, { scopes, label, configId, createdBy }) {
const list = Array.isArray(scopes) && scopes.length ? [...new Set(scopes.map(String))] : SCOPES;
if (list.some((s) => !SCOPES.includes(s))) fail(422, 'tips.invalid_input', `scopes must be among ${SCOPES.join(', ')}`);
if (configId) {
-- const c = getConfig(configId);
++ const c = await getConfig(db, configId);
if (!c || c.creator_subject !== creator) fail(404, 'tips.overlay_config_not_found', 'no such overlay config for this creator');
}
-- const active = db.prepare('SELECT COUNT(*) AS n FROM overlay_tokens WHERE creator_subject = ? AND revoked_at IS NULL').get(creator).n;
-- if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');
const secret = `tovl_${crypto.randomBytes(32).toString('base64url')}`;
-- const id = prefixedId('tovt', ctx.now());
-- db.prepare(`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)
-- VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, sha256(secret), JSON.stringify(list), configId || null, text(label, 'label', 80), createdBy, iso(ctx.now()));
-- return { token: presentToken(getToken(id)), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };
++ const row = await ctx.tx(async (t) => {
++ await ctx.lockCreator(t, creator);
++ const active = await t.value(sql`SELECT count(*) FROM overlay_tokens WHERE creator_subject = ${creator} AND revoked_at IS NULL`);
++ if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');
++ return t.one(sql`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)
++ VALUES (${prefixedId('tovt', ctx.now())}, ${creator}, ${sha256(secret)}, ${sql.json(list)}, ${configId || null}, ${text(label, 'label', 80)}, ${createdBy}, ${iso(ctx.now())})
++ RETURNING *`);
++ });
++ return { token: presentToken(row), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };
}
-- const getToken = (id) => db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(String(id || '')) || null;
-- const listTokens = (creator) => db.prepare('SELECT * FROM overlay_tokens WHERE creator_subject = ? ORDER BY revoked_at IS NOT NULL, created_at DESC').all(creator);
++ const getToken = (q, id) => q.maybe(sql`SELECT * FROM overlay_tokens WHERE id = ${String(id || '')}`);
++ const listTokens = (q, creator) => q.many(sql`SELECT * FROM overlay_tokens WHERE creator_subject = ${creator} ORDER BY revoked_at IS NOT NULL, created_at DESC`);
/** The active token row for a presented secret, or null (unknown, malformed or revoked). */
-- function authenticate(secret) {
++ async function authenticate(q, secret) {
if (!TOKEN_RE.test(String(secret || ''))) return null;
-- const row = db.prepare('SELECT * FROM overlay_tokens WHERE token_hash = ?').get(sha256(secret));
++ const row = await q.maybe(sql`SELECT * FROM overlay_tokens WHERE token_hash = ${sha256(secret)}`);
if (!row || row.revoked_at) return null;
-- return { ...row, scopes: json(row.scopes, []) };
++ return row;
}
-- function revokeToken(row) {
-- if (!row.revoked_at) db.prepare('UPDATE overlay_tokens SET revoked_at = ? WHERE id = ? AND revoked_at IS NULL').run(iso(ctx.now()), row.id);
-- closeToken(row.id);
-- return presentToken(getToken(row.id));
++ async function revokeToken(row) {
++ const out = await db.maybe(sql`UPDATE overlay_tokens SET revoked_at = ${iso(ctx.now())} WHERE id = ${row.id} AND revoked_at IS NULL RETURNING *`);
++ await hub.revoke(row.creator_subject, row.id);
++ return presentToken(out || await getToken(db, row.id));
}
function presentToken(t) {
if (!t) return null;
return {
-- id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: json(t.scopes, []), label: t.label || null, config_id: t.config_id || null,
++ id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes, label: t.label || null, config_id: t.config_id || null,
created_at: t.created_at, last_used_at: t.last_used_at || null, revoked_at: t.revoked_at || null, active: !t.revoked_at,
};
}
// ── Configs ──────────────────────────────────────────────
-- const getConfig = (id) => db.prepare('SELECT * FROM overlay_configs WHERE id = ?').get(String(id || '')) || null;
-- const listConfigs = (creator) => db.prepare('SELECT * FROM overlay_configs WHERE creator_subject = ? ORDER BY created_at').all(creator);
++ const getConfig = (q, id) => q.maybe(sql`SELECT * FROM overlay_configs WHERE id = ${String(id || '')}`);
++ const listConfigs = (q, creator) => q.many(sql`SELECT * FROM overlay_configs WHERE creator_subject = ${creator} ORDER BY created_at`);
function settingsOf(kind, input = {}, base = {}) {
const s = { ...base };
@@@ -110,236 +119,248 @@ function createOverlays(ctx) {
goal: { show_amounts: true, show_percent: true },
};
-- function createConfig(creator, input = {}) {
++ async function createConfig(creator, input = {}) {
const kind = input.kind === 'goal' ? 'goal' : input.kind === 'alerts' ? 'alerts' : fail(422, 'tips.invalid_input', "kind must be 'alerts' or 'goal'");
const name = text(input.name, 'name', 80) || (kind === 'alerts' ? 'Alerts' : 'Goal');
let goalId = null;
if (kind === 'goal' && input.goal_id) {
-- const g = ctx.goals.get(input.goal_id);
++ const g = await ctx.goals.get(db, input.goal_id);
if (!g || g.creator_subject !== creator) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
goalId = g.id;
}
-- const id = prefixedId('tovc', ctx.now());
const at = iso(ctx.now());
-- db.prepare(`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`)
-- .run(id, creator, kind, name, JSON.stringify(settingsOf(kind, input, DEFAULTS[kind])), goalId, at, at);
-- return presentConfig(getConfig(id));
++ const row = await db.one(sql`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at)
++ VALUES (${prefixedId('tovc', ctx.now())}, ${creator}, ${kind}, ${name}, ${sql.json(settingsOf(kind, input, DEFAULTS[kind]))}, ${goalId}, ${at}, ${at}) RETURNING *`);
++ return presentConfig(row);
}
-- function updateConfig(c, input = {}) {
-- if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);
-- const settings = settingsOf(c.kind, input, json(c.settings, {}));
++ /**
++ * A settings change on the locked row (two at once apply one after the other); with `revision` it
++ * applies only to that revision (else 409).
++ */
++ function updateConfig(config, input = {}) {
++ return ctx.tx(async (t) => {
++ const c = await t.one(sql`SELECT * FROM overlay_configs WHERE id = ${config.id} FOR UPDATE`);
++ if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);
++ return changeConfig(t, c, input);
++ });
++ }
++ async function changeConfig(t, c, input) {
++ const settings = settingsOf(c.kind, input, c.settings || {});
let goalId = c.goal_id;
if (c.kind === 'goal' && input.goal_id !== undefined) {
if (!input.goal_id) goalId = null;
else {
-- const g = ctx.goals.get(input.goal_id);
++ const g = await ctx.goals.get(t, input.goal_id);
if (!g || g.creator_subject !== c.creator_subject) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
goalId = g.id;
}
}
const name = input.name !== undefined ? (text(input.name, 'name', 80) || c.name) : c.name;
-- db.prepare('UPDATE overlay_configs SET name = ?, settings = ?, goal_id = ?, revision = revision + 1, updated_at = ? WHERE id = ?')
-- .run(name, JSON.stringify(settings), goalId, iso(ctx.now()), c.id);
-- const out = presentConfig(getConfig(c.id));
-- // Open overlays using this config pick the change up at once.
-- for (const cl of clients.get(c.creator_subject) || []) if (cl.configId === c.id) write(cl, 'config', null, { config: out });
++ const out = presentConfig(await t.one(sql`UPDATE overlay_configs SET name = ${name}, settings = ${sql.json(settings)}, goal_id = ${goalId},
++ revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${c.id} RETURNING *`));
++ // Open overlays using this config, in any process, pick the change up once it is committed.
++ t.after(() => hub.config(c.creator_subject, out));
return out;
}
function presentConfig(c) {
if (!c) return null;
-- return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: json(c.settings, {}), goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };
++ return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: c.settings || {}, goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };
}
// ── Deliveries ───────────────────────────────────────────
-- function insertDelivery(creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {
-- const id = prefixedId('tovd', ctx.now());
-- const r = db.prepare(`INSERT OR IGNORE INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)
-- VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)`).run(id, creator, kind, interactionId, goalId, JSON.stringify(payload), test ? 1 : 0, dedupe, iso(ctx.now()));
-- if (r.changes) ctx.afterCommit(() => notify(creator));
-- return r.changes ? id : null;
++ /**
++ * Inside a transaction. Open overlays hear of it once the transaction committed.
++ *
++ * A row's seq comes from its identity when it is inserted, but other processes see it only at commit.
++ * Two transactions of one creator committing out of order (seq 101 before 100) would let a stream move
++ * past 100 before 100 exists for it, and never send it. So a creator's deliveries are inserted under
++ * a transaction-scoped advisory lock (allowed through PgBouncer): the next insert for that creator
++ * waits for the previous transaction to end, and a creator's seqs become visible in order.
++ */
++ async function insertDelivery(t, creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {
++ await t.query(sql`SELECT pg_advisory_xact_lock(${STREAM_LOCK}, hashtext(${creator}))`);
++ const row = await t.maybe(sql`INSERT INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)
++ VALUES (${prefixedId('tovd', ctx.now())}, ${creator}, ${kind}, ${interactionId}, ${goalId}, ${sql.json(payload)}, ${!!test}, 'pending', ${dedupe}, ${iso(ctx.now())})
++ ON CONFLICT DO NOTHING RETURNING id`);
++ if (row) ctx.afterCommit(t, () => hub.notify(creator));
++ return row ? row.id : null;
}
-- /** Inside the settlement transaction (or a simulation). */
-- function addAlert(interaction) {
-- const payload = ctx.view(interaction, { at: iso(ctx.now()) });
-- return insertDelivery(interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });
++ /** Inside the settlement transaction (or a simulation). `filter`: the creator's word filter. */
++ function addAlert(t, interaction, filter) {
++ const payload = publicView(interaction, { at: iso(ctx.now()), filter });
++ return insertDelivery(t, interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });
}
/**
-- * Inside a transaction: the interaction's public view changed. Its alert payload is rewritten, and
-- * the goal updates it caused no longer name anyone.
++ * Inside a transaction: these interactions' public view changed (an erasure, a moderation). Their
++ * alert payloads are rewritten and the goal updates they caused no longer name anyone, in one read
++ * and one write. `filters`: Map(creator → the creator's word filter).
*/
-- function refreshInteraction(interaction) {
-- const view = ctx.view(interaction);
-- for (const row of db.prepare('SELECT seq, kind, payload FROM overlay_deliveries WHERE interaction_id = ?').all(interaction.id)) {
-- const old = json(row.payload, {});
-- const payload = row.kind === 'alert' ? ctx.view(interaction, { at: old.at }) : { ...old, by: interaction.hide_amount || isHidden(interaction) ? null : view.supporter_name };
-- db.prepare('UPDATE overlay_deliveries SET payload = ? WHERE seq = ?').run(JSON.stringify(payload), row.seq);
-- }
++ async function refreshInteractions(t, interactions, filters) {
++ if (!interactions.length) return;
++ const byId = new Map(interactions.map((i) => [i.id, i]));
++ const rows = await t.many(sql`SELECT seq, kind, payload, interaction_id FROM overlay_deliveries WHERE interaction_id = ANY(${[...byId.keys()]})`);
++ if (!rows.length) return;
++ const updates = rows.map((row) => {
++ const i = byId.get(row.interaction_id);
++ const filter = filters.get(i.creator_subject) || null;
++ const old = row.payload || {};
++ const payload = row.kind === 'alert' ? publicView(i, { at: old.at, filter })
++ : { ...old, by: i.hide_amount || isHidden(i) ? null : publicView(i, { filter }).supporter_name };
++ return { seq: row.seq, payload };
++ });
++ await t.exec(sql`UPDATE overlay_deliveries d SET payload = v.payload
++ FROM jsonb_to_recordset(${sql.json(updates)}) AS v(seq bigint, payload jsonb) WHERE d.seq = v.seq`);
}
++ const refreshInteraction = (t, interaction, filter) => refreshInteractions(t, [interaction], new Map([[interaction.creator_subject, filter]]));
/** Inside a transaction: a moderator hid the interaction. Its alert is never sent or replayed again. */
-- function retract(interaction) {
-- db.prepare("UPDATE overlay_deliveries SET hidden = 1 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);
-- refreshInteraction(interaction);
-- const rows = db.prepare("SELECT id, seq FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").all(interaction.id);
-- ctx.afterCommit(() => {
-- for (const c of clients.get(interaction.creator_subject) || []) {
-- if (!c.scopes.includes('alerts')) continue;
-- for (const r of rows) write(c, 'retract', null, { interaction_id: interaction.id, delivery_id: r.id, seq: r.seq });
-- }
-- });
++ async function retract(t, interaction, filter) {
++ const rows = await t.many(sql`UPDATE overlay_deliveries SET hidden = true WHERE interaction_id = ${interaction.id} AND kind = 'alert' RETURNING id, seq`);
++ await refreshInteraction(t, interaction, filter);
++ if (rows.length) ctx.afterCommit(t, () => hub.retract(interaction.creator_subject, interaction.id, rows.map((r) => ({ id: r.id, seq: r.seq }))));
}
/** Inside a transaction: shown again. Back in /state and replays; open overlays are not re-alerted. */
-- function unretract(interaction) {
-- db.prepare("UPDATE overlay_deliveries SET hidden = 0 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);
-- refreshInteraction(interaction);
++ async function unretract(t, interaction, filter) {
++ await t.exec(sql`UPDATE overlay_deliveries SET hidden = false WHERE interaction_id = ${interaction.id} AND kind = 'alert'`);
++ await refreshInteraction(t, interaction, filter);
}
-- function addGoalDelivery(creator, goalView, { reason, interactionId, by, dedupe, test = false }) {
-- return insertDelivery(creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });
++ function addGoalDelivery(t, creator, goalView, { reason, interactionId, by, dedupe, test = false }) {
++ return insertDelivery(t, creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });
}
-- /** First write of a pending row: delivered + the event (never for tests). */
-- function markDelivered(row) {
-- ctx.tx(() => {
-- const r = db.prepare("UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);
-- if (r.changes && !row.test) {
-- ctx.outbox.emit('tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {
++ /**
++ * About to write these rows to an overlay: count the sends, and the first write of a pending row
++ * makes it delivered + the event (never for tests). One transaction for the batch.
++ */
++ async function recordSends(rows) {
++ const pending = rows.filter((r) => r.status === 'pending').map((r) => r.seq);
++ await ctx.tx(async (t) => {
++ await t.exec(sql`UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ANY(${rows.map((r) => r.seq)})`);
++ if (!pending.length) return;
++ const done = await t.many(sql`UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ${iso(ctx.now())}
++ WHERE seq = ANY(${pending}) AND status = 'pending' RETURNING id, creator_subject, kind, interaction_id, goal_id, test`);
++ for (const row of done) {
++ if (row.test) continue;
++ await ctx.outbox.emit(t, 'tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {
delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id, goal_id: row.goal_id,
});
}
++ if (done.some((r) => !r.test)) t.after(ctx.outboxKick);
});
-- ctx.outboxKick();
}
-- /** Pending rows older than the window → failed (tips.overlay.failed). Returns how many. */
-- function sweepFailed() {
++ /**
++ * Pending rows older than the window → failed (tips.overlay.failed). Returns how many. Rows are
++ * claimed with SKIP LOCKED, so processes sweeping at once never fail a row twice.
++ */
++ async function sweepFailed() {
const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE status = 'pending' AND hidden = 0 AND created_at < ? ORDER BY seq LIMIT 500").all(cutoff);
-- if (!rows.length) return 0;
-- ctx.tx(() => {
++ const n = await ctx.tx(async (t) => {
++ const rows = await t.many(sql`UPDATE overlay_deliveries SET status = 'failed', failed_at = ${iso(ctx.now())}
++ WHERE seq IN (SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < ${cutoff}
++ ORDER BY seq LIMIT 500 FOR UPDATE SKIP LOCKED)
++ RETURNING id, seq, creator_subject, kind, interaction_id, goal_id, test`);
++ rows.sort((a, b) => a.seq - b.seq);
for (const row of rows) {
-- const r = db.prepare("UPDATE overlay_deliveries SET status = 'failed', failed_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);
-- if (r.changes && !row.test) {
-- ctx.outbox.emit('tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {
-- delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,
-- goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',
-- });
-- }
++ if (row.test) continue;
++ await ctx.outbox.emit(t, 'tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {
++ delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,
++ goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',
++ });
}
++ return rows.length;
});
-- ctx.outboxKick();
-- return rows.length;
++ if (n) ctx.outboxKick();
++ return n;
}
-- // ── SSE hub ──────────────────────────────────────────────
++ // ── Streams (overlay-hub.js) ─────────────────────────────
const scopeOf = (kind) => (kind === 'alert' ? 'alerts' : 'goals');
-- function write(client, event, id, data) {
-- if (client.closed) return false;
-- try {
-- client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
-- return true;
-- } catch { return false; }
-- }
--
-- function send(client, row) {
-- client.lastSeq = Math.max(client.lastSeq, row.seq);
-- if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) return;
-- const payload = json(row.payload, {});
-- if (row.kind === 'alert' && client.minAmount && !row.test) {
-- // A hidden amount is not in the payload; the threshold still applies to the real one.
-- const amount = payload.amount != null ? payload.amount : (db.prepare('SELECT amount FROM tip_interactions WHERE id = ?').get(row.interaction_id) || {}).amount;
-- if (amount < client.minAmount) return;
++ /** Write these rows to a client (scopes, retractions and its minimum applied), after recording them. */
++ async function deliver(client, rows) {
++ const shown = [];
++ for (const row of rows) {
++ client.lastSeq = Math.max(client.lastSeq, row.seq);
++ if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) continue;
++ const payload = row.payload || {};
++ if (row.kind === 'alert' && client.minAmount && !row.test) {
++ // A hidden amount is not in the payload; the threshold still applies to the real one.
++ const amount = payload.amount != null ? payload.amount : row.interaction_amount;
++ if (amount < client.minAmount) continue;
++ }
++ shown.push({ row, payload });
}
-- if (!write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload })) return;
-- db.prepare('UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ?').run(row.seq);
-- if (row.status === 'pending') markDelivered(row);
++ if (!shown.length || client.closed) return;
++ await recordSends(shown.map((x) => x.row));
++ for (const { row, payload } of shown) hub.write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload });
}
-- function pushNew(client) {
-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq LIMIT 200").all(client.creator, client.lastSeq);
-- for (const row of rows) send(client, row);
++ async function pushNew(client) {
++ const rows = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
++ WHERE d.creator_subject = ${client.creator} AND d.seq > ${client.lastSeq} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq LIMIT 200`);
++ await deliver(client, rows);
}
-- function notify(creator) {
-- for (const c of clients.get(creator) || []) pushNew(c);
-- }
++ /** A stream place for this token (TIPS_OVERLAY_MAX_STREAMS across processes), or null: answer 429. */
++ const reserveStream = (token) => hub.reserve(token, ctx.config.overlays.maxStreamsPerToken);
++ const releaseStream = (slot) => hub.release(slot);
/**
-- * Attach an SSE response. lastEventId (Last-Event-ID): replay what came after it (bounded).
-- * Without it: what is still pending within the window, then everything new.
++ * Attach an SSE response (its slot from reserveStream). lastEventId (Last-Event-ID): replay what
++ * came after it (bounded). Without it: what is still pending within the window, then everything new.
*/
-- function attach(token, res, { lastEventId, config } = {}) {
-- const settings = config ? json(config.settings, {}) : {};
-- const client = {
-- id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId: token.config_id || null,
-- minAmount: config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0, res, lastSeq: 0, closed: false,
-- };
-- if (!clients.has(client.creator)) clients.set(client.creator, new Set());
-- clients.get(client.creator).add(client);
-- db.prepare('UPDATE overlay_tokens SET last_used_at = ? WHERE id = ?').run(iso(ctx.now()), token.id);
--
-- const maxSeq = db.prepare('SELECT COALESCE(MAX(seq), 0) AS n FROM overlay_deliveries WHERE creator_subject = ?').get(client.creator).n;
-- write(client, 'hello', null, {
-- creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,
-- goals: client.scopes.includes('goals') ? ctx.goals.list(client.creator, { status: 'active' }).map((g) => ctx.goals.present(g)) : [],
-- resumed_from: lastEventId != null ? lastEventId : null,
++ function attach(token, res, slot, { lastEventId, config } = {}) {
++ const settings = config ? config.settings || {} : {};
++ const minAmount = config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0;
++ return hub.attach(token, res, slot, { minAmount, configId: token.config_id || null }, async (client) => {
++ await db.exec(sql`UPDATE overlay_tokens SET last_used_at = ${iso(ctx.now())} WHERE id = ${token.id}`);
++ const maxSeq = await db.value(sql`SELECT COALESCE(MAX(seq), 0) FROM overlay_deliveries WHERE creator_subject = ${client.creator}`);
++ const goals = client.scopes.includes('goals') ? await ctx.goals.presentMany(db, await ctx.goals.list(db, client.creator, { status: 'active' })) : [];
++ hub.write(client, 'hello', null, {
++ creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,
++ goals, resumed_from: lastEventId != null ? lastEventId : null,
++ });
++ const last = Number(lastEventId);
++ if (lastEventId != null && Number.isInteger(last) && last >= 0) {
++ const limit = ctx.config.overlays.replayLimit;
++ const rows = (await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
++ WHERE d.creator_subject = ${client.creator} AND d.seq > ${last} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq DESC LIMIT ${limit}`)).reverse();
++ client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;
++ await deliver(client, rows);
++ client.lastSeq = Math.max(client.lastSeq, maxSeq);
++ } else {
++ const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
++ const pending = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
++ WHERE d.creator_subject = ${client.creator} AND d.status = 'pending' AND NOT d.hidden AND d.created_at >= ${cutoff} ORDER BY d.seq LIMIT 100`);
++ await deliver(client, pending);
++ client.lastSeq = Math.max(client.lastSeq, maxSeq);
++ }
});
-- const last = Number(lastEventId);
-- if (lastEventId != null && Number.isInteger(last) && last >= 0) {
-- const limit = ctx.config.overlays.replayLimit;
-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq DESC LIMIT ?").all(client.creator, last, limit).reverse();
-- client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;
-- for (const row of rows) send(client, row);
-- client.lastSeq = Math.max(client.lastSeq, maxSeq);
-- } else {
-- const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
-- const pending = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND status = 'pending' AND hidden = 0 AND created_at >= ? ORDER BY seq LIMIT 100").all(client.creator, cutoff);
-- for (const row of pending) send(client, row);
-- client.lastSeq = Math.max(client.lastSeq, maxSeq);
-- }
-- pushNew(client);
-- const beat = setInterval(() => { if (!write(client, 'ping', null, { t: Date.now() })) detach(client); }, ctx.config.overlays.heartbeatMs);
-- if (beat.unref) beat.unref();
-- client.beat = beat;
-- res.on('close', () => detach(client));
-- return client;
}
-- function detach(client) {
-- if (client.closed) return;
-- client.closed = true;
-- clearInterval(client.beat);
-- const set = clients.get(client.creator);
-- if (set) { set.delete(client); if (!set.size) clients.delete(client.creator); }
-- try { client.res.end(); } catch { /* already gone */ }
++ function recentDeliveries(q, creator, limit = 50) {
++ return q.many(sql`SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at
++ FROM overlay_deliveries WHERE creator_subject = ${creator} ORDER BY seq DESC LIMIT ${limit}`);
}
-- function closeToken(tokenId) {
-- for (const set of clients.values()) {
-- for (const c of [...set]) {
-- if (c.tokenId === tokenId) { write(c, 'revoked', null, { reason: 'this overlay token was revoked' }); detach(c); }
-- }
-- }
-- }
--
-- function connected(creator) { return (clients.get(creator) || new Set()).size; }
-- function streamsFor(token) { let n = 0; for (const c of clients.get(token.creator_subject) || []) if (c.tokenId === token.id) n++; return n; }
-- function closeAll() { for (const set of clients.values()) for (const c of [...set]) detach(c); }
--
-- function recentDeliveries(creator, limit = 50) {
-- return db.prepare('SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at FROM overlay_deliveries WHERE creator_subject = ? ORDER BY seq DESC LIMIT ?').all(creator, limit);
++ /** The newest alerts of a creator, for GET /overlay/:token/state. */
++ async function recentAlerts(q, creator) {
++ return (await q.many(sql`SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ${creator} AND kind = 'alert' AND NOT hidden
++ ORDER BY seq DESC LIMIT 20`)).map((d) => ({ seq: d.seq, ...(d.payload || {}), test: !!d.test, created_at: d.created_at }));
}
return {
SCOPES, createToken, getToken, listTokens, authenticate, revokeToken, presentToken,
getConfig, listConfigs, createConfig, updateConfig, presentConfig,
-- addAlert, refreshInteraction, retract, unretract, addGoalDelivery, sweepFailed, attach, detach, closeToken, notify, connected, streamsFor, closeAll, recentDeliveries,
++ addAlert, refreshInteraction, refreshInteractions, retract, unretract, addGoalDelivery, sweepFailed, reserveStream, releaseStream, attach,
++ connected: hub.connected, closeAll: () => hub.closeAll(), close: () => hub.close(), recentDeliveries, recentAlerts,
};
}
ddiff --git a/server/domain/privacy.js b/server/domain/privacy.js
iindex 4e5ea03..bcaefec 100644
---- a/server/domain/privacy.js
++++ b/server/domain/privacy.js
@@@ -25,7 +25,7 @@
* word filter (filter.js) stars blocked words out of the name and message and drops them from what
* text-to-speech reads.
*/
--const { fail, json } = require('../util');
++const { fail } = require('../util');
const filter = require('./filter');
const ANONYMOUS = 'Anonymous';
@@@ -64,10 +64,11 @@ function shownName(i, settings) {
/**
* The public shape of an interaction. `at` is kept from an earlier overlay payload when given, so a
-- * rewritten alert keeps its time; `filter` is the creator's filter settings.
++ * rewritten alert keeps its time; `filter` is the creator's filter settings (profiles.filterOf, or a
++ * profile's `filter`), fetched once per creator by the caller.
*/
function publicView(i, { at, filter: settings = null } = {}) {
-- const req = json(i.request, {});
++ const req = i.request || {};
const privateMsg = !!i.private_message || !!i.erased_at;
const base = {
interaction_id: i.id,
ddiff --git a/server/domain/profiles.js b/server/domain/profiles.js
iindex a91fcad..fbc0caa 100644
---- a/server/domain/profiles.js
++++ b/server/domain/profiles.js
@@@ -19,7 +19,8 @@
* filter_words / filter_action / filter_links are the creator's word filter for paid messages
* (filter.js; `filter` in the API): filterOf(subject) is what every public view applies.
*/
--const { fail, iso, text, json } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { fail, iso, text } = require('../util');
const filterLib = require('./filter');
const HANDLE_RE = /^[a-z0-9][a-z0-9_.-]{0,39}$/;
@@@ -36,55 +37,70 @@ function normalizeHandle(v) {
const PAGE_DEFAULTS = Object.freeze({ goal_amounts: true, goal_supporters: false, supporters_page: false, supporters_amounts: false, supporters_messages: false });
--function pageSettings(raw) {
-- const stored = json(raw, {});
++/** page_settings (jsonb) → every setting, defaults filled in. */
++function pageSettings(stored) {
++ const s = stored && typeof stored === 'object' ? stored : {};
const out = {};
-- for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof stored[k] === 'boolean' ? stored[k] : PAGE_DEFAULTS[k];
++ for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof s[k] === 'boolean' ? s[k] : PAGE_DEFAULTS[k];
return out;
}
++/** The creator's filter settings for publicView() from a profile row (filter_* columns). */
++const filterOfRow = (row) => ({
++ words: Array.isArray(row.filter_words) ? row.filter_words : [],
++ action: row.filter_action === 'hold' ? 'hold' : 'mask',
++ links: row.filter_links !== false,
++});
++
function parse(row) {
if (!row) return null;
-- const b = (k) => !!row[k];
-- return {
-- ...row, page_enabled: b('page_enabled'), accepting: b('accepting'), tts_enabled: b('tts_enabled'), media_requests_enabled: b('media_requests_enabled'),
-- page: pageSettings(row.page_settings),
-- filter: { words: json(row.filter_words, []), action: filterLib.ACTIONS.includes(row.filter_action) ? row.filter_action : 'mask', links: row.filter_links !== 0 },
-- };
++ return { ...row, page: pageSettings(row.page_settings), filter: filterOfRow(row) };
}
++const HANDLE_TAKEN = 'creator_tip_profiles_handle_key';
++
function createProfiles(ctx) {
const { db } = ctx;
-- const bySubject = (s) => parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE creator_subject = ?').get(s));
-- const byHandle = (h) => { const n = normalizeHandle(h); return n ? parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE handle = ?').get(n)) : null; };
++ const bySubject = async (q, s) => parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(s || '')}`));
++ const byHandle = async (q, h) => { const n = normalizeHandle(h); return n ? parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE handle = ${n}`)) : null; };
/** usr_… or a handle (with or without @). */
-- const resolve = (ref) => (/^usr_/.test(String(ref || '')) ? bySubject(String(ref)) : byHandle(ref));
++ const resolve = (q, ref) => (/^usr_/.test(String(ref || '')) ? bySubject(q, String(ref)) : byHandle(q, ref));
/** A handle not held by another creator: the username, else username-<suffix of the subject>. */
-- function freeHandle(subject, username) {
++ async function freeHandle(q, subject, username) {
let h = normalizeHandle(username) || `creator-${subject.slice(-8).toLowerCase()}`;
-- const holder = db.prepare('SELECT creator_subject FROM creator_tip_profiles WHERE handle = ?').get(h);
++ const holder = await q.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE handle = ${h}`);
if (holder && holder.creator_subject !== subject) h = `${h.slice(0, 30)}-${subject.slice(-6).toLowerCase()}`;
return h;
}
-- /** Create or refresh the profile from the creator's own Network account (claims). */
-- function ensure(subject, { username, displayName, avatarUrl } = {}) {
++ /**
++ * Create or refresh the profile from the creator's own Network account (claims). Two first visits
++ * at once create it once; a handle another creator took meanwhile is looked for again.
++ */
++ async function ensure(subject, claims = {}, attempt = 0) {
++ try { return await ensureOnce(subject, claims); } catch (e) {
++ if (e.code === '23505' && e.constraint === HANDLE_TAKEN && attempt < 2) return ensure(subject, claims, attempt + 1);
++ throw e;
++ }
++ }
++ async function ensureOnce(subject, { username, displayName, avatarUrl } = {}) {
const at = iso(ctx.now());
-- const existing = bySubject(subject);
++ const existing = await bySubject(db, subject);
const name = String(displayName || username || 'Creator').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80) || 'Creator';
if (!existing) {
-- db.prepare(`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)
-- VALUES (?, ?, ?, ?, ?, ?)`).run(subject, freeHandle(subject, username), name, safeUrl(avatarUrl), at, at);
-- return bySubject(subject);
++ const row = await db.maybe(sql`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)
++ VALUES (${subject}, ${await freeHandle(db, subject, username)}, ${name}, ${safeUrl(avatarUrl)}, ${at}, ${at})
++ ON CONFLICT (creator_subject) DO NOTHING RETURNING *`);
++ return row ? parse(row) : bySubject(db, subject);
}
-- const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? freeHandle(subject, username) : existing.handle;
++ const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? await freeHandle(db, subject, username) : existing.handle;
if (handle !== existing.handle || name !== existing.display_name || (safeUrl(avatarUrl) || null) !== (existing.avatar_url || null)) {
-- db.prepare('UPDATE creator_tip_profiles SET handle = ?, display_name = ?, avatar_url = ?, updated_at = ? WHERE creator_subject = ?')
-- .run(handle, name, safeUrl(avatarUrl), at, subject);
++ return parse(await db.one(sql`UPDATE creator_tip_profiles SET handle = ${handle}, display_name = ${name}, avatar_url = ${safeUrl(avatarUrl)}, updated_at = ${at}
++ WHERE creator_subject = ${subject} RETURNING *`));
}
-- return bySubject(subject);
++ return existing;
}
const LIMITS = {
@@@ -92,13 +108,25 @@ function createProfiles(ctx) {
media_request_min: [1, 1_000_000], media_max_seconds: [10, 3 * 3600],
};
const BOOLS = ['page_enabled', 'accepting', 'tts_enabled', 'media_requests_enabled'];
++ const on = (v) => v === true || v === 'on' || v === '1' || v === 1;
++ /**
++ * A settings change, on the locked row (two changes at once apply one after the other, neither lost).
++ * With expectedRevision it applies only to that revision (else 409).
++ */
function update(subject, patch = {}, { expectedRevision } = {}) {
-- const p = bySubject(subject);
-- if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');
-- if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);
++ return ctx.tx(async (t) => {
++ const p = parse(await t.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(subject || '')} FOR UPDATE`));
++ if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');
++ if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);
++ return change(t, p, patch);
++ });
++ }
++
++ async function change(t, p, patch) {
++ const subject = p.creator_subject;
const set = {};
-- for (const k of BOOLS) if (patch[k] !== undefined) set[k] = patch[k] === true || patch[k] === 'on' || patch[k] === '1' || patch[k] === 1 ? 1 : 0;
++ for (const k of BOOLS) if (patch[k] !== undefined) set[k] = on(patch[k]);
for (const [k, [lo, hi]] of Object.entries(LIMITS)) {
if (patch[k] === undefined || patch[k] === '') continue;
const n = Number(patch[k]);
@@@ -120,31 +148,29 @@ function createProfiles(ctx) {
const unknown = Object.keys(patch.page).filter((k) => !(k in PAGE_DEFAULTS));
if (unknown.length) fail(422, 'tips.invalid_input', `page has no setting ${unknown[0]}`);
const next = { ...p.page };
-- for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = patch.page[k] === true || patch.page[k] === 'on' || patch.page[k] === '1' || patch.page[k] === 1;
-- set.page_settings = JSON.stringify(next);
++ for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = on(patch.page[k]);
++ set.page_settings = sql.json(next);
}
if (patch.filter !== undefined) {
const f = patch.filter;
if (!f || typeof f !== 'object' || Array.isArray(f)) fail(422, 'tips.invalid_input', 'filter must be an object { words, action, links }');
const unknown = Object.keys(f).filter((k) => !['words', 'action', 'links'].includes(k));
if (unknown.length) fail(422, 'tips.invalid_input', `filter has no setting ${unknown[0]}`);
-- if (f.words !== undefined) set.filter_words = JSON.stringify(filterLib.normalizeWords(f.words));
++ if (f.words !== undefined) set.filter_words = sql.json(filterLib.normalizeWords(f.words));
if (f.action !== undefined) {
if (!filterLib.ACTIONS.includes(f.action)) fail(422, 'tips.invalid_input', `filter.action must be one of ${filterLib.ACTIONS.join(', ')}`);
set.filter_action = f.action;
}
-- if (f.links !== undefined) set.filter_links = f.links === true || f.links === 'on' || f.links === '1' || f.links === 1 ? 1 : 0;
++ if (f.links !== undefined) set.filter_links = on(f.links);
}
if (patch.display_name !== undefined) {
const n = text(patch.display_name, 'display_name', 80);
if (!n) fail(422, 'tips.invalid_input', 'display_name cannot be empty');
set.display_name = n.replace(/[<>]/g, '');
}
-- const keys = Object.keys(set);
-- if (!keys.length) return p;
-- db.prepare(`UPDATE creator_tip_profiles SET ${keys.map((k) => `${k} = @${k}`).join(', ')}, revision = revision + 1, updated_at = @at WHERE creator_subject = @s`)
-- .run({ ...set, at: iso(ctx.now()), s: subject });
-- return bySubject(subject);
++ if (!Object.keys(set).length) return p;
++ return parse(await t.one(sql`UPDATE creator_tip_profiles SET ${sql.set({ ...set, updated_at: iso(ctx.now()) })}, revision = revision + 1
++ WHERE creator_subject = ${subject} RETURNING *`));
}
/** Public shape; `owner` adds the settings only the creator sees. */
@@@ -165,13 +191,31 @@ function createProfiles(ctx) {
}
/** The creator's filter settings for publicView() (null when they have no profile). */
-- function filterOf(subject) {
-- const row = db.prepare('SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ?').get(subject);
-- if (!row) return null;
-- return { words: json(row.filter_words, []), action: row.filter_action === 'hold' ? 'hold' : 'mask', links: row.filter_links !== 0 };
++ async function filterOf(q, subject) {
++ const row = await q.maybe(sql`SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ${subject}`);
++ return row ? filterOfRow(row) : null;
++ }
++ /** Several creators' profiles in one query: Map(subject → profile | null). */
++ async function bySubjects(q, subjects) {
++ const list = [...new Set(subjects)];
++ const out = new Map(list.map((s) => [s, null]));
++ if (!list.length) return out;
++ for (const row of await q.many(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) out.set(row.creator_subject, parse(row));
++ return out;
++ }
++
++ /** Several creators' filters in one query: Map(subject → settings | null). */
++ async function filtersOf(q, subjects) {
++ const list = [...new Set(subjects)];
++ const out = new Map(list.map((s) => [s, null]));
++ if (!list.length) return out;
++ for (const row of await q.many(sql`SELECT creator_subject, filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) {
++ out.set(row.creator_subject, filterOfRow(row));
++ }
++ return out;
}
-- return { bySubject, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf };
++ return { bySubject, bySubjects, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf, filtersOf };
}
function safeUrl(u) {
ddiff --git a/server/events/consumer.js b/server/events/consumer.js
iindex cd1833c..7f7bb53 100644
---- a/server/events/consumer.js
++++ b/server/events/consumer.js
@@@ -14,41 +14,41 @@
* chat line, overlay alert, goal. Billing sends it only once it is the
* money authority (while Live is, Live's webhook announces the tip)
*
-- * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same SQLite
++ * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same
* transaction as the change, so a redelivered event does nothing; and the Billing transaction id is
* UNIQUE on tip_interactions, so the same transaction arriving under another event id (a replay, a
-- * republish) still yields one logical interaction.
++ * republish) still yields one logical interaction. That transaction moves money state, so it is
++ * SERIALIZABLE (domain MONEY): two deliveries racing in two processes settle once.
*
* The signature (X-OpenVibe-Signature, HMAC-SHA256 of the raw body with TIPS_EVENTS_SECRET) is
* verified with openvibe-sdk's parseDelivery. Only events whose source is `billing` are applied.
*/
const express = require('express');
const { http } = require('openvibe-contracts');
--const { parseDelivery, createInbox } = require('openvibe-sdk/events');
++const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');
const CONSUMER = 'tips-billing';
++const TABLE = 'tips_event_inbox'; // migrations/0001_initial.sql (inboxSchema)
function consumerRouter({ domain, config, log = console }) {
const router = express.Router();
-- const inbox = createInbox(domain.db, { now: domain.now });
-- inbox.ensureSchema();
++ // The inbox's transaction is the domain's MONEY one (serializable, with after-commit hooks).
++ const inbox = createPgInbox({ tx: (fn) => domain.tx(fn, domain.MONEY), maybe: (...a) => domain.db.maybe(...a) }, { table: TABLE, now: domain.now });
/** Apply one envelope (also used by tests and the replay tool). Returns { duplicate, outcome }. */
-- function apply(event) {
-- return domain.tx(() => {
-- const r = inbox.once(CONSUMER, event.event_id, () => {
-- if (event.source !== 'billing') return 'ignored:source';
-- const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};
-- if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(payload);
-- if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(payload);
-- if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(payload);
-- return 'ignored:type';
-- });
-- return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };
++ async function apply(event) {
++ const r = await inbox.once(CONSUMER, event.event_id, async (t) => {
++ if (event.source !== 'billing') return 'ignored:source';
++ const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};
++ if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(t, payload);
++ if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(t, payload);
++ if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(t, payload);
++ return 'ignored:type';
});
++ return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };
}
-- router.post('/events', express.raw({ type: () => true, limit: '256kb' }), (req, res) => {
++ router.post('/events', express.raw({ type: () => true, limit: '256kb' }), async (req, res) => {
const secrets = config.events.webhookSecrets;
if (!secrets.length) return http.sendProblem(res, 503, 'tips.webhook_disabled', { detail: 'TIPS_EVENTS_SECRET is not set', ctx: req.ov });
const raw = Buffer.isBuffer(req.body) ? req.body : Buffer.alloc(0);
@@@ -62,7 +62,7 @@ function consumerRouter({ domain, config, log = console }) {
}
let out;
try {
-- out = apply(event);
++ out = await apply(event);
} catch (e) {
// Not acknowledged: Events retries it, and the inbox claim rolled back with the change.
log.error(`[Tips] event ${event.event_id} (${event.event_type}) failed:`, e.message);
ddiff --git a/server/events/outbox.js b/server/events/outbox.js
iindex 932346a..6258449 100644
---- a/server/events/outbox.js
++++ b/server/events/outbox.js
@@@ -10,16 +10,18 @@
* tips.overlay.delivered an overlay delivery reached at least one overlay (first time only)
* tips.overlay.failed no overlay received it within the delivery window
*
-- * enqueue() runs inside the SQLite transaction that makes the change, so an event exists if and
-- * only if its change committed. Simulations (test) never produce events. The relay publishes with
++ * emit(t, …) runs inside the transaction that makes the change (its handle `t`), so an event exists if
++ * and only if its change committed. Simulations (test) never produce events. The relay publishes with
* Tips' service token (events.event.publish, audience openvibe.events) only when EVENTS_URL and
-- * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in event_outbox.
++ * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in tips_event_outbox. Any number of processes
++ * relay the one table: the SDK claims due rows with a lease (FOR UPDATE SKIP LOCKED).
*/
const { createClient } = require('openvibe-sdk/core');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
--const { createEventsClient, createOutbox } = require('openvibe-sdk/events');
++const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
const ACTOR = { type: 'service', id: 'tips' };
++const TABLE = 'tips_event_outbox'; // migrations/0001_initial.sql (outboxSchema)
function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
const enabled = !!(config.events.url && config.oauth.clientSecret);
@@@ -35,8 +37,9 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
}
const events = createEventsClient(createClient(clientOpts), { source: 'tips' });
let lastError = null;
-- const outbox = createOutbox(db, {
++ const outbox = createPgOutbox(db, {
events,
++ table: TABLE,
intervalMs: config.events.intervalMs,
now,
onError: (err) => {
@@@ -45,11 +48,10 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
lastError = msg;
},
});
-- outbox.ensureSchema();
-- /** Inside the caller's transaction. */
-- function emit(eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {
-- return outbox.enqueue({ event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });
++ /** Inside the caller's transaction `t`. */
++ function emit(t, eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {
++ return outbox.enqueue(t, { event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });
}
return {
@@@ -59,8 +61,8 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
start() { if (enabled) outbox.start(); },
stop: () => outbox.stop(),
kick() { if (enabled) outbox.kick(); },
-- status: () => ({ enabled, pending: outbox.pending(), rejected: outbox.rejected(), last_error: lastError }),
++ status: async () => ({ enabled, pending: await outbox.pending(), rejected: await outbox.rejected(), last_error: lastError }),
};
}
--module.exports = { createTipsOutbox };
++module.exports = { createTipsOutbox, TABLE };
ddiff --git a/server/importer/live.js b/server/importer/live.js
iindex 74fa04f..5b2d81b 100644
---- a/server/importer/live.js
++++ b/server/importer/live.js
@@@ -32,8 +32,13 @@
* Idempotent: every row is keyed by legacy_source / migration_maps (live, table, id); a re-run changes
* nothing, and a later snapshot only adds what is new or releases what was held. --dry-run does it all
* inside a transaction that is rolled back.
++ *
++ * The snapshots are other services' SQLite files, read synchronously (better-sqlite3 handles passed in
++ * by scripts/import-live.js); everything written goes to Tips' PostgreSQL in one SERIALIZABLE
++ * transaction (it records settled money and applies refunds), whose report starts over on a retry.
*/
--const { iso, prefixedId, json } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { iso, prefixedId, json, storable } = require('../util');
const liveTs = (s) => {
if (!s) return null;
@@@ -41,8 +46,9 @@ const liveTs = (s) => {
const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);
return Number.isFinite(t) ? new Date(t).toISOString() : null;
};
--const tableExists = (db, name) => !!db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);
--const columns = (db, t) => (tableExists(db, t) ? db.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);
++// The snapshot (SQLite, read-only) side.
++const tableExists = (snap, name) => !!snap.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);
++const columns = (snap, t) => (tableExists(snap, t) ? snap.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);
class DryRun extends Error {}
@@@ -79,15 +85,15 @@ function billingSnapshotSource(bdb) {
}
async function importLive(domain, { live, billingSource = null, resolveLiveUsers, dryRun = false, log = console }) {
-- const { db } = domain;
const started = domain.now();
const runId = prefixedId('timp', started);
const at = iso(started);
const report = {
run_id: runId, dry_run: !!dryRun, started_at: at, source: 'openvibe-live snapshot', billing_source: billingSource ? billingSource.name : null,
-- counts: { imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 },
-- holds: [], excluded: [], unlinked: [], reconciliation: null,
++ counts: {}, holds: [], excluded: [], unlinked: [], reconciliation: null,
};
++ const fresh = () => ({ imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 });
++ Object.assign(report.counts, fresh());
// ── Read the snapshot ────────────────────────────────────
const txns = tableExists(live, 'transactions') ? live.prepare("SELECT * FROM transactions WHERE type IN ('donation', 'refund') ORDER BY id").all() : [];
@@@ -114,25 +120,36 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
for (const g of goalRows) if (g.user_id) ids.add(String(g.user_id));
const map = ids.size ? await resolveLiveUsers([...ids]) : new Map();
const subjectOf = (liveId) => { const v = liveId == null ? null : map.get(String(liveId)); return v ? (typeof v === 'string' ? v : v.subject) : null; };
-- const nameOf = (liveId) => { const u = users.get(String(liveId)); return u ? (u.display_name || u.username) : null; };
++ const nameOf = (liveId) => { const u = users.get(String(liveId)); return u && (u.display_name || u.username) ? storable(u.display_name || u.username) : null; };
report.counts.identities = { resolved: [...ids].filter((i) => subjectOf(i)).length, unmapped: [...ids].filter((i) => !subjectOf(i)).length };
-- const mapRow = db.prepare('SELECT * FROM migration_maps WHERE source = ? AND source_table = ? AND source_id = ?');
-- const upsertMap = db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)
-- VALUES ('live', @table, @id, @target_type, @target_id, @status, @reason, @billing, @run, @at, @at)
-- ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,
-- reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);
-- const record = (table, id, fields) => upsertMap.run({ table, id: String(id), target_type: null, target_id: null, reason: null, billing: null, run: runId, at, ...fields });
++ let q = null; // the import's transaction
++ const mapRow = (_source, table, id) => q.maybe(sql`SELECT * FROM migration_maps WHERE source = 'live' AND source_table = ${table} AND source_id = ${id}`);
++ const record = (table, id, fields) => {
++ const f = { target_type: null, target_id: null, reason: null, billing: null, ...fields };
++ return q.exec(sql`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)
++ VALUES ('live', ${table}, ${String(id)}, ${f.target_type}, ${f.target_id}, ${f.status}, ${f.reason}, ${f.billing}, ${runId}, ${at}, ${at})
++ ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,
++ reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);
++ };
-- const run = () => {
++ const run = async (t0) => {
++ q = t0;
++ // A retried attempt (serialization failure) reports from scratch.
++ Object.assign(report.counts, fresh(), { live: report.counts.live, identities: report.counts.identities });
++ report.holds = []; report.excluded = []; report.unlinked = [];
// ── 2. donations ─────────────────────────────────────
for (const t of txns.filter((x) => x.type === 'donation')) {
-- const prev = mapRow.get('live', 'transactions', String(t.id));
++ const prev = await mapRow('live', 'transactions', String(t.id));
if (prev && prev.status !== 'held') {
// Re-link to Billing when an earlier run had no Billing transaction for it yet.
if (prev.status === 'imported' && !prev.billing_txn_id && billingSource) {
const b = billingSource.linkLiveTxn(t.id);
-- if (b) { db.prepare('UPDATE tip_interactions SET billing_txn_id = ? WHERE id = ? AND billing_txn_id IS NULL').run(b, prev.target_id); record('transactions', t.id, { ...pick(prev), billing: b }); report.counts.linked_to_billing++; }
++ if (b) {
++ await q.exec(sql`UPDATE tip_interactions SET billing_txn_id = ${b} WHERE id = ${prev.target_id} AND billing_txn_id IS NULL`);
++ await record('transactions', t.id, { ...pick(prev), billing: b });
++ report.counts.linked_to_billing++;
++ }
}
report.counts.unchanged++;
continue;
@@@ -141,33 +158,33 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
const creator = subjectOf(t.to_user_id);
if (t.status !== 'completed' || amount <= 0) {
const reason = amount <= 0 ? 'non-positive amount' : `status ${t.status}: no balance moved on Live`;
-- record('transactions', t.id, { status: 'excluded', reason });
++ await record('transactions', t.id, { status: 'excluded', reason });
report.counts.excluded++; report.excluded.push({ live_txn: t.id, reason });
continue;
}
if (!creator) {
const reason = `creator (Live user ${t.to_user_id}) has no Network subject yet`;
-- record('transactions', t.id, { status: 'held', reason });
++ await record('transactions', t.id, { status: 'held', reason });
report.counts.held++; report.holds.push({ live_txn: t.id, live_creator: t.to_user_id, amount });
continue;
}
const created = liveTs(t.created_at) || at;
const test = !!(resetAt && created < resetAt);
const billingTxn = billingSource ? billingSource.linkLiveTxn(t.id) : null;
-- const message = t.message ? String(t.message).slice(0, 500) : null;
++ const message = t.message ? storable(String(t.message).slice(0, 500)) : null;
const media = /^Media request:/i.test(message || '');
const site = /^PowerChat tip via site account/i.test(message || '');
-- const i = domain.interactions.insert({
++ const i = await domain.interactions.insert(q, {
id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_subject: subjectOf(t.from_user_id),
supporter_name: nameOf(t.from_user_id) || (site ? 'PowerChat supporter' : null), kind: media ? 'media_request' : 'tip', amount,
message: media || site ? null : message, request: { legacy_message: message, live_stream_id: t.stream_id || null },
funding: site ? 'provider' : 'credit', settlement: 'imported', payment_state: 'settled', delivery_state: 'delivered',
-- test: test ? 1 : 0, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,
++ test, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,
origin: 'import', created_at: created,
target: t.stream_id ? { service: 'live', type: 'stream', id: String(t.stream_id) } : null,
});
-- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);
-- record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });
++ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);
++ await record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });
if (prev) report.counts.released++;
report.counts.imported++;
if (billingTxn) report.counts.linked_to_billing++;
@@@ -176,15 +193,17 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
// ── 3. refunds (streamer → donor) ────────────────────
for (const t of txns.filter((x) => x.type === 'refund')) {
-- const prev = mapRow.get('live', 'transactions', String(t.id));
++ const prev = await mapRow('live', 'transactions', String(t.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const amount = Math.round(Number(t.amount) || 0);
-- if (t.status !== 'completed' || amount <= 0) { record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }
++ if (t.status !== 'completed' || amount <= 0) { await record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }
const creator = subjectOf(t.from_user_id);
const donor = subjectOf(t.to_user_id);
const created = liveTs(t.created_at) || at;
-- const candidates = creator ? db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND settlement = 'imported' AND legacy_source LIKE 'live:transactions:%'
-- AND (supporter_subject IS ? ) AND created_at <= ? AND amount - reversed_bits > 0 ORDER BY created_at DESC`).all(creator, donor, created) : [];
++ // The donations a refund may apply to, locked in id order (the domain's lock order).
++ const candidates = creator ? (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND settlement = 'imported'
++ AND starts_with(legacy_source, 'live:transactions:') AND supporter_subject IS NOT DISTINCT FROM ${donor}::text AND created_at <= ${created}
++ AND amount - reversed_bits > 0 ORDER BY id FOR UPDATE`)).sort((a, b) => (a.created_at < b.created_at ? 1 : a.created_at > b.created_at ? -1 : 0)) : [];
// Plan first; apply only when the whole refund can be placed.
let left = amount;
const plan = [];
@@@ -195,17 +214,18 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
left -= take;
}
if (left > 0) {
-- record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });
++ await record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });
report.counts.held++; report.holds.push({ live_txn: t.id, refund: true, amount });
continue;
}
const applied = [];
for (const { c, take } of plan) {
-- const ids = json(c.reversal_txn_ids, []); ids.push(`live:transactions:${t.id}`);
-- db.prepare("UPDATE tip_interactions SET reversed_bits = reversed_bits + ?, payment_state = 'reversed', reversed_at = ?, reversal_txn_ids = ? WHERE id = ?").run(take, created, JSON.stringify(ids), c.id);
++ const ids = [...(c.reversal_txn_ids || []), `live:transactions:${t.id}`];
++ await q.exec(sql`UPDATE tip_interactions SET reversed_bits = reversed_bits + ${take}, payment_state = 'reversed', reversed_at = ${created},
++ reversal_txn_ids = ${sql.json(ids)} WHERE id = ${c.id}`);
applied.push({ interaction: c.id, bits: take });
}
-- record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });
++ await record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });
report.counts.refunds_applied++;
}
@@@ -215,7 +235,7 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
for (const c of chatDonations) {
const m = json(c.metadata, {});
if (m.source !== 'powerchat' || m.kind !== 'donation') continue;
-- const prev = mapRow.get('live', 'chat_messages', String(c.id));
++ const prev = await mapRow('live', 'chat_messages', String(c.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const amount = Math.round(Number(m.amount) || 0);
const created = liveTs(c.created_at) || at;
@@@ -223,53 +243,55 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
&& Math.abs(Date.parse(liveTs(t.created_at) || at) - Date.parse(created)) <= 10 * 60 * 1000);
if (twin) {
usedSite.add(twin.id);
-- record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });
++ await record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });
report.counts.excluded++;
continue;
}
-- if (amount <= 0) { record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }
++ if (amount <= 0) { await record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }
const creator = subjectOf(c.channel_user_id);
if (!creator) {
-- record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });
++ await record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });
report.counts.held++; report.holds.push({ live_chat_message: c.id, live_creator: c.channel_user_id, amount });
continue;
}
-- const i = domain.interactions.insert({
-- id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? String(m.username).slice(0, 80) : 'PowerChat supporter',
-- kind: 'tip', amount, amount_cents: amount, message: m.message ? String(m.message).slice(0, 500) : null, funding: 'external', settlement: 'external',
-- payment_state: 'settled', delivery_state: 'delivered', test: resetAt && created < resetAt ? 1 : 0, provider: 'powerchat',
++ const i = await domain.interactions.insert(q, {
++ id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? storable(String(m.username).slice(0, 80)) : 'PowerChat supporter',
++ kind: 'tip', amount, amount_cents: amount, message: m.message ? storable(String(m.message).slice(0, 500)) : null, funding: 'external', settlement: 'external',
++ payment_state: 'settled', delivery_state: 'delivered', test: !!(resetAt && created < resetAt), provider: 'powerchat',
provider_ref: `live-chat:${c.id}`, legacy_source: `live:chat_messages:${c.id}`, origin: 'import', created_at: created,
});
-- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);
-- record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });
++ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);
++ await record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });
report.counts.external++;
}
// ── 5. goals ─────────────────────────────────────────
for (const g of goalRows) {
-- const prev = mapRow.get('live', 'donation_goals', String(g.id));
++ const prev = await mapRow('live', 'donation_goals', String(g.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const creator = subjectOf(g.user_id);
-- if (!creator) { record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }
++ if (!creator) { await record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }
const target = Math.round(Number(g.target_amount) || 0);
-- if (target <= 0 || !g.title) { record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }
++ if (target <= 0 || !g.title) { await record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }
const created = liveTs(g.created_at) || at;
const id = prefixedId('tgoal', Date.parse(created));
-- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)
-- VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, String(g.title).slice(0, 120), target,
-- /^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null, g.is_active ? 'active' : 'closed', Number(g.sort_order) || 0,
-- liveTs(g.reached_at), g.is_active ? null : at, Math.max(0, Math.round(Number(g.current_amount) || 0)), `live:donation_goals:${g.id}`, created, at);
-- record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });
++ await q.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)
++ VALUES (${id}, ${creator}, ${storable(String(g.title).slice(0, 120))}, ${target}, ${/^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null},
++ ${g.is_active ? 'active' : 'closed'}, ${Number(g.sort_order) || 0}, ${liveTs(g.reached_at)}, ${g.is_active ? null : at},
++ ${Math.max(0, Math.round(Number(g.current_amount) || 0))}, ${`live:donation_goals:${g.id}`}, ${created}, ${at})`);
++ await record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });
report.counts.goals++;
}
-- if (dryRun) { report.reconciliation = reconcile(domain, billingSource); throw new DryRun(); }
++ if (dryRun) { report.reconciliation = await reconcile(domain, billingSource, q); throw new DryRun(); }
};
-- try { db.transaction(run)(); } catch (e) { if (!(e instanceof DryRun)) throw e; }
-- if (!dryRun) report.reconciliation = reconcile(domain, billingSource);
++ // Serializable while the service writes too: a conflict retries the whole run (idempotent), up to 10 times.
++ try { await domain.tx(run, { ...domain.MONEY, retries: 10 }); } catch (e) { if (!(e instanceof DryRun)) throw e; }
++ if (!dryRun) report.reconciliation = await reconcile(domain, billingSource);
report.finished_at = iso(domain.now());
-- db.prepare('INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report) VALUES (?, ?, ?, ?, ?, ?)').run(runId, 'live', dryRun ? 1 : 0, report.started_at, report.finished_at, JSON.stringify(report));
++ await domain.db.exec(sql`INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report)
++ VALUES (${runId}, 'live', ${!!dryRun}, ${report.started_at}, ${report.finished_at}, ${sql.json(report)})`);
if (log && log.log) log.log(`[Tips] import ${runId}${dryRun ? ' (dry run, rolled back)' : ''}: ${report.counts.imported} tips, ${report.counts.external} external, ${report.counts.goals} goals, ${report.counts.held} held, ${report.counts.excluded} excluded; reconciliation ${report.reconciliation ? (report.reconciliation.ok ? 'OK' : 'MISMATCH') : 'skipped (no Billing source)'}`);
return report;
}
@@@ -277,11 +299,11 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
function pick(row) { return { status: row.status, target_type: row.target_type, target_id: row.target_id, reason: row.reason }; }
/** Per-creator totals: Tips (settled via Billing or imported, non-test, minus refunds) vs Billing. */
--function reconcile(domain, billingSource) {
++async function reconcile(domain, billingSource, q = domain.db) {
if (!billingSource) return null;
const billing = billingSource.creatorTotals();
-- const tips = new Map(domain.db.prepare(`SELECT creator_subject AS s, SUM(amount - reversed_bits) AS n FROM tip_interactions
-- WHERE test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`).all().map((r) => [r.s, r.n]));
++ const tips = new Map((await q.many(sql`SELECT creator_subject AS s, SUM(amount - reversed_bits)::bigint AS n FROM tip_interactions
++ WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`)).map((r) => [r.s, r.n]));
const creators = [...new Set([...billing.keys(), ...tips.keys()])].sort();
const rows = creators.map((s) => ({ creator: s, tips: tips.get(s) || 0, billing: billing.get(s) || 0, difference: (tips.get(s) || 0) - (billing.get(s) || 0) }));
const mismatches = rows.filter((r) => r.difference !== 0);
ddiff --git a/server/index.js b/server/index.js
iindex 5312f60..f02d369 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -4,44 +4,73 @@
* OpenVibe.Tips — process entry. `node server/index.js`
* Listens on PORT (4610) behind nginx (openvibe.tips); see deploy/.
*
-- * Background jobs (TIPS_JOBS=off disables them): the effects worker (chat delivery with retries),
-- * due Billing transfers (funded checkouts, retries after Billing was unreachable), the overlay
-- * delivery window (pending → failed), stored API answers older than a week, and the events outbox
-- * relay when EVENTS_URL is set.
++ * Boot (ADR-035): apply migrations/ with the owner role (DATABASE_DIRECT_URL, one direct connection,
++ * closed afterwards), then serve on the pooled runtime role (DATABASE_URL, through PgBouncer) with
++ * Valkey (VALKEY_URL) for what the processes share. Several processes may start together: the
++ * migration run is serialised by an advisory lock.
++ *
++ * Background jobs (TIPS_JOBS=off disables them), in every process: the effects worker (chat delivery
++ * with retries), due Billing transfers (funded checkouts, retries after Billing was unreachable), the
++ * overlay delivery window (pending → failed), stored API answers and sent events older than a week,
++ * and the events outbox relay when EVENTS_URL is set. Running them in several processes is safe: the
++ * effects, the transfers, the sweep and the relay claim their rows (FOR UPDATE SKIP LOCKED, with a
++ * lease), and the prunes are idempotent deletes.
*/
const { loadConfig } = require('./config');
++const { openDb, migrate } = require('./db');
const { createApp } = require('./app');
++const { createValkey } = require('openvibe-sdk/valkey');
++const { createRegistry } = require('openvibe-shared/metrics');
--const config = loadConfig();
--const app = createApp({ config });
--const { domain, keys, outbox } = app.locals;
--keys.start();
++async function main() {
++ const config = loadConfig();
++ const registry = createRegistry();
++ const db = openDb(config, { registry });
++ const m = await migrate(config, { serving: db });
++ if (m.held.length) console.warn(`[Tips] migrations held: ${m.held.map((h) => `${h.id} (${h.reason})`).join('; ')}`);
++ const valkey = createValkey({ url: config.valkey.url, prefix: config.valkey.prefix });
--const timers = [];
--if (config.jobs.enabled) {
-- const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };
-- every(config.jobs.intervalMs, () => domain.effects.drain());
-- every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());
-- every(30_000, () => domain.overlays.sweepFailed());
-- every(6 * 3600 * 1000, () => outbox.outbox.prune());
-- every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(domain.db));
-- outbox.start();
--}
++ const app = createApp({ config, db, valkey, registry });
++ const { domain, keys, outbox } = app.locals;
++ keys.start();
--const server = app.listen(config.port, config.host, () => {
-- console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (db ${config.dbPath})`);
-- console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);
--});
--server.keepAliveTimeout = 65_000;
++ const timers = [];
++ if (config.jobs.enabled) {
++ const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };
++ every(config.jobs.intervalMs, () => domain.effects.drain());
++ every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());
++ every(30_000, () => domain.overlays.sweepFailed());
++ every(6 * 3600 * 1000, () => outbox.outbox.prune());
++ every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(db));
++ outbox.start();
++ }
--function shutdown(signal) {
-- console.log(`[Tips] ${signal} — closing`);
-- timers.forEach(clearInterval);
-- domain.overlays.closeAll();
-- outbox.stop();
-- keys.stop();
-- server.close(() => { try { domain.db.close(); } catch { /* */ } process.exit(0); });
-- setTimeout(() => process.exit(0), 5000).unref();
++ const server = app.listen(config.port, config.host, () => {
++ console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (store ${db.store}; valkey ${valkey ? 'on' : 'off: per process'})`);
++ console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);
++ });
++ server.keepAliveTimeout = 65_000;
++
++ let closing = false;
++ async function shutdown(signal) {
++ if (closing) return;
++ closing = true;
++ console.log(`[Tips] ${signal} — closing`);
++ setTimeout(() => process.exit(0), 5000).unref();
++ timers.forEach(clearInterval);
++ keys.stop();
++ await domain.overlays.close().catch(() => {});
++ await outbox.stop();
++ await new Promise((resolve) => server.close(resolve));
++ await db.close().catch(() => {});
++ if (valkey) await valkey.close().catch(() => {});
++ process.exit(0);
++ }
++ process.on('SIGTERM', () => { shutdown('SIGTERM'); });
++ process.on('SIGINT', () => { shutdown('SIGINT'); });
}
--process.on('SIGTERM', () => shutdown('SIGTERM'));
--process.on('SIGINT', () => shutdown('SIGINT'));
++
++main().catch((e) => {
++ console.error(`[Tips] could not start: ${e.message}`);
++ process.exit(1);
++});
ddiff --git a/server/observability.js b/server/observability.js
iindex f1f3330..7c3a593 100644
---- a/server/observability.js
++++ b/server/observability.js
@@@ -3,7 +3,12 @@
* Track O: truthful readiness for GET /api/ready and the Tips gauges on GET /metrics
* (openvibe-shared/ready and openvibe-shared/metrics).
*
-- * db required a real query on Tips' SQLite (the settings row is there and answers)
++ * db required a real round trip to PostgreSQL through the pool (db.ready(): the store
++ * that answered — postgresql, or pglite in development and tests — and the
++ * pool's state)
++ * valkey optional a real PING (valkey.ready()). Without it (VALKEY_URL unset: skipped, or
++ * down: degraded) per-actor limits, overlay fan-out and stream slots are
++ * counted in each process, which is right for one process only
* network_jwks optional the Network signing key has loaded. Without it pages, overlays and the
* signed Billing webhook still work, but no service or user token can be
* verified (API calls and sign-in fail), so it degrades rather than fails
@@@ -13,12 +18,16 @@
* EVENTS_URL or the client secret unset) tips.* events wait in the outbox
*
* Gauges: pending deliveries (chat/TTS/media effects waiting for their adapter, overlay alerts no
-- * overlay has shown yet) and the events outbox backlog. Counts only, never subjects.
++ * overlay has shown yet) and the events outbox backlog. Counts only, never subjects. They are read in
++ * one query just before a direct scrape renders (refresh()); a scrape whose read failed leaves them
++ * out rather than reporting stale or invented values.
*/
--const { createReadiness } = require('openvibe-shared/ready');
++const { sql } = require('openvibe-sdk/db');
++const { createReadiness, skip } = require('openvibe-shared/ready');
++const { TABLE: OUTBOX } = require('./events/outbox');
const PING_TTL_MS = 15_000;
--// interaction_effects.effect (the CHECK constraint in db.js): each reported, 0 when none waits.
++// interaction_effects.effect (the CHECK constraint in migrations/0001_initial.sql): each reported, 0 when none waits.
const EFFECTS = ['chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert'];
function probe(url, fetchImpl) {
@@@ -29,7 +38,7 @@ function probe(url, fetchImpl) {
};
}
--function createTipsReadiness({ db, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {
++function createTipsReadiness({ db, valkey = null, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {
const events = outbox.enabled
? probe(`${config.events.url}/api/health`, fetchImpl)
: () => 'events relay off (EVENTS_URL or OV_OAUTH_CLIENT_SECRET unset): tips.* events wait in the outbox';
@@@ -37,45 +46,56 @@ function createTipsReadiness({ db, keys, config, outbox, release = null, fetchIm
service: 'tips',
release,
checks: [
-- { name: 'db', required: true, check: () => (db.prepare('SELECT id FROM settings WHERE id = 1').get() ? true : 'database has no settings row') },
++ { name: 'db', required: true, check: () => db.ready() },
++ { name: 'valkey', required: false, check: () => (valkey ? valkey.ready() : skip('VALKEY_URL unset: limits, overlay fan-out and stream slots are per process (one process only)')) },
{ name: 'network_jwks', required: false, check: () => (keys.get() ? true : 'Network signing key not loaded yet: tokens cannot be verified') },
{ name: 'billing', required: false, cacheMs: PING_TTL_MS, timeoutMs: 2500, check: probe(`${config.billing.url}/api/health`, fetchImpl) },
{ name: 'events', required: false, cacheMs: outbox.enabled ? PING_TTL_MS : 0, timeoutMs: 2500, check: events },
],
-- details: (body) => ({
-- events_outbox: body.checks.db.status === 'ok' ? outbox.status() : null,
-- billing_events_accepted: config.events.webhookSecrets.length > 0,
-- pending_deliveries: body.checks.db.status === 'ok' ? pendingDeliveries(db) : null,
-- }),
++ details: async (body) => (body.checks.db.status === 'ok'
++ ? { events_outbox: await outbox.status(), billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: await pendingDeliveries(db) }
++ : { events_outbox: null, billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: null }),
});
}
--function pendingDeliveries(db) {
-- const effects = {};
-- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) effects[r.effect] = r.n;
-- const overlay = db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n;
-- return { effects, overlay };
++async function pendingDeliveries(db) {
++ const r = await db.one(sql`SELECT
++ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,
++ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay`);
++ return { effects: r.effects, overlay: r.overlay };
}
--/** Tips gauges on the openvibe-shared/metrics registry. */
--function registerTipsGauges(registry, { db, outbox, now = () => Date.now() }) {
++/** Tips gauges on the openvibe-shared/metrics registry; refresh() reads them all in one query. */
++function registerTipsGauges(registry, { db, now = () => Date.now(), log = console }) {
++ let snap = null;
++ const read = (f) => () => (snap ? f(snap) : undefined);
registry.gauge({
name: 'tips_effects_pending', help: 'Chat, TTS, media and alert effects waiting for their delivery adapter, by effect', labelNames: ['effect'],
-- collect: () => {
-- const n = Object.fromEntries(EFFECTS.map((e) => [e, 0]));
-- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) n[r.effect] = r.n;
-- return Object.entries(n).map(([effect, value]) => ({ labels: { effect }, value }));
-- },
++ collect: read((s) => EFFECTS.map((effect) => ({ labels: { effect }, value: Number(s.effects[effect] || 0) }))),
});
registry.gauge({
name: 'tips_effects_due', help: 'Queued effects whose next attempt is due now (a growing number means the worker is behind)',
-- collect: () => db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ?").get(now()).n,
-- });
-- registry.gauge({
-- name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet',
-- collect: () => db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n,
++ collect: read((s) => s.due),
});
-- registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: () => outbox.status().pending });
++ registry.gauge({ name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet', collect: read((s) => s.overlay) });
++ registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: read((s) => s.outbox) });
++ return {
++ async refresh() {
++ try {
++ snap = await Promise.race([
++ db.one(sql`SELECT
++ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,
++ (SELECT count(*) FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now()}) AS due,
++ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay,
++ (SELECT count(*) FROM ${sql.ident(OUTBOX)} WHERE sent_at IS NULL AND rejected_at IS NULL) AS outbox`),
++ new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000).unref()),
++ ]);
++ } catch (e) {
++ snap = null;
++ log.warn(`[Tips] gauges not read: ${e.message}`);
++ }
++ },
++ };
}
module.exports = { createTipsReadiness, registerTipsGauges };
ddiff --git a/server/util.js b/server/util.js
iindex c018b58..c120385 100644
---- a/server/util.js
++++ b/server/util.js
@@@ -19,6 +19,7 @@ function fail(status, code, detail, extra) { throw new TipsError(status, code, d
const prefixedId = (prefix, ms = Date.now()) => `${prefix}_${ids.ulid(ms)}`;
const iso = (ms) => new Date(ms).toISOString();
const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');
++/** JSON text (a SQLite snapshot's column) → value, or `d`. PostgreSQL jsonb columns come back parsed. */
const json = (v, d) => { if (v == null || v === '') return d; try { return JSON.parse(v); } catch { return d; } };
function positiveInt(v, field, max) {
@@@ -32,10 +33,16 @@ function positiveInt(v, field, max) {
// from the creator's filter and turn text around on stream (docs/threat-review.md).
const INVISIBLE = /[\u00AD\u200B-\u200F\u202A-\u202E\u2060-\u2064\u2066-\u2069\uFEFF]/g;
--/** Optional trimmed text: null when empty, refused when longer than max. Invisible characters are dropped. */
++// PostgreSQL stores no NUL character in text or jsonb, and jsonb refuses an unpaired surrogate (SQLite
++// took both): every text from outside goes through storable() before it reaches the database.
++const NUL = /\u0000/g;
++/** A string PostgreSQL can store: NUL characters dropped, unpaired surrogates replaced by U+FFFD. */
++const storable = (v) => String(v).toWellFormed().replace(NUL, '');
++
++/** Optional trimmed text: null when empty, refused when longer than max. Invisible and NUL characters are dropped. */
function text(v, field, max) {
if (v == null) return null;
-- const s = String(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();
++ const s = storable(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();
if (!s) return null;
if (s.length > max) fail(422, 'tips.text_too_long', `${field} must be at most ${max} characters`);
return s;
@@@ -53,14 +60,36 @@ function userSubject(v, field = 'subject') {
function entityRef(v, field = 'target') {
if (v == null) return null;
if (!validate('common.entity-ref@1', v).valid) fail(422, 'tips.invalid_input', `${field} must be an EntityRef ({ service, type, id })`);
-- return { service: v.service, type: v.type, id: String(v.id) };
++ return { service: v.service, type: v.type, id: storable(v.id) };
}
/** The display name a supporter chose, kept short, without markup characters. */
function displayName(v) {
if (v == null) return null;
-- const s = String(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80);
++ const s = storable(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80).toWellFormed();
return s || null;
}
--module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE };
++/** A keyset cursor: base64url JSON [created_at ISO, id] → the pair, or 422; cursorOf(row) makes one. */
++const ISO_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,6})?Z$/;
++function readCursor(cursor) {
++ let cur = null;
++ try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { /* refused below */ }
++ if (!Array.isArray(cur) || typeof cur[0] !== 'string' || !ISO_RE.test(cur[0]) || typeof cur[1] !== 'string' || cur[1].length > 64) fail(422, 'tips.invalid_input', 'bad cursor');
++ return cur;
++}
++const cursorOf = (row) => Buffer.from(JSON.stringify([row.created_at, row.id])).toString('base64url');
++
++/**
++ * A database refusal the caller caused (text PostgreSQL cannot store: a NUL character, a malformed
++ * sequence) → the TipsError to answer with, else null (a real failure: 500).
++ */
++function inputError(e) {
++ const code = e && (e.code || (e.cause && e.cause.code));
++ // 22021/22P05: a character the encoding or jsonb refuses; 22P02, 22007, 22008: a value that is not
++ // what its column takes (an unreadable number, JSON text or time).
++ return ['22021', '22P05', '22P02', '22007', '22008'].includes(code)
++ ? new TipsError(422, 'tips.invalid_input', 'the request carries a value that cannot be stored (such as a NUL character)') : null;
++}
++
++module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, storable, userSubject, entityRef, displayName, inputError, readCursor, cursorOf, INVISIBLE };
ddiff --git a/server/web/async-router.js b/server/web/async-router.js
nnew file mode 100644
iindex 0000000..943565b
---- /dev/null
++++ b/server/web/async-router.js
@@@ -0,0 +1,31 @@
++'use strict';
++
++/**
++ * Express 4 ignores a rejected promise, so an async handler's error would never reach the error
++ * handler (the request would hang). A router made here wraps every handler it is given: a returned
++ * promise that rejects is passed to next(err). Error handlers (four arguments) are left as they are.
++ *
++ * const r = asyncRouter(express.Router());
++ * r.get('/x', async (req, res) => { res.json(await something()); });
++ */
++const METHODS = ['all', 'get', 'post', 'put', 'patch', 'delete', 'head', 'options', 'use'];
++
++function catching(fn) {
++ if (typeof fn !== 'function' || fn.length >= 4) return fn;
++ return function asyncHandler(req, res, next) {
++ let out;
++ try { out = fn(req, res, next); } catch (e) { return next(e); }
++ if (out && typeof out.then === 'function') out.then(undefined, next);
++ return undefined;
++ };
++}
++
++function asyncRouter(router) {
++ for (const m of METHODS) {
++ const orig = router[m].bind(router);
++ router[m] = (...args) => orig(...args.map((a) => (Array.isArray(a) ? a.map(catching) : catching(a))));
++ }
++ return router;
++}
++
++module.exports = { asyncRouter, catching };
ddiff --git a/server/web/routes.js b/server/web/routes.js
iindex 647ab32..d27a6ac 100644
---- a/server/web/routes.js
++++ b/server/web/routes.js
@@@ -25,8 +25,10 @@ const crypto = require('crypto');
const ovServe = require('openvibe-shared/serve');
const frame = require('openvibe-shared/frame');
const express = require('express');
--const { TipsError, json: parseJson } = require('../util');
++const { sql } = require('openvibe-sdk/db');
++const { TipsError } = require('../util');
const { viewerMiddleware } = require('./session');
++const { asyncRouter } = require('./async-router');
const pages = require('./pages');
const { esc, asset } = require('./layout');
const { PAGE_DEFAULTS } = require('../domain/profiles');
@@@ -37,7 +39,8 @@ const MOD_FLASH = new Set(['Hidden', 'Shown', 'You are now a moderator']);
const RECEIPT_FLASH = new Set(['Your data was erased from your tips']);
function createWebRoutes({ domain, config, layout, userAuth }) {
-- const r = express.Router();
++ // Every handler below may be async: its errors reach the app's error handler (the 500 page).
++ const r = asyncRouter(express.Router());
const { profiles, goals, interactions, overlays, moderation, db } = domain;
const withViewer = viewerMiddleware(userAuth);
const form = express.urlencoded({ extended: false, limit: '32kb' });
@@@ -64,62 +67,66 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
const bool = (v) => v === '1' || v === 'on' || v === true;
// ── Home, robots, sitemap ────────────────────────────────
-- r.get('/', withViewer, (req, res) => {
-- const creators = db.prepare(`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY updated_at DESC LIMIT 24`).all();
++ r.get('/', withViewer, async (req, res) => {
++ const creators = await db.many(sql`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`);
html(res, pageFor(req, { active: 'home', canonicalPath: '/', body: pages.home({ creators }) }));
});
// What shipped on OpenVibe.Tips: the shared update log every OpenVibe site has.
r.get('/updates', withViewer, (req, res) => html(res, pageFor(req, { canonicalPath: '/updates', title: 'What shipped on OpenVibe.Tips', body: frame.updatesBody({ service: 'tips', siteName: 'OpenVibe.Tips' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` })));
r.get('/robots.txt', (req, res) => res.type('text/plain').set('Cache-Control', 'public, max-age=3600').send(
`User-agent: *\nAllow: /\nDisallow: /dashboard\nDisallow: /receipts\nDisallow: /overlay/\nDisallow: /moderate/\nDisallow: /auth/\nDisallow: /api/\nDisallow: /internal/\nSitemap: ${config.baseUrl}/sitemap.xml\n`));
-- r.get('/sitemap.xml', (req, res) => {
-- const rows = db.prepare('SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY handle').all();
++ r.get('/sitemap.xml', async (req, res) => {
++ const rows = await db.many(sql`SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled ORDER BY handle`);
const urls = [`<url><loc>${esc(config.baseUrl)}/</loc></url>`, ...rows.map((p) => `<url><loc>${esc(`${config.baseUrl}/${p.handle}`)}</loc><lastmod>${esc(p.updated_at.slice(0, 10))}</lastmod></url>`)];
res.type('application/xml').set('Cache-Control', 'public, max-age=900').send(`<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls.join('')}</urlset>\n`);
});
// ── Receipts ─────────────────────────────────────────────
-- const decorate = (i) => {
-- const p = profiles.bySubject(i.creator_subject);
-- return { ...interactions.present(i, { viewer: 'supporter' }), creator_handle: p && p.handle, creator_name: p && p.display_name };
-- };
-- r.get('/receipts', withViewer, (req, res) => {
++ /** The supporter's view of each receipt, with its creator's handle and name (one query for all creators). */
++ async function decorate(rows) {
++ const [views, byCreator] = await Promise.all([
++ interactions.presentMany(db, rows, { viewer: 'supporter' }),
++ profiles.bySubjects(db, rows.map((i) => i.creator_subject)),
++ ]);
++ return views.map((v, k) => { const p = byCreator.get(rows[k].creator_subject); return { ...v, creator_handle: p && p.handle, creator_name: p && p.display_name }; });
++ }
++ r.get('/receipts', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
-- const out = interactions.list({ supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });
++ const out = await interactions.list(db, { supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });
const flash = RECEIPT_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
-- html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: out.rows.map(decorate), next: out.next_cursor, flash }) }));
++ return html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: await decorate(out.rows), next: out.next_cursor, flash }) }));
});
// The supporter's own data: a download, and erasure (privacy.js, interactions.erase()).
-- r.get('/receipts/export', withViewer, (req, res) => {
++ r.get('/receipts/export', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
-- const day = new Date(domain.now()).toISOString().slice(0, 10);
-- res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${day}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })
-- .type('application/json').send(`${JSON.stringify(interactions.exportFor(req.viewer.subject), null, 2)}\n`);
++ const today = new Date(domain.now()).toISOString().slice(0, 10);
++ return res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${today}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })
++ .type('application/json').send(`${JSON.stringify(await interactions.exportFor(req.viewer.subject), null, 2)}\n`);
});
-- function renderErase(req, res, status = 200) {
-- const rows = db.prepare("SELECT payment_state FROM tip_interactions WHERE supporter_subject = ?").all(req.viewer.subject);
++ async function renderErase(req, res, status = 200) {
++ const c = await db.one(sql`SELECT count(*) AS n, count(*) FILTER (WHERE payment_state = 'pending') AS pending FROM tip_interactions WHERE supporter_subject = ${req.viewer.subject}`);
html(res, pageFor(req, {
title: 'Erase your data', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts/erase',
-- body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: rows.length, pending: rows.filter((x) => x.payment_state === 'pending').length }),
++ body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: c.n, pending: c.pending }),
}), status);
}
-- r.get('/receipts/erase', withViewer, (req, res) => {
++ r.get('/receipts/erase', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
-- renderErase(req, res);
++ return renderErase(req, res);
});
-- r.post('/receipts/erase', withViewer, form, (req, res) => {
++ r.post('/receipts/erase', withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
if (!csrfOk(req) || !bool(req.body.confirm)) return renderErase(req, res, 403);
-- interactions.erase(req.viewer.subject);
++ await interactions.erase(req.viewer.subject);
return res.redirect(303, `/receipts?done=${encodeURIComponent('Your data was erased from your tips')}`);
});
-- r.get('/receipts/:id', withViewer, (req, res) => {
++ r.get('/receipts/:id', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
-- const i = interactions.get(req.params.id);
++ const i = await interactions.get(db, req.params.id);
const as = i && (i.supporter_subject === req.viewer.subject ? 'supporter' : i.creator_subject === req.viewer.subject ? 'creator' : null);
if (!i || !as) return notFound(req, res);
-- const view = interactions.present(i, { viewer: as === 'creator' ? 'owner' : 'supporter' });
-- html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile: profiles.bySubject(i.creator_subject), as, cancelled: req.query.cancelled === '1' }) }));
++ const [view, profile] = await Promise.all([interactions.present(db, i, { viewer: as === 'creator' ? 'owner' : 'supporter' }), profiles.bySubject(db, i.creator_subject)]);
++ return html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile, as, cancelled: req.query.cancelled === '1' }) }));
});
// ── Dashboard ────────────────────────────────────────────
@@@ -128,42 +135,45 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
const claims = req.viewerClaims || {};
return profiles.ensure(v.subject, { username: v.username, displayName: v.name, avatarUrl: claims.avatar_url });
}
-- function ensureAlertsConfig(subject) {
-- if (!overlays.listConfigs(subject).some((c) => c.kind === 'alerts')) overlays.createConfig(subject, { kind: 'alerts', name: 'Alerts' });
-- }
-- function renderDashboard(req, res, { flash, error, status = 200 } = {}) {
-- const p = ensureProfile(req);
-- ensureAlertsConfig(p.creator_subject);
-- const recent = interactions.list({ creator: p.creator_subject, limit: 20 }).rows.map((i) => interactions.present(i, { viewer: 'owner' }));
++ async function renderDashboard(req, res, { flash, error, status = 200 } = {}) {
++ const p = await ensureProfile(req);
++ let configs = await overlays.listConfigs(db, p.creator_subject);
++ if (!configs.some((c) => c.kind === 'alerts')) {
++ await overlays.createConfig(p.creator_subject, { kind: 'alerts', name: 'Alerts' });
++ configs = await overlays.listConfigs(db, p.creator_subject);
++ }
++ const s = p.creator_subject;
++ const [recentRows, goalRows, tokens, totals, deliveries, connected, held, moderators, invites, log, moderating] = await Promise.all([
++ interactions.list(db, { creator: s, limit: 20 }).then((out) => interactions.presentMany(db, out.rows, { viewer: 'owner' })),
++ goals.list(db, s).then((list) => goals.presentMany(db, list)),
++ overlays.listTokens(db, s), interactions.totals(db, s), overlays.recentDeliveries(db, s, 20), overlays.connected(s),
++ moderation.heldCount(db, s), moderation.listModerators(db, s), moderation.openInvites(db, s), moderation.log(db, s, 20), moderation.moderatedBy(db, req.viewer.subject),
++ ]);
html(res, pageFor(req, {
title: 'Dashboard', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard',
body: pages.dashboard({
-- profile: p, goals: goals.list(p.creator_subject).map((g) => goals.present(g)), tokens: overlays.listTokens(p.creator_subject).map(overlays.presentToken),
-- configs: overlays.listConfigs(p.creator_subject).map(overlays.presentConfig), totals: interactions.totals(p.creator_subject), recent,
-- deliveries: overlays.recentDeliveries(p.creator_subject, 20), csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,
-- connected: overlays.connected(p.creator_subject), chatAdapter: config.chat.adapter,
-- mod: {
-- held: db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND moderation = 'held' AND payment_state = 'settled'").get(p.creator_subject).n,
-- moderators: moderation.listModerators(p.creator_subject).map(moderation.presentModerator), invites: moderation.openInvites(p.creator_subject),
-- log: moderation.log(p.creator_subject, 20), moderating: moderation.moderatedBy(req.viewer.subject),
-- },
++ profile: p, goals: goalRows, tokens: tokens.map(overlays.presentToken),
++ configs: configs.map(overlays.presentConfig), totals, recent: recentRows,
++ deliveries, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,
++ connected, chatAdapter: config.chat.adapter,
++ mod: { held, moderators: moderators.map(moderation.presentModerator), invites, log, moderating },
}),
}), status);
}
-- r.get('/dashboard', withViewer, (req, res) => {
++ r.get('/dashboard', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
// Only our own confirmations are shown: a crafted ?done= link cannot put words on the page.
-- renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });
++ return renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });
});
/** A dashboard form action: sign-in + anti-forgery, errors re-render the dashboard. */
function action(path, fn) {
-- r.post(path, withViewer, form, (req, res) => {
++ r.post(path, withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
if (!csrfOk(req)) return renderDashboard(req, res, { error: 'That form expired. Please try again.', status: 403 });
try {
-- const p = ensureProfile(req);
-- const out = fn(req, p);
++ const p = await ensureProfile(req);
++ const out = await fn(req, p);
if (out && out.render) return out.render(res);
return res.redirect(303, `/dashboard?done=${encodeURIComponent((out && out.done) || 'Saved')}`);
} catch (e) {
@@@ -172,9 +182,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}
});
}
-- action('/dashboard/profile', (req, p) => {
++ action('/dashboard/profile', async (req, p) => {
const b = req.body;
-- profiles.update(p.creator_subject, {
++ await profiles.update(p.creator_subject, {
page_enabled: bool(b.page_enabled), accepting: bool(b.accepting), tts_enabled: bool(b.tts_enabled), media_requests_enabled: bool(b.media_requests_enabled),
headline: b.headline, min_amount: b.min_amount, paid_message_min: b.paid_message_min, tts_min_amount: b.tts_min_amount, tts_max_chars: b.tts_max_chars,
tts_voice: b.tts_voice, media_request_min: b.media_request_min, media_max_seconds: b.media_max_seconds,
@@@ -182,46 +192,46 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}, { expectedRevision: b.revision });
return { done: 'Page settings saved' };
});
-- action('/dashboard/goals', (req, p) => { goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });
-- const ownGoal = (req, p) => {
-- const g = goals.get(req.params.id);
++ action('/dashboard/goals', async (req, p) => { await goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });
++ const ownGoal = async (req, p) => {
++ const g = await goals.get(db, req.params.id);
if (!g || g.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.goal_not_found', 'no such goal');
return g;
};
-- action('/dashboard/goals/:id', (req, p) => { goals.update(ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });
-- action('/dashboard/goals/:id/close', (req, p) => { goals.close(ownGoal(req, p)); return { done: 'Goal closed' }; });
-- action('/dashboard/overlay-tokens', (req, p) => {
++ action('/dashboard/goals/:id', async (req, p) => { await goals.update(await ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });
++ action('/dashboard/goals/:id/close', async (req, p) => { await goals.close(await ownGoal(req, p)); return { done: 'Goal closed' }; });
++ action('/dashboard/overlay-tokens', async (req, p) => {
const scopes = [bool(req.body.scope_alerts) && 'alerts', bool(req.body.scope_goals) && 'goals'].filter(Boolean);
-- const alertCfg = overlays.listConfigs(p.creator_subject).find((c) => c.kind === 'alerts');
-- const out = overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });
++ const alertCfg = (await overlays.listConfigs(db, p.creator_subject)).find((c) => c.kind === 'alerts');
++ const out = await overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });
return { render: (res) => html(res, pageFor(req, { title: 'Overlay link', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.tokenCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
});
-- action('/dashboard/overlay-tokens/:id/revoke', (req, p) => {
-- const t = overlays.getToken(req.params.id);
++ action('/dashboard/overlay-tokens/:id/revoke', async (req, p) => {
++ const t = await overlays.getToken(db, req.params.id);
if (!t || t.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_token_not_found', 'no such overlay token');
-- overlays.revokeToken(t);
++ await overlays.revokeToken(t);
return { done: 'Overlay link revoked' };
});
-- action('/dashboard/overlay-configs/:id', (req, p) => {
-- const c = overlays.getConfig(req.params.id);
++ action('/dashboard/overlay-configs/:id', async (req, p) => {
++ const c = await overlays.getConfig(db, req.params.id);
if (!c || c.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_config_not_found', 'no such overlay config');
const b = req.body;
-- overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });
++ await overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });
return { done: 'Alert settings saved' };
});
-- action('/dashboard/filter', (req, p) => {
-- profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });
++ action('/dashboard/filter', async (req, p) => {
++ await profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });
return { done: 'Filter saved' };
});
-- action('/dashboard/moderators/:subject/remove', (req, p) => { moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });
-- action('/dashboard/moderator-invites', (req, p) => {
-- const out = moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });
++ action('/dashboard/moderators/:subject/remove', async (req, p) => { await moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });
++ action('/dashboard/moderator-invites', async (req, p) => {
++ const out = await moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });
return { render: (res) => html(res, pageFor(req, { title: 'Moderator invitation', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.inviteCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
});
-- action('/dashboard/moderator-invites/:id/revoke', (req, p) => { moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });
-- action('/dashboard/simulate', (req, p) => {
++ action('/dashboard/moderator-invites/:id/revoke', async (req, p) => { await moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });
++ action('/dashboard/simulate', async (req, p) => {
const b = req.body;
-- interactions.simulate(p, {
++ await interactions.simulate(p, {
kind: b.kind, amount: b.amount, message: b.message, supporter_name: b.supporter_name,
tts: b.kind === 'tts' ? { text: b.message } : undefined, media: b.kind === 'media_request' ? { url: b.media_url } : undefined,
}, { by: req.viewer.subject });
@@@ -230,13 +240,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
// ── Overlays (token in the path; no cookies involved) ────
const overlayHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
-- function tokenOr404(req, res) {
-- const t = overlays.authenticate(req.params.token);
++ async function tokenOr404(req, res) {
++ const t = await overlays.authenticate(db, req.params.token);
if (!t) { res.status(404).set(overlayHeaders).type('text/plain').send('This overlay link is not valid (it may have been revoked).'); return null; }
return t;
}
-- r.get('/overlay/:token', (req, res) => {
-- const t = tokenOr404(req, res);
++ r.get('/overlay/:token', async (req, res) => {
++ const t = await tokenOr404(req, res);
if (!t) return;
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/html; charset=utf-8' }).send(`<!DOCTYPE html>
<html lang="en"><head><meta charset="utf-8"><meta name="robots" content="noindex,nofollow"><meta name="referrer" content="no-referrer">
@@@ -246,59 +256,59 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
<noscript><p class="nojs">This overlay needs JavaScript (OBS Browser Sources have it on).</p></noscript>
<script src="${asset('js/overlay.js')}" defer></script></body></html>`);
});
-- r.get('/overlay/:token/events', (req, res) => {
-- const t = tokenOr404(req, res);
++ r.get('/overlay/:token/events', async (req, res) => {
++ const t = await tokenOr404(req, res);
if (!t) return;
-- if (overlays.streamsFor(t) >= config.overlays.maxStreamsPerToken) {
++ // A bounded number of open streams per token, counted across every process (overlay-hub.js).
++ const slot = await overlays.reserveStream(t);
++ if (!slot) {
res.status(429).set({ ...overlayHeaders, 'Retry-After': '30' }).type('text/plain').send('Too many open streams for this overlay link.');
return;
}
++ let overlayConfig = null;
++ try { overlayConfig = t.config_id ? await overlays.getConfig(db, t.config_id) : null; } catch (e) { overlays.releaseStream(slot); throw e; }
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/event-stream; charset=utf-8', Connection: 'keep-alive', 'X-Accel-Buffering': 'no' });
res.flushHeaders();
res.write('retry: 3000\n\n');
-- const overlayConfig = t.config_id ? overlays.getConfig(t.config_id) : null;
const lastEventId = req.get('last-event-id') != null ? req.get('last-event-id') : req.query.last_event_id;
-- overlays.attach(t, res, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });
++ overlays.attach(t, res, slot, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });
});
-- r.get('/overlay/:token/state', (req, res) => {
-- const t = tokenOr404(req, res);
++ r.get('/overlay/:token/state', async (req, res) => {
++ const t = await tokenOr404(req, res);
if (!t) return;
const out = { creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes };
-- if (t.scopes.includes('goals')) out.goals = goals.list(t.creator_subject, { status: 'active' }).map((g) => goals.present(g));
-- if (t.scopes.includes('alerts')) {
-- out.alerts = db.prepare("SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ? AND kind = 'alert' AND hidden = 0 ORDER BY seq DESC LIMIT 20")
-- .all(t.creator_subject).map((d) => ({ seq: d.seq, ...parseJson(d.payload, {}), test: !!d.test, created_at: d.created_at }));
-- }
++ if (t.scopes.includes('goals')) out.goals = await goals.presentMany(db, await goals.list(db, t.creator_subject, { status: 'active' }));
++ if (t.scopes.includes('alerts')) out.alerts = await overlays.recentAlerts(db, t.creator_subject);
res.set(overlayHeaders).json(out);
});
// ── Moderation (the creator and their moderators) ─────────
// Invitation links carry a secret in the path, like overlay links: no-store, no referrer, noindex.
const inviteHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
-- function inviteOr404(req, res) {
-- const inv = moderation.findInvite(req.params.token);
-- const profile = inv && profiles.bySubject(inv.creator_subject);
++ async function inviteOr404(req, res) {
++ const inv = await moderation.findInvite(db, req.params.token);
++ const profile = inv && await profiles.bySubject(db, inv.creator_subject);
if (!inv || !profile) {
html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 404, title: 'This invitation is not valid', message: 'It may have been used, revoked or expired. Ask the creator for a new one.' }) }), 404, inviteHeaders);
return null;
}
return { inv, profile };
}
-- r.get('/moderate/invite/:token', withViewer, (req, res) => {
++ r.get('/moderate/invite/:token', withViewer, async (req, res) => {
if (!req.viewer) return res.set(inviteHeaders).redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
-- const found = inviteOr404(req, res);
++ const found = await inviteOr404(req, res);
if (!found) return undefined;
return html(res, pageFor(req, {
title: `Moderate ${found.profile.display_name}`, robots: 'noindex,nofollow', canonicalPath: '/',
body: pages.invitePage({ profile: found.profile, csrf: csrfFor(req.viewer.subject), idem: idem(), secret: req.params.token, isSelf: found.profile.creator_subject === req.viewer.subject }),
}), 200, inviteHeaders);
});
-- r.post('/moderate/invite/:token', withViewer, form, (req, res) => {
++ r.post('/moderate/invite/:token', withViewer, form, async (req, res) => {
if (!req.viewer) return res.set(inviteHeaders).redirect(303, '/auth/login');
if (!csrfOk(req)) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 403, title: 'That form expired', message: 'Open the invitation link again.' }) }), 403, inviteHeaders);
try {
-- const out = moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });
-- const profile = profiles.bySubject(out.creator);
++ const out = await moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });
++ const profile = await profiles.bySubject(db, out.creator);
return res.set(inviteHeaders).redirect(303, `/moderate/${encodeURIComponent(profile.handle)}?done=${encodeURIComponent('You are now a moderator')}`);
} catch (e) {
if (e instanceof TipsError) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: e.status, title: 'This invitation is not valid', message: e.detail || e.message }) }), e.status, inviteHeaders);
@@@ -307,36 +317,35 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
});
/** The creator or one of their moderators, for /moderate/:handle; else null (the page is a 404). */
-- function modAccess(req) {
-- const p = profiles.byHandle(req.params.handle);
++ async function modAccess(req) {
++ const p = await profiles.byHandle(db, req.params.handle);
if (!p || !req.viewer) return { p };
-- const actor = moderation.actorFor(req.viewer, p.creator_subject, false);
-- return { p, actor };
++ return { p, actor: await moderation.actorFor(db, req.viewer, p.creator_subject, false) };
}
-- function renderModerate(req, res, p, { error, status = 200 } = {}) {
++ async function renderModerate(req, res, p, { error, status = 200 } = {}) {
const state = ['held', 'hidden', 'visible', 'all'].includes(req.query.state) ? req.query.state : 'all';
-- const out = moderation.queue(p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });
++ const out = await moderation.queue(db, p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });
const flash = MOD_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
html(res, pageFor(req, {
title: `Paid messages — ${p.display_name}`, robots: 'noindex,nofollow', canonicalPath: `/moderate/${p.handle}`,
body: pages.moderatePage({ profile: p, rows: out.rows, state, next: out.next_cursor, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error, isCreator: req.viewer.subject === p.creator_subject }),
}), status);
}
-- r.get('/moderate/:handle', withViewer, (req, res) => {
++ r.get('/moderate/:handle', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
-- const { p, actor } = modAccess(req);
++ const { p, actor } = await modAccess(req);
if (!p || !actor) return notFound(req, res);
return renderModerate(req, res, p);
});
-- r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, (req, res) => {
++ r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/moderate/${req.params.handle}`)}`);
-- const { p, actor } = modAccess(req);
++ const { p, actor } = await modAccess(req);
if (!p || !actor) return notFound(req, res);
if (!csrfOk(req)) return renderModerate(req, res, p, { error: 'That form expired. Please try again.', status: 403 });
-- const i = interactions.get(req.params.id);
++ const i = await interactions.get(db, req.params.id);
if (!i || i.creator_subject !== p.creator_subject) return notFound(req, res);
try {
-- moderation[req.params.act](i, { ...actor, reason: req.body.reason });
++ await moderation[req.params.act](i, { ...actor, reason: req.body.reason });
} catch (e) {
if (e instanceof TipsError) return renderModerate(req, res, p, { error: e.detail || e.message, status: 400 });
throw e;
@@@ -346,13 +355,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
// ── Creator pages (last: /:handle catches everything else) ──
const providers = config.billing.providers;
-- function creatorView(req, res, { values, error, status = 200 } = {}) {
-- const p = profiles.byHandle(req.params.handle);
++ async function creatorView(req, res, { values, error, status = 200 } = {}) {
++ const p = await profiles.byHandle(db, req.params.handle);
const owner = !!(p && req.viewer && req.viewer.subject === p.creator_subject);
if (!p || (!p.page_enabled && !owner)) return notFound(req, res);
if (p.handle !== String(req.params.handle)) return res.redirect(301, `/${p.handle}`); // @name, Name → the canonical path
// The page shows what the public sees, for the creator too (a preview); the dashboard has the rest.
-- const list = goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page));
++ const list = await goals.publicGoals(db, await goals.list(db, p.creator_subject), p);
const csrf = req.viewer ? csrfFor(req.viewer.subject) : '';
const jsonLd = p.page_enabled ? [{ '@context': 'https://schema.org', '@type': 'ProfilePage', name: `${p.display_name} on ${'OpenVibe.Tips'}`, url: `${config.baseUrl}/${p.handle}`, mainEntity: { '@type': 'Person', name: p.display_name, alternateName: p.handle, image: p.avatar_url || undefined } }] : [];
return html(res, pageFor(req, {
@@@ -363,33 +372,34 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}), status, p.page_enabled ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
}
r.get('/:handle', withViewer, (req, res, next) => (profiles.normalizeHandle(req.params.handle) ? creatorView(req, res) : next()));
-- r.get('/:handle/goals', withViewer, (req, res, next) => {
-- const p = profiles.byHandle(req.params.handle);
++ r.get('/:handle/goals', withViewer, async (req, res, next) => {
++ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
const owner = req.viewer && req.viewer.subject === p.creator_subject;
if (!p.page_enabled && !owner) return notFound(req, res);
-- html(res, pageFor(req, {
++ return html(res, pageFor(req, {
title: `${p.display_name} — goals`, canonicalPath: `/${p.handle}/goals`, robots: p.page_enabled ? 'index,follow' : 'noindex,nofollow',
-- body: pages.goalsPage({ profile: p, goals: goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page)) }),
++ body: pages.goalsPage({ profile: p, goals: await goals.publicGoals(db, await goals.list(db, p.creator_subject), p) }),
}));
});
-- r.get('/:handle/supporters', withViewer, (req, res, next) => {
-- const p = profiles.byHandle(req.params.handle);
++ r.get('/:handle/supporters', withViewer, async (req, res, next) => {
++ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
const owner = req.viewer && req.viewer.subject === p.creator_subject;
// Off unless the creator shows it; the creator gets a preview while it is off.
if ((!p.page_enabled || !p.page.supporters_page) && !owner) return notFound(req, res);
-- const leaderboard = interactions.leaderboard(p.creator_subject).map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));
-- const recent = p.page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;
++ const [top, recentRows] = await Promise.all([interactions.leaderboard(db, p), p.page.supporters_messages ? interactions.recentPublic(db, p) : null]);
++ const leaderboard = top.map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));
++ const recent = recentRows ? recentRows.map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;
const indexable = p.page_enabled && p.page.supporters_page;
-- html(res, pageFor(req, {
++ return html(res, pageFor(req, {
title: `${p.display_name} — supporters`, canonicalPath: `/${p.handle}/supporters`, robots: indexable ? 'index,follow' : 'noindex,nofollow',
body: pages.supportersPage({ profile: p, leaderboard, recent }),
}), 200, indexable ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
});
r.post('/:handle/tip', withViewer, form, async (req, res, next) => {
-- const p = profiles.byHandle(req.params.handle);
++ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/${p.handle}`)}`);
const b = req.body || {};
@@@ -409,7 +419,7 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
privacy: { anonymous: values.anonymous, hide_amount: values.hide_amount, private_message: values.private_message },
};
const funding = b.pay_with === 'checkout' ? 'checkout' : 'credit';
-- const { interaction, replay } = interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });
++ const { interaction, replay } = await interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });
let i = interaction;
if (!replay) {
if (funding === 'credit') {
ddiff --git a/test/api.test.js b/test/api.test.js
iindex 5d13cad..5535c2e 100644
---- a/test/api.test.js
++++ b/test/api.test.js
@@@ -17,6 +17,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(h.json.service, 'tips');
const r = await t.call('GET', '/api/ready', { token: null });
assert.strictEqual(r.json.ready, true);
++ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered: pglite here, postgresql on the containers');
const rel = await t.call('GET', '/release.json', { token: null });
assert.strictEqual(rel.json.service, 'tips');
assert.deepStrictEqual(require('openvibe-contracts').validate('registry.release-manifest@1', rel.json).errors, []);
@@@ -107,6 +108,38 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(none.status, 400);
});
++ await check('an Idempotency-Key is claimed before the handler runs: once at a time, a stale claim taken over, refusals released', async () => {
++ const goals = () => t.db.value('SELECT count(*) FROM tip_goals WHERE creator_subject = $1', [alex.subject]);
++ const before = await goals();
++ // Two at once: the handler runs once; the other answer is "in progress" or the replay.
++ const body = { title: 'Once only', target_amount: 70 };
++ const both = await Promise.all([1, 2].map(() => t.call('POST', '/api/v1/goals', { user: alex, body, key: 'goal-key-race-01' })));
++ assert.ok(both.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(both.map((r) => r.status)));
++ assert.strictEqual(new Set(both.filter((r) => r.status === 201).map((r) => r.json.goal.id)).size, 1);
++ assert.strictEqual(await goals(), before + 1);
++ // A claim still in progress (as another process holds it): 409 with Retry-After.
++ const { stable } = require('../server/api/idempotency');
++ const hashOf = (b) => require('crypto').createHash('sha256').update(`POST /api/v1/goals\n${stable(b)}`).digest('hex');
++ const claim = (key, b, ageMs) => t.db.exec(`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
++ VALUES ($1, $2, 'POST', '/api/v1/goals', 0, '', $3)`, [`${alex.subject}:${key}`, hashOf(b), new Date(Date.now() - ageMs).toISOString()]);
++ await claim('goal-key-busy-0001', { title: 'Busy', target_amount: 5 }, 1000);
++ const busy = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Busy', target_amount: 5 }, key: 'goal-key-busy-0001' });
++ assert.deepStrictEqual([busy.status, busy.json.code, busy.headers.get('retry-after')], [409, 'idempotency.in_progress', '1']);
++ // A claim left by a process that died over a minute ago is taken over.
++ await claim('goal-key-stale-001', { title: 'Stale', target_amount: 5 }, 120_000);
++ const taken = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });
++ assert.strictEqual(taken.status, 201, taken.text);
++ const replay = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });
++ assert.deepStrictEqual([replay.headers.get('idempotent-replayed'), replay.json.goal.id], ['true', taken.json.goal.id]);
++ // A refusal is not stored: its claim is released, and a retry of the refused request runs again.
++ const bad = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' });
++ assert.strictEqual(bad.status, 422);
++ await new Promise((ok) => setTimeout(ok, 50));
++ assert.strictEqual(await t.db.value("SELECT count(*) FROM api_idempotency WHERE key LIKE '%goal-key-refused-1'"), 0, 'refusals are not stored');
++ assert.strictEqual((await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' })).status, 422);
++ for (const id of [taken.json.goal.id, both.find((r) => r.status === 201).json.goal.id]) await t.call('POST', `/api/v1/goals/${id}/close`, { user: alex });
++ });
++
await check('receipts: supporters see their own, creators theirs, nobody else', async () => {
const mine = await t.call('GET', '/api/v1/interactions', { user: viewer });
assert.ok(mine.json.interactions.length >= 2);
@@@ -158,7 +191,7 @@ const { boot, check, done } = require('./helpers/app');
await check('totals: derived from settled interactions, never from a stored balance', async () => {
const r = await t.call('GET', '/api/v1/profiles/alex/totals', { user: alex });
-- const sum = t.db.prepare("SELECT SUM(amount - reversed_bits) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')").get(alex.subject).n;
++ const sum = await t.db.value("SELECT SUM(amount - reversed_bits)::bigint FROM tip_interactions WHERE creator_subject = $1 AND NOT test AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')", [alex.subject]);
assert.strictEqual(r.json.totals.settled_via_billing, sum);
assert.strictEqual(r.json.totals.settled_via_billing, billing.payable.get(alex.subject), 'equals what Billing holds for the creator');
assert.strictEqual((await t.call('GET', '/api/v1/profiles/alex/totals', { user: viewer })).status, 403);
ddiff --git a/test/contracts.test.js b/test/contracts.test.js
iindex e0f2771..7c87a4f 100644
---- a/test/contracts.test.js
++++ b/test/contracts.test.js
@@@ -32,15 +32,15 @@ const TYPES = ['tips.interaction.ready', 'tips.interaction.failed', 'tips.intera
await domain.effects.drain();
live.state.fail = null;
t.clock.offset += 11 * 60 * 1000;
-- domain.overlays.sweepFailed();
++ await domain.overlays.sweepFailed();
t.clock.offset = 0;
// A reversal before delivery (tips.interaction.cancelled) and one after it.
const pending = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'read me' } } });
-- const txn = domain.interactions.get(pending.json.interaction.id).billing_txn_id;
++ const txn = (await domain.interactions.get(t.db, pending.json.interaction.id)).billing_txn_id;
await t.deliver(billing.refund(txn));
-- await t.deliver(billing.refund(domain.interactions.get(tip.json.interaction.id).billing_txn_id, 100));
++ await t.deliver(billing.refund((await domain.interactions.get(t.db, tip.json.interaction.id)).billing_txn_id, 100));
-- const rows = t.outboxRows();
++ const rows = await t.outboxRows();
for (const type of TYPES) assert.ok(rows.some((e) => e.event_type === type), `no ${type} was produced`);
for (const e of rows) {
const env = contracts.validate('events.event-envelope@1', e);
ddiff --git a/test/delivery.test.js b/test/delivery.test.js
iindex 2aae6d0..f1774d8 100644
---- a/test/delivery.test.js
++++ b/test/delivery.test.js
@@@ -24,8 +24,8 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(d.job.test, false);
const grant = t.network.grants.find((g) => g.audience === 'openvibe.live');
assert.strictEqual(grant.scope, 'live.tips_delivery.write');
-- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(r.json.interaction.id);
-- assert.deepStrictEqual(JSON.parse(e.result).ref, { chat_message_id: 1 });
++ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [r.json.interaction.id]);
++ assert.deepStrictEqual(e.result.ref, { chat_message_id: 1 }, 'jsonb comes back parsed');
});
await check('TTS goes out as its own effect with the text and voice', async () => {
@@@ -41,17 +41,17 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 7 } });
const id = r.json.interaction.id;
await domain.effects.drain();
-- let e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
++ let e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'queued');
assert.match(e.last_error, /Live 503/);
-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
live.state.fail = null;
t.clock.offset += 5000;
await domain.effects.drain();
t.clock.offset = 0;
-- e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
++ e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'delivered');
-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
});
await check('Live refuses (422): the effect fails at once, tips.interaction.failed, payment untouched', async () => {
@@@ -60,17 +60,17 @@ const { boot, check, done } = require('./helpers/app');
const id = r.json.interaction.id;
await domain.effects.drain();
live.state.fail = null;
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.delivery_state, 'failed');
assert.strictEqual(i.payment_state, 'settled');
-- assert.strictEqual(billing.payable.get(t.domain.profiles.byHandle('alex').creator_subject) >= 8, true);
-- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((ev) => ev.subject.id === id).length, 1);
++ assert.strictEqual(billing.payable.get((await t.domain.profiles.byHandle(t.db, 'alex')).creator_subject) >= 8, true);
++ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((ev) => ev.subject.id === id).length, 1);
});
await check('simulations never reach Live: their chat effects stay in the test adapter', async () => {
-- const alex = t.domain.profiles.byHandle('alex');
++ const alex = await t.domain.profiles.byHandle(t.db, 'alex');
const before = live.deliveries.length;
-- const sim = domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });
++ const sim = await domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });
await domain.effects.drain();
assert.strictEqual(live.deliveries.length, before);
assert.ok(t.adapters.test.jobs.some((j) => j.interaction.id === sim.id && j.test));
ddiff --git a/test/external.test.js b/test/external.test.js
iindex a1eb55c..58a2c0d 100644
---- a/test/external.test.js
++++ b/test/external.test.js
@@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');
},
};
};
-- const byRef = (ref) => t.db.prepare("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = ?").get(ref);
++ const byRef = (ref) => t.db.maybe("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = $1", [ref]);
let first;
await check('an EXTERNAL receipt is recorded, settled and announced in Live chat as Live\'s webhook wrote it', async () => {
@@@ -37,7 +37,7 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.deliver(first);
assert.strictEqual(r.status, 200, JSON.stringify(r.json));
assert.strictEqual(r.json.outcome, 'recorded_external');
-- const i = byRef('ext-1');
++ const i = await byRef('ext-1');
assert.deepStrictEqual([i.creator_subject, i.kind, i.amount, i.amount_cents, i.supporter_name, i.message], [alex.subject, 'tip', 1234, 1234, 'Generous Fan', 'love the stream']);
assert.deepStrictEqual([i.funding, i.settlement, i.origin, i.payment_state, i.billing_txn_id], ['external', 'external', 'billing-external', 'settled', null]);
await domain.effects.drain();
@@@ -46,12 +46,12 @@ const { boot, check, done } = require('./helpers/app');
assert.deepStrictEqual([d[0].key, d[0].job.effect, d[0].job.test], [`${i.id}:chat_line`, 'chat_line', false]);
assert.strictEqual(d[0].job.text, 'Generous Fan tipped 1,234 Vibes: love the stream (PowerChat)');
assert.deepStrictEqual(d[0].job.creator, { type: 'user', id: alex.subject, handle: 'alex' });
-- const effects = t.db.prepare('SELECT effect, state FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);
++ const effects = await t.db.many('SELECT effect, state FROM interaction_effects WHERE interaction_id = $1 ORDER BY id', [i.id]);
assert.deepStrictEqual(effects, [{ effect: 'overlay_alert', state: 'delivered' }, { effect: 'chat_line', state: 'delivered' }]);
-- const goal = domain.goals.present(domain.goals.get(g.json.goal.id));
++ const goal = await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id));
assert.strictEqual(goal.current_amount, 1234, 'the sole active goal advances, as on Live');
-- assert.ok(t.outboxRows('tips.goal.updated').some((e) => e.payload.interaction_id === i.id));
-- assert.ok(t.outboxRows('tips.interaction.ready').some((e) => e.subject.id === i.id));
++ assert.ok((await t.outboxRows('tips.goal.updated')).some((e) => e.payload.interaction_id === i.id));
++ assert.ok((await t.outboxRows('tips.interaction.ready')).some((e) => e.subject.id === i.id));
});
await check('a redelivery or a republish of the same payment announces nothing more', async () => {
@@@ -61,12 +61,13 @@ const { boot, check, done } = require('./helpers/app');
const viaApi = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.interaction.record'], body: { creator: 'alex', provider: 'powerchat', provider_ref: 'ext-1', amount_cents: 1234 } });
assert.strictEqual(viaApi.json.duplicate, true, 'the same key as POST /interactions/external');
await domain.effects.drain();
-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE provider_ref = 'ext-1'").get().n, 1);
-- assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === byRef('ext-1').id).length, 1);
++ assert.strictEqual(await t.db.value("SELECT count(*) FROM tip_interactions WHERE provider_ref = 'ext-1'"), 1);
++ const one = await byRef('ext-1');
++ assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === one.id).length, 1);
});
await check('EXTERNAL money stays out of the Billing total', async () => {
-- const totals = domain.interactions.totals(alex.subject);
++ const totals = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totals.external, 1234);
assert.strictEqual(totals.settled_via_billing, 0);
});
@@@ -75,19 +76,19 @@ const { boot, check, done } = require('./helpers/app');
const a = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 900 } })).json.goal;
const b = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Chair', target_amount: 900 } })).json.goal;
await t.deliver(receipt({ provider_event_id: 'ext-goal-tips', amount_cents: 100, value_bits: 100, app_purpose: `goal:${a.id}` }));
-- t.db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)
-- VALUES ('live', 'donation_goals', '12', 'goal', ?, 'imported', 'imp_test', ?, ?)`).run(b.id, new Date().toISOString(), new Date().toISOString());
++ await t.db.exec(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)
++ VALUES ('live', 'donation_goals', '12', 'goal', $1, 'imported', 'imp_test', $2, $2)`, [b.id, new Date().toISOString()]);
await t.deliver(receipt({ provider_event_id: 'ext-goal-live', amount_cents: 200, value_bits: 200, app_purpose: 'goal:12' }));
await t.deliver(receipt({ provider_event_id: 'ext-goal-none', amount_cents: 300, value_bits: 300 }));
-- assert.strictEqual(domain.goals.present(domain.goals.get(a.id)).current_amount, 100);
-- assert.strictEqual(domain.goals.present(domain.goals.get(b.id)).current_amount, 200);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(byRef('ext-goal-none').id).n, 0, 'several active goals and no pick: none (Live\'s rule)');
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, a.id))).current_amount, 100);
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, b.id))).current_amount, 200);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [(await byRef('ext-goal-none')).id]), 0, 'several active goals and no pick: none (Live\'s rule)');
});
await check('anonymous: no donor name; a creator with no tip page is still announced', async () => {
const bob = t.network.newUser('bob');
await t.deliver(receipt({ provider_event_id: 'ext-anon', donor_name: null, anonymous: true, message: 'hi' }, { streamer: bob.subject }));
-- const i = byRef('ext-anon');
++ const i = await byRef('ext-anon');
assert.deepStrictEqual([i.creator_subject, i.supporter_name], [bob.subject, 'Anonymous']);
await domain.effects.drain();
const d = live.deliveries.find((x) => x.job.interaction.id === i.id);
@@@ -99,20 +100,20 @@ const { boot, check, done } = require('./helpers/app');
const before = live.deliveries.length;
await t.deliver(receipt({ provider_event_id: 'ext-test', test: true }));
await domain.effects.drain();
-- const i = byRef('ext-test');
-- assert.strictEqual(i.test, 1);
++ const i = await byRef('ext-test');
++ assert.strictEqual(i.test, true);
assert.strictEqual(live.deliveries.length, before);
assert.strictEqual(t.adapters.test.jobs.filter((j) => j.interaction.id === i.id).length, 1);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);
});
await check('malformed or foreign receipts are ignored, not recorded', async () => {
-- const n = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;
++ const n = await t.db.value('SELECT count(*) FROM tip_interactions');
assert.strictEqual((await t.deliver(receipt({ streamer: { type: 'user', id: '42' } }))).json.outcome, 'ignored:no_streamer');
assert.strictEqual((await t.deliver(receipt({ amount_cents: 1e20 }))).json.outcome, 'ignored:no_amount');
assert.strictEqual((await t.deliver(receipt({ provider_event_id: '' }))).json.outcome, 'ignored:no_reference');
assert.strictEqual((await t.deliver({ ...receipt(), source: 'live' })).json.outcome, 'ignored:source');
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, n);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), n);
});
await t.close();
ddiff --git a/test/fixtures/sqlite-v3-schema.sql b/test/fixtures/sqlite-v3-schema.sql
nnew file mode 100644
iindex 0000000..ee02962
---- /dev/null
++++ b/test/fixtures/sqlite-v3-schema.sql
@@@ -0,0 +1,301 @@
++-- The SQLite schema of the last SQLite release of OpenVibe.Tips (schema v3: server/db.js with its added
++-- columns, plus the openvibe-sdk outbox and inbox), as sqlite_master holds it in a database that release made.
++-- test/sqlite-import.test.js builds its fixture from it. Never edit: it is the source side of the one-time import.
++
++CREATE TABLE settings (
++ id INTEGER PRIMARY KEY CHECK (id = 1),
++ schema_version INTEGER NOT NULL,
++ created_at TEXT NOT NULL
++);
++
++CREATE TABLE creator_tip_profiles (
++ creator_subject TEXT PRIMARY KEY, -- usr_…
++ handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)
++ display_name TEXT NOT NULL,
++ avatar_url TEXT,
++ headline TEXT,
++ page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing
++ accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted
++ min_amount INTEGER NOT NULL DEFAULT 1,
++ paid_message_min INTEGER NOT NULL DEFAULT 100,
++ tts_enabled INTEGER NOT NULL DEFAULT 0,
++ tts_min_amount INTEGER NOT NULL DEFAULT 100,
++ tts_max_chars INTEGER NOT NULL DEFAULT 200,
++ tts_voice TEXT NOT NULL DEFAULT 'gary',
++ media_requests_enabled INTEGER NOT NULL DEFAULT 0,
++ media_request_min INTEGER NOT NULL DEFAULT 25,
++ media_max_seconds INTEGER NOT NULL DEFAULT 600,
++ revision INTEGER NOT NULL DEFAULT 1,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL
++, page_settings TEXT NOT NULL DEFAULT '{}', filter_words TEXT NOT NULL DEFAULT '[]', filter_action TEXT NOT NULL DEFAULT 'mask', filter_links INTEGER NOT NULL DEFAULT 1);
++
++CREATE TABLE tip_interactions (
++ id TEXT PRIMARY KEY, -- tint_<ULID>
++ creator_subject TEXT NOT NULL,
++ supporter_subject TEXT, -- null: external/anonymous/unmapped
++ supporter_name TEXT, -- display projection at the time
++ kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
++ amount INTEGER NOT NULL CHECK (amount > 0),
++ currency TEXT NOT NULL DEFAULT 'vibes-bits',
++ amount_cents INTEGER, -- money that arrived (provider/external)
++ message TEXT,
++ request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target
++ funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
++ settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
++ payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
++ delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
++ test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted
++ billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it
++ billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent
++ funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it
++ reversal_txn_ids TEXT NOT NULL DEFAULT '[]',
++ reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator
++ checkout_url TEXT,
++ checkout_ref TEXT,
++ provider TEXT,
++ provider_ref TEXT,
++ legacy_source TEXT, -- 'live:transactions:<id>' etc.
++ idempotency_key TEXT UNIQUE,
++ failure TEXT,
++ origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import
++ transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried
++ transfer_attempts INTEGER NOT NULL DEFAULT 0,
++ next_transfer_at INTEGER NOT NULL DEFAULT 0,
++ target TEXT, -- JSON EntityRef (e.g. a Live stream)
++ created_at TEXT NOT NULL,
++ settled_at TEXT,
++ reversed_at TEXT,
++ delivered_at TEXT,
++ updated_at TEXT NOT NULL, anonymous INTEGER NOT NULL DEFAULT 0, hide_amount INTEGER NOT NULL DEFAULT 0, private_message INTEGER NOT NULL DEFAULT 0, erased_at TEXT, moderation TEXT NOT NULL DEFAULT 'visible', moderated_at TEXT, moderated_by TEXT, filtered INTEGER NOT NULL DEFAULT 0,
++ UNIQUE (provider, provider_ref),
++ UNIQUE (legacy_source)
++);
++
++CREATE TABLE tip_goals (
++ id TEXT PRIMARY KEY, -- tgoal_<ULID>
++ creator_subject TEXT NOT NULL,
++ title TEXT NOT NULL,
++ description TEXT,
++ target_amount INTEGER NOT NULL CHECK (target_amount > 0),
++ currency TEXT NOT NULL DEFAULT 'vibes-bits',
++ image_url TEXT,
++ status TEXT NOT NULL CHECK (status IN ('active', 'closed')),
++ sort_order INTEGER NOT NULL DEFAULT 0,
++ reached_at TEXT,
++ closed_at TEXT,
++ opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)
++ legacy_source TEXT UNIQUE,
++ revision INTEGER NOT NULL DEFAULT 1,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL
++);
++
++CREATE TABLE tip_goal_contributions (
++ id INTEGER PRIMARY KEY AUTOINCREMENT,
++ goal_id TEXT NOT NULL REFERENCES tip_goals(id),
++ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
++ amount INTEGER NOT NULL CHECK (amount > 0),
++ reversed_amount INTEGER NOT NULL DEFAULT 0,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL,
++ UNIQUE (goal_id, interaction_id)
++);
++
++CREATE TABLE paid_messages (
++ id TEXT PRIMARY KEY, -- tpm_<ULID>
++ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
++ creator_subject TEXT NOT NULL,
++ kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),
++ text TEXT NOT NULL,
++ voice TEXT,
++ highlight_seconds INTEGER NOT NULL DEFAULT 0,
++ status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
++ chat_ref TEXT, -- JSON reference the chat adapter returned
++ test INTEGER NOT NULL DEFAULT 0,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL
++);
++
++CREATE TABLE paid_media_requests (
++ id TEXT PRIMARY KEY, -- tmr_<ULID>
++ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
++ creator_subject TEXT NOT NULL,
++ url TEXT NOT NULL,
++ provider TEXT,
++ status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
++ queue_ref TEXT, -- JSON reference returned by the queue owner
++ test INTEGER NOT NULL DEFAULT 0,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL
++);
++
++CREATE TABLE overlay_configs (
++ id TEXT PRIMARY KEY, -- tovc_<ULID>
++ creator_subject TEXT NOT NULL,
++ kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),
++ name TEXT NOT NULL,
++ settings TEXT NOT NULL DEFAULT '{}',
++ goal_id TEXT,
++ revision INTEGER NOT NULL DEFAULT 1,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL
++);
++
++CREATE TABLE overlay_tokens (
++ id TEXT PRIMARY KEY, -- tovt_<ULID>
++ creator_subject TEXT NOT NULL,
++ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
++ scopes TEXT NOT NULL, -- JSON ["alerts","goals"]
++ config_id TEXT,
++ label TEXT,
++ created_by TEXT NOT NULL,
++ created_at TEXT NOT NULL,
++ last_used_at TEXT,
++ revoked_at TEXT
++);
++
++CREATE TABLE overlay_deliveries (
++ seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)
++ id TEXT NOT NULL UNIQUE, -- tovd_<ULID>
++ creator_subject TEXT NOT NULL,
++ kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),
++ interaction_id TEXT,
++ goal_id TEXT,
++ payload TEXT NOT NULL,
++ test INTEGER NOT NULL DEFAULT 0,
++ status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
++ delivered_at TEXT,
++ failed_at TEXT,
++ sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
++ dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
++ created_at TEXT NOT NULL
++, hidden INTEGER NOT NULL DEFAULT 0);
++
++CREATE TABLE interaction_effects (
++ id INTEGER PRIMARY KEY AUTOINCREMENT,
++ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
++ effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
++ adapter TEXT NOT NULL,
++ state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
++ attempts INTEGER NOT NULL DEFAULT 0,
++ next_attempt_at INTEGER NOT NULL DEFAULT 0,
++ last_error TEXT,
++ result TEXT,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL,
++ UNIQUE (interaction_id, effect)
++);
++
++CREATE TABLE migration_maps (
++ id INTEGER PRIMARY KEY AUTOINCREMENT,
++ source TEXT NOT NULL, -- 'live'
++ source_table TEXT NOT NULL,
++ source_id TEXT NOT NULL,
++ target_type TEXT, -- 'interaction' | 'goal'
++ target_id TEXT,
++ status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
++ reason TEXT,
++ billing_txn_id TEXT,
++ run_id TEXT NOT NULL,
++ created_at TEXT NOT NULL,
++ updated_at TEXT NOT NULL,
++ UNIQUE (source, source_table, source_id)
++);
++
++CREATE TABLE import_runs (
++ id TEXT PRIMARY KEY,
++ source TEXT NOT NULL,
++ dry_run INTEGER NOT NULL,
++ started_at TEXT NOT NULL,
++ finished_at TEXT,
++ report TEXT
++);
++
++CREATE TABLE tip_moderators (
++ creator_subject TEXT NOT NULL,
++ moderator_subject TEXT NOT NULL,
++ name TEXT, -- display name when they joined (for the creator's list)
++ added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'
++ created_at TEXT NOT NULL,
++ removed_at TEXT,
++ PRIMARY KEY (creator_subject, moderator_subject)
++);
++
++CREATE TABLE tip_moderator_invites (
++ id TEXT PRIMARY KEY, -- tmin_<ULID>
++ creator_subject TEXT NOT NULL,
++ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
++ created_by TEXT NOT NULL,
++ created_at TEXT NOT NULL,
++ expires_at TEXT NOT NULL,
++ used_at TEXT,
++ used_by TEXT,
++ revoked_at TEXT
++);
++
++CREATE TABLE tip_moderation_log (
++ id INTEGER PRIMARY KEY AUTOINCREMENT,
++ creator_subject TEXT NOT NULL,
++ interaction_id TEXT NOT NULL,
++ action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
++ by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
++ actor TEXT, -- usr_… / svc:… ; null for the filter
++ reason TEXT,
++ created_at TEXT NOT NULL
++);
++
++CREATE TABLE api_idempotency (
++ key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>
++ request_hash TEXT NOT NULL,
++ method TEXT NOT NULL,
++ path TEXT NOT NULL,
++ status INTEGER NOT NULL,
++ response TEXT NOT NULL,
++ created_at TEXT NOT NULL
++);
++
++CREATE TABLE event_outbox (
++ id INTEGER PRIMARY KEY AUTOINCREMENT,
++ event_id TEXT NOT NULL UNIQUE,
++ envelope TEXT NOT NULL,
++ traceparent TEXT,
++ created_at INTEGER NOT NULL,
++ attempts INTEGER NOT NULL DEFAULT 0,
++ next_attempt_at INTEGER NOT NULL DEFAULT 0,
++ sent_at INTEGER,
++ seq INTEGER,
++ rejected_at INTEGER,
++ last_error TEXT
++ );
++
++CREATE TABLE idempotency_receipts (
++ consumer TEXT NOT NULL,
++ event_id TEXT NOT NULL,
++ processed_at INTEGER NOT NULL,
++ PRIMARY KEY (consumer, event_id)
++ );
++
++CREATE INDEX idx_tint_creator ON tip_interactions (creator_subject, created_at);
++
++CREATE INDEX idx_tint_supporter ON tip_interactions (supporter_subject, created_at);
++
++CREATE INDEX idx_tint_pending ON tip_interactions (payment_state, funding);
++
++CREATE INDEX idx_goal_creator ON tip_goals (creator_subject, status);
++
++CREATE INDEX idx_ovc_creator ON overlay_configs (creator_subject);
++
++CREATE INDEX idx_ovt_creator ON overlay_tokens (creator_subject);
++
++CREATE INDEX idx_ovd_creator ON overlay_deliveries (creator_subject, seq);
++
++CREATE INDEX idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';
++
++CREATE INDEX idx_effects_due ON interaction_effects (state, next_attempt_at);
++
++CREATE INDEX idx_tmod_moderator ON tip_moderators (moderator_subject);
++
++CREATE INDEX idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);
++
++CREATE INDEX idx_event_outbox_due ON event_outbox(sent_at, rejected_at, next_attempt_at);
ddiff --git a/test/helpers/app.js b/test/helpers/app.js
iindex ea32e9a..32961eb 100644
---- a/test/helpers/app.js
++++ b/test/helpers/app.js
@@@ -1,8 +1,13 @@
'use strict';
/**
-- * Boots Tips against the stubs on a random port with a temp database. Jobs are off: tests drive the
++ * Boots Tips against the stubs on a random port with a migrated database of its own (helpers/db.js:
++ * PGlite, or the containers with opts.store 'pg' / TIPS_TEST_STORE=pg). Jobs are off: tests drive the
* effects worker and transfer retries explicitly (domain.effects.drain(), processDueTransfers()).
-- * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow).
++ * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow);
++ * opts.valkey an openvibe-sdk/valkey handle (default: the containers' Valkey with TIPS_TEST_STORE=pg,
++ * else none: everything shared stays in the process);
++ * opts.db an existing handle (a second process on the same database); opts.share another instance
++ * whose stubs (Network, Billing, Events, Live) this one uses.
*
* t.call(method, path, { body, user, cap, sub, key }) user: a stub user → Bearer user JWT;
* otherwise a service token with `cap`
@@@ -16,18 +21,20 @@ const http = require('http');
const crypto = require('crypto');
const { signDelivery, signDeliveryHeaders } = require('openvibe-sdk/events');
const { startNetwork, startBilling, startEvents, startLive } = require('./stubs');
++const { testDb, testValkey } = require('./db');
const EVENTS_SECRET = 'e'.repeat(48);
async function boot(opts = {}) {
-- const network = await startNetwork();
-- const billing = await startBilling(network);
-- const events = await startEvents();
-- const live = opts.live ? await startLive(network) : null;
++ // opts.share: another booted instance whose stubs this one uses (a second process of one service).
++ const shared = opts.share || null;
++ const network = shared ? shared.network : await startNetwork();
++ const billing = shared ? shared.billing : await startBilling(network);
++ const events = shared ? shared.events : await startEvents();
++ const live = shared ? shared.live : (opts.live ? await startLive(network) : null);
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-test-'));
const env = {
NODE_ENV: 'test',
-- TIPS_DB_PATH: path.join(dir, 'tips.db'),
BASE_URL: 'http://tips.test',
OV_NETWORK_URL: network.url,
OV_NETWORK_INTERNAL_URL: network.url,
@@@ -53,7 +60,10 @@ async function boot(opts = {}) {
const clock = { offset: 0 };
const logs = [];
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')) };
-- const app = createApp({ config, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });
++ const store = opts.db ? { db: opts.db, store: opts.db.store, close: async () => {} } : await testDb({ store: opts.store });
++ const ownValkey = opts.valkey === undefined && process.env.TIPS_TEST_STORE === 'pg' ? testValkey() : null;
++ const valkey = opts.valkey || ownValkey;
++ const app = createApp({ config, db: store.db, valkey, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });
await app.locals.keys.load();
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
@@@ -135,17 +145,20 @@ async function boot(opts = {}) {
return u;
}
-- const outboxRows = (type) => domain.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope)).filter((e) => !type || e.event_type === type);
++ /** The events in the outbox (envelopes, oldest first), of one type or all. */
++ const outboxRows = async (type) => (await domain.db.many('SELECT envelope FROM tips_event_outbox ORDER BY id')).map((r) => r.envelope).filter((e) => !type || e.event_type === type);
return {
-- app, base, call, deliver, sse, creator, domain, db: domain.db, config, clock, network, billing, events, live, logs, dir, outboxRows,
++ app, base, call, deliver, sse, creator, domain, db: domain.db, store, valkey, config, clock, network, billing, events, live, logs, dir, outboxRows,
adapters: app.locals.adapters,
close: async () => {
-- domain.overlays.closeAll();
++ await domain.overlays.close();
++ server.closeAllConnections();
await new Promise((r) => server.close(r));
await app.locals.outbox.stop();
-- await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);
-- try { domain.db.close(); } catch { /* */ }
++ if (!shared) await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);
++ await store.close();
++ if (ownValkey) await ownValkey.close();
fs.rmSync(dir, { recursive: true, force: true });
},
};
ddiff --git a/test/helpers/db.js b/test/helpers/db.js
nnew file mode 100644
iindex 0000000..7dfdaa5
---- /dev/null
++++ b/test/helpers/db.js
@@@ -0,0 +1,85 @@
++'use strict';
++/**
++ * A migrated database for one test run (ADR-035).
++ *
++ * store 'pglite' (the default): real PostgreSQL in-process, migrated (openvibe-sdk/db { pglite }).
++ * store 'pg': the production-shaped containers (openvibe-sdk scripts/test-services.sh up), when
++ * OV_TEST_PG_URL (PgBouncer, transaction mode) and OV_TEST_PG_DIRECT_URL are set. The run gets roles
++ * and a schema of its own, shaped as OpenVibe.Host's roles/data/add-service.sh makes them: an owner
++ * role that migrates on the direct connection, and a runtime role (DML only, statement_timeout 15 s,
++ * lock_timeout 5 s) that serves through PgBouncer. Other suites sharing the database never see them;
++ * close() drops them.
++ *
++ * TIPS_TEST_STORE=pg runs every test file on the containers instead of PGlite.
++ *
++ * const { db, close } = await testDb();
++ * testValkey() the containers' Valkey (OV_TEST_VALKEY_URL) under a prefix of its own, or null
++ */
++const crypto = require('crypto');
++const { createDb } = require('openvibe-sdk/db');
++const { createValkey } = require('openvibe-sdk/valkey');
++const { MIGRATIONS } = require('../../server/db');
++
++const quiet = { log() {}, warn() {}, error: (...a) => console.error(...a) };
++const pgAvailable = () => !!(process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL);
++const valkeyAvailable = () => !!process.env.OV_TEST_VALKEY_URL;
++
++async function testDb({ store = process.env.TIPS_TEST_STORE || 'pglite', max = 4 } = {}) {
++ if (store !== 'pg') {
++ const db = createDb({ pglite: true, service: 'tips-test', log: quiet });
++ await db.migrate({ dir: MIGRATIONS, log: quiet });
++ return { db, store: 'pglite', close: () => db.close().catch(() => {}) };
++ }
++ if (!pgAvailable()) throw new Error('store pg needs OV_TEST_PG_URL and OV_TEST_PG_DIRECT_URL (openvibe-sdk scripts/test-services.sh up)');
++ const name = `tips_t${process.pid}_${crypto.randomBytes(4).toString('hex')}`;
++ const owner = `${name}_owner`;
++ const pw = crypto.randomBytes(16).toString('hex');
++ const su = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL, service: 'tips-test-admin', max: 1, log: quiet });
++ const database = await su.value('SELECT current_database()');
++ for (const stmt of [
++ `CREATE ROLE ${owner} LOGIN PASSWORD '${pw}'`,
++ `CREATE ROLE ${name} LOGIN PASSWORD '${pw}'`,
++ `GRANT CONNECT ON DATABASE ${database} TO ${owner}, ${name}`,
++ `CREATE SCHEMA ${name} AUTHORIZATION ${owner}`,
++ `ALTER ROLE ${owner} SET search_path = ${name}`,
++ `ALTER ROLE ${name} SET search_path = ${name}`,
++ `ALTER ROLE ${name} SET statement_timeout = '15s'`,
++ `ALTER ROLE ${name} SET lock_timeout = '5s'`,
++ `GRANT USAGE ON SCHEMA ${name} TO ${name}`,
++ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ${name}`,
++ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO ${name}`,
++ ]) await su.query(stmt);
++ const as = (url, user) => { const u = new URL(url); u.username = user; u.password = pw; return u.toString(); };
++ async function drop() {
++ try {
++ // PgBouncer keeps its server connections after the client pool closes: end them here, or
++ // every run would leave its roles' idle connections holding the server's slots.
++ await su.query('SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE usename = ANY($1)', [[name, owner]]);
++ await su.query(`DROP SCHEMA IF EXISTS ${name} CASCADE`);
++ for (const r of [name, owner]) { await su.query(`DROP OWNED BY ${r}`); await su.query(`DROP ROLE ${r}`); }
++ } finally { await su.close(); }
++ }
++ let db;
++ try {
++ const ownerDb = createDb({ url: as(process.env.OV_TEST_PG_DIRECT_URL, owner), service: 'tips-test-migrate', max: 1, log: quiet });
++ try { await ownerDb.migrate({ dir: MIGRATIONS, log: quiet }); } finally { await ownerDb.close(); }
++ db = createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet });
++ } catch (e) { await drop().catch(() => {}); throw e; } // a failed setup leaves nothing behind
++ return {
++ db, store: 'postgresql', schema: name,
++ /** Another pooled handle on the same database (a second process). */
++ open: (o = {}) => createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet, ...o }),
++ async close() {
++ await db.close().catch(() => {});
++ await drop();
++ },
++ };
++}
++
++/** The containers' Valkey under a prefix no other run uses (ov:tips-test:<random>:), or null without it. */
++function testValkey() {
++ if (!valkeyAvailable()) return null;
++ return createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: `ov:tips-test:${crypto.randomBytes(4).toString('hex')}:`, log: quiet });
++}
++
++module.exports = { testDb, testValkey, pgAvailable, valkeyAvailable };
ddiff --git a/test/import.test.js b/test/import.test.js
iindex 77aa464..4e80699 100644
---- a/test/import.test.js
++++ b/test/import.test.js
@@@ -81,61 +81,61 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
const r = await run({ dryRun: true });
assert.strictEqual(r.dry_run, true);
assert.ok(r.counts.imported > 0);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 0);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM migration_maps').get().n, 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM migration_maps'), 0);
});
let report;
await check('donations import settled and linked to Billing; failed excluded; unmapped creators held; test era flagged', async () => {
report = await run();
-- const byLegacy = (id) => t.db.prepare('SELECT * FROM tip_interactions WHERE legacy_source = ?').get(`live:transactions:${id}`);
-- const two = byLegacy(2);
++ const byLegacy = (id) => t.db.maybe('SELECT * FROM tip_interactions WHERE legacy_source = $1', [`live:transactions:${id}`]);
++ const two = await byLegacy(2);
assert.strictEqual(two.payment_state, 'settled');
assert.strictEqual(two.settlement, 'imported');
assert.strictEqual(two.billing_txn_id, 'txn_import_2');
assert.strictEqual(two.supporter_subject, subjects[2]);
assert.strictEqual(two.message, 'nice');
assert.strictEqual(two.created_at, '2026-02-01T10:00:00.000Z');
-- assert.strictEqual(byLegacy(1).test, 1, 'before stats_vibes_reset_at');
-- const ghost = byLegacy(3);
++ assert.strictEqual((await byLegacy(1)).test, true, 'before stats_vibes_reset_at');
++ const ghost = await byLegacy(3);
assert.strictEqual(ghost.supporter_subject, null);
assert.strictEqual(ghost.supporter_name, 'Ghost');
-- assert.strictEqual(byLegacy(6).kind, 'media_request');
-- assert.strictEqual(byLegacy(8).funding, 'provider');
-- assert.strictEqual(byLegacy(4), undefined, 'held, not dropped');
-- const held = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get();
++ assert.strictEqual((await byLegacy(6)).kind, 'media_request');
++ assert.strictEqual((await byLegacy(8)).funding, 'provider');
++ assert.strictEqual(await byLegacy(4), null, 'held, not dropped');
++ const held = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'");
assert.strictEqual(held.status, 'held');
-- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'").get();
++ const excluded = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'");
assert.strictEqual(excluded.status, 'excluded');
assert.match(excluded.reason, /status failed/);
assert.strictEqual(report.counts.linked_to_billing, report.counts.imported);
// Imports are history: no effects, no overlay alerts, no events.
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM interaction_effects').get().n, 0);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM overlay_deliveries').get().n, 0);
-- assert.strictEqual(t.outboxRows('tips.interaction.ready').length, 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM interaction_effects'), 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM overlay_deliveries'), 0);
++ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).length, 0);
});
await check('the refund nets out against the donation it undoes', async () => {
-- const six = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'").get();
++ const six = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'");
assert.strictEqual(six.reversed_bits, 250);
assert.strictEqual(six.payment_state, 'reversed');
assert.strictEqual(report.counts.refunds_applied, 1);
});
await check('external PowerChat tips: the direct one is imported as EXTERNAL; the site-routed celebration is not counted twice', async () => {
-- const ext = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'").get();
++ const ext = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'");
assert.strictEqual(ext.settlement, 'external');
assert.strictEqual(ext.amount, 700);
assert.strictEqual(ext.supporter_name, 'Direct');
-- const twin = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'").get();
++ const twin = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'");
assert.strictEqual(twin.status, 'excluded');
assert.match(twin.reason, /site-routed tip \(Live transaction 8\)/);
-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'").get().n, 0, 'on-site donation lines are not PowerChat');
++ assert.strictEqual(await t.db.value("SELECT count(*) FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'"), 0, 'on-site donation lines are not PowerChat');
});
await check('goals import with Live\'s current amount carried over (not invented contributions)', async () => {
-- const g = t.db.prepare("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'").get();
-- const view = domain.goals.present(g);
++ const g = await t.db.maybe("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'");
++ const view = await domain.goals.present(t.db, g);
assert.strictEqual(view.current_amount, 1200);
assert.strictEqual(view.carried_over_amount, 1200);
assert.strictEqual(view.supporters_count, 0);
@@@ -154,34 +154,34 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
const evTest = t.billing.envelope('billing.transaction.settled', { transaction_id: 'txn_native_test', type: 'donation', test: true, from_subject: subjects[2], to_subject: subjects[1], provider: null, metadata: { kind: 'donation', amount_bits: 999 } }, 'txn_native_test');
await t.deliver(evTest);
const { reconcile } = require('../server/importer/live');
-- const r = reconcile(domain, billingSnapshotSource(bdb));
++ const r = await reconcile(domain, billingSnapshotSource(bdb));
assert.strictEqual(r.ok, true, JSON.stringify(r.mismatches));
-- assert.strictEqual(domain.interactions.totals(subjects[1]).settled_via_billing, 740);
-- assert.strictEqual(domain.interactions.totals(subjects[1]).external, 700);
++ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).settled_via_billing, 740);
++ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).external, 700);
});
await check('a re-run changes nothing; a newly mapped creator is released from hold', async () => {
-- const before = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;
++ const before = await t.db.value('SELECT count(*) FROM tip_interactions');
const again = await run();
assert.strictEqual(again.counts.imported, 0);
assert.strictEqual(again.counts.goals, 0);
assert.ok(again.counts.unchanged > 0);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before);
const heldUser = t.network.newUser('held', 4);
const r = await run();
assert.strictEqual(r.counts.released, 1);
assert.strictEqual(r.counts.goals, 1);
-- const four = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'").get();
++ const four = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'");
assert.strictEqual(four.creator_subject, heldUser.subject);
assert.strictEqual(four.billing_txn_id, 'txn_import_4');
-- assert.strictEqual(t.db.prepare("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get().status, 'imported');
++ assert.strictEqual(await t.db.value("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'"), 'imported');
});
await check('every source row is imported, excluded with a reason, or held — none silently dropped', async () => {
const tipRows = live.prepare("SELECT id FROM transactions WHERE type IN ('donation', 'refund')").all().map((x) => String(x.id));
-- const mapped = new Set(t.db.prepare("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'").all().map((x) => x.source_id));
++ const mapped = new Set((await t.db.many("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'")).map((x) => x.source_id));
assert.deepStrictEqual(tipRows.filter((id) => !mapped.has(id)), []);
-- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE status = 'excluded'").all();
++ const excluded = await t.db.many("SELECT * FROM migration_maps WHERE status = 'excluded'");
assert.ok(excluded.every((x) => x.reason));
});
ddiff --git a/test/integration.test.js b/test/integration.test.js
nnew file mode 100644
iindex 0000000..40b02c9
---- /dev/null
++++ b/test/integration.test.js
@@@ -0,0 +1,237 @@
++'use strict';
++// Tips on the production-shaped data tier (ADR-035; the tests of ADR-007's 2026-09-24 amendment):
++// PostgreSQL 18 behind PgBouncer in transaction mode and Valkey 9, from
++// `eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)"`. Two Tips processes (two app
++// instances, each with its own pool and Valkey connection) share one database, one Valkey and the same
++// Billing and Network:
++// - money races: one settlement or one reversal delivered many times at once, to both processes,
++// counts once; the unpaid-checkout limit holds under concurrent requests (serializable)
++// - work claimed with a lease: two effects workers and two transfer retries at once act once each
++// - per-process state in Valkey: per-actor limits, overlay fan-out (a stream on one process hears
++// tips, config changes and revocations made on the other), overlay stream slots
++// - flushing Valkey changes no money answer
++// Without the containers it prints why it skipped; the rest of the suite runs on PGlite.
++const assert = require('assert');
++const { ids } = require('openvibe-contracts');
++const { createValkey } = require('openvibe-sdk/valkey');
++const { testDb, testValkey, pgAvailable, valkeyAvailable } = require('./helpers/db');
++const { boot, check, done } = require('./helpers/app');
++
++if (!pgAvailable()) {
++ console.log('integration (PgBouncer + Valkey): skipped (OV_TEST_PG_URL not set; eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)")');
++ process.exit(0);
++}
++
++const LIMIT_CLOCK = Date.UTC(2026, 8, 28, 12, 0, 10); // one fixed minute for both processes' limiters
++
++(async () => {
++ const store = await testDb({ store: 'pg', max: 6 });
++ const vA = testValkey();
++ const vB = vA ? createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: vA.prefix, log: { warn() {} } }) : null;
++ const env = { TIPS_OVERLAY_MAX_STREAMS: '2', TIPS_MAX_PENDING_CHECKOUTS: '3' };
++ const appOpts = { limitsNow: () => LIMIT_CLOCK };
++ const A = await boot({ db: store.db, valkey: vA, env, appOpts });
++ const dbB = store.open();
++ const B = await boot({ db: dbB, valkey: vB, env, appOpts, share: A });
++ const { billing } = A;
++ const db = store.db;
++ const alex = await A.creator('alex');
++ const fan = A.network.newUser('fan');
++ billing.fund(fan.subject, 1_000_000);
++ const goal = (await A.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 100_000 } })).json.goal;
++ const fresh = (e) => ({ ...e, event_id: ids.newId('event') });
++ const both = (n, fn) => Promise.all(Array.from({ length: n }, (_, k) => fn(k % 2 ? B : A, k)));
++
++ await check('the store is PostgreSQL through PgBouncer; Valkey answers', async () => {
++ const r = await A.call('GET', '/api/ready', { token: null });
++ assert.strictEqual(r.status, 200, r.text);
++ assert.strictEqual(r.json.checks.db.detail.store, 'postgresql');
++ assert.strictEqual(r.json.checks.valkey.status, vA ? 'ok' : 'skipped');
++ });
++
++ await check('one Billing settlement delivered 6 times at once, to both processes, settles once', async () => {
++ const ev = billing.foreignDonation({ from: fan.subject, to: alex.subject, amount: 300, provider: 'powerchat' });
++ const out = await both(6, (t) => t.deliver(fresh(ev)));
++ assert.ok(out.every((r) => r.status === 200), JSON.stringify(out.map((r) => [r.status, r.json])));
++ assert.deepStrictEqual(out.map((r) => r.json.outcome).sort(), ['duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'recorded']);
++ const rows = await db.many('SELECT id FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);
++ assert.strictEqual(rows.length, 1);
++ assert.strictEqual(await db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [rows[0].id]), 1);
++ assert.strictEqual((await A.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === rows[0].id).length, 1);
++ assert.strictEqual(await db.value("SELECT count(*) FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [rows[0].id]), 1);
++ });
++
++ await check('one reversal delivered 6 times at once takes the bits back once, from the goal too', async () => {
++ const r = await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 400, goal_id: goal.id } });
++ assert.strictEqual(r.status, 201, r.text);
++ const id = r.json.interaction.id;
++ const before = (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount;
++ const rev = billing.refund(r.json.interaction.payment.billing_txn_id);
++ const out = await both(6, (t) => t.deliver(fresh(rev)));
++ assert.ok(out.every((x) => x.status === 200), JSON.stringify(out.map((x) => [x.status, x.json])));
++ assert.strictEqual(out.filter((x) => x.json.outcome === 'duplicate_reversal').length, 5);
++ const i = await db.one('SELECT reversed_bits, reversal_txn_ids, payment_state FROM tip_interactions WHERE id = $1', [id]);
++ assert.deepStrictEqual([i.reversed_bits, i.reversal_txn_ids.length, i.payment_state], [400, 1, 'reversed']);
++ assert.strictEqual((await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount, before - 400);
++ assert.strictEqual(await A.domain.interactions.totals(db, alex.subject).then((x) => x.settled_via_billing), billing.payable.get(alex.subject), 'reconciles with Billing');
++ });
++
++ await check('one Idempotency-Key sent at once to both processes runs once: one interaction, one Billing transfer', async () => {
++ const key = `race-${Date.now()}-key`;
++ const body = { creator: 'alex', amount: 33, message: 'once' };
++ const out = await Promise.all([A, B, A, B].map((t) => t.call('POST', '/api/v1/checkout', { user: fan, body, key })));
++ // The first runs; the others are told it is in progress (409, Retry-After) or get its stored answer.
++ assert.ok(out.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(out.map((r) => [r.status, r.json && r.json.code])));
++ const ok = out.filter((r) => r.status === 201);
++ assert.ok(ok.length >= 1);
++ assert.strictEqual(new Set(ok.map((r) => r.json.interaction.id)).size, 1, 'every answer names the one interaction');
++ const id = ok[0].json.interaction.id;
++ assert.strictEqual(billing.transfers().filter((c) => c.key === `tips:transfer:${id}`).length, 1);
++ assert.strictEqual(await db.value('SELECT count(*) FROM tip_interactions WHERE idempotency_key = $1', [`api:${fan.subject}:${key}`]), 1);
++ const later = await B.call('POST', '/api/v1/checkout', { user: fan, body, key });
++ assert.deepStrictEqual([later.status, later.headers.get('idempotent-replayed'), later.json.interaction.id], [201, 'true', id]);
++ });
++
++ await check('a creator\'s overlay deliveries become visible in seq order, however their transactions interleave', async () => {
++ // T1 inserts a delivery and holds its transaction open; T2 (the other process) inserts one for the
++ // same creator. T2 waits for T1, so seq order is commit order and no stream skips a seq.
++ const view = { id: goal.id };
++ let release;
++ const hold = new Promise((ok) => { release = ok; });
++ let t1Seq = null; let t2Done = false;
++ const t1 = A.domain.tx(async (t) => {
++ await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-1-${Date.now()}` });
++ t1Seq = await t.value('SELECT max(seq) FROM overlay_deliveries');
++ await hold;
++ });
++ while (t1Seq === null) await new Promise((ok) => setTimeout(ok, 10));
++ const t2 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-2-${Date.now()}` })).then(() => { t2Done = true; });
++ await new Promise((ok) => setTimeout(ok, 300));
++ const waited = !t2Done;
++ release(); // before any assertion, so a failure reports instead of holding the transaction
++ await Promise.all([t1, t2]);
++ assert.strictEqual(waited, true, 'the second insert waits for the first transaction');
++ const [first, second] = (await db.many("SELECT seq FROM overlay_deliveries WHERE dedupe_key LIKE 'seq-order-%' ORDER BY seq")).map((r) => r.seq);
++ assert.strictEqual(first, t1Seq);
++ assert.ok(second > first);
++ // Another creator is not held up by it.
++ const other = A.network.newUser('other-creator');
++ let otherDone = false;
++ let release2;
++ const hold2 = new Promise((ok) => { release2 = ok; });
++ const t3 = A.domain.tx(async (t) => { await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-3-${Date.now()}` }); await hold2; });
++ await new Promise((ok) => setTimeout(ok, 50));
++ const t4 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, other.subject, view, { reason: 'test', dedupe: `seq-order-4-${Date.now()}` })).then(() => { otherDone = true; });
++ await Promise.race([t4, new Promise((ok) => setTimeout(ok, 2000))]);
++ const unblocked = otherDone;
++ release2();
++ await Promise.all([t3, t4]);
++ assert.strictEqual(unblocked, true, 'another creator\'s insert does not wait');
++ });
++
++ await check('the unpaid-checkout limit (3) holds for 6 concurrent requests on two processes', async () => {
++ const buyer = A.network.newUser('buyer');
++ const out = await both(6, (t, k) => t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 100 + k, pay_with: 'checkout', provider: 'stripe' } }));
++ assert.deepStrictEqual(out.map((r) => r.status).sort(), [201, 201, 201, 429, 429, 429], JSON.stringify(out.map((r) => r.json && r.json.code)));
++ assert.ok(out.filter((r) => r.status === 429).every((r) => r.json.code === 'tips.too_many_pending'));
++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1 AND payment_state = 'pending'", [buyer.subject]), 3);
++ });
++
++ await check('two effects workers draining at once deliver each chat line once (leased claims)', async () => {
++ const ids6 = [];
++ for (let k = 0; k < 6; k++) ids6.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 10 + k, message: `m${k}` } })).json.interaction.id);
++ await Promise.all([A.domain.effects.drain(), B.domain.effects.drain(), A.domain.effects.drain(), B.domain.effects.drain()]);
++ const jobs = [...A.adapters.test.jobs, ...B.adapters.test.jobs].filter((j) => ids6.includes(j.interaction.id));
++ const per = new Map();
++ for (const j of jobs) per.set(j.delivery_id, (per.get(j.delivery_id) || 0) + 1);
++ assert.strictEqual(per.size, 6);
++ assert.ok([...per.values()].every((n) => n === 1), JSON.stringify([...per]));
++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND delivery_state = 'delivered'", [ids6]), 6);
++ });
++
++ await check('two processes retrying due transfers at once call Billing once per tip', async () => {
++ billing.state.down = true;
++ const pending = [];
++ for (let k = 0; k < 4; k++) pending.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 20 + k } })).json.interaction.id);
++ billing.state.down = false;
++ const mark = billing.calls.length;
++ A.clock.offset = B.clock.offset = 60_000;
++ await Promise.all([A.domain.interactions.processDueTransfers(), B.domain.interactions.processDueTransfers(), A.domain.interactions.processDueTransfers()]);
++ A.clock.offset = B.clock.offset = 0;
++ const calls = billing.calls.slice(mark).filter((c) => c.url === '/api/v1/transfers');
++ assert.deepStrictEqual(calls.map((c) => c.key).sort(), pending.map((id) => `tips:transfer:${id}`).sort(), 'one call each');
++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND payment_state = 'settled'", [pending]), 4);
++ });
++
++ if (!valkeyAvailable()) {
++ console.log('integration, shared state in Valkey: skipped (OV_TEST_VALKEY_URL not set)');
++ } else {
++ await check('per-actor limits are counted across processes (POST /me/erase: 3 a minute a person)', async () => {
++ const person = A.network.newUser('eraser');
++ const codes = [];
++ for (const t of [A, B, A, B]) codes.push((await t.call('POST', '/api/v1/me/erase', { user: person })).status);
++ assert.deepStrictEqual(codes, [200, 200, 200, 429]);
++ });
++
++ let secret;
++ let cfg;
++ await check('a stream on process A hears a tip, a config change and a revocation made on process B', async () => {
++ cfg = (await B.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', name: 'Main' } })).json.config;
++ const tok = (await B.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'], config_id: cfg.id } })).json;
++ secret = tok.secret;
++ const s = await A.sse(`/overlay/${secret}/events`);
++ await s.waitFor((e) => e.event === 'hello');
++ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 77, message: 'from B' } });
++ const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === r.json.interaction.id);
++ assert.strictEqual(alert.data.amount, 77);
++ await B.call('PATCH', `/api/v1/overlay-configs/${cfg.id}`, { user: alex, body: { duration_ms: 4000 } });
++ const c = await s.waitFor((e) => e.event === 'config');
++ assert.strictEqual(c.data.config.settings.duration_ms, 4000);
++ await B.call('POST', `/api/v1/overlay-tokens/${tok.token.id}/revoke`, { user: alex });
++ assert.ok(await s.waitFor((e) => e.event === 'revoked'));
++ await new Promise((ok) => setTimeout(ok, 100));
++ assert.strictEqual(s.ended, true);
++ });
++
++ await check('overlay stream slots (2 a token) are counted across processes', async () => {
++ const tok = (await A.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts'] } })).json.secret;
++ const s1 = await A.sse(`/overlay/${tok}/events`);
++ const s2 = await B.sse(`/overlay/${tok}/events`);
++ assert.deepStrictEqual([s1.statusCode, s2.statusCode], [200, 200]);
++ assert.strictEqual((await fetch(`${A.base}/overlay/${tok}/events`)).status, 429, 'the third stream, on either process');
++ assert.strictEqual(await B.domain.overlays.connected(alex.subject), 2);
++ s1.close();
++ await new Promise((ok) => setTimeout(ok, 150));
++ const s3 = await B.sse(`/overlay/${tok}/events`);
++ assert.strictEqual(s3.statusCode, 200, 'a closed stream frees its place for the other process');
++ s2.close(); s3.close();
++ });
++
++ await check('flushing Valkey changes no money answer', async () => {
++ const snapshot = async () => ({
++ totals: await A.domain.interactions.totals(db, alex.subject),
++ goal: (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount,
++ receipts: (await B.call('GET', '/api/v1/interactions?limit=200', { user: fan })).json.interactions.map((i) => [i.id, i.payment.state, i.amount]),
++ });
++ const before = await snapshot();
++ let cursor = '0'; let deleted = 0;
++ do {
++ const [next, keys] = await vA.client.scan(cursor, 'MATCH', `${vA.prefix}*`, 'COUNT', 500);
++ cursor = next;
++ if (keys.length) deleted += await vA.client.del(...keys);
++ } while (cursor !== '0');
++ assert.ok(deleted > 0, 'there was shared state to lose');
++ assert.deepStrictEqual(await snapshot(), before);
++ assert.strictEqual(before.totals.settled_via_billing, billing.payable.get(alex.subject), 'and it still reconciles with Billing');
++ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 5 } });
++ assert.strictEqual(r.json.interaction.payment.state, 'settled', 'tips still settle');
++ });
++ }
++
++ await B.close();
++ await A.close();
++ await dbB.close();
++ await store.close();
++ if (vA) { await vA.close(); await vB.close(); }
++ done();
++})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/migrations.test.js b/test/migrations.test.js
nnew file mode 100644
iindex 0000000..bb5f228
---- /dev/null
++++ b/test/migrations.test.js
@@@ -0,0 +1,161 @@
++'use strict';
++// migrations/ (ADR-035, ADR-028): the files parse with their phase, apply in order, and apply again
++// without change; the schema carries every column the SQLite release had (nothing lost in the import);
++// every query shape the code runs can be served by an index; and no runtime file loads better-sqlite3.
++const assert = require('assert');
++const fs = require('fs');
++const path = require('path');
++const Database = require('better-sqlite3');
++const { createDb, sql } = require('openvibe-sdk/db');
++const { MIGRATIONS } = require('../server/db');
++const { TABLES } = require('../scripts/migrate-to-postgres');
++const { check, done } = require('./helpers/app');
++
++const quiet = { log() {} };
++const ROOT = path.join(__dirname, '..');
++
++/** The migration files as the SDK reads them: NNNN_name.sql with a phase header (and after: for a contract). */
++function parse(dir) {
++ return fs.readdirSync(dir).filter((f) => f.endsWith('.sql')).sort().map((file) => {
++ const m = /^(\d{4,})_([a-z0-9_-]+)\.sql$/.exec(file);
++ assert.ok(m, `${file} is named NNNN_description.sql`);
++ const head = fs.readFileSync(path.join(dir, file), 'utf8').split('\n').slice(0, 12).join('\n');
++ const phase = (/^--\s*phase:\s*(expand|migrate|contract)\s*$/m.exec(head) || [])[1];
++ assert.ok(phase, `${file} has a phase header`);
++ return { id: m[1], name: m[2], file, phase, after: (/^--\s*after:\s*(\d{4,})\s*$/m.exec(head) || [])[1] || null, transaction: !/^--\s*no-transaction\s*$/m.test(head) };
++ });
++}
++
++(async () => {
++ const db = createDb({ pglite: true, service: 'tips-test', log: { warn() {}, error: console.error } });
++
++ await check('the files parse: numbered, each with its phase; 0001 is the initial expand', async () => {
++ const files = parse(MIGRATIONS);
++ assert.ok(files.length >= 1);
++ assert.deepStrictEqual([files[0].id, files[0].name, files[0].phase, files[0].transaction], ['0001', 'initial', 'expand', true]);
++ assert.deepStrictEqual(files.map((f) => f.id), [...files.map((f) => f.id)].sort(), 'applied in order');
++ for (const f of files) if (f.phase === 'contract') assert.ok(f.after, `${f.file} names its expand`);
++ });
++
++ await check('they apply once, in order, and a second run changes nothing', async () => {
++ const first = await db.migrate({ dir: MIGRATIONS, log: quiet });
++ assert.deepStrictEqual(first.applied.map((m) => m.id), parse(MIGRATIONS).map((m) => m.id));
++ const schema = async () => db.many(sql`SELECT table_name, column_name, data_type, is_nullable, column_default FROM information_schema.columns
++ WHERE table_schema = 'public' ORDER BY table_name, ordinal_position`);
++ const before = await schema();
++ const second = await db.migrate({ dir: MIGRATIONS, log: quiet });
++ assert.deepStrictEqual([second.applied, second.held], [[], []]);
++ assert.deepStrictEqual(await schema(), before, 'the schema is the same after a second run');
++ assert.strictEqual(await db.value(sql`SELECT count(*) FROM ov_migrations`), parse(MIGRATIONS).length);
++ });
++
++ await check('every table has a primary key; money and time columns have their types', async () => {
++ const noKey = await db.many(sql`SELECT c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
++ WHERE n.nspname = 'public' AND c.relkind = 'r' AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.oid AND i.indisprimary)`);
++ assert.deepStrictEqual(noKey, []);
++ const type = async (t, c) => db.value(sql`SELECT data_type FROM information_schema.columns WHERE table_schema = 'public' AND table_name = ${t} AND column_name = ${c}`);
++ for (const [t, c] of [['tip_interactions', 'amount'], ['tip_interactions', 'reversed_bits'], ['tip_goals', 'target_amount'], ['tip_goal_contributions', 'amount'], ['tip_interactions', 'next_transfer_at'], ['interaction_effects', 'next_attempt_at']]) {
++ assert.strictEqual(await type(t, c), 'bigint', `${t}.${c}`);
++ }
++ for (const [t, c] of [['tip_interactions', 'created_at'], ['tip_interactions', 'settled_at'], ['overlay_deliveries', 'created_at'], ['api_idempotency', 'created_at']]) {
++ assert.strictEqual(await type(t, c), 'timestamp with time zone', `${t}.${c}`);
++ }
++ for (const [t, c] of [['tip_interactions', 'test'], ['tip_interactions', 'transfer_due'], ['creator_tip_profiles', 'page_enabled'], ['overlay_deliveries', 'hidden']]) {
++ assert.strictEqual(await type(t, c), 'boolean', `${t}.${c}`);
++ }
++ for (const [t, c] of [['tip_interactions', 'request'], ['overlay_deliveries', 'payload'], ['tips_event_outbox', 'envelope'], ['overlay_tokens', 'scopes']]) {
++ assert.strictEqual(await type(t, c), 'jsonb', `${t}.${c}`);
++ }
++ assert.strictEqual(await type('api_idempotency', 'response'), 'text', 'replayed byte for byte');
++ });
++
++ await check('nothing the SQLite release stored is left behind: every source table and column has a target', async () => {
++ const s = new Database(':memory:');
++ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));
++ const from = (target) => (TABLES[target] && TABLES[target].from) || target;
++ const targets = new Map();
++ for (const r of await db.many(sql`SELECT table_name, column_name FROM information_schema.columns WHERE table_schema = 'public' AND table_name <> 'ov_migrations'`)) {
++ const src = from(r.table_name);
++ if (!targets.has(src)) targets.set(src, new Set());
++ targets.get(src).add(r.column_name);
++ }
++ const missing = [];
++ for (const { name } of s.prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'").all()) {
++ if (name === 'settings') continue; // not carried on purpose: ov_migrations records the schema (0001_initial.sql)
++ if (!targets.has(name)) { missing.push(name); continue; }
++ for (const c of s.prepare(`PRAGMA table_info(${name})`).all()) if (!targets.get(name).has(c.name)) missing.push(`${name}.${c.name}`);
++ }
++ s.close();
++ assert.deepStrictEqual(missing, [], 'no table or column dropped (dropColumns stays empty)');
++ assert.ok(Object.values(TABLES).every((o) => !o.dropColumns), 'no dropColumns');
++ });
++
++ await check('every query shape the code runs can be served by an index (no sequential scan of its table)', async () => {
++ // With sequential scans switched off for the plan, a shape that has no usable index still plans a
++ // Seq Scan (at a prohibitive cost): that is what this looks for.
++ const shapes = {
++ 'interactions of a creator, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND (created_at, id) < (now(), 'x') ORDER BY created_at DESC, id DESC LIMIT 51`],
++ 'receipts of a supporter, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE supporter_subject = 's' AND NOT test ORDER BY created_at DESC, id DESC LIMIT 51`],
++ 'the moderation queue by state': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state IN ('settled', 'reversed', 'pending') AND moderation = 'held' ORDER BY created_at DESC, id DESC LIMIT 51`],
++ 'unpaid checkouts of a supporter': ['tip_interactions', sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = 's' AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= now()`],
++ 'due transfers (the lease claim)': ['tip_interactions', sql`SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= 5 ORDER BY next_transfer_at LIMIT 10 FOR UPDATE SKIP LOCKED`],
++ 'the supporters leaderboard': ['tip_interactions', sql`SELECT supporter_subject, SUM(amount - reversed_bits) FROM tip_interactions WHERE creator_subject = 'c' AND supporter_subject IS NOT NULL AND NOT test
++ AND payment_state IN ('settled', 'reversed') AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible' GROUP BY supporter_subject`],
++ 'recent public messages': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state = 'settled' AND message IS NOT NULL AND NOT test ORDER BY settled_at DESC, id DESC LIMIT 20`],
++ 'an interaction by Billing transaction': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_txn_id = 't'`],
++ 'an interaction by checkout intent': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_intent_id = 'i'`],
++ 'an interaction by provider reference': ['tip_interactions', sql`SELECT 1 FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = 'r'`],
++ 'goals of a creator': ['tip_goals', sql`SELECT * FROM tip_goals WHERE creator_subject = 'c' AND status = 'active' ORDER BY sort_order, created_at`],
++ 'goal totals': ['tip_goal_contributions', sql`SELECT goal_id, SUM(amount - reversed_amount) FROM tip_goal_contributions WHERE goal_id = ANY('{a,b}'::text[]) GROUP BY goal_id`],
++ 'the latest supporters of a goal': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE goal_id = 'g' ORDER BY id DESC LIMIT 10`],
++ 'the contributions of an interaction': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = 'i'`],
++ 'due effects (the lease claim)': ['interaction_effects', sql`SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= 5 ORDER BY id LIMIT 20 FOR UPDATE SKIP LOCKED`],
++ 'the effects of a page of interactions': ['interaction_effects', sql`SELECT * FROM interaction_effects WHERE interaction_id = ANY('{a,b}'::text[]) ORDER BY interaction_id, id`],
++ 'an overlay stream after a seq': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE creator_subject = 'c' AND seq > 5 AND status <> 'failed' AND NOT hidden ORDER BY seq LIMIT 200`],
++ 'the delivery window sweep': ['overlay_deliveries', sql`SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < now() ORDER BY seq LIMIT 500`],
++ 'the deliveries of an interaction': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE interaction_id = 'i'`],
++ 'overlay tokens of a creator': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE creator_subject = 'c' ORDER BY revoked_at IS NOT NULL, created_at DESC`],
++ 'an overlay token by its secret': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE token_hash = 'h'`],
++ 'overlay configs of a creator': ['overlay_configs', sql`SELECT * FROM overlay_configs WHERE creator_subject = 'c' ORDER BY created_at`],
++ 'a creator by handle': ['creator_tip_profiles', sql`SELECT * FROM creator_tip_profiles WHERE handle = 'h'`],
++ 'switched-on pages (home, sitemap)': ['creator_tip_profiles', sql`SELECT handle FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`],
++ 'the creators a person moderates': ['tip_moderators', sql`SELECT * FROM tip_moderators WHERE moderator_subject = 's' AND removed_at IS NULL`],
++ 'open invitations of a creator': ['tip_moderator_invites', sql`SELECT * FROM tip_moderator_invites WHERE creator_subject = 'c' AND used_at IS NULL ORDER BY created_at`],
++ 'the moderation log of a creator': ['tip_moderation_log', sql`SELECT * FROM tip_moderation_log WHERE creator_subject = 'c' ORDER BY id DESC LIMIT 100`],
++ 'the weekly prune of stored answers': ['api_idempotency', sql`DELETE FROM api_idempotency WHERE created_at < now()`],
++ 'an erasure scrubbing outbox copies': ['tips_event_outbox', sql`SELECT id FROM tips_event_outbox WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY('{a}'::text[])`],
++ 'the rejected-events count': ['tips_event_outbox', sql`SELECT count(*) FROM tips_event_outbox WHERE rejected_at IS NOT NULL`],
++ 'a Live goal through the import map': ['migration_maps', sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = '12'`],
++ };
++ const scans = [];
++ for (const [name, [table, q]] of Object.entries(shapes)) {
++ const plan = await db.tx(async (t) => {
++ await t.query('SET LOCAL enable_seqscan = off');
++ return t.value(sql`EXPLAIN (FORMAT JSON) ${q}`);
++ });
++ const nodes = [];
++ const walk = (n) => { nodes.push(n); (n.Plans || []).forEach(walk); };
++ walk(plan[0].Plan);
++ if (nodes.some((n) => n['Node Type'] === 'Seq Scan' && n['Relation Name'] === table)) scans.push(name);
++ }
++ assert.deepStrictEqual(scans, []);
++ });
++
++ await check('no runtime file loads better-sqlite3 (only the import tools may)', async () => {
++ const offenders = [];
++ const walk = (dir) => {
++ for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
++ const p = path.join(dir, e.name);
++ if (e.isDirectory()) walk(p);
++ else if (e.name.endsWith('.js') && /require\(['"]better-sqlite3['"]\)|\.prepare\(|db\.transaction\(/.test(fs.readFileSync(p, 'utf8').replace(/(live|bdb|snap)\.prepare\(/g, ''))) offenders.push(path.relative(ROOT, p));
++ }
++ };
++ walk(path.join(ROOT, 'server'));
++ assert.deepStrictEqual(offenders, []);
++ const loaders = fs.readdirSync(path.join(ROOT, 'scripts')).filter((f) => /require\(['"]better-sqlite3['"]\)/.test(fs.readFileSync(path.join(ROOT, 'scripts', f), 'utf8')));
++ assert.deepStrictEqual(loaders, ['import-live.js'], "the Live import reads Live's and Billing's snapshot files; migrate-to-postgres.js reads Tips' through importSqlite");
++ });
++
++ await db.close();
++ done();
++})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/moderation.test.js b/test/moderation.test.js
iindex bd0f320..cb7a90e 100644
---- a/test/moderation.test.js
++++ b/test/moderation.test.js
@@@ -29,7 +29,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const pay = (kind, body) => t.call('POST', { tip: '/api/v1/checkout', paid_message: '/api/v1/paid-messages', tts: '/api/v1/tts-requests' }[kind], { user: viewer, body: { creator: 'alex', ...body } });
const state = async () => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts;
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
-- const moderated = (id) => t.outboxRows('tips.interaction.moderated').filter((e) => e.subject.id === id).map((e) => e.payload);
++ const moderated = async (id) => (await t.outboxRows('tips.interaction.moderated')).filter((e) => e.subject.id === id).map((e) => e.payload);
const cookie = (u) => `ov_token=${t.network.signUser(u)}`;
const get = (p, u) => fetch(t.base + p, { headers: u ? { Cookie: cookie(u) } : {}, redirect: 'manual' }).then(async (r) => ({ status: r.status, headers: r.headers, text: await r.text() }));
const post = (p, u, form) => fetch(t.base + p, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie(u) }, body: new URLSearchParams(form).toString() })
@@@ -62,7 +62,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.match(own.message, /BADWORD/, 'the creator reads the original');
assert.strictEqual(own.moderation.filtered, true);
assert.strictEqual(own.moderation.state, 'visible');
-- const [ev] = moderated(id);
++ const [ev] = await moderated(id);
assert.deepStrictEqual(ev, { interaction_id: id, creator: { type: 'user', id: alex.subject }, action: 'filtered', by: 'filter', moderation_state: 'visible', cancelled_effects: [] });
const tts = await pay('tts', { amount: 150, tts: { text: 'say badword now please' } });
await domain.effects.drain();
@@@ -74,7 +74,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
let heldId;
await check('hold: nothing is shown, posted or read until someone reviews it; the money counts at once', async () => {
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { action: 'hold' } } });
-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
++ const goalBefore = await goalNow();
const s = await t.sse(`/overlay/${secret}/events`);
await s.waitFor((e) => e.event === 'hello');
const r = await pay('tts', { amount: 300, supporter_name: 'Viewer', tts: { text: 'read this badword aloud' } });
@@@ -85,13 +86,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
s.close();
assert.ok(!(await state()).some((x) => x.interaction_id === heldId));
assert.strictEqual(jobsOf(heldId).length, 0, 'no chat line, no TTS');
-- const i = domain.interactions.get(heldId);
++ const i = await domain.interactions.get(t.db, heldId);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.moderation, 'held');
assert.strictEqual(i.delivery_state, 'queued', 'waiting for review, not "delivered"');
-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 300);
-- assert.strictEqual(t.outboxRows('tips.interaction.ready').filter((e) => e.subject.id === heldId).length, 1);
-- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);
++ assert.strictEqual(await goalNow(), goalBefore + 300);
++ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === heldId).length, 1);
++ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);
const q = await t.call('GET', '/api/v1/moderation?creator=alex&state=held', { user: alex });
assert.deepStrictEqual(q.json.interactions.map((x) => x.id), [heldId]);
assert.strictEqual(q.json.interactions[0].tts_text, 'read this badword aloud', 'reviewers read what was written');
@@@ -108,7 +109,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const url = inv.text.match(/value="(http:\/\/tips\.test\/moderate\/invite\/tmin_[^"]+)"/)[1];
const path = url.replace('http://tips.test', '');
assert.ok(!(await get('/dashboard', alex)).text.includes(url), 'shown once');
-- assert.ok(!JSON.stringify(t.db.prepare('SELECT * FROM tip_moderator_invites').all()).includes(path.split('/').pop()), 'stored hashed');
++ assert.ok(!JSON.stringify(await t.db.many('SELECT * FROM tip_moderator_invites')).includes(path.split('/').pop()), 'stored hashed');
assert.match((await get(path)).headers.get('location'), /^\/auth\/login/);
const page = await get(path, mod);
assert.strictEqual(page.status, 200);
@@@ -139,10 +140,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.strictEqual(alert.data.tts.text, 'read this aloud', 'blocked words stay out of what is read');
await domain.effects.drain();
assert.deepStrictEqual(jobsOf(heldId).map((j) => j.effect).sort(), ['chat_line', 'tts']);
-- const i = domain.interactions.get(heldId);
++ const i = await domain.interactions.get(t.db, heldId);
assert.strictEqual(i.moderation, 'visible');
assert.strictEqual(i.delivery_state, 'delivered');
-- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);
++ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);
});
let hiddenId;
@@@ -154,7 +155,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r = await pay('tts', { amount: 500, supporter_name: 'Viewer', message: 'nasty but not blocked', tts: { text: 'nasty but not blocked' } });
hiddenId = r.json.interaction.id;
const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === hiddenId);
-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
++ const goalBefore = await goalNow();
const h = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: mod, body: { reason: 'abuse' } });
assert.strictEqual(h.status, 200, h.text);
assert.strictEqual(h.json.changed, true);
@@@ -164,11 +166,11 @@ const validModerated = payloadValidator('tips.interaction.moderated');
s.close();
await domain.effects.drain();
assert.strictEqual(jobsOf(hiddenId).length, 0, 'nothing was posted or read');
-- const i = domain.interactions.get(hiddenId);
++ const i = await domain.interactions.get(t.db, hiddenId);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'cancelled');
assert.strictEqual(billing.payable.get(alex.subject), payableBefore + 500, 'the payment stands');
-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore, 'the goal keeps it');
++ assert.strictEqual(await goalNow(), goalBefore, 'the goal keeps it');
assert.ok(!(await state()).some((x) => x.interaction_id === hiddenId));
const replay = await t.sse(`/overlay/${secret}/events`, { lastEventId: alert.id - 1 });
await replay.waitFor((e) => e.event === 'hello');
@@@ -177,13 +179,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
replay.close();
const sup = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json;
assert.ok(!sup.recent.some((v) => v.interaction_id === hiddenId));
-- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');
-- const [ev] = moderated(hiddenId);
++ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');
++ const [ev] = await moderated(hiddenId);
assert.deepStrictEqual({ ...ev, cancelled_effects: ev.cancelled_effects.sort() }, { interaction_id: hiddenId, creator: { type: 'user', id: alex.subject }, action: 'hidden', by: 'moderator', moderation_state: 'hidden', cancelled_effects: ['chat_line', 'tts'] });
assert.ok(!JSON.stringify(ev).includes('abuse') && !JSON.stringify(ev).includes(mod.subject), 'the event carries no note and no moderator');
const again = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: alex });
assert.strictEqual(again.json.changed, false);
-- assert.strictEqual(moderated(hiddenId).length, 1, 'hiding twice publishes once');
++ assert.strictEqual((await moderated(hiddenId)).length, 1, 'hiding twice publishes once');
const receipt = (await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: viewer })).json.interaction;
assert.strictEqual(receipt.moderation.state, 'hidden');
assert.strictEqual(receipt.payment.state, 'settled');
@@@ -196,7 +198,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
await domain.effects.drain();
assert.strictEqual(jobsOf(hiddenId).length, 0, 'no late TTS');
assert.ok((await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.recent.some((v) => v.interaction_id === hiddenId));
-- assert.deepStrictEqual(moderated(hiddenId).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);
++ assert.deepStrictEqual((await moderated(hiddenId)).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);
});
await check('who may moderate: the creator, their moderators, services with tips.interaction.moderate; nobody else', async () => {
@@@ -208,7 +210,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.get'] })).status, 403);
const svc = await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.moderate'], sub: 'svc:chat' });
assert.strictEqual(svc.status, 200, svc.text);
-- assert.strictEqual(moderated(id)[0].by, 'service');
++ assert.strictEqual((await moderated(id))[0].by, 'service');
assert.strictEqual((await get('/moderate/alex', stranger)).status, 404);
assert.strictEqual((await t.call('GET', '/api/v1/moderation?creator=alex', { user: stranger })).status, 403);
const log = await t.call('GET', '/api/v1/moderation/log?creator=alex', { user: alex });
@@@ -227,10 +229,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r = await t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 150, message: 'later', pay_with: 'checkout', provider: 'stripe' } });
const id = r.json.interaction.id;
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { user: alex })).status, 200);
-- await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));
++ await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));
await new Promise((ok) => setTimeout(ok, 50));
await domain.interactions.processDueTransfers();
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'cancelled');
await domain.effects.drain();
@@@ -242,7 +244,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { words: ['badword'], action: 'hold' } } });
const r = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'a badword test' } });
assert.strictEqual(r.json.interaction.moderation.state, 'held');
-- assert.strictEqual(moderated(r.json.interaction.id).length, 0);
++ assert.strictEqual((await moderated(r.json.interaction.id)).length, 0);
const shown = await t.call('POST', `/api/v1/interactions/${r.json.interaction.id}/restore`, { user: alex });
assert.strictEqual(shown.json.interaction.moderation.state, 'visible');
assert.strictEqual((await state()).find((x) => x.interaction_id === r.json.interaction.id).message, 'a ******* test');
@@@ -250,16 +252,16 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r2 = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'another badword' } });
const id2 = r2.json.interaction.id;
await t.call('POST', `/api/v1/interactions/${id2}/hide`, { user: alex });
-- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'cancelled');
++ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'cancelled');
await t.call('POST', `/api/v1/interactions/${id2}/restore`, { user: alex });
await domain.effects.drain();
-- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'delivered');
++ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'delivered');
assert.strictEqual(jobsOf(id2).length, 1);
assert.ok((await state()).some((x) => x.interaction_id === id2));
});
await check('every event is a valid envelope with a valid payload', async () => {
-- for (const e of t.outboxRows()) {
++ for (const e of await t.outboxRows()) {
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
const v = e.event_type === 'tips.interaction.moderated' ? validModerated(e.payload) : payloadValidator(e.event_type)(e.payload);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)} ${JSON.stringify(e.payload)}`);
ddiff --git a/test/observability.test.js b/test/observability.test.js
iindex 0f5bcb7..7f76cf0 100644
---- a/test/observability.test.js
++++ b/test/observability.test.js
@@@ -1,6 +1,7 @@
'use strict';
// Track O: GET /metrics answers direct loopback callers only, labels requests by route template and
--// carries the Tips gauges; /api/ready is 503 only when the database fails, and a missing Billing,
++// carries the Tips gauges; /api/ready is 503 only when the database fails, reports the store that
++// answered (db.ready()) and Valkey (valkey.ready(), skipped without VALKEY_URL), and a missing Billing,
// Events or Network key degrades it instead.
const assert = require('assert');
const nodeHttp = require('http');
@@@ -21,13 +22,17 @@ function get(base, p, headers = {}) {
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.json.status, 'ready');
assert.strictEqual(r.json.service, 'tips');
-- assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'network_jwks', 'billing', 'events']);
++ assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'valkey', 'network_jwks', 'billing', 'events']);
for (const [name, c] of Object.entries(r.json.checks)) {
-- assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);
++ // Without Valkey: skipped with its reason, never claimed "ok"; with it, a real PING.
++ if (name === 'valkey' && !t.valkey) { assert.strictEqual(c.status, 'skipped', JSON.stringify(c)); assert.match(c.reason, /VALKEY_URL unset/); assert.deepStrictEqual(r.json.skipped, ['valkey']); }
++ else assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);
assert.strictEqual(c.required, name === 'db', name);
assert.strictEqual(typeof c.latency_ms, 'number');
assert.ok(Date.parse(c.checked_at));
}
++ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered, from a real round trip');
++ assert.ok(r.json.checks.db.detail.pool && typeof r.json.checks.db.detail.pool.total === 'number');
assert.ok(r.json.events_outbox && r.json.pending_deliveries);
assert.strictEqual(r.headers.get('cache-control'), 'no-store');
});
@@@ -80,7 +85,7 @@ function get(base, p, headers = {}) {
await check('a broken database makes the service unready (503); /metrics still answers', async () => {
const d = await boot();
-- d.db.close();
++ await d.db.close();
const r = await d.call('GET', '/api/ready', { token: null });
assert.strictEqual(r.status, 503, r.text);
assert.strictEqual(r.json.ready, false);
ddiff --git a/test/overlays.test.js b/test/overlays.test.js
iindex 46b6d16..f375e0c 100644
---- a/test/overlays.test.js
++++ b/test/overlays.test.js
@@@ -21,7 +21,7 @@ const { boot, check, done } = require('./helpers/app');
tokenId = r.json.token.id;
assert.match(secret, /^tovl_[A-Za-z0-9_-]{43}$/);
assert.strictEqual(r.json.overlay_url, `http://tips.test/overlay/${secret}`);
-- const row = t.db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(tokenId);
++ const row = await t.db.maybe('SELECT * FROM overlay_tokens WHERE id = $1', [tokenId]);
assert.notStrictEqual(row.token_hash, secret);
assert.ok(!JSON.stringify(row).includes(secret), 'the secret is not stored');
const list = await t.call('GET', '/api/v1/overlay-tokens', { user: alex });
@@@ -48,16 +48,17 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(g.data.goal.id, goal.id);
firstAlertSeq = alert.id;
s.close();
-- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r.json.interaction.id);
++ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r.json.interaction.id]);
assert.strictEqual(d.status, 'delivered');
-- assert.strictEqual(t.outboxRows('tips.overlay.delivered').filter((e) => e.payload.delivery_id === d.id).length, 1);
++ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).filter((e) => e.payload.delivery_id === d.id).length, 1);
});
await check('overlay replay (Last-Event-ID) resends but never charges, re-counts or re-emits', async () => {
++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
const before = {
-- transfers: billing.transfers().length, contributions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n,
-- goal: domain.goals.present(domain.goals.get(goal.id)).current_amount, delivered: t.outboxRows('tips.overlay.delivered').length,
-- interactions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, payable: billing.payable.get(alex.subject),
++ transfers: billing.transfers().length, contributions: await t.db.value('SELECT count(*) FROM tip_goal_contributions'),
++ goal: await goalNow(), delivered: (await t.outboxRows('tips.overlay.delivered')).length,
++ interactions: await t.db.value('SELECT count(*) FROM tip_interactions'), payable: billing.payable.get(alex.subject),
};
for (let k = 0; k < 3; k++) {
const s = await t.sse(`/overlay/${secret}/events`, { lastEventId: firstAlertSeq - 1 });
@@@ -66,12 +67,12 @@ const { boot, check, done } = require('./helpers/app');
s.close();
}
assert.strictEqual(billing.transfers().length, before.transfers);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n, before.contributions);
-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal);
-- assert.strictEqual(t.outboxRows('tips.overlay.delivered').length, before.delivered);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before.interactions);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions'), before.contributions);
++ assert.strictEqual(await goalNow(), before.goal);
++ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).length, before.delivered);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before.interactions);
assert.strictEqual(billing.payable.get(alex.subject), before.payable);
-- assert.ok(t.db.prepare('SELECT sends FROM overlay_deliveries WHERE seq = ?').get(firstAlertSeq).sends >= 4);
++ assert.ok(await t.db.value('SELECT sends FROM overlay_deliveries WHERE seq = $1', [firstAlertSeq]) >= 4);
});
await check('a tip while no overlay is connected waits, is delivered on connect, else fails after the window', async () => {
@@@ -83,20 +84,21 @@ const { boot, check, done } = require('./helpers/app');
await new Promise((ok) => setTimeout(ok, 50));
const r2 = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 12 } });
t.clock.offset += 11 * 60 * 1000;
-- const n = domain.overlays.sweepFailed();
++ const n = await domain.overlays.sweepFailed();
t.clock.offset = 0;
assert.ok(n >= 1);
-- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r2.json.interaction.id);
++ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r2.json.interaction.id]);
assert.strictEqual(d.status, 'failed');
-- assert.strictEqual(t.outboxRows('tips.overlay.failed').filter((e) => e.payload.delivery_id === d.id).length, 1);
++ assert.strictEqual((await t.outboxRows('tips.overlay.failed')).filter((e) => e.payload.delivery_id === d.id).length, 1);
// The payment is untouched by the overlay failure.
-- assert.strictEqual(domain.interactions.get(r2.json.interaction.id).payment_state, 'settled');
++ assert.strictEqual((await domain.interactions.get(t.db, r2.json.interaction.id)).payment_state, 'settled');
});
await check('simulation runs the full path with test = 1, no Billing call, never counted', async () => {
++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
const before = {
-- transfers: billing.calls.length, goal: domain.goals.present(domain.goals.get(goal.id)).current_amount,
-- totals: domain.interactions.totals(alex.subject), events: t.outboxRows().length,
++ transfers: billing.calls.length, goal: await goalNow(),
++ totals: await domain.interactions.totals(t.db, alex.subject), events: (await t.outboxRows()).length,
};
const s = await t.sse(`/overlay/${secret}/events`);
await s.waitFor((e) => e.event === 'hello');
@@@ -114,13 +116,13 @@ const { boot, check, done } = require('./helpers/app');
const job = t.adapters.test.jobs.find((j) => j.interaction.id === i.id);
assert.ok(job && job.test === true, 'chat effect delivered to the test adapter, flagged test');
assert.strictEqual(billing.calls.length, before.transfers, 'no Billing call');
-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal, 'goal unchanged');
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);
-- const after = domain.interactions.totals(alex.subject);
++ assert.strictEqual(await goalNow(), before.goal, 'goal unchanged');
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);
++ const after = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(after.settled_via_billing, before.totals.settled_via_billing);
assert.strictEqual(after.interactions, before.totals.interactions);
assert.strictEqual(after.simulations, before.totals.simulations + 1);
-- assert.strictEqual(t.outboxRows().length, before.events, 'no durable events for a simulation');
++ assert.strictEqual((await t.outboxRows()).length, before.events, 'no durable events for a simulation');
const denied = await t.call('POST', '/api/v1/simulate', { user: viewer, body: { creator: 'alex' } });
assert.strictEqual(denied.status, 403);
});
ddiff --git a/test/pages.test.js b/test/pages.test.js
iindex 06946ad..1693959 100644
---- a/test/pages.test.js
++++ b/test/pages.test.js
@@@ -82,7 +82,7 @@ const { boot, check, done } = require('./helpers/app');
const loc = r.headers.get('location');
assert.match(loc, /^\/receipts\/tint_/);
const id = loc.split('/').pop();
-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
// Double submit (same form nonce): the same interaction, no second transfer.
const before = billing.transfers().length;
const again = await post('/alex/tip', viewer, form);
@@@ -106,7 +106,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r2.status, 403, 'a token minted for someone else does not work');
const d = await post('/dashboard/profile', alex, { csrf: 'nope', page_enabled: '0' });
assert.strictEqual(d.status, 403);
-- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, true);
++ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, true);
});
await check('the form reports Billing\'s refusal in words; checkout redirects to the provider', async () => {
@@@ -138,10 +138,10 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(g.status, 303);
const sim = await post('/dashboard/simulate', alex, { csrf, idem: 'x', kind: 'tip', amount: '100', supporter_name: 'Test supporter', message: 'test' });
assert.strictEqual(sim.status, 303);
-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 1").get(alex.subject).n, 1);
-- const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String(domain.profiles.byHandle('alex').revision) });
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE creator_subject = $1 AND test', [alex.subject]), 1);
++ const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String((await domain.profiles.byHandle(t.db, 'alex')).revision) });
assert.strictEqual(off.status, 303);
-- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, false, 'an unchecked box switches the page off');
++ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, false, 'an unchecked box switches the page off');
assert.strictEqual((await get('/alex')).status, 404);
});
ddiff --git a/test/privacy.test.js b/test/privacy.test.js
iindex f921c7c..a26b9ed 100644
---- a/test/privacy.test.js
++++ b/test/privacy.test.js
@@@ -28,7 +28,7 @@ function payloadValidator(type) {
const tip = (user, body) => t.call('POST', '/api/v1/checkout', { user, body: { creator: 'alex', ...body } });
const alertOf = async (id) => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts.find((a) => a.interaction_id === id);
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
-- const eventsOf = (type, id) => t.outboxRows(type).filter((e) => e.subject.id === id);
++ const eventsOf = async (type, id) => (await t.outboxRows(type)).filter((e) => e.subject.id === id);
let anonId;
await check('anonymous: "Anonymous" to the creator, overlays, chat and events; the supporter keeps their receipt', async () => {
@@@ -46,19 +46,19 @@ function payloadValidator(type) {
assert.strictEqual(a.data.supporter_name, 'Anonymous');
assert.strictEqual(a.data.message, 'from nobody');
assert.ok(!JSON.stringify(a.data).includes('Viewer Real Name') && !JSON.stringify(a.data).includes(viewer.subject));
-- const g = t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(anonId);
-- assert.strictEqual(JSON.parse(g.payload).by, 'Anonymous');
++ const g = await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [anonId]);
++ assert.strictEqual(g.payload.by, 'Anonymous');
await domain.effects.drain();
const [job] = jobsOf(anonId);
assert.deepStrictEqual(job.supporter, { name: 'Anonymous', subject: null });
assert.strictEqual(job.text, 'Anonymous tipped 120 Vibes: from nobody');
-- const [ready] = eventsOf('tips.interaction.ready', anonId);
++ const [ready] = await eventsOf('tips.interaction.ready', anonId);
assert.strictEqual(ready.payload.supporter, null);
assert.strictEqual(ready.payload.supporter_name, 'Anonymous');
assert.ok(contracts.validate('tips.interaction.ready', ready.payload).valid);
-- assert.ok(!JSON.stringify(t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');
++ assert.ok(!JSON.stringify(await t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');
for (const as of [{ user: alex }, { cap: ['tips.interaction.get'] }]) {
const v = (await t.call('GET', `/api/v1/interactions/${anonId}`, as)).json.interaction;
@@@ -80,7 +80,7 @@ function payloadValidator(type) {
assert.strictEqual(a.amount, null);
assert.strictEqual(a.amount_hidden, true);
assert.strictEqual(a.supporter_name, 'Viewer');
-- assert.strictEqual(JSON.parse(t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(hiddenId).payload).by, null,
++ assert.strictEqual((await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [hiddenId])).payload.by, null,
'the goal update does not pin the jump on them');
await domain.effects.drain();
const [job] = jobsOf(hiddenId);
@@@ -88,7 +88,7 @@ function payloadValidator(type) {
assert.strictEqual(job.interaction.amount, null);
assert.strictEqual(job.privacy.hide_amount, true);
assert.strictEqual((await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: alex })).json.interaction.amount, 222);
-- assert.strictEqual(eventsOf('tips.interaction.ready', hiddenId)[0].payload.amount, 222);
++ assert.strictEqual((await eventsOf('tips.interaction.ready', hiddenId))[0].payload.amount, 222);
});
await check('an overlay\'s minimum still applies to a hidden amount (without revealing it)', async () => {
@@@ -183,10 +183,10 @@ function payloadValidator(type) {
body: new URLSearchParams({ csrf, idem: 'form-privacy-00001', kind: 'tip', amount: '15', message: 'shh', pay_with: 'credit', supporter_name: 'Other', anonymous: '1', private_message: '1' }).toString(),
});
assert.strictEqual(res.status, 303);
-- const i = domain.interactions.get(res.headers.get('location').split('/').pop());
-- assert.strictEqual(i.anonymous, 1);
-- assert.strictEqual(i.private_message, 1);
-- assert.strictEqual(i.hide_amount, 0);
++ const i = await domain.interactions.get(t.db, res.headers.get('location').split('/').pop());
++ assert.strictEqual(i.anonymous, true);
++ assert.strictEqual(i.private_message, true);
++ assert.strictEqual(i.hide_amount, false);
});
await check('export: a supporter downloads their own tips, with messages and choices, nobody else\'s', async () => {
@@@ -207,14 +207,15 @@ function payloadValidator(type) {
});
await check('erasure: the person goes, the money record stays; tips.interaction.erased redacts the earlier events', async () => {
-- const totalsBefore = domain.interactions.totals(alex.subject);
-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
++ const totalsBefore = await domain.interactions.totals(t.db, alex.subject);
++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
++ const goalBefore = await goalNow();
const buyer = t.network.newUser('buyer');
billing.fund(buyer.subject, 1000);
const kept = await tip(buyer, { amount: 150, pay_with: 'checkout', provider: 'stripe', message: 'pending one' });
const done1 = await tip(buyer, { amount: 40, message: 'my secret words', supporter_name: 'Buyer Person' });
-- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM api_idempotency WHERE key LIKE ?').get(`${buyer.subject}:%`).n >= 2);
-- const unsent = eventsOf('tips.interaction.ready', done1.json.interaction.id)[0];
++ assert.ok(await t.db.value('SELECT count(*) FROM api_idempotency WHERE key LIKE $1', [`${buyer.subject}:%`]) >= 2);
++ const unsent = (await eventsOf('tips.interaction.ready', done1.json.interaction.id))[0];
assert.strictEqual(unsent.payload.supporter.id, buyer.subject);
const r = await t.call('POST', '/api/v1/me/erase', { user: buyer });
@@@ -223,36 +224,36 @@ function payloadValidator(type) {
assert.strictEqual(r.json.kept_pending, 1);
assert.ok(r.json.stored_answers_removed >= 2);
-- const i = domain.interactions.get(done1.json.interaction.id);
++ const i = await domain.interactions.get(t.db, done1.json.interaction.id);
assert.strictEqual(i.supporter_subject, null);
assert.strictEqual(i.supporter_name, null);
assert.strictEqual(i.message, null);
assert.ok(i.erased_at);
assert.strictEqual(i.amount, 40);
assert.ok(i.billing_txn_id, 'the Billing reference stays');
-- assert.strictEqual(domain.interactions.get(kept.json.interaction.id).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');
++ assert.strictEqual((await domain.interactions.get(t.db, kept.json.interaction.id)).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');
const a = await alertOf(i.id);
assert.strictEqual(a.supporter_name, 'Anonymous');
assert.strictEqual(a.message, null);
-- const row = t.db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(i.id);
++ const row = await t.db.maybe('SELECT * FROM tip_interactions WHERE id = $1', [i.id]);
assert.ok(!JSON.stringify(row).includes('my secret words') && !JSON.stringify(row).includes('Buyer Person') && !JSON.stringify(row).includes(buyer.subject));
-- const everything = JSON.stringify(t.db.prepare('SELECT * FROM paid_messages').all()) + JSON.stringify(t.db.prepare('SELECT payload FROM overlay_deliveries').all())
-- + JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all());
++ const everything = JSON.stringify(await t.db.many('SELECT * FROM paid_messages')) + JSON.stringify(await t.db.many('SELECT payload FROM overlay_deliveries'))
++ + JSON.stringify(await t.db.many('SELECT * FROM api_idempotency'));
assert.ok(!everything.includes('my secret words') && !everything.includes('Buyer Person'));
-- const scrubbed = eventsOf('tips.interaction.ready', i.id)[0];
++ const scrubbed = (await eventsOf('tips.interaction.ready', i.id))[0];
assert.strictEqual(scrubbed.payload.supporter, null, 'the local outbox copy no longer names them');
assert.strictEqual(scrubbed.payload.supporter_name, 'Anonymous');
-- const [erased] = eventsOf('tips.interaction.erased', i.id);
++ const [erased] = await eventsOf('tips.interaction.erased', i.id);
assert.deepStrictEqual(erased.payload.redacts, { subject_type: 'interaction', subject_ids: [i.id] });
const v = payloadValidator('tips.interaction.erased')(erased.payload);
assert.ok(v.valid, JSON.stringify(v.errors));
assert.ok(contracts.validate('events.event-envelope@1', erased).valid);
-- const totalsAfter = domain.interactions.totals(alex.subject);
++ const totalsAfter = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totalsAfter.settled_via_billing, totalsBefore.settled_via_billing + 40, 'the erased tip still counts');
assert.strictEqual(totalsAfter.settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 40, 'the goal keeps the money');
++ assert.strictEqual(await goalNow(), goalBefore + 40, 'the goal keeps the money');
const receipts = (await t.call('GET', '/api/v1/interactions', { user: buyer })).json.interactions;
assert.deepStrictEqual(receipts.map((x) => x.id), [kept.json.interaction.id]);
});
@@@ -265,17 +266,17 @@ function payloadValidator(type) {
const post = (form) => fetch(`${t.base}/receipts/erase`, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie }, body: new URLSearchParams(form).toString() });
assert.strictEqual((await post({ csrf, idem: 'form-erase-000001' })).status, 403, 'no confirmation');
assert.strictEqual((await post({ csrf: 'forged', idem: 'form-erase-000002', confirm: '1' })).status, 403);
-- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n > 0);
++ assert.ok(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]) > 0);
const ok = await post({ csrf, idem: 'form-erase-000003', confirm: '1' });
assert.strictEqual(ok.status, 303);
assert.match(ok.headers.get('location'), /^\/receipts\?done=/);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n, 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]), 0);
const lb = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.leaderboard;
assert.ok(!lb.some((x) => x.name === 'Other'), 'an erased supporter leaves the leaderboard');
});
await check('every event is a valid envelope with a valid payload', async () => {
-- for (const e of t.outboxRows()) {
++ for (const e of await t.outboxRows()) {
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
const v = payloadValidator(e.event_type)(e.payload);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)}`);
ddiff --git a/test/query-budget.test.js b/test/query-budget.test.js
nnew file mode 100644
iindex 0000000..a26f40d
---- /dev/null
++++ b/test/query-budget.test.js
@@@ -0,0 +1,65 @@
++'use strict';
++// The hot list routes run a bounded number of queries, whatever the page size (no N+1): the receipts
++// and creator lists (GET /api/v1/interactions, the hottest), goals, the supporters page and an
++// overlay's /state. Counted on the database handle (db.stats().queries) around one request each.
++const assert = require('assert');
++const { boot, check, done } = require('./helpers/app');
++
++(async () => {
++ const t = await boot();
++ const { billing } = t;
++ const alex = await t.creator('alex', { settings: { page: { supporters_page: true, supporters_messages: true, goal_supporters: true } } });
++ const fans = ['ann', 'bob', 'cat', 'dan', 'eve'].map((n) => t.network.newUser(n));
++ for (const f of fans) billing.fund(f.subject, 10_000);
++ const tip = (f, body) => t.call('POST', '/api/v1/checkout', { user: f, body: { creator: 'alex', amount: 10, ...body } });
++ const goals = [];
++ for (const title of ['Desk', 'Chair', 'Lamp']) goals.push((await t.call('POST', '/api/v1/goals', { user: alex, body: { title, target_amount: 5000 } })).json.goal);
++ // Several tips per supporter, messages, goal contributions: every list has many rows.
++ for (let k = 0; k < 3; k++) for (const f of fans) assert.strictEqual((await tip(f, { message: `tip ${k} from ${f.username}`, goal_id: goals[k].id })).status, 201);
++ const token = (await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'] } })).json.secret;
++
++ /** Queries one request runs. */
++ async function queries(method, p, opts) {
++ const before = t.db.stats().queries;
++ const r = await t.call(method, p, opts);
++ assert.strictEqual(r.status, 200, `${p}: ${r.text}`);
++ return { n: t.db.stats().queries - before, r };
++ }
++
++ await check('GET /api/v1/interactions (the creator\'s list, with effects): 3 queries for 1 row or 15', async () => {
++ const one = await queries('GET', '/api/v1/interactions?as=creator&limit=1', { user: alex });
++ const all = await queries('GET', '/api/v1/interactions?as=creator&limit=50', { user: alex });
++ assert.strictEqual(one.r.json.interactions.length, 1);
++ assert.strictEqual(all.r.json.interactions.length, 15);
++ assert.ok(all.r.json.interactions.every((i) => Array.isArray(i.effects) && i.effects.length >= 1));
++ assert.deepStrictEqual([one.n, all.n], [3, 3], 'the page, the creators\' filters, the effects');
++ });
++
++ await check('GET /api/v1/interactions (a supporter\'s receipts; a service\'s view of a creator): bounded too', async () => {
++ const mine = await queries('GET', '/api/v1/interactions', { user: fans[0] });
++ assert.strictEqual(mine.r.json.interactions.length, 3);
++ assert.strictEqual(mine.n, 2, 'the page and the filters (no effects for a supporter)');
++ const svc = await queries('GET', '/api/v1/interactions?creator=alex&limit=50', { cap: ['tips.interaction.list'] });
++ assert.strictEqual(svc.r.json.interactions.length, 15);
++ assert.strictEqual(svc.n, 4, 'the creator, the page, the filters, the effects');
++ });
++
++ await check('goals, the supporters page and an overlay\'s state: a fixed number of queries for any number of rows', async () => {
++ const pub = await queries('GET', '/api/v1/goals?creator=alex', { token: null });
++ assert.strictEqual(pub.r.json.goals.length, 3);
++ assert.ok(pub.r.json.goals.every((g) => g.supporters.length === 5), 'each goal lists its supporters');
++ assert.strictEqual(pub.n, 4, 'the profile, the goals, their totals, their supporters');
++ const sup = await queries('GET', '/api/v1/profiles/alex/supporters', { token: null });
++ assert.strictEqual(sup.r.json.leaderboard.length, 5);
++ assert.strictEqual(sup.r.json.recent.length, 15);
++ assert.strictEqual(sup.n, 3, 'the profile, the leaderboard, the recent messages');
++ const before = t.db.stats().queries;
++ const state = await (await fetch(`${t.base}/overlay/${token}/state`)).json();
++ assert.strictEqual(state.alerts.length, 15);
++ assert.strictEqual(state.goals.length, 3);
++ assert.strictEqual(t.db.stats().queries - before, 4, 'the token, the goals, their totals, the alerts');
++ });
++
++ await t.close();
++ done();
++})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/security.test.js b/test/security.test.js
iindex 8c91c7d..9133c5c 100644
---- a/test/security.test.js
++++ b/test/security.test.js
@@@ -30,13 +30,13 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(d.status, 200);
assert.strictEqual(d.json.outcome, 'recorded', 'recorded on its own');
}
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'pending', 'the paid message is still unpaid');
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM paid_messages WHERE interaction_id = ?').get(id).n, 0);
-- const spoof = domain.interactions.byBillingTxn(cases[0].payload.transaction_id);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM paid_messages WHERE interaction_id = $1', [id]), 0);
++ const spoof = await domain.interactions.byBillingTxn(t.db, cases[0].payload.transaction_id);
assert.strictEqual(spoof.amount, 1);
assert.strictEqual(spoof.origin, 'billing');
-- assert.strictEqual(domain.interactions.totals(alex.subject).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
++ assert.strictEqual((await domain.interactions.totals(t.db, alex.subject)).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
});
await check('overlay stream floods: a token opens a bounded number of streams at once', async () => {
@@@ -83,7 +83,7 @@ const { boot, check, done } = require('./helpers/app');
tts: { text: '<speak><break time="9s"/>hello⁦ world</speak>' },
} });
assert.strictEqual(r.status, 201, r.text);
-- const i = domain.interactions.get(r.json.interaction.id);
++ const i = await domain.interactions.get(t.db, r.json.interaction.id);
assert.strictEqual(i.supporter_name, 'Viewer');
assert.strictEqual(i.message, 'hi there');
await domain.effects.drain();
@@@ -92,6 +92,24 @@ const { boot, check, done } = require('./helpers/app');
assert.ok(!/[⁦‮​]/.test(JSON.stringify(job)));
});
++ await check('text PostgreSQL cannot store (NUL, an unpaired surrogate) is cleaned, from people and from Billing events alike', async () => {
++ const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 6, message: 'a\u0000b \ud83d', supporter_name: 'N\u0000ul' } });
++ assert.strictEqual(r.status, 201, r.text);
++ const i = await domain.interactions.get(t.db, r.json.interaction.id);
++ assert.deepStrictEqual([i.message, i.supporter_name], ['ab �', 'Nul']);
++ const tts = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'say \udc00 this\u0000' } } });
++ assert.strictEqual(tts.status, 201, tts.text);
++ const cfg = await t.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', template: '{name} \ud800 tipped' } });
++ assert.strictEqual(cfg.status, 201, cfg.text);
++ // A Billing event is redelivered until it is accepted: its text must never make it unstorable.
++ const ev = billing.foreignDonation({ from: other.subject, to: alex.subject, amount: 9, message: 'from\u0000 billing \ud83d' });
++ const d = await t.deliver(ev);
++ assert.deepStrictEqual([d.status, d.json.outcome], [200, 'recorded']);
++ assert.strictEqual((await domain.interactions.byBillingTxn(t.db, ev.payload.transaction_id)).message, 'from billing �');
++ const cursor = Buffer.from(JSON.stringify(['1', 'x'])).toString('base64url');
++ assert.strictEqual((await t.call('GET', `/api/v1/interactions?cursor=${cursor}`, { user: viewer })).status, 422, 'a cursor PostgreSQL could not read is refused, not a 500');
++ });
++
await check('an overlay secret is shown once: an Idempotency-Key replay does not return it, the database never holds it', async () => {
const first = await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { label: 'replay' }, key: 'overlay-key-0001' });
assert.match(first.json.secret, /^tovl_/);
@@@ -99,7 +117,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(replay.headers.get('idempotent-replayed'), 'true');
assert.strictEqual(replay.json.token.id, first.json.token.id);
assert.strictEqual(replay.json.secret, null);
-- const everything = JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all()) + JSON.stringify(t.db.prepare('SELECT * FROM overlay_tokens').all());
++ const everything = JSON.stringify(await t.db.many('SELECT * FROM api_idempotency')) + JSON.stringify(await t.db.many('SELECT * FROM overlay_tokens'));
assert.ok(!everything.includes(first.json.secret));
});
ddiff --git a/test/settlement.test.js b/test/settlement.test.js
iindex 274ff3d..c930d90 100644
---- a/test/settlement.test.js
++++ b/test/settlement.test.js
@@@ -38,21 +38,21 @@ const { boot, check, done } = require('./helpers/app');
// Same transaction republished under a new event id (a replay): still one interaction.
const c = await t.deliver({ ...ev, event_id: billing.envelope('x', {}, 'x').event_id });
assert.strictEqual(c.json.duplicate, false);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 1);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 1);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 1);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 1);
});
await check('a duplicate provider webhook (Billing settles once) → one logical interaction for a donation Tips did not start', async () => {
// Billing turned one PowerChat site-routed tip into one transaction; Events may deliver it more than once.
const ev = billing.foreignDonation({ from: null, to: alex.subject, amount: 300, provider: 'powerchat', message: 'from PowerChat' });
for (let k = 0; k < 3; k++) await t.deliver(k === 2 ? { ...ev, event_id: billing.envelope('x', {}, 'x').event_id } : ev);
-- const rows = t.db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').all(ev.payload.transaction_id);
++ const rows = await t.db.many('SELECT * FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);
assert.strictEqual(rows.length, 1);
assert.strictEqual(rows[0].origin, 'billing');
assert.strictEqual(rows[0].funding, 'provider');
assert.strictEqual(rows[0].payment_state, 'settled');
// Tips did not start it, so Tips does not announce it in chat a second time — overlay only.
-- const effects = t.db.prepare('SELECT effect FROM interaction_effects WHERE interaction_id = ?').all(rows[0].id).map((e) => e.effect);
++ const effects = (await t.db.many('SELECT effect FROM interaction_effects WHERE interaction_id = $1', [rows[0].id])).map((e) => e.effect);
assert.deepStrictEqual(effects, ['overlay_alert']);
assert.strictEqual(rows[0].delivery_state, 'delivered');
});
@@@ -60,16 +60,16 @@ const { boot, check, done } = require('./helpers/app');
await check('delivery runs after settlement; a delivery failure never touches payment state', async () => {
const test = t.adapters.test;
await domain.effects.drain(); // the first tip delivers normally
-- assert.strictEqual(domain.interactions.get(tipId).delivery_state, 'delivered');
++ assert.strictEqual((await domain.interactions.get(t.db, tipId)).delivery_state, 'delivered');
test.state.failWith = 'chat is down';
test.state.permanent = true;
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 40 } });
const id = r.json.interaction.id;
await domain.effects.drain();
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'failed');
-- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((e) => e.subject.id === id).length, 1);
++ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((e) => e.subject.id === id).length, 1);
test.state.failWith = null;
assert.strictEqual(test.jobs.find((j) => j.interaction.id === tipId).text, 'Viewer tipped 250 Vibes: great stream');
});
@@@ -80,13 +80,13 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 10 } });
const id = r.json.interaction.id;
await domain.effects.drain();
-- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
++ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'queued');
assert.strictEqual(e.attempts, 1);
test.state.failWith = null;
t.clock.offset += 60_000;
await domain.effects.drain();
-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
t.clock.offset = 0;
});
@@@ -96,21 +96,21 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 400 } });
const id = r.json.interaction.id;
await domain.effects.drain();
-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 400);
++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 400);
const rev = billing.refund(r.json.interaction.payment.billing_txn_id);
const d = await t.deliver(rev);
assert.strictEqual(d.json.outcome, 'reversed');
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.delivery_state, 'delivered');
assert.strictEqual(i.reversed_bits, 400);
-- assert.ok(t.db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE interaction_id = ? AND state = 'delivered'").get(id).n >= 2);
-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0);
++ assert.ok(await t.db.value("SELECT count(*) FROM interaction_effects WHERE interaction_id = $1 AND state = 'delivered'", [id]) >= 2);
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0);
// The same reversal again changes nothing.
const again = await t.deliver({ ...rev, event_id: billing.envelope('x', {}, 'x').event_id });
assert.strictEqual(again.json.outcome, 'duplicate_reversal');
-- assert.strictEqual(domain.interactions.get(id).reversed_bits, 400);
++ assert.strictEqual((await domain.interactions.get(t.db, id)).reversed_bits, 400);
await t.call('POST', `/api/v1/goals/${g.json.goal.id}/close`, { user: alex });
});
@@@ -118,12 +118,12 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 20 } });
const id = r.json.interaction.id;
await t.deliver(billing.refund(r.json.interaction.payment.billing_txn_id));
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.delivery_state, 'cancelled');
await domain.effects.drain();
assert.ok(!t.adapters.test.jobs.some((j) => j.interaction.id === id), 'nothing delivered after the reversal');
-- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === id).length, 1);
++ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === id).length, 1);
});
await check('insufficient credit: payment failed, nothing delivered, a replay of the key answers the same', async () => {
@@@ -134,7 +134,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r.json.details.interaction.payment.state, 'failed');
const again = await t.call('POST', '/api/v1/checkout', { user: poor, body: { creator: 'alex', amount: 50 }, key: 'poor-key-00001' });
assert.strictEqual(again.status, 409);
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(poor.subject).n, 1);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [poor.subject]), 1);
});
await check('self-tipping is refused by subject', async () => {
@@@ -151,18 +151,18 @@ const { boot, check, done } = require('./helpers/app');
assert.match(r.json.checkout_url, /^https:\/\/checkout\.stripe\.test\/pi_/);
const id = r.json.interaction.id;
assert.strictEqual(r.json.interaction.payment.state, 'pending');
-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0, 'pending never counts');
-- const intentId = domain.interactions.get(id).billing_intent_id;
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0, 'pending never counts');
++ const intentId = (await domain.interactions.get(t.db, id)).billing_intent_id;
assert.strictEqual(billing.calls.find((c) => c.url === '/api/v1/intents' && c.body.subject.id === buyer.subject).key, `tips:intent:${id}`);
const purchase = billing.settlePurchase(intentId);
const d = await t.deliver(purchase);
assert.strictEqual(d.json.outcome, 'funded');
await new Promise((ok) => setTimeout(ok, 100));
await domain.interactions.processDueTransfers();
-- const i = domain.interactions.get(id);
++ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.funding_txn_id, purchase.payload.transaction_id);
-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 300);
++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 300);
// The purchase event again, and the transfer's own event: nothing new.
assert.strictEqual((await t.deliver(purchase)).json.duplicate, true);
const own = billing.events.find((e) => e.payload.metadata && e.payload.metadata.target && e.payload.metadata.target.id === id);
@@@ -188,7 +188,7 @@ const { boot, check, done } = require('./helpers/app');
t.clock.offset += 5000;
await domain.interactions.processDueTransfers();
t.clock.offset = 0;
-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
});
await check('paid messages, TTS and media requests exist only after settlement', async () => {
@@@ -205,17 +205,17 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r.status, 201, r.text);
ids.push(r.json.interaction.id);
}
-- const count = () => t.db.prepare('SELECT (SELECT COUNT(*) FROM paid_messages WHERE interaction_id IN (?, ?, ?)) + (SELECT COUNT(*) FROM paid_media_requests WHERE interaction_id IN (?, ?, ?)) AS n').get(...ids, ...ids).n;
-- assert.strictEqual(count(), 0);
-- for (const id of ids) await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));
++ const count = () => t.db.value('SELECT (SELECT count(*) FROM paid_messages WHERE interaction_id = ANY($1)) + (SELECT count(*) FROM paid_media_requests WHERE interaction_id = ANY($1))', [ids]);
++ assert.strictEqual(await count(), 0);
++ for (const id of ids) await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));
await new Promise((ok) => setTimeout(ok, 100));
await domain.interactions.processDueTransfers();
-- assert.strictEqual(count(), 3);
-- const pm = t.db.prepare('SELECT * FROM paid_messages WHERE interaction_id = ?').get(ids[0]);
++ assert.strictEqual(await count(), 3);
++ const pm = await t.db.maybe('SELECT * FROM paid_messages WHERE interaction_id = $1', [ids[0]]);
assert.strictEqual(pm.highlight_seconds, 60);
await domain.effects.drain();
-- assert.strictEqual(t.db.prepare('SELECT status FROM paid_messages WHERE interaction_id = ?').get(ids[1]).status, 'delivered');
-- assert.strictEqual(t.db.prepare('SELECT status FROM paid_media_requests WHERE interaction_id = ?').get(ids[2]).status, 'accepted');
++ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_messages WHERE interaction_id = $1', [ids[1]])).status, 'delivered');
++ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_media_requests WHERE interaction_id = $1', [ids[2]])).status, 'accepted');
const tts = t.adapters.test.jobs.find((j) => j.effect === 'tts' && j.interaction.id === ids[1]);
assert.deepStrictEqual(tts.tts, { text: 'read me please', voice: 'amy' });
});
@@@ -230,7 +230,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(a.json.interaction.id, b.json.interaction.id);
const denied = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.checkout.create'], body: { ...body, provider_ref: 'x2' } });
assert.strictEqual(denied.status, 403);
-- const totals = domain.interactions.totals(alex.subject);
++ const totals = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totals.external, 500);
});
@@@ -246,7 +246,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(proxied.status, 404);
const forged = await t.deliver({ ...ev, source: 'live' });
assert.strictEqual(forged.json.outcome, 'ignored:source');
-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 0);
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 0);
});
await check('events: ready/goal events are relayed to OpenVibe.Events with the tips service token', async () => {
ddiff --git a/test/sqlite-import.test.js b/test/sqlite-import.test.js
nnew file mode 100644
iindex 0000000..edb09e0
---- /dev/null
++++ b/test/sqlite-import.test.js
@@@ -0,0 +1,189 @@
++'use strict';
++// The one-time import (ADR-035; scripts/migrate-to-postgres.js): a SQLite file with the last SQLite
++// release's schema (test/fixtures/sqlite-v3-schema.sql) and rows of every table goes through
++// openvibe-sdk's importSqlite into the migrated schema: report.ok, every row counted and checksummed,
++// values converted by column type (0/1 → boolean, ISO text → timestamptz, JSON text → jsonb), identities
++// kept and continued, the SQLite file untouched; and the service then serves the imported data.
++const assert = require('assert');
++const crypto = require('crypto');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const { execFileSync } = require('child_process');
++const Database = require('better-sqlite3');
++const { ids } = require('openvibe-contracts');
++const { importSqlite } = require('openvibe-sdk/db');
++const { testDb } = require('./helpers/db');
++const { boot, check, done } = require('./helpers/app');
++const { importOptions } = require('../scripts/migrate-to-postgres');
++
++const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-import-'));
++const quiet = { log() {} };
++const creator = ids.newId('user');
++const supporter = ids.newId('user');
++const T = (s) => `2026-09-${s}Z`; // an ISO time, as the SQLite release wrote them
++
++/** A SQLite file as the SQLite release left it: its schema and a few rows in every table. */
++function fixture(file) {
++ const s = new Database(file);
++ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));
++ const ins = (table, row) => s.prepare(`INSERT INTO ${table} (${Object.keys(row).join(', ')}) VALUES (${Object.keys(row).map(() => '?').join(', ')})`).run(...Object.values(row));
++ ins('settings', { id: 1, schema_version: 3, created_at: T('01T00:00:00.000') });
++ ins('creator_tip_profiles', {
++ creator_subject: creator, handle: 'alex', display_name: 'Alex', page_enabled: 1, accepting: 1, tts_enabled: 0, media_requests_enabled: 1,
++ revision: 4, created_at: T('01T10:00:00.000'), updated_at: T('02T11:30:00.250'),
++ page_settings: '{"goal_amounts":false,"supporters_page":true}', filter_words: '["badword","two words"]', filter_action: 'hold', filter_links: 0,
++ });
++ const tip = (id, fields) => ins('tip_interactions', {
++ id, creator_subject: creator, supporter_subject: supporter, supporter_name: 'Viewer', kind: 'tip', amount: 250, funding: 'credit', settlement: 'billing',
++ payment_state: 'settled', delivery_state: 'delivered', created_at: T('03T12:00:00.000'), updated_at: T('03T12:00:01.000'), settled_at: T('03T12:00:00.500'), ...fields,
++ });
++ tip('tint_01', { billing_txn_id: 'txn_1', request: '{"goal_id":"tgoal_01"}', target: '{"service":"live","type":"stream","id":"42"}', message: 'great stream' });
++ // Two things SQLite took and PostgreSQL does not: a NUL in text, an unpaired surrogate inside JSON.
++ tip('tint_02', { message: 'nul\u0000here', request: '{"legacy_message":"cut \\ud83d","n":1}', billing_txn_id: 'txn_2', payment_state: 'reversed', reversal_txn_ids: '["txn_9"]', reversed_bits: 100, reversed_at: T('04T09:00:00.000'), anonymous: 1, hide_amount: 1, test: 0 });
++ tip('tint_03', { kind: 'paid_message', amount: 500, message: 'hello chat', funding: 'checkout', payment_state: 'pending', delivery_state: 'awaiting_payment',
++ settled_at: null, billing_intent_id: 'pi_1', checkout_url: 'https://checkout.example/pi_1', idempotency_key: 'api:usr:k1', transfer_due: 1, transfer_attempts: 2, next_transfer_at: 1790600000123 });
++ tip('tint_04', { supporter_subject: null, supporter_name: null, erased_at: T('05T08:00:00.000'), moderation: 'hidden', moderated_by: 'filter', filtered: 1, test: 1, origin: 'import',
++ settlement: 'imported', legacy_source: 'live:transactions:7', provider: 'powerchat', provider_ref: 'live-chat:3', request: '{}' });
++ ins('tip_goals', { id: 'tgoal_01', creator_subject: creator, title: 'Desk', target_amount: 1000, status: 'active', opening_amount: 50, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000'), reached_at: null });
++ ins('tip_goal_contributions', { id: 7, goal_id: 'tgoal_01', interaction_id: 'tint_01', amount: 250, reversed_amount: 0, created_at: T('03T12:00:00.600'), updated_at: T('03T12:00:00.600') });
++ ins('tip_goal_contributions', { id: 12, goal_id: 'tgoal_01', interaction_id: 'tint_02', amount: 250, reversed_amount: 100, created_at: T('03T12:00:00.700'), updated_at: T('04T09:00:00.000') });
++ ins('paid_messages', { id: 'tpm_01', interaction_id: 'tint_01', creator_subject: creator, kind: 'paid_message', text: 'hello', highlight_seconds: 30, status: 'delivered', chat_ref: 'null', test: 0, created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
++ ins('paid_media_requests', { id: 'tmr_01', interaction_id: 'tint_02', creator_subject: creator, url: 'https://youtu.be/x', provider: 'youtube', status: 'accepted', queue_ref: '{"queue_id":5}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
++ ins('overlay_configs', { id: 'tovc_01', creator_subject: creator, kind: 'alerts', name: 'Alerts', settings: '{"min_amount":1,"sound_url":null}', revision: 2, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000') });
++ ins('overlay_tokens', { id: 'tovt_01', creator_subject: creator, token_hash: 'a'.repeat(64), scopes: '["alerts","goals"]', config_id: 'tovc_01', created_by: creator, created_at: T('02T00:00:00.000'), revoked_at: T('06T00:00:00.000') });
++ ins('overlay_deliveries', { seq: 3, id: 'tovd_03', creator_subject: creator, kind: 'alert', interaction_id: 'tint_01', payload: '{"interaction_id":"tint_01","amount":250}', status: 'delivered', sends: 4, dedupe_key: 'alert:tint_01', created_at: T('03T12:00:01.000'), delivered_at: T('03T12:00:01.500'), hidden: 0 });
++ ins('overlay_deliveries', { seq: 9, id: 'tovd_09', creator_subject: creator, kind: 'goal', goal_id: 'tgoal_01', payload: '{"goal":{"id":"tgoal_01"},"by":null}', status: 'pending', dedupe_key: 'goal:tgoal_01:r2', created_at: T('03T12:00:01.000'), hidden: 1 });
++ ins('interaction_effects', { id: 4, interaction_id: 'tint_01', effect: 'chat_line', adapter: 'live-chat', state: 'delivered', attempts: 1, result: '{"ref":{"chat_message_id":1}}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
++ ins('interaction_effects', { id: 5, interaction_id: 'tint_01', effect: 'overlay_alert', adapter: 'overlay', state: 'delivered', result: 'null', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });
++ ins('interaction_effects', { id: 6, interaction_id: 'tint_03', effect: 'paid_message', adapter: 'live-chat', state: 'queued', attempts: 2, next_attempt_at: 1790600000999, last_error: 'Live 503', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });
++ ins('migration_maps', { id: 2, source: 'live', source_table: 'transactions', source_id: '7', target_type: 'interaction', target_id: 'tint_04', status: 'imported', run_id: 'timp_1', created_at: T('01T00:00:00.000'), updated_at: T('01T00:00:00.000') });
++ ins('import_runs', { id: 'timp_1', source: 'live', dry_run: 0, started_at: T('01T00:00:00.000'), finished_at: T('01T00:00:05.000'), report: '{"counts":{"imported":1},"holds":[]}' });
++ ins('tip_moderators', { creator_subject: creator, moderator_subject: supporter, name: 'Mod', added_by: 'invite:tmin_1', created_at: T('02T00:00:00.000') });
++ ins('tip_moderator_invites', { id: 'tmin_1', creator_subject: creator, token_hash: 'b'.repeat(64), created_by: creator, created_at: T('02T00:00:00.000'), expires_at: T('09T00:00:00.000'), used_at: T('02T01:00:00.000'), used_by: supporter });
++ ins('tip_moderation_log', { id: 3, creator_subject: creator, interaction_id: 'tint_04', action: 'hidden', by_role: 'moderator', actor: supporter, reason: 'abuse', created_at: T('05T08:00:00.000') });
++ // A stored answer is replayed byte for byte: its text keeps its key order and spacing.
++ ins('api_idempotency', { key: `${supporter}:tip-key-0001`, request_hash: 'h1', method: 'POST', path: '/api/v1/checkout', status: 201, response: '{"z":1,"a":{"b":2}}', created_at: T('03T12:00:00.000') });
++ ins('event_outbox', { id: 1, event_id: 'evt_01', envelope: '{"event_type":"tips.interaction.ready","subject":{"type":"interaction","id":"tint_01"},"payload":{"supporter":null}}', created_at: 1790500000000, attempts: 1, sent_at: 1790500001000, seq: 17 });
++ ins('event_outbox', { id: 2, event_id: 'evt_02', envelope: '{"event_type":"tips.goal.updated","subject":{"type":"goal","id":"tgoal_01"},"payload":{}}', created_at: 1790500002000, attempts: 3, next_attempt_at: 1790500999000, last_error: 'timeout' });
++ ins('idempotency_receipts', { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });
++ s.close();
++ return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex');
++}
++
++(async () => {
++ const file = path.join(tmp, 'tips.db');
++ const hash = fixture(file);
++
++ const imported = await testDb({ store: 'pglite' });
++ const db = imported.db;
++ let report;
++ const options = await importOptions(db);
++ const TABLES = options.tables;
++ await check('the importer copies every table, verified by counts and checksums (report.ok)', async () => {
++ report = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
++ assert.strictEqual(report.ok, true, JSON.stringify(report.problems));
++ const rows = Object.fromEntries(report.tables.map((t) => [t.table, t.rows]));
++ assert.deepStrictEqual(rows, {
++ creator_tip_profiles: 1, tip_interactions: 4, tip_goals: 1, tip_goal_contributions: 2, paid_messages: 1, paid_media_requests: 1,
++ overlay_configs: 1, overlay_tokens: 1, overlay_deliveries: 2, interaction_effects: 3, migration_maps: 1, import_runs: 1,
++ tip_moderators: 1, tip_moderator_invites: 1, tip_moderation_log: 1, api_idempotency: 1, tips_event_outbox: 2, tips_event_inbox: 1,
++ });
++ assert.ok(report.tables.every((t) => /^[0-9a-f]{16}$/.test(t.checksum)), 'a checksum per table');
++ const renamed = report.tables.filter((t) => t.source !== t.table).map((t) => `${t.source} → ${t.table}`).sort();
++ assert.deepStrictEqual(renamed, ['event_outbox → tips_event_outbox', 'idempotency_receipts → tips_event_inbox']);
++ assert.strictEqual(crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'), hash, 'the SQLite file is unchanged');
++ });
++
++ await check('values are converted by column type', async () => {
++ const p = await db.one('SELECT * FROM creator_tip_profiles');
++ assert.deepStrictEqual([p.page_enabled, p.tts_enabled, p.filter_links, p.revision], [true, false, false, 4]);
++ assert.deepStrictEqual(p.page_settings, { goal_amounts: false, supporters_page: true });
++ assert.deepStrictEqual(p.filter_words, ['badword', 'two words']);
++ assert.strictEqual(p.updated_at, T('02T11:30:00.250'), 'the same ISO time the API sent before');
++ const i = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_01'");
++ assert.deepStrictEqual(i.request, { goal_id: 'tgoal_01' });
++ assert.deepStrictEqual(i.target, { service: 'live', type: 'stream', id: '42' });
++ assert.deepStrictEqual([i.test, i.anonymous, i.transfer_due, i.amount], [false, false, false, 250]);
++ assert.deepStrictEqual(i.reversal_txn_ids, []);
++ const r = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_02'");
++ assert.deepStrictEqual([r.anonymous, r.hide_amount, r.reversed_bits, r.reversal_txn_ids], [true, true, 100, ['txn_9']]);
++ assert.strictEqual(r.message, 'nulhere', 'a NUL is dropped (PostgreSQL text cannot hold it)');
++ assert.deepStrictEqual(r.request, { legacy_message: 'cut \ufffd', n: 1 }, 'an unpaired surrogate becomes U+FFFD (jsonb refuses it)');
++ assert.deepStrictEqual(options.cleaned().sort((a, b) => a.column.localeCompare(b.column)),
++ [{ column: 'tip_interactions.message', values: 1 }, { column: 'tip_interactions.request', values: 1 }], 'and the report says where');
++ const pending = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_03'");
++ assert.deepStrictEqual([pending.transfer_due, pending.transfer_attempts, pending.next_transfer_at, pending.settled_at], [true, 2, 1790600000123, null]);
++ const erased = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_04'");
++ assert.deepStrictEqual([erased.test, erased.filtered, erased.erased_at, erased.target], [true, true, T('05T08:00:00.000'), null]);
++ assert.strictEqual(await db.value("SELECT chat_ref FROM paid_messages WHERE id = 'tpm_01'"), null, "JSON text 'null' → NULL (no reference)");
++ assert.deepStrictEqual(await db.value("SELECT queue_ref FROM paid_media_requests WHERE id = 'tmr_01'"), { queue_id: 5 });
++ assert.deepStrictEqual((await db.many('SELECT id, result FROM interaction_effects ORDER BY id')).map((e) => [e.id, e.result]),
++ [[4, { ref: { chat_message_id: 1 } }], [5, null], [6, null]]);
++ assert.deepStrictEqual(await db.value("SELECT scopes FROM overlay_tokens"), ['alerts', 'goals']);
++ assert.deepStrictEqual((await db.many('SELECT seq, hidden, test FROM overlay_deliveries ORDER BY seq')).map((d) => [d.seq, d.hidden, d.test]), [[3, false, false], [9, true, false]]);
++ assert.strictEqual(await db.value('SELECT response FROM api_idempotency'), '{"z":1,"a":{"b":2}}', 'a stored answer stays byte for byte');
++ assert.strictEqual(await db.value('SELECT dry_run FROM import_runs'), false);
++ const out = await db.many('SELECT id, envelope, sent_at, next_attempt_at FROM tips_event_outbox ORDER BY id');
++ assert.deepStrictEqual(out.map((o) => [o.id, o.envelope.event_type, o.sent_at, o.next_attempt_at]), [[1, 'tips.interaction.ready', 1790500001000, 0], [2, 'tips.goal.updated', null, 1790500999000]]);
++ assert.deepStrictEqual(await db.one('SELECT * FROM tips_event_inbox'), { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });
++ });
++
++ await check('identities keep their values and continue after the imported maximum', async () => {
++ const at = new Date().toISOString();
++ assert.strictEqual(await db.value(`INSERT INTO overlay_deliveries (id, creator_subject, kind, payload, status, dedupe_key, created_at)
++ VALUES ('tovd_new', $1, 'alert', '{}', 'pending', 'alert:new', $2) RETURNING seq`, [creator, at]), 10, 'the SSE ids go on from 9');
++ assert.strictEqual(await db.value(`INSERT INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at)
++ VALUES ('tint_02', 'chat_line', 'test', 'queued', $1, $1) RETURNING id`, [at]), 7);
++ assert.strictEqual(await db.value(`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)
++ VALUES ('tgoal_01', 'tint_03', 5, $1, $1) RETURNING id`, [at]), 13);
++ assert.strictEqual(await db.value("INSERT INTO tips_event_outbox (event_id, envelope, created_at) VALUES ('evt_03', '{}', 1) RETURNING id"), 3);
++ });
++
++ await check('a rehearsal can be repeated (truncate), and a value the checksum would miss is caught', async () => {
++ const again = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
++ assert.strictEqual(again.ok, true, JSON.stringify(again.problems));
++ // Without the 'null' mapping, JSON null and SQL NULL differ: the verification says so.
++ const bad = await importSqlite({ sqlite: file, db, truncate: true, tables: { ...TABLES, paid_messages: {} }, only: ['paid_messages', 'tip_interactions', 'creator_tip_profiles'], log: quiet });
++ assert.strictEqual(bad.ok, false);
++ assert.match(bad.problems.map((x) => `${x.table}: ${x.problem}`).join('\n'), /paid_messages: verification failed/);
++ await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
++ });
++
++ await check('the service serves the imported data as it did from SQLite', async () => {
++ const t = await boot({ db });
++ const who = { subject: supporter, username: 'viewer' };
++ const r = await t.call('GET', '/api/v1/interactions', { user: who });
++ assert.strictEqual(r.status, 200, r.text);
++ const byId = Object.fromEntries(r.json.interactions.map((i) => [i.id, i]));
++ assert.deepStrictEqual(Object.keys(byId).sort(), ['tint_01', 'tint_02', 'tint_03']);
++ assert.strictEqual(byId.tint_01.created_at, T('03T12:00:00.000'));
++ assert.strictEqual(byId.tint_01.payment.settled_at, T('03T12:00:00.500'));
++ assert.deepStrictEqual(byId.tint_02.payment.reversal_txn_ids, ['txn_9']);
++ assert.strictEqual(byId.tint_02.public.supporter_name, 'Anonymous');
++ const goal = (await t.call('GET', '/api/v1/goals/tgoal_01', { user: { subject: creator, username: 'alex' } })).json.goal;
++ assert.strictEqual(goal.current_amount, 50 + 250 + 150, 'opening amount plus what the contributions keep');
++ const replay = await t.call('POST', '/api/v1/checkout', { user: who, key: 'tip-key-0001', body: {} });
++ assert.strictEqual(replay.status, 422, 'a different request under a stored key is refused');
++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tips_event_inbox'), 1);
++ await t.close();
++ });
++
++ await check('scripts/migrate-to-postgres.js: OK exits 0; a column the schema lacks is a problem and exits 1', async () => {
++ const out = execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', file], { encoding: 'utf8' });
++ assert.match(out, /→ pglite \(\d+ ms\): OK/);
++ assert.match(out, /tips_event_outbox\s+2 rows\s+[0-9a-f]{16}\s+\(from event_outbox\)/);
++ assert.match(out, /cleaned: tip_interactions\.message: 1 value\(s\) with a NUL or an unpaired surrogate/);
++ const extra = path.join(tmp, 'extra.db');
++ fs.copyFileSync(file, extra);
++ const s = new Database(extra); s.exec('ALTER TABLE tip_goals ADD COLUMN mystery TEXT'); s.close();
++ let code = 0; let text = '';
++ try { execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', extra], { encoding: 'utf8' }); } catch (e) { code = e.status; text = e.stdout; }
++ assert.strictEqual(code, 1);
++ assert.match(text, /problem: tip_goals: source columns with no target column .*mystery/);
++ });
++
++ await imported.close();
++ fs.rmSync(tmp, { recursive: true, force: true });
++ done();
++})().catch((e) => { console.error(e); process.exit(1); });