openvibers/openvibe.tips Commits
Tips moves to PostgreSQL and Valkey (ADR-035 and its amendment, roadmap WS-X2, following openvibe-sdk's docs/migrating-to-postgresql.md). Everything is async on openvibe-sdk/db v0.15.0: node-postgres through PgBouncer in production, PGlite in tests and in development without DATABASE_URL. migrations/0001_initial.sql (expand) is the SQLite v3 schema in PostgreSQL types: ISO text times become timestamptz, epoch-ms stays bigint, 0/1 becomes boolean, JSON text becomes jsonb, amounts become bigint with their sums cast back, and the AUTOINCREMENT ids become identities. Two exceptions: the stored API answers stay text so a replay is byte for byte, and the SQLite settings row is not carried (ov_migrations records the schema). The SDK outbox and inbox are tips_event_outbox and tips_event_inbox, and every query shape has an index. Boot migrates with DATABASE_DIRECT_URL and then serves on DATABASE_URL; /api/ready reports db.ready() (the store that answered, and the pool) and valkey.ready(). A function that touches the database takes the handle first (db, or the transaction's t), after-commit work hangs on the transaction (t.after, per attempt), and Express handlers are async with their errors reaching the error handler.
Changes
@@@ -11,8 +11,22 @@ NODE_ENV=developmentBASE_URL=http://localhost:4610TRUST_PROXY=1--# SQLite (one database per service). The systemd unit sets /var/lib/openvibe-tips/tips.db.--TIPS_DB_PATH=./data/tips.db++# ── Data: PostgreSQL + Valkey (ADR-035) ──────────────────────++# In production OpenVibe.Host writes all four (sudo /opt/openvibe.host/roles/data/add-service.sh tips).++# DATABASE_URL: the runtime role through PgBouncer (transaction mode; DML only). Unset in development:++# an embedded PGlite database in data/pglite (one process). Tests need neither (PGlite, or the containers++# of node_modules/openvibe-sdk/scripts/test-services.sh).++DATABASE_URL=++# DATABASE_DIRECT_URL: the owner role on a direct connection, for the migrations at boot and++# scripts/migrate-to-postgres.js.++DATABASE_DIRECT_URL=++# VALKEY_URL: shared, never authoritative state (per-actor limits, overlay fan-out and stream slots).++# Unset: kept in each process, right for one process only.++VALKEY_URL=++VALKEY_PREFIX=ov:tips:++# The SQLite file of the release before PostgreSQL: read (never written) only by++# scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.++#TIPS_DB_PATH=./data/tips.db# ── Identity: OpenVibe.Network ──────────────────────────────OV_NETWORK_URL=https://openvibe.network@@@ -28,7 +42,8 @@ OV_OAUTH_CLIENT_ID=tipsOV_OAUTH_CLIENT_SECRET=#OV_OAUTH_REDIRECT_URI=https://openvibe.tips/auth/callback#COOKIE_SECURE=true--# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32).++# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32). Every process must++# share it; unset, it is derived from OV_OAUTH_CLIENT_SECRET (and random per process without either).TIPS_FORM_SECRET=# ── OpenVibe.Billing (money moves only there) ───────────────@@@ -9,6 +9,10 @@ jobs:uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176with:contracts-service: tips++ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers++ # (openvibe-sdk scripts/test-services.sh). npm test runs every file on PGlite and++ # test/integration.test.js on the containers; test:pg runs the whole suite through PgBouncer and Valkey.++ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).security:uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176@@@ -4,7 +4,9 @@**Status:** alpha — runtime built and tested against stubs (roadmap Wave 9). Deployed internally, notlaunched: running on the host since 2026-09-23 on `127.0.0.1:4610` only, holding the Live import and no--creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).++creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).++Data: PostgreSQL through PgBouncer, and Valkey for shared state ([ADR-035](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-035-postgresql-and-valkey.md));++production runs the SQLite release until the switch in *Deploy*.**Domain:** `openvibe.tips` (keeps its OpenVibe.Sites placeholder until the launch rule below holds)**Port / service id:** 4610 / `tips`**Decision:** [ADR-012](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-012-economic-classification.md) — Tips never moves money; Billing does.@@@ -21,8 +23,8 @@ from *interaction delivered*.- `tip_interactions`, `creator_tip_profiles`, `tip_goals`, `tip_goal_contributions`, `paid_messages`,`paid_media_requests`, `overlay_configs`, `overlay_deliveries`, `interaction_effects`, `migration_maps`-- (its own SQLite, `server/db.js`) — interaction state and **references** to Billing transaction ids,-- never a mutable cash balance++ (its own PostgreSQL database `ov_tips`, schema in [migrations/](migrations/), access in `server/db.js`) —++ interaction state and **references** to Billing transaction ids, never a mutable cash balance- revocable scoped overlay tokens (`overlay_tokens`, hashed at rest; never creator cookies)## Does not own@@@ -34,6 +36,10 @@ from *interaction delivered*.## Depends on++- **PostgreSQL 18 through PgBouncer (transaction mode) and Valkey 9** — OpenVibe.Host's data role++ (`roles/data/add-service.sh tips`: database `ov_tips`, owner and runtime roles, a Valkey user on++ `ov:tips:*`). PostgreSQL is required (the system of record); Valkey holds only what the processes share++ (per-actor limit counters, overlay fan-out, overlay stream slots) and its loss changes no answer about money- **OpenVibe.Billing** — `POST /api/v1/intents` (checkout), `POST /api/v1/transfers` (a tip from credit),`billing.transaction.settled|reversed` events, and `billing.receipt.external` (a tip on a creator's ownPowerChat, once Billing receives the PowerChat webhook)@@@ -42,8 +48,9 @@ from *interaction delivered*.- **OpenVibe.Network** — SSO for people, service tokens, JWKS, identity resolve (importer)- **OpenVibe.Live** — chat delivery through `/internal/tips/deliveries` (deployed in Live since `f11f809`; [docs/live-patch.diff](docs/live-patch.diff) is the original patch) until OpenVibe.Chat exposes paid messages; overlay consumer- **OpenVibe.Shared** v1.25.0 (app icon, SSR footer, noscript nav, release manifest, legal pages) and-- the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.12.0 (service tokens,-- outbox and inbox, per-actor limits), pinned by release tarball++ the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.15.0 (service tokens,++ the async data layer `openvibe-sdk/db`, the PostgreSQL outbox and inbox, Valkey, pub/sub and the shared++ limit store), pinned by release tarball; `pg` and `iovalkey` (drivers), PGlite for tests## Capabilities@@@ -66,10 +73,14 @@ Called elsewhere, as the service principal `tips`:```bashfnm exec --using=22.22.1 npm installcp .env.example .env # OV_OAUTH_CLIENT_SECRET, TIPS_FORM_SECRET, TIPS_EVENTS_SECRET, …--npm run dev # http://localhost:4610--npm test # every test/*.test.js: stub Network/Billing/Events/Live, temp DBs, random ports++npm run dev # http://localhost:4610 (without DATABASE_URL: an embedded PGlite database in data/pglite)++npm test # every test/*.test.js: stub Network/Billing/Events/Live, PGlite, random ports++eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" # PostgreSQL 18 + PgBouncer + Valkey 9 containers++npm test # … now also test/integration.test.js on the containers++npm run test:pg # every test file through PgBouncer and Valkeynpm run subscribe # create the billing.transaction.* and billing.receipt.* subscriptions in OpenVibe.Eventsnode scripts/import-live.js --live-db <live snapshot> --billing-db <billing snapshot> [--dry-run] [--json]++npm run migrate-to-postgres -- [--sqlite <copy>] [--pglite] [--json] # the one-time SQLite import (Deploy)```Production (deployed, loopback only): `/opt/openvibe.tips`, env `/etc/openvibe/tips.env`, unit@@@ -101,7 +112,7 @@ delivery record; a reversal before delivery cancels the queued effects (`tips.in| `external` | nothing — a tip on the creator's own PowerChat (ADR-012 EXTERNAL) | `POST /api/v1/interactions/external`, or Billing's `billing.receipt.external` (origin `billing-external`: Tips posts the chat line — `… tipped N Vibes: msg (PowerChat)` — plus overlay alert and goal, as Live's webhook did); once per (provider, provider_ref); never in Billing totals || `none` (simulation) | nothing | at once, `test = 1`: full effect path, no goal contribution, no durable event, excluded from totals |--**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same SQLite++**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same serializabletransaction as the change, so a redelivered event does nothing; `billing_txn_id` is UNIQUE, so the sameBilling transaction under another event id (republish, replay) is still one interaction; API calls carry`Idempotency-Key` (stored responses + the interaction's own unique key); the no-JS form carries a nonce.@@@ -117,9 +128,35 @@ from Live (whose `current_amount` cannot be traced to individual donations).minus what Billing took back) — the number that must equal Billing's books — with `external` and`simulations` reported apart.++**Data** (ADR-035; `server/db.js`, [migrations/](migrations/)):++- Everything is async on `openvibe-sdk/db`: a function that touches the database takes the query handle++ first (`db`, or the transaction's `t`), and after-commit work (overlay pushes, worker kicks) hangs on++ the transaction (`t.after`). The schema is `migrations/NNNN_*.sql` (expand/migrate/contract), applied++ at boot with the owner role; the service then serves through PgBouncer with no session state.++- **Money paths are SERIALIZABLE with row locks** (ADR-007 amendment 2026-09-24, rule 3): a request and++ its unpaid-checkout limit, settlement (paid message, media request, goal contribution), payment failure,++ funding, reversal, the Billing inbox, and what cancels or releases what a payment bought (moderation's++ hide and restore, the effects worker's writes). Locks are taken in one order: a creator's profile row++ (settings changes and per-creator limits), the interaction row, its goals by id, the creator's overlay++ stream (an advisory lock, so a creator's deliveries commit in seq order), then the rest. Serialization++ failures and deadlocks are retried by the data layer.++- Text is made storable on the way in: PostgreSQL keeps no NUL character, and jsonb no unpaired surrogate++ (SQLite kept both), so they are dropped or replaced (U+FFFD) in what people and Billing's events send.++- **Background work runs in every process** and claims its rows: due effects and due transfers with a++ lease (`FOR UPDATE SKIP LOCKED`, the lease in `next_attempt_at` / `next_transfer_at`), the overlay++ window sweep with `SKIP LOCKED`, the outbox relay by the SDK's lease; prunes are idempotent deletes.++- **Shared state in Valkey** (none of it authoritative): per-actor limit counters (the SDK's shared limit++ store), overlay fan-out (`overlay:<creator>` on pub/sub: a stream in one process hears a tip, a config++ change, a retraction or a revocation made in another), overlay stream slots (a sorted set per creator,++ refreshed by heartbeats). Without `VALKEY_URL` each process keeps its own, which is right for one process.++- Every query shape has an index (`test/migrations.test.js` checks the plans), lists page by keyset, and a++ list costs a fixed number of queries whatever its size (`test/query-budget.test.js`).++## API--`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH. Services present a Network++`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH (claimed before the handler runs:++the same key twice at once runs once, the other answered `409 idempotency.in_progress` with `Retry-After`++until the first answer is stored and replayed). Services present a Networkservice token (audience `openvibe.tips`, one capability per route); people present their Network usertoken as a Bearer and act on their own things only. The API never reads cookies.@@@ -218,8 +255,8 @@ toward its goal and the creator's totals.## Events--Produced through the openvibe-sdk outbox (table `event_outbox`, source `tips`, same transaction as the--change; relayed when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,++Produced through the openvibe-sdk outbox (table `tips_event_outbox`, source `tips`, same transaction as++the change; relayed by any process, rows claimed with a lease, when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,`tips.interaction.failed`, `tips.interaction.cancelled`, `tips.goal.updated`, `tips.overlay.delivered`,`tips.overlay.failed`, and two whose schemas were proposed in [docs/events-proposal/](docs/events-proposal/)and are in openvibe-contracts since v0.32.0: `tips.interaction.erased` (a supporter's erasure; `{ interaction_id,@@@ -319,6 +356,7 @@ service counts as its principal (`svc:live`), a person as `user:usr_…`, anyoneNever limited: `/api/health`, `/api/ready`, `/release.json`, `/metrics`, the overlays (bounded by`TIPS_OVERLAY_MAX_STREAMS`) and the signed Events deliveries at `/internal/events`. `test/actor-limits.test.js`.++With `VALKEY_URL` every process counts one caller together (`test/integration.test.js`).## Import and reconciliation@@@ -347,6 +385,8 @@ a copy of Billing's database for the links and totals:| simulation never counted; token revocation immediate; goals from settled only; reversal keeps the delivery record | `test/overlays.test.js`, `test/api.test.js`, `test/settlement.test.js` || an EXTERNAL PowerChat tip Billing announced is celebrated once: Live chat line, overlay, goal; never Billing money | `test/external.test.js` (redelivery and republish, same key as `POST /interactions/external`, Tips and Live goal ids, anonymous, test receipts, malformed payloads) || paid messages are filtered before they are shown or read; the creator and their moderators hide and show them without touching the money | `test/moderation.test.js` (mask, hold and release, invisible/full-width evasion, TTS text, invitation links, hide cancels queued chat/TTS and retracts the overlay live/replay/state, restore, pending payments, who may moderate, event payloads) |++| the suite passes on PostgreSQL behind PgBouncer in transaction mode; one settlement or reversal raced across two processes counts once; the unpaid-checkout limit holds under concurrency; leased work runs once; flushing Valkey changes no money answer (ADR-007 2026-09-24, ADR-035) | `npm run test:pg`; `test/integration.test.js` (two app instances on one database and one Valkey) |++| the SQLite data moves whole: every table and column, verified by counts and checksums | `test/sqlite-import.test.js`, `test/migrations.test.js` (no column left behind) || a supporter's privacy holds everywhere; the creator chooses what public pages show; export and erasure keep the books reconciled | `test/privacy.test.js` (anonymous / hidden amount / private message across API, overlays, chat jobs, pages and events; goal and supporters page settings; export; erasure scrubs rows, overlay payloads, stored answers and unsent events, emits `tips.interaction.erased`, totals still equal Billing) |## What waits@@@ -375,8 +415,25 @@ The unit is `openvibe-tips.service` on `127.0.0.1:4610`, the env file `/etc/open`/var/lib/openvibe-tips`. The vhost [deploy/nginx/openvibe.tips.conf](deploy/nginx/openvibe.tips.conf)waits for the launch: `openvibe.tips` serves the Sites placeholder.Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the--restart; afterwards `sudo ovhost rollback tips --to <sha>`. Nothing blocks a rollback: the schema--code only adds tables and columns.++restart; afterwards `sudo ovhost rollback tips --to <sha>`. Migrations are expand-only until a++`contract` migration, which waits out the 7-day N-1 window.++++**The switch from SQLite to PostgreSQL** (once; the procedure is the SDK's++[migrating-to-postgresql.md](https://github.com/OpenVibers/OpenVibe.SDK/blob/main/docs/migrating-to-postgresql.md), section 6):++1. `sudo /opt/openvibe.host/roles/data/add-service.sh tips` — database `ov_tips`, its roles and Valkey++ user; it writes `DATABASE_URL`, `DATABASE_DIRECT_URL`, `VALKEY_URL`, `VALKEY_PREFIX` into++ `/etc/openvibe/tips.env`.++2. Rehearse on a copy of `/var/lib/openvibe-tips/tips.db` (`sqlite3 … ".backup …"` as the service user):++ `npm run migrate-to-postgres -- --sqlite <copy> --pglite` must end OK.++3. Merge the PostgreSQL release; do not deploy it yet.++4. `sudo systemctl stop openvibe-tips` (the write freeze), then back up `tips.db`.++5. From the new release's directory, as the service user with the service's environment:++ `node scripts/migrate-to-postgres.js` — it migrates as the owner, imports every table with++ `truncate`, verifies counts and checksums, changes nothing in the SQLite file, and exits 1 unless OK.++6. `sudo ovhost deploy tips`; check `/api/ready` shows `db` `store: postgresql` and `valkey` ok, the++ dashboard, a receipt and an overlay, and a flat error rate.++7. Keep `tips.db` read-only for 7 days (the rollback: the previous release reads it, losing the writes++ made since the switch unless they are replayed), then `ovhost archive push` it and delete it.## Launch rule@@@ -385,8 +442,8 @@ This repository does not make the product real, and the domain keeps its placehoexist here (plan §12.12):1. an owning runtime with health/readiness endpoints and observability — ✔ `/api/health`, `/api/ready`-- (openvibe-shared/ready: the database required; Network key, Billing and Events optional, so-- their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals by++ (openvibe-shared/ready: the database required and reporting the store that answered; Valkey, the++ Network key, Billing and Events optional, so their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals byroute template, pending effects and overlay deliveries, outbox backlog);2. canonical identity/auth integration (Network subjects, scoped service principals) — ✔; the `tips`principal is provisioned on the host;@@@ -8,6 +8,7 @@"authoritative": false,"deployed": true,"deployment": "Deployed internally, not launched: deployed with `sudo ovhost deploy tips`, /opt/openvibe.tips, unit openvibe-tips, 127.0.0.1:4610 loopback only; https://openvibe.tips still serves the OpenVibe.Sites placeholder. Principal tips; Events subscriptions billing.transaction.* and billing.receipt.* -> /internal/events (secret rotated 2026-09-24); TIPS_CHAT_ADAPTER is empty (no chat side effects).",++ "database": "PostgreSQL 18 through PgBouncer (transaction mode) as the system of record, Valkey 9 for shared state (per-actor limits, overlay fan-out, overlay stream slots), ADR-035: database ov_tips from OpenVibe.Host's data role, schema in migrations/ (0001_initial, expand), async on openvibe-sdk/db v0.15.0, money paths serializable with row locks, PostgreSQL outbox and inbox. The deployed release still runs on SQLite (/var/lib/openvibe-tips/tips.db) until the switch in README Deploy: scripts/migrate-to-postgres.js imports it verified by counts and checksums (rehearsed on the SQLite databases the previous release makes: all tables and columns, report.ok), and the file stays read-only for the 7-day N-1 window.","contracts": "openvibe-contracts v0.49.0","notes": "Runtime, API, SSR pages, overlays, the Billing settlement consumer, events outbox, Live import and reconciliation are built, tested and deployed, together with /metrics (loopback only), truthful /api/ready (Billing and Events checks), supporter privacy (anonymous, hidden amount, private message), goal and supporter-page settings, export and erasure (tips.interaction.erased), moderation (word filter, hide/show, invited moderators, tips.interaction.moderated) and per-actor limits. Capabilities (18 tips.*) and payload schemas are in openvibe-contracts. On 2026-09-24 production held the Live import (3 import runs, the last 2026-09-23 18:31 UTC: 139 interactions, 4 goals, 143 migration maps); 0 creator profiles and 0 overlay tokens (nobody had set up the product). No billing.* event had arrived: Billing runs in shadow. A written internal threat review (docs/threat-review.md) with its fixes and test/security.test.js. Not done: an independent security review; the openvibe.tips launch. Live stays the place tips happen until the Billing cutover (blocked on the owner re-pointing the PowerChat webhook) and the launch rule in README.md holds.","updated": "2026-09-28",@@@ -22,7 +23,8 @@"Live import and reconciliation against Billing","server-rendered creator pages useful without JavaScript","/api/v1 by capability",-- "SDK outbox and inbox",-- "per-actor limits (openvibe-sdk/limits)"++ "per-actor limits (openvibe-sdk/limits)",++ "PostgreSQL + Valkey data tier (ADR-035): serializable money paths, leased background work, shared limits and overlay fan-out",++ "SDK outbox and inbox on PostgreSQL"]}@@@ -3,8 +3,11 @@ Description=OpenVibe.Tips — creator support: tips, goals, paid messages, TTS a# Network signs the tokens Tips verifies (its key is fetched at boot and retried); Billing moves# the money; Events carries Billing's settlements in and Tips' events out. Tips starts without any# of them: pages and overlays keep working, payments wait.--After=network.target--Wants=openvibe-network.service openvibe-billing.service++# Its own data (ADR-035, OpenVibe.Host's data role): PostgreSQL (migrations at boot on the direct++# connection, then PgBouncer) is required, the process exits and is restarted until it answers;++# Valkey is shared state only (without it everything shared is counted per process).++After=network.target postgresql.service pgbouncer.service valkey-server.service++Wants=openvibe-network.service openvibe-billing.service postgresql.service pgbouncer.service valkey-server.service[Service]Type=simple@@@ -19,7 +22,9 @@ Environment=PATH=/usr/local/bin:/usr/bin:/binExecStart=/usr/bin/env node /opt/openvibe.tips/server/index.jsRestart=alwaysRestartSec=5--# Tips' own database (ADR-007: one database per service); the code tree stays read-only.++# DATABASE_URL, DATABASE_DIRECT_URL, VALKEY_URL and VALKEY_PREFIX come from the env file++# (roles/data/add-service.sh tips). The state directory keeps the SQLite file of the release before++# PostgreSQL: read-only rollback for the 7-day N-1 window, read by scripts/migrate-to-postgres.js.StateDirectory=openvibe-tipsStateDirectoryMode=0700Environment=TIPS_DB_PATH=/var/lib/openvibe-tips/tips.dbnnew file mode 100644@@@ -0,0 +1,365 @@++-- phase: expand++--++-- OpenVibe.Tips on PostgreSQL (ADR-035): the SQLite schema of the last SQLite release (server/db.js,++-- schema v3 with its added columns; the SDK outbox and inbox) in PostgreSQL types, so++-- scripts/migrate-to-postgres.js can import the SQLite file as it is.++--++-- ISO-8601 text times -> timestamptz (the data layer returns the same ISO strings the API sent)++-- epoch-ms integers -> bigint (next_transfer_at, next_attempt_at, the outbox's times)++-- 0/1 integers -> boolean++-- JSON text -> jsonb, except api_idempotency.response (below)++-- amounts (bits/cents) -> bigint; SUM() of a bigint is numeric, so queries cast sums back ::bigint++-- AUTOINCREMENT -> bigint GENERATED ALWAYS AS IDENTITY (the importer keeps the values)++--++-- Not carried over: the SQLite `settings` table (one row, schema_version 3): ov_migrations records the++-- schema now, and readiness asks the database itself (db.ready()).++--++-- Every query shape the code runs has an index; the comment above each index names the queries.++-- Locks are always taken in one order: a creator's profile row, tip_interactions, tip_goals by id, the++-- creator's overlay stream (an advisory lock), then the rest (server/domain/index.js).++++-- ── Creator profiles ────────────────────────────────────────────────────────────────────────────++CREATE TABLE creator_tip_profiles (++ creator_subject text PRIMARY KEY, -- usr_…++ handle text NOT NULL UNIQUE, -- lower-case Network username (projection)++ display_name text NOT NULL,++ avatar_url text,++ headline text,++ page_enabled boolean NOT NULL DEFAULT false, -- public page + indexing++ accepting boolean NOT NULL DEFAULT true, -- new tips accepted++ min_amount bigint NOT NULL DEFAULT 1,++ paid_message_min bigint NOT NULL DEFAULT 100,++ tts_enabled boolean NOT NULL DEFAULT false,++ tts_min_amount bigint NOT NULL DEFAULT 100,++ tts_max_chars integer NOT NULL DEFAULT 200,++ tts_voice text NOT NULL DEFAULT 'gary',++ media_requests_enabled boolean NOT NULL DEFAULT false,++ media_request_min bigint NOT NULL DEFAULT 25,++ media_max_seconds integer NOT NULL DEFAULT 600,++ revision integer NOT NULL DEFAULT 1,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL,++ page_settings jsonb NOT NULL DEFAULT '{}', -- what the public goal / supporters pages show++ filter_words jsonb NOT NULL DEFAULT '[]', -- the creator's blocklist (filter.js)++ filter_action text NOT NULL DEFAULT 'mask', -- mask | hold++ filter_links boolean NOT NULL DEFAULT true -- links in paid messages shown as [link]++);++-- Home page and sitemap: switched-on pages, newest first / by handle.++CREATE INDEX creator_tip_profiles_enabled ON creator_tip_profiles (updated_at DESC) WHERE page_enabled;++++-- ── Interactions ────────────────────────────────────────────────────────────────────────────────++CREATE TABLE tip_interactions (++ id text PRIMARY KEY, -- tint_<ULID>++ creator_subject text NOT NULL,++ supporter_subject text, -- null: external/anonymous/unmapped/erased++ supporter_name text, -- display projection at the time++ kind text NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),++ amount bigint NOT NULL CHECK (amount > 0),++ currency text NOT NULL DEFAULT 'vibes-bits',++ amount_cents bigint, -- money that arrived (provider/external)++ message text,++ request jsonb NOT NULL DEFAULT '{}', -- tts text/voice, media url, goal_id, highlight++ funding text NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),++ settlement text NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),++ payment_state text NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),++ delivery_state text NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),++ test boolean NOT NULL DEFAULT false, -- simulation or Billing test money: never counted++ billing_txn_id text UNIQUE, -- the Billing transaction that settled it++ billing_intent_id text UNIQUE, -- checkout: the Billing payment intent++ funding_txn_id text UNIQUE, -- checkout: the purchase that funded it++ reversal_txn_ids jsonb NOT NULL DEFAULT '[]',++ reversed_bits bigint NOT NULL DEFAULT 0, -- bits Billing took back from the creator++ checkout_url text,++ checkout_ref text,++ provider text,++ provider_ref text,++ legacy_source text, -- 'live:transactions:<id>' etc.++ idempotency_key text UNIQUE,++ failure text,++ origin text NOT NULL DEFAULT 'tips', -- tips | billing | external | billing-external | import++ transfer_due boolean NOT NULL DEFAULT false, -- a Billing transfer still has to be (re)tried++ transfer_attempts integer NOT NULL DEFAULT 0,++ next_transfer_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the lease of the transfer job++ target jsonb, -- EntityRef (e.g. a Live stream)++ created_at timestamptz NOT NULL,++ settled_at timestamptz,++ reversed_at timestamptz,++ delivered_at timestamptz,++ updated_at timestamptz NOT NULL,++ anonymous boolean NOT NULL DEFAULT false, -- "Anonymous" to everyone but the supporter++ hide_amount boolean NOT NULL DEFAULT false, -- amount left out of overlays, chat, public pages++ private_message boolean NOT NULL DEFAULT false, -- a tip's message is for the creator only++ erased_at timestamptz, -- the supporter erased their data from it++ moderation text NOT NULL DEFAULT 'visible', -- visible | held | hidden++ moderated_at timestamptz,++ moderated_by text, -- 'filter', usr_… or svc:…++ filtered boolean NOT NULL DEFAULT false, -- the word filter matched it at settlement++ UNIQUE (provider, provider_ref),++ UNIQUE (legacy_source)++);++-- A creator's interactions, newest first, keyset-paged (list, dashboard, moderation queue "all"); totals.++CREATE INDEX tip_interactions_creator ON tip_interactions (creator_subject, created_at, id);++-- A supporter's receipts, newest first, keyset-paged; export; erasure.++CREATE INDEX tip_interactions_supporter ON tip_interactions (supporter_subject, created_at, id) WHERE supporter_subject IS NOT NULL;++-- The moderation queue by state, keyset-paged; the dashboard's held count.++CREATE INDEX tip_interactions_moderation ON tip_interactions (creator_subject, moderation, created_at, id);++-- Unpaid checkouts one supporter holds (the pending-checkout limit).++CREATE INDEX tip_interactions_open_checkouts ON tip_interactions (supporter_subject, created_at)++ WHERE funding = 'checkout' AND payment_state = 'pending';++-- Due Billing transfers, claimed with a lease (FOR UPDATE SKIP LOCKED).++CREATE INDEX tip_interactions_transfer_due ON tip_interactions (next_transfer_at) WHERE payment_state = 'pending' AND transfer_due;++-- The public supporters page: the leaderboard (settled or reversed, public name and amount).++CREATE INDEX tip_interactions_leaderboard ON tip_interactions (creator_subject, supporter_subject)++ WHERE supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed') AND NOT anonymous++ AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible';++-- The public supporters page: recent public messages.++CREATE INDEX tip_interactions_recent_public ON tip_interactions (creator_subject, settled_at, id)++ WHERE payment_state = 'settled' AND message IS NOT NULL;++++-- ── Goals ───────────────────────────────────────────────────────────────────────────────────────++CREATE TABLE tip_goals (++ id text PRIMARY KEY, -- tgoal_<ULID>++ creator_subject text NOT NULL,++ title text NOT NULL,++ description text,++ target_amount bigint NOT NULL CHECK (target_amount > 0),++ currency text NOT NULL DEFAULT 'vibes-bits',++ image_url text,++ status text NOT NULL CHECK (status IN ('active', 'closed')),++ sort_order integer NOT NULL DEFAULT 0,++ reached_at timestamptz,++ closed_at timestamptz,++ opening_amount bigint NOT NULL DEFAULT 0, -- carried over from Live's goal column++ legacy_source text UNIQUE,++ revision integer NOT NULL DEFAULT 1,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL++);++-- A creator's goals (by status), in the creator's order; active count; the goal a tip counts toward.++CREATE INDEX tip_goals_creator ON tip_goals (creator_subject, status, sort_order, created_at);++++CREATE TABLE tip_goal_contributions (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ goal_id text NOT NULL REFERENCES tip_goals(id),++ interaction_id text NOT NULL REFERENCES tip_interactions(id),++ amount bigint NOT NULL CHECK (amount > 0),++ reversed_amount bigint NOT NULL DEFAULT 0,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL,++ UNIQUE (goal_id, interaction_id)++);++-- Goal totals (goal_id = ANY), the latest contributions of a goal (goal_id, id DESC).++CREATE INDEX tip_goal_contributions_goal ON tip_goal_contributions (goal_id, id);++-- A reversal's contributions; a supporter's export.++CREATE INDEX tip_goal_contributions_interaction ON tip_goal_contributions (interaction_id);++++-- ── What settlement creates ─────────────────────────────────────────────────────────────────────++CREATE TABLE paid_messages (++ id text PRIMARY KEY, -- tpm_<ULID>++ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),++ creator_subject text NOT NULL,++ kind text NOT NULL CHECK (kind IN ('paid_message', 'tts')),++ text text NOT NULL,++ voice text,++ highlight_seconds integer NOT NULL DEFAULT 0,++ status text NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),++ chat_ref jsonb, -- the reference the chat adapter returned++ test boolean NOT NULL DEFAULT false,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL++);++++CREATE TABLE paid_media_requests (++ id text PRIMARY KEY, -- tmr_<ULID>++ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),++ creator_subject text NOT NULL,++ url text NOT NULL,++ provider text,++ status text NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),++ queue_ref jsonb, -- the reference returned by the queue owner++ test boolean NOT NULL DEFAULT false,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL++);++++-- ── Overlays ────────────────────────────────────────────────────────────────────────────────────++CREATE TABLE overlay_configs (++ id text PRIMARY KEY, -- tovc_<ULID>++ creator_subject text NOT NULL,++ kind text NOT NULL CHECK (kind IN ('alerts', 'goal')),++ name text NOT NULL,++ settings jsonb NOT NULL DEFAULT '{}',++ goal_id text,++ revision integer NOT NULL DEFAULT 1,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL++);++-- A creator's configs in creation order.++CREATE INDEX overlay_configs_creator ON overlay_configs (creator_subject, created_at);++++CREATE TABLE overlay_tokens (++ id text PRIMARY KEY, -- tovt_<ULID>++ creator_subject text NOT NULL,++ token_hash text NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once++ scopes jsonb NOT NULL, -- ["alerts","goals"]++ config_id text,++ label text,++ created_by text NOT NULL,++ created_at timestamptz NOT NULL,++ last_used_at timestamptz,++ revoked_at timestamptz++);++-- A creator's tokens (listing, the active count).++CREATE INDEX overlay_tokens_creator ON overlay_tokens (creator_subject, created_at);++++CREATE TABLE overlay_deliveries (++ seq bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, -- the SSE event id (monotonic)++ id text NOT NULL UNIQUE, -- tovd_<ULID>++ creator_subject text NOT NULL,++ kind text NOT NULL CHECK (kind IN ('alert', 'goal')),++ interaction_id text,++ goal_id text,++ payload jsonb NOT NULL,++ test boolean NOT NULL DEFAULT false,++ status text NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),++ delivered_at timestamptz,++ failed_at timestamptz,++ sends integer NOT NULL DEFAULT 0, -- times written to an overlay (replays included)++ dedupe_key text NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>++ created_at timestamptz NOT NULL,++ hidden boolean NOT NULL DEFAULT false -- retracted: never sent or replayed again++);++-- An overlay's stream: a creator's rows after a seq (push, replay, pending on connect, /state, max seq).++CREATE INDEX overlay_deliveries_creator ON overlay_deliveries (creator_subject, seq);++-- The delivery window sweep and the pending gauge.++CREATE INDEX overlay_deliveries_pending ON overlay_deliveries (seq) WHERE status = 'pending';++-- An interaction's deliveries (erasure rewrite, moderation retract/restore).++CREATE INDEX overlay_deliveries_interaction ON overlay_deliveries (interaction_id) WHERE interaction_id IS NOT NULL;++++-- ── Delivery effects (the effects worker claims due rows with a lease) ──────────────────────────++CREATE TABLE interaction_effects (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ interaction_id text NOT NULL REFERENCES tip_interactions(id),++ effect text NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),++ adapter text NOT NULL,++ state text NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),++ attempts integer NOT NULL DEFAULT 0,++ next_attempt_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the worker's lease++ last_error text,++ result jsonb,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL,++ UNIQUE (interaction_id, effect)++);++-- Due effects (the worker's claim) and the pending/due gauges.++CREATE INDEX interaction_effects_due ON interaction_effects (next_attempt_at, id) WHERE state = 'queued';++++-- ── Live import bookkeeping ─────────────────────────────────────────────────────────────────────++CREATE TABLE migration_maps (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ source text NOT NULL, -- 'live'++ source_table text NOT NULL,++ source_id text NOT NULL,++ target_type text, -- 'interaction' | 'goal' | 'refund'++ target_id text,++ status text NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),++ reason text,++ billing_txn_id text,++ run_id text NOT NULL,++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL,++ UNIQUE (source, source_table, source_id)++);++++CREATE TABLE import_runs (++ id text PRIMARY KEY,++ source text NOT NULL,++ dry_run boolean NOT NULL,++ started_at timestamptz NOT NULL,++ finished_at timestamptz,++ report jsonb++);++++-- ── Moderation ──────────────────────────────────────────────────────────────────────────────────++CREATE TABLE tip_moderators (++ creator_subject text NOT NULL,++ moderator_subject text NOT NULL,++ name text,++ added_by text NOT NULL, -- the creator, a service, or 'invite:<id>'++ created_at timestamptz NOT NULL,++ removed_at timestamptz,++ PRIMARY KEY (creator_subject, moderator_subject)++);++-- The creators a person moderates.++CREATE INDEX tip_moderators_moderator ON tip_moderators (moderator_subject);++++CREATE TABLE tip_moderator_invites (++ id text PRIMARY KEY, -- tmin_<ULID>++ creator_subject text NOT NULL,++ token_hash text NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)++ created_by text NOT NULL,++ created_at timestamptz NOT NULL,++ expires_at timestamptz NOT NULL,++ used_at timestamptz,++ used_by text,++ revoked_at timestamptz++);++-- A creator's open invitations (listing and the open count).++CREATE INDEX tip_moderator_invites_creator ON tip_moderator_invites (creator_subject, created_at);++++CREATE TABLE tip_moderation_log (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ creator_subject text NOT NULL,++ interaction_id text NOT NULL,++ action text NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),++ by_role text NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),++ actor text, -- usr_… / svc:… ; null for the filter++ reason text,++ created_at timestamptz NOT NULL++);++-- A creator's log, newest first.++CREATE INDEX tip_moderation_log_creator ON tip_moderation_log (creator_subject, id);++++-- ── Stored API answers (Idempotency-Key) ────────────────────────────────────────────────────────++-- response stays text: a replay returns the stored answer byte for byte, and jsonb would reorder keys.++-- A supporter's erasure deletes theirs by key prefix or by their subject in the answer: a scan of at++-- most a week of answers (pruned daily), for a rare action.++CREATE TABLE api_idempotency (++ key text PRIMARY KEY, -- <principal>:<Idempotency-Key>++ request_hash text NOT NULL,++ method text NOT NULL,++ path text NOT NULL,++ status integer NOT NULL,++ response text NOT NULL,++ created_at timestamptz NOT NULL++);++-- The weekly prune.++CREATE INDEX api_idempotency_created ON api_idempotency (created_at);++++-- ── Events: the openvibe-sdk outbox and inbox (outboxSchema('tips_event_outbox'), inboxSchema('tips_event_inbox')) ──++CREATE TABLE IF NOT EXISTS tips_event_outbox (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ event_id text NOT NULL UNIQUE,++ envelope jsonb NOT NULL,++ traceparent text,++ created_at bigint NOT NULL,++ attempts integer NOT NULL DEFAULT 0,++ next_attempt_at bigint NOT NULL DEFAULT 0,++ sent_at bigint,++ seq bigint,++ rejected_at bigint,++ last_error text++);++CREATE INDEX IF NOT EXISTS tips_event_outbox_due ON tips_event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;++CREATE INDEX IF NOT EXISTS tips_event_outbox_sent ON tips_event_outbox (sent_at) WHERE sent_at IS NOT NULL;++-- Tips' own queries on it: the rejected count (/api/ready), and an erasure scrubbing the local copies++-- of an interaction's events.++CREATE INDEX tips_event_outbox_rejected ON tips_event_outbox (id) WHERE rejected_at IS NOT NULL;++CREATE INDEX tips_event_outbox_interaction ON tips_event_outbox ((envelope->'subject'->>'id')) WHERE envelope->'subject'->>'type' = 'interaction';++++CREATE TABLE IF NOT EXISTS tips_event_inbox (++ consumer text NOT NULL,++ event_id text NOT NULL,++ processed_at bigint NOT NULL,++ PRIMARY KEY (consumer, event_id)++);@@@ -13,15 +13,33 @@"cookie-parser": "^1.4.7","dotenv": "^16.4.7","express": "^4.21.2",++ "iovalkey": "^0.4.0","jsonwebtoken": "^9.0.2","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",++ "pg": "^8.23.0"++ },++ "devDependencies": {++ "@electric-sql/pglite": "^0.5.8"},"engines": {"node": ">=22.0.0"}},++ "node_modules/@electric-sql/pglite": {++ "version": "0.5.8",++ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",++ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",++ "devOptional": true,++ "license": "Apache-2.0"++ },++ "node_modules/@iovalkey/commands": {++ "version": "0.1.0",++ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",++ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",++ "license": "MIT"++ },"node_modules/accepts": {"version": "1.3.8","resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",@@@ -223,6 +241,15 @@"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==","license": "ISC"},++ "node_modules/cluster-key-slot": {++ "version": "1.1.2",++ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",++ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/content-disposition": {"version": "0.5.4","resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",@@@ -305,6 +332,15 @@"node": ">=4.0.0"}},++ "node_modules/denque": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",++ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10"++ }++ },"node_modules/depd": {"version": "2.0.0","resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",@@@ -714,6 +750,49 @@"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==","license": "ISC"},++ "node_modules/iovalkey": {++ "version": "0.4.0",++ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",++ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",++ "license": "MIT",++ "dependencies": {++ "@iovalkey/commands": "^0.1.0",++ "cluster-key-slot": "^1.1.0",++ "debug": "^4.3.4",++ "denque": "^2.1.0",++ "lodash.defaults": "^4.2.0",++ "lodash.isarguments": "^3.1.0",++ "redis-errors": "^1.2.0",++ "redis-parser": "^3.0.0",++ "standard-as-callback": "^2.1.0"++ },++ "engines": {++ "node": ">=18.12.0"++ }++ },++ "node_modules/iovalkey/node_modules/debug": {++ "version": "4.4.3",++ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",++ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",++ "license": "MIT",++ "dependencies": {++ "ms": "^2.1.3"++ },++ "engines": {++ "node": ">=6.0"++ },++ "peerDependenciesMeta": {++ "supports-color": {++ "optional": true++ }++ }++ },++ "node_modules/iovalkey/node_modules/ms": {++ "version": "2.1.3",++ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",++ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",++ "license": "MIT"++ },"node_modules/ipaddr.js": {"version": "1.9.1","resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",@@@ -778,12 +857,24 @@"safe-buffer": "^5.0.1"}},++ "node_modules/lodash.defaults": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",++ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",++ "license": "MIT"++ },"node_modules/lodash.includes": {"version": "4.3.0","resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz","integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==","license": "MIT"},++ "node_modules/lodash.isarguments": {++ "version": "3.1.0",++ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",++ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",++ "license": "MIT"++ },"node_modules/lodash.isboolean": {"version": "3.0.3","resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",@@@ -999,12 +1090,32 @@}},"node_modules/openvibe-sdk": {-- "version": "0.12.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "integrity": "sha512-iG1h905O1f7y/PnP8xLA6GCVzXEBDfSaBj2tc1ucNreWg2sT/AJWosb8NW0WRMK5WS4sIQ6tMsprmDLj/fjjeQ==",++ "version": "0.15.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",++ "integrity": "sha512-5ZcqEZxRQjT15erqtEdJ4js3roSijUFiQZPbGIHpaOVFRJSI45ZCEgTBb9ByKNRdUNBl4ub9Vb1dF2CcCVJjrg==","license": "MIT","engines": {"node": ">=20"++ },++ "peerDependencies": {++ "@electric-sql/pglite": "^0.5.8",++ "better-sqlite3": ">=11",++ "iovalkey": "^0.4.0",++ "pg": "^8.23.0"++ },++ "peerDependenciesMeta": {++ "@electric-sql/pglite": {++ "optional": true++ },++ "better-sqlite3": {++ "optional": true++ },++ "iovalkey": {++ "optional": true++ },++ "pg": {++ "optional": true++ }}},"node_modules/openvibe-shared": {@@@ -1031,6 +1142,134 @@"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==","license": "MIT"},++ "node_modules/pg": {++ "version": "8.23.0",++ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",++ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",++ "license": "MIT",++ "dependencies": {++ "pg-connection-string": "^2.14.0",++ "pg-pool": "^3.14.0",++ "pg-protocol": "^1.16.0",++ "pg-types": "2.2.0",++ "pgpass": "1.0.5"++ },++ "engines": {++ "node": ">= 16.0.0"++ },++ "optionalDependencies": {++ "pg-cloudflare": "^1.4.0"++ },++ "peerDependencies": {++ "pg-native": ">=3.0.1"++ },++ "peerDependenciesMeta": {++ "pg-native": {++ "optional": true++ }++ }++ },++ "node_modules/pg-cloudflare": {++ "version": "1.4.0",++ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",++ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",++ "license": "MIT",++ "optional": true++ },++ "node_modules/pg-connection-string": {++ "version": "2.14.0",++ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",++ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",++ "license": "MIT"++ },++ "node_modules/pg-int8": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",++ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",++ "license": "ISC",++ "engines": {++ "node": ">=4.0.0"++ }++ },++ "node_modules/pg-pool": {++ "version": "3.14.0",++ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",++ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",++ "license": "MIT",++ "peerDependencies": {++ "pg": ">=8.0"++ }++ },++ "node_modules/pg-protocol": {++ "version": "1.16.0",++ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",++ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",++ "license": "MIT"++ },++ "node_modules/pg-types": {++ "version": "2.2.0",++ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",++ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",++ "license": "MIT",++ "dependencies": {++ "pg-int8": "1.0.1",++ "postgres-array": "~2.0.0",++ "postgres-bytea": "~1.0.0",++ "postgres-date": "~1.0.4",++ "postgres-interval": "^1.1.0"++ },++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/pgpass": {++ "version": "1.0.5",++ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",++ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",++ "license": "MIT",++ "dependencies": {++ "split2": "^4.1.0"++ }++ },++ "node_modules/postgres-array": {++ "version": "2.0.0",++ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",++ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/postgres-bytea": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",++ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-date": {++ "version": "1.0.7",++ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",++ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-interval": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",++ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",++ "license": "MIT",++ "dependencies": {++ "xtend": "^4.0.0"++ },++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/prebuild-install": {"version": "7.1.3","resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",@@@ -1154,6 +1393,27 @@"node": ">= 6"}},++ "node_modules/redis-errors": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",++ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/redis-parser": {++ "version": "3.0.0",++ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",++ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",++ "license": "MIT",++ "dependencies": {++ "redis-errors": "^1.0.0"++ },++ "engines": {++ "node": ">=4"++ }++ },"node_modules/require-from-string": {"version": "2.0.2","resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",@@@ -1369,6 +1629,21 @@"simple-concat": "^1.0.0"}},++ "node_modules/split2": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",++ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",++ "license": "ISC",++ "engines": {++ "node": ">= 10.x"++ }++ },++ "node_modules/standard-as-callback": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",++ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",++ "license": "MIT"++ },"node_modules/statuses": {"version": "2.0.2","resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",@@@ -1496,6 +1771,15 @@"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz","integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==","license": "ISC"++ },++ "node_modules/xtend": {++ "version": "4.0.2",++ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",++ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.4"++ }}}}@@@ -8,10 +8,17 @@"start": "node server/index.js","dev": "NODE_ENV=development node server/index.js","test": "node test/run.js",++ "test:pg": "TIPS_TEST_STORE=pg node test/run.js","subscribe": "node scripts/subscribe.js",-- "import-live": "node scripts/import-live.js"++ "import-live": "node scripts/import-live.js",++ "migrate-to-postgres": "node scripts/migrate-to-postgres.js"},-- "keywords": ["openvibe", "tips", "overlays", "creator"],++ "keywords": [++ "openvibe",++ "tips",++ "overlays",++ "creator"++ ],"author": "OpenVibers","license": "AGPL-3.0-only","repository": "github:OpenVibers/OpenVibe.Tips",@@@ -20,12 +27,17 @@"cookie-parser": "^1.4.7","dotenv": "^16.4.7","express": "^4.21.2",++ "iovalkey": "^0.4.0","jsonwebtoken": "^9.0.2","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",++ "pg": "^8.23.0"},"engines": {"node": ">=22.0.0"++ },++ "devDependencies": {++ "@electric-sql/pglite": "^0.5.8"}}@@@ -12,11 +12,15 @@** With --billing-db the report ends with the reconciliation: per creator, Tips' settled total must* equal Billing's; the exit code is 1 when it does not. Safe to re-run.++ *++ * Tips' side is its PostgreSQL database (DATABASE_URL; migrations applied with DATABASE_DIRECT_URL++ * first). The two snapshots are other services' SQLite files, which is why this operator tool, and++ * scripts/migrate-to-postgres.js, are the only code here that loads better-sqlite3.*/const path = require('path');const Database = require('better-sqlite3');const { loadConfig } = require('../server/config');--const { openDb } = require('../server/db');++const { openDb, migrate } = require('../server/db');const { createIdentity } = require('../server/network');const { createTipsOutbox } = require('../server/events/outbox');const { createDomain } = require('../server/domain');@@@ -30,10 +34,11 @@ async function main() {const livePath = opt('live-db');if (!livePath) { console.error('usage: import-live.js --live-db <snapshot> [--billing-db <snapshot>] [--dry-run] [--json]'); process.exit(2); }const config = loadConfig();-- for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.dbPath)) throw new Error(`${p} is the Tips database`);++ for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.sqlitePath)) throw new Error(`${p} is Tips' own (pre-PostgreSQL) database`);const live = new Database(livePath, { readonly: true, fileMustExist: true });const bdb = opt('billing-db') ? new Database(opt('billing-db'), { readonly: true, fileMustExist: true }) : null;-- const db = openDb(config.dbPath);++ const db = openDb(config);++ await migrate(config, { serving: db });const outbox = createTipsOutbox({ db, config });const domain = createDomain({ db, config, outbox, billing: null, adapters: {}, log: console });const identity = createIdentity(config);@@@ -55,7 +60,7 @@ async function main() {}live.close();if (bdb) bdb.close();-- db.close();++ await db.close();if (report.reconciliation && !report.reconciliation.ok) process.exit(1);}nnew file mode 100644@@@ -0,0 +1,142 @@++#!/usr/bin/env node++'use strict';++/**++ * The one-time move of Tips' SQLite database into its PostgreSQL schema (ADR-035; the procedure is++ * openvibe-sdk docs/migrating-to-postgresql.md, section 6).++ *++ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]++ *++ * 1. applies migrations/ as the owner (DATABASE_DIRECT_URL, a direct connection);++ * 2. copies every table with openvibe-sdk's importSqlite (batched, parents first, identities kept and++ * their sequences advanced) into emptied tables (truncate: a rehearsal can be repeated);++ * 3. verifies each table's row count and a checksum of every row on both sides, prints the report,++ * and exits 1 unless report.ok.++ *++ * The SQLite file (TIPS_DB_PATH, config.sqlitePath; --sqlite for a copy) is opened read-only: nothing in++ * it changes. --pglite imports into an in-memory PostgreSQL instead (a rehearsal with nothing to set up;++ * the report is all it leaves). --json prints the whole report.++ *++ * In production it runs once, from the new release's directory, as the service user with the service's++ * environment, while the service is stopped (the write freeze), before the PostgreSQL release starts.++ *++ * What does not map one to one:++ * settings not carried: the SQLite bookkeeping row (schema_version 3); ov_migrations++ * records the schema now (migrations/0001_initial.sql)++ * event_outbox → tips_event_outbox (the SDK outbox, outboxSchema('tips_event_outbox'))++ * idempotency_receipts → tips_event_inbox (the SDK inbox, inboxSchema('tips_event_inbox'))++ * JSON text 'null' → SQL NULL in the reference columns (interaction_effects.result,++ * paid_messages.chat_ref, paid_media_requests.queue_ref): the SQLite code++ * wrote JSON.stringify(null) when an adapter returned no reference, the++ * PostgreSQL code writes NULL, and both mean "no reference"++ * NUL, unpaired surrogate PostgreSQL stores no NUL character in text or jsonb and no unpaired surrogate++ * in jsonb (SQLite took both; the code strips them now): a NUL is dropped and an++ * unpaired surrogate becomes U+FFFD, in text and inside JSON values alike. The++ * report lists every column where that happened and how many values changed++ * No column is dropped.++ */++const path = require('path');++const { createDb, importSqlite } = require('openvibe-sdk/db');++const { loadConfig } = require('../server/config');++const { migrate, MIGRATIONS } = require('../server/db');++++const args = process.argv.slice(2);++const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };++const flag = (name) => args.includes(`--${name}`);++++/** A row map turning the JSON text 'null' into SQL NULL for these columns. */++const jsonNullToSql = (...cols) => (row) => {++ for (const c of cols) if (row[c] === 'null') row[c] = null;++ return row;++};++++const TABLES = {++ tips_event_outbox: { from: 'event_outbox' },++ tips_event_inbox: { from: 'idempotency_receipts' },++ interaction_effects: { map: jsonNullToSql('result') },++ paid_messages: { map: jsonNullToSql('chat_ref') },++ paid_media_requests: { map: jsonNullToSql('queue_ref') },++};++++const cleanText = (v) => v.toWellFormed().replace(/\u0000/g, '');++function cleanJson(v) {++ if (typeof v === 'string') return cleanText(v);++ if (Array.isArray(v)) return v.map(cleanJson);++ if (v && typeof v === 'object') return Object.fromEntries(Object.entries(v).map(([k, x]) => [cleanText(k), cleanJson(x)]));++ return v;++}++++/**++ * importSqlite's `tables` for this schema: TABLES, and for every table a map that makes its text and++ * JSON values storable (see above). `cleaned` counts the distinct values changed per column (the++ * verification sees the same mapped values on both sides, so this is the record of what changed).++ */++async function importOptions(db) {++ const cols = await db.many(`SELECT table_name, column_name, udt_name FROM information_schema.columns++ WHERE table_schema = current_schema() AND table_name <> 'ov_migrations' AND udt_name IN ('text', 'varchar', 'json', 'jsonb')`);++ const byTable = new Map();++ for (const c of cols) { if (!byTable.has(c.table_name)) byTable.set(c.table_name, []); byTable.get(c.table_name).push(c); }++ const cleaned = new Map(); // "table.column" → Set of the original values changed++ const note = (t, c, v) => { const k = `${t}.${c}`; if (!cleaned.has(k)) cleaned.set(k, new Set()); cleaned.get(k).add(v); };++ const tables = {};++ for (const [table, list] of byTable) {++ const base = TABLES[table] || {};++ tables[table] = {++ ...base,++ map(row) {++ const r = base.map ? base.map(row) : row;++ if (!r) return r;++ for (const { column_name: c, udt_name: udt } of list) {++ const v = r[c];++ if (typeof v !== 'string') continue;++ if (udt === 'json' || udt === 'jsonb') {++ let parsed;++ try { parsed = JSON.parse(v); } catch { continue; } // left for the importer to report++ const fixed = JSON.stringify(cleanJson(parsed));++ if (fixed !== JSON.stringify(parsed)) { note(table, c, v); r[c] = fixed; }++ } else {++ const fixed = cleanText(v);++ if (fixed !== v) { note(table, c, v); r[c] = fixed; }++ }++ }++ return r;++ },++ };++ }++ return { tables, cleaned: () => [...cleaned].map(([column, values]) => ({ column, values: values.size })) };++}++++async function main() {++ const config = loadConfig();++ const sqlite = path.resolve(opt('sqlite') || config.sqlitePath);++ const quiet = { log() {}, warn: console.warn, error: console.error };++ let owner;++ if (flag('pglite')) {++ owner = createDb({ pglite: true, service: 'tips-import' });++ await owner.migrate({ dir: MIGRATIONS, log: quiet });++ } else {++ await migrate(config, { log: quiet });++ owner = createDb({ url: config.db.directUrl, service: 'tips-import', max: 2 });++ }++ try {++ const t0 = Date.now();++ const options = await importOptions(owner);++ const report = await importSqlite({ sqlite, db: owner, truncate: true, tables: options.tables, log: quiet });++ const cleaned = options.cleaned();++ if (flag('json')) console.log(JSON.stringify({ sqlite, into: owner.store, ...report, cleaned }, null, 2));++ else {++ console.log(`import ${sqlite} → ${owner.store} (${Date.now() - t0} ms): ${report.ok ? 'OK' : 'PROBLEMS'}`);++ for (const t of report.tables) console.log(` ${t.table.padEnd(24)} ${String(t.rows).padStart(8)} rows ${t.checksum || '-'}${t.source !== t.table ? ` (from ${t.source})` : ''}`);++ for (const c of cleaned) console.log(` cleaned: ${c.column}: ${c.values} value(s) with a NUL or an unpaired surrogate`);++ for (const p of report.problems) console.log(` problem: ${p.table}: ${p.problem}`);++ }++ return report.ok ? 0 : 1;++ } finally {++ await owner.close();++ }++}++++if (require.main === module) {++ main().then((code) => process.exit(code), (e) => { console.error(`migrate-to-postgres failed: ${e.message}`); process.exit(1); });++}++++module.exports = { TABLES, importOptions };@@@ -8,13 +8,16 @@* problem+json `rate_limited` with Retry-After before it does any work (before the idempotency store,* Billing or chat delivery); the refusal is logged once and counted in tips_rate_limited_total{limit,window}.* Reads get TIPS_LIMITS_MINUTE / TIPS_LIMITS_HOUR (120 and 3000); writes and payments set their own,-- * tighter numbers in api/v1.js. Counters live in this process: a restart forgets them.++ * tighter numbers in api/v1.js. With VALKEY_URL the counters are shared by every process and host++ * (openvibe-sdk createValkeyLimitStore: one atomic script per request); if Valkey fails, a request is++ * counted by its process's own counters instead, so an outage neither opens nor closes the gates.++ * Without Valkey they live in this process (one process only).** Never limited: /api/health, /api/ready, /release.json, /metrics, the overlays (a scoped token each,* capped by TIPS_OVERLAY_MAX_STREAMS) and the signed Events deliveries at /internal/events (Billing's* settlements: Events pushes at its own pace, and a 429 would only make it retry and fall behind).*/--const { createActorLimiter, defaultActor } = require('openvibe-sdk/limits');++const { createActorLimiter, createValkeyLimitStore, defaultActor } = require('openvibe-sdk/limits');function actor(req) {const p = req.principal;@@@ -24,7 +27,7 @@ function actor(req) {}/** limits(name, own) middleware for one app, plus limits.reads(name): the defaults on GET/HEAD. */--function createActorLimits({ config, now = () => Date.now(), registry = null, log = console }) {++function createActorLimits({ config, valkey = null, now = () => Date.now(), registry = null, log = console }) {const refused = registry? registry.counter({ name: 'tips_rate_limited_total', help: 'Requests refused 429 by a per-actor limit, by limit name and window', labelNames: ['limit', 'window'] }): null;@@@ -32,6 +35,8 @@ function createActorLimits({ config, now = () => Date.now(), registry = null, lolimits: { minute: config.actorLimits.minute, hour: config.actorLimits.hour },actor,now,++ store: createValkeyLimitStore(valkey),++ log,onLimited(e) {// The actor is a principal, a subject id or an address, never a token.log.warn(`[Tips] limit ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);@@@ -6,12 +6,21 @@* same key and request returns it verbatim (Idempotent-Replayed: true) and does nothing; the same* key with another request is 422 idempotency.key_reused. Refusals are not stored.*++ * The key is claimed before the handler runs (a row with status 0), so the same key sent twice at once,++ * to one process or two, runs the handler once: the other request is answered 409++ * idempotency.in_progress with Retry-After, and a retry after the first finished gets the stored answer.++ * A 2xx answer is written into the claim before it is sent; any other outcome releases the claim. A++ * claim older than a minute (its process died mid-request) may be taken over.++ ** The scoped key also becomes the interaction's idempotency_key (UNIQUE), the second line of defence.*/const crypto = require('crypto');const { http } = require('openvibe-contracts');++const { sql } = require('openvibe-sdk/db');const KEY_RE = /^[A-Za-z0-9._:-]{8,200}$/;++const CLAIM_MS = 60_000;++const PENDING = 0; // api_idempotency.status of a claim whose answer is not in yetfunction stable(v) {if (Array.isArray(v)) return `[${v.map(stable).join(',')}]`;@@@ -21,31 +30,62 @@ function stable(v) {const principalKey = (p) => (p.kind === 'service' ? p.sub : p.kind === 'user' ? p.subject : 'anonymous');--function idempotent(db, now) {-- return function idempotencyKey(req, res, next) {++function idempotent(db, now, log = console) {++ const iso = (ms) => new Date(ms).toISOString();++ return async function idempotencyKey(req, res, next) {const key = req.get('Idempotency-Key');if (!key || !KEY_RE.test(key)) {return http.sendProblem(res, 400, 'idempotency.key_required', { detail: 'mutating calls need an Idempotency-Key header (8-200 of A-Z a-z 0-9 . _ : -)', ctx: req.ov });}const scoped = `${principalKey(req.principal)}:${key}`;const hash = crypto.createHash('sha256').update(`${req.method} ${req.baseUrl}${req.path}\n${stable(req.body || {})}`).digest('hex');-- const row = db.prepare('SELECT * FROM api_idempotency WHERE key = ?').get(scoped);-- if (row) {-- if (row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });++ const path = `${req.baseUrl}${req.path}`;++ let claimed;++ let row = null;++ try {++ const at = now();++ claimed = await db.maybe(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)++ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${PENDING}, '', ${iso(at)})++ ON CONFLICT (key) DO UPDATE SET request_hash = excluded.request_hash, method = excluded.method, path = excluded.path, created_at = excluded.created_at++ WHERE api_idempotency.status = ${PENDING} AND api_idempotency.created_at < ${iso(at - CLAIM_MS)}++ RETURNING key`);++ if (!claimed) row = await db.maybe(sql`SELECT request_hash, status, response FROM api_idempotency WHERE key = ${scoped}`);++ } catch (e) { return next(e); }++ if (!claimed) {++ if (row && row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });++ if (!row || row.status === PENDING) {++ res.setHeader('Retry-After', '1');++ return http.sendProblem(res, 409, 'idempotency.in_progress', { detail: 'a request with this Idempotency-Key is still being handled; retry shortly for its answer', ctx: req.ov });++ }res.setHeader('Idempotent-Replayed', 'true');-- return res.status(row.status).json(JSON.parse(row.response));++ // The stored text is the answer as first sent (jsonb would reorder its keys).++ return res.status(row.status).type('application/json').send(row.response);}req.idempotencyKey = `api:${scoped}`;++ let answered = false;++ const release = () => {++ if (answered) return;++ answered = true;++ db.exec(sql`DELETE FROM api_idempotency WHERE key = ${scoped} AND status = ${PENDING} AND request_hash = ${hash}`)++ .catch((e) => log.warn(`[Tips] Idempotency-Key claim not released: ${e.message}`));++ };++ res.on('finish', release);++ res.on('close', release);const json = res.json.bind(res);res.json = (body) => {-- if (res.statusCode >= 200 && res.statusCode < 300) {-- // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the-- // replay gets that instead, so no secret is kept in plain text here.-- const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;-- db.prepare('INSERT OR IGNORE INTO api_idempotency (key, request_hash, method, path, status, response, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')-- .run(scoped, hash, req.method, `${req.baseUrl}${req.path}`, res.statusCode, JSON.stringify(stored), new Date(now()).toISOString());-- }-- return json(body);++ if (answered || res.statusCode < 200 || res.statusCode >= 300) return json(body);++ answered = true;++ // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the++ // replay gets that instead, so no secret is kept in plain text here. An upsert: an erasure++ // during the request may have removed the claim.++ const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;++ db.exec(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)++ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${res.statusCode}, ${JSON.stringify(stored)}, ${iso(now())})++ ON CONFLICT (key) DO UPDATE SET status = excluded.status, response = excluded.response, created_at = excluded.created_at++ WHERE api_idempotency.request_hash = excluded.request_hash`)++ .catch((e) => log.warn(`[Tips] stored answer for an Idempotency-Key not kept: ${e.message}`))++ .then(() => json(body));++ return res;};return next();};@@@ -57,7 +97,7 @@ function idempotent(db, now) {*/const RETAIN_MS = 7 * 24 * 3600 * 1000;function pruneAnswers(db, now = Date.now(), retainMs = RETAIN_MS) {-- return db.prepare('DELETE FROM api_idempotency WHERE created_at < ?').run(new Date(now - retainMs).toISOString()).changes;++ return db.exec(sql`DELETE FROM api_idempotency WHERE created_at < ${new Date(now - retainMs).toISOString()}`);}module.exports = { idempotent, stable, principalKey, pruneAnswers, RETAIN_MS };@@@ -40,7 +40,7 @@*/const express = require('express');const { http } = require('openvibe-contracts');--const { TipsError, fail, userSubject, displayName } = require('../util');++const { TipsError, fail, userSubject, displayName, inputError } = require('../util');const { idempotent } = require('./idempotency');const CAP = {@@@ -67,7 +67,7 @@ const CAP = {function v1Router({ domain, apiAuth, limits }) {const r = express.Router();const { db, profiles, goals, interactions, overlays } = domain;-- const idem = idempotent(db, domain.now);++ const idem = idempotent(db, domain.now, domain.log);// Per-actor limits (api/actor-limits.js), after the credential and before any work: every read takes// the defaults; each write below names its own, tighter numbers.@@@ -86,40 +86,40 @@ function v1Router({ domain, apiAuth, limits }) {return null;}/** The creator a call is about: body/query `creator` (subject or handle), or the signed-in user. */-- function creatorOf(req, value, { mustExist = true } = {}) {++ async function creatorOf(req, value, { mustExist = true } = {}) {const v = value != null && value !== '' ? value : (req.principal.kind === 'user' ? req.principal.subject : null);if (!v) fail(422, 'tips.invalid_input', 'creator is required');const ref = typeof v === 'object' ? userSubject(v, 'creator') : String(v);-- const profile = ref === 'me' && req.principal.kind === 'user' ? profiles.bySubject(req.principal.subject) : profiles.resolve(ref);++ const profile = ref === 'me' && req.principal.kind === 'user' ? await profiles.bySubject(db, req.principal.subject) : await profiles.resolve(db, ref);if (!profile && mustExist) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');return profile;}const write = (handler) => [idem, wrap(handler)];// ── Profiles ─────────────────────────────────────────────-- r.get('/profiles/:creator', wrap((req, res) => {-- const p = creatorOf(req, req.params.creator);++ r.get('/profiles/:creator', wrap(async (req, res) => {++ const p = await creatorOf(req, req.params.creator);const who = req.principal;const mine = who.kind === 'user' && who.subject === p.creator_subject;const svc = who.kind === 'service' && apiAuth.granted(who, CAP.profileGet);// A page that is switched off does not exist for anyone but its creator and granted services.if (!p.page_enabled && !mine && !svc) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');const owner = mine || svc;-- const active = goals.list(p.creator_subject, { status: 'active' });-- res.json({ profile: profiles.present(p, { owner }), goals: active.map((g) => (owner ? goals.present(g) : goals.publicGoal(g, p.page))) });++ const active = await goals.list(db, p.creator_subject, { status: 'active' });++ res.json({ profile: profiles.present(p, { owner }), goals: owner ? await goals.presentMany(db, active) : await goals.publicGoals(db, active, p) });}));/** The supporters page: top supporters and recent public messages, as the creator chose to show them. */-- r.get('/profiles/:creator/supporters', wrap((req, res) => {-- const p = creatorOf(req, req.params.creator);++ r.get('/profiles/:creator/supporters', wrap(async (req, res) => {++ const p = await creatorOf(req, req.params.creator);const who = req.principal;const owner = (who.kind === 'user' && who.subject === p.creator_subject) || (who.kind === 'service' && apiAuth.granted(who, CAP.profileGet));if (!owner && (!p.page_enabled || !p.page.supporters_page)) fail(404, 'tips.supporters_not_public', 'this creator does not show their supporters');const page = p.page;res.json({creator: { type: 'user', id: p.creator_subject }, page,-- leaderboard: interactions.leaderboard(p.creator_subject).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),-- recent: page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,++ leaderboard: (await interactions.leaderboard(db, p)).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),++ recent: page.supporters_messages ? (await interactions.recentPublic(db, p)).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,});}));@@@ -128,34 +128,34 @@ function v1Router({ domain, apiAuth, limits }) {if (req.principal.kind !== 'user') fail(req.principal.kind === 'anonymous' ? 401 : 403, req.principal.kind === 'anonymous' ? 'token.missing' : 'tips.people_only', 'only the person themselves (a Network user token) can do this');return req.principal.subject;};-- r.get('/me/export', wrap((req, res) => res.json(interactions.exportFor(person(req)))));++ r.get('/me/export', wrap(async (req, res) => res.json(await interactions.exportFor(person(req)))));// A person erasing their own tips does it once; the network's account deletion arrives through// /internal/events, which is never limited.-- r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write((req, res) => res.json(interactions.erase(person(req)))));++ r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write(async (req, res) => res.json(await interactions.erase(person(req)))));-- r.patch('/profiles/:creator', edit, ...write((req, res) => {++ r.patch('/profiles/:creator', edit, ...write(async (req, res) => {const b = req.body || {};let p;if (req.params.creator === 'me') {if (req.principal.kind !== 'user') fail(401, 'token.missing', '`me` needs a user token');-- p = profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });-- } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !profiles.bySubject(req.params.creator)) {++ p = await profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });++ } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !(await profiles.bySubject(db, req.params.creator))) {allow(req, CAP.profileUpdate);// A service opening a profile for a creator names the account projection it knows.if (!b.handle) fail(422, 'tips.invalid_input', 'handle is required to create a profile');-- p = profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });++ p = await profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });} else {-- p = creatorOf(req, req.params.creator);++ p = await creatorOf(req, req.params.creator);}allow(req, CAP.profileUpdate, p.creator_subject);-- const out = profiles.update(p.creator_subject, b, { expectedRevision: b.revision });++ const out = await profiles.update(p.creator_subject, b, { expectedRevision: b.revision });res.json({ profile: profiles.present(out, { owner: true }) });}));-- r.get('/profiles/:creator/totals', wrap((req, res) => {-- const p = creatorOf(req, req.params.creator);++ r.get('/profiles/:creator/totals', wrap(async (req, res) => {++ const p = await creatorOf(req, req.params.creator);allow(req, CAP.interactionList, p.creator_subject);-- res.json({ creator: { type: 'user', id: p.creator_subject }, totals: interactions.totals(p.creator_subject) });++ res.json({ creator: { type: 'user', id: p.creator_subject }, totals: await interactions.totals(db, p.creator_subject) });}));// ── Paid requests (checkout / credit) ────────────────────@@@ -174,28 +174,27 @@ function v1Router({ domain, apiAuth, limits }) {supporter = p.subject;supporterName = displayName(b.supporter_name) || p.name || p.username;}-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);const funding = b.pay_with || 'credit';const tp = req.ov && req.ov.traceparent;-- const { interaction, replay } = interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });++ const { interaction, replay } = await interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });let i = interaction;-- let checkout = null;++ const viewer = p.kind === 'service' ? 'service' : 'supporter';if (!replay) {if (funding === 'credit') {const out = await interactions.transfer(i, { traceparent: tp });-- i = interactions.get(i.id);++ i = await interactions.get(db, i.id);if (out.refused) {-- fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: interactions.present(i, { viewer: 'supporter' }) });++ fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });}} else {-- checkout = await interactions.startCheckout(i, { provider: b.provider, traceparent: tp });-- i = checkout.interaction;++ i = (await interactions.startCheckout(i, { provider: b.provider, traceparent: tp })).interaction;}} else if (i.payment_state === 'failed') {-- fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: interactions.present(i, { viewer: 'supporter' }) });++ fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });}res.status(replay ? 200 : 201).json({-- interaction: interactions.present(i, { viewer: p.kind === 'service' ? 'service' : 'supporter' }),++ interaction: await interactions.present(db, i, { viewer }),checkout_url: i.checkout_url || null, checkout_ref: i.checkout_ref || null,});});@@@ -212,27 +211,29 @@ function v1Router({ domain, apiAuth, limits }) {r.post('/media-requests', pays, ...paid('media_request', CAP.media));// ── Interactions ─────────────────────────────────────────-- r.get('/interactions', wrap((req, res) => {++ // The hottest list: a keyset page of any size costs three queries (the page, the creators' filters,++ // the effects) — test/query-budget.test.js holds it there.++ r.get('/interactions', wrap(async (req, res) => {const p = req.principal;const q = req.query;let filter;if (p.kind === 'service') {allow(req, CAP.interactionList);filter = {-- creator: q.creator ? (creatorOf(req, q.creator).creator_subject) : null,++ creator: q.creator ? (await creatorOf(req, q.creator)).creator_subject : null,supporter: q.supporter ? userSubject(String(q.supporter), 'supporter') : null,};if (!filter.creator && !filter.supporter) fail(422, 'tips.invalid_input', 'creator or supporter is required');} else if (p.kind === 'user') {filter = q.as === 'creator' ? { creator: p.subject } : { supporter: p.subject };} else fail(401, 'token.missing', 'sign in to see receipts');-- const out = interactions.list({ ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });++ const out = await interactions.list(db, { ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });const viewer = p.kind === 'service' ? 'service' : (filter.creator ? 'owner' : 'supporter');-- res.json({ interactions: out.rows.map((i) => interactions.present(i, { viewer })), next_cursor: out.next_cursor });++ res.json({ interactions: await interactions.presentMany(db, out.rows, { viewer }), next_cursor: out.next_cursor });}));-- r.get('/interactions/:id', wrap((req, res) => {-- const i = interactions.get(req.params.id);++ r.get('/interactions/:id', wrap(async (req, res) => {++ const i = await interactions.get(db, req.params.id);const p = req.principal;if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');let viewer;@@@ -241,164 +242,160 @@ function v1Router({ domain, apiAuth, limits }) {else if (p.kind === 'user' && p.subject === i.supporter_subject) viewer = 'supporter';else if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in to see this receipt');else fail(404, 'tips.interaction_not_found', 'no such interaction');-- res.json({ interaction: interactions.present(i, { viewer }) });++ res.json({ interaction: await interactions.present(db, i, { viewer }) });}));// A service importing tips made elsewhere records one per tip, at most one a second.-- r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write((req, res) => {++ r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write(async (req, res) => {allow(req, CAP.interactionRecord);const b = req.body || {};-- const profile = creatorOf(req, b.creator);-- const out = interactions.recordExternal(profile, b);-- res.status(out.replay ? 200 : 201).json({ interaction: interactions.present(out.interaction, { viewer: 'service' }), duplicate: out.replay });++ const profile = await creatorOf(req, b.creator);++ const out = await interactions.recordExternal(profile, b);++ res.status(out.replay ? 200 : 201).json({ interaction: await interactions.present(db, out.interaction, { viewer: 'service' }), duplicate: out.replay });}));// ── Goals ────────────────────────────────────────────────-- function goalVisible(req, g) {-- const profile = profiles.bySubject(g.creator_subject);-- if (profile && profile.page_enabled) return true;-- const p = req.principal;-- return (p.kind === 'user' && p.subject === g.creator_subject) || (p.kind === 'service' && apiAuth.granted(p, CAP.goalUpdate));-- }-- const goalOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));-- r.get('/goals', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);-- const sample = { creator_subject: profile.creator_subject };-- if (!goalVisible(req, sample)) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');++ const isOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));++ r.get('/goals', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);++ const full = isOwner(req, profile.creator_subject);++ if (!profile.page_enabled && !full) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');const status = ['active', 'closed'].includes(req.query.status) ? req.query.status : undefined;-- const full = goalOwner(req, profile.creator_subject);-- res.json({ goals: goals.list(profile.creator_subject, { status }).map((g) => (full ? goals.present(g) : goals.publicGoal(g, profile.page))) });++ const list = await goals.list(db, profile.creator_subject, { status });++ res.json({ goals: full ? await goals.presentMany(db, list) : await goals.publicGoals(db, list, profile) });}));-- r.get('/goals/:id', wrap((req, res) => {-- const g = goals.get(req.params.id);-- if (!g || !goalVisible(req, g)) fail(404, 'tips.goal_not_found', 'no such goal');-- if (!goalOwner(req, g.creator_subject)) return res.json({ goal: goals.publicGoal(g, profiles.bySubject(g.creator_subject).page) });-- res.json({ goal: goals.present(g, { contributions: true }) });++ r.get('/goals/:id', wrap(async (req, res) => {++ const g = await goals.get(db, req.params.id);++ const profile = g && await profiles.bySubject(db, g.creator_subject);++ const full = g && isOwner(req, g.creator_subject);++ if (!g || (!(profile && profile.page_enabled) && !full)) fail(404, 'tips.goal_not_found', 'no such goal');++ if (!full) return res.json({ goal: await goals.publicGoal(db, g, profile) });++ res.json({ goal: await goals.present(db, g, { contributions: true }) });}));-- r.post('/goals', edit, ...write((req, res) => {++ r.post('/goals', edit, ...write(async (req, res) => {const b = req.body || {};-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);allow(req, CAP.goalCreate, profile.creator_subject);-- res.status(201).json({ goal: goals.create(profile.creator_subject, b) });++ res.status(201).json({ goal: await goals.create(profile.creator_subject, b) });}));-- const ownGoal = (req, cap) => {-- const g = goals.get(req.params.id);++ const ownGoal = async (req, cap) => {++ const g = await goals.get(db, req.params.id);if (!g) fail(404, 'tips.goal_not_found', 'no such goal');allow(req, cap, g.creator_subject);return g;};-- r.patch('/goals/:id', edit, ...write((req, res) => res.json({ goal: goals.update(ownGoal(req, CAP.goalUpdate), req.body || {}) })));-- r.post('/goals/:id/close', edit, ...write((req, res) => res.json({ goal: goals.close(ownGoal(req, CAP.goalClose)) })));++ r.patch('/goals/:id', edit, ...write(async (req, res) => res.json({ goal: await goals.update(await ownGoal(req, CAP.goalUpdate), req.body || {}) })));++ r.post('/goals/:id/close', edit, ...write(async (req, res) => res.json({ goal: await goals.close(await ownGoal(req, CAP.goalClose)) })));// ── Overlay tokens and configs ───────────────────────────-- r.get('/overlay-tokens', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);++ r.get('/overlay-tokens', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);allow(req, CAP.tokenCreate, profile.creator_subject);-- res.json({ tokens: overlays.listTokens(profile.creator_subject).map(overlays.presentToken) });++ res.json({ tokens: (await overlays.listTokens(db, profile.creator_subject)).map(overlays.presentToken) });}));// A token opens an overlay's event stream: a creator mints a few for their scenes.-- r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write((req, res) => {++ r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write(async (req, res) => {const b = req.body || {};-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);allow(req, CAP.tokenCreate, profile.creator_subject);const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;-- const out = overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });++ const out = await overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });// Shown once: only its hash is stored, and the stored answer for an Idempotency-Key replay has no// secret either (a caller who lost the answer revokes the token and creates another).res.locals.storedBody = { token: out.token, secret: null, overlay_url: null, events_url: null, note: 'the secret is shown once; revoke this token and create another' };res.status(201).json({ token: out.token, secret: out.secret, overlay_url: out.overlay_url, events_url: out.events_url });}));-- r.post('/overlay-tokens/:id/revoke', edit, ...write((req, res) => {-- const t = overlays.getToken(req.params.id);++ r.post('/overlay-tokens/:id/revoke', edit, ...write(async (req, res) => {++ const t = await overlays.getToken(db, req.params.id);if (!t) fail(404, 'tips.overlay_token_not_found', 'no such overlay token');allow(req, CAP.tokenRevoke, t.creator_subject);-- res.json({ token: overlays.revokeToken(t) });++ res.json({ token: await overlays.revokeToken(t) });}));-- r.get('/overlay-configs', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);++ r.get('/overlay-configs', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);allow(req, CAP.configGet, profile.creator_subject);-- res.json({ configs: overlays.listConfigs(profile.creator_subject).map(overlays.presentConfig) });++ res.json({ configs: (await overlays.listConfigs(db, profile.creator_subject)).map(overlays.presentConfig) });}));-- r.get('/overlay-configs/:id', wrap((req, res) => {-- const c = overlays.getConfig(req.params.id);++ r.get('/overlay-configs/:id', wrap(async (req, res) => {++ const c = await overlays.getConfig(db, req.params.id);if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');allow(req, CAP.configGet, c.creator_subject);res.json({ config: overlays.presentConfig(c) });}));-- r.post('/overlay-configs', edit, ...write((req, res) => {++ r.post('/overlay-configs', edit, ...write(async (req, res) => {const b = req.body || {};-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);allow(req, CAP.configUpdate, profile.creator_subject);-- res.status(201).json({ config: overlays.createConfig(profile.creator_subject, b) });++ res.status(201).json({ config: await overlays.createConfig(profile.creator_subject, b) });}));-- r.patch('/overlay-configs/:id', edit, ...write((req, res) => {-- const c = overlays.getConfig(req.params.id);++ r.patch('/overlay-configs/:id', edit, ...write(async (req, res) => {++ const c = await overlays.getConfig(db, req.params.id);if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');allow(req, CAP.configUpdate, c.creator_subject);-- res.json({ config: overlays.updateConfig(c, req.body || {}) });++ res.json({ config: await overlays.updateConfig(c, req.body || {}) });}));// ── Moderation (server/domain/moderation.js) ─────────────const { moderation } = domain;/** The creator, one of their moderators, or a service with tips.interaction.moderate. */-- function moderatorOf(req, creator, { hideExistence = false } = {}) {++ async function moderatorOf(req, creator, { hideExistence = false } = {}) {const p = req.principal;if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in (a Network user token) or present a service token');-- const a = moderation.actorFor(p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));++ const a = await moderation.actorFor(db, p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));if (a) return a;if (p.kind === 'service') fail(403, 'capability.denied', `${CAP.moderate} not granted`);if (hideExistence) fail(404, 'tips.interaction_not_found', 'no such interaction');fail(403, 'tips.forbidden', 'only the creator and their moderators moderate this page');return null;}-- r.get('/moderation', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);-- moderatorOf(req, profile.creator_subject);-- const out = moderation.queue(profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });++ r.get('/moderation', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);++ await moderatorOf(req, profile.creator_subject);++ const out = await moderation.queue(db, profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });res.json({ creator: { type: 'user', id: profile.creator_subject }, interactions: out.rows, next_cursor: out.next_cursor });}));-- r.get('/moderation/log', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);++ r.get('/moderation/log', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);allow(req, CAP.moderate, profile.creator_subject);-- res.json({ log: moderation.log(profile.creator_subject) });++ res.json({ log: await moderation.log(db, profile.creator_subject) });}));for (const action of ['hide', 'restore']) {-- r.post(`/interactions/:id/${action}`, moderate, ...write((req, res) => {-- const i = interactions.get(req.params.id);++ r.post(`/interactions/:id/${action}`, moderate, ...write(async (req, res) => {++ const i = await interactions.get(db, req.params.id);if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');-- const actor = moderatorOf(req, i.creator_subject, { hideExistence: true });-- const out = moderation[action](i, { ...actor, reason: (req.body || {}).reason });-- res.json({ interaction: moderation.present(out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });++ const actor = await moderatorOf(req, i.creator_subject, { hideExistence: true });++ const out = await moderation[action](i, { ...actor, reason: (req.body || {}).reason });++ res.json({ interaction: await moderation.present(db, out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });}));}-- r.get('/moderators', wrap((req, res) => {-- const profile = creatorOf(req, req.query.creator);++ r.get('/moderators', wrap(async (req, res) => {++ const profile = await creatorOf(req, req.query.creator);allow(req, CAP.profileUpdate, profile.creator_subject);-- res.json({ moderators: moderation.listModerators(profile.creator_subject).map(moderation.presentModerator) });++ res.json({ moderators: (await moderation.listModerators(db, profile.creator_subject)).map(moderation.presentModerator) });}));-- r.post('/moderators', staffing, ...write((req, res) => {++ r.post('/moderators', staffing, ...write(async (req, res) => {const b = req.body || {};-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);allow(req, CAP.profileUpdate, profile.creator_subject);const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;-- res.status(201).json({ moderator: moderation.addModerator(profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });++ res.status(201).json({ moderator: await moderation.addModerator(db, profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });}));-- r.post('/moderators/:subject/remove', staffing, ...write((req, res) => {-- const profile = creatorOf(req, (req.body || {}).creator);++ r.post('/moderators/:subject/remove', staffing, ...write(async (req, res) => {++ const profile = await creatorOf(req, (req.body || {}).creator);allow(req, CAP.profileUpdate, profile.creator_subject);-- res.json(moderation.removeModerator(profile.creator_subject, req.params.subject));++ res.json(await moderation.removeModerator(profile.creator_subject, req.params.subject));}));// ── Simulation ───────────────────────────────────────────// A simulated tip plays on the creator's overlays and chat like a real one.-- r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write((req, res) => {++ r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write(async (req, res) => {const b = req.body || {};-- const profile = creatorOf(req, b.creator);++ const profile = await creatorOf(req, b.creator);allow(req, CAP.simulate, profile.creator_subject);const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;-- const i = interactions.simulate(profile, b, { by });-- res.status(201).json({ interaction: interactions.present(i, { viewer: 'owner' }) });++ const i = await interactions.simulate(profile, b, { by });++ res.status(201).json({ interaction: await interactions.present(db, i, { viewer: 'owner' }) });}));return r;@@@ -416,6 +413,7 @@ function wrap(fn) {function sendError(req, res, e) {if (res.headersSent) return;++ e = inputError(e) || e;if (e instanceof TipsError) {return http.sendProblem(res, e.status, e.code, { detail: e.detail || e.message, ctx: req.ov, extra: e.extra && e.status < 500 ? { details: e.extra } : undefined });}@@@ -11,12 +11,16 @@* /overlay/:token[/events|/state] overlays (scoped revocable tokens)* /, /dashboard, /receipts, /:handle … server-rendered pages*-- * createApp({ config, db, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.++ * createApp({ config, db, valkey, registry, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.++ * `db` is an openvibe-sdk/db handle with the schema migrated (server/index.js migrates at boot; tests++ * pass a PGlite one); `valkey` an openvibe-sdk/valkey handle or null (everything shared stays in this++ * process).*/const path = require('path');const express = require('express');const cookieParser = require('cookie-parser');const { http } = require('openvibe-contracts');++const { isLoopbackDirect } = require('openvibe-shared/metrics');const { loadConfig } = require('./config');const { openDb } = require('./db');const { createKeyProvider, createUserAuth } = require('./network');@@@ -32,6 +36,7 @@ const { createSessionRoutes } = require('./web/session');const { createWebRoutes } = require('./web/routes');const { createLayout, assetVersion } = require('./web/layout');const pages = require('./web/pages');++const { inputError } = require('./util');const { createTipsReadiness, registerTipsGauges } = require('./observability');const VERSION = require('../package.json').version;@@@ -39,7 +44,8 @@ const PUBLIC_DIR = path.join(__dirname, '..', 'public');function createApp(opts = {}) {const config = opts.config || loadConfig();-- const db = opts.db || openDb(config.dbPath);++ const db = opts.db || openDb(config);++ const valkey = opts.valkey || null;const fetchImpl = opts.fetchImpl || globalThis.fetch;const log = opts.log || console;const now = opts.now || (() => Date.now());@@@ -48,7 +54,7 @@ function createApp(opts = {}) {const billing = opts.billing || createBillingClient(config, { fetchImpl });const adapters = opts.adapters || createAdapters(config, { fetchImpl });const outbox = opts.outbox || createTipsOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });-- const domain = createDomain({ db, config, outbox, billing, adapters, now, log });++ const domain = createDomain({ db, config, outbox, billing, adapters, valkey, now, log });const apiAuth = createApiAuth({ config, keys, userAuth });const release = require('openvibe-shared/release').createRelease({ service: 'tips', root: path.join(__dirname, '..') });const layout = createLayout({ config, release });@@@ -58,10 +64,13 @@ function createApp(opts = {}) {app.set('trust proxy', config.trustProxy);// HTTP golden signals by route template, process metrics, release_info; GET /metrics answers// direct loopback callers only (Track O). An overlay's SSE stream is a session, not a request.++ // The Tips gauges are read from the database just before a direct scrape renders (one query).++ const registry = opts.registry || require('openvibe-shared/metrics').createRegistry();++ const gauges = registerTipsGauges(registry, { db, now, log });++ app.get('/metrics', (req, res, next) => (isLoopbackDirect(req) ? gauges.refresh().then(() => next(), () => next()) : next()));const metrics = require('openvibe-shared/metrics').instrument(app, {-- service: 'tips', release: release.release, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),++ service: 'tips', release: release.release, registry, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),});-- registerTipsGauges(metrics.registry, { db, outbox, now });app.use(http.middleware());app.use((req, res, next) => {res.setHeader('X-Content-Type-Options', 'nosniff');@@@ -77,12 +86,12 @@ function createApp(opts = {}) {});app.use(cookieParser());-- app.get('/api/health', (req, res) => res.json({-- ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events: outbox.status(),-- }));-- // Readiness (openvibe-shared/ready): 503 only when the database fails; Network key, Billing and-- // Events are optional and degrade it (see observability.js).-- const readiness = createTipsReadiness({ db, keys, config, outbox, release: release.release, fetchImpl });++ app.get('/api/health', (req, res, next) => outbox.status().then((events) => res.json({++ ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events,++ }), next));++ // Readiness (openvibe-shared/ready): 503 only when the database fails; Valkey, the Network key,++ // Billing and Events are optional and degrade it (see observability.js).++ const readiness = createTipsReadiness({ db, valkey, keys, config, outbox, release: release.release, fetchImpl });app.get('/api/ready', readiness.handler);// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports into /metrics.release.mount(app, { registry: metrics.registry });@@@ -94,7 +103,7 @@ function createApp(opts = {}) {? http.sendProblem(res, 404, 'not_found', { ctx: req.ov }) : next()), consumer.router);// Per-actor limits on /api/v1 (api/actor-limits.js), counted once apiAuth resolved the caller.// opts.limitsNow: the limiter's clock (tests).-- const limits = createActorLimits({ config, now: opts.limitsNow || now, registry: metrics.registry, log });++ const limits = createActorLimits({ config, valkey, now: opts.limitsNow || now, registry: metrics.registry, log });app.use('/api/v1', express.json({ limit: '64kb' }), (req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); }, apiAuth.middleware, v1Router({ domain, apiAuth, limits }));app.use('/auth', createSessionRoutes(config, userAuth, { fetchImpl }));{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'tips', service: 'tips', host: 'openvibe.tips', name: 'OpenVibe.Tips', profile: 'ugc' })); }@@@ -119,13 +128,18 @@ function createApp(opts = {}) {app.use((err, req, res, next) => {if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { ctx: req.ov });if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });++ const refused = inputError(err);++ if (refused && !res.headersSent) {++ if (req.path.startsWith('/api/')) return http.sendProblem(res, 422, refused.code, { detail: refused.detail, ctx: req.ov });++ return res.status(400).type('html').send(layout.page({ title: 'Not accepted', robots: 'noindex', body: pages.errorPage({ status: 400, title: 'That could not be saved', message: refused.detail }) }));++ }log.error('[Tips] unhandled error:', err);if (res.headersSent) return undefined;if (req.path.startsWith('/api/')) return http.sendProblem(res, 500, 'tips.internal', { ctx: req.ov });return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));});-- Object.assign(app.locals, { config, db, domain, keys, outbox, adapters, billing, consumer, metrics });++ Object.assign(app.locals, { config, db, valkey, domain, keys, outbox, adapters, billing, consumer, metrics });return app;}@@@ -8,6 +8,7 @@* never prices or rates. Prices and the value of a bit are Billing's.*/require('dotenv').config();++const crypto = require('crypto');const int = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) ? n : d; };const bool = (v, d = false) => (v == null || v === '' ? d : ['1', 'true', 'yes', 'on'].includes(String(v).toLowerCase()));@@@ -28,7 +29,23 @@ function loadConfig(env = process.env) {host: env.HOST || '127.0.0.1',baseUrl,trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1,-- dbPath: env.TIPS_DB_PATH || './data/tips.db',++++ // PostgreSQL (ADR-035; OpenVibe.Host's data role writes all four into /etc/openvibe/tips.env):++ // DATABASE_URL is the pooled runtime role through PgBouncer (transaction mode), DATABASE_DIRECT_URL++ // the owner role on a direct connection, for migrations at boot and the one-time import.++ db: {++ url: env.DATABASE_URL || '',++ directUrl: env.DATABASE_DIRECT_URL || '',++ },++ // Valkey: shared, never authoritative (per-actor limit counters, overlay fan-out between++ // processes, overlay stream slots). Unset: all of it stays in this process (one process only).++ valkey: {++ url: env.VALKEY_URL || '',++ prefix: env.VALKEY_PREFIX || 'ov:tips:',++ },++ // The SQLite file of the release before PostgreSQL: read (never written) only by++ // scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.++ sqlitePath: env.TIPS_DB_PATH || './data/tips.db',// Identity: service tokens and user tokens are RS256 JWTs signed by OpenVibe.Network.network: {@@@ -48,8 +65,11 @@ function loadConfig(env = process.env) {scope: env.OV_OAUTH_SCOPE || 'profile',},cookies: { secure: env.COOKIE_SECURE != null ? bool(env.COOKIE_SECURE) : isProduction },-- // Signs the anti-forgery tokens of the server-rendered forms.-- formSecret: env.TIPS_FORM_SECRET || '',++ // Signs the anti-forgery tokens of the server-rendered forms. Every process must use the same++ // one (a form rendered by one process is posted to another), so without TIPS_FORM_SECRET it is++ // derived from the client secret; only with neither is it per process (development).++ formSecret: env.TIPS_FORM_SECRET++ || (env.OV_OAUTH_CLIENT_SECRET ? crypto.createHmac('sha256', env.OV_OAUTH_CLIENT_SECRET).update('openvibe.tips form secret').digest('hex') : ''),billing: {url: trim(env.BILLING_URL || 'http://127.0.0.1:4600'),@@@ -1,346 +1,62 @@'use strict';/**-- * Tips' own SQLite database (WAL), created on boot, idempotent.++ * Tips' own PostgreSQL database (ADR-035), through openvibe-sdk/db: async, pooled, one dialect.*-- * The ten product tables (roadmap §15.11) store interaction state and REFERENCES to Billing-- * transactions — never a mutable cash balance. Amounts are integer vibes-bits as Billing counts-- * them; a creator's totals are always derived from settled interactions.++ * The schema is migrations/NNNN_*.sql (expand/migrate/contract, ADR-028), applied at boot by++ * migrate() on the owner's direct connection (DATABASE_DIRECT_URL); the service then serves on the++ * pooled runtime role (DATABASE_URL, PgBouncer in transaction mode: no session state, 10 connections++ * per process at most).++ *++ * The product tables (roadmap §15.11) store interaction state and REFERENCES to Billing transactions,++ * never a mutable cash balance. Amounts are integer vibes-bits as Billing counts them; a creator's++ * totals are always derived from settled interactions.** creator_tip_profiles one per creator subject: page switch, minimums, TTS/media settings* tip_interactions one logical interaction; payment_state (mirrored from Billing, keyed by* Billing txn id) is separate from delivery_state (Tips' own effects)* tip_goals creator goals* tip_goal_contributions settled interactions counted toward a goal (never tests, never pending)-- * paid_messages highlighted paid chat messages and TTS requests — created on settlement-- * paid_media_requests paid media requests — created on settlement++ * paid_messages highlighted paid chat messages and TTS requests, created on settlement++ * paid_media_requests paid media requests, created on settlement* overlay_configs alert / goal overlay settings* overlay_deliveries what overlays were sent (monotonic seq per creator, replayable)* interaction_effects the retryable delivery work of an interaction (chat line, TTS, media, overlay)* migration_maps legacy (Live) row → Tips entity, with held/excluded reasons*-- * Supporting: overlay_tokens (hashed, scoped, revocable), api_idempotency, import_runs,-- * event_outbox + idempotency_receipts (openvibe-sdk outbox and inbox), settings. Columns added after-- * v1 are in COLUMNS below.++ * Supporting: overlay_tokens (hashed, scoped, revocable), tip_moderators / _invites / _moderation_log,++ * api_idempotency, import_runs, tips_event_outbox + tips_event_inbox (openvibe-sdk outbox and inbox).++ *++ * openDb(config) the serving handle: DATABASE_URL; in development without it, an embedded++ * PGlite database in data/pglite (one process, nothing to install)++ * migrate(config, {serving}) apply migrations/ with the owner role (DATABASE_DIRECT_URL), then close++ * that connection; the embedded database migrates on its serving handle*/const fs = require('fs');const path = require('path');--const Database = require('better-sqlite3');----const SCHEMA = `--CREATE TABLE IF NOT EXISTS settings (-- id INTEGER PRIMARY KEY CHECK (id = 1),-- schema_version INTEGER NOT NULL,-- created_at TEXT NOT NULL--);----CREATE TABLE IF NOT EXISTS creator_tip_profiles (-- creator_subject TEXT PRIMARY KEY, -- usr_…-- handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)-- display_name TEXT NOT NULL,-- avatar_url TEXT,-- headline TEXT,-- page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing-- accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted-- min_amount INTEGER NOT NULL DEFAULT 1,-- paid_message_min INTEGER NOT NULL DEFAULT 100,-- tts_enabled INTEGER NOT NULL DEFAULT 0,-- tts_min_amount INTEGER NOT NULL DEFAULT 100,-- tts_max_chars INTEGER NOT NULL DEFAULT 200,-- tts_voice TEXT NOT NULL DEFAULT 'gary',-- media_requests_enabled INTEGER NOT NULL DEFAULT 0,-- media_request_min INTEGER NOT NULL DEFAULT 25,-- media_max_seconds INTEGER NOT NULL DEFAULT 600,-- revision INTEGER NOT NULL DEFAULT 1,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);----CREATE TABLE IF NOT EXISTS tip_interactions (-- id TEXT PRIMARY KEY, -- tint_<ULID>-- creator_subject TEXT NOT NULL,-- supporter_subject TEXT, -- null: external/anonymous/unmapped-- supporter_name TEXT, -- display projection at the time-- kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),-- amount INTEGER NOT NULL CHECK (amount > 0),-- currency TEXT NOT NULL DEFAULT 'vibes-bits',-- amount_cents INTEGER, -- money that arrived (provider/external)-- message TEXT,-- request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target-- funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),-- settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),-- payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),-- delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),-- test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted-- billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it-- billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent-- funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it-- reversal_txn_ids TEXT NOT NULL DEFAULT '[]',-- reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator-- checkout_url TEXT,-- checkout_ref TEXT,-- provider TEXT,-- provider_ref TEXT,-- legacy_source TEXT, -- 'live:transactions:<id>' etc.-- idempotency_key TEXT UNIQUE,-- failure TEXT,-- origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import-- transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried-- transfer_attempts INTEGER NOT NULL DEFAULT 0,-- next_transfer_at INTEGER NOT NULL DEFAULT 0,-- target TEXT, -- JSON EntityRef (e.g. a Live stream)-- created_at TEXT NOT NULL,-- settled_at TEXT,-- reversed_at TEXT,-- delivered_at TEXT,-- updated_at TEXT NOT NULL,-- UNIQUE (provider, provider_ref),-- UNIQUE (legacy_source)--);--CREATE INDEX IF NOT EXISTS idx_tint_creator ON tip_interactions (creator_subject, created_at);--CREATE INDEX IF NOT EXISTS idx_tint_supporter ON tip_interactions (supporter_subject, created_at);--CREATE INDEX IF NOT EXISTS idx_tint_pending ON tip_interactions (payment_state, funding);----CREATE TABLE IF NOT EXISTS tip_goals (-- id TEXT PRIMARY KEY, -- tgoal_<ULID>-- creator_subject TEXT NOT NULL,-- title TEXT NOT NULL,-- description TEXT,-- target_amount INTEGER NOT NULL CHECK (target_amount > 0),-- currency TEXT NOT NULL DEFAULT 'vibes-bits',-- image_url TEXT,-- status TEXT NOT NULL CHECK (status IN ('active', 'closed')),-- sort_order INTEGER NOT NULL DEFAULT 0,-- reached_at TEXT,-- closed_at TEXT,-- opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)-- legacy_source TEXT UNIQUE,-- revision INTEGER NOT NULL DEFAULT 1,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_goal_creator ON tip_goals (creator_subject, status);----CREATE TABLE IF NOT EXISTS tip_goal_contributions (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- goal_id TEXT NOT NULL REFERENCES tip_goals(id),-- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),-- amount INTEGER NOT NULL CHECK (amount > 0),-- reversed_amount INTEGER NOT NULL DEFAULT 0,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL,-- UNIQUE (goal_id, interaction_id)--);----CREATE TABLE IF NOT EXISTS paid_messages (-- id TEXT PRIMARY KEY, -- tpm_<ULID>-- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),-- creator_subject TEXT NOT NULL,-- kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),-- text TEXT NOT NULL,-- voice TEXT,-- highlight_seconds INTEGER NOT NULL DEFAULT 0,-- status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),-- chat_ref TEXT, -- JSON reference the chat adapter returned-- test INTEGER NOT NULL DEFAULT 0,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);----CREATE TABLE IF NOT EXISTS paid_media_requests (-- id TEXT PRIMARY KEY, -- tmr_<ULID>-- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),-- creator_subject TEXT NOT NULL,-- url TEXT NOT NULL,-- provider TEXT,-- status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),-- queue_ref TEXT, -- JSON reference returned by the queue owner-- test INTEGER NOT NULL DEFAULT 0,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);----CREATE TABLE IF NOT EXISTS overlay_configs (-- id TEXT PRIMARY KEY, -- tovc_<ULID>-- creator_subject TEXT NOT NULL,-- kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),-- name TEXT NOT NULL,-- settings TEXT NOT NULL DEFAULT '{}',-- goal_id TEXT,-- revision INTEGER NOT NULL DEFAULT 1,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_ovc_creator ON overlay_configs (creator_subject);----CREATE TABLE IF NOT EXISTS overlay_tokens (-- id TEXT PRIMARY KEY, -- tovt_<ULID>-- creator_subject TEXT NOT NULL,-- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once-- scopes TEXT NOT NULL, -- JSON ["alerts","goals"]-- config_id TEXT,-- label TEXT,-- created_by TEXT NOT NULL,-- created_at TEXT NOT NULL,-- last_used_at TEXT,-- revoked_at TEXT--);--CREATE INDEX IF NOT EXISTS idx_ovt_creator ON overlay_tokens (creator_subject);----CREATE TABLE IF NOT EXISTS overlay_deliveries (-- seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)-- id TEXT NOT NULL UNIQUE, -- tovd_<ULID>-- creator_subject TEXT NOT NULL,-- kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),-- interaction_id TEXT,-- goal_id TEXT,-- payload TEXT NOT NULL,-- test INTEGER NOT NULL DEFAULT 0,-- status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),-- delivered_at TEXT,-- failed_at TEXT,-- sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)-- dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>-- created_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_ovd_creator ON overlay_deliveries (creator_subject, seq);--CREATE INDEX IF NOT EXISTS idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';----CREATE TABLE IF NOT EXISTS interaction_effects (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),-- effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),-- adapter TEXT NOT NULL,-- state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),-- attempts INTEGER NOT NULL DEFAULT 0,-- next_attempt_at INTEGER NOT NULL DEFAULT 0,-- last_error TEXT,-- result TEXT,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL,-- UNIQUE (interaction_id, effect)--);--CREATE INDEX IF NOT EXISTS idx_effects_due ON interaction_effects (state, next_attempt_at);++const { createDb } = require('openvibe-sdk/db');--CREATE TABLE IF NOT EXISTS migration_maps (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- source TEXT NOT NULL, -- 'live'-- source_table TEXT NOT NULL,-- source_id TEXT NOT NULL,-- target_type TEXT, -- 'interaction' | 'goal'-- target_id TEXT,-- status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),-- reason TEXT,-- billing_txn_id TEXT,-- run_id TEXT NOT NULL,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL,-- UNIQUE (source, source_table, source_id)--);++const MIGRATIONS = path.join(__dirname, '..', 'migrations');++const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');--CREATE TABLE IF NOT EXISTS import_runs (-- id TEXT PRIMARY KEY,-- source TEXT NOT NULL,-- dry_run INTEGER NOT NULL,-- started_at TEXT NOT NULL,-- finished_at TEXT,-- report TEXT--);----CREATE TABLE IF NOT EXISTS tip_moderators (-- creator_subject TEXT NOT NULL,-- moderator_subject TEXT NOT NULL,-- name TEXT, -- display name when they joined (for the creator's list)-- added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'-- created_at TEXT NOT NULL,-- removed_at TEXT,-- PRIMARY KEY (creator_subject, moderator_subject)--);--CREATE INDEX IF NOT EXISTS idx_tmod_moderator ON tip_moderators (moderator_subject);----CREATE TABLE IF NOT EXISTS tip_moderator_invites (-- id TEXT PRIMARY KEY, -- tmin_<ULID>-- creator_subject TEXT NOT NULL,-- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)-- created_by TEXT NOT NULL,-- created_at TEXT NOT NULL,-- expires_at TEXT NOT NULL,-- used_at TEXT,-- used_by TEXT,-- revoked_at TEXT--);----CREATE TABLE IF NOT EXISTS tip_moderation_log (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- creator_subject TEXT NOT NULL,-- interaction_id TEXT NOT NULL,-- action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),-- by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),-- actor TEXT, -- usr_… / svc:… ; null for the filter-- reason TEXT,-- created_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);----CREATE TABLE IF NOT EXISTS api_idempotency (-- key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>-- request_hash TEXT NOT NULL,-- method TEXT NOT NULL,-- path TEXT NOT NULL,-- status INTEGER NOT NULL,-- response TEXT NOT NULL,-- created_at TEXT NOT NULL--);--`;----/**-- * Columns added after schema v1, for new and existing databases alike (ALTER TABLE … ADD COLUMN when-- * missing; idempotent, never rewrites a row).-- *-- * v2 privacy: what the supporter lets the public see of an interaction (server/domain/privacy.js),-- * their erasure, and what the creator's public goal and supporters pages show-- * v3 moderation (server/domain/moderation.js): an interaction's moderation state, the creator's-- * word filter, overlay alerts retracted by a moderator-- */--const COLUMNS = [-- ['tip_interactions', 'anonymous', 'INTEGER NOT NULL DEFAULT 0'], // name shown as "Anonymous" to everyone but the supporter-- ['tip_interactions', 'hide_amount', 'INTEGER NOT NULL DEFAULT 0'], // amount left out of overlays, chat lines, public pages-- ['tip_interactions', 'private_message', 'INTEGER NOT NULL DEFAULT 0'], // a tip's message is for the creator only-- ['tip_interactions', 'erased_at', 'TEXT'], // the supporter erased their data from it-- ['creator_tip_profiles', 'page_settings', "TEXT NOT NULL DEFAULT '{}'"], // JSON: what the public goal / supporters pages show-- ['tip_interactions', 'moderation', "TEXT NOT NULL DEFAULT 'visible'"], // visible | held (the filter, awaiting review) | hidden (a moderator)-- ['tip_interactions', 'moderated_at', 'TEXT'],-- ['tip_interactions', 'moderated_by', 'TEXT'], // 'filter', usr_… or svc:…-- ['tip_interactions', 'filtered', 'INTEGER NOT NULL DEFAULT 0'], // the word filter matched it at settlement-- ['creator_tip_profiles', 'filter_words', "TEXT NOT NULL DEFAULT '[]'"], // JSON: the creator's blocklist (filter.js)-- ['creator_tip_profiles', 'filter_action', "TEXT NOT NULL DEFAULT 'mask'"],// mask | hold-- ['creator_tip_profiles', 'filter_links', 'INTEGER NOT NULL DEFAULT 1'], // links in paid messages shown as [link]-- ['overlay_deliveries', 'hidden', 'INTEGER NOT NULL DEFAULT 0'], // retracted: never sent or replayed again--];----const SCHEMA_VERSION = 3;----function addColumns(db) {-- const have = new Map();-- for (const [table, name, def] of COLUMNS) {-- if (!have.has(table)) have.set(table, new Set(db.prepare(`PRAGMA table_info(${table})`).all().map((c) => c.name)));-- if (!have.get(table).has(name)) { db.exec(`ALTER TABLE ${table} ADD COLUMN ${name} ${def}`); have.get(table).add(name); }++function openDb(config, { registry, log = console } = {}) {++ if (!config.db.url && !config.isProduction) {++ log.warn(`[Tips] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);++ fs.mkdirSync(DEV_PGLITE, { recursive: true });++ return createDb({ pglite: DEV_PGLITE, service: 'tips', registry, log });}++ return createDb({ url: config.db.url, service: 'tips', registry, log });}--function openDb(file) {-- if (file !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(file)), { recursive: true });-- const db = new Database(file);-- db.pragma('journal_mode = WAL');-- db.pragma('foreign_keys = ON');-- db.pragma('busy_timeout = 5000');-- db.exec(SCHEMA);-- db.transaction(() => addColumns(db))();-- db.prepare('INSERT OR IGNORE INTO settings (id, schema_version, created_at) VALUES (1, ?, ?)').run(SCHEMA_VERSION, new Date().toISOString());-- db.prepare('UPDATE settings SET schema_version = ? WHERE id = 1 AND schema_version < ?').run(SCHEMA_VERSION, SCHEMA_VERSION);-- return db;++/**++ * Apply pending migrations. Several processes starting together are safe: the SDK serialises runs++ * with an advisory lock held on the owner's one direct session.++ */++async function migrate(config, { serving = null, log = console } = {}) {++ if (serving && serving.store === 'pglite') return serving.migrate({ dir: MIGRATIONS, log });++ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');++ const owner = createDb({ url: config.db.directUrl, service: 'tips-migrate', max: 1, log });++ try { return await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }}--module.exports = { openDb, SCHEMA_VERSION, COLUMNS };++module.exports = { openDb, migrate, MIGRATIONS };@@@ -14,9 +14,21 @@* is not sent, and the creator's word filter has starred blocked words out of the line and dropped* them from the TTS text. `privacy` tells the receiving product what was withheld. An effect whose* interaction a moderator hid (or whose effect was cancelled meanwhile) is never delivered.++ *++ * Writes to an effect and what it delivers (paid message, media request, the interaction's delivery++ * state) run SERIALIZABLE with the interaction row locked first, as the money paths do.++ *++ * Every process runs the worker; each batch of due effects is claimed with a lease (next_attempt_at++ * moved past the batch's worst case, FOR UPDATE SKIP LOCKED), so no two processes deliver one effect++ * at once, and a process that dies mid-batch leaves its effects to the others when the lease ends.*/--const { iso, json } = require('../util');--const { privacyOf, isAnonymous } = require('./privacy');++const { sql } = require('openvibe-sdk/db');++const { iso } = require('../util');++const { privacyOf, isAnonymous, publicView } = require('./privacy');++++// A delivery takes up to 10 s (twice with a token refresh): 20 of them finish well inside the lease.++const BATCH = 20;++const LEASE_MS = 5 * 60_000;function createEffects(ctx) {const { db, config } = ctx;@@@ -24,8 +36,8 @@ function createEffects(ctx) {let kickTimer = null;function jobFor(effect, i, profile) {-- const req = json(i.request, {});-- const pv = ctx.view(i);++ const req = i.request || {};++ const pv = publicView(i, { filter: profile ? profile.filter : null });const name = pv.supporter_name;const amount = pv.amount != null ? `${pv.amount.toLocaleString('en-US')} Vibes` : null;const msg = pv.message ? `: ${pv.message}` : '';@@@ -39,7 +51,7 @@ function createEffects(ctx) {privacy: privacyOf(i),// A tip on the creator's own PowerChat reads as Live's webhook wrote it: "… (PowerChat)".text: `${name} ${amount ? `tipped ${amount}` : 'sent a tip'}${msg}${i.settlement === 'external' && i.provider === 'powerchat' ? ' (PowerChat)' : ''}`,-- target: json(i.target, null),++ target: i.target || null,};if (effect.effect === 'paid_message') job.highlight_seconds = req.highlight_seconds || 0;if (effect.effect === 'tts') job.tts = pv.tts;@@@ -48,78 +60,92 @@ function createEffects(ctx) {}function markDone(effect, result) {-- ctx.tx(() => {++ return ctx.tx(async (t) => {++ await ctx.interactions.lock(t, effect.interaction_id);const at = iso(ctx.now());-- const r = db.prepare("UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ?, updated_at = ? WHERE id = ? AND state = 'queued'")-- .run(JSON.stringify(result || null), at, effect.id);-- if (!r.changes) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side-- const ref = JSON.stringify((result && result.ref) || null);++ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ${sql.json(result || null)},++ updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);++ if (!n) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side++ const ref = sql.json((result && result.ref) || null);if (effect.effect === 'paid_message' || effect.effect === 'tts') {-- db.prepare("UPDATE paid_messages SET status = 'delivered', chat_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);++ await t.exec(sql`UPDATE paid_messages SET status = 'delivered', chat_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);}if (effect.effect === 'media_request') {-- db.prepare("UPDATE paid_media_requests SET status = 'accepted', queue_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);++ await t.exec(sql`UPDATE paid_media_requests SET status = 'accepted', queue_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);}-- ctx.interactions.recomputeDelivery(effect.interaction_id);-- });++ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);++ }, ctx.MONEY);}-- function markFailed(effect, err) {-- ctx.tx(() => {++ async function markFailed(effect, err) {++ await ctx.tx(async (t) => {++ await ctx.interactions.lock(t, effect.interaction_id);const at = iso(ctx.now());const attempts = effect.attempts + 1;const final = err.permanent || attempts >= config.chat.maxAttempts;const backoff = config.chat.backoffMs[Math.min(attempts - 1, config.chat.backoffMs.length - 1)] || 60000;-- db.prepare('UPDATE interaction_effects SET state = ?, attempts = ?, next_attempt_at = ?, last_error = ?, updated_at = ? WHERE id = ? AND state = \'queued\'')-- .run(final ? 'failed' : 'queued', attempts, ctx.now() + backoff, String(err.message || err).slice(0, 500), at, effect.id);++ await t.exec(sql`UPDATE interaction_effects SET state = ${final ? 'failed' : 'queued'}, attempts = ${attempts}, next_attempt_at = ${ctx.now() + backoff},++ last_error = ${String(err.message || err).slice(0, 500)}, updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);if (final) {-- if (effect.effect === 'paid_message' || effect.effect === 'tts') db.prepare("UPDATE paid_messages SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);-- if (effect.effect === 'media_request') db.prepare("UPDATE paid_media_requests SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);-- ctx.interactions.recomputeDelivery(effect.interaction_id);++ if (effect.effect === 'paid_message' || effect.effect === 'tts') await t.exec(sql`UPDATE paid_messages SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);++ if (effect.effect === 'media_request') await t.exec(sql`UPDATE paid_media_requests SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);++ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);}-- });++ }, ctx.MONEY);ctx.outboxKick();}async function runOne(effect) {-- // Re-read: the batch was selected before earlier deliveries awaited; a hide or a reversal may++ // Re-read: the batch was claimed before earlier deliveries awaited; a hide or a reversal may// have cancelled this one since.-- const current = db.prepare('SELECT state FROM interaction_effects WHERE id = ?').get(effect.id);++ const current = await db.maybe(sql`SELECT state FROM interaction_effects WHERE id = ${effect.id}`);if (!current || current.state !== 'queued') return 'cancelled';-- const i = ctx.interactions.get(effect.interaction_id);++ const i = await ctx.interactions.get(db, effect.interaction_id);if (!i || i.payment_state !== 'settled') {// Reversed or failed before its turn: nothing to deliver.-- ctx.tx(() => ctx.interactions.cancelQueued(i, 'payment_not_settled'));++ await ctx.tx(async (t) => { const locked = await ctx.interactions.lock(t, effect.interaction_id); if (locked) await ctx.interactions.cancelQueued(t, locked, 'payment_not_settled'); }, ctx.MONEY);return 'cancelled';}if (i.moderation !== 'visible') {// Hidden or held: moderation.js cancels or releases these; never deliver one meanwhile.-- db.prepare("UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE id = ? AND state = 'queued'").run(iso(ctx.now()), effect.id);++ await db.exec(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${iso(ctx.now())}++ WHERE id = ${effect.id} AND state = 'queued'`);return 'cancelled';}const adapter = ctx.adapters[effect.adapter];-- if (!adapter) { markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }++ if (!adapter) { await markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }++ let result;try {-- const result = await adapter.deliver(jobFor(effect, i, ctx.profiles.bySubject(i.creator_subject)));-- markDone(effect, result);-- return 'delivered';++ result = await adapter.deliver(jobFor(effect, i, await ctx.profiles.bySubject(db, i.creator_subject)));} catch (e) {-- markFailed(effect, e);++ await markFailed(effect, e);return 'retry';}++ await markDone(effect, result);++ return 'delivered';++ }++++ /** Claim a batch of due effects (a lease each), oldest first. */++ async function claim() {++ const now = ctx.now();++ const rows = await db.many(sql`UPDATE interaction_effects SET next_attempt_at = ${now + LEASE_MS}++ WHERE id IN (SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now}++ ORDER BY id LIMIT ${BATCH} FOR UPDATE SKIP LOCKED)++ RETURNING *`);++ return rows.sort((a, b) => a.id - b.id);}-- /** Deliver every due effect once. */++ /** Deliver every due effect once (this process's share of them). */async function drain() {if (running) return { busy: true };running = true;const out = { delivered: 0, retry: 0, failed: 0, cancelled: 0 };try {for (;;) {-- const due = db.prepare("SELECT * FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ? ORDER BY id LIMIT 50").all(ctx.now());++ const due = await claim();if (!due.length) break;for (const e of due) { const r = await runOne(e); out[r] = (out[r] || 0) + 1; }-- if (due.length < 50) break;++ if (due.length < BATCH) break;}} finally { running = false; }return out;@@@ -25,9 +25,9 @@ const INVISIBLE = /[----]/g;const COMBINING_RUN = /(\p{M}{2})\p{M}+/gu;const URL_RE = /\b(?:https?:\/\/|www\.)[^\s<>"']+/gi;--/** Invisible characters and combining-mark runs out, NFKC. */++/** Invisible and NUL characters and combining-mark runs out, unpaired surrogates replaced, NFKC. */function clean(s) {-- return String(s == null ? '' : s).replace(INVISIBLE, '').normalize('NFKC').replace(COMBINING_RUN, '$1');++ return String(s == null ? '' : s).toWellFormed().replace(INVISIBLE, '').replace(/\u0000/g, '').normalize('NFKC').replace(COMBINING_RUN, '$1');}/** The creator's input (a list, or text with one entry per line or comma) → a clean, bounded list. */@@@ -16,75 +16,97 @@* when goal_supporters is on, each supporter's own privacy applying (an anonymous one reads* "Anonymous", a hidden amount is left out).*/++const { sql } = require('openvibe-sdk/db');const { fail, iso, prefixedId, text, positiveInt } = require('../util');const { safeUrl } = require('./profiles');--const { publicName, shownName } = require('./privacy');++const { publicName, shownName, publicView } = require('./privacy');function createGoals(ctx) {const { db } = ctx;-- const get = (id) => db.prepare('SELECT * FROM tip_goals WHERE id = ?').get(String(id || '')) || null;-- /** Settled contributions, plus the opening amount a goal imported from Live carried over. */-- const totalOf = (goalId) => {-- const r = db.prepare('SELECT COALESCE(SUM(amount - reversed_amount), 0) AS n, COUNT(*) AS c FROM tip_goal_contributions WHERE goal_id = ?').get(goalId);-- const g = db.prepare('SELECT opening_amount FROM tip_goals WHERE id = ?').get(goalId);-- return { n: r.n + ((g && g.opening_amount) || 0), c: r.c, contributions: r.n, opening: (g && g.opening_amount) || 0 };-- };---- function list(creator, { status } = {}) {-- const rows = status-- ? db.prepare('SELECT * FROM tip_goals WHERE creator_subject = ? AND status = ? ORDER BY sort_order, created_at').all(creator, status)-- : db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? ORDER BY status = 'active' DESC, sort_order, created_at DESC").all(creator);-- return rows;++ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_goals WHERE id = ${String(id || '')}`);++++ /** Settled contributions per goal, in one query: Map(goal id → { n: sum, c: count }). */++ async function totalsOf(q, goalIds) {++ const out = new Map(goalIds.map((id) => [id, { n: 0, c: 0 }]));++ if (!goalIds.length) return out;++ const rows = await q.many(sql`SELECT goal_id, COALESCE(SUM(amount - reversed_amount), 0)::bigint AS n, count(*) AS c++ FROM tip_goal_contributions WHERE goal_id = ANY(${goalIds}) GROUP BY goal_id`);++ for (const r of rows) out.set(r.goal_id, { n: r.n, c: r.c });++ return out;++ }++++ function list(q, creator, { status } = {}) {++ return status++ ? q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = ${status} ORDER BY sort_order, created_at`)++ : q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} ORDER BY status = 'active' DESC, sort_order, created_at DESC`);++ }++++ /** The full view of each goal (the creator's, granted services', overlays'): one query for all totals. */++ async function presentMany(q, goals) {++ const totals = await totalsOf(q, goals.map((g) => g.id));++ return goals.map((g) => {++ const t = totals.get(g.id);++ const n = t.n + (g.opening_amount || 0);++ return {++ id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,++ target_amount: g.target_amount, current_amount: n, currency: g.currency, supporters_count: t.c,++ carried_over_amount: g.opening_amount || 0,++ percent: Math.min(100, Math.floor((n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,++ image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,++ created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,++ };++ });}-- function present(g, { contributions = false } = {}) {++ async function present(q, g, { contributions = false } = {}) {if (!g) return null;-- const t = totalOf(g.id);-- const out = {-- id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,-- target_amount: g.target_amount, current_amount: t.n, currency: g.currency, supporters_count: t.c,-- carried_over_amount: t.opening,-- percent: Math.min(100, Math.floor((t.n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,-- image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,-- created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,-- };++ const [out] = await presentMany(q, [g]);if (contributions) {// The creator's view: amounts always (their money), names as the supporter allowed.-- out.contributions = db.prepare(`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at-- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? ORDER BY c.id DESC LIMIT 200`).all(g.id)++ out.contributions = (await q.many(sql`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at++ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ${g.id} ORDER BY c.id DESC LIMIT 200`)).map(({ anonymous, erased_at: erasedAt, ...c }) => ({ ...c, supporter_name: publicName({ ...c, anonymous, erased_at: erasedAt }) }));}return out;}-- /** The public shape of a goal under the creator's page settings (`page`, profiles.js). */-- function publicGoal(g, page) {-- const out = present(g);-- if (!out) return null;-- if (!page.goal_amounts) Object.assign(out, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });-- if (page.goal_supporters) {++ /**++ * The public shape of goals under the creator's page settings (`profile.page`, profiles.js), with++ * the creator's word filter (`profile.filter`) on supporter names: totals in one query, and the++ * latest supporters of every goal in one more when the page shows them.++ */++ async function publicGoals(q, goals, profile) {++ const page = profile.page;++ const views = await presentMany(q, goals);++ if (!page.goal_amounts) for (const v of views) Object.assign(v, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });++ if (page.goal_supporters && goals.length) {// Moderated (held or hidden) interactions are left out; names go through the creator's filter.-- const settings = ctx.profiles.filterOf(g.creator_subject);-- out.supporters = db.prepare(`SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at-- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? AND c.amount > c.reversed_amount-- AND i.moderation = 'visible' ORDER BY c.id DESC LIMIT 10`).all(g.id)-- .map((c) => ({ name: shownName(c, settings), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));++ const rows = await q.many(sql`SELECT gid AS goal_id, s.amount, s.created_at, s.supporter_name, s.anonymous, s.hide_amount, s.erased_at++ FROM unnest(${goals.map((g) => g.id)}::text[]) AS gid++ CROSS JOIN LATERAL (++ SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at++ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id++ WHERE c.goal_id = gid AND c.amount > c.reversed_amount AND i.moderation = 'visible'++ ORDER BY c.id DESC LIMIT 10) s`);++ for (const v of views) {++ v.supporters = rows.filter((r) => r.goal_id === v.id)++ .map((c) => ({ name: shownName(c, profile.filter), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));++ }}-- return out;++ return views;}++ const publicGoal = async (q, g, profile) => (g ? (await publicGoals(q, [g], profile))[0] : null);/** Inside a transaction: bump revision, emit the event, queue the overlay delivery. */-- function changed(goalId, reason, { interactionId = null, by = null } = {}) {-- const at = iso(ctx.now());-- db.prepare('UPDATE tip_goals SET revision = revision + 1, updated_at = ? WHERE id = ?').run(at, goalId);-- const g = get(goalId);-- const view = present(g);-- ctx.outbox.emit('tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {++ async function changed(t, goalId, reason, { interactionId = null, by = null } = {}) {++ const g = await t.one(sql`UPDATE tip_goals SET revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${goalId} RETURNING *`);++ const [view] = await presentMany(t, [g]);++ await ctx.outbox.emit(t, 'tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {goal_id: g.id, creator: view.creator, title: g.title, target_amount: g.target_amount, current_amount: view.current_amount,currency: g.currency, status: g.status, reached: view.reached, reason, interaction_id: interactionId,});-- ctx.overlays.addGoalDelivery(g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });++ await ctx.overlays.addGoalDelivery(t, g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });return view;}@@@ -109,88 +131,103 @@ function createGoals(ctx) {return out;}-- function create(creator, input = {}) {++ async function create(creator, input = {}) {const f = fields(input, false);-- const count = db.prepare("SELECT COUNT(*) AS n FROM tip_goals WHERE creator_subject = ? AND status = 'active'").get(creator).n;-- if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');-- return ctx.tx(() => {++ return ctx.tx(async (t) => {++ await ctx.lockCreator(t, creator);++ const count = await t.value(sql`SELECT count(*) FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active'`);++ if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');const at = iso(ctx.now());const id = prefixedId('tgoal', ctx.now());-- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)-- VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?, ?)`).run(id, creator, f.title, f.description || null, f.target_amount, f.image_url || null, f.sort_order || 0, at, at);-- return changed(id, 'created');++ await t.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)++ VALUES (${id}, ${creator}, ${f.title}, ${f.description || null}, ${f.target_amount}, ${f.image_url || null}, 'active', ${f.sort_order || 0}, ${at}, ${at})`);++ return changed(t, id, 'created');});}-- function update(goal, input = {}) {++ /** An edit by the creator. The goal is locked, and its revision checked against the locked row. */++ async function update(goal, input = {}) {if (goal.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');if (input.revision != null && Number(input.revision) !== goal.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${goal.revision}`);const f = fields(input, true);-- const keys = Object.keys(f);-- if (!keys.length) return present(goal);-- return ctx.tx(() => {-- db.prepare(`UPDATE tip_goals SET ${keys.map((k) => `${k} = @${k}`).join(', ')} WHERE id = @id`).run({ ...f, id: goal.id });-- markReached(goal.id);-- return changed(goal.id, 'updated');++ if (!Object.keys(f).length) return present(db, goal);++ return ctx.tx(async (t) => {++ const cur = await t.one(sql`SELECT * FROM tip_goals WHERE id = ${goal.id} FOR UPDATE`);++ if (cur.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');++ if (input.revision != null && Number(input.revision) !== cur.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${cur.revision}`);++ await t.exec(sql`UPDATE tip_goals SET ${sql.set(f)} WHERE id = ${goal.id}`);++ await markReached(t, goal.id);++ return changed(t, goal.id, 'updated');});}-- function close(goal) {-- if (goal.status === 'closed') return present(goal);-- return ctx.tx(() => {-- db.prepare("UPDATE tip_goals SET status = 'closed', closed_at = ? WHERE id = ?").run(iso(ctx.now()), goal.id);-- return changed(goal.id, 'closed');++ async function close(goal) {++ if (goal.status === 'closed') return present(db, goal);++ return ctx.tx(async (t) => {++ const n = await t.exec(sql`UPDATE tip_goals SET status = 'closed', closed_at = ${iso(ctx.now())} WHERE id = ${goal.id} AND status = 'active'`);++ if (!n) return present(t, await get(t, goal.id));++ return changed(t, goal.id, 'closed');});}-- function markReached(goalId) {-- const g = get(goalId);-- const reached = totalOf(goalId).n >= g.target_amount;-- if (reached && !g.reached_at) { db.prepare('UPDATE tip_goals SET reached_at = ? WHERE id = ?').run(iso(ctx.now()), goalId); return true; }-- if (!reached && g.reached_at) db.prepare('UPDATE tip_goals SET reached_at = NULL WHERE id = ?').run(goalId);++ /** Inside a transaction: set or clear reached_at from the goal's total. True when reached just now. */++ async function markReached(t, goalId) {++ const g = await t.one(sql`SELECT g.target_amount, g.reached_at,++ g.opening_amount + COALESCE((SELECT SUM(c.amount - c.reversed_amount) FROM tip_goal_contributions c WHERE c.goal_id = g.id), 0)::bigint AS total++ FROM tip_goals g WHERE g.id = ${goalId}`);++ const reached = g.total >= g.target_amount;++ if (reached && !g.reached_at) { await t.exec(sql`UPDATE tip_goals SET reached_at = ${iso(ctx.now())} WHERE id = ${goalId}`); return true; }++ if (!reached && g.reached_at) await t.exec(sql`UPDATE tip_goals SET reached_at = NULL WHERE id = ${goalId}`);return false;}/** The goal an interaction counts toward, or null. */-- function pick(creator, requestedGoalId) {++ async function pick(q, creator, requestedGoalId) {if (requestedGoalId) {-- const g = get(requestedGoalId);++ const g = await get(q, requestedGoalId);if (g && g.creator_subject === creator && g.status === 'active') return g;}-- const active = db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? AND status = 'active' LIMIT 2").all(creator);++ const active = await q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active' LIMIT 2`);return active.length === 1 ? active[0] : null;}-- /** Inside the settlement transaction. Simulations never get here. */-- function contribute(interaction, requestedGoalId) {-- const g = pick(interaction.creator_subject, requestedGoalId);-- if (!g) return null;++ /**++ * Inside the settlement transaction (the interaction row is locked already; the goal is locked++ * next, keeping the lock order). Simulations never get here. `filter`: the creator's word filter.++ */++ async function contribute(t, interaction, requestedGoalId, filter) {++ const picked = await pick(t, interaction.creator_subject, requestedGoalId);++ if (!picked) return null;++ const g = await t.one(sql`SELECT id, status FROM tip_goals WHERE id = ${picked.id} FOR UPDATE`);++ if (g.status !== 'active') return null;const at = iso(ctx.now());-- const r = db.prepare(`INSERT OR IGNORE INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at) VALUES (?, ?, ?, ?, ?)`)-- .run(g.id, interaction.id, interaction.amount, at, at);-- if (!r.changes) return null;-- const reachedNow = markReached(g.id);++ const n = await t.exec(sql`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)++ VALUES (${g.id}, ${interaction.id}, ${interaction.amount}, ${at}, ${at}) ON CONFLICT DO NOTHING`);++ if (!n) return null;++ const reachedNow = await markReached(t, g.id);// The overlay's goal update names the supporter as the alert does; not when the amount is hidden// (the bar's jump would otherwise pin the amount on them), nor while moderation holds it.-- const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : ctx.view(interaction).supporter_name;-- return changed(g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });++ const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : publicView(interaction, { filter }).supporter_name;++ return changed(t, g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });}/** Inside the reversal transaction: take `bits` back off every goal the interaction counted toward. */-- function reverse(interaction, bits) {-- const rows = db.prepare('SELECT * FROM tip_goal_contributions WHERE interaction_id = ?').all(interaction.id);++ async function reverse(t, interaction, bits) {++ const rows = await t.many(sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = ${interaction.id} ORDER BY goal_id`);++ if (!rows.length) return [];++ await t.many(sql`SELECT id FROM tip_goals WHERE id = ANY(${rows.map((c) => c.goal_id)}) ORDER BY id FOR UPDATE`);const out = [];for (const c of rows) {const take = Math.min(bits, c.amount - c.reversed_amount);if (take <= 0) continue;-- db.prepare('UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ?, updated_at = ? WHERE id = ?').run(take, iso(ctx.now()), c.id);-- markReached(c.goal_id);-- out.push(changed(c.goal_id, 'reversal', { interactionId: interaction.id }));++ await t.exec(sql`UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ${take}, updated_at = ${iso(ctx.now())} WHERE id = ${c.id}`);++ await markReached(t, c.goal_id);++ out.push(await changed(t, c.goal_id, 'reversal', { interactionId: interaction.id }));}return out;}-- return { get, list, present, publicGoal, create, update, close, contribute, reverse, pick, totalOf };++ return { get, list, present, presentMany, publicGoal, publicGoals, create, update, close, contribute, reverse, pick, totalsOf };}module.exports = { createGoals };@@@ -3,12 +3,30 @@/*** The Tips domain: one context shared by profiles, goals, interactions, overlays and effects.*-- * ctx.tx(fn) run fn in one SQLite transaction; ctx.afterCommit(hook) queues work (overlay-- * pushes, worker kicks) that must only happen once the transaction committed-- * ctx.outbox.emit() durable event, inside the transaction (openvibe-sdk outbox)-- * ctx.view(i) the interaction's public view (privacy.js) through its creator's word filter++ * Every function that touches the database takes the query handle first: `db` outside a transaction,++ * or the transaction's `t` inside one (never `db` inside a transaction: that would be another++ * connection, outside it). Operations that open their own transaction take no handle.++ *++ * ctx.tx(fn, { isolation }) run fn(t) in one transaction. t.after(hook) queues work (overlay pushes,++ * worker and relay kicks) for after the commit: a retried attempt starts++ * with no hooks, a rolled-back one runs none. Anything that changes money++ * state (payment state, reversed bits, goal contributions, what a payment++ * entitles: paid messages, media requests) runs { isolation: 'serializable' }++ * (ADR-007 amendment 2026-09-24, rule 3); the SDK retries serialization++ * failures and deadlocks.++ * ctx.within(q, fn, opts) fn(q) when q is a transaction handle, else fn(t) in a new transaction++ * ctx.afterCommit(q, hook) t.after(hook) inside a transaction; at once on `db` (already committed)++ * ctx.lockCreator(t, s) lock a creator's profile row: settings changes and per-creator limits++ * (20 active goals, 25 tokens, 50 moderators, 10 invitations) take it, so two++ * requests at once count and write one after the other++ * ctx.outbox.emit(t, …) a durable event, inside the transaction (openvibe-sdk outbox)++ *++ * Lock order (so two transactions never wait on each other in a cycle): a creator's profile row, then++ * tip_interactions rows (SELECT … FOR UPDATE, several by id), then tip_goals rows by id, then the++ * creator's overlay stream (an advisory lock taken before a delivery is inserted: overlays.js), then++ * everything else (effects, paid messages and media requests, overlay deliveries, the outbox).*/--const { publicView } = require('./privacy');++const { sql } = require('openvibe-sdk/db');const { createProfiles } = require('./profiles');const { createGoals } = require('./goals');const { createOverlays } = require('./overlays');@@@ -16,23 +34,30 @@ const { createInteractions } = require('./interactions');const { createEffects } = require('./effects');const { createModeration } = require('./moderation');--function createDomain({ db, config, outbox, billing, adapters, now = () => Date.now(), log = console }) {-- const ctx = { db, config, now, log, outbox, billing, adapters, _after: null };++const MONEY = { isolation: 'serializable' };++++function createDomain({ db, config, outbox, billing, adapters, valkey = null, now = () => Date.now(), log = console }) {++ const ctx = { db, config, now, log, outbox, billing, adapters, valkey, MONEY };-- ctx.tx = (fn) => {-- if (ctx._after) return db.transaction(fn)();-- const hooks = [];-- ctx._after = hooks;-- let result;-- try { result = db.transaction(fn)(); } finally { ctx._after = null; }-- for (const h of hooks) { try { h(); } catch (e) { log.warn('[Tips] after-commit:', e.message); } }-- return result;++ const runHook = (h) => { Promise.resolve().then(h).catch((e) => log.warn('[Tips] after-commit:', e.message)); };++ ctx.tx = async (fn, opts = {}) => {++ let hooks = [];++ const out = await db.tx(async (t) => {++ const mine = []; // this attempt's hooks only: a retried or rolled-back attempt's never run++ hooks = mine;++ t.after = (hook) => { mine.push(hook); };++ return fn(t);++ }, opts);++ hooks.forEach(runHook);++ return out;};-- ctx.afterCommit = (hook) => { if (ctx._after) ctx._after.push(hook); else hook(); };++ const inTx = (q) => !!(q && typeof q.after === 'function');++ ctx.within = (q, fn, opts) => (inTx(q) ? fn(q) : ctx.tx(fn, opts));++ ctx.afterCommit = (q, hook) => { if (inTx(q)) q.after(hook); else runHook(hook); };++ ctx.lockCreator = (t, subject) => t.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE creator_subject = ${subject} FOR UPDATE`);ctx.outboxKick = () => { try { outbox.kick(); } catch { /* relay off */ } };ctx.profiles = createProfiles(ctx);-- ctx.view = (i, opts = {}) => publicView(i, { ...opts, filter: ctx.profiles.filterOf(i.creator_subject) });ctx.goals = createGoals(ctx);ctx.overlays = createOverlays(ctx);ctx.interactions = createInteractions(ctx);@@@ -41,4 +66,4 @@ function createDomain({ db, config, outbox, billing, adapters, now = () => Date.return ctx;}--module.exports = { createDomain };++module.exports = { createDomain, MONEY };@@@ -32,15 +32,20 @@* contribution, the overlay alert, the delivery effects and tips.interaction.ready. Delivery then* runs in the effects worker; its failure never touches payment_state.*++ * Every transaction that changes money state (a request and its pending-checkout limit, settlement,++ * failure, funding, reversal) is SERIALIZABLE (ctx.MONEY) and locks the interaction row first++ * (SELECT … FOR UPDATE), then the goals it counts toward: ADR-007 amendment 2026-09-24, rule 3.++ ** Privacy (privacy.js): the supporter's choices are stored with the interaction (anonymous,* hide_amount, private_message); everything that leaves Tips for the public is publicView(), and* events and answers to anyone but the supporter never name an anonymous supporter. A supporter can* export their tips and erase their data from them (erase(): the money record stays, the person* goes; tips.interaction.erased redacts the earlier events in OpenVibe.Events).*/--const { fail, iso, prefixedId, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { fail, iso, prefixedId, positiveInt, text, storable, userSubject, entityRef, displayName, readCursor, cursorOf, INVISIBLE } = require('../util');const { VOICES } = require('./profiles');--const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous } = require('./privacy');++const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous, publicView, ANONYMOUS } = require('./privacy');const { BillingCallError } = require('../billing-client');const KINDS = ['tip', 'paid_message', 'tts', 'media_request'];@@@ -53,18 +58,28 @@ const MAX_EXTERNAL_CENTS = 100_000_000; // Billing's per-receipt maximumconst HIGHLIGHT = [[5000, 300], [1000, 120], [500, 60], [100, 30]];const highlightFor = (bits) => (HIGHLIGHT.find(([min]) => bits >= min) || [0, 0])[1];const TRANSFER_BACKOFF = [2000, 10000, 30000, 60000, 300000, 600000];++// Due transfers are claimed in small batches with a lease: a Billing call takes up to 10 s (twice++// with a token refresh), so a batch finishes well inside the lease, and a process that dies with a++// claimed batch leaves it to the others once the lease ends.++const TRANSFER_BATCH = 10;++const TRANSFER_LEASE_MS = 5 * 60_000;++const COLUMNS = ['id', 'creator_subject', 'supporter_subject', 'supporter_name', 'kind', 'amount', 'amount_cents', 'message', 'request',++ 'funding', 'settlement', 'payment_state', 'delivery_state', 'test', 'billing_txn_id', 'billing_intent_id', 'funding_txn_id', 'provider', 'provider_ref',++ 'legacy_source', 'idempotency_key', 'origin', 'target', 'anonymous', 'hide_amount', 'private_message', 'created_at', 'updated_at'];function createInteractions(ctx) {-- const { db, config } = ctx;++ const { db, config, MONEY } = ctx;-- const get = (id) => db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(String(id || '')) || null;-- const byBillingTxn = (txnId) => db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').get(txnId) || null;++ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')}`);++ /** Inside a transaction: the interaction row, locked (the first lock any transaction takes). */++ const lock = (t, id) => t.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')} FOR UPDATE`);++ const byBillingTxn = (q, txnId) => q.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${txnId}`);// ── Input rules (the creator's settings) ─────────────────// Plain words only: no control or invisible characters, no links, no markup a speech engine could// take as SSML.function cleanTts(s) {-- return String(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ')++ return storable(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ').replace(/\s+/g, ' ').trim();}@@@ -89,7 +104,7 @@ function createInteractions(ctx) {* Validate a request against the creator's profile. Returns the normalised fields:* { kind, amount, message, request: { goal_id, tts, media, highlight_seconds } }.*/-- function validate(profile, input, { simulation = false } = {}) {++ async function validate(q, profile, input, { simulation = false } = {}) {if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip page');if (!profile.accepting && !simulation) fail(409, 'tips.not_accepting', `${profile.display_name} is not accepting tips right now`);const kind = input.kind || 'tip';@@@ -121,7 +136,7 @@ function createInteractions(ctx) {request.media = { ...mediaUrl(input.media && input.media.url), max_seconds: profile.media_max_seconds };}if (input.goal_id) {-- const g = ctx.goals.get(input.goal_id);++ const g = await ctx.goals.get(q, input.goal_id);if (!g || g.creator_subject !== profile.creator_subject || g.status !== 'active') fail(422, 'tips.goal_not_found', 'that goal is not open on this page');request.goal_id = g.id;}@@@ -129,29 +144,27 @@ function createInteractions(ctx) {}// ── Creation ─────────────────────────────────────────────-- function insert(row) {++ async function insert(q, row) {const at = iso(ctx.now());const r = {-- supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: 0, billing_txn_id: null,++ supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: false, billing_txn_id: null,billing_intent_id: null, funding_txn_id: null, provider: null, provider_ref: null, legacy_source: null, idempotency_key: null,-- target: null, origin: 'tips', anonymous: 0, hide_amount: 0, private_message: 0, created_at: at, ...row,++ target: null, origin: 'tips', anonymous: false, hide_amount: false, private_message: false, created_at: at, ...row,++ };++ const values = {++ ...r, request: sql.json(r.request || {}), target: r.target ? sql.json(r.target) : null, test: !!r.test,++ anonymous: !!r.anonymous, hide_amount: !!r.hide_amount, private_message: !!r.private_message, updated_at: r.created_at,};-- db.prepare(`INSERT INTO tip_interactions (id, creator_subject, supporter_subject, supporter_name, kind, amount, amount_cents, message, request,-- funding, settlement, payment_state, delivery_state, test, billing_txn_id, billing_intent_id, funding_txn_id, provider, provider_ref,-- legacy_source, idempotency_key, origin, target, anonymous, hide_amount, private_message, created_at, updated_at)-- VALUES (@id, @creator_subject, @supporter_subject, @supporter_name, @kind, @amount, @amount_cents, @message, @request,-- @funding, @settlement, @payment_state, @delivery_state, @test, @billing_txn_id, @billing_intent_id, @funding_txn_id, @provider, @provider_ref,-- @legacy_source, @idempotency_key, @origin, @target, @anonymous, @hide_amount, @private_message, @created_at, @created_at)`)-- .run({ ...r, request: JSON.stringify(r.request || {}), target: r.target ? JSON.stringify(r.target) : null, anonymous: r.anonymous ? 1 : 0, hide_amount: r.hide_amount ? 1 : 0, private_message: r.private_message ? 1 : 0 });-- return get(r.id);++ return q.one(sql`INSERT INTO tip_interactions ${sql.insert([values], COLUMNS)} RETURNING *`);}/*** A supporter's request (checkout or credit). Idempotent by `idempotencyKey` (the caller's key,-- * scoped to the caller): a repeat returns the interaction it created the first time.++ * scoped to the caller): a repeat returns the interaction it created the first time. Serializable:++ * the unpaid-checkout count and the new row are one decision.*/-- function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {-- const v = validate(profile, input);++ async function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {++ const v = await validate(db, profile, input);const from = userSubject(supporter, 'supporter');if (from === profile.creator_subject) fail(422, 'tips.self_dealing', 'you cannot tip yourself');if (!['credit', 'checkout'].includes(funding)) fail(422, 'tips.invalid_input', "pay_with must be 'credit' or 'checkout'");@@@ -161,25 +174,26 @@ function createInteractions(ctx) {const target = entityRef(input.target);const privacy = parsePrivacy(input.privacy, v.kind);if (!privacy.anonymous) checkName(profile, displayName(supporterName));-- return ctx.tx(() => {++ return ctx.tx(async (t) => {if (idempotencyKey) {-- const prev = db.prepare('SELECT * FROM tip_interactions WHERE idempotency_key = ?').get(idempotencyKey);++ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE idempotency_key = ${idempotencyKey}`);if (prev) return { interaction: prev, replay: true };}if (funding === 'checkout') {// Each unpaid checkout is a Billing intent and a row here: bounded per supporter.const since = iso(ctx.now() - 24 * 3600 * 1000);-- const open = db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ? AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= ?").get(from, since).n;++ const open = await t.value(sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = ${from} AND funding = 'checkout'++ AND payment_state = 'pending' AND created_at >= ${since}`);if (open >= config.limits.pendingCheckouts) fail(429, 'tips.too_many_pending', `you have ${open} unpaid checkouts; finish or wait for them before starting another`);}-- const i = insert({++ const i = await insert(t, {id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: from, supporter_name: displayName(supporterName) || 'Someone',kind: v.kind, amount: v.amount, message: v.message, request: v.request, funding, settlement: 'billing', payment_state: 'pending',delivery_state: 'awaiting_payment', idempotency_key: idempotencyKey || null, target,...privacy, private_message: privacy.private_message && !!v.message,});return { interaction: i, replay: false };-- });++ }, MONEY);}// ── Paying through Billing ───────────────────────────────@@@ -194,17 +208,18 @@ function createInteractions(ctx) {key: `tips:transfer:${i.id}`, traceparent,});const txn = out.transaction;-- ctx.tx(() => settleByTransaction(get(i.id), { id: txn.id, test: !!txn.test }));++ await ctx.tx(async (t) => settleByTransaction(t, await lock(t, i.id), { id: txn.id, test: !!txn.test }), MONEY);return { settled: true };} catch (e) {if (e instanceof BillingCallError && !e.retryable) {-- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));++ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);return { settled: false, refused: e.code, detail: e.body && e.body.detail };}-- const cur = get(i.id);-- const n = cur.transfer_attempts + 1;-- db.prepare('UPDATE tip_interactions SET transfer_due = 1, transfer_attempts = ?, next_transfer_at = ?, failure = ?, updated_at = ? WHERE id = ? AND payment_state = \'pending\'')-- .run(n, ctx.now() + TRANSFER_BACKOFF[Math.min(n - 1, TRANSFER_BACKOFF.length - 1)], String(e.message).slice(0, 300), iso(ctx.now()), i.id);++ // Due again after the attempt's backoff (TRANSFER_BACKOFF[attempts - 1], the last one repeating).++ await db.exec(sql`UPDATE tip_interactions SET transfer_due = true, transfer_attempts = transfer_attempts + 1,++ next_transfer_at = ${ctx.now()} + (${TRANSFER_BACKOFF}::bigint[])[LEAST(transfer_attempts + 1, ${TRANSFER_BACKOFF.length})],++ failure = ${String(e.message).slice(0, 300)}, updated_at = ${iso(ctx.now())}++ WHERE id = ${i.id} AND payment_state = 'pending'`);ctx.log.warn(`[Tips] transfer for ${i.id} will be retried: ${e.message}`);return { settled: false, retrying: true };}@@@ -222,7 +237,7 @@ function createInteractions(ctx) {});} catch (e) {if (e instanceof BillingCallError && !e.retryable) {-- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));++ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);fail(e.status === 409 ? 409 : 422, e.code || 'billing.refused', (e.body && e.body.detail) || 'Billing refused the checkout');}fail(502, 'tips.billing_unavailable', 'checkout is unavailable right now; nothing was charged — try again shortly');@@@ -235,63 +250,72 @@ function createInteractions(ctx) {}// Pages link and redirect to it: only an https URL is ever kept.if (url && !/^https:\/\/[^\s"'<>]+$/.test(url)) url = null;-- db.prepare('UPDATE tip_interactions SET billing_intent_id = ?, checkout_url = ?, checkout_ref = ?, provider = ?, updated_at = ? WHERE id = ?')-- .run(intent.id || null, url, ref, p, iso(ctx.now()), i.id);-- return { interaction: get(i.id), checkout_url: url, checkout_ref: ref, intent };++ const row = await db.one(sql`UPDATE tip_interactions SET billing_intent_id = ${intent.id || null}, checkout_url = ${url}, checkout_ref = ${ref}, provider = ${p},++ updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);++ return { interaction: row, checkout_url: url, checkout_ref: ref, intent };}-- /** Due transfers (credit retries and funded checkouts). Called by the jobs loop. */++ /**++ * Due transfers (credit retries and funded checkouts). Called by the jobs loop in every process:++ * each batch is claimed with a lease (FOR UPDATE SKIP LOCKED), so no two processes try one at once.++ */async function processDueTransfers() {-- const rows = db.prepare("SELECT * FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due = 1 AND next_transfer_at <= ? ORDER BY next_transfer_at LIMIT 20").all(ctx.now());++ const now = ctx.now();++ const rows = await db.many(sql`UPDATE tip_interactions SET next_transfer_at = ${now + TRANSFER_LEASE_MS}++ WHERE id IN (SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= ${now}++ ORDER BY next_transfer_at LIMIT ${TRANSFER_BATCH} FOR UPDATE SKIP LOCKED)++ RETURNING *`);++ rows.sort((a, b) => (a.created_at < b.created_at ? -1 : 1));let settled = 0;for (const i of rows) { const r = await transfer(i); if (r.settled) settled++; }return { attempted: rows.length, settled };}// ── Settlement ───────────────────────────────────────────-- /** Inside a transaction: settle `i` by Billing transaction `txn` ({ id, test }). Idempotent. */-- function settleByTransaction(i, txn) {++ /** Inside a MONEY transaction, `i` locked: settle it by Billing transaction `txn` ({ id, test }). Idempotent. */++ async function settleByTransaction(t, i, txn) {if (!i) return null;if (i.billing_txn_id === txn.id || i.payment_state === 'settled' || i.payment_state === 'reversed') return i;-- const other = byBillingTxn(txn.id);++ const other = await byBillingTxn(t, txn.id);if (other && other.id !== i.id) return other;-- db.prepare('UPDATE tip_interactions SET billing_txn_id = ?, test = MAX(test, ?), transfer_due = 0, failure = NULL WHERE id = ?').run(txn.id, txn.test ? 1 : 0, i.id);-- return settle(get(i.id));++ const row = await t.one(sql`UPDATE tip_interactions SET billing_txn_id = ${txn.id}, test = test OR ${!!txn.test}, transfer_due = false, failure = NULL++ WHERE id = ${i.id} RETURNING *`);++ return settle(t, row);}-- /** Inside a transaction: pending/failed → settled, and the whole effect path. */-- function settle(i) {++ /** Inside a MONEY transaction, `i` locked (or created by it): pending/failed → settled, and the whole effect path. */++ async function settle(t, i) {const at = iso(ctx.now());const test = !!i.test;-- const req = json(i.request, {});++ const req = i.request || {};// A tip hidden by a moderator while its payment was pending will deliver nothing.-- db.prepare(`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,-- settled_at = ?, updated_at = ? WHERE id = ?`).run(at, at, i.id);-- i = get(i.id);++ i = await t.one(sql`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,++ settled_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);if (i.kind === 'paid_message' || i.kind === 'tts') {-- db.prepare(`INSERT OR IGNORE INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)-- VALUES (?, ?, ?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tpm', ctx.now()), i.id, i.creator_subject, i.kind,-- i.kind === 'tts' ? req.tts.text : (i.message || ''), i.kind === 'tts' ? req.tts.voice : null, req.highlight_seconds || 0, test ? 1 : 0, at, at);++ await t.exec(sql`INSERT INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)++ VALUES (${prefixedId('tpm', ctx.now())}, ${i.id}, ${i.creator_subject}, ${i.kind}, ${i.kind === 'tts' ? req.tts.text : (i.message || '')},++ ${i.kind === 'tts' ? req.tts.voice : null}, ${req.highlight_seconds || 0}, 'queued', ${test}, ${at}, ${at}) ON CONFLICT DO NOTHING`);}if (i.kind === 'media_request' && req.media) {-- db.prepare(`INSERT OR IGNORE INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)-- VALUES (?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tmr', ctx.now()), i.id, i.creator_subject, req.media.url, req.media.provider, test ? 1 : 0, at, at);++ await t.exec(sql`INSERT INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)++ VALUES (${prefixedId('tmr', ctx.now())}, ${i.id}, ${i.creator_subject}, ${req.media.url}, ${req.media.provider}, 'queued', ${test}, ${at}, ${at})++ ON CONFLICT DO NOTHING`);}// The creator's word filter runs before anything is shown, posted or read (moderation.js): a// match is masked, or held for review when the creator chose that. History is not screened.-- const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i) : { hit: false };-- if (verdict.hit) ctx.moderation.markScreened(i, verdict);-- i = get(i.id);-- if (!test && i.origin !== 'import') ctx.goals.contribute(i, req.goal_id);-- if (i.origin !== 'import' && i.moderation === 'visible') release(i);-- recomputeDelivery(i.id);++ const filter = await ctx.profiles.filterOf(t, i.creator_subject);++ const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i, filter) : { hit: false };++ if (verdict.hit) i = await ctx.moderation.markScreened(t, i, verdict);++ if (!test && i.origin !== 'import') await ctx.goals.contribute(t, i, req.goal_id, filter);++ if (i.origin !== 'import' && i.moderation === 'visible') await release(t, i, filter);++ i = await recomputeDelivery(t, i.id);if (!test && i.origin !== 'import') {-- ctx.outbox.emit('tips.interaction.ready', { type: 'interaction', id: i.id }, summary(get(i.id)));++ await ctx.outbox.emit(t, 'tips.interaction.ready', { type: 'interaction', id: i.id }, summary(i));}-- if (verdict.hit) ctx.moderation.recordScreened(get(i.id), verdict);-- ctx.afterCommit(() => { ctx.effects.kick(); ctx.outboxKick(); });-- return get(i.id);++ if (verdict.hit) await ctx.moderation.recordScreened(t, i, verdict);++ ctx.afterCommit(t, () => { ctx.effects.kick(); ctx.outboxKick(); });++ return i;}/**@@@ -299,97 +323,90 @@ function createInteractions(ctx) {* deliveries. Settlement calls it, and moderation's restore() for one the filter held. Idempotent:* nothing that exists (delivered or cancelled) is created again.*/-- function release(i) {++ async function release(t, i, filter) {const test = !!i.test;-- ctx.overlays.addAlert(i);-- addEffect(i, 'overlay_alert', 'overlay', 'delivered');++ await ctx.overlays.addAlert(t, i, filter);++ const at = iso(ctx.now());++ const effects = [{ effect: 'overlay_alert', adapter: 'overlay', state: 'delivered' }];// Chat effects: never for imports (history), never twice for a donation another product already// announced (origin billing), never to a real chat room for a simulation.const adapter = test ? 'test' : config.chat.adapter;if (ANNOUNCED_HERE.includes(i.origin) && adapter !== 'none') {-- if (i.kind === 'tip') addEffect(i, 'chat_line', adapter);-- if (i.kind === 'paid_message') addEffect(i, 'paid_message', adapter);-- if (i.kind === 'tts') { addEffect(i, 'chat_line', adapter); addEffect(i, 'tts', adapter); }-- if (i.kind === 'media_request') addEffect(i, 'media_request', adapter);++ const chat = { tip: ['chat_line'], paid_message: ['paid_message'], tts: ['chat_line', 'tts'], media_request: ['media_request'] }[i.kind] || [];++ for (const effect of chat) effects.push({ effect, adapter, state: 'queued' });}-- recomputeDelivery(i.id);-- ctx.afterCommit(() => ctx.effects.kick());++ await t.exec(sql`INSERT INTO interaction_effects ${sql.insert(effects.map((e) => ({ interaction_id: i.id, ...e, created_at: at, updated_at: at })))}++ ON CONFLICT DO NOTHING`);++ await recomputeDelivery(t, i.id);++ ctx.afterCommit(t, () => ctx.effects.kick());}-- function addEffect(i, effect, adapter, state = 'queued') {-- const at = iso(ctx.now());-- db.prepare(`INSERT OR IGNORE INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?)`)-- .run(i.id, effect, adapter, state, at, at);-- }---- /** Inside a transaction: pending → failed (nothing was delivered; nothing moved in Billing). */-- function failPayment(i, code, detail) {++ /** Inside a MONEY transaction, `i` locked: pending → failed (nothing was delivered; nothing moved in Billing). */++ async function failPayment(t, i, code, detail) {if (!i || i.payment_state !== 'pending') return i;-- const at = iso(ctx.now());-- db.prepare("UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = 0, failure = ?, updated_at = ? WHERE id = ?")-- .run(`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`, at, i.id);-- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_failed', code });-- return get(i.id);++ const row = await t.one(sql`UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = false,++ failure = ${`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`}, updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);++ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_failed', code });++ return row;}-- /** Billing settled a transaction. Inside the inbox transaction. */-- function onBillingSettled(p) {++ /** Billing settled a transaction. Inside the inbox's MONEY transaction. */++ async function onBillingSettled(t, p) {const meta = p.metadata || {};const txn = { id: p.transaction_id, test: !!p.test };if (!txn.id) return 'ignored:no_transaction';if (p.type === 'donation') {-- const known = byBillingTxn(txn.id);-- if (known) return 'duplicate_transaction';++ if (await byBillingTxn(t, txn.id)) return 'duplicate_transaction';const target = meta.target;if (target && target.service === 'tips' && target.type === 'interaction') {-- const i = get(target.id);++ const i = await lock(t, target.id);if (!i) return 'ignored:unknown_interaction';// Only the transfer Tips asked for settles the interaction: same amount, same creator, same// supporter. Another service's donation that merely names it (a 1-bit transfer claiming a// 10,000-bit paid message) is recorded as what it is, a donation of its own amount.const matches = Number(meta.amount_bits) === i.amount && p.to_subject === i.creator_subject && (!i.supporter_subject || p.from_subject === i.supporter_subject);-- if (matches) { settleByTransaction(i, txn); return 'settled'; }++ if (matches) { await settleByTransaction(t, i, txn); return 'settled'; }ctx.log.warn(`[Tips] donation ${txn.id} names interaction ${i.id} but does not match it (amount, creator or supporter); recorded on its own`);}// A donation Tips did not start: record it once, by its transaction id.if (!p.to_subject) return 'ignored:no_recipient';const amount = Number(meta.amount_bits) || 0;if (amount <= 0) return 'ignored:no_amount';-- const i = insert({++ const i = await insert(t, {id: prefixedId('tint', ctx.now()), creator_subject: p.to_subject, supporter_subject: p.from_subject || null,supporter_name: displayName(meta.donor_name) || null, anonymous: meta.anonymous === true, kind: meta.kind === 'paid_interaction' ? 'media_request' : 'tip', amount, amount_cents: meta.paid_cents || null,-- message: meta.message ? String(meta.message).slice(0, config.limits.messageChars) : null,++ message: meta.message ? storable(String(meta.message).slice(0, config.limits.messageChars)) : null,funding: p.provider ? 'provider' : 'credit', settlement: 'billing', payment_state: 'pending', delivery_state: 'awaiting_payment',-- test: txn.test ? 1 : 0, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',++ test: txn.test, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',target: meta.target && meta.target.service ? meta.target : null,});-- settle(i);++ await settle(t, i);return 'recorded';}if (p.type === 'purchase' && meta.intent_id) {-- const i = db.prepare('SELECT * FROM tip_interactions WHERE billing_intent_id = ?').get(meta.intent_id);++ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_intent_id = ${meta.intent_id} FOR UPDATE`);if (!i) return 'ignored:not_a_tips_checkout';if (i.funding_txn_id) return 'duplicate_funding';-- db.prepare("UPDATE tip_interactions SET funding_txn_id = ?, test = MAX(test, ?), transfer_due = CASE WHEN payment_state = 'pending' THEN 1 ELSE 0 END, next_transfer_at = 0, updated_at = ? WHERE id = ?")-- .run(txn.id, txn.test ? 1 : 0, iso(ctx.now()), i.id);-- ctx.afterCommit(() => { processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)); });++ await t.exec(sql`UPDATE tip_interactions SET funding_txn_id = ${txn.id}, test = test OR ${txn.test}, transfer_due = (payment_state = 'pending'),++ next_transfer_at = 0, updated_at = ${iso(ctx.now())} WHERE id = ${i.id}`);++ ctx.afterCommit(t, () => processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)));return 'funded';}return 'ignored';}-- /** Billing reversed a transaction. Inside the inbox transaction. */-- function onBillingReversed(p) {++ /** Billing reversed a transaction. Inside the inbox's MONEY transaction. */++ async function onBillingReversed(t, p) {const meta = p.metadata || {};const orig = p.reverses_txn;if (!orig) return 'ignored:no_original';-- const i = byBillingTxn(orig);++ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${orig} FOR UPDATE`);if (!i) {-- const funded = db.prepare('SELECT * FROM tip_interactions WHERE funding_txn_id = ?').get(orig);-- if (funded && funded.payment_state === 'pending') { failPayment(funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }++ const funded = await t.maybe(sql`SELECT * FROM tip_interactions WHERE funding_txn_id = ${orig} FOR UPDATE`);++ if (funded && funded.payment_state === 'pending') { await failPayment(t, funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }return 'ignored';}-- const ids = json(i.reversal_txn_ids, []);++ const ids = Array.isArray(i.reversal_txn_ids) ? [...i.reversal_txn_ids] : [];if (ids.includes(p.transaction_id)) return 'duplicate_reversal';ids.push(p.transaction_id);// Bits Billing took back from the creator: a transfer refund moves them out of the payable. A@@@ -397,52 +414,54 @@ function createInteractions(ctx) {const clawed = p.type === 'refund' && p.from_subject === i.creator_subject ? Math.max(0, Number(meta.amount_bits) || 0) : 0;const take = Math.min(clawed, i.amount - i.reversed_bits);const at = iso(ctx.now());-- db.prepare("UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ?, reversed_bits = reversed_bits + ?, reversed_at = ?, updated_at = ? WHERE id = ?")-- .run(JSON.stringify(ids), take, at, at, i.id);-- if (take > 0 && !i.test) ctx.goals.reverse(i, take);++ const reversed = await t.one(sql`UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ${sql.json(ids)}, reversed_bits = reversed_bits + ${take},++ reversed_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);++ if (take > 0 && !i.test) await ctx.goals.reverse(t, reversed, take);// Undelivered effects are cancelled; a delivery that already happened stays on record.-- let cancelled = cancelQueued(get(i.id), 'payment_reversed');-- const now = get(i.id);++ let cancelled = await cancelQueued(t, reversed, 'payment_reversed');++ const now = await get(t, i.id);if (!cancelled && now.moderation === 'held' && now.delivery_state === 'queued') {// Held by the filter and never shown: it will not be released now.-- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);-- if (!now.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_reversed' });++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);++ if (!now.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_reversed' });cancelled = true;}return cancelled ? 'reversed_cancelled' : 'reversed';}-- /** Inside a transaction: queued effects → cancelled. True when anything was cancelled. */-- function cancelQueued(i, reason) {++ /** Inside a transaction, the interaction locked: queued effects → cancelled. True when anything was cancelled. */++ async function cancelQueued(t, i, reason) {const at = iso(ctx.now());-- const r = db.prepare("UPDATE interaction_effects SET state = 'cancelled', updated_at = ? WHERE interaction_id = ? AND state = 'queued'").run(at, i.id);-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);-- if (!r.changes) return false;-- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);-- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason });++ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND state = 'queued'`);++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);++ if (!n) return false;++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);++ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason });return true;}-- /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. */-- function recomputeDelivery(id) {-- const i = get(id);++ /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. Returns the row. */++ async function recomputeDelivery(t, id) {++ const i = await get(t, id);if (!i || i.delivery_state === 'cancelled' || i.delivery_state === 'awaiting_payment') return i;-- const effects = db.prepare('SELECT state FROM interaction_effects WHERE interaction_id = ?').all(id).map((e) => e.state);-- if (i.moderation === 'held' && !effects.length) return i; // waiting for a moderator: still to deliver++ const effects = await t.many(sql`SELECT effect, adapter, state, last_error FROM interaction_effects WHERE interaction_id = ${id} ORDER BY id`);++ const states = effects.map((e) => e.state);++ if (i.moderation === 'held' && !states.length) return i; // waiting for a moderator: still to deliverlet next = 'queued';-- if (effects.includes('failed') && !effects.includes('queued')) next = 'failed';-- else if (!effects.includes('queued')) next = 'delivered';++ if (states.includes('failed') && !states.includes('queued')) next = 'failed';++ else if (!states.includes('queued')) next = 'delivered';if (next === i.delivery_state) return i;const at = iso(ctx.now());-- db.prepare(`UPDATE tip_interactions SET delivery_state = ?, delivered_at = CASE WHEN ? = 'delivered' THEN ? ELSE delivered_at END, updated_at = ? WHERE id = ?`).run(next, next, at, at, id);++ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = ${next},++ delivered_at = CASE WHEN ${next === 'delivered'} THEN ${at}::timestamptz ELSE delivered_at END, updated_at = ${at} WHERE id = ${id} RETURNING *`);if (next === 'failed' && !i.test) {-- const failed = db.prepare("SELECT effect, adapter, last_error FROM interaction_effects WHERE interaction_id = ? AND state = 'failed'").all(id);-- ctx.outbox.emit('tips.interaction.failed', { type: 'interaction', id }, { ...summary(get(id)), failed_effects: failed });++ const failed = effects.filter((e) => e.state === 'failed').map(({ effect, adapter, last_error: lastError }) => ({ effect, adapter, last_error: lastError }));++ await ctx.outbox.emit(t, 'tips.interaction.failed', { type: 'interaction', id }, { ...summary(row), failed_effects: failed });}-- return get(id);++ return row;}/**@@@ -450,12 +469,12 @@ function createInteractions(ctx) {* or a Live donation_goals id (links minted by Live), which the Live import mapped to a Tips goal.* goals.contribute() still checks it is the creator's and active, else falls back to Live's rule.*/-- function goalFromPurpose(...refs) {++ async function goalFromPurpose(q, ...refs) {for (const v of refs) {-- const m = String(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);++ const m = storable(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);if (!m) continue;if (/^\d+$/.test(m[1])) {-- const map = db.prepare("SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ? AND status = 'imported'").get(m[1]);++ const map = await q.maybe(sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ${m[1]} AND status = 'imported'`);if (map && map.target_id) return map.target_id;continue;}@@@ -466,84 +485,84 @@ function createInteractions(ctx) {/*** Billing announced an EXTERNAL receipt (billing.receipt.external): a tip paid on the creator's own-- * PowerChat. Inside the inbox transaction. Recorded once by (provider, provider event id) — the same-- * key POST /interactions/external uses — settled at once (the provider already confirmed it), and-- * announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.++ * PowerChat. Inside the inbox's MONEY transaction. Recorded once by (provider, provider event id) —++ * the same key POST /interactions/external uses — settled at once (the provider already confirmed++ * it), and announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.*/-- function onBillingExternal(p) {++ async function onBillingExternal(t, p) {const provider = String(p.provider || '').toLowerCase();-- const ref = String(p.provider_event_id || '').trim();++ const ref = storable(p.provider_event_id || '').trim();if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider) || !ref || ref.length > 200) return 'ignored:no_reference';const creator = p.streamer && p.streamer.type === 'user' && SUBJECT_RE.test(String(p.streamer.id || '')) ? p.streamer.id : null;if (!creator) return 'ignored:no_streamer';const cents = Number(p.amount_cents);if (!Number.isSafeInteger(cents) || cents < 1 || cents > MAX_EXTERNAL_CENTS) return 'ignored:no_amount';-- if (db.prepare('SELECT 1 FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) return 'duplicate_receipt';++ if (await t.maybe(sql`SELECT 1 FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`)) return 'duplicate_receipt';// Goals and the chat line count Vibes: the value Billing put on the money (1 bit = 1 cent today).const bits = Number(p.value_bits);const amount = Math.min(Number.isSafeInteger(bits) && bits > 0 ? bits : cents, config.limits.maxBits);-- const message = p.message ? String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars) || null : null;-- const goalId = goalFromPurpose(p.app_purpose, p.app_ref);-- const i = insert({++ const message = p.message ? storable(String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars)) || null : null;++ const goalId = await goalFromPurpose(t, p.app_purpose, p.app_ref);++ const i = await insert(t, {id: prefixedId('tint', ctx.now()), creator_subject: creator, supporter_subject: null,supporter_name: p.anonymous ? 'Anonymous' : (displayName(p.donor_name) || 'Someone'), anonymous: !!p.anonymous, kind: 'tip', amount, amount_cents: cents,message, request: goalId ? { goal_id: goalId } : {}, funding: 'external', settlement: 'external', payment_state: 'pending',-- delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: p.test ? 1 : 0,++ delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: !!p.test,});-- settle(i);++ await settle(t, i);return 'recorded_external';}// ── External and simulated ───────────────────────────────/** A tip on the creator's own PowerChat (EXTERNAL, ADR-012): recorded once by provider ref. */-- function recordExternal(profile, input) {++ async function recordExternal(profile, input) {if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip profile');const provider = String(input.provider || '').toLowerCase();if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider)) fail(422, 'tips.invalid_input', 'provider is required');-- const ref = String(input.provider_ref || '').trim();++ const ref = storable(input.provider_ref || '').trim();if (!ref || ref.length > 200) fail(422, 'tips.invalid_input', 'provider_ref (the provider event id) is required');const cents = positiveInt(input.amount_cents, 'amount_cents', 100_000_000);const privacy = parsePrivacy(input.privacy, 'tip');-- return ctx.tx(() => {-- const prev = db.prepare('SELECT * FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref);++ return ctx.tx(async (t) => {++ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`);if (prev) return { interaction: prev, replay: true };-- const i = insert({++ const i = await insert(t, {id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: input.supporter ? userSubject(input.supporter, 'supporter') : null,supporter_name: displayName(input.supporter_name) || 'Someone', kind: 'tip', amount: cents, amount_cents: cents,message: text(input.message, 'message', 500), request: input.goal_id ? { goal_id: String(input.goal_id) } : {},funding: 'external', settlement: 'external', payment_state: 'pending', delivery_state: 'awaiting_payment',// origin external: the provider (or Live's webhook) already announced it in chat; with// announce: true Tips delivers the chat line itself.-- provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: input.test ? 1 : 0,++ provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: !!input.test,...privacy, private_message: privacy.private_message && !!input.message,});-- return { interaction: settle(i), replay: false };-- });++ return { interaction: await settle(t, i), replay: false };++ }, MONEY);}-- /** The full effect path with test = 1 and no Billing call. */-- function simulate(profile, input, { by }) {-- const v = validate(profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });++ /** The full effect path with test = true and no Billing call. */++ async function simulate(profile, input, { by }) {++ const v = await validate(db, profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });const privacy = parsePrivacy(input.privacy, v.kind);-- return ctx.tx(() => {-- const i = insert({++ return ctx.tx(async (t) => {++ const i = await insert(t, {id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: null,supporter_name: displayName(input.supporter_name) || 'Test supporter', kind: v.kind, amount: v.amount, message: v.message,request: { ...v.request, simulated_by: by }, funding: 'none', settlement: 'simulated', payment_state: 'pending',-- delivery_state: 'awaiting_payment', test: 1, ...privacy, private_message: privacy.private_message && !!v.message,++ delivery_state: 'awaiting_payment', test: true, ...privacy, private_message: privacy.private_message && !!v.message,});-- const out = settle(i);++ const out = await settle(t, i);// Show the goal widget moving without counting anything: a test goal delivery with the// would-be total. No contribution row is written.-- const g = ctx.goals.pick(profile.creator_subject, v.request.goal_id);++ const g = await ctx.goals.pick(t, profile.creator_subject, v.request.goal_id);if (g) {-- const view = ctx.goals.present(g);++ const [view] = await ctx.goals.presentMany(t, [g]);const would = { ...view, current_amount: view.current_amount + v.amount, percent: Math.min(100, Math.floor(((view.current_amount + v.amount) * 100) / view.target_amount)) };-- const shown = get(i.id);-- ctx.overlays.addGoalDelivery(profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: shown.hide_amount || shown.moderation !== 'visible' ? null : ctx.view(shown).supporter_name, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });++ const by2 = out.hide_amount || out.moderation !== 'visible' ? null : publicView(out, { filter: profile.filter }).supporter_name;++ await ctx.overlays.addGoalDelivery(t, profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: by2, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });}return out;-- });++ }, MONEY);}// ── Reads ────────────────────────────────────────────────@@@ -557,14 +576,8 @@ function createInteractions(ctx) {};}-- /**-- * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or-- * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;-- * `public` is what may be shown to the public, `privacy` the supporter's choices.-- */-- function present(i, { viewer = 'owner' } = {}) {-- if (!i) return null;-- const req = json(i.request, {});++ function shape(i, viewer, filter, effects) {++ const req = i.request || {};const own = viewer === 'supporter';const named = own || !isAnonymous(i);const out = {@@@ -572,15 +585,15 @@ function createInteractions(ctx) {supporter_name: named ? (i.supporter_name || null) : publicName(i), kind: i.kind, amount: i.amount, currency: i.currency, message: i.message || null,tts: req.tts || null, media: req.media ? { url: req.media.url, provider: req.media.provider } : null, goal_id: req.goal_id || null,funding: i.funding, settlement: i.settlement, origin: i.origin, test: !!i.test,-- privacy: privacyOf(i), public: ctx.view(i), erased_at: i.erased_at || null,++ privacy: privacyOf(i), public: publicView(i, { filter }), erased_at: i.erased_at || null,moderation: { state: i.moderation, filtered: !!i.filtered, at: i.moderated_at || null },-- payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: json(i.reversal_txn_ids, []), reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },++ payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: i.reversal_txn_ids || [], reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },delivery: { state: i.delivery_state, delivered_at: i.delivered_at || null },created_at: i.created_at, updated_at: i.updated_at,};if (i.funding === 'checkout') out.checkout = { url: i.checkout_url || null, ref: i.checkout_ref || null, provider: i.provider || null, intent_id: i.billing_intent_id || null };if (viewer === 'owner' || viewer === 'service') {-- out.effects = db.prepare('SELECT effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);++ out.effects = effects;out.provider = i.provider || null;out.provider_ref = i.provider_ref || null;out.legacy_source = i.legacy_source || null;@@@ -588,55 +601,83 @@ function createInteractions(ctx) {return out;}-- /** Cursor-paged list, newest first. filter: { creator?, supporter?, includeTest? } */-- function list({ creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {++ /**++ * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or++ * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;++ * `public` is what may be shown to the public, `privacy` the supporter's choices. A page of any++ * size costs two queries: the creators' filters, and (owner, service) every row's effects.++ */++ async function presentMany(q, rows, { viewer = 'owner' } = {}) {++ if (!rows.length) return [];++ const filters = await ctx.profiles.filtersOf(q, rows.map((i) => i.creator_subject));++ const effects = new Map();++ if (viewer === 'owner' || viewer === 'service') {++ const list = await q.many(sql`SELECT interaction_id, effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects++ WHERE interaction_id = ANY(${rows.map((i) => i.id)}) ORDER BY interaction_id, id`);++ for (const { interaction_id: id, ...e } of list) { if (!effects.has(id)) effects.set(id, []); effects.get(id).push(e); }++ }++ return rows.map((i) => shape(i, viewer, filters.get(i.creator_subject), effects.get(i.id) || []));++ }++ const present = async (q, i, opts) => (i ? (await presentMany(q, [i], opts))[0] : null);++++ /** A keyset page, newest first. filter: { creator?, supporter?, includeTest? } (one of the first two). */++ async function list(q, { creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {const n = Math.min(200, Math.max(1, Number(limit) || 50));-- let cur = null;-- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }-- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE (@c IS NULL OR creator_subject = @c) AND (@s IS NULL OR supporter_subject = @s)-- AND (@t = 1 OR test = 0) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))-- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator || null, s: supporter || null, t: includeTest ? 1 : 0, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });++ const cur = cursor ? readCursor(cursor) : null;++ const where = [];++ if (creator) where.push(sql`creator_subject = ${creator}`);++ if (supporter) where.push(sql`supporter_subject = ${supporter}`);++ if (!where.length) fail(422, 'tips.invalid_input', 'creator or supporter is required');++ if (!includeTest) where.push(sql`NOT test`);++ if (cur) where.push(sql`(created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)`);++ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE ${sql.join(where, sql` AND `)} ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);const page = rows.slice(0, n);-- const next = rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null;-- return { rows: page, next_cursor: next };++ return { rows: page, next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };}// ── The public supporters page ───────────────────────────-- // Only settled, non-test interactions of supporters who kept both their name and their amount-- // public count toward the leaderboard (a hidden amount must not be inferable from a rank).-- const leaderboardWhere = (t) => `${t}.creator_subject = @c AND ${t}.supporter_subject IS NOT NULL AND ${t}.test = 0-- AND ${t}.payment_state IN ('settled', 'reversed') AND ${t}.anonymous = 0 AND ${t}.hide_amount = 0 AND ${t}.erased_at IS NULL-- AND ${t}.moderation = 'visible'`;---- /** Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). */-- function leaderboard(creator, { limit = 20 } = {}) {-- const rows = db.prepare(`SELECT t.supporter_subject AS s, SUM(t.amount - t.reversed_bits) AS total, COUNT(*) AS tips, MIN(t.created_at) AS first_at,-- (SELECT j.supporter_name FROM tip_interactions j WHERE ${leaderboardWhere('j')} AND j.supporter_subject = t.supporter_subject-- ORDER BY j.created_at DESC LIMIT 1) AS name-- FROM tip_interactions t WHERE ${leaderboardWhere('t')}-- GROUP BY t.supporter_subject HAVING total > 0 ORDER BY total DESC, first_at LIMIT @n`).all({ c: creator, n: Math.min(100, Math.max(1, limit)) });-- const settings = ctx.profiles.filterOf(creator);-- return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, settings), total: r.total, tips: r.tips }));++ /**++ * Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). Only++ * settled, non-test interactions of supporters who kept both their name and their amount public++ * count (a hidden amount must not be inferable from a rank).++ */++ async function leaderboard(q, profile, { limit = 20 } = {}) {++ const rows = await q.many(sql`SELECT supporter_subject AS s, SUM(amount - reversed_bits)::bigint AS total, count(*) AS tips, MIN(created_at) AS first_at,++ (array_agg(supporter_name ORDER BY created_at DESC))[1] AS name++ FROM tip_interactions++ WHERE creator_subject = ${profile.creator_subject} AND supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed')++ AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible'++ GROUP BY supporter_subject HAVING SUM(amount - reversed_bits) > 0++ ORDER BY total DESC, first_at LIMIT ${Math.min(100, Math.max(1, limit))}`);++ return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, profile.filter), total: r.total, tips: r.tips }));}/** Recent settled interactions with a public message, as publicView(). */-- function recentPublic(creator, { limit = 20 } = {}) {-- return db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND payment_state = 'settled' AND private_message = 0-- AND erased_at IS NULL AND moderation = 'visible' AND message IS NOT NULL ORDER BY settled_at DESC, id DESC LIMIT ?`).all(creator, Math.min(100, Math.max(1, limit)))-- .map((i) => ctx.view(i));++ async function recentPublic(q, profile, { limit = 20 } = {}) {++ return (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${profile.creator_subject} AND payment_state = 'settled' AND message IS NOT NULL++ AND NOT test AND NOT private_message AND erased_at IS NULL AND moderation = 'visible' ORDER BY settled_at DESC, id DESC LIMIT ${Math.min(100, Math.max(1, limit))}`))++ .map((i) => publicView(i, { filter: profile.filter }));}// ── A supporter's own data ───────────────────────────────-- /** Everything Tips holds about a supporter's tips (their own view), for a download. */-- function exportFor(subject) {-- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ? ORDER BY created_at').all(subject);++ /** Everything Tips holds about a supporter's tips (their own view), for a download: five queries. */++ async function exportFor(subject) {++ const rows = await db.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY created_at`);++ const ids = rows.map((i) => i.id);++ const views = await presentMany(db, rows, { viewer: 'supporter' });++ const group = (list) => { const m = new Map(); for (const { interaction_id: id, ...r } of list) { if (!m.has(id)) m.set(id, []); m.get(id).push(r); } return m; };++ const [contributions, paid, media] = ids.length ? await Promise.all([++ db.many(sql`SELECT interaction_id, goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ANY(${ids}) ORDER BY id`),++ db.many(sql`SELECT interaction_id, kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ANY(${ids})`),++ db.many(sql`SELECT interaction_id, url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ANY(${ids})`),++ ]).then((r) => r.map(group)) : [new Map(), new Map(), new Map()];return {service: 'tips', subject: { type: 'user', id: subject }, exported_at: iso(ctx.now()),-- interactions: rows.map((i) => ({-- ...present(i, { viewer: 'supporter' }),-- goal_contributions: db.prepare('SELECT goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ?').all(i.id),-- paid_message: db.prepare('SELECT kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ?').get(i.id) || null,-- media_request: db.prepare('SELECT url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ?').get(i.id) || null,++ interactions: views.map((v) => ({++ ...v,++ goal_contributions: contributions.get(v.id) || [],++ paid_message: (paid.get(v.id) || [])[0] || null,++ media_request: (media.get(v.id) || [])[0] || null,})),not_included: 'Payments, balances and refunds are OpenVibe.Billing\'s records; ask Billing for them.',};@@@ -649,74 +690,67 @@ function createInteractions(ctx) {* answers, overlay payloads and the unsent events that named them. Each erased interaction emits* tips.interaction.erased, whose `redacts` has OpenVibe.Events tombstone the earlier events about* it. Interactions still waiting for their payment are kept (the payment needs the supporter) and-- * counted in kept_pending.++ * counted in kept_pending. The supporter's rows are locked in id order first; the writes are one++ * statement per table.*/function erase(subject) {-- return ctx.tx(() => {-- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ?').all(subject);++ return ctx.tx(async (t) => {++ const rows = await t.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY id FOR UPDATE`);const at = iso(ctx.now());-- let erased = 0;-- for (const i of rows) {-- if (i.payment_state === 'pending') continue;-- const req = json(i.request, {});-- const keep = {};-- if (req.goal_id) keep.goal_id = req.goal_id;-- if (req.highlight_seconds) keep.highlight_seconds = req.highlight_seconds;-- db.prepare(`UPDATE tip_interactions SET supporter_subject = NULL, supporter_name = NULL, anonymous = 1, message = NULL, request = ?,-- idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ?, updated_at = ? WHERE id = ?`)-- .run(JSON.stringify(keep), at, at, i.id);-- db.prepare("UPDATE paid_messages SET text = '', voice = NULL, updated_at = ? WHERE interaction_id = ?").run(at, i.id);-- db.prepare("UPDATE paid_media_requests SET url = '', updated_at = ? WHERE interaction_id = ?").run(at, i.id);-- const now = get(i.id);-- ctx.overlays.refreshInteraction(now);-- scrubOutbox(now);-- if (!i.test) {-- ctx.outbox.emit('tips.interaction.erased', { type: 'interaction', id: i.id }, {++ const done = rows.filter((i) => i.payment_state !== 'pending');++ if (done.length) {++ const keep = done.map((i) => {++ const req = i.request || {};++ const k = {};++ if (req.goal_id) k.goal_id = req.goal_id;++ if (req.highlight_seconds) k.highlight_seconds = req.highlight_seconds;++ return { id: i.id, request: k };++ });++ const ids = done.map((i) => i.id);++ const erased = await t.many(sql`UPDATE tip_interactions x SET supporter_subject = NULL, supporter_name = NULL, anonymous = true, message = NULL,++ request = v.request, idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ${at}, updated_at = ${at}++ FROM jsonb_to_recordset(${sql.json(keep)}) AS v(id text, request jsonb) WHERE x.id = v.id RETURNING x.*`);++ await t.exec(sql`UPDATE paid_messages SET text = '', voice = NULL, updated_at = ${at} WHERE interaction_id = ANY(${ids})`);++ await t.exec(sql`UPDATE paid_media_requests SET url = '', updated_at = ${at} WHERE interaction_id = ANY(${ids})`);++ await ctx.overlays.refreshInteractions(t, erased, await ctx.profiles.filtersOf(t, erased.map((i) => i.creator_subject)));++ // Events about them still in the local outbox lose the supporter's name.++ await t.exec(sql`UPDATE tips_event_outbox++ SET envelope = jsonb_set(jsonb_set(envelope, '{payload,supporter}', 'null'::jsonb), '{payload,supporter_name}', ${sql.json(ANONYMOUS)})++ WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY(${ids}) AND envelope->'payload' ? 'supporter'`);++ for (const i of done) {++ if (i.test) continue;++ await ctx.outbox.emit(t, 'tips.interaction.erased', { type: 'interaction', id: i.id }, {interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, erased_at: at,redacts: { subject_type: 'interaction', subject_ids: [i.id] },});}-- erased++;++ t.after(ctx.outboxKick);}// Stored API answers (Idempotency-Key replays) carry messages and the subject.-- const prefix = `${subject}:`;-- const answers = db.prepare('DELETE FROM api_idempotency WHERE substr(key, 1, ?) = ? OR instr(response, ?) > 0').run(prefix.length, prefix, subject).changes;-- if (erased) ctx.afterCommit(() => ctx.outboxKick());-- return { erased, kept_pending: rows.length - erased, stored_answers_removed: answers };++ const answers = await t.exec(sql`DELETE FROM api_idempotency WHERE starts_with(key, ${`${subject}:`}) OR strpos(response, ${subject}) > 0`);++ return { erased: done.length, kept_pending: rows.length - done.length, stored_answers_removed: answers };});}-- /** Events about an erased interaction that are still in the local outbox lose the supporter's name. */-- function scrubOutbox(i) {-- const rows = db.prepare("SELECT id, envelope FROM event_outbox WHERE json_extract(envelope, '$.subject.type') = 'interaction' AND json_extract(envelope, '$.subject.id') = ?").all(i.id);-- for (const r of rows) {-- const env = JSON.parse(r.envelope);-- if (!env.payload || !('supporter' in env.payload)) continue;-- env.payload.supporter = null;-- env.payload.supporter_name = publicName(i);-- db.prepare('UPDATE event_outbox SET envelope = ? WHERE id = ?').run(JSON.stringify(env), r.id);-- }-- }--/*** A creator's totals, derived from settled interactions. `billing` is what must equal Billing's* books (settled through Billing or imported from Live's ledger, minus what Billing took back);* external and test interactions are reported apart and never mixed in.*/-- function totals(creator) {-- const r = db.prepare(`SELECT-- COALESCE(SUM(CASE WHEN test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') THEN amount - reversed_bits END), 0) AS billing,-- COALESCE(SUM(CASE WHEN test = 0 AND settlement = 'external' AND payment_state = 'settled' THEN amount END), 0) AS external,-- COALESCE(SUM(CASE WHEN test = 0 AND payment_state IN ('settled', 'reversed') THEN 1 END), 0) AS count,-- COALESCE(SUM(CASE WHEN test = 1 THEN 1 END), 0) AS test_count,-- COALESCE(SUM(CASE WHEN payment_state = 'pending' THEN 1 END), 0) AS pending_count-- FROM tip_interactions WHERE creator_subject = ?`).get(creator);++ async function totals(q, creator) {++ const r = await q.one(sql`SELECT++ COALESCE(SUM(amount - reversed_bits) FILTER (WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed')), 0)::bigint AS billing,++ COALESCE(SUM(amount) FILTER (WHERE NOT test AND settlement = 'external' AND payment_state = 'settled'), 0)::bigint AS external,++ count(*) FILTER (WHERE NOT test AND payment_state IN ('settled', 'reversed')) AS count,++ count(*) FILTER (WHERE test) AS test_count,++ count(*) FILTER (WHERE payment_state = 'pending') AS pending_count++ FROM tip_interactions WHERE creator_subject = ${creator}`);return { currency: 'vibes-bits', settled_via_billing: r.billing, external: r.external, interactions: r.count, simulations: r.test_count, pending: r.pending_count };}return {-- get, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,-- onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, list, totals, insert, KINDS,++ get, lock, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,++ onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, presentMany, list, totals, insert, KINDS,leaderboard, recentPublic, exportFor, erase, release,};}@@@ -18,9 +18,15 @@* tips.interaction.moderate. Every outcome is recorded in tip_moderation_log and published as* tips.interaction.moderated (never for simulations): { interaction_id, creator, action, by,* moderation_state, cancelled_effects }. The payload names no supporter, message or moderator.++ *++ * hide() and restore() cancel or release what a payment bought (paid messages, TTS, media requests), so++ * they run SERIALIZABLE and lock the interaction row first (the domain's lock order): two moderators++ * acting at once apply one after the other.*/const crypto = require('crypto');--const { fail, iso, prefixedId, sha256, userSubject, displayName, text } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { fail, iso, prefixedId, sha256, userSubject, displayName, text, readCursor, cursorOf } = require('../util');++const { publicView } = require('./privacy');const filter = require('./filter');const INVITE_RE = /^tmin_[A-Za-z0-9_-]{43}$/;@@@ -32,139 +38,150 @@ function createModeration(ctx) {const { db } = ctx;// ── Moderators ───────────────────────────────────────────-- const isModerator = (creator, subject) => !!(subject && db.prepare('SELECT 1 FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').get(creator, subject));-- const listModerators = (creator) => db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL ORDER BY created_at').all(creator);++ const isModerator = async (q, creator, subject) => !!(subject && await q.maybe(sql`SELECT 1 FROM tip_moderators++ WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`));++ const listModerators = (q, creator) => q.many(sql`SELECT * FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL ORDER BY created_at`);/** Creators whose interactions this person moderates (with their handles). */-- const moderatedBy = (subject) => db.prepare(`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m-- JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ? AND m.removed_at IS NULL ORDER BY p.handle`).all(subject);++ const moderatedBy = (q, subject) => q.many(sql`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m++ JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ${subject} AND m.removed_at IS NULL ORDER BY p.handle`);-- function addModerator(creator, moderator, { name, addedBy }) {++ /** In q's transaction, or a new one: the creator's row is locked while the limit is counted. */++ function addModerator(q, creator, moderator, { name, addedBy }) {const subject = userSubject(moderator, 'moderator');if (subject === creator) fail(422, 'tips.invalid_input', 'the creator moderates their own page already');-- const count = db.prepare('SELECT COUNT(*) AS n FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL').get(creator).n;-- if (!isModerator(creator, subject) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);-- db.prepare(`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at) VALUES (?, ?, ?, ?, ?)-- ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, name), added_by = excluded.added_by`)-- .run(creator, subject, displayName(name), addedBy, iso(ctx.now()));-- return presentModerator(db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ?').get(creator, subject));++ return ctx.within(q, async (t) => {++ await ctx.lockCreator(t, creator);++ const count = await t.value(sql`SELECT count(*) FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL`);++ if (!(await isModerator(t, creator, subject)) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);++ return presentModerator(await t.one(sql`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at)++ VALUES (${creator}, ${subject}, ${displayName(name)}, ${addedBy}, ${iso(ctx.now())})++ ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, tip_moderators.name), added_by = excluded.added_by++ RETURNING *`));++ });}-- function removeModerator(creator, moderator) {++ async function removeModerator(creator, moderator) {const subject = userSubject(moderator, 'moderator');-- const r = db.prepare('UPDATE tip_moderators SET removed_at = ? WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').run(iso(ctx.now()), creator, subject);-- return { removed: r.changes > 0 };++ const n = await db.exec(sql`UPDATE tip_moderators SET removed_at = ${iso(ctx.now())} WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`);++ return { removed: n > 0 };}const presentModerator = (m) => (m ? { moderator: { type: 'user', id: m.moderator_subject }, name: m.name || null, added_at: m.created_at } : null);// ── Invitations (a show-once link the creator hands to a moderator) ──-- function createInvite(creator, { createdBy }) {-- const open = db.prepare('SELECT COUNT(*) AS n FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ?').get(creator, iso(ctx.now())).n;-- if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');++ async function createInvite(creator, { createdBy }) {const secret = `tmin_${crypto.randomBytes(32).toString('base64url')}`;const id = prefixedId('tmin', ctx.now());const at = ctx.now();-- db.prepare('INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)')-- .run(id, creator, sha256(secret), createdBy, iso(at), iso(at + INVITE_TTL_MS));++ await ctx.tx(async (t) => {++ await ctx.lockCreator(t, creator);++ const open = await t.value(sql`SELECT count(*) FROM tip_moderator_invites WHERE creator_subject = ${creator} AND used_at IS NULL AND revoked_at IS NULL++ AND expires_at > ${iso(at)}`);++ if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');++ await t.exec(sql`INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at)++ VALUES (${id}, ${creator}, ${sha256(secret)}, ${createdBy}, ${iso(at)}, ${iso(at + INVITE_TTL_MS)})`);++ });return { id, url: `${ctx.config.baseUrl}/moderate/invite/${secret}`, expires_at: iso(at + INVITE_TTL_MS) };}/** The open invitation for a presented secret, or null (unknown, used, revoked or expired). */-- function findInvite(secret) {++ async function findInvite(q, secret) {if (!INVITE_RE.test(String(secret || ''))) return null;-- const row = db.prepare('SELECT * FROM tip_moderator_invites WHERE token_hash = ?').get(sha256(secret));-- if (!row || row.used_at || row.revoked_at || row.expires_at <= iso(ctx.now())) return null;++ const row = await q.maybe(sql`SELECT * FROM tip_moderator_invites WHERE token_hash = ${sha256(secret)}`);++ if (!row || row.used_at || row.revoked_at || Date.parse(row.expires_at) <= ctx.now()) return null;return row;}++ /** Accept once: the invitation is taken by a conditional update, so two people at once cannot both use it. */function acceptInvite(secret, { subject, name }) {-- return ctx.tx(() => {-- const inv = findInvite(secret);++ return ctx.tx(async (t) => {++ const inv = await findInvite(t, secret);if (!inv) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');if (inv.creator_subject === subject) fail(422, 'tips.invalid_input', 'this is your own page');-- db.prepare('UPDATE tip_moderator_invites SET used_at = ?, used_by = ? WHERE id = ? AND used_at IS NULL').run(iso(ctx.now()), subject, inv.id);-- return { creator: inv.creator_subject, moderator: addModerator(inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };++ const n = await t.exec(sql`UPDATE tip_moderator_invites SET used_at = ${iso(ctx.now())}, used_by = ${subject} WHERE id = ${inv.id} AND used_at IS NULL AND revoked_at IS NULL`);++ if (!n) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');++ return { creator: inv.creator_subject, moderator: await addModerator(t, inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };});}-- const openInvites = (creator) => db.prepare('SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ? ORDER BY created_at').all(creator, iso(ctx.now()));-- function revokeInvite(creator, id) {-- return { revoked: db.prepare('UPDATE tip_moderator_invites SET revoked_at = ? WHERE id = ? AND creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL').run(iso(ctx.now()), String(id || ''), creator).changes > 0 };++ const openInvites = (q, creator) => q.many(sql`SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ${creator}++ AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ${iso(ctx.now())} ORDER BY created_at`);++ async function revokeInvite(creator, id) {++ const n = await db.exec(sql`UPDATE tip_moderator_invites SET revoked_at = ${iso(ctx.now())} WHERE id = ${String(id || '')} AND creator_subject = ${creator}++ AND used_at IS NULL AND revoked_at IS NULL`);++ return { revoked: n > 0 };}// ── The filter at settlement ─────────────────────────────/**-- * Inside the settlement transaction, before anything is shown: does the creator's filter match the-- * name or what will be shown or read? Returns { hit, hold }.++ * Before anything is shown or read: does the creator's filter (`settings`, profiles.filterOf) match++ * the name or what will be shown or read? Returns { hit, hold }.*/-- function screen(i) {-- const settings = ctx.profiles.filterOf(i.creator_subject);++ function screen(i, settings) {if (!settings || !settings.words.length) return { hit: false, hold: false };-- const req = JSON.parse(i.request || '{}');++ const req = i.request || {};const texts = [i.anonymous ? null : i.supporter_name, i.private_message ? null : i.message, req.tts && req.tts.text];const hit = filter.hits(settings, ...texts);return { hit, hold: hit && settings.action === 'hold' };}// ── Outcomes ─────────────────────────────────────────────-- function record(i, action, { role, actor = null, reason = null, cancelled = [] }) {-- db.prepare('INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')-- .run(i.creator_subject, i.id, action, role, actor, reason, iso(ctx.now()));++ async function record(t, i, action, { role, actor = null, reason = null, cancelled = [] }) {++ await t.exec(sql`INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at)++ VALUES (${i.creator_subject}, ${i.id}, ${action}, ${role}, ${actor}, ${reason}, ${iso(ctx.now())})`);if (!i.test) {-- const now = ctx.interactions.get(i.id);-- ctx.outbox.emit('tips.interaction.moderated', { type: 'interaction', id: i.id }, {++ await ctx.outbox.emit(t, 'tips.interaction.moderated', { type: 'interaction', id: i.id }, {interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, action, by: role,-- moderation_state: now.moderation, cancelled_effects: cancelled,++ moderation_state: i.moderation, cancelled_effects: cancelled,});-- ctx.afterCommit(() => ctx.outboxKick());++ ctx.afterCommit(t, () => ctx.outboxKick());}}/*** Settlement found a filter match (inside its transaction). markScreened() sets the state before-- * anything is shown (held, or only filtered = masked); recordScreened() logs and publishes it once-- * the settlement's own event is out.++ * anything is shown (held, or only filtered = masked) and returns the row; recordScreened() logs and++ * publishes it once the settlement's own event is out.*/-- function markScreened(i, verdict) {++ function markScreened(t, i, verdict) {const at = iso(ctx.now());-- if (verdict.hold) db.prepare("UPDATE tip_interactions SET filtered = 1, moderation = 'held', moderated_at = ?, moderated_by = 'filter', updated_at = ? WHERE id = ?").run(at, at, i.id);-- else db.prepare('UPDATE tip_interactions SET filtered = 1, updated_at = ? WHERE id = ?').run(at, i.id);++ return verdict.hold++ ? t.one(sql`UPDATE tip_interactions SET filtered = true, moderation = 'held', moderated_at = ${at}, moderated_by = 'filter', updated_at = ${at} WHERE id = ${i.id} RETURNING *`)++ : t.one(sql`UPDATE tip_interactions SET filtered = true, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);}-- function recordScreened(i, verdict) {-- record(i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });++ function recordScreened(t, i, verdict) {++ return record(t, i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });}-- /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else 403. */-- function actorFor(principal, creator, granted) {++ /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else null. */++ async function actorFor(q, principal, creator, granted) {if (principal.kind === 'service' && granted) return { role: 'service', actor: principal.sub };if (principal.kind === 'user' && principal.subject === creator) return { role: 'creator', actor: principal.subject };-- if (principal.kind === 'user' && isModerator(creator, principal.subject)) return { role: 'moderator', actor: principal.subject };++ if (principal.kind === 'user' && await isModerator(q, creator, principal.subject)) return { role: 'moderator', actor: principal.subject };return null;}/** Hide from overlays, chat/TTS still to come and public pages. The money is untouched. */function hide(i, { role, actor, reason }) {const note = text(reason, 'reason', 200);-- return ctx.tx(() => {-- const cur = ctx.interactions.get(i.id);++ return ctx.tx(async (t) => {++ const cur = await ctx.interactions.lock(t, i.id);if (cur.moderation === 'hidden') return { interaction: cur, changed: false };const at = iso(ctx.now());-- const cancelled = db.prepare(`SELECT effect FROM interaction_effects WHERE interaction_id = ? AND state = 'queued' AND effect IN (${CHAT_EFFECTS.map(() => '?').join(',')})`)-- .all(cur.id, ...CHAT_EFFECTS).map((e) => e.effect);++ const cancelled = (await t.many(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${at}++ WHERE interaction_id = ${cur.id} AND state = 'queued' RETURNING effect`)).map((e) => e.effect).filter((e) => CHAT_EFFECTS.includes(e));if (cancelled.length) {-- db.prepare(`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE interaction_id = ? AND state = 'queued'`).run(at, cur.id);-- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);-- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);++ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);++ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);}const undelivered = cancelled.length > 0 || cur.moderation === 'held' || cur.payment_state === 'pending';-- db.prepare(`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ?, moderated_by = ?,-- delivery_state = CASE WHEN ? AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END, updated_at = ? WHERE id = ?`)-- .run(at, actor || role, undelivered ? 1 : 0, at, cur.id);-- const now = ctx.interactions.get(cur.id);-- ctx.overlays.retract(now);-- record(now, 'hidden', { role, actor, reason: note, cancelled });-- return { interaction: ctx.interactions.get(cur.id), changed: true, cancelled_effects: cancelled };-- });++ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ${at}, moderated_by = ${actor || role},++ delivery_state = CASE WHEN ${undelivered} AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END,++ updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);++ await ctx.overlays.retract(t, now, await ctx.profiles.filterOf(t, now.creator_subject));++ await record(t, now, 'hidden', { role, actor, reason: note, cancelled });++ return { interaction: now, changed: true, cancelled_effects: cancelled };++ }, ctx.MONEY);}/**@@@ -174,26 +191,26 @@ function createModeration(ctx) {*/function restore(i, { role, actor, reason }) {const note = text(reason, 'reason', 200);-- return ctx.tx(() => {-- const cur = ctx.interactions.get(i.id);++ return ctx.tx(async (t) => {++ const cur = await ctx.interactions.lock(t, i.id);if (cur.moderation === 'visible') return { interaction: cur, changed: false };const at = iso(ctx.now());// Never shown (held, or hidden before it was): released now, its delivery starting over.-- const neverShown = !db.prepare("SELECT 1 FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").get(cur.id);++ const neverShown = !(await t.maybe(sql`SELECT 1 FROM overlay_deliveries WHERE interaction_id = ${cur.id} AND kind = 'alert'`));const release = cur.payment_state === 'settled' && cur.origin !== 'import' && neverShown;-- db.prepare(`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ?, moderated_by = ?,-- delivery_state = CASE WHEN ? THEN 'queued' ELSE delivery_state END, updated_at = ? WHERE id = ?`).run(at, actor || role, release ? 1 : 0, at, cur.id);-- const now = ctx.interactions.get(cur.id);-- ctx.overlays.unretract(now);-- if (release) ctx.interactions.release(now);-- record(now, 'restored', { role, actor, reason: note });-- return { interaction: ctx.interactions.get(cur.id), changed: true };-- });++ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ${at}, moderated_by = ${actor || role},++ delivery_state = CASE WHEN ${release} THEN 'queued' ELSE delivery_state END, updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);++ const settings = await ctx.profiles.filterOf(t, now.creator_subject);++ await ctx.overlays.unretract(t, now, settings);++ if (release) await ctx.interactions.release(t, now, settings);++ await record(t, now, 'restored', { role, actor, reason: note });++ return { interaction: await ctx.interactions.get(t, cur.id), changed: true };++ }, ctx.MONEY);}/** The moderators' view: what was written (not private things), and what the public sees. */-- function present(i) {-- const req = JSON.parse(i.request || '{}');++ function shape(i, settings) {++ const req = i.request || {};return {id: i.id, kind: i.kind, created_at: i.created_at, settled_at: i.settled_at || null, payment_state: i.payment_state, test: !!i.test,moderation: { state: i.moderation, at: i.moderated_at || null, filtered: !!i.filtered },@@@ -202,29 +219,34 @@ function createModeration(ctx) {message: i.private_message || i.erased_at ? null : (i.message || null),tts_text: req.tts && !i.erased_at ? req.tts.text : null,media_url: req.media && !i.erased_at ? req.media.url : null,-- public: ctx.view(i),++ public: publicView(i, { filter: settings }),};}++ const present = async (q, i) => shape(i, await ctx.profiles.filterOf(q, i.creator_subject));-- /** A creator's interactions for review, newest first. state: held | hidden | visible | all. */-- function queue(creator, { state = 'all', cursor, limit = 50 } = {}) {++ /** A creator's interactions for review, newest first, keyset-paged. state: held | hidden | visible | all. */++ async function queue(q, creator, { state = 'all', cursor, limit = 50 } = {}) {if (!['held', 'hidden', 'visible', 'all'].includes(state)) fail(422, 'tips.invalid_input', 'state must be held, hidden, visible or all');const n = Math.min(200, Math.max(1, Number(limit) || 50));-- let cur = null;-- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }-- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = @c AND payment_state IN ('settled', 'reversed', 'pending')-- AND (@s = 'all' OR moderation = @s) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))-- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator, s: state, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });++ const cur = cursor ? readCursor(cursor) : null;++ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND payment_state IN ('settled', 'reversed', 'pending')++ ${state === 'all' ? sql`` : sql`AND moderation = ${state}`} ${cur ? sql`AND (created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)` : sql``}++ ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);const page = rows.slice(0, n);-- return { rows: page.map(present), next_cursor: rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null };++ const settings = await ctx.profiles.filterOf(q, creator);++ return { rows: page.map((i) => shape(i, settings)), next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };}-- const log = (creator, limit = 100) => db.prepare('SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log WHERE creator_subject = ? ORDER BY id DESC LIMIT ?').all(creator, limit);++ const log = (q, creator, limit = 100) => q.many(sql`SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log++ WHERE creator_subject = ${creator} ORDER BY id DESC LIMIT ${limit}`);++++ /** Held paid messages waiting for review (the dashboard's count). */++ const heldCount = (q, creator) => q.value(sql`SELECT count(*) FROM tip_interactions WHERE creator_subject = ${creator} AND moderation = 'held' AND payment_state = 'settled'`);return {isModerator, listModerators, moderatedBy, addModerator, removeModerator, presentModerator,createInvite, findInvite, acceptInvite, openInvites, revokeInvite,-- screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log,++ screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log, heldCount,};}nnew file mode 100644@@@ -0,0 +1,223 @@++'use strict';++++/**++ * The overlays' live side: open SSE streams, fan-out between processes, and the per-token stream cap.++ *++ * Streams are sockets of one process; what they must hear can happen in any process (a tip settled by++ * another process, a moderator's hide, a config change, a revocation). Each process subscribes, per++ * creator with an open stream here, to `overlay:<creator>` on openvibe-sdk/pubsub (Valkey, or in-process++ * without it) and acts on what arrives:++ *++ * { t: 'new' } push the creator's new deliveries (read from the database)++ * { t: 'retract', interaction_id, rows } tell alert streams to drop an alert a moderator hid++ * { t: 'config', config } a changed overlay config, to the streams using it++ * { t: 'revoke', token_id } close the token's streams at once++ *++ * Nothing authoritative travels here: a lost message costs a push that the next one (or a reconnect's++ * replay from the database) makes up.++ *++ * Stream slots (TIPS_OVERLAY_MAX_STREAMS per token) are counted across processes in a Valkey sorted set++ * per creator (member <token>|<stream>, score = expiry); a stream refreshes its slot on every heartbeat,++ * so a dead process's slots lapse after three missed heartbeats. `connected(creator)` counts the live++ * slots. Without Valkey, or when it fails, the slots are counted in this process.++ */++const crypto = require('crypto');++const { createPubSub } = require('openvibe-sdk/pubsub');++++// Drop lapsed slots, count the token's, and take one when under the cap, atomically.++// KEYS[1] the creator's set; ARGV: now, '<token>|', max, expiry, member, ttl (ms).++const ACQUIRE = `++redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])++local n = 0++for _, m in ipairs(redis.call('ZRANGE', KEYS[1], 0, -1)) do++ if string.sub(m, 1, #ARGV[2]) == ARGV[2] then n = n + 1 end++end++if n >= tonumber(ARGV[3]) then return 0 end++redis.call('ZADD', KEYS[1], ARGV[4], ARGV[5])++redis.call('PEXPIRE', KEYS[1], ARGV[6])++return 1`;++++function memorySlots(now) {++ const byCreator = new Map(); // creator → Map(member → expiry)++ const live = (creator) => {++ const m = byCreator.get(creator);++ if (!m) return new Map();++ for (const [k, exp] of m) if (exp <= now()) m.delete(k);++ if (!m.size) byCreator.delete(creator);++ return m;++ };++ return {++ async acquire(creator, tokenId, member, max, ttlMs) {++ const m = live(creator);++ if ([...m.keys()].filter((k) => k.startsWith(`${tokenId}|`)).length >= max) return false;++ if (!byCreator.has(creator)) byCreator.set(creator, m);++ m.set(member, now() + ttlMs);++ return true;++ },++ async refresh(creator, member, ttlMs) { const m = byCreator.get(creator); if (m && m.has(member)) m.set(member, now() + ttlMs); },++ async release(creator, member) { const m = byCreator.get(creator); if (m) { m.delete(member); if (!m.size) byCreator.delete(creator); } },++ async count(creator) { return live(creator).size; },++ };++}++++function valkeySlots(valkey, now) {++ const c = valkey.client;++ if (typeof c.tipsOverlayAcquire !== 'function') c.defineCommand('tipsOverlayAcquire', { numberOfKeys: 1, lua: ACQUIRE });++ const key = (creator) => valkey.key('overlay', 'streams', creator);++ return {++ async acquire(creator, tokenId, member, max, ttlMs) {++ const t = now();++ return (await c.tipsOverlayAcquire(key(creator), String(t), `${tokenId}|`, String(max), String(t + ttlMs), member, String(ttlMs))) === 1;++ },++ async refresh(creator, member, ttlMs) { await c.multi().zadd(key(creator), 'XX', now() + ttlMs, member).pexpire(key(creator), ttlMs).exec(); },++ async release(creator, member) { await c.zrem(key(creator), member); },++ async count(creator) { return c.zcount(key(creator), now(), '+inf'); },++ };++}++++/**++ * @param {object} o++ * @param {object|null} o.valkey openvibe-sdk/valkey handle, or null (in-process)++ * @param {(client) => Promise<void>} o.pushNew read and write a client's new deliveries++ * @param {number} o.heartbeatMs++ */++function createOverlayHub({ valkey = null, pushNew, heartbeatMs, now = () => Date.now(), log = console }) {++ const ps = createPubSub({ valkey, log });++ const local = memorySlots(now);++ const shared = valkey ? valkeySlots(valkey, now) : null;++ const ttlMs = heartbeatMs * 3;++ let warned = false;++ /** The shared slot store; on a Valkey error this call counts in the process instead. */++ const slots = async (op, ...args) => {++ if (shared) {++ try { return await shared[op](...args); } catch (e) {++ if (!warned) { warned = true; log.warn(`[Tips] overlay stream slots: Valkey failed (${e.message}); counting in this process`); }++ }++ }++ return local[op](...args);++ };++ const creators = new Map(); // creator → { clients: Set, off: Promise<() => Promise> }++ let nextClient = 1;++++ function write(client, event, id, data) {++ if (client.closed) return false;++ try {++ client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);++ return true;++ } catch { return false; }++ }++++ /** Run fn for a client after whatever it is already doing (its pushes never interleave). */++ function queue(client, fn) {++ client.chain = client.chain.then(() => (client.closed ? null : fn())).catch((e) => log.warn('[Tips] overlay push:', e.message));++ return client.chain;++ }++ /** New deliveries for this client: coalesced, so a burst of notifications reads once. */++ function schedulePush(client) {++ if (client.pushQueued) return;++ client.pushQueued = true;++ queue(client, () => { client.pushQueued = false; return pushNew(client); });++ }++++ function onMessage(creator, msg) {++ const entry = creators.get(creator);++ if (!entry || !msg) return;++ for (const c of [...entry.clients]) {++ if (msg.t === 'new') schedulePush(c);++ else if (msg.t === 'retract' && c.scopes.includes('alerts')) {++ queue(c, () => { for (const r of msg.rows || []) write(c, 'retract', null, { interaction_id: msg.interaction_id, delivery_id: r.id, seq: r.seq }); });++ } else if (msg.t === 'config' && msg.config && c.configId === msg.config.id) {++ queue(c, () => { write(c, 'config', null, { config: msg.config }); });++ } else if (msg.t === 'revoke' && c.tokenId === msg.token_id) {++ write(c, 'revoked', null, { reason: 'this overlay token was revoked' });++ detach(c);++ }++ }++ }++++ // Through Valkey every process hears it, this one included. If publishing fails, this process's own++ // streams still hear it (the others catch up from the database on their next push or reconnect).++ const publish = (creator, msg) => ps.publish(`overlay:${creator}`, msg).then(() => undefined, (e) => {++ log.warn('[Tips] overlay fan-out:', e.message);++ onMessage(creator, JSON.parse(JSON.stringify(msg)));++ });++++ /** Subscribe for a creator's messages; a failure is retried while the creator has streams here. */++ function subscribe(creator, entry) {++ entry.off = ps.subscribe(`overlay:${creator}`, (msg) => onMessage(creator, msg)).catch((e) => {++ log.warn('[Tips] overlay subscribe:', e.message);++ const retry = setTimeout(() => { if (creators.get(creator) === entry) subscribe(creator, entry); }, 5000);++ if (retry.unref) retry.unref();++ return async () => {};++ });++ }++++ /** A stream place for this token, or null past the cap (the route answers 429). */++ async function reserve(token, max) {++ const slot = { creator: token.creator_subject, member: `${token.id}|${crypto.randomBytes(8).toString('hex')}` };++ return (await slots('acquire', slot.creator, token.id, slot.member, max, ttlMs)) ? slot : null;++ }++++ /**++ * Register a stream (its slot reserved) and run `start(client)` (hello, replay) before any push.++ * The subscription is in place before start reads, so nothing committed meanwhile is missed.++ */++ function attach(token, res, slot, { minAmount = 0, configId = null }, start) {++ const client = {++ id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId,++ minAmount, res, lastSeq: 0, closed: false, slot, chain: Promise.resolve(), pushQueued: false,++ };++ let entry = creators.get(client.creator);++ if (!entry) {++ entry = { clients: new Set(), off: null };++ creators.set(client.creator, entry);++ subscribe(client.creator, entry);++ }++ entry.clients.add(client);++ res.on('close', () => detach(client));++ if (res.destroyed) { detach(client); return client; } // gone before it was attached++ client.beat = setInterval(() => {++ if (!write(client, 'ping', null, { t: Date.now() })) { detach(client); return; }++ slots('refresh', slot.creator, slot.member, ttlMs).catch(() => {});++ }, heartbeatMs);++ if (client.beat.unref) client.beat.unref();++ queue(client, async () => { await entry.off; await start(client); });++ schedulePush(client);++ return client;++ }++++ function detach(client) {++ if (client.closed) return;++ client.closed = true;++ clearInterval(client.beat);++ slots('release', client.slot.creator, client.slot.member).catch(() => {});++ const entry = creators.get(client.creator);++ if (entry) {++ entry.clients.delete(client);++ if (!entry.clients.size) {++ creators.delete(client.creator);++ entry.off.then((off) => off()).catch(() => {});++ }++ }++ try { client.res.end(); } catch { /* already gone */ }++ }++++ return {++ write,++ attach,++ reserve,++ /** Give a reserved slot back (the stream never started). */++ release: (slot) => slots('release', slot.creator, slot.member).catch(() => {}),++ /** Tell every process's streams of this creator: new deliveries are in the database. */++ notify: (creator) => publish(creator, { t: 'new' }),++ retract: (creator, interactionId, rows) => publish(creator, { t: 'retract', interaction_id: interactionId, rows }),++ config: (creator, config) => publish(creator, { t: 'config', config }),++ revoke: (creator, tokenId) => publish(creator, { t: 'revoke', token_id: tokenId }),++ /** Open streams of this creator's overlays, in every process. */++ connected: (creator) => slots('count', creator),++ closeAll() { for (const entry of [...creators.values()]) for (const c of [...entry.clients]) detach(c); },++ async close() { this.closeAll(); await ps.close(); },++ };++}++++module.exports = { createOverlayHub };@@@ -1,16 +1,17 @@'use strict';/**-- * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE hub.++ * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE streams (overlay-hub.js).** Tokens. `tovl_<43 base64url chars>` shown ONCE at creation; only its SHA-256 is stored. A token* carries scopes (alerts, goals), optionally a config, and is never a creator cookie: an OBS* browser source holds it in its URL and nothing else. Revocation is immediate: the row is marked-- * and every open stream of that token is closed in the same call.++ * and every open stream of that token, in any process, is told and closed.** Deliveries. Every alert (a settled interaction) and goal change becomes one overlay_deliveries* row with a monotonic seq (the SSE event id). The first time a row is written to any overlay it-- * becomes `delivered` and emits tips.overlay.delivered; a row no overlay received within the++ * becomes `delivered` and emits tips.overlay.delivered (recorded just before the bytes are written);++ * a row no overlay received within the* delivery window becomes `failed` (tips.overlay.failed). A reconnect with Last-Event-ID* REPLAYS rows after that id: replay only writes bytes to a socket — it never charges, never* re-counts a goal, never emits another event. Simulated (test) rows are flagged and emit nothing.@@@ -22,62 +23,70 @@* sent, replayed or listed again — and tells connected overlays to drop it (`retract` event).*/const crypto = require('crypto');--const { fail, iso, prefixedId, sha256, json, text } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { fail, iso, prefixedId, sha256, text } = require('../util');const { safeUrl } = require('./profiles');--const { isHidden } = require('./privacy');++const { isHidden, publicView } = require('./privacy');++const { createOverlayHub } = require('./overlay-hub');const SCOPES = ['alerts', 'goals'];++const STREAM_LOCK = 4610; // the advisory-lock namespace of a creator's overlay stream (Tips' port)const TOKEN_RE = /^tovl_[A-Za-z0-9_-]{43}$/;++// A delivery row with the interaction's real amount (an overlay's minimum applies to a hidden amount too).++const DELIVERY_COLUMNS = sql`d.*, i.amount AS interaction_amount`;++const DELIVERY_FROM = sql`overlay_deliveries d LEFT JOIN tip_interactions i ON i.id = d.interaction_id`;function createOverlays(ctx) {const { db } = ctx;-- const clients = new Map(); // creator subject → Set<client>-- let nextClient = 1;++ const hub = createOverlayHub({ valkey: ctx.valkey, pushNew, heartbeatMs: ctx.config.overlays.heartbeatMs, log: ctx.log });// ── Tokens ───────────────────────────────────────────────-- function createToken(creator, { scopes, label, configId, createdBy }) {++ async function createToken(creator, { scopes, label, configId, createdBy }) {const list = Array.isArray(scopes) && scopes.length ? [...new Set(scopes.map(String))] : SCOPES;if (list.some((s) => !SCOPES.includes(s))) fail(422, 'tips.invalid_input', `scopes must be among ${SCOPES.join(', ')}`);if (configId) {-- const c = getConfig(configId);++ const c = await getConfig(db, configId);if (!c || c.creator_subject !== creator) fail(404, 'tips.overlay_config_not_found', 'no such overlay config for this creator');}-- const active = db.prepare('SELECT COUNT(*) AS n FROM overlay_tokens WHERE creator_subject = ? AND revoked_at IS NULL').get(creator).n;-- if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');const secret = `tovl_${crypto.randomBytes(32).toString('base64url')}`;-- const id = prefixedId('tovt', ctx.now());-- db.prepare(`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)-- VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, sha256(secret), JSON.stringify(list), configId || null, text(label, 'label', 80), createdBy, iso(ctx.now()));-- return { token: presentToken(getToken(id)), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };++ const row = await ctx.tx(async (t) => {++ await ctx.lockCreator(t, creator);++ const active = await t.value(sql`SELECT count(*) FROM overlay_tokens WHERE creator_subject = ${creator} AND revoked_at IS NULL`);++ if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');++ return t.one(sql`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)++ VALUES (${prefixedId('tovt', ctx.now())}, ${creator}, ${sha256(secret)}, ${sql.json(list)}, ${configId || null}, ${text(label, 'label', 80)}, ${createdBy}, ${iso(ctx.now())})++ RETURNING *`);++ });++ return { token: presentToken(row), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };}-- const getToken = (id) => db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(String(id || '')) || null;-- const listTokens = (creator) => db.prepare('SELECT * FROM overlay_tokens WHERE creator_subject = ? ORDER BY revoked_at IS NOT NULL, created_at DESC').all(creator);++ const getToken = (q, id) => q.maybe(sql`SELECT * FROM overlay_tokens WHERE id = ${String(id || '')}`);++ const listTokens = (q, creator) => q.many(sql`SELECT * FROM overlay_tokens WHERE creator_subject = ${creator} ORDER BY revoked_at IS NOT NULL, created_at DESC`);/** The active token row for a presented secret, or null (unknown, malformed or revoked). */-- function authenticate(secret) {++ async function authenticate(q, secret) {if (!TOKEN_RE.test(String(secret || ''))) return null;-- const row = db.prepare('SELECT * FROM overlay_tokens WHERE token_hash = ?').get(sha256(secret));++ const row = await q.maybe(sql`SELECT * FROM overlay_tokens WHERE token_hash = ${sha256(secret)}`);if (!row || row.revoked_at) return null;-- return { ...row, scopes: json(row.scopes, []) };++ return row;}-- function revokeToken(row) {-- if (!row.revoked_at) db.prepare('UPDATE overlay_tokens SET revoked_at = ? WHERE id = ? AND revoked_at IS NULL').run(iso(ctx.now()), row.id);-- closeToken(row.id);-- return presentToken(getToken(row.id));++ async function revokeToken(row) {++ const out = await db.maybe(sql`UPDATE overlay_tokens SET revoked_at = ${iso(ctx.now())} WHERE id = ${row.id} AND revoked_at IS NULL RETURNING *`);++ await hub.revoke(row.creator_subject, row.id);++ return presentToken(out || await getToken(db, row.id));}function presentToken(t) {if (!t) return null;return {-- id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: json(t.scopes, []), label: t.label || null, config_id: t.config_id || null,++ id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes, label: t.label || null, config_id: t.config_id || null,created_at: t.created_at, last_used_at: t.last_used_at || null, revoked_at: t.revoked_at || null, active: !t.revoked_at,};}// ── Configs ──────────────────────────────────────────────-- const getConfig = (id) => db.prepare('SELECT * FROM overlay_configs WHERE id = ?').get(String(id || '')) || null;-- const listConfigs = (creator) => db.prepare('SELECT * FROM overlay_configs WHERE creator_subject = ? ORDER BY created_at').all(creator);++ const getConfig = (q, id) => q.maybe(sql`SELECT * FROM overlay_configs WHERE id = ${String(id || '')}`);++ const listConfigs = (q, creator) => q.many(sql`SELECT * FROM overlay_configs WHERE creator_subject = ${creator} ORDER BY created_at`);function settingsOf(kind, input = {}, base = {}) {const s = { ...base };@@@ -110,236 +119,248 @@ function createOverlays(ctx) {goal: { show_amounts: true, show_percent: true },};-- function createConfig(creator, input = {}) {++ async function createConfig(creator, input = {}) {const kind = input.kind === 'goal' ? 'goal' : input.kind === 'alerts' ? 'alerts' : fail(422, 'tips.invalid_input', "kind must be 'alerts' or 'goal'");const name = text(input.name, 'name', 80) || (kind === 'alerts' ? 'Alerts' : 'Goal');let goalId = null;if (kind === 'goal' && input.goal_id) {-- const g = ctx.goals.get(input.goal_id);++ const g = await ctx.goals.get(db, input.goal_id);if (!g || g.creator_subject !== creator) fail(404, 'tips.goal_not_found', 'no such goal for this creator');goalId = g.id;}-- const id = prefixedId('tovc', ctx.now());const at = iso(ctx.now());-- db.prepare(`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`)-- .run(id, creator, kind, name, JSON.stringify(settingsOf(kind, input, DEFAULTS[kind])), goalId, at, at);-- return presentConfig(getConfig(id));++ const row = await db.one(sql`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at)++ VALUES (${prefixedId('tovc', ctx.now())}, ${creator}, ${kind}, ${name}, ${sql.json(settingsOf(kind, input, DEFAULTS[kind]))}, ${goalId}, ${at}, ${at}) RETURNING *`);++ return presentConfig(row);}-- function updateConfig(c, input = {}) {-- if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);-- const settings = settingsOf(c.kind, input, json(c.settings, {}));++ /**++ * A settings change on the locked row (two at once apply one after the other); with `revision` it++ * applies only to that revision (else 409).++ */++ function updateConfig(config, input = {}) {++ return ctx.tx(async (t) => {++ const c = await t.one(sql`SELECT * FROM overlay_configs WHERE id = ${config.id} FOR UPDATE`);++ if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);++ return changeConfig(t, c, input);++ });++ }++ async function changeConfig(t, c, input) {++ const settings = settingsOf(c.kind, input, c.settings || {});let goalId = c.goal_id;if (c.kind === 'goal' && input.goal_id !== undefined) {if (!input.goal_id) goalId = null;else {-- const g = ctx.goals.get(input.goal_id);++ const g = await ctx.goals.get(t, input.goal_id);if (!g || g.creator_subject !== c.creator_subject) fail(404, 'tips.goal_not_found', 'no such goal for this creator');goalId = g.id;}}const name = input.name !== undefined ? (text(input.name, 'name', 80) || c.name) : c.name;-- db.prepare('UPDATE overlay_configs SET name = ?, settings = ?, goal_id = ?, revision = revision + 1, updated_at = ? WHERE id = ?')-- .run(name, JSON.stringify(settings), goalId, iso(ctx.now()), c.id);-- const out = presentConfig(getConfig(c.id));-- // Open overlays using this config pick the change up at once.-- for (const cl of clients.get(c.creator_subject) || []) if (cl.configId === c.id) write(cl, 'config', null, { config: out });++ const out = presentConfig(await t.one(sql`UPDATE overlay_configs SET name = ${name}, settings = ${sql.json(settings)}, goal_id = ${goalId},++ revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${c.id} RETURNING *`));++ // Open overlays using this config, in any process, pick the change up once it is committed.++ t.after(() => hub.config(c.creator_subject, out));return out;}function presentConfig(c) {if (!c) return null;-- return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: json(c.settings, {}), goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };++ return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: c.settings || {}, goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };}// ── Deliveries ───────────────────────────────────────────-- function insertDelivery(creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {-- const id = prefixedId('tovd', ctx.now());-- const r = db.prepare(`INSERT OR IGNORE INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)-- VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)`).run(id, creator, kind, interactionId, goalId, JSON.stringify(payload), test ? 1 : 0, dedupe, iso(ctx.now()));-- if (r.changes) ctx.afterCommit(() => notify(creator));-- return r.changes ? id : null;++ /**++ * Inside a transaction. Open overlays hear of it once the transaction committed.++ *++ * A row's seq comes from its identity when it is inserted, but other processes see it only at commit.++ * Two transactions of one creator committing out of order (seq 101 before 100) would let a stream move++ * past 100 before 100 exists for it, and never send it. So a creator's deliveries are inserted under++ * a transaction-scoped advisory lock (allowed through PgBouncer): the next insert for that creator++ * waits for the previous transaction to end, and a creator's seqs become visible in order.++ */++ async function insertDelivery(t, creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {++ await t.query(sql`SELECT pg_advisory_xact_lock(${STREAM_LOCK}, hashtext(${creator}))`);++ const row = await t.maybe(sql`INSERT INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)++ VALUES (${prefixedId('tovd', ctx.now())}, ${creator}, ${kind}, ${interactionId}, ${goalId}, ${sql.json(payload)}, ${!!test}, 'pending', ${dedupe}, ${iso(ctx.now())})++ ON CONFLICT DO NOTHING RETURNING id`);++ if (row) ctx.afterCommit(t, () => hub.notify(creator));++ return row ? row.id : null;}-- /** Inside the settlement transaction (or a simulation). */-- function addAlert(interaction) {-- const payload = ctx.view(interaction, { at: iso(ctx.now()) });-- return insertDelivery(interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });++ /** Inside the settlement transaction (or a simulation). `filter`: the creator's word filter. */++ function addAlert(t, interaction, filter) {++ const payload = publicView(interaction, { at: iso(ctx.now()), filter });++ return insertDelivery(t, interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });}/**-- * Inside a transaction: the interaction's public view changed. Its alert payload is rewritten, and-- * the goal updates it caused no longer name anyone.++ * Inside a transaction: these interactions' public view changed (an erasure, a moderation). Their++ * alert payloads are rewritten and the goal updates they caused no longer name anyone, in one read++ * and one write. `filters`: Map(creator → the creator's word filter).*/-- function refreshInteraction(interaction) {-- const view = ctx.view(interaction);-- for (const row of db.prepare('SELECT seq, kind, payload FROM overlay_deliveries WHERE interaction_id = ?').all(interaction.id)) {-- const old = json(row.payload, {});-- const payload = row.kind === 'alert' ? ctx.view(interaction, { at: old.at }) : { ...old, by: interaction.hide_amount || isHidden(interaction) ? null : view.supporter_name };-- db.prepare('UPDATE overlay_deliveries SET payload = ? WHERE seq = ?').run(JSON.stringify(payload), row.seq);-- }++ async function refreshInteractions(t, interactions, filters) {++ if (!interactions.length) return;++ const byId = new Map(interactions.map((i) => [i.id, i]));++ const rows = await t.many(sql`SELECT seq, kind, payload, interaction_id FROM overlay_deliveries WHERE interaction_id = ANY(${[...byId.keys()]})`);++ if (!rows.length) return;++ const updates = rows.map((row) => {++ const i = byId.get(row.interaction_id);++ const filter = filters.get(i.creator_subject) || null;++ const old = row.payload || {};++ const payload = row.kind === 'alert' ? publicView(i, { at: old.at, filter })++ : { ...old, by: i.hide_amount || isHidden(i) ? null : publicView(i, { filter }).supporter_name };++ return { seq: row.seq, payload };++ });++ await t.exec(sql`UPDATE overlay_deliveries d SET payload = v.payload++ FROM jsonb_to_recordset(${sql.json(updates)}) AS v(seq bigint, payload jsonb) WHERE d.seq = v.seq`);}++ const refreshInteraction = (t, interaction, filter) => refreshInteractions(t, [interaction], new Map([[interaction.creator_subject, filter]]));/** Inside a transaction: a moderator hid the interaction. Its alert is never sent or replayed again. */-- function retract(interaction) {-- db.prepare("UPDATE overlay_deliveries SET hidden = 1 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);-- refreshInteraction(interaction);-- const rows = db.prepare("SELECT id, seq FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").all(interaction.id);-- ctx.afterCommit(() => {-- for (const c of clients.get(interaction.creator_subject) || []) {-- if (!c.scopes.includes('alerts')) continue;-- for (const r of rows) write(c, 'retract', null, { interaction_id: interaction.id, delivery_id: r.id, seq: r.seq });-- }-- });++ async function retract(t, interaction, filter) {++ const rows = await t.many(sql`UPDATE overlay_deliveries SET hidden = true WHERE interaction_id = ${interaction.id} AND kind = 'alert' RETURNING id, seq`);++ await refreshInteraction(t, interaction, filter);++ if (rows.length) ctx.afterCommit(t, () => hub.retract(interaction.creator_subject, interaction.id, rows.map((r) => ({ id: r.id, seq: r.seq }))));}/** Inside a transaction: shown again. Back in /state and replays; open overlays are not re-alerted. */-- function unretract(interaction) {-- db.prepare("UPDATE overlay_deliveries SET hidden = 0 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);-- refreshInteraction(interaction);++ async function unretract(t, interaction, filter) {++ await t.exec(sql`UPDATE overlay_deliveries SET hidden = false WHERE interaction_id = ${interaction.id} AND kind = 'alert'`);++ await refreshInteraction(t, interaction, filter);}-- function addGoalDelivery(creator, goalView, { reason, interactionId, by, dedupe, test = false }) {-- return insertDelivery(creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });++ function addGoalDelivery(t, creator, goalView, { reason, interactionId, by, dedupe, test = false }) {++ return insertDelivery(t, creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });}-- /** First write of a pending row: delivered + the event (never for tests). */-- function markDelivered(row) {-- ctx.tx(() => {-- const r = db.prepare("UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);-- if (r.changes && !row.test) {-- ctx.outbox.emit('tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {++ /**++ * About to write these rows to an overlay: count the sends, and the first write of a pending row++ * makes it delivered + the event (never for tests). One transaction for the batch.++ */++ async function recordSends(rows) {++ const pending = rows.filter((r) => r.status === 'pending').map((r) => r.seq);++ await ctx.tx(async (t) => {++ await t.exec(sql`UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ANY(${rows.map((r) => r.seq)})`);++ if (!pending.length) return;++ const done = await t.many(sql`UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ${iso(ctx.now())}++ WHERE seq = ANY(${pending}) AND status = 'pending' RETURNING id, creator_subject, kind, interaction_id, goal_id, test`);++ for (const row of done) {++ if (row.test) continue;++ await ctx.outbox.emit(t, 'tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id, goal_id: row.goal_id,});}++ if (done.some((r) => !r.test)) t.after(ctx.outboxKick);});-- ctx.outboxKick();}-- /** Pending rows older than the window → failed (tips.overlay.failed). Returns how many. */-- function sweepFailed() {++ /**++ * Pending rows older than the window → failed (tips.overlay.failed). Returns how many. Rows are++ * claimed with SKIP LOCKED, so processes sweeping at once never fail a row twice.++ */++ async function sweepFailed() {const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE status = 'pending' AND hidden = 0 AND created_at < ? ORDER BY seq LIMIT 500").all(cutoff);-- if (!rows.length) return 0;-- ctx.tx(() => {++ const n = await ctx.tx(async (t) => {++ const rows = await t.many(sql`UPDATE overlay_deliveries SET status = 'failed', failed_at = ${iso(ctx.now())}++ WHERE seq IN (SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < ${cutoff}++ ORDER BY seq LIMIT 500 FOR UPDATE SKIP LOCKED)++ RETURNING id, seq, creator_subject, kind, interaction_id, goal_id, test`);++ rows.sort((a, b) => a.seq - b.seq);for (const row of rows) {-- const r = db.prepare("UPDATE overlay_deliveries SET status = 'failed', failed_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);-- if (r.changes && !row.test) {-- ctx.outbox.emit('tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {-- delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,-- goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',-- });-- }++ if (row.test) continue;++ await ctx.outbox.emit(t, 'tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {++ delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,++ goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',++ });}++ return rows.length;});-- ctx.outboxKick();-- return rows.length;++ if (n) ctx.outboxKick();++ return n;}-- // ── SSE hub ──────────────────────────────────────────────++ // ── Streams (overlay-hub.js) ─────────────────────────────const scopeOf = (kind) => (kind === 'alert' ? 'alerts' : 'goals');-- function write(client, event, id, data) {-- if (client.closed) return false;-- try {-- client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);-- return true;-- } catch { return false; }-- }---- function send(client, row) {-- client.lastSeq = Math.max(client.lastSeq, row.seq);-- if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) return;-- const payload = json(row.payload, {});-- if (row.kind === 'alert' && client.minAmount && !row.test) {-- // A hidden amount is not in the payload; the threshold still applies to the real one.-- const amount = payload.amount != null ? payload.amount : (db.prepare('SELECT amount FROM tip_interactions WHERE id = ?').get(row.interaction_id) || {}).amount;-- if (amount < client.minAmount) return;++ /** Write these rows to a client (scopes, retractions and its minimum applied), after recording them. */++ async function deliver(client, rows) {++ const shown = [];++ for (const row of rows) {++ client.lastSeq = Math.max(client.lastSeq, row.seq);++ if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) continue;++ const payload = row.payload || {};++ if (row.kind === 'alert' && client.minAmount && !row.test) {++ // A hidden amount is not in the payload; the threshold still applies to the real one.++ const amount = payload.amount != null ? payload.amount : row.interaction_amount;++ if (amount < client.minAmount) continue;++ }++ shown.push({ row, payload });}-- if (!write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload })) return;-- db.prepare('UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ?').run(row.seq);-- if (row.status === 'pending') markDelivered(row);++ if (!shown.length || client.closed) return;++ await recordSends(shown.map((x) => x.row));++ for (const { row, payload } of shown) hub.write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload });}-- function pushNew(client) {-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq LIMIT 200").all(client.creator, client.lastSeq);-- for (const row of rows) send(client, row);++ async function pushNew(client) {++ const rows = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}++ WHERE d.creator_subject = ${client.creator} AND d.seq > ${client.lastSeq} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq LIMIT 200`);++ await deliver(client, rows);}-- function notify(creator) {-- for (const c of clients.get(creator) || []) pushNew(c);-- }++ /** A stream place for this token (TIPS_OVERLAY_MAX_STREAMS across processes), or null: answer 429. */++ const reserveStream = (token) => hub.reserve(token, ctx.config.overlays.maxStreamsPerToken);++ const releaseStream = (slot) => hub.release(slot);/**-- * Attach an SSE response. lastEventId (Last-Event-ID): replay what came after it (bounded).-- * Without it: what is still pending within the window, then everything new.++ * Attach an SSE response (its slot from reserveStream). lastEventId (Last-Event-ID): replay what++ * came after it (bounded). Without it: what is still pending within the window, then everything new.*/-- function attach(token, res, { lastEventId, config } = {}) {-- const settings = config ? json(config.settings, {}) : {};-- const client = {-- id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId: token.config_id || null,-- minAmount: config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0, res, lastSeq: 0, closed: false,-- };-- if (!clients.has(client.creator)) clients.set(client.creator, new Set());-- clients.get(client.creator).add(client);-- db.prepare('UPDATE overlay_tokens SET last_used_at = ? WHERE id = ?').run(iso(ctx.now()), token.id);---- const maxSeq = db.prepare('SELECT COALESCE(MAX(seq), 0) AS n FROM overlay_deliveries WHERE creator_subject = ?').get(client.creator).n;-- write(client, 'hello', null, {-- creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,-- goals: client.scopes.includes('goals') ? ctx.goals.list(client.creator, { status: 'active' }).map((g) => ctx.goals.present(g)) : [],-- resumed_from: lastEventId != null ? lastEventId : null,++ function attach(token, res, slot, { lastEventId, config } = {}) {++ const settings = config ? config.settings || {} : {};++ const minAmount = config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0;++ return hub.attach(token, res, slot, { minAmount, configId: token.config_id || null }, async (client) => {++ await db.exec(sql`UPDATE overlay_tokens SET last_used_at = ${iso(ctx.now())} WHERE id = ${token.id}`);++ const maxSeq = await db.value(sql`SELECT COALESCE(MAX(seq), 0) FROM overlay_deliveries WHERE creator_subject = ${client.creator}`);++ const goals = client.scopes.includes('goals') ? await ctx.goals.presentMany(db, await ctx.goals.list(db, client.creator, { status: 'active' })) : [];++ hub.write(client, 'hello', null, {++ creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,++ goals, resumed_from: lastEventId != null ? lastEventId : null,++ });++ const last = Number(lastEventId);++ if (lastEventId != null && Number.isInteger(last) && last >= 0) {++ const limit = ctx.config.overlays.replayLimit;++ const rows = (await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}++ WHERE d.creator_subject = ${client.creator} AND d.seq > ${last} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq DESC LIMIT ${limit}`)).reverse();++ client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;++ await deliver(client, rows);++ client.lastSeq = Math.max(client.lastSeq, maxSeq);++ } else {++ const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);++ const pending = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}++ WHERE d.creator_subject = ${client.creator} AND d.status = 'pending' AND NOT d.hidden AND d.created_at >= ${cutoff} ORDER BY d.seq LIMIT 100`);++ await deliver(client, pending);++ client.lastSeq = Math.max(client.lastSeq, maxSeq);++ }});-- const last = Number(lastEventId);-- if (lastEventId != null && Number.isInteger(last) && last >= 0) {-- const limit = ctx.config.overlays.replayLimit;-- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq DESC LIMIT ?").all(client.creator, last, limit).reverse();-- client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;-- for (const row of rows) send(client, row);-- client.lastSeq = Math.max(client.lastSeq, maxSeq);-- } else {-- const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);-- const pending = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND status = 'pending' AND hidden = 0 AND created_at >= ? ORDER BY seq LIMIT 100").all(client.creator, cutoff);-- for (const row of pending) send(client, row);-- client.lastSeq = Math.max(client.lastSeq, maxSeq);-- }-- pushNew(client);-- const beat = setInterval(() => { if (!write(client, 'ping', null, { t: Date.now() })) detach(client); }, ctx.config.overlays.heartbeatMs);-- if (beat.unref) beat.unref();-- client.beat = beat;-- res.on('close', () => detach(client));-- return client;}-- function detach(client) {-- if (client.closed) return;-- client.closed = true;-- clearInterval(client.beat);-- const set = clients.get(client.creator);-- if (set) { set.delete(client); if (!set.size) clients.delete(client.creator); }-- try { client.res.end(); } catch { /* already gone */ }++ function recentDeliveries(q, creator, limit = 50) {++ return q.many(sql`SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at++ FROM overlay_deliveries WHERE creator_subject = ${creator} ORDER BY seq DESC LIMIT ${limit}`);}-- function closeToken(tokenId) {-- for (const set of clients.values()) {-- for (const c of [...set]) {-- if (c.tokenId === tokenId) { write(c, 'revoked', null, { reason: 'this overlay token was revoked' }); detach(c); }-- }-- }-- }---- function connected(creator) { return (clients.get(creator) || new Set()).size; }-- function streamsFor(token) { let n = 0; for (const c of clients.get(token.creator_subject) || []) if (c.tokenId === token.id) n++; return n; }-- function closeAll() { for (const set of clients.values()) for (const c of [...set]) detach(c); }---- function recentDeliveries(creator, limit = 50) {-- return db.prepare('SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at FROM overlay_deliveries WHERE creator_subject = ? ORDER BY seq DESC LIMIT ?').all(creator, limit);++ /** The newest alerts of a creator, for GET /overlay/:token/state. */++ async function recentAlerts(q, creator) {++ return (await q.many(sql`SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ${creator} AND kind = 'alert' AND NOT hidden++ ORDER BY seq DESC LIMIT 20`)).map((d) => ({ seq: d.seq, ...(d.payload || {}), test: !!d.test, created_at: d.created_at }));}return {SCOPES, createToken, getToken, listTokens, authenticate, revokeToken, presentToken,getConfig, listConfigs, createConfig, updateConfig, presentConfig,-- addAlert, refreshInteraction, retract, unretract, addGoalDelivery, sweepFailed, attach, detach, closeToken, notify, connected, streamsFor, closeAll, recentDeliveries,++ addAlert, refreshInteraction, refreshInteractions, retract, unretract, addGoalDelivery, sweepFailed, reserveStream, releaseStream, attach,++ connected: hub.connected, closeAll: () => hub.closeAll(), close: () => hub.close(), recentDeliveries, recentAlerts,};}@@@ -25,7 +25,7 @@* word filter (filter.js) stars blocked words out of the name and message and drops them from what* text-to-speech reads.*/--const { fail, json } = require('../util');++const { fail } = require('../util');const filter = require('./filter');const ANONYMOUS = 'Anonymous';@@@ -64,10 +64,11 @@ function shownName(i, settings) {/*** The public shape of an interaction. `at` is kept from an earlier overlay payload when given, so a-- * rewritten alert keeps its time; `filter` is the creator's filter settings.++ * rewritten alert keeps its time; `filter` is the creator's filter settings (profiles.filterOf, or a++ * profile's `filter`), fetched once per creator by the caller.*/function publicView(i, { at, filter: settings = null } = {}) {-- const req = json(i.request, {});++ const req = i.request || {};const privateMsg = !!i.private_message || !!i.erased_at;const base = {interaction_id: i.id,@@@ -19,7 +19,8 @@* filter_words / filter_action / filter_links are the creator's word filter for paid messages* (filter.js; `filter` in the API): filterOf(subject) is what every public view applies.*/--const { fail, iso, text, json } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { fail, iso, text } = require('../util');const filterLib = require('./filter');const HANDLE_RE = /^[a-z0-9][a-z0-9_.-]{0,39}$/;@@@ -36,55 +37,70 @@ function normalizeHandle(v) {const PAGE_DEFAULTS = Object.freeze({ goal_amounts: true, goal_supporters: false, supporters_page: false, supporters_amounts: false, supporters_messages: false });--function pageSettings(raw) {-- const stored = json(raw, {});++/** page_settings (jsonb) → every setting, defaults filled in. */++function pageSettings(stored) {++ const s = stored && typeof stored === 'object' ? stored : {};const out = {};-- for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof stored[k] === 'boolean' ? stored[k] : PAGE_DEFAULTS[k];++ for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof s[k] === 'boolean' ? s[k] : PAGE_DEFAULTS[k];return out;}++/** The creator's filter settings for publicView() from a profile row (filter_* columns). */++const filterOfRow = (row) => ({++ words: Array.isArray(row.filter_words) ? row.filter_words : [],++ action: row.filter_action === 'hold' ? 'hold' : 'mask',++ links: row.filter_links !== false,++});++function parse(row) {if (!row) return null;-- const b = (k) => !!row[k];-- return {-- ...row, page_enabled: b('page_enabled'), accepting: b('accepting'), tts_enabled: b('tts_enabled'), media_requests_enabled: b('media_requests_enabled'),-- page: pageSettings(row.page_settings),-- filter: { words: json(row.filter_words, []), action: filterLib.ACTIONS.includes(row.filter_action) ? row.filter_action : 'mask', links: row.filter_links !== 0 },-- };++ return { ...row, page: pageSettings(row.page_settings), filter: filterOfRow(row) };}++const HANDLE_TAKEN = 'creator_tip_profiles_handle_key';++function createProfiles(ctx) {const { db } = ctx;-- const bySubject = (s) => parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE creator_subject = ?').get(s));-- const byHandle = (h) => { const n = normalizeHandle(h); return n ? parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE handle = ?').get(n)) : null; };++ const bySubject = async (q, s) => parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(s || '')}`));++ const byHandle = async (q, h) => { const n = normalizeHandle(h); return n ? parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE handle = ${n}`)) : null; };/** usr_… or a handle (with or without @). */-- const resolve = (ref) => (/^usr_/.test(String(ref || '')) ? bySubject(String(ref)) : byHandle(ref));++ const resolve = (q, ref) => (/^usr_/.test(String(ref || '')) ? bySubject(q, String(ref)) : byHandle(q, ref));/** A handle not held by another creator: the username, else username-<suffix of the subject>. */-- function freeHandle(subject, username) {++ async function freeHandle(q, subject, username) {let h = normalizeHandle(username) || `creator-${subject.slice(-8).toLowerCase()}`;-- const holder = db.prepare('SELECT creator_subject FROM creator_tip_profiles WHERE handle = ?').get(h);++ const holder = await q.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE handle = ${h}`);if (holder && holder.creator_subject !== subject) h = `${h.slice(0, 30)}-${subject.slice(-6).toLowerCase()}`;return h;}-- /** Create or refresh the profile from the creator's own Network account (claims). */-- function ensure(subject, { username, displayName, avatarUrl } = {}) {++ /**++ * Create or refresh the profile from the creator's own Network account (claims). Two first visits++ * at once create it once; a handle another creator took meanwhile is looked for again.++ */++ async function ensure(subject, claims = {}, attempt = 0) {++ try { return await ensureOnce(subject, claims); } catch (e) {++ if (e.code === '23505' && e.constraint === HANDLE_TAKEN && attempt < 2) return ensure(subject, claims, attempt + 1);++ throw e;++ }++ }++ async function ensureOnce(subject, { username, displayName, avatarUrl } = {}) {const at = iso(ctx.now());-- const existing = bySubject(subject);++ const existing = await bySubject(db, subject);const name = String(displayName || username || 'Creator').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80) || 'Creator';if (!existing) {-- db.prepare(`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)-- VALUES (?, ?, ?, ?, ?, ?)`).run(subject, freeHandle(subject, username), name, safeUrl(avatarUrl), at, at);-- return bySubject(subject);++ const row = await db.maybe(sql`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)++ VALUES (${subject}, ${await freeHandle(db, subject, username)}, ${name}, ${safeUrl(avatarUrl)}, ${at}, ${at})++ ON CONFLICT (creator_subject) DO NOTHING RETURNING *`);++ return row ? parse(row) : bySubject(db, subject);}-- const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? freeHandle(subject, username) : existing.handle;++ const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? await freeHandle(db, subject, username) : existing.handle;if (handle !== existing.handle || name !== existing.display_name || (safeUrl(avatarUrl) || null) !== (existing.avatar_url || null)) {-- db.prepare('UPDATE creator_tip_profiles SET handle = ?, display_name = ?, avatar_url = ?, updated_at = ? WHERE creator_subject = ?')-- .run(handle, name, safeUrl(avatarUrl), at, subject);++ return parse(await db.one(sql`UPDATE creator_tip_profiles SET handle = ${handle}, display_name = ${name}, avatar_url = ${safeUrl(avatarUrl)}, updated_at = ${at}++ WHERE creator_subject = ${subject} RETURNING *`));}-- return bySubject(subject);++ return existing;}const LIMITS = {@@@ -92,13 +108,25 @@ function createProfiles(ctx) {media_request_min: [1, 1_000_000], media_max_seconds: [10, 3 * 3600],};const BOOLS = ['page_enabled', 'accepting', 'tts_enabled', 'media_requests_enabled'];++ const on = (v) => v === true || v === 'on' || v === '1' || v === 1;++ /**++ * A settings change, on the locked row (two changes at once apply one after the other, neither lost).++ * With expectedRevision it applies only to that revision (else 409).++ */function update(subject, patch = {}, { expectedRevision } = {}) {-- const p = bySubject(subject);-- if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');-- if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);++ return ctx.tx(async (t) => {++ const p = parse(await t.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(subject || '')} FOR UPDATE`));++ if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');++ if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);++ return change(t, p, patch);++ });++ }++++ async function change(t, p, patch) {++ const subject = p.creator_subject;const set = {};-- for (const k of BOOLS) if (patch[k] !== undefined) set[k] = patch[k] === true || patch[k] === 'on' || patch[k] === '1' || patch[k] === 1 ? 1 : 0;++ for (const k of BOOLS) if (patch[k] !== undefined) set[k] = on(patch[k]);for (const [k, [lo, hi]] of Object.entries(LIMITS)) {if (patch[k] === undefined || patch[k] === '') continue;const n = Number(patch[k]);@@@ -120,31 +148,29 @@ function createProfiles(ctx) {const unknown = Object.keys(patch.page).filter((k) => !(k in PAGE_DEFAULTS));if (unknown.length) fail(422, 'tips.invalid_input', `page has no setting ${unknown[0]}`);const next = { ...p.page };-- for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = patch.page[k] === true || patch.page[k] === 'on' || patch.page[k] === '1' || patch.page[k] === 1;-- set.page_settings = JSON.stringify(next);++ for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = on(patch.page[k]);++ set.page_settings = sql.json(next);}if (patch.filter !== undefined) {const f = patch.filter;if (!f || typeof f !== 'object' || Array.isArray(f)) fail(422, 'tips.invalid_input', 'filter must be an object { words, action, links }');const unknown = Object.keys(f).filter((k) => !['words', 'action', 'links'].includes(k));if (unknown.length) fail(422, 'tips.invalid_input', `filter has no setting ${unknown[0]}`);-- if (f.words !== undefined) set.filter_words = JSON.stringify(filterLib.normalizeWords(f.words));++ if (f.words !== undefined) set.filter_words = sql.json(filterLib.normalizeWords(f.words));if (f.action !== undefined) {if (!filterLib.ACTIONS.includes(f.action)) fail(422, 'tips.invalid_input', `filter.action must be one of ${filterLib.ACTIONS.join(', ')}`);set.filter_action = f.action;}-- if (f.links !== undefined) set.filter_links = f.links === true || f.links === 'on' || f.links === '1' || f.links === 1 ? 1 : 0;++ if (f.links !== undefined) set.filter_links = on(f.links);}if (patch.display_name !== undefined) {const n = text(patch.display_name, 'display_name', 80);if (!n) fail(422, 'tips.invalid_input', 'display_name cannot be empty');set.display_name = n.replace(/[<>]/g, '');}-- const keys = Object.keys(set);-- if (!keys.length) return p;-- db.prepare(`UPDATE creator_tip_profiles SET ${keys.map((k) => `${k} = @${k}`).join(', ')}, revision = revision + 1, updated_at = @at WHERE creator_subject = @s`)-- .run({ ...set, at: iso(ctx.now()), s: subject });-- return bySubject(subject);++ if (!Object.keys(set).length) return p;++ return parse(await t.one(sql`UPDATE creator_tip_profiles SET ${sql.set({ ...set, updated_at: iso(ctx.now()) })}, revision = revision + 1++ WHERE creator_subject = ${subject} RETURNING *`));}/** Public shape; `owner` adds the settings only the creator sees. */@@@ -165,13 +191,31 @@ function createProfiles(ctx) {}/** The creator's filter settings for publicView() (null when they have no profile). */-- function filterOf(subject) {-- const row = db.prepare('SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ?').get(subject);-- if (!row) return null;-- return { words: json(row.filter_words, []), action: row.filter_action === 'hold' ? 'hold' : 'mask', links: row.filter_links !== 0 };++ async function filterOf(q, subject) {++ const row = await q.maybe(sql`SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ${subject}`);++ return row ? filterOfRow(row) : null;++ }++ /** Several creators' profiles in one query: Map(subject → profile | null). */++ async function bySubjects(q, subjects) {++ const list = [...new Set(subjects)];++ const out = new Map(list.map((s) => [s, null]));++ if (!list.length) return out;++ for (const row of await q.many(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) out.set(row.creator_subject, parse(row));++ return out;++ }++++ /** Several creators' filters in one query: Map(subject → settings | null). */++ async function filtersOf(q, subjects) {++ const list = [...new Set(subjects)];++ const out = new Map(list.map((s) => [s, null]));++ if (!list.length) return out;++ for (const row of await q.many(sql`SELECT creator_subject, filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) {++ out.set(row.creator_subject, filterOfRow(row));++ }++ return out;}-- return { bySubject, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf };++ return { bySubject, bySubjects, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf, filtersOf };}function safeUrl(u) {@@@ -14,41 +14,41 @@* chat line, overlay alert, goal. Billing sends it only once it is the* money authority (while Live is, Live's webhook announces the tip)*-- * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same SQLite++ * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same* transaction as the change, so a redelivered event does nothing; and the Billing transaction id is* UNIQUE on tip_interactions, so the same transaction arriving under another event id (a replay, a-- * republish) still yields one logical interaction.++ * republish) still yields one logical interaction. That transaction moves money state, so it is++ * SERIALIZABLE (domain MONEY): two deliveries racing in two processes settle once.** The signature (X-OpenVibe-Signature, HMAC-SHA256 of the raw body with TIPS_EVENTS_SECRET) is* verified with openvibe-sdk's parseDelivery. Only events whose source is `billing` are applied.*/const express = require('express');const { http } = require('openvibe-contracts');--const { parseDelivery, createInbox } = require('openvibe-sdk/events');++const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');const CONSUMER = 'tips-billing';++const TABLE = 'tips_event_inbox'; // migrations/0001_initial.sql (inboxSchema)function consumerRouter({ domain, config, log = console }) {const router = express.Router();-- const inbox = createInbox(domain.db, { now: domain.now });-- inbox.ensureSchema();++ // The inbox's transaction is the domain's MONEY one (serializable, with after-commit hooks).++ const inbox = createPgInbox({ tx: (fn) => domain.tx(fn, domain.MONEY), maybe: (...a) => domain.db.maybe(...a) }, { table: TABLE, now: domain.now });/** Apply one envelope (also used by tests and the replay tool). Returns { duplicate, outcome }. */-- function apply(event) {-- return domain.tx(() => {-- const r = inbox.once(CONSUMER, event.event_id, () => {-- if (event.source !== 'billing') return 'ignored:source';-- const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};-- if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(payload);-- if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(payload);-- if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(payload);-- return 'ignored:type';-- });-- return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };++ async function apply(event) {++ const r = await inbox.once(CONSUMER, event.event_id, async (t) => {++ if (event.source !== 'billing') return 'ignored:source';++ const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};++ if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(t, payload);++ if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(t, payload);++ if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(t, payload);++ return 'ignored:type';});++ return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };}-- router.post('/events', express.raw({ type: () => true, limit: '256kb' }), (req, res) => {++ router.post('/events', express.raw({ type: () => true, limit: '256kb' }), async (req, res) => {const secrets = config.events.webhookSecrets;if (!secrets.length) return http.sendProblem(res, 503, 'tips.webhook_disabled', { detail: 'TIPS_EVENTS_SECRET is not set', ctx: req.ov });const raw = Buffer.isBuffer(req.body) ? req.body : Buffer.alloc(0);@@@ -62,7 +62,7 @@ function consumerRouter({ domain, config, log = console }) {}let out;try {-- out = apply(event);++ out = await apply(event);} catch (e) {// Not acknowledged: Events retries it, and the inbox claim rolled back with the change.log.error(`[Tips] event ${event.event_id} (${event.event_type}) failed:`, e.message);@@@ -10,16 +10,18 @@* tips.overlay.delivered an overlay delivery reached at least one overlay (first time only)* tips.overlay.failed no overlay received it within the delivery window*-- * enqueue() runs inside the SQLite transaction that makes the change, so an event exists if and-- * only if its change committed. Simulations (test) never produce events. The relay publishes with++ * emit(t, …) runs inside the transaction that makes the change (its handle `t`), so an event exists if++ * and only if its change committed. Simulations (test) never produce events. The relay publishes with* Tips' service token (events.event.publish, audience openvibe.events) only when EVENTS_URL and-- * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in event_outbox.++ * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in tips_event_outbox. Any number of processes++ * relay the one table: the SDK claims due rows with a lease (FOR UPDATE SKIP LOCKED).*/const { createClient } = require('openvibe-sdk/core');const { createServiceTokenClient } = require('openvibe-sdk/auth');--const { createEventsClient, createOutbox } = require('openvibe-sdk/events');++const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');const ACTOR = { type: 'service', id: 'tips' };++const TABLE = 'tips_event_outbox'; // migrations/0001_initial.sql (outboxSchema)function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {const enabled = !!(config.events.url && config.oauth.clientSecret);@@@ -35,8 +37,9 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {}const events = createEventsClient(createClient(clientOpts), { source: 'tips' });let lastError = null;-- const outbox = createOutbox(db, {++ const outbox = createPgOutbox(db, {events,++ table: TABLE,intervalMs: config.events.intervalMs,now,onError: (err) => {@@@ -45,11 +48,10 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {lastError = msg;},});-- outbox.ensureSchema();-- /** Inside the caller's transaction. */-- function emit(eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {-- return outbox.enqueue({ event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });++ /** Inside the caller's transaction `t`. */++ function emit(t, eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {++ return outbox.enqueue(t, { event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });}return {@@@ -59,8 +61,8 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {start() { if (enabled) outbox.start(); },stop: () => outbox.stop(),kick() { if (enabled) outbox.kick(); },-- status: () => ({ enabled, pending: outbox.pending(), rejected: outbox.rejected(), last_error: lastError }),++ status: async () => ({ enabled, pending: await outbox.pending(), rejected: await outbox.rejected(), last_error: lastError }),};}--module.exports = { createTipsOutbox };++module.exports = { createTipsOutbox, TABLE };@@@ -32,8 +32,13 @@* Idempotent: every row is keyed by legacy_source / migration_maps (live, table, id); a re-run changes* nothing, and a later snapshot only adds what is new or releases what was held. --dry-run does it all* inside a transaction that is rolled back.++ *++ * The snapshots are other services' SQLite files, read synchronously (better-sqlite3 handles passed in++ * by scripts/import-live.js); everything written goes to Tips' PostgreSQL in one SERIALIZABLE++ * transaction (it records settled money and applies refunds), whose report starts over on a retry.*/--const { iso, prefixedId, json } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { iso, prefixedId, json, storable } = require('../util');const liveTs = (s) => {if (!s) return null;@@@ -41,8 +46,9 @@ const liveTs = (s) => {const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);return Number.isFinite(t) ? new Date(t).toISOString() : null;};--const tableExists = (db, name) => !!db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);--const columns = (db, t) => (tableExists(db, t) ? db.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);++// The snapshot (SQLite, read-only) side.++const tableExists = (snap, name) => !!snap.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);++const columns = (snap, t) => (tableExists(snap, t) ? snap.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);class DryRun extends Error {}@@@ -79,15 +85,15 @@ function billingSnapshotSource(bdb) {}async function importLive(domain, { live, billingSource = null, resolveLiveUsers, dryRun = false, log = console }) {-- const { db } = domain;const started = domain.now();const runId = prefixedId('timp', started);const at = iso(started);const report = {run_id: runId, dry_run: !!dryRun, started_at: at, source: 'openvibe-live snapshot', billing_source: billingSource ? billingSource.name : null,-- counts: { imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 },-- holds: [], excluded: [], unlinked: [], reconciliation: null,++ counts: {}, holds: [], excluded: [], unlinked: [], reconciliation: null,};++ const fresh = () => ({ imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 });++ Object.assign(report.counts, fresh());// ── Read the snapshot ────────────────────────────────────const txns = tableExists(live, 'transactions') ? live.prepare("SELECT * FROM transactions WHERE type IN ('donation', 'refund') ORDER BY id").all() : [];@@@ -114,25 +120,36 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsersfor (const g of goalRows) if (g.user_id) ids.add(String(g.user_id));const map = ids.size ? await resolveLiveUsers([...ids]) : new Map();const subjectOf = (liveId) => { const v = liveId == null ? null : map.get(String(liveId)); return v ? (typeof v === 'string' ? v : v.subject) : null; };-- const nameOf = (liveId) => { const u = users.get(String(liveId)); return u ? (u.display_name || u.username) : null; };++ const nameOf = (liveId) => { const u = users.get(String(liveId)); return u && (u.display_name || u.username) ? storable(u.display_name || u.username) : null; };report.counts.identities = { resolved: [...ids].filter((i) => subjectOf(i)).length, unmapped: [...ids].filter((i) => !subjectOf(i)).length };-- const mapRow = db.prepare('SELECT * FROM migration_maps WHERE source = ? AND source_table = ? AND source_id = ?');-- const upsertMap = db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)-- VALUES ('live', @table, @id, @target_type, @target_id, @status, @reason, @billing, @run, @at, @at)-- ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,-- reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);-- const record = (table, id, fields) => upsertMap.run({ table, id: String(id), target_type: null, target_id: null, reason: null, billing: null, run: runId, at, ...fields });++ let q = null; // the import's transaction++ const mapRow = (_source, table, id) => q.maybe(sql`SELECT * FROM migration_maps WHERE source = 'live' AND source_table = ${table} AND source_id = ${id}`);++ const record = (table, id, fields) => {++ const f = { target_type: null, target_id: null, reason: null, billing: null, ...fields };++ return q.exec(sql`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)++ VALUES ('live', ${table}, ${String(id)}, ${f.target_type}, ${f.target_id}, ${f.status}, ${f.reason}, ${f.billing}, ${runId}, ${at}, ${at})++ ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,++ reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);++ };-- const run = () => {++ const run = async (t0) => {++ q = t0;++ // A retried attempt (serialization failure) reports from scratch.++ Object.assign(report.counts, fresh(), { live: report.counts.live, identities: report.counts.identities });++ report.holds = []; report.excluded = []; report.unlinked = [];// ── 2. donations ─────────────────────────────────────for (const t of txns.filter((x) => x.type === 'donation')) {-- const prev = mapRow.get('live', 'transactions', String(t.id));++ const prev = await mapRow('live', 'transactions', String(t.id));if (prev && prev.status !== 'held') {// Re-link to Billing when an earlier run had no Billing transaction for it yet.if (prev.status === 'imported' && !prev.billing_txn_id && billingSource) {const b = billingSource.linkLiveTxn(t.id);-- if (b) { db.prepare('UPDATE tip_interactions SET billing_txn_id = ? WHERE id = ? AND billing_txn_id IS NULL').run(b, prev.target_id); record('transactions', t.id, { ...pick(prev), billing: b }); report.counts.linked_to_billing++; }++ if (b) {++ await q.exec(sql`UPDATE tip_interactions SET billing_txn_id = ${b} WHERE id = ${prev.target_id} AND billing_txn_id IS NULL`);++ await record('transactions', t.id, { ...pick(prev), billing: b });++ report.counts.linked_to_billing++;++ }}report.counts.unchanged++;continue;@@@ -141,33 +158,33 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsersconst creator = subjectOf(t.to_user_id);if (t.status !== 'completed' || amount <= 0) {const reason = amount <= 0 ? 'non-positive amount' : `status ${t.status}: no balance moved on Live`;-- record('transactions', t.id, { status: 'excluded', reason });++ await record('transactions', t.id, { status: 'excluded', reason });report.counts.excluded++; report.excluded.push({ live_txn: t.id, reason });continue;}if (!creator) {const reason = `creator (Live user ${t.to_user_id}) has no Network subject yet`;-- record('transactions', t.id, { status: 'held', reason });++ await record('transactions', t.id, { status: 'held', reason });report.counts.held++; report.holds.push({ live_txn: t.id, live_creator: t.to_user_id, amount });continue;}const created = liveTs(t.created_at) || at;const test = !!(resetAt && created < resetAt);const billingTxn = billingSource ? billingSource.linkLiveTxn(t.id) : null;-- const message = t.message ? String(t.message).slice(0, 500) : null;++ const message = t.message ? storable(String(t.message).slice(0, 500)) : null;const media = /^Media request:/i.test(message || '');const site = /^PowerChat tip via site account/i.test(message || '');-- const i = domain.interactions.insert({++ const i = await domain.interactions.insert(q, {id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_subject: subjectOf(t.from_user_id),supporter_name: nameOf(t.from_user_id) || (site ? 'PowerChat supporter' : null), kind: media ? 'media_request' : 'tip', amount,message: media || site ? null : message, request: { legacy_message: message, live_stream_id: t.stream_id || null },funding: site ? 'provider' : 'credit', settlement: 'imported', payment_state: 'settled', delivery_state: 'delivered',-- test: test ? 1 : 0, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,++ test, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,origin: 'import', created_at: created,target: t.stream_id ? { service: 'live', type: 'stream', id: String(t.stream_id) } : null,});-- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);-- record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });++ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);++ await record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });if (prev) report.counts.released++;report.counts.imported++;if (billingTxn) report.counts.linked_to_billing++;@@@ -176,15 +193,17 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers// ── 3. refunds (streamer → donor) ────────────────────for (const t of txns.filter((x) => x.type === 'refund')) {-- const prev = mapRow.get('live', 'transactions', String(t.id));++ const prev = await mapRow('live', 'transactions', String(t.id));if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }const amount = Math.round(Number(t.amount) || 0);-- if (t.status !== 'completed' || amount <= 0) { record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }++ if (t.status !== 'completed' || amount <= 0) { await record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }const creator = subjectOf(t.from_user_id);const donor = subjectOf(t.to_user_id);const created = liveTs(t.created_at) || at;-- const candidates = creator ? db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND settlement = 'imported' AND legacy_source LIKE 'live:transactions:%'-- AND (supporter_subject IS ? ) AND created_at <= ? AND amount - reversed_bits > 0 ORDER BY created_at DESC`).all(creator, donor, created) : [];++ // The donations a refund may apply to, locked in id order (the domain's lock order).++ const candidates = creator ? (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND settlement = 'imported'++ AND starts_with(legacy_source, 'live:transactions:') AND supporter_subject IS NOT DISTINCT FROM ${donor}::text AND created_at <= ${created}++ AND amount - reversed_bits > 0 ORDER BY id FOR UPDATE`)).sort((a, b) => (a.created_at < b.created_at ? 1 : a.created_at > b.created_at ? -1 : 0)) : [];// Plan first; apply only when the whole refund can be placed.let left = amount;const plan = [];@@@ -195,17 +214,18 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsersleft -= take;}if (left > 0) {-- record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });++ await record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });report.counts.held++; report.holds.push({ live_txn: t.id, refund: true, amount });continue;}const applied = [];for (const { c, take } of plan) {-- const ids = json(c.reversal_txn_ids, []); ids.push(`live:transactions:${t.id}`);-- db.prepare("UPDATE tip_interactions SET reversed_bits = reversed_bits + ?, payment_state = 'reversed', reversed_at = ?, reversal_txn_ids = ? WHERE id = ?").run(take, created, JSON.stringify(ids), c.id);++ const ids = [...(c.reversal_txn_ids || []), `live:transactions:${t.id}`];++ await q.exec(sql`UPDATE tip_interactions SET reversed_bits = reversed_bits + ${take}, payment_state = 'reversed', reversed_at = ${created},++ reversal_txn_ids = ${sql.json(ids)} WHERE id = ${c.id}`);applied.push({ interaction: c.id, bits: take });}-- record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });++ await record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });report.counts.refunds_applied++;}@@@ -215,7 +235,7 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsersfor (const c of chatDonations) {const m = json(c.metadata, {});if (m.source !== 'powerchat' || m.kind !== 'donation') continue;-- const prev = mapRow.get('live', 'chat_messages', String(c.id));++ const prev = await mapRow('live', 'chat_messages', String(c.id));if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }const amount = Math.round(Number(m.amount) || 0);const created = liveTs(c.created_at) || at;@@@ -223,53 +243,55 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers&& Math.abs(Date.parse(liveTs(t.created_at) || at) - Date.parse(created)) <= 10 * 60 * 1000);if (twin) {usedSite.add(twin.id);-- record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });++ await record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });report.counts.excluded++;continue;}-- if (amount <= 0) { record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }++ if (amount <= 0) { await record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }const creator = subjectOf(c.channel_user_id);if (!creator) {-- record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });++ await record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });report.counts.held++; report.holds.push({ live_chat_message: c.id, live_creator: c.channel_user_id, amount });continue;}-- const i = domain.interactions.insert({-- id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? String(m.username).slice(0, 80) : 'PowerChat supporter',-- kind: 'tip', amount, amount_cents: amount, message: m.message ? String(m.message).slice(0, 500) : null, funding: 'external', settlement: 'external',-- payment_state: 'settled', delivery_state: 'delivered', test: resetAt && created < resetAt ? 1 : 0, provider: 'powerchat',++ const i = await domain.interactions.insert(q, {++ id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? storable(String(m.username).slice(0, 80)) : 'PowerChat supporter',++ kind: 'tip', amount, amount_cents: amount, message: m.message ? storable(String(m.message).slice(0, 500)) : null, funding: 'external', settlement: 'external',++ payment_state: 'settled', delivery_state: 'delivered', test: !!(resetAt && created < resetAt), provider: 'powerchat',provider_ref: `live-chat:${c.id}`, legacy_source: `live:chat_messages:${c.id}`, origin: 'import', created_at: created,});-- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);-- record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });++ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);++ await record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });report.counts.external++;}// ── 5. goals ─────────────────────────────────────────for (const g of goalRows) {-- const prev = mapRow.get('live', 'donation_goals', String(g.id));++ const prev = await mapRow('live', 'donation_goals', String(g.id));if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }const creator = subjectOf(g.user_id);-- if (!creator) { record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }++ if (!creator) { await record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }const target = Math.round(Number(g.target_amount) || 0);-- if (target <= 0 || !g.title) { record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }++ if (target <= 0 || !g.title) { await record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }const created = liveTs(g.created_at) || at;const id = prefixedId('tgoal', Date.parse(created));-- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)-- VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, String(g.title).slice(0, 120), target,-- /^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null, g.is_active ? 'active' : 'closed', Number(g.sort_order) || 0,-- liveTs(g.reached_at), g.is_active ? null : at, Math.max(0, Math.round(Number(g.current_amount) || 0)), `live:donation_goals:${g.id}`, created, at);-- record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });++ await q.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)++ VALUES (${id}, ${creator}, ${storable(String(g.title).slice(0, 120))}, ${target}, ${/^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null},++ ${g.is_active ? 'active' : 'closed'}, ${Number(g.sort_order) || 0}, ${liveTs(g.reached_at)}, ${g.is_active ? null : at},++ ${Math.max(0, Math.round(Number(g.current_amount) || 0))}, ${`live:donation_goals:${g.id}`}, ${created}, ${at})`);++ await record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });report.counts.goals++;}-- if (dryRun) { report.reconciliation = reconcile(domain, billingSource); throw new DryRun(); }++ if (dryRun) { report.reconciliation = await reconcile(domain, billingSource, q); throw new DryRun(); }};-- try { db.transaction(run)(); } catch (e) { if (!(e instanceof DryRun)) throw e; }-- if (!dryRun) report.reconciliation = reconcile(domain, billingSource);++ // Serializable while the service writes too: a conflict retries the whole run (idempotent), up to 10 times.++ try { await domain.tx(run, { ...domain.MONEY, retries: 10 }); } catch (e) { if (!(e instanceof DryRun)) throw e; }++ if (!dryRun) report.reconciliation = await reconcile(domain, billingSource);report.finished_at = iso(domain.now());-- db.prepare('INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report) VALUES (?, ?, ?, ?, ?, ?)').run(runId, 'live', dryRun ? 1 : 0, report.started_at, report.finished_at, JSON.stringify(report));++ await domain.db.exec(sql`INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report)++ VALUES (${runId}, 'live', ${!!dryRun}, ${report.started_at}, ${report.finished_at}, ${sql.json(report)})`);if (log && log.log) log.log(`[Tips] import ${runId}${dryRun ? ' (dry run, rolled back)' : ''}: ${report.counts.imported} tips, ${report.counts.external} external, ${report.counts.goals} goals, ${report.counts.held} held, ${report.counts.excluded} excluded; reconciliation ${report.reconciliation ? (report.reconciliation.ok ? 'OK' : 'MISMATCH') : 'skipped (no Billing source)'}`);return report;}@@@ -277,11 +299,11 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsersfunction pick(row) { return { status: row.status, target_type: row.target_type, target_id: row.target_id, reason: row.reason }; }/** Per-creator totals: Tips (settled via Billing or imported, non-test, minus refunds) vs Billing. */--function reconcile(domain, billingSource) {++async function reconcile(domain, billingSource, q = domain.db) {if (!billingSource) return null;const billing = billingSource.creatorTotals();-- const tips = new Map(domain.db.prepare(`SELECT creator_subject AS s, SUM(amount - reversed_bits) AS n FROM tip_interactions-- WHERE test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`).all().map((r) => [r.s, r.n]));++ const tips = new Map((await q.many(sql`SELECT creator_subject AS s, SUM(amount - reversed_bits)::bigint AS n FROM tip_interactions++ WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`)).map((r) => [r.s, r.n]));const creators = [...new Set([...billing.keys(), ...tips.keys()])].sort();const rows = creators.map((s) => ({ creator: s, tips: tips.get(s) || 0, billing: billing.get(s) || 0, difference: (tips.get(s) || 0) - (billing.get(s) || 0) }));const mismatches = rows.filter((r) => r.difference !== 0);@@@ -4,44 +4,73 @@* OpenVibe.Tips — process entry. `node server/index.js`* Listens on PORT (4610) behind nginx (openvibe.tips); see deploy/.*-- * Background jobs (TIPS_JOBS=off disables them): the effects worker (chat delivery with retries),-- * due Billing transfers (funded checkouts, retries after Billing was unreachable), the overlay-- * delivery window (pending → failed), stored API answers older than a week, and the events outbox-- * relay when EVENTS_URL is set.++ * Boot (ADR-035): apply migrations/ with the owner role (DATABASE_DIRECT_URL, one direct connection,++ * closed afterwards), then serve on the pooled runtime role (DATABASE_URL, through PgBouncer) with++ * Valkey (VALKEY_URL) for what the processes share. Several processes may start together: the++ * migration run is serialised by an advisory lock.++ *++ * Background jobs (TIPS_JOBS=off disables them), in every process: the effects worker (chat delivery++ * with retries), due Billing transfers (funded checkouts, retries after Billing was unreachable), the++ * overlay delivery window (pending → failed), stored API answers and sent events older than a week,++ * and the events outbox relay when EVENTS_URL is set. Running them in several processes is safe: the++ * effects, the transfers, the sweep and the relay claim their rows (FOR UPDATE SKIP LOCKED, with a++ * lease), and the prunes are idempotent deletes.*/const { loadConfig } = require('./config');++const { openDb, migrate } = require('./db');const { createApp } = require('./app');++const { createValkey } = require('openvibe-sdk/valkey');++const { createRegistry } = require('openvibe-shared/metrics');--const config = loadConfig();--const app = createApp({ config });--const { domain, keys, outbox } = app.locals;--keys.start();++async function main() {++ const config = loadConfig();++ const registry = createRegistry();++ const db = openDb(config, { registry });++ const m = await migrate(config, { serving: db });++ if (m.held.length) console.warn(`[Tips] migrations held: ${m.held.map((h) => `${h.id} (${h.reason})`).join('; ')}`);++ const valkey = createValkey({ url: config.valkey.url, prefix: config.valkey.prefix });--const timers = [];--if (config.jobs.enabled) {-- const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };-- every(config.jobs.intervalMs, () => domain.effects.drain());-- every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());-- every(30_000, () => domain.overlays.sweepFailed());-- every(6 * 3600 * 1000, () => outbox.outbox.prune());-- every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(domain.db));-- outbox.start();--}++ const app = createApp({ config, db, valkey, registry });++ const { domain, keys, outbox } = app.locals;++ keys.start();--const server = app.listen(config.port, config.host, () => {-- console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (db ${config.dbPath})`);-- console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);--});--server.keepAliveTimeout = 65_000;++ const timers = [];++ if (config.jobs.enabled) {++ const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };++ every(config.jobs.intervalMs, () => domain.effects.drain());++ every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());++ every(30_000, () => domain.overlays.sweepFailed());++ every(6 * 3600 * 1000, () => outbox.outbox.prune());++ every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(db));++ outbox.start();++ }--function shutdown(signal) {-- console.log(`[Tips] ${signal} — closing`);-- timers.forEach(clearInterval);-- domain.overlays.closeAll();-- outbox.stop();-- keys.stop();-- server.close(() => { try { domain.db.close(); } catch { /* */ } process.exit(0); });-- setTimeout(() => process.exit(0), 5000).unref();++ const server = app.listen(config.port, config.host, () => {++ console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (store ${db.store}; valkey ${valkey ? 'on' : 'off: per process'})`);++ console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);++ });++ server.keepAliveTimeout = 65_000;++++ let closing = false;++ async function shutdown(signal) {++ if (closing) return;++ closing = true;++ console.log(`[Tips] ${signal} — closing`);++ setTimeout(() => process.exit(0), 5000).unref();++ timers.forEach(clearInterval);++ keys.stop();++ await domain.overlays.close().catch(() => {});++ await outbox.stop();++ await new Promise((resolve) => server.close(resolve));++ await db.close().catch(() => {});++ if (valkey) await valkey.close().catch(() => {});++ process.exit(0);++ }++ process.on('SIGTERM', () => { shutdown('SIGTERM'); });++ process.on('SIGINT', () => { shutdown('SIGINT'); });}--process.on('SIGTERM', () => shutdown('SIGTERM'));--process.on('SIGINT', () => shutdown('SIGINT'));++++main().catch((e) => {++ console.error(`[Tips] could not start: ${e.message}`);++ process.exit(1);++});@@@ -3,7 +3,12 @@* Track O: truthful readiness for GET /api/ready and the Tips gauges on GET /metrics* (openvibe-shared/ready and openvibe-shared/metrics).*-- * db required a real query on Tips' SQLite (the settings row is there and answers)++ * db required a real round trip to PostgreSQL through the pool (db.ready(): the store++ * that answered — postgresql, or pglite in development and tests — and the++ * pool's state)++ * valkey optional a real PING (valkey.ready()). Without it (VALKEY_URL unset: skipped, or++ * down: degraded) per-actor limits, overlay fan-out and stream slots are++ * counted in each process, which is right for one process only* network_jwks optional the Network signing key has loaded. Without it pages, overlays and the* signed Billing webhook still work, but no service or user token can be* verified (API calls and sign-in fail), so it degrades rather than fails@@@ -13,12 +18,16 @@* EVENTS_URL or the client secret unset) tips.* events wait in the outbox** Gauges: pending deliveries (chat/TTS/media effects waiting for their adapter, overlay alerts no-- * overlay has shown yet) and the events outbox backlog. Counts only, never subjects.++ * overlay has shown yet) and the events outbox backlog. Counts only, never subjects. They are read in++ * one query just before a direct scrape renders (refresh()); a scrape whose read failed leaves them++ * out rather than reporting stale or invented values.*/--const { createReadiness } = require('openvibe-shared/ready');++const { sql } = require('openvibe-sdk/db');++const { createReadiness, skip } = require('openvibe-shared/ready');++const { TABLE: OUTBOX } = require('./events/outbox');const PING_TTL_MS = 15_000;--// interaction_effects.effect (the CHECK constraint in db.js): each reported, 0 when none waits.++// interaction_effects.effect (the CHECK constraint in migrations/0001_initial.sql): each reported, 0 when none waits.const EFFECTS = ['chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert'];function probe(url, fetchImpl) {@@@ -29,7 +38,7 @@ function probe(url, fetchImpl) {};}--function createTipsReadiness({ db, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {++function createTipsReadiness({ db, valkey = null, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {const events = outbox.enabled? probe(`${config.events.url}/api/health`, fetchImpl): () => 'events relay off (EVENTS_URL or OV_OAUTH_CLIENT_SECRET unset): tips.* events wait in the outbox';@@@ -37,45 +46,56 @@ function createTipsReadiness({ db, keys, config, outbox, release = null, fetchImservice: 'tips',release,checks: [-- { name: 'db', required: true, check: () => (db.prepare('SELECT id FROM settings WHERE id = 1').get() ? true : 'database has no settings row') },++ { name: 'db', required: true, check: () => db.ready() },++ { name: 'valkey', required: false, check: () => (valkey ? valkey.ready() : skip('VALKEY_URL unset: limits, overlay fan-out and stream slots are per process (one process only)')) },{ name: 'network_jwks', required: false, check: () => (keys.get() ? true : 'Network signing key not loaded yet: tokens cannot be verified') },{ name: 'billing', required: false, cacheMs: PING_TTL_MS, timeoutMs: 2500, check: probe(`${config.billing.url}/api/health`, fetchImpl) },{ name: 'events', required: false, cacheMs: outbox.enabled ? PING_TTL_MS : 0, timeoutMs: 2500, check: events },],-- details: (body) => ({-- events_outbox: body.checks.db.status === 'ok' ? outbox.status() : null,-- billing_events_accepted: config.events.webhookSecrets.length > 0,-- pending_deliveries: body.checks.db.status === 'ok' ? pendingDeliveries(db) : null,-- }),++ details: async (body) => (body.checks.db.status === 'ok'++ ? { events_outbox: await outbox.status(), billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: await pendingDeliveries(db) }++ : { events_outbox: null, billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: null }),});}--function pendingDeliveries(db) {-- const effects = {};-- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) effects[r.effect] = r.n;-- const overlay = db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n;-- return { effects, overlay };++async function pendingDeliveries(db) {++ const r = await db.one(sql`SELECT++ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,++ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay`);++ return { effects: r.effects, overlay: r.overlay };}--/** Tips gauges on the openvibe-shared/metrics registry. */--function registerTipsGauges(registry, { db, outbox, now = () => Date.now() }) {++/** Tips gauges on the openvibe-shared/metrics registry; refresh() reads them all in one query. */++function registerTipsGauges(registry, { db, now = () => Date.now(), log = console }) {++ let snap = null;++ const read = (f) => () => (snap ? f(snap) : undefined);registry.gauge({name: 'tips_effects_pending', help: 'Chat, TTS, media and alert effects waiting for their delivery adapter, by effect', labelNames: ['effect'],-- collect: () => {-- const n = Object.fromEntries(EFFECTS.map((e) => [e, 0]));-- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) n[r.effect] = r.n;-- return Object.entries(n).map(([effect, value]) => ({ labels: { effect }, value }));-- },++ collect: read((s) => EFFECTS.map((effect) => ({ labels: { effect }, value: Number(s.effects[effect] || 0) }))),});registry.gauge({name: 'tips_effects_due', help: 'Queued effects whose next attempt is due now (a growing number means the worker is behind)',-- collect: () => db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ?").get(now()).n,-- });-- registry.gauge({-- name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet',-- collect: () => db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n,++ collect: read((s) => s.due),});-- registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: () => outbox.status().pending });++ registry.gauge({ name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet', collect: read((s) => s.overlay) });++ registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: read((s) => s.outbox) });++ return {++ async refresh() {++ try {++ snap = await Promise.race([++ db.one(sql`SELECT++ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,++ (SELECT count(*) FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now()}) AS due,++ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay,++ (SELECT count(*) FROM ${sql.ident(OUTBOX)} WHERE sent_at IS NULL AND rejected_at IS NULL) AS outbox`),++ new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000).unref()),++ ]);++ } catch (e) {++ snap = null;++ log.warn(`[Tips] gauges not read: ${e.message}`);++ }++ },++ };}module.exports = { createTipsReadiness, registerTipsGauges };@@@ -19,6 +19,7 @@ function fail(status, code, detail, extra) { throw new TipsError(status, code, dconst prefixedId = (prefix, ms = Date.now()) => `${prefix}_${ids.ulid(ms)}`;const iso = (ms) => new Date(ms).toISOString();const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');++/** JSON text (a SQLite snapshot's column) → value, or `d`. PostgreSQL jsonb columns come back parsed. */const json = (v, d) => { if (v == null || v === '') return d; try { return JSON.parse(v); } catch { return d; } };function positiveInt(v, field, max) {@@@ -32,10 +33,16 @@ function positiveInt(v, field, max) {// from the creator's filter and turn text around on stream (docs/threat-review.md).const INVISIBLE = /[\u00AD\u200B-\u200F\u202A-\u202E\u2060-\u2064\u2066-\u2069\uFEFF]/g;--/** Optional trimmed text: null when empty, refused when longer than max. Invisible characters are dropped. */++// PostgreSQL stores no NUL character in text or jsonb, and jsonb refuses an unpaired surrogate (SQLite++// took both): every text from outside goes through storable() before it reaches the database.++const NUL = /\u0000/g;++/** A string PostgreSQL can store: NUL characters dropped, unpaired surrogates replaced by U+FFFD. */++const storable = (v) => String(v).toWellFormed().replace(NUL, '');++++/** Optional trimmed text: null when empty, refused when longer than max. Invisible and NUL characters are dropped. */function text(v, field, max) {if (v == null) return null;-- const s = String(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();++ const s = storable(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();if (!s) return null;if (s.length > max) fail(422, 'tips.text_too_long', `${field} must be at most ${max} characters`);return s;@@@ -53,14 +60,36 @@ function userSubject(v, field = 'subject') {function entityRef(v, field = 'target') {if (v == null) return null;if (!validate('common.entity-ref@1', v).valid) fail(422, 'tips.invalid_input', `${field} must be an EntityRef ({ service, type, id })`);-- return { service: v.service, type: v.type, id: String(v.id) };++ return { service: v.service, type: v.type, id: storable(v.id) };}/** The display name a supporter chose, kept short, without markup characters. */function displayName(v) {if (v == null) return null;-- const s = String(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80);++ const s = storable(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80).toWellFormed();return s || null;}--module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE };++/** A keyset cursor: base64url JSON [created_at ISO, id] → the pair, or 422; cursorOf(row) makes one. */++const ISO_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,6})?Z$/;++function readCursor(cursor) {++ let cur = null;++ try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { /* refused below */ }++ if (!Array.isArray(cur) || typeof cur[0] !== 'string' || !ISO_RE.test(cur[0]) || typeof cur[1] !== 'string' || cur[1].length > 64) fail(422, 'tips.invalid_input', 'bad cursor');++ return cur;++}++const cursorOf = (row) => Buffer.from(JSON.stringify([row.created_at, row.id])).toString('base64url');++++/**++ * A database refusal the caller caused (text PostgreSQL cannot store: a NUL character, a malformed++ * sequence) → the TipsError to answer with, else null (a real failure: 500).++ */++function inputError(e) {++ const code = e && (e.code || (e.cause && e.cause.code));++ // 22021/22P05: a character the encoding or jsonb refuses; 22P02, 22007, 22008: a value that is not++ // what its column takes (an unreadable number, JSON text or time).++ return ['22021', '22P05', '22P02', '22007', '22008'].includes(code)++ ? new TipsError(422, 'tips.invalid_input', 'the request carries a value that cannot be stored (such as a NUL character)') : null;++}++++module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, storable, userSubject, entityRef, displayName, inputError, readCursor, cursorOf, INVISIBLE };nnew file mode 100644@@@ -0,0 +1,31 @@++'use strict';++++/**++ * Express 4 ignores a rejected promise, so an async handler's error would never reach the error++ * handler (the request would hang). A router made here wraps every handler it is given: a returned++ * promise that rejects is passed to next(err). Error handlers (four arguments) are left as they are.++ *++ * const r = asyncRouter(express.Router());++ * r.get('/x', async (req, res) => { res.json(await something()); });++ */++const METHODS = ['all', 'get', 'post', 'put', 'patch', 'delete', 'head', 'options', 'use'];++++function catching(fn) {++ if (typeof fn !== 'function' || fn.length >= 4) return fn;++ return function asyncHandler(req, res, next) {++ let out;++ try { out = fn(req, res, next); } catch (e) { return next(e); }++ if (out && typeof out.then === 'function') out.then(undefined, next);++ return undefined;++ };++}++++function asyncRouter(router) {++ for (const m of METHODS) {++ const orig = router[m].bind(router);++ router[m] = (...args) => orig(...args.map((a) => (Array.isArray(a) ? a.map(catching) : catching(a))));++ }++ return router;++}++++module.exports = { asyncRouter, catching };@@@ -25,8 +25,10 @@ const crypto = require('crypto');const ovServe = require('openvibe-shared/serve');const frame = require('openvibe-shared/frame');const express = require('express');--const { TipsError, json: parseJson } = require('../util');++const { sql } = require('openvibe-sdk/db');++const { TipsError } = require('../util');const { viewerMiddleware } = require('./session');++const { asyncRouter } = require('./async-router');const pages = require('./pages');const { esc, asset } = require('./layout');const { PAGE_DEFAULTS } = require('../domain/profiles');@@@ -37,7 +39,8 @@ const MOD_FLASH = new Set(['Hidden', 'Shown', 'You are now a moderator']);const RECEIPT_FLASH = new Set(['Your data was erased from your tips']);function createWebRoutes({ domain, config, layout, userAuth }) {-- const r = express.Router();++ // Every handler below may be async: its errors reach the app's error handler (the 500 page).++ const r = asyncRouter(express.Router());const { profiles, goals, interactions, overlays, moderation, db } = domain;const withViewer = viewerMiddleware(userAuth);const form = express.urlencoded({ extended: false, limit: '32kb' });@@@ -64,62 +67,66 @@ function createWebRoutes({ domain, config, layout, userAuth }) {const bool = (v) => v === '1' || v === 'on' || v === true;// ── Home, robots, sitemap ────────────────────────────────-- r.get('/', withViewer, (req, res) => {-- const creators = db.prepare(`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY updated_at DESC LIMIT 24`).all();++ r.get('/', withViewer, async (req, res) => {++ const creators = await db.many(sql`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`);html(res, pageFor(req, { active: 'home', canonicalPath: '/', body: pages.home({ creators }) }));});// What shipped on OpenVibe.Tips: the shared update log every OpenVibe site has.r.get('/updates', withViewer, (req, res) => html(res, pageFor(req, { canonicalPath: '/updates', title: 'What shipped on OpenVibe.Tips', body: frame.updatesBody({ service: 'tips', siteName: 'OpenVibe.Tips' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` })));r.get('/robots.txt', (req, res) => res.type('text/plain').set('Cache-Control', 'public, max-age=3600').send(`User-agent: *\nAllow: /\nDisallow: /dashboard\nDisallow: /receipts\nDisallow: /overlay/\nDisallow: /moderate/\nDisallow: /auth/\nDisallow: /api/\nDisallow: /internal/\nSitemap: ${config.baseUrl}/sitemap.xml\n`));-- r.get('/sitemap.xml', (req, res) => {-- const rows = db.prepare('SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY handle').all();++ r.get('/sitemap.xml', async (req, res) => {++ const rows = await db.many(sql`SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled ORDER BY handle`);const urls = [`<url><loc>${esc(config.baseUrl)}/</loc></url>`, ...rows.map((p) => `<url><loc>${esc(`${config.baseUrl}/${p.handle}`)}</loc><lastmod>${esc(p.updated_at.slice(0, 10))}</lastmod></url>`)];res.type('application/xml').set('Cache-Control', 'public, max-age=900').send(`<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls.join('')}</urlset>\n`);});// ── Receipts ─────────────────────────────────────────────-- const decorate = (i) => {-- const p = profiles.bySubject(i.creator_subject);-- return { ...interactions.present(i, { viewer: 'supporter' }), creator_handle: p && p.handle, creator_name: p && p.display_name };-- };-- r.get('/receipts', withViewer, (req, res) => {++ /** The supporter's view of each receipt, with its creator's handle and name (one query for all creators). */++ async function decorate(rows) {++ const [views, byCreator] = await Promise.all([++ interactions.presentMany(db, rows, { viewer: 'supporter' }),++ profiles.bySubjects(db, rows.map((i) => i.creator_subject)),++ ]);++ return views.map((v, k) => { const p = byCreator.get(rows[k].creator_subject); return { ...v, creator_handle: p && p.handle, creator_name: p && p.display_name }; });++ }++ r.get('/receipts', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);-- const out = interactions.list({ supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });++ const out = await interactions.list(db, { supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });const flash = RECEIPT_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;-- html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: out.rows.map(decorate), next: out.next_cursor, flash }) }));++ return html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: await decorate(out.rows), next: out.next_cursor, flash }) }));});// The supporter's own data: a download, and erasure (privacy.js, interactions.erase()).-- r.get('/receipts/export', withViewer, (req, res) => {++ r.get('/receipts/export', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);-- const day = new Date(domain.now()).toISOString().slice(0, 10);-- res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${day}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })-- .type('application/json').send(`${JSON.stringify(interactions.exportFor(req.viewer.subject), null, 2)}\n`);++ const today = new Date(domain.now()).toISOString().slice(0, 10);++ return res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${today}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })++ .type('application/json').send(`${JSON.stringify(await interactions.exportFor(req.viewer.subject), null, 2)}\n`);});-- function renderErase(req, res, status = 200) {-- const rows = db.prepare("SELECT payment_state FROM tip_interactions WHERE supporter_subject = ?").all(req.viewer.subject);++ async function renderErase(req, res, status = 200) {++ const c = await db.one(sql`SELECT count(*) AS n, count(*) FILTER (WHERE payment_state = 'pending') AS pending FROM tip_interactions WHERE supporter_subject = ${req.viewer.subject}`);html(res, pageFor(req, {title: 'Erase your data', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts/erase',-- body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: rows.length, pending: rows.filter((x) => x.payment_state === 'pending').length }),++ body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: c.n, pending: c.pending }),}), status);}-- r.get('/receipts/erase', withViewer, (req, res) => {++ r.get('/receipts/erase', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts/erase')}`);-- renderErase(req, res);++ return renderErase(req, res);});-- r.post('/receipts/erase', withViewer, form, (req, res) => {++ r.post('/receipts/erase', withViewer, form, async (req, res) => {if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent('/receipts/erase')}`);if (!csrfOk(req) || !bool(req.body.confirm)) return renderErase(req, res, 403);-- interactions.erase(req.viewer.subject);++ await interactions.erase(req.viewer.subject);return res.redirect(303, `/receipts?done=${encodeURIComponent('Your data was erased from your tips')}`);});-- r.get('/receipts/:id', withViewer, (req, res) => {++ r.get('/receipts/:id', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);-- const i = interactions.get(req.params.id);++ const i = await interactions.get(db, req.params.id);const as = i && (i.supporter_subject === req.viewer.subject ? 'supporter' : i.creator_subject === req.viewer.subject ? 'creator' : null);if (!i || !as) return notFound(req, res);-- const view = interactions.present(i, { viewer: as === 'creator' ? 'owner' : 'supporter' });-- html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile: profiles.bySubject(i.creator_subject), as, cancelled: req.query.cancelled === '1' }) }));++ const [view, profile] = await Promise.all([interactions.present(db, i, { viewer: as === 'creator' ? 'owner' : 'supporter' }), profiles.bySubject(db, i.creator_subject)]);++ return html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile, as, cancelled: req.query.cancelled === '1' }) }));});// ── Dashboard ────────────────────────────────────────────@@@ -128,42 +135,45 @@ function createWebRoutes({ domain, config, layout, userAuth }) {const claims = req.viewerClaims || {};return profiles.ensure(v.subject, { username: v.username, displayName: v.name, avatarUrl: claims.avatar_url });}-- function ensureAlertsConfig(subject) {-- if (!overlays.listConfigs(subject).some((c) => c.kind === 'alerts')) overlays.createConfig(subject, { kind: 'alerts', name: 'Alerts' });-- }-- function renderDashboard(req, res, { flash, error, status = 200 } = {}) {-- const p = ensureProfile(req);-- ensureAlertsConfig(p.creator_subject);-- const recent = interactions.list({ creator: p.creator_subject, limit: 20 }).rows.map((i) => interactions.present(i, { viewer: 'owner' }));++ async function renderDashboard(req, res, { flash, error, status = 200 } = {}) {++ const p = await ensureProfile(req);++ let configs = await overlays.listConfigs(db, p.creator_subject);++ if (!configs.some((c) => c.kind === 'alerts')) {++ await overlays.createConfig(p.creator_subject, { kind: 'alerts', name: 'Alerts' });++ configs = await overlays.listConfigs(db, p.creator_subject);++ }++ const s = p.creator_subject;++ const [recentRows, goalRows, tokens, totals, deliveries, connected, held, moderators, invites, log, moderating] = await Promise.all([++ interactions.list(db, { creator: s, limit: 20 }).then((out) => interactions.presentMany(db, out.rows, { viewer: 'owner' })),++ goals.list(db, s).then((list) => goals.presentMany(db, list)),++ overlays.listTokens(db, s), interactions.totals(db, s), overlays.recentDeliveries(db, s, 20), overlays.connected(s),++ moderation.heldCount(db, s), moderation.listModerators(db, s), moderation.openInvites(db, s), moderation.log(db, s, 20), moderation.moderatedBy(db, req.viewer.subject),++ ]);html(res, pageFor(req, {title: 'Dashboard', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard',body: pages.dashboard({-- profile: p, goals: goals.list(p.creator_subject).map((g) => goals.present(g)), tokens: overlays.listTokens(p.creator_subject).map(overlays.presentToken),-- configs: overlays.listConfigs(p.creator_subject).map(overlays.presentConfig), totals: interactions.totals(p.creator_subject), recent,-- deliveries: overlays.recentDeliveries(p.creator_subject, 20), csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,-- connected: overlays.connected(p.creator_subject), chatAdapter: config.chat.adapter,-- mod: {-- held: db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND moderation = 'held' AND payment_state = 'settled'").get(p.creator_subject).n,-- moderators: moderation.listModerators(p.creator_subject).map(moderation.presentModerator), invites: moderation.openInvites(p.creator_subject),-- log: moderation.log(p.creator_subject, 20), moderating: moderation.moderatedBy(req.viewer.subject),-- },++ profile: p, goals: goalRows, tokens: tokens.map(overlays.presentToken),++ configs: configs.map(overlays.presentConfig), totals, recent: recentRows,++ deliveries, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,++ connected, chatAdapter: config.chat.adapter,++ mod: { held, moderators: moderators.map(moderation.presentModerator), invites, log, moderating },}),}), status);}-- r.get('/dashboard', withViewer, (req, res) => {++ r.get('/dashboard', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);// Only our own confirmations are shown: a crafted ?done= link cannot put words on the page.-- renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });++ return renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });});/** A dashboard form action: sign-in + anti-forgery, errors re-render the dashboard. */function action(path, fn) {-- r.post(path, withViewer, form, (req, res) => {++ r.post(path, withViewer, form, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);if (!csrfOk(req)) return renderDashboard(req, res, { error: 'That form expired. Please try again.', status: 403 });try {-- const p = ensureProfile(req);-- const out = fn(req, p);++ const p = await ensureProfile(req);++ const out = await fn(req, p);if (out && out.render) return out.render(res);return res.redirect(303, `/dashboard?done=${encodeURIComponent((out && out.done) || 'Saved')}`);} catch (e) {@@@ -172,9 +182,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {}});}-- action('/dashboard/profile', (req, p) => {++ action('/dashboard/profile', async (req, p) => {const b = req.body;-- profiles.update(p.creator_subject, {++ await profiles.update(p.creator_subject, {page_enabled: bool(b.page_enabled), accepting: bool(b.accepting), tts_enabled: bool(b.tts_enabled), media_requests_enabled: bool(b.media_requests_enabled),headline: b.headline, min_amount: b.min_amount, paid_message_min: b.paid_message_min, tts_min_amount: b.tts_min_amount, tts_max_chars: b.tts_max_chars,tts_voice: b.tts_voice, media_request_min: b.media_request_min, media_max_seconds: b.media_max_seconds,@@@ -182,46 +192,46 @@ function createWebRoutes({ domain, config, layout, userAuth }) {}, { expectedRevision: b.revision });return { done: 'Page settings saved' };});-- action('/dashboard/goals', (req, p) => { goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });-- const ownGoal = (req, p) => {-- const g = goals.get(req.params.id);++ action('/dashboard/goals', async (req, p) => { await goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });++ const ownGoal = async (req, p) => {++ const g = await goals.get(db, req.params.id);if (!g || g.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.goal_not_found', 'no such goal');return g;};-- action('/dashboard/goals/:id', (req, p) => { goals.update(ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });-- action('/dashboard/goals/:id/close', (req, p) => { goals.close(ownGoal(req, p)); return { done: 'Goal closed' }; });-- action('/dashboard/overlay-tokens', (req, p) => {++ action('/dashboard/goals/:id', async (req, p) => { await goals.update(await ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });++ action('/dashboard/goals/:id/close', async (req, p) => { await goals.close(await ownGoal(req, p)); return { done: 'Goal closed' }; });++ action('/dashboard/overlay-tokens', async (req, p) => {const scopes = [bool(req.body.scope_alerts) && 'alerts', bool(req.body.scope_goals) && 'goals'].filter(Boolean);-- const alertCfg = overlays.listConfigs(p.creator_subject).find((c) => c.kind === 'alerts');-- const out = overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });++ const alertCfg = (await overlays.listConfigs(db, p.creator_subject)).find((c) => c.kind === 'alerts');++ const out = await overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });return { render: (res) => html(res, pageFor(req, { title: 'Overlay link', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.tokenCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };});-- action('/dashboard/overlay-tokens/:id/revoke', (req, p) => {-- const t = overlays.getToken(req.params.id);++ action('/dashboard/overlay-tokens/:id/revoke', async (req, p) => {++ const t = await overlays.getToken(db, req.params.id);if (!t || t.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_token_not_found', 'no such overlay token');-- overlays.revokeToken(t);++ await overlays.revokeToken(t);return { done: 'Overlay link revoked' };});-- action('/dashboard/overlay-configs/:id', (req, p) => {-- const c = overlays.getConfig(req.params.id);++ action('/dashboard/overlay-configs/:id', async (req, p) => {++ const c = await overlays.getConfig(db, req.params.id);if (!c || c.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_config_not_found', 'no such overlay config');const b = req.body;-- overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });++ await overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });return { done: 'Alert settings saved' };});-- action('/dashboard/filter', (req, p) => {-- profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });++ action('/dashboard/filter', async (req, p) => {++ await profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });return { done: 'Filter saved' };});-- action('/dashboard/moderators/:subject/remove', (req, p) => { moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });-- action('/dashboard/moderator-invites', (req, p) => {-- const out = moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });++ action('/dashboard/moderators/:subject/remove', async (req, p) => { await moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });++ action('/dashboard/moderator-invites', async (req, p) => {++ const out = await moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });return { render: (res) => html(res, pageFor(req, { title: 'Moderator invitation', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.inviteCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };});-- action('/dashboard/moderator-invites/:id/revoke', (req, p) => { moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });-- action('/dashboard/simulate', (req, p) => {++ action('/dashboard/moderator-invites/:id/revoke', async (req, p) => { await moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });++ action('/dashboard/simulate', async (req, p) => {const b = req.body;-- interactions.simulate(p, {++ await interactions.simulate(p, {kind: b.kind, amount: b.amount, message: b.message, supporter_name: b.supporter_name,tts: b.kind === 'tts' ? { text: b.message } : undefined, media: b.kind === 'media_request' ? { url: b.media_url } : undefined,}, { by: req.viewer.subject });@@@ -230,13 +240,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {// ── Overlays (token in the path; no cookies involved) ────const overlayHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };-- function tokenOr404(req, res) {-- const t = overlays.authenticate(req.params.token);++ async function tokenOr404(req, res) {++ const t = await overlays.authenticate(db, req.params.token);if (!t) { res.status(404).set(overlayHeaders).type('text/plain').send('This overlay link is not valid (it may have been revoked).'); return null; }return t;}-- r.get('/overlay/:token', (req, res) => {-- const t = tokenOr404(req, res);++ r.get('/overlay/:token', async (req, res) => {++ const t = await tokenOr404(req, res);if (!t) return;res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/html; charset=utf-8' }).send(`<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="robots" content="noindex,nofollow"><meta name="referrer" content="no-referrer">@@@ -246,59 +256,59 @@ function createWebRoutes({ domain, config, layout, userAuth }) {<noscript><p class="nojs">This overlay needs JavaScript (OBS Browser Sources have it on).</p></noscript><script src="${asset('js/overlay.js')}" defer></script></body></html>`);});-- r.get('/overlay/:token/events', (req, res) => {-- const t = tokenOr404(req, res);++ r.get('/overlay/:token/events', async (req, res) => {++ const t = await tokenOr404(req, res);if (!t) return;-- if (overlays.streamsFor(t) >= config.overlays.maxStreamsPerToken) {++ // A bounded number of open streams per token, counted across every process (overlay-hub.js).++ const slot = await overlays.reserveStream(t);++ if (!slot) {res.status(429).set({ ...overlayHeaders, 'Retry-After': '30' }).type('text/plain').send('Too many open streams for this overlay link.');return;}++ let overlayConfig = null;++ try { overlayConfig = t.config_id ? await overlays.getConfig(db, t.config_id) : null; } catch (e) { overlays.releaseStream(slot); throw e; }res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/event-stream; charset=utf-8', Connection: 'keep-alive', 'X-Accel-Buffering': 'no' });res.flushHeaders();res.write('retry: 3000\n\n');-- const overlayConfig = t.config_id ? overlays.getConfig(t.config_id) : null;const lastEventId = req.get('last-event-id') != null ? req.get('last-event-id') : req.query.last_event_id;-- overlays.attach(t, res, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });++ overlays.attach(t, res, slot, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });});-- r.get('/overlay/:token/state', (req, res) => {-- const t = tokenOr404(req, res);++ r.get('/overlay/:token/state', async (req, res) => {++ const t = await tokenOr404(req, res);if (!t) return;const out = { creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes };-- if (t.scopes.includes('goals')) out.goals = goals.list(t.creator_subject, { status: 'active' }).map((g) => goals.present(g));-- if (t.scopes.includes('alerts')) {-- out.alerts = db.prepare("SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ? AND kind = 'alert' AND hidden = 0 ORDER BY seq DESC LIMIT 20")-- .all(t.creator_subject).map((d) => ({ seq: d.seq, ...parseJson(d.payload, {}), test: !!d.test, created_at: d.created_at }));-- }++ if (t.scopes.includes('goals')) out.goals = await goals.presentMany(db, await goals.list(db, t.creator_subject, { status: 'active' }));++ if (t.scopes.includes('alerts')) out.alerts = await overlays.recentAlerts(db, t.creator_subject);res.set(overlayHeaders).json(out);});// ── Moderation (the creator and their moderators) ─────────// Invitation links carry a secret in the path, like overlay links: no-store, no referrer, noindex.const inviteHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };-- function inviteOr404(req, res) {-- const inv = moderation.findInvite(req.params.token);-- const profile = inv && profiles.bySubject(inv.creator_subject);++ async function inviteOr404(req, res) {++ const inv = await moderation.findInvite(db, req.params.token);++ const profile = inv && await profiles.bySubject(db, inv.creator_subject);if (!inv || !profile) {html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 404, title: 'This invitation is not valid', message: 'It may have been used, revoked or expired. Ask the creator for a new one.' }) }), 404, inviteHeaders);return null;}return { inv, profile };}-- r.get('/moderate/invite/:token', withViewer, (req, res) => {++ r.get('/moderate/invite/:token', withViewer, async (req, res) => {if (!req.viewer) return res.set(inviteHeaders).redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);-- const found = inviteOr404(req, res);++ const found = await inviteOr404(req, res);if (!found) return undefined;return html(res, pageFor(req, {title: `Moderate ${found.profile.display_name}`, robots: 'noindex,nofollow', canonicalPath: '/',body: pages.invitePage({ profile: found.profile, csrf: csrfFor(req.viewer.subject), idem: idem(), secret: req.params.token, isSelf: found.profile.creator_subject === req.viewer.subject }),}), 200, inviteHeaders);});-- r.post('/moderate/invite/:token', withViewer, form, (req, res) => {++ r.post('/moderate/invite/:token', withViewer, form, async (req, res) => {if (!req.viewer) return res.set(inviteHeaders).redirect(303, '/auth/login');if (!csrfOk(req)) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 403, title: 'That form expired', message: 'Open the invitation link again.' }) }), 403, inviteHeaders);try {-- const out = moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });-- const profile = profiles.bySubject(out.creator);++ const out = await moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });++ const profile = await profiles.bySubject(db, out.creator);return res.set(inviteHeaders).redirect(303, `/moderate/${encodeURIComponent(profile.handle)}?done=${encodeURIComponent('You are now a moderator')}`);} catch (e) {if (e instanceof TipsError) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: e.status, title: 'This invitation is not valid', message: e.detail || e.message }) }), e.status, inviteHeaders);@@@ -307,36 +317,35 @@ function createWebRoutes({ domain, config, layout, userAuth }) {});/** The creator or one of their moderators, for /moderate/:handle; else null (the page is a 404). */-- function modAccess(req) {-- const p = profiles.byHandle(req.params.handle);++ async function modAccess(req) {++ const p = await profiles.byHandle(db, req.params.handle);if (!p || !req.viewer) return { p };-- const actor = moderation.actorFor(req.viewer, p.creator_subject, false);-- return { p, actor };++ return { p, actor: await moderation.actorFor(db, req.viewer, p.creator_subject, false) };}-- function renderModerate(req, res, p, { error, status = 200 } = {}) {++ async function renderModerate(req, res, p, { error, status = 200 } = {}) {const state = ['held', 'hidden', 'visible', 'all'].includes(req.query.state) ? req.query.state : 'all';-- const out = moderation.queue(p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });++ const out = await moderation.queue(db, p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });const flash = MOD_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;html(res, pageFor(req, {title: `Paid messages — ${p.display_name}`, robots: 'noindex,nofollow', canonicalPath: `/moderate/${p.handle}`,body: pages.moderatePage({ profile: p, rows: out.rows, state, next: out.next_cursor, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error, isCreator: req.viewer.subject === p.creator_subject }),}), status);}-- r.get('/moderate/:handle', withViewer, (req, res) => {++ r.get('/moderate/:handle', withViewer, async (req, res) => {if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);-- const { p, actor } = modAccess(req);++ const { p, actor } = await modAccess(req);if (!p || !actor) return notFound(req, res);return renderModerate(req, res, p);});-- r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, (req, res) => {++ r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, async (req, res) => {if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/moderate/${req.params.handle}`)}`);-- const { p, actor } = modAccess(req);++ const { p, actor } = await modAccess(req);if (!p || !actor) return notFound(req, res);if (!csrfOk(req)) return renderModerate(req, res, p, { error: 'That form expired. Please try again.', status: 403 });-- const i = interactions.get(req.params.id);++ const i = await interactions.get(db, req.params.id);if (!i || i.creator_subject !== p.creator_subject) return notFound(req, res);try {-- moderation[req.params.act](i, { ...actor, reason: req.body.reason });++ await moderation[req.params.act](i, { ...actor, reason: req.body.reason });} catch (e) {if (e instanceof TipsError) return renderModerate(req, res, p, { error: e.detail || e.message, status: 400 });throw e;@@@ -346,13 +355,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {// ── Creator pages (last: /:handle catches everything else) ──const providers = config.billing.providers;-- function creatorView(req, res, { values, error, status = 200 } = {}) {-- const p = profiles.byHandle(req.params.handle);++ async function creatorView(req, res, { values, error, status = 200 } = {}) {++ const p = await profiles.byHandle(db, req.params.handle);const owner = !!(p && req.viewer && req.viewer.subject === p.creator_subject);if (!p || (!p.page_enabled && !owner)) return notFound(req, res);if (p.handle !== String(req.params.handle)) return res.redirect(301, `/${p.handle}`); // @name, Name → the canonical path// The page shows what the public sees, for the creator too (a preview); the dashboard has the rest.-- const list = goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page));++ const list = await goals.publicGoals(db, await goals.list(db, p.creator_subject), p);const csrf = req.viewer ? csrfFor(req.viewer.subject) : '';const jsonLd = p.page_enabled ? [{ '@context': 'https://schema.org', '@type': 'ProfilePage', name: `${p.display_name} on ${'OpenVibe.Tips'}`, url: `${config.baseUrl}/${p.handle}`, mainEntity: { '@type': 'Person', name: p.display_name, alternateName: p.handle, image: p.avatar_url || undefined } }] : [];return html(res, pageFor(req, {@@@ -363,33 +372,34 @@ function createWebRoutes({ domain, config, layout, userAuth }) {}), status, p.page_enabled ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });}r.get('/:handle', withViewer, (req, res, next) => (profiles.normalizeHandle(req.params.handle) ? creatorView(req, res) : next()));-- r.get('/:handle/goals', withViewer, (req, res, next) => {-- const p = profiles.byHandle(req.params.handle);++ r.get('/:handle/goals', withViewer, async (req, res, next) => {++ const p = await profiles.byHandle(db, req.params.handle);if (!p) return next();const owner = req.viewer && req.viewer.subject === p.creator_subject;if (!p.page_enabled && !owner) return notFound(req, res);-- html(res, pageFor(req, {++ return html(res, pageFor(req, {title: `${p.display_name} — goals`, canonicalPath: `/${p.handle}/goals`, robots: p.page_enabled ? 'index,follow' : 'noindex,nofollow',-- body: pages.goalsPage({ profile: p, goals: goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page)) }),++ body: pages.goalsPage({ profile: p, goals: await goals.publicGoals(db, await goals.list(db, p.creator_subject), p) }),}));});-- r.get('/:handle/supporters', withViewer, (req, res, next) => {-- const p = profiles.byHandle(req.params.handle);++ r.get('/:handle/supporters', withViewer, async (req, res, next) => {++ const p = await profiles.byHandle(db, req.params.handle);if (!p) return next();const owner = req.viewer && req.viewer.subject === p.creator_subject;// Off unless the creator shows it; the creator gets a preview while it is off.if ((!p.page_enabled || !p.page.supporters_page) && !owner) return notFound(req, res);-- const leaderboard = interactions.leaderboard(p.creator_subject).map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));-- const recent = p.page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;++ const [top, recentRows] = await Promise.all([interactions.leaderboard(db, p), p.page.supporters_messages ? interactions.recentPublic(db, p) : null]);++ const leaderboard = top.map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));++ const recent = recentRows ? recentRows.map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;const indexable = p.page_enabled && p.page.supporters_page;-- html(res, pageFor(req, {++ return html(res, pageFor(req, {title: `${p.display_name} — supporters`, canonicalPath: `/${p.handle}/supporters`, robots: indexable ? 'index,follow' : 'noindex,nofollow',body: pages.supportersPage({ profile: p, leaderboard, recent }),}), 200, indexable ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });});r.post('/:handle/tip', withViewer, form, async (req, res, next) => {-- const p = profiles.byHandle(req.params.handle);++ const p = await profiles.byHandle(db, req.params.handle);if (!p) return next();if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/${p.handle}`)}`);const b = req.body || {};@@@ -409,7 +419,7 @@ function createWebRoutes({ domain, config, layout, userAuth }) {privacy: { anonymous: values.anonymous, hide_amount: values.hide_amount, private_message: values.private_message },};const funding = b.pay_with === 'checkout' ? 'checkout' : 'credit';-- const { interaction, replay } = interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });++ const { interaction, replay } = await interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });let i = interaction;if (!replay) {if (funding === 'credit') {@@@ -17,6 +17,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(h.json.service, 'tips');const r = await t.call('GET', '/api/ready', { token: null });assert.strictEqual(r.json.ready, true);++ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered: pglite here, postgresql on the containers');const rel = await t.call('GET', '/release.json', { token: null });assert.strictEqual(rel.json.service, 'tips');assert.deepStrictEqual(require('openvibe-contracts').validate('registry.release-manifest@1', rel.json).errors, []);@@@ -107,6 +108,38 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(none.status, 400);});++ await check('an Idempotency-Key is claimed before the handler runs: once at a time, a stale claim taken over, refusals released', async () => {++ const goals = () => t.db.value('SELECT count(*) FROM tip_goals WHERE creator_subject = $1', [alex.subject]);++ const before = await goals();++ // Two at once: the handler runs once; the other answer is "in progress" or the replay.++ const body = { title: 'Once only', target_amount: 70 };++ const both = await Promise.all([1, 2].map(() => t.call('POST', '/api/v1/goals', { user: alex, body, key: 'goal-key-race-01' })));++ assert.ok(both.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(both.map((r) => r.status)));++ assert.strictEqual(new Set(both.filter((r) => r.status === 201).map((r) => r.json.goal.id)).size, 1);++ assert.strictEqual(await goals(), before + 1);++ // A claim still in progress (as another process holds it): 409 with Retry-After.++ const { stable } = require('../server/api/idempotency');++ const hashOf = (b) => require('crypto').createHash('sha256').update(`POST /api/v1/goals\n${stable(b)}`).digest('hex');++ const claim = (key, b, ageMs) => t.db.exec(`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)++ VALUES ($1, $2, 'POST', '/api/v1/goals', 0, '', $3)`, [`${alex.subject}:${key}`, hashOf(b), new Date(Date.now() - ageMs).toISOString()]);++ await claim('goal-key-busy-0001', { title: 'Busy', target_amount: 5 }, 1000);++ const busy = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Busy', target_amount: 5 }, key: 'goal-key-busy-0001' });++ assert.deepStrictEqual([busy.status, busy.json.code, busy.headers.get('retry-after')], [409, 'idempotency.in_progress', '1']);++ // A claim left by a process that died over a minute ago is taken over.++ await claim('goal-key-stale-001', { title: 'Stale', target_amount: 5 }, 120_000);++ const taken = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });++ assert.strictEqual(taken.status, 201, taken.text);++ const replay = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });++ assert.deepStrictEqual([replay.headers.get('idempotent-replayed'), replay.json.goal.id], ['true', taken.json.goal.id]);++ // A refusal is not stored: its claim is released, and a retry of the refused request runs again.++ const bad = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' });++ assert.strictEqual(bad.status, 422);++ await new Promise((ok) => setTimeout(ok, 50));++ assert.strictEqual(await t.db.value("SELECT count(*) FROM api_idempotency WHERE key LIKE '%goal-key-refused-1'"), 0, 'refusals are not stored');++ assert.strictEqual((await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' })).status, 422);++ for (const id of [taken.json.goal.id, both.find((r) => r.status === 201).json.goal.id]) await t.call('POST', `/api/v1/goals/${id}/close`, { user: alex });++ });++await check('receipts: supporters see their own, creators theirs, nobody else', async () => {const mine = await t.call('GET', '/api/v1/interactions', { user: viewer });assert.ok(mine.json.interactions.length >= 2);@@@ -158,7 +191,7 @@ const { boot, check, done } = require('./helpers/app');await check('totals: derived from settled interactions, never from a stored balance', async () => {const r = await t.call('GET', '/api/v1/profiles/alex/totals', { user: alex });-- const sum = t.db.prepare("SELECT SUM(amount - reversed_bits) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')").get(alex.subject).n;++ const sum = await t.db.value("SELECT SUM(amount - reversed_bits)::bigint FROM tip_interactions WHERE creator_subject = $1 AND NOT test AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')", [alex.subject]);assert.strictEqual(r.json.totals.settled_via_billing, sum);assert.strictEqual(r.json.totals.settled_via_billing, billing.payable.get(alex.subject), 'equals what Billing holds for the creator');assert.strictEqual((await t.call('GET', '/api/v1/profiles/alex/totals', { user: viewer })).status, 403);@@@ -32,15 +32,15 @@ const TYPES = ['tips.interaction.ready', 'tips.interaction.failed', 'tips.interaawait domain.effects.drain();live.state.fail = null;t.clock.offset += 11 * 60 * 1000;-- domain.overlays.sweepFailed();++ await domain.overlays.sweepFailed();t.clock.offset = 0;// A reversal before delivery (tips.interaction.cancelled) and one after it.const pending = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'read me' } } });-- const txn = domain.interactions.get(pending.json.interaction.id).billing_txn_id;++ const txn = (await domain.interactions.get(t.db, pending.json.interaction.id)).billing_txn_id;await t.deliver(billing.refund(txn));-- await t.deliver(billing.refund(domain.interactions.get(tip.json.interaction.id).billing_txn_id, 100));++ await t.deliver(billing.refund((await domain.interactions.get(t.db, tip.json.interaction.id)).billing_txn_id, 100));-- const rows = t.outboxRows();++ const rows = await t.outboxRows();for (const type of TYPES) assert.ok(rows.some((e) => e.event_type === type), `no ${type} was produced`);for (const e of rows) {const env = contracts.validate('events.event-envelope@1', e);@@@ -24,8 +24,8 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(d.job.test, false);const grant = t.network.grants.find((g) => g.audience === 'openvibe.live');assert.strictEqual(grant.scope, 'live.tips_delivery.write');-- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(r.json.interaction.id);-- assert.deepStrictEqual(JSON.parse(e.result).ref, { chat_message_id: 1 });++ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [r.json.interaction.id]);++ assert.deepStrictEqual(e.result.ref, { chat_message_id: 1 }, 'jsonb comes back parsed');});await check('TTS goes out as its own effect with the text and voice', async () => {@@@ -41,17 +41,17 @@ const { boot, check, done } = require('./helpers/app');const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 7 } });const id = r.json.interaction.id;await domain.effects.drain();-- let e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);++ let e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);assert.strictEqual(e.state, 'queued');assert.match(e.last_error, /Live 503/);-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');live.state.fail = null;t.clock.offset += 5000;await domain.effects.drain();t.clock.offset = 0;-- e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);++ e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);assert.strictEqual(e.state, 'delivered');-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');});await check('Live refuses (422): the effect fails at once, tips.interaction.failed, payment untouched', async () => {@@@ -60,17 +60,17 @@ const { boot, check, done } = require('./helpers/app');const id = r.json.interaction.id;await domain.effects.drain();live.state.fail = null;-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.delivery_state, 'failed');assert.strictEqual(i.payment_state, 'settled');-- assert.strictEqual(billing.payable.get(t.domain.profiles.byHandle('alex').creator_subject) >= 8, true);-- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((ev) => ev.subject.id === id).length, 1);++ assert.strictEqual(billing.payable.get((await t.domain.profiles.byHandle(t.db, 'alex')).creator_subject) >= 8, true);++ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((ev) => ev.subject.id === id).length, 1);});await check('simulations never reach Live: their chat effects stay in the test adapter', async () => {-- const alex = t.domain.profiles.byHandle('alex');++ const alex = await t.domain.profiles.byHandle(t.db, 'alex');const before = live.deliveries.length;-- const sim = domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });++ const sim = await domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });await domain.effects.drain();assert.strictEqual(live.deliveries.length, before);assert.ok(t.adapters.test.jobs.some((j) => j.interaction.id === sim.id && j.test));@@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');},};};-- const byRef = (ref) => t.db.prepare("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = ?").get(ref);++ const byRef = (ref) => t.db.maybe("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = $1", [ref]);let first;await check('an EXTERNAL receipt is recorded, settled and announced in Live chat as Live\'s webhook wrote it', async () => {@@@ -37,7 +37,7 @@ const { boot, check, done } = require('./helpers/app');const r = await t.deliver(first);assert.strictEqual(r.status, 200, JSON.stringify(r.json));assert.strictEqual(r.json.outcome, 'recorded_external');-- const i = byRef('ext-1');++ const i = await byRef('ext-1');assert.deepStrictEqual([i.creator_subject, i.kind, i.amount, i.amount_cents, i.supporter_name, i.message], [alex.subject, 'tip', 1234, 1234, 'Generous Fan', 'love the stream']);assert.deepStrictEqual([i.funding, i.settlement, i.origin, i.payment_state, i.billing_txn_id], ['external', 'external', 'billing-external', 'settled', null]);await domain.effects.drain();@@@ -46,12 +46,12 @@ const { boot, check, done } = require('./helpers/app');assert.deepStrictEqual([d[0].key, d[0].job.effect, d[0].job.test], [`${i.id}:chat_line`, 'chat_line', false]);assert.strictEqual(d[0].job.text, 'Generous Fan tipped 1,234 Vibes: love the stream (PowerChat)');assert.deepStrictEqual(d[0].job.creator, { type: 'user', id: alex.subject, handle: 'alex' });-- const effects = t.db.prepare('SELECT effect, state FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);++ const effects = await t.db.many('SELECT effect, state FROM interaction_effects WHERE interaction_id = $1 ORDER BY id', [i.id]);assert.deepStrictEqual(effects, [{ effect: 'overlay_alert', state: 'delivered' }, { effect: 'chat_line', state: 'delivered' }]);-- const goal = domain.goals.present(domain.goals.get(g.json.goal.id));++ const goal = await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id));assert.strictEqual(goal.current_amount, 1234, 'the sole active goal advances, as on Live');-- assert.ok(t.outboxRows('tips.goal.updated').some((e) => e.payload.interaction_id === i.id));-- assert.ok(t.outboxRows('tips.interaction.ready').some((e) => e.subject.id === i.id));++ assert.ok((await t.outboxRows('tips.goal.updated')).some((e) => e.payload.interaction_id === i.id));++ assert.ok((await t.outboxRows('tips.interaction.ready')).some((e) => e.subject.id === i.id));});await check('a redelivery or a republish of the same payment announces nothing more', async () => {@@@ -61,12 +61,13 @@ const { boot, check, done } = require('./helpers/app');const viaApi = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.interaction.record'], body: { creator: 'alex', provider: 'powerchat', provider_ref: 'ext-1', amount_cents: 1234 } });assert.strictEqual(viaApi.json.duplicate, true, 'the same key as POST /interactions/external');await domain.effects.drain();-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE provider_ref = 'ext-1'").get().n, 1);-- assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === byRef('ext-1').id).length, 1);++ assert.strictEqual(await t.db.value("SELECT count(*) FROM tip_interactions WHERE provider_ref = 'ext-1'"), 1);++ const one = await byRef('ext-1');++ assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === one.id).length, 1);});await check('EXTERNAL money stays out of the Billing total', async () => {-- const totals = domain.interactions.totals(alex.subject);++ const totals = await domain.interactions.totals(t.db, alex.subject);assert.strictEqual(totals.external, 1234);assert.strictEqual(totals.settled_via_billing, 0);});@@@ -75,19 +76,19 @@ const { boot, check, done } = require('./helpers/app');const a = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 900 } })).json.goal;const b = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Chair', target_amount: 900 } })).json.goal;await t.deliver(receipt({ provider_event_id: 'ext-goal-tips', amount_cents: 100, value_bits: 100, app_purpose: `goal:${a.id}` }));-- t.db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)-- VALUES ('live', 'donation_goals', '12', 'goal', ?, 'imported', 'imp_test', ?, ?)`).run(b.id, new Date().toISOString(), new Date().toISOString());++ await t.db.exec(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)++ VALUES ('live', 'donation_goals', '12', 'goal', $1, 'imported', 'imp_test', $2, $2)`, [b.id, new Date().toISOString()]);await t.deliver(receipt({ provider_event_id: 'ext-goal-live', amount_cents: 200, value_bits: 200, app_purpose: 'goal:12' }));await t.deliver(receipt({ provider_event_id: 'ext-goal-none', amount_cents: 300, value_bits: 300 }));-- assert.strictEqual(domain.goals.present(domain.goals.get(a.id)).current_amount, 100);-- assert.strictEqual(domain.goals.present(domain.goals.get(b.id)).current_amount, 200);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(byRef('ext-goal-none').id).n, 0, 'several active goals and no pick: none (Live\'s rule)');++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, a.id))).current_amount, 100);++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, b.id))).current_amount, 200);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [(await byRef('ext-goal-none')).id]), 0, 'several active goals and no pick: none (Live\'s rule)');});await check('anonymous: no donor name; a creator with no tip page is still announced', async () => {const bob = t.network.newUser('bob');await t.deliver(receipt({ provider_event_id: 'ext-anon', donor_name: null, anonymous: true, message: 'hi' }, { streamer: bob.subject }));-- const i = byRef('ext-anon');++ const i = await byRef('ext-anon');assert.deepStrictEqual([i.creator_subject, i.supporter_name], [bob.subject, 'Anonymous']);await domain.effects.drain();const d = live.deliveries.find((x) => x.job.interaction.id === i.id);@@@ -99,20 +100,20 @@ const { boot, check, done } = require('./helpers/app');const before = live.deliveries.length;await t.deliver(receipt({ provider_event_id: 'ext-test', test: true }));await domain.effects.drain();-- const i = byRef('ext-test');-- assert.strictEqual(i.test, 1);++ const i = await byRef('ext-test');++ assert.strictEqual(i.test, true);assert.strictEqual(live.deliveries.length, before);assert.strictEqual(t.adapters.test.jobs.filter((j) => j.interaction.id === i.id).length, 1);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);});await check('malformed or foreign receipts are ignored, not recorded', async () => {-- const n = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;++ const n = await t.db.value('SELECT count(*) FROM tip_interactions');assert.strictEqual((await t.deliver(receipt({ streamer: { type: 'user', id: '42' } }))).json.outcome, 'ignored:no_streamer');assert.strictEqual((await t.deliver(receipt({ amount_cents: 1e20 }))).json.outcome, 'ignored:no_amount');assert.strictEqual((await t.deliver(receipt({ provider_event_id: '' }))).json.outcome, 'ignored:no_reference');assert.strictEqual((await t.deliver({ ...receipt(), source: 'live' })).json.outcome, 'ignored:source');-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, n);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), n);});await t.close();nnew file mode 100644@@@ -0,0 +1,301 @@++-- The SQLite schema of the last SQLite release of OpenVibe.Tips (schema v3: server/db.js with its added++-- columns, plus the openvibe-sdk outbox and inbox), as sqlite_master holds it in a database that release made.++-- test/sqlite-import.test.js builds its fixture from it. Never edit: it is the source side of the one-time import.++++CREATE TABLE settings (++ id INTEGER PRIMARY KEY CHECK (id = 1),++ schema_version INTEGER NOT NULL,++ created_at TEXT NOT NULL++);++++CREATE TABLE creator_tip_profiles (++ creator_subject TEXT PRIMARY KEY, -- usr_…++ handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)++ display_name TEXT NOT NULL,++ avatar_url TEXT,++ headline TEXT,++ page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing++ accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted++ min_amount INTEGER NOT NULL DEFAULT 1,++ paid_message_min INTEGER NOT NULL DEFAULT 100,++ tts_enabled INTEGER NOT NULL DEFAULT 0,++ tts_min_amount INTEGER NOT NULL DEFAULT 100,++ tts_max_chars INTEGER NOT NULL DEFAULT 200,++ tts_voice TEXT NOT NULL DEFAULT 'gary',++ media_requests_enabled INTEGER NOT NULL DEFAULT 0,++ media_request_min INTEGER NOT NULL DEFAULT 25,++ media_max_seconds INTEGER NOT NULL DEFAULT 600,++ revision INTEGER NOT NULL DEFAULT 1,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++, page_settings TEXT NOT NULL DEFAULT '{}', filter_words TEXT NOT NULL DEFAULT '[]', filter_action TEXT NOT NULL DEFAULT 'mask', filter_links INTEGER NOT NULL DEFAULT 1);++++CREATE TABLE tip_interactions (++ id TEXT PRIMARY KEY, -- tint_<ULID>++ creator_subject TEXT NOT NULL,++ supporter_subject TEXT, -- null: external/anonymous/unmapped++ supporter_name TEXT, -- display projection at the time++ kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),++ amount INTEGER NOT NULL CHECK (amount > 0),++ currency TEXT NOT NULL DEFAULT 'vibes-bits',++ amount_cents INTEGER, -- money that arrived (provider/external)++ message TEXT,++ request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target++ funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),++ settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),++ payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),++ delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),++ test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted++ billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it++ billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent++ funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it++ reversal_txn_ids TEXT NOT NULL DEFAULT '[]',++ reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator++ checkout_url TEXT,++ checkout_ref TEXT,++ provider TEXT,++ provider_ref TEXT,++ legacy_source TEXT, -- 'live:transactions:<id>' etc.++ idempotency_key TEXT UNIQUE,++ failure TEXT,++ origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import++ transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried++ transfer_attempts INTEGER NOT NULL DEFAULT 0,++ next_transfer_at INTEGER NOT NULL DEFAULT 0,++ target TEXT, -- JSON EntityRef (e.g. a Live stream)++ created_at TEXT NOT NULL,++ settled_at TEXT,++ reversed_at TEXT,++ delivered_at TEXT,++ updated_at TEXT NOT NULL, anonymous INTEGER NOT NULL DEFAULT 0, hide_amount INTEGER NOT NULL DEFAULT 0, private_message INTEGER NOT NULL DEFAULT 0, erased_at TEXT, moderation TEXT NOT NULL DEFAULT 'visible', moderated_at TEXT, moderated_by TEXT, filtered INTEGER NOT NULL DEFAULT 0,++ UNIQUE (provider, provider_ref),++ UNIQUE (legacy_source)++);++++CREATE TABLE tip_goals (++ id TEXT PRIMARY KEY, -- tgoal_<ULID>++ creator_subject TEXT NOT NULL,++ title TEXT NOT NULL,++ description TEXT,++ target_amount INTEGER NOT NULL CHECK (target_amount > 0),++ currency TEXT NOT NULL DEFAULT 'vibes-bits',++ image_url TEXT,++ status TEXT NOT NULL CHECK (status IN ('active', 'closed')),++ sort_order INTEGER NOT NULL DEFAULT 0,++ reached_at TEXT,++ closed_at TEXT,++ opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)++ legacy_source TEXT UNIQUE,++ revision INTEGER NOT NULL DEFAULT 1,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++);++++CREATE TABLE tip_goal_contributions (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ goal_id TEXT NOT NULL REFERENCES tip_goals(id),++ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),++ amount INTEGER NOT NULL CHECK (amount > 0),++ reversed_amount INTEGER NOT NULL DEFAULT 0,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL,++ UNIQUE (goal_id, interaction_id)++);++++CREATE TABLE paid_messages (++ id TEXT PRIMARY KEY, -- tpm_<ULID>++ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),++ creator_subject TEXT NOT NULL,++ kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),++ text TEXT NOT NULL,++ voice TEXT,++ highlight_seconds INTEGER NOT NULL DEFAULT 0,++ status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),++ chat_ref TEXT, -- JSON reference the chat adapter returned++ test INTEGER NOT NULL DEFAULT 0,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++);++++CREATE TABLE paid_media_requests (++ id TEXT PRIMARY KEY, -- tmr_<ULID>++ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),++ creator_subject TEXT NOT NULL,++ url TEXT NOT NULL,++ provider TEXT,++ status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),++ queue_ref TEXT, -- JSON reference returned by the queue owner++ test INTEGER NOT NULL DEFAULT 0,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++);++++CREATE TABLE overlay_configs (++ id TEXT PRIMARY KEY, -- tovc_<ULID>++ creator_subject TEXT NOT NULL,++ kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),++ name TEXT NOT NULL,++ settings TEXT NOT NULL DEFAULT '{}',++ goal_id TEXT,++ revision INTEGER NOT NULL DEFAULT 1,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL++);++++CREATE TABLE overlay_tokens (++ id TEXT PRIMARY KEY, -- tovt_<ULID>++ creator_subject TEXT NOT NULL,++ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once++ scopes TEXT NOT NULL, -- JSON ["alerts","goals"]++ config_id TEXT,++ label TEXT,++ created_by TEXT NOT NULL,++ created_at TEXT NOT NULL,++ last_used_at TEXT,++ revoked_at TEXT++);++++CREATE TABLE overlay_deliveries (++ seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)++ id TEXT NOT NULL UNIQUE, -- tovd_<ULID>++ creator_subject TEXT NOT NULL,++ kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),++ interaction_id TEXT,++ goal_id TEXT,++ payload TEXT NOT NULL,++ test INTEGER NOT NULL DEFAULT 0,++ status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),++ delivered_at TEXT,++ failed_at TEXT,++ sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)++ dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>++ created_at TEXT NOT NULL++, hidden INTEGER NOT NULL DEFAULT 0);++++CREATE TABLE interaction_effects (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),++ effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),++ adapter TEXT NOT NULL,++ state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),++ attempts INTEGER NOT NULL DEFAULT 0,++ next_attempt_at INTEGER NOT NULL DEFAULT 0,++ last_error TEXT,++ result TEXT,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL,++ UNIQUE (interaction_id, effect)++);++++CREATE TABLE migration_maps (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ source TEXT NOT NULL, -- 'live'++ source_table TEXT NOT NULL,++ source_id TEXT NOT NULL,++ target_type TEXT, -- 'interaction' | 'goal'++ target_id TEXT,++ status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),++ reason TEXT,++ billing_txn_id TEXT,++ run_id TEXT NOT NULL,++ created_at TEXT NOT NULL,++ updated_at TEXT NOT NULL,++ UNIQUE (source, source_table, source_id)++);++++CREATE TABLE import_runs (++ id TEXT PRIMARY KEY,++ source TEXT NOT NULL,++ dry_run INTEGER NOT NULL,++ started_at TEXT NOT NULL,++ finished_at TEXT,++ report TEXT++);++++CREATE TABLE tip_moderators (++ creator_subject TEXT NOT NULL,++ moderator_subject TEXT NOT NULL,++ name TEXT, -- display name when they joined (for the creator's list)++ added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'++ created_at TEXT NOT NULL,++ removed_at TEXT,++ PRIMARY KEY (creator_subject, moderator_subject)++);++++CREATE TABLE tip_moderator_invites (++ id TEXT PRIMARY KEY, -- tmin_<ULID>++ creator_subject TEXT NOT NULL,++ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)++ created_by TEXT NOT NULL,++ created_at TEXT NOT NULL,++ expires_at TEXT NOT NULL,++ used_at TEXT,++ used_by TEXT,++ revoked_at TEXT++);++++CREATE TABLE tip_moderation_log (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ creator_subject TEXT NOT NULL,++ interaction_id TEXT NOT NULL,++ action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),++ by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),++ actor TEXT, -- usr_… / svc:… ; null for the filter++ reason TEXT,++ created_at TEXT NOT NULL++);++++CREATE TABLE api_idempotency (++ key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>++ request_hash TEXT NOT NULL,++ method TEXT NOT NULL,++ path TEXT NOT NULL,++ status INTEGER NOT NULL,++ response TEXT NOT NULL,++ created_at TEXT NOT NULL++);++++CREATE TABLE event_outbox (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ event_id TEXT NOT NULL UNIQUE,++ envelope TEXT NOT NULL,++ traceparent TEXT,++ created_at INTEGER NOT NULL,++ attempts INTEGER NOT NULL DEFAULT 0,++ next_attempt_at INTEGER NOT NULL DEFAULT 0,++ sent_at INTEGER,++ seq INTEGER,++ rejected_at INTEGER,++ last_error TEXT++ );++++CREATE TABLE idempotency_receipts (++ consumer TEXT NOT NULL,++ event_id TEXT NOT NULL,++ processed_at INTEGER NOT NULL,++ PRIMARY KEY (consumer, event_id)++ );++++CREATE INDEX idx_tint_creator ON tip_interactions (creator_subject, created_at);++++CREATE INDEX idx_tint_supporter ON tip_interactions (supporter_subject, created_at);++++CREATE INDEX idx_tint_pending ON tip_interactions (payment_state, funding);++++CREATE INDEX idx_goal_creator ON tip_goals (creator_subject, status);++++CREATE INDEX idx_ovc_creator ON overlay_configs (creator_subject);++++CREATE INDEX idx_ovt_creator ON overlay_tokens (creator_subject);++++CREATE INDEX idx_ovd_creator ON overlay_deliveries (creator_subject, seq);++++CREATE INDEX idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';++++CREATE INDEX idx_effects_due ON interaction_effects (state, next_attempt_at);++++CREATE INDEX idx_tmod_moderator ON tip_moderators (moderator_subject);++++CREATE INDEX idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);++++CREATE INDEX idx_event_outbox_due ON event_outbox(sent_at, rejected_at, next_attempt_at);@@@ -1,8 +1,13 @@'use strict';/**-- * Boots Tips against the stubs on a random port with a temp database. Jobs are off: tests drive the++ * Boots Tips against the stubs on a random port with a migrated database of its own (helpers/db.js:++ * PGlite, or the containers with opts.store 'pg' / TIPS_TEST_STORE=pg). Jobs are off: tests drive the* effects worker and transfer retries explicitly (domain.effects.drain(), processDueTransfers()).-- * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow).++ * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow);++ * opts.valkey an openvibe-sdk/valkey handle (default: the containers' Valkey with TIPS_TEST_STORE=pg,++ * else none: everything shared stays in the process);++ * opts.db an existing handle (a second process on the same database); opts.share another instance++ * whose stubs (Network, Billing, Events, Live) this one uses.** t.call(method, path, { body, user, cap, sub, key }) user: a stub user → Bearer user JWT;* otherwise a service token with `cap`@@@ -16,18 +21,20 @@ const http = require('http');const crypto = require('crypto');const { signDelivery, signDeliveryHeaders } = require('openvibe-sdk/events');const { startNetwork, startBilling, startEvents, startLive } = require('./stubs');++const { testDb, testValkey } = require('./db');const EVENTS_SECRET = 'e'.repeat(48);async function boot(opts = {}) {-- const network = await startNetwork();-- const billing = await startBilling(network);-- const events = await startEvents();-- const live = opts.live ? await startLive(network) : null;++ // opts.share: another booted instance whose stubs this one uses (a second process of one service).++ const shared = opts.share || null;++ const network = shared ? shared.network : await startNetwork();++ const billing = shared ? shared.billing : await startBilling(network);++ const events = shared ? shared.events : await startEvents();++ const live = shared ? shared.live : (opts.live ? await startLive(network) : null);const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-test-'));const env = {NODE_ENV: 'test',-- TIPS_DB_PATH: path.join(dir, 'tips.db'),BASE_URL: 'http://tips.test',OV_NETWORK_URL: network.url,OV_NETWORK_INTERNAL_URL: network.url,@@@ -53,7 +60,10 @@ async function boot(opts = {}) {const clock = { offset: 0 };const logs = [];const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')) };-- const app = createApp({ config, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });++ const store = opts.db ? { db: opts.db, store: opts.db.store, close: async () => {} } : await testDb({ store: opts.store });++ const ownValkey = opts.valkey === undefined && process.env.TIPS_TEST_STORE === 'pg' ? testValkey() : null;++ const valkey = opts.valkey || ownValkey;++ const app = createApp({ config, db: store.db, valkey, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });await app.locals.keys.load();const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;@@@ -135,17 +145,20 @@ async function boot(opts = {}) {return u;}-- const outboxRows = (type) => domain.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope)).filter((e) => !type || e.event_type === type);++ /** The events in the outbox (envelopes, oldest first), of one type or all. */++ const outboxRows = async (type) => (await domain.db.many('SELECT envelope FROM tips_event_outbox ORDER BY id')).map((r) => r.envelope).filter((e) => !type || e.event_type === type);return {-- app, base, call, deliver, sse, creator, domain, db: domain.db, config, clock, network, billing, events, live, logs, dir, outboxRows,++ app, base, call, deliver, sse, creator, domain, db: domain.db, store, valkey, config, clock, network, billing, events, live, logs, dir, outboxRows,adapters: app.locals.adapters,close: async () => {-- domain.overlays.closeAll();++ await domain.overlays.close();++ server.closeAllConnections();await new Promise((r) => server.close(r));await app.locals.outbox.stop();-- await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);-- try { domain.db.close(); } catch { /* */ }++ if (!shared) await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);++ await store.close();++ if (ownValkey) await ownValkey.close();fs.rmSync(dir, { recursive: true, force: true });},};nnew file mode 100644@@@ -0,0 +1,85 @@++'use strict';++/**++ * A migrated database for one test run (ADR-035).++ *++ * store 'pglite' (the default): real PostgreSQL in-process, migrated (openvibe-sdk/db { pglite }).++ * store 'pg': the production-shaped containers (openvibe-sdk scripts/test-services.sh up), when++ * OV_TEST_PG_URL (PgBouncer, transaction mode) and OV_TEST_PG_DIRECT_URL are set. The run gets roles++ * and a schema of its own, shaped as OpenVibe.Host's roles/data/add-service.sh makes them: an owner++ * role that migrates on the direct connection, and a runtime role (DML only, statement_timeout 15 s,++ * lock_timeout 5 s) that serves through PgBouncer. Other suites sharing the database never see them;++ * close() drops them.++ *++ * TIPS_TEST_STORE=pg runs every test file on the containers instead of PGlite.++ *++ * const { db, close } = await testDb();++ * testValkey() the containers' Valkey (OV_TEST_VALKEY_URL) under a prefix of its own, or null++ */++const crypto = require('crypto');++const { createDb } = require('openvibe-sdk/db');++const { createValkey } = require('openvibe-sdk/valkey');++const { MIGRATIONS } = require('../../server/db');++++const quiet = { log() {}, warn() {}, error: (...a) => console.error(...a) };++const pgAvailable = () => !!(process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL);++const valkeyAvailable = () => !!process.env.OV_TEST_VALKEY_URL;++++async function testDb({ store = process.env.TIPS_TEST_STORE || 'pglite', max = 4 } = {}) {++ if (store !== 'pg') {++ const db = createDb({ pglite: true, service: 'tips-test', log: quiet });++ await db.migrate({ dir: MIGRATIONS, log: quiet });++ return { db, store: 'pglite', close: () => db.close().catch(() => {}) };++ }++ if (!pgAvailable()) throw new Error('store pg needs OV_TEST_PG_URL and OV_TEST_PG_DIRECT_URL (openvibe-sdk scripts/test-services.sh up)');++ const name = `tips_t${process.pid}_${crypto.randomBytes(4).toString('hex')}`;++ const owner = `${name}_owner`;++ const pw = crypto.randomBytes(16).toString('hex');++ const su = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL, service: 'tips-test-admin', max: 1, log: quiet });++ const database = await su.value('SELECT current_database()');++ for (const stmt of [++ `CREATE ROLE ${owner} LOGIN PASSWORD '${pw}'`,++ `CREATE ROLE ${name} LOGIN PASSWORD '${pw}'`,++ `GRANT CONNECT ON DATABASE ${database} TO ${owner}, ${name}`,++ `CREATE SCHEMA ${name} AUTHORIZATION ${owner}`,++ `ALTER ROLE ${owner} SET search_path = ${name}`,++ `ALTER ROLE ${name} SET search_path = ${name}`,++ `ALTER ROLE ${name} SET statement_timeout = '15s'`,++ `ALTER ROLE ${name} SET lock_timeout = '5s'`,++ `GRANT USAGE ON SCHEMA ${name} TO ${name}`,++ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ${name}`,++ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO ${name}`,++ ]) await su.query(stmt);++ const as = (url, user) => { const u = new URL(url); u.username = user; u.password = pw; return u.toString(); };++ async function drop() {++ try {++ // PgBouncer keeps its server connections after the client pool closes: end them here, or++ // every run would leave its roles' idle connections holding the server's slots.++ await su.query('SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE usename = ANY($1)', [[name, owner]]);++ await su.query(`DROP SCHEMA IF EXISTS ${name} CASCADE`);++ for (const r of [name, owner]) { await su.query(`DROP OWNED BY ${r}`); await su.query(`DROP ROLE ${r}`); }++ } finally { await su.close(); }++ }++ let db;++ try {++ const ownerDb = createDb({ url: as(process.env.OV_TEST_PG_DIRECT_URL, owner), service: 'tips-test-migrate', max: 1, log: quiet });++ try { await ownerDb.migrate({ dir: MIGRATIONS, log: quiet }); } finally { await ownerDb.close(); }++ db = createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet });++ } catch (e) { await drop().catch(() => {}); throw e; } // a failed setup leaves nothing behind++ return {++ db, store: 'postgresql', schema: name,++ /** Another pooled handle on the same database (a second process). */++ open: (o = {}) => createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet, ...o }),++ async close() {++ await db.close().catch(() => {});++ await drop();++ },++ };++}++++/** The containers' Valkey under a prefix no other run uses (ov:tips-test:<random>:), or null without it. */++function testValkey() {++ if (!valkeyAvailable()) return null;++ return createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: `ov:tips-test:${crypto.randomBytes(4).toString('hex')}:`, log: quiet });++}++++module.exports = { testDb, testValkey, pgAvailable, valkeyAvailable };@@@ -81,61 +81,61 @@ function billingFixture(file, subjects, { withNative = false } = {}) {const r = await run({ dryRun: true });assert.strictEqual(r.dry_run, true);assert.ok(r.counts.imported > 0);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 0);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM migration_maps').get().n, 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM migration_maps'), 0);});let report;await check('donations import settled and linked to Billing; failed excluded; unmapped creators held; test era flagged', async () => {report = await run();-- const byLegacy = (id) => t.db.prepare('SELECT * FROM tip_interactions WHERE legacy_source = ?').get(`live:transactions:${id}`);-- const two = byLegacy(2);++ const byLegacy = (id) => t.db.maybe('SELECT * FROM tip_interactions WHERE legacy_source = $1', [`live:transactions:${id}`]);++ const two = await byLegacy(2);assert.strictEqual(two.payment_state, 'settled');assert.strictEqual(two.settlement, 'imported');assert.strictEqual(two.billing_txn_id, 'txn_import_2');assert.strictEqual(two.supporter_subject, subjects[2]);assert.strictEqual(two.message, 'nice');assert.strictEqual(two.created_at, '2026-02-01T10:00:00.000Z');-- assert.strictEqual(byLegacy(1).test, 1, 'before stats_vibes_reset_at');-- const ghost = byLegacy(3);++ assert.strictEqual((await byLegacy(1)).test, true, 'before stats_vibes_reset_at');++ const ghost = await byLegacy(3);assert.strictEqual(ghost.supporter_subject, null);assert.strictEqual(ghost.supporter_name, 'Ghost');-- assert.strictEqual(byLegacy(6).kind, 'media_request');-- assert.strictEqual(byLegacy(8).funding, 'provider');-- assert.strictEqual(byLegacy(4), undefined, 'held, not dropped');-- const held = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get();++ assert.strictEqual((await byLegacy(6)).kind, 'media_request');++ assert.strictEqual((await byLegacy(8)).funding, 'provider');++ assert.strictEqual(await byLegacy(4), null, 'held, not dropped');++ const held = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'");assert.strictEqual(held.status, 'held');-- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'").get();++ const excluded = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'");assert.strictEqual(excluded.status, 'excluded');assert.match(excluded.reason, /status failed/);assert.strictEqual(report.counts.linked_to_billing, report.counts.imported);// Imports are history: no effects, no overlay alerts, no events.-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM interaction_effects').get().n, 0);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM overlay_deliveries').get().n, 0);-- assert.strictEqual(t.outboxRows('tips.interaction.ready').length, 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM interaction_effects'), 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM overlay_deliveries'), 0);++ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).length, 0);});await check('the refund nets out against the donation it undoes', async () => {-- const six = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'").get();++ const six = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'");assert.strictEqual(six.reversed_bits, 250);assert.strictEqual(six.payment_state, 'reversed');assert.strictEqual(report.counts.refunds_applied, 1);});await check('external PowerChat tips: the direct one is imported as EXTERNAL; the site-routed celebration is not counted twice', async () => {-- const ext = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'").get();++ const ext = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'");assert.strictEqual(ext.settlement, 'external');assert.strictEqual(ext.amount, 700);assert.strictEqual(ext.supporter_name, 'Direct');-- const twin = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'").get();++ const twin = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'");assert.strictEqual(twin.status, 'excluded');assert.match(twin.reason, /site-routed tip \(Live transaction 8\)/);-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'").get().n, 0, 'on-site donation lines are not PowerChat');++ assert.strictEqual(await t.db.value("SELECT count(*) FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'"), 0, 'on-site donation lines are not PowerChat');});await check('goals import with Live\'s current amount carried over (not invented contributions)', async () => {-- const g = t.db.prepare("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'").get();-- const view = domain.goals.present(g);++ const g = await t.db.maybe("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'");++ const view = await domain.goals.present(t.db, g);assert.strictEqual(view.current_amount, 1200);assert.strictEqual(view.carried_over_amount, 1200);assert.strictEqual(view.supporters_count, 0);@@@ -154,34 +154,34 @@ function billingFixture(file, subjects, { withNative = false } = {}) {const evTest = t.billing.envelope('billing.transaction.settled', { transaction_id: 'txn_native_test', type: 'donation', test: true, from_subject: subjects[2], to_subject: subjects[1], provider: null, metadata: { kind: 'donation', amount_bits: 999 } }, 'txn_native_test');await t.deliver(evTest);const { reconcile } = require('../server/importer/live');-- const r = reconcile(domain, billingSnapshotSource(bdb));++ const r = await reconcile(domain, billingSnapshotSource(bdb));assert.strictEqual(r.ok, true, JSON.stringify(r.mismatches));-- assert.strictEqual(domain.interactions.totals(subjects[1]).settled_via_billing, 740);-- assert.strictEqual(domain.interactions.totals(subjects[1]).external, 700);++ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).settled_via_billing, 740);++ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).external, 700);});await check('a re-run changes nothing; a newly mapped creator is released from hold', async () => {-- const before = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;++ const before = await t.db.value('SELECT count(*) FROM tip_interactions');const again = await run();assert.strictEqual(again.counts.imported, 0);assert.strictEqual(again.counts.goals, 0);assert.ok(again.counts.unchanged > 0);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before);const heldUser = t.network.newUser('held', 4);const r = await run();assert.strictEqual(r.counts.released, 1);assert.strictEqual(r.counts.goals, 1);-- const four = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'").get();++ const four = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'");assert.strictEqual(four.creator_subject, heldUser.subject);assert.strictEqual(four.billing_txn_id, 'txn_import_4');-- assert.strictEqual(t.db.prepare("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get().status, 'imported');++ assert.strictEqual(await t.db.value("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'"), 'imported');});await check('every source row is imported, excluded with a reason, or held — none silently dropped', async () => {const tipRows = live.prepare("SELECT id FROM transactions WHERE type IN ('donation', 'refund')").all().map((x) => String(x.id));-- const mapped = new Set(t.db.prepare("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'").all().map((x) => x.source_id));++ const mapped = new Set((await t.db.many("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'")).map((x) => x.source_id));assert.deepStrictEqual(tipRows.filter((id) => !mapped.has(id)), []);-- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE status = 'excluded'").all();++ const excluded = await t.db.many("SELECT * FROM migration_maps WHERE status = 'excluded'");assert.ok(excluded.every((x) => x.reason));});nnew file mode 100644@@@ -0,0 +1,237 @@++'use strict';++// Tips on the production-shaped data tier (ADR-035; the tests of ADR-007's 2026-09-24 amendment):++// PostgreSQL 18 behind PgBouncer in transaction mode and Valkey 9, from++// `eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)"`. Two Tips processes (two app++// instances, each with its own pool and Valkey connection) share one database, one Valkey and the same++// Billing and Network:++// - money races: one settlement or one reversal delivered many times at once, to both processes,++// counts once; the unpaid-checkout limit holds under concurrent requests (serializable)++// - work claimed with a lease: two effects workers and two transfer retries at once act once each++// - per-process state in Valkey: per-actor limits, overlay fan-out (a stream on one process hears++// tips, config changes and revocations made on the other), overlay stream slots++// - flushing Valkey changes no money answer++// Without the containers it prints why it skipped; the rest of the suite runs on PGlite.++const assert = require('assert');++const { ids } = require('openvibe-contracts');++const { createValkey } = require('openvibe-sdk/valkey');++const { testDb, testValkey, pgAvailable, valkeyAvailable } = require('./helpers/db');++const { boot, check, done } = require('./helpers/app');++++if (!pgAvailable()) {++ console.log('integration (PgBouncer + Valkey): skipped (OV_TEST_PG_URL not set; eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)")');++ process.exit(0);++}++++const LIMIT_CLOCK = Date.UTC(2026, 8, 28, 12, 0, 10); // one fixed minute for both processes' limiters++++(async () => {++ const store = await testDb({ store: 'pg', max: 6 });++ const vA = testValkey();++ const vB = vA ? createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: vA.prefix, log: { warn() {} } }) : null;++ const env = { TIPS_OVERLAY_MAX_STREAMS: '2', TIPS_MAX_PENDING_CHECKOUTS: '3' };++ const appOpts = { limitsNow: () => LIMIT_CLOCK };++ const A = await boot({ db: store.db, valkey: vA, env, appOpts });++ const dbB = store.open();++ const B = await boot({ db: dbB, valkey: vB, env, appOpts, share: A });++ const { billing } = A;++ const db = store.db;++ const alex = await A.creator('alex');++ const fan = A.network.newUser('fan');++ billing.fund(fan.subject, 1_000_000);++ const goal = (await A.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 100_000 } })).json.goal;++ const fresh = (e) => ({ ...e, event_id: ids.newId('event') });++ const both = (n, fn) => Promise.all(Array.from({ length: n }, (_, k) => fn(k % 2 ? B : A, k)));++++ await check('the store is PostgreSQL through PgBouncer; Valkey answers', async () => {++ const r = await A.call('GET', '/api/ready', { token: null });++ assert.strictEqual(r.status, 200, r.text);++ assert.strictEqual(r.json.checks.db.detail.store, 'postgresql');++ assert.strictEqual(r.json.checks.valkey.status, vA ? 'ok' : 'skipped');++ });++++ await check('one Billing settlement delivered 6 times at once, to both processes, settles once', async () => {++ const ev = billing.foreignDonation({ from: fan.subject, to: alex.subject, amount: 300, provider: 'powerchat' });++ const out = await both(6, (t) => t.deliver(fresh(ev)));++ assert.ok(out.every((r) => r.status === 200), JSON.stringify(out.map((r) => [r.status, r.json])));++ assert.deepStrictEqual(out.map((r) => r.json.outcome).sort(), ['duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'recorded']);++ const rows = await db.many('SELECT id FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);++ assert.strictEqual(rows.length, 1);++ assert.strictEqual(await db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [rows[0].id]), 1);++ assert.strictEqual((await A.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === rows[0].id).length, 1);++ assert.strictEqual(await db.value("SELECT count(*) FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [rows[0].id]), 1);++ });++++ await check('one reversal delivered 6 times at once takes the bits back once, from the goal too', async () => {++ const r = await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 400, goal_id: goal.id } });++ assert.strictEqual(r.status, 201, r.text);++ const id = r.json.interaction.id;++ const before = (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount;++ const rev = billing.refund(r.json.interaction.payment.billing_txn_id);++ const out = await both(6, (t) => t.deliver(fresh(rev)));++ assert.ok(out.every((x) => x.status === 200), JSON.stringify(out.map((x) => [x.status, x.json])));++ assert.strictEqual(out.filter((x) => x.json.outcome === 'duplicate_reversal').length, 5);++ const i = await db.one('SELECT reversed_bits, reversal_txn_ids, payment_state FROM tip_interactions WHERE id = $1', [id]);++ assert.deepStrictEqual([i.reversed_bits, i.reversal_txn_ids.length, i.payment_state], [400, 1, 'reversed']);++ assert.strictEqual((await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount, before - 400);++ assert.strictEqual(await A.domain.interactions.totals(db, alex.subject).then((x) => x.settled_via_billing), billing.payable.get(alex.subject), 'reconciles with Billing');++ });++++ await check('one Idempotency-Key sent at once to both processes runs once: one interaction, one Billing transfer', async () => {++ const key = `race-${Date.now()}-key`;++ const body = { creator: 'alex', amount: 33, message: 'once' };++ const out = await Promise.all([A, B, A, B].map((t) => t.call('POST', '/api/v1/checkout', { user: fan, body, key })));++ // The first runs; the others are told it is in progress (409, Retry-After) or get its stored answer.++ assert.ok(out.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(out.map((r) => [r.status, r.json && r.json.code])));++ const ok = out.filter((r) => r.status === 201);++ assert.ok(ok.length >= 1);++ assert.strictEqual(new Set(ok.map((r) => r.json.interaction.id)).size, 1, 'every answer names the one interaction');++ const id = ok[0].json.interaction.id;++ assert.strictEqual(billing.transfers().filter((c) => c.key === `tips:transfer:${id}`).length, 1);++ assert.strictEqual(await db.value('SELECT count(*) FROM tip_interactions WHERE idempotency_key = $1', [`api:${fan.subject}:${key}`]), 1);++ const later = await B.call('POST', '/api/v1/checkout', { user: fan, body, key });++ assert.deepStrictEqual([later.status, later.headers.get('idempotent-replayed'), later.json.interaction.id], [201, 'true', id]);++ });++++ await check('a creator\'s overlay deliveries become visible in seq order, however their transactions interleave', async () => {++ // T1 inserts a delivery and holds its transaction open; T2 (the other process) inserts one for the++ // same creator. T2 waits for T1, so seq order is commit order and no stream skips a seq.++ const view = { id: goal.id };++ let release;++ const hold = new Promise((ok) => { release = ok; });++ let t1Seq = null; let t2Done = false;++ const t1 = A.domain.tx(async (t) => {++ await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-1-${Date.now()}` });++ t1Seq = await t.value('SELECT max(seq) FROM overlay_deliveries');++ await hold;++ });++ while (t1Seq === null) await new Promise((ok) => setTimeout(ok, 10));++ const t2 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-2-${Date.now()}` })).then(() => { t2Done = true; });++ await new Promise((ok) => setTimeout(ok, 300));++ const waited = !t2Done;++ release(); // before any assertion, so a failure reports instead of holding the transaction++ await Promise.all([t1, t2]);++ assert.strictEqual(waited, true, 'the second insert waits for the first transaction');++ const [first, second] = (await db.many("SELECT seq FROM overlay_deliveries WHERE dedupe_key LIKE 'seq-order-%' ORDER BY seq")).map((r) => r.seq);++ assert.strictEqual(first, t1Seq);++ assert.ok(second > first);++ // Another creator is not held up by it.++ const other = A.network.newUser('other-creator');++ let otherDone = false;++ let release2;++ const hold2 = new Promise((ok) => { release2 = ok; });++ const t3 = A.domain.tx(async (t) => { await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-3-${Date.now()}` }); await hold2; });++ await new Promise((ok) => setTimeout(ok, 50));++ const t4 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, other.subject, view, { reason: 'test', dedupe: `seq-order-4-${Date.now()}` })).then(() => { otherDone = true; });++ await Promise.race([t4, new Promise((ok) => setTimeout(ok, 2000))]);++ const unblocked = otherDone;++ release2();++ await Promise.all([t3, t4]);++ assert.strictEqual(unblocked, true, 'another creator\'s insert does not wait');++ });++++ await check('the unpaid-checkout limit (3) holds for 6 concurrent requests on two processes', async () => {++ const buyer = A.network.newUser('buyer');++ const out = await both(6, (t, k) => t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 100 + k, pay_with: 'checkout', provider: 'stripe' } }));++ assert.deepStrictEqual(out.map((r) => r.status).sort(), [201, 201, 201, 429, 429, 429], JSON.stringify(out.map((r) => r.json && r.json.code)));++ assert.ok(out.filter((r) => r.status === 429).every((r) => r.json.code === 'tips.too_many_pending'));++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1 AND payment_state = 'pending'", [buyer.subject]), 3);++ });++++ await check('two effects workers draining at once deliver each chat line once (leased claims)', async () => {++ const ids6 = [];++ for (let k = 0; k < 6; k++) ids6.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 10 + k, message: `m${k}` } })).json.interaction.id);++ await Promise.all([A.domain.effects.drain(), B.domain.effects.drain(), A.domain.effects.drain(), B.domain.effects.drain()]);++ const jobs = [...A.adapters.test.jobs, ...B.adapters.test.jobs].filter((j) => ids6.includes(j.interaction.id));++ const per = new Map();++ for (const j of jobs) per.set(j.delivery_id, (per.get(j.delivery_id) || 0) + 1);++ assert.strictEqual(per.size, 6);++ assert.ok([...per.values()].every((n) => n === 1), JSON.stringify([...per]));++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND delivery_state = 'delivered'", [ids6]), 6);++ });++++ await check('two processes retrying due transfers at once call Billing once per tip', async () => {++ billing.state.down = true;++ const pending = [];++ for (let k = 0; k < 4; k++) pending.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 20 + k } })).json.interaction.id);++ billing.state.down = false;++ const mark = billing.calls.length;++ A.clock.offset = B.clock.offset = 60_000;++ await Promise.all([A.domain.interactions.processDueTransfers(), B.domain.interactions.processDueTransfers(), A.domain.interactions.processDueTransfers()]);++ A.clock.offset = B.clock.offset = 0;++ const calls = billing.calls.slice(mark).filter((c) => c.url === '/api/v1/transfers');++ assert.deepStrictEqual(calls.map((c) => c.key).sort(), pending.map((id) => `tips:transfer:${id}`).sort(), 'one call each');++ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND payment_state = 'settled'", [pending]), 4);++ });++++ if (!valkeyAvailable()) {++ console.log('integration, shared state in Valkey: skipped (OV_TEST_VALKEY_URL not set)');++ } else {++ await check('per-actor limits are counted across processes (POST /me/erase: 3 a minute a person)', async () => {++ const person = A.network.newUser('eraser');++ const codes = [];++ for (const t of [A, B, A, B]) codes.push((await t.call('POST', '/api/v1/me/erase', { user: person })).status);++ assert.deepStrictEqual(codes, [200, 200, 200, 429]);++ });++++ let secret;++ let cfg;++ await check('a stream on process A hears a tip, a config change and a revocation made on process B', async () => {++ cfg = (await B.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', name: 'Main' } })).json.config;++ const tok = (await B.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'], config_id: cfg.id } })).json;++ secret = tok.secret;++ const s = await A.sse(`/overlay/${secret}/events`);++ await s.waitFor((e) => e.event === 'hello');++ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 77, message: 'from B' } });++ const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === r.json.interaction.id);++ assert.strictEqual(alert.data.amount, 77);++ await B.call('PATCH', `/api/v1/overlay-configs/${cfg.id}`, { user: alex, body: { duration_ms: 4000 } });++ const c = await s.waitFor((e) => e.event === 'config');++ assert.strictEqual(c.data.config.settings.duration_ms, 4000);++ await B.call('POST', `/api/v1/overlay-tokens/${tok.token.id}/revoke`, { user: alex });++ assert.ok(await s.waitFor((e) => e.event === 'revoked'));++ await new Promise((ok) => setTimeout(ok, 100));++ assert.strictEqual(s.ended, true);++ });++++ await check('overlay stream slots (2 a token) are counted across processes', async () => {++ const tok = (await A.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts'] } })).json.secret;++ const s1 = await A.sse(`/overlay/${tok}/events`);++ const s2 = await B.sse(`/overlay/${tok}/events`);++ assert.deepStrictEqual([s1.statusCode, s2.statusCode], [200, 200]);++ assert.strictEqual((await fetch(`${A.base}/overlay/${tok}/events`)).status, 429, 'the third stream, on either process');++ assert.strictEqual(await B.domain.overlays.connected(alex.subject), 2);++ s1.close();++ await new Promise((ok) => setTimeout(ok, 150));++ const s3 = await B.sse(`/overlay/${tok}/events`);++ assert.strictEqual(s3.statusCode, 200, 'a closed stream frees its place for the other process');++ s2.close(); s3.close();++ });++++ await check('flushing Valkey changes no money answer', async () => {++ const snapshot = async () => ({++ totals: await A.domain.interactions.totals(db, alex.subject),++ goal: (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount,++ receipts: (await B.call('GET', '/api/v1/interactions?limit=200', { user: fan })).json.interactions.map((i) => [i.id, i.payment.state, i.amount]),++ });++ const before = await snapshot();++ let cursor = '0'; let deleted = 0;++ do {++ const [next, keys] = await vA.client.scan(cursor, 'MATCH', `${vA.prefix}*`, 'COUNT', 500);++ cursor = next;++ if (keys.length) deleted += await vA.client.del(...keys);++ } while (cursor !== '0');++ assert.ok(deleted > 0, 'there was shared state to lose');++ assert.deepStrictEqual(await snapshot(), before);++ assert.strictEqual(before.totals.settled_via_billing, billing.payable.get(alex.subject), 'and it still reconciles with Billing');++ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 5 } });++ assert.strictEqual(r.json.interaction.payment.state, 'settled', 'tips still settle');++ });++ }++++ await B.close();++ await A.close();++ await dbB.close();++ await store.close();++ if (vA) { await vA.close(); await vB.close(); }++ done();++})().catch((e) => { console.error(e); process.exit(1); });nnew file mode 100644@@@ -0,0 +1,161 @@++'use strict';++// migrations/ (ADR-035, ADR-028): the files parse with their phase, apply in order, and apply again++// without change; the schema carries every column the SQLite release had (nothing lost in the import);++// every query shape the code runs can be served by an index; and no runtime file loads better-sqlite3.++const assert = require('assert');++const fs = require('fs');++const path = require('path');++const Database = require('better-sqlite3');++const { createDb, sql } = require('openvibe-sdk/db');++const { MIGRATIONS } = require('../server/db');++const { TABLES } = require('../scripts/migrate-to-postgres');++const { check, done } = require('./helpers/app');++++const quiet = { log() {} };++const ROOT = path.join(__dirname, '..');++++/** The migration files as the SDK reads them: NNNN_name.sql with a phase header (and after: for a contract). */++function parse(dir) {++ return fs.readdirSync(dir).filter((f) => f.endsWith('.sql')).sort().map((file) => {++ const m = /^(\d{4,})_([a-z0-9_-]+)\.sql$/.exec(file);++ assert.ok(m, `${file} is named NNNN_description.sql`);++ const head = fs.readFileSync(path.join(dir, file), 'utf8').split('\n').slice(0, 12).join('\n');++ const phase = (/^--\s*phase:\s*(expand|migrate|contract)\s*$/m.exec(head) || [])[1];++ assert.ok(phase, `${file} has a phase header`);++ return { id: m[1], name: m[2], file, phase, after: (/^--\s*after:\s*(\d{4,})\s*$/m.exec(head) || [])[1] || null, transaction: !/^--\s*no-transaction\s*$/m.test(head) };++ });++}++++(async () => {++ const db = createDb({ pglite: true, service: 'tips-test', log: { warn() {}, error: console.error } });++++ await check('the files parse: numbered, each with its phase; 0001 is the initial expand', async () => {++ const files = parse(MIGRATIONS);++ assert.ok(files.length >= 1);++ assert.deepStrictEqual([files[0].id, files[0].name, files[0].phase, files[0].transaction], ['0001', 'initial', 'expand', true]);++ assert.deepStrictEqual(files.map((f) => f.id), [...files.map((f) => f.id)].sort(), 'applied in order');++ for (const f of files) if (f.phase === 'contract') assert.ok(f.after, `${f.file} names its expand`);++ });++++ await check('they apply once, in order, and a second run changes nothing', async () => {++ const first = await db.migrate({ dir: MIGRATIONS, log: quiet });++ assert.deepStrictEqual(first.applied.map((m) => m.id), parse(MIGRATIONS).map((m) => m.id));++ const schema = async () => db.many(sql`SELECT table_name, column_name, data_type, is_nullable, column_default FROM information_schema.columns++ WHERE table_schema = 'public' ORDER BY table_name, ordinal_position`);++ const before = await schema();++ const second = await db.migrate({ dir: MIGRATIONS, log: quiet });++ assert.deepStrictEqual([second.applied, second.held], [[], []]);++ assert.deepStrictEqual(await schema(), before, 'the schema is the same after a second run');++ assert.strictEqual(await db.value(sql`SELECT count(*) FROM ov_migrations`), parse(MIGRATIONS).length);++ });++++ await check('every table has a primary key; money and time columns have their types', async () => {++ const noKey = await db.many(sql`SELECT c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace++ WHERE n.nspname = 'public' AND c.relkind = 'r' AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.oid AND i.indisprimary)`);++ assert.deepStrictEqual(noKey, []);++ const type = async (t, c) => db.value(sql`SELECT data_type FROM information_schema.columns WHERE table_schema = 'public' AND table_name = ${t} AND column_name = ${c}`);++ for (const [t, c] of [['tip_interactions', 'amount'], ['tip_interactions', 'reversed_bits'], ['tip_goals', 'target_amount'], ['tip_goal_contributions', 'amount'], ['tip_interactions', 'next_transfer_at'], ['interaction_effects', 'next_attempt_at']]) {++ assert.strictEqual(await type(t, c), 'bigint', `${t}.${c}`);++ }++ for (const [t, c] of [['tip_interactions', 'created_at'], ['tip_interactions', 'settled_at'], ['overlay_deliveries', 'created_at'], ['api_idempotency', 'created_at']]) {++ assert.strictEqual(await type(t, c), 'timestamp with time zone', `${t}.${c}`);++ }++ for (const [t, c] of [['tip_interactions', 'test'], ['tip_interactions', 'transfer_due'], ['creator_tip_profiles', 'page_enabled'], ['overlay_deliveries', 'hidden']]) {++ assert.strictEqual(await type(t, c), 'boolean', `${t}.${c}`);++ }++ for (const [t, c] of [['tip_interactions', 'request'], ['overlay_deliveries', 'payload'], ['tips_event_outbox', 'envelope'], ['overlay_tokens', 'scopes']]) {++ assert.strictEqual(await type(t, c), 'jsonb', `${t}.${c}`);++ }++ assert.strictEqual(await type('api_idempotency', 'response'), 'text', 'replayed byte for byte');++ });++++ await check('nothing the SQLite release stored is left behind: every source table and column has a target', async () => {++ const s = new Database(':memory:');++ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));++ const from = (target) => (TABLES[target] && TABLES[target].from) || target;++ const targets = new Map();++ for (const r of await db.many(sql`SELECT table_name, column_name FROM information_schema.columns WHERE table_schema = 'public' AND table_name <> 'ov_migrations'`)) {++ const src = from(r.table_name);++ if (!targets.has(src)) targets.set(src, new Set());++ targets.get(src).add(r.column_name);++ }++ const missing = [];++ for (const { name } of s.prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'").all()) {++ if (name === 'settings') continue; // not carried on purpose: ov_migrations records the schema (0001_initial.sql)++ if (!targets.has(name)) { missing.push(name); continue; }++ for (const c of s.prepare(`PRAGMA table_info(${name})`).all()) if (!targets.get(name).has(c.name)) missing.push(`${name}.${c.name}`);++ }++ s.close();++ assert.deepStrictEqual(missing, [], 'no table or column dropped (dropColumns stays empty)');++ assert.ok(Object.values(TABLES).every((o) => !o.dropColumns), 'no dropColumns');++ });++++ await check('every query shape the code runs can be served by an index (no sequential scan of its table)', async () => {++ // With sequential scans switched off for the plan, a shape that has no usable index still plans a++ // Seq Scan (at a prohibitive cost): that is what this looks for.++ const shapes = {++ 'interactions of a creator, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND (created_at, id) < (now(), 'x') ORDER BY created_at DESC, id DESC LIMIT 51`],++ 'receipts of a supporter, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE supporter_subject = 's' AND NOT test ORDER BY created_at DESC, id DESC LIMIT 51`],++ 'the moderation queue by state': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state IN ('settled', 'reversed', 'pending') AND moderation = 'held' ORDER BY created_at DESC, id DESC LIMIT 51`],++ 'unpaid checkouts of a supporter': ['tip_interactions', sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = 's' AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= now()`],++ 'due transfers (the lease claim)': ['tip_interactions', sql`SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= 5 ORDER BY next_transfer_at LIMIT 10 FOR UPDATE SKIP LOCKED`],++ 'the supporters leaderboard': ['tip_interactions', sql`SELECT supporter_subject, SUM(amount - reversed_bits) FROM tip_interactions WHERE creator_subject = 'c' AND supporter_subject IS NOT NULL AND NOT test++ AND payment_state IN ('settled', 'reversed') AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible' GROUP BY supporter_subject`],++ 'recent public messages': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state = 'settled' AND message IS NOT NULL AND NOT test ORDER BY settled_at DESC, id DESC LIMIT 20`],++ 'an interaction by Billing transaction': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_txn_id = 't'`],++ 'an interaction by checkout intent': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_intent_id = 'i'`],++ 'an interaction by provider reference': ['tip_interactions', sql`SELECT 1 FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = 'r'`],++ 'goals of a creator': ['tip_goals', sql`SELECT * FROM tip_goals WHERE creator_subject = 'c' AND status = 'active' ORDER BY sort_order, created_at`],++ 'goal totals': ['tip_goal_contributions', sql`SELECT goal_id, SUM(amount - reversed_amount) FROM tip_goal_contributions WHERE goal_id = ANY('{a,b}'::text[]) GROUP BY goal_id`],++ 'the latest supporters of a goal': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE goal_id = 'g' ORDER BY id DESC LIMIT 10`],++ 'the contributions of an interaction': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = 'i'`],++ 'due effects (the lease claim)': ['interaction_effects', sql`SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= 5 ORDER BY id LIMIT 20 FOR UPDATE SKIP LOCKED`],++ 'the effects of a page of interactions': ['interaction_effects', sql`SELECT * FROM interaction_effects WHERE interaction_id = ANY('{a,b}'::text[]) ORDER BY interaction_id, id`],++ 'an overlay stream after a seq': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE creator_subject = 'c' AND seq > 5 AND status <> 'failed' AND NOT hidden ORDER BY seq LIMIT 200`],++ 'the delivery window sweep': ['overlay_deliveries', sql`SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < now() ORDER BY seq LIMIT 500`],++ 'the deliveries of an interaction': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE interaction_id = 'i'`],++ 'overlay tokens of a creator': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE creator_subject = 'c' ORDER BY revoked_at IS NOT NULL, created_at DESC`],++ 'an overlay token by its secret': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE token_hash = 'h'`],++ 'overlay configs of a creator': ['overlay_configs', sql`SELECT * FROM overlay_configs WHERE creator_subject = 'c' ORDER BY created_at`],++ 'a creator by handle': ['creator_tip_profiles', sql`SELECT * FROM creator_tip_profiles WHERE handle = 'h'`],++ 'switched-on pages (home, sitemap)': ['creator_tip_profiles', sql`SELECT handle FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`],++ 'the creators a person moderates': ['tip_moderators', sql`SELECT * FROM tip_moderators WHERE moderator_subject = 's' AND removed_at IS NULL`],++ 'open invitations of a creator': ['tip_moderator_invites', sql`SELECT * FROM tip_moderator_invites WHERE creator_subject = 'c' AND used_at IS NULL ORDER BY created_at`],++ 'the moderation log of a creator': ['tip_moderation_log', sql`SELECT * FROM tip_moderation_log WHERE creator_subject = 'c' ORDER BY id DESC LIMIT 100`],++ 'the weekly prune of stored answers': ['api_idempotency', sql`DELETE FROM api_idempotency WHERE created_at < now()`],++ 'an erasure scrubbing outbox copies': ['tips_event_outbox', sql`SELECT id FROM tips_event_outbox WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY('{a}'::text[])`],++ 'the rejected-events count': ['tips_event_outbox', sql`SELECT count(*) FROM tips_event_outbox WHERE rejected_at IS NOT NULL`],++ 'a Live goal through the import map': ['migration_maps', sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = '12'`],++ };++ const scans = [];++ for (const [name, [table, q]] of Object.entries(shapes)) {++ const plan = await db.tx(async (t) => {++ await t.query('SET LOCAL enable_seqscan = off');++ return t.value(sql`EXPLAIN (FORMAT JSON) ${q}`);++ });++ const nodes = [];++ const walk = (n) => { nodes.push(n); (n.Plans || []).forEach(walk); };++ walk(plan[0].Plan);++ if (nodes.some((n) => n['Node Type'] === 'Seq Scan' && n['Relation Name'] === table)) scans.push(name);++ }++ assert.deepStrictEqual(scans, []);++ });++++ await check('no runtime file loads better-sqlite3 (only the import tools may)', async () => {++ const offenders = [];++ const walk = (dir) => {++ for (const e of fs.readdirSync(dir, { withFileTypes: true })) {++ const p = path.join(dir, e.name);++ if (e.isDirectory()) walk(p);++ else if (e.name.endsWith('.js') && /require\(['"]better-sqlite3['"]\)|\.prepare\(|db\.transaction\(/.test(fs.readFileSync(p, 'utf8').replace(/(live|bdb|snap)\.prepare\(/g, ''))) offenders.push(path.relative(ROOT, p));++ }++ };++ walk(path.join(ROOT, 'server'));++ assert.deepStrictEqual(offenders, []);++ const loaders = fs.readdirSync(path.join(ROOT, 'scripts')).filter((f) => /require\(['"]better-sqlite3['"]\)/.test(fs.readFileSync(path.join(ROOT, 'scripts', f), 'utf8')));++ assert.deepStrictEqual(loaders, ['import-live.js'], "the Live import reads Live's and Billing's snapshot files; migrate-to-postgres.js reads Tips' through importSqlite");++ });++++ await db.close();++ done();++})().catch((e) => { console.error(e); process.exit(1); });@@@ -29,7 +29,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');const pay = (kind, body) => t.call('POST', { tip: '/api/v1/checkout', paid_message: '/api/v1/paid-messages', tts: '/api/v1/tts-requests' }[kind], { user: viewer, body: { creator: 'alex', ...body } });const state = async () => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts;const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);-- const moderated = (id) => t.outboxRows('tips.interaction.moderated').filter((e) => e.subject.id === id).map((e) => e.payload);++ const moderated = async (id) => (await t.outboxRows('tips.interaction.moderated')).filter((e) => e.subject.id === id).map((e) => e.payload);const cookie = (u) => `ov_token=${t.network.signUser(u)}`;const get = (p, u) => fetch(t.base + p, { headers: u ? { Cookie: cookie(u) } : {}, redirect: 'manual' }).then(async (r) => ({ status: r.status, headers: r.headers, text: await r.text() }));const post = (p, u, form) => fetch(t.base + p, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie(u) }, body: new URLSearchParams(form).toString() })@@@ -62,7 +62,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');assert.match(own.message, /BADWORD/, 'the creator reads the original');assert.strictEqual(own.moderation.filtered, true);assert.strictEqual(own.moderation.state, 'visible');-- const [ev] = moderated(id);++ const [ev] = await moderated(id);assert.deepStrictEqual(ev, { interaction_id: id, creator: { type: 'user', id: alex.subject }, action: 'filtered', by: 'filter', moderation_state: 'visible', cancelled_effects: [] });const tts = await pay('tts', { amount: 150, tts: { text: 'say badword now please' } });await domain.effects.drain();@@@ -74,7 +74,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');let heldId;await check('hold: nothing is shown, posted or read until someone reviews it; the money counts at once', async () => {await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { action: 'hold' } } });-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;++ const goalBefore = await goalNow();const s = await t.sse(`/overlay/${secret}/events`);await s.waitFor((e) => e.event === 'hello');const r = await pay('tts', { amount: 300, supporter_name: 'Viewer', tts: { text: 'read this badword aloud' } });@@@ -85,13 +86,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');s.close();assert.ok(!(await state()).some((x) => x.interaction_id === heldId));assert.strictEqual(jobsOf(heldId).length, 0, 'no chat line, no TTS');-- const i = domain.interactions.get(heldId);++ const i = await domain.interactions.get(t.db, heldId);assert.strictEqual(i.payment_state, 'settled');assert.strictEqual(i.moderation, 'held');assert.strictEqual(i.delivery_state, 'queued', 'waiting for review, not "delivered"');-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 300);-- assert.strictEqual(t.outboxRows('tips.interaction.ready').filter((e) => e.subject.id === heldId).length, 1);-- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);++ assert.strictEqual(await goalNow(), goalBefore + 300);++ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === heldId).length, 1);++ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);const q = await t.call('GET', '/api/v1/moderation?creator=alex&state=held', { user: alex });assert.deepStrictEqual(q.json.interactions.map((x) => x.id), [heldId]);assert.strictEqual(q.json.interactions[0].tts_text, 'read this badword aloud', 'reviewers read what was written');@@@ -108,7 +109,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');const url = inv.text.match(/value="(http:\/\/tips\.test\/moderate\/invite\/tmin_[^"]+)"/)[1];const path = url.replace('http://tips.test', '');assert.ok(!(await get('/dashboard', alex)).text.includes(url), 'shown once');-- assert.ok(!JSON.stringify(t.db.prepare('SELECT * FROM tip_moderator_invites').all()).includes(path.split('/').pop()), 'stored hashed');++ assert.ok(!JSON.stringify(await t.db.many('SELECT * FROM tip_moderator_invites')).includes(path.split('/').pop()), 'stored hashed');assert.match((await get(path)).headers.get('location'), /^\/auth\/login/);const page = await get(path, mod);assert.strictEqual(page.status, 200);@@@ -139,10 +140,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');assert.strictEqual(alert.data.tts.text, 'read this aloud', 'blocked words stay out of what is read');await domain.effects.drain();assert.deepStrictEqual(jobsOf(heldId).map((j) => j.effect).sort(), ['chat_line', 'tts']);-- const i = domain.interactions.get(heldId);++ const i = await domain.interactions.get(t.db, heldId);assert.strictEqual(i.moderation, 'visible');assert.strictEqual(i.delivery_state, 'delivered');-- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);++ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);});let hiddenId;@@@ -154,7 +155,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');const r = await pay('tts', { amount: 500, supporter_name: 'Viewer', message: 'nasty but not blocked', tts: { text: 'nasty but not blocked' } });hiddenId = r.json.interaction.id;const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === hiddenId);-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;++ const goalBefore = await goalNow();const h = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: mod, body: { reason: 'abuse' } });assert.strictEqual(h.status, 200, h.text);assert.strictEqual(h.json.changed, true);@@@ -164,11 +166,11 @@ const validModerated = payloadValidator('tips.interaction.moderated');s.close();await domain.effects.drain();assert.strictEqual(jobsOf(hiddenId).length, 0, 'nothing was posted or read');-- const i = domain.interactions.get(hiddenId);++ const i = await domain.interactions.get(t.db, hiddenId);assert.strictEqual(i.payment_state, 'settled');assert.strictEqual(i.delivery_state, 'cancelled');assert.strictEqual(billing.payable.get(alex.subject), payableBefore + 500, 'the payment stands');-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore, 'the goal keeps it');++ assert.strictEqual(await goalNow(), goalBefore, 'the goal keeps it');assert.ok(!(await state()).some((x) => x.interaction_id === hiddenId));const replay = await t.sse(`/overlay/${secret}/events`, { lastEventId: alert.id - 1 });await replay.waitFor((e) => e.event === 'hello');@@@ -177,13 +179,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');replay.close();const sup = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json;assert.ok(!sup.recent.some((v) => v.interaction_id === hiddenId));-- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');-- const [ev] = moderated(hiddenId);++ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');++ const [ev] = await moderated(hiddenId);assert.deepStrictEqual({ ...ev, cancelled_effects: ev.cancelled_effects.sort() }, { interaction_id: hiddenId, creator: { type: 'user', id: alex.subject }, action: 'hidden', by: 'moderator', moderation_state: 'hidden', cancelled_effects: ['chat_line', 'tts'] });assert.ok(!JSON.stringify(ev).includes('abuse') && !JSON.stringify(ev).includes(mod.subject), 'the event carries no note and no moderator');const again = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: alex });assert.strictEqual(again.json.changed, false);-- assert.strictEqual(moderated(hiddenId).length, 1, 'hiding twice publishes once');++ assert.strictEqual((await moderated(hiddenId)).length, 1, 'hiding twice publishes once');const receipt = (await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: viewer })).json.interaction;assert.strictEqual(receipt.moderation.state, 'hidden');assert.strictEqual(receipt.payment.state, 'settled');@@@ -196,7 +198,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');await domain.effects.drain();assert.strictEqual(jobsOf(hiddenId).length, 0, 'no late TTS');assert.ok((await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.recent.some((v) => v.interaction_id === hiddenId));-- assert.deepStrictEqual(moderated(hiddenId).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);++ assert.deepStrictEqual((await moderated(hiddenId)).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);});await check('who may moderate: the creator, their moderators, services with tips.interaction.moderate; nobody else', async () => {@@@ -208,7 +210,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.get'] })).status, 403);const svc = await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.moderate'], sub: 'svc:chat' });assert.strictEqual(svc.status, 200, svc.text);-- assert.strictEqual(moderated(id)[0].by, 'service');++ assert.strictEqual((await moderated(id))[0].by, 'service');assert.strictEqual((await get('/moderate/alex', stranger)).status, 404);assert.strictEqual((await t.call('GET', '/api/v1/moderation?creator=alex', { user: stranger })).status, 403);const log = await t.call('GET', '/api/v1/moderation/log?creator=alex', { user: alex });@@@ -227,10 +229,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');const r = await t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 150, message: 'later', pay_with: 'checkout', provider: 'stripe' } });const id = r.json.interaction.id;assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { user: alex })).status, 200);-- await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));++ await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));await new Promise((ok) => setTimeout(ok, 50));await domain.interactions.processDueTransfers();-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'settled');assert.strictEqual(i.delivery_state, 'cancelled');await domain.effects.drain();@@@ -242,7 +244,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { words: ['badword'], action: 'hold' } } });const r = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'a badword test' } });assert.strictEqual(r.json.interaction.moderation.state, 'held');-- assert.strictEqual(moderated(r.json.interaction.id).length, 0);++ assert.strictEqual((await moderated(r.json.interaction.id)).length, 0);const shown = await t.call('POST', `/api/v1/interactions/${r.json.interaction.id}/restore`, { user: alex });assert.strictEqual(shown.json.interaction.moderation.state, 'visible');assert.strictEqual((await state()).find((x) => x.interaction_id === r.json.interaction.id).message, 'a ******* test');@@@ -250,16 +252,16 @@ const validModerated = payloadValidator('tips.interaction.moderated');const r2 = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'another badword' } });const id2 = r2.json.interaction.id;await t.call('POST', `/api/v1/interactions/${id2}/hide`, { user: alex });-- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'cancelled');++ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'cancelled');await t.call('POST', `/api/v1/interactions/${id2}/restore`, { user: alex });await domain.effects.drain();-- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'delivered');++ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'delivered');assert.strictEqual(jobsOf(id2).length, 1);assert.ok((await state()).some((x) => x.interaction_id === id2));});await check('every event is a valid envelope with a valid payload', async () => {-- for (const e of t.outboxRows()) {++ for (const e of await t.outboxRows()) {assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);const v = e.event_type === 'tips.interaction.moderated' ? validModerated(e.payload) : payloadValidator(e.event_type)(e.payload);assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)} ${JSON.stringify(e.payload)}`);@@@ -1,6 +1,7 @@'use strict';// Track O: GET /metrics answers direct loopback callers only, labels requests by route template and--// carries the Tips gauges; /api/ready is 503 only when the database fails, and a missing Billing,++// carries the Tips gauges; /api/ready is 503 only when the database fails, reports the store that++// answered (db.ready()) and Valkey (valkey.ready(), skipped without VALKEY_URL), and a missing Billing,// Events or Network key degrades it instead.const assert = require('assert');const nodeHttp = require('http');@@@ -21,13 +22,17 @@ function get(base, p, headers = {}) {assert.strictEqual(r.status, 200, r.text);assert.strictEqual(r.json.status, 'ready');assert.strictEqual(r.json.service, 'tips');-- assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'network_jwks', 'billing', 'events']);++ assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'valkey', 'network_jwks', 'billing', 'events']);for (const [name, c] of Object.entries(r.json.checks)) {-- assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);++ // Without Valkey: skipped with its reason, never claimed "ok"; with it, a real PING.++ if (name === 'valkey' && !t.valkey) { assert.strictEqual(c.status, 'skipped', JSON.stringify(c)); assert.match(c.reason, /VALKEY_URL unset/); assert.deepStrictEqual(r.json.skipped, ['valkey']); }++ else assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);assert.strictEqual(c.required, name === 'db', name);assert.strictEqual(typeof c.latency_ms, 'number');assert.ok(Date.parse(c.checked_at));}++ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered, from a real round trip');++ assert.ok(r.json.checks.db.detail.pool && typeof r.json.checks.db.detail.pool.total === 'number');assert.ok(r.json.events_outbox && r.json.pending_deliveries);assert.strictEqual(r.headers.get('cache-control'), 'no-store');});@@@ -80,7 +85,7 @@ function get(base, p, headers = {}) {await check('a broken database makes the service unready (503); /metrics still answers', async () => {const d = await boot();-- d.db.close();++ await d.db.close();const r = await d.call('GET', '/api/ready', { token: null });assert.strictEqual(r.status, 503, r.text);assert.strictEqual(r.json.ready, false);@@@ -21,7 +21,7 @@ const { boot, check, done } = require('./helpers/app');tokenId = r.json.token.id;assert.match(secret, /^tovl_[A-Za-z0-9_-]{43}$/);assert.strictEqual(r.json.overlay_url, `http://tips.test/overlay/${secret}`);-- const row = t.db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(tokenId);++ const row = await t.db.maybe('SELECT * FROM overlay_tokens WHERE id = $1', [tokenId]);assert.notStrictEqual(row.token_hash, secret);assert.ok(!JSON.stringify(row).includes(secret), 'the secret is not stored');const list = await t.call('GET', '/api/v1/overlay-tokens', { user: alex });@@@ -48,16 +48,17 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(g.data.goal.id, goal.id);firstAlertSeq = alert.id;s.close();-- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r.json.interaction.id);++ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r.json.interaction.id]);assert.strictEqual(d.status, 'delivered');-- assert.strictEqual(t.outboxRows('tips.overlay.delivered').filter((e) => e.payload.delivery_id === d.id).length, 1);++ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).filter((e) => e.payload.delivery_id === d.id).length, 1);});await check('overlay replay (Last-Event-ID) resends but never charges, re-counts or re-emits', async () => {++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;const before = {-- transfers: billing.transfers().length, contributions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n,-- goal: domain.goals.present(domain.goals.get(goal.id)).current_amount, delivered: t.outboxRows('tips.overlay.delivered').length,-- interactions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, payable: billing.payable.get(alex.subject),++ transfers: billing.transfers().length, contributions: await t.db.value('SELECT count(*) FROM tip_goal_contributions'),++ goal: await goalNow(), delivered: (await t.outboxRows('tips.overlay.delivered')).length,++ interactions: await t.db.value('SELECT count(*) FROM tip_interactions'), payable: billing.payable.get(alex.subject),};for (let k = 0; k < 3; k++) {const s = await t.sse(`/overlay/${secret}/events`, { lastEventId: firstAlertSeq - 1 });@@@ -66,12 +67,12 @@ const { boot, check, done } = require('./helpers/app');s.close();}assert.strictEqual(billing.transfers().length, before.transfers);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n, before.contributions);-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal);-- assert.strictEqual(t.outboxRows('tips.overlay.delivered').length, before.delivered);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before.interactions);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions'), before.contributions);++ assert.strictEqual(await goalNow(), before.goal);++ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).length, before.delivered);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before.interactions);assert.strictEqual(billing.payable.get(alex.subject), before.payable);-- assert.ok(t.db.prepare('SELECT sends FROM overlay_deliveries WHERE seq = ?').get(firstAlertSeq).sends >= 4);++ assert.ok(await t.db.value('SELECT sends FROM overlay_deliveries WHERE seq = $1', [firstAlertSeq]) >= 4);});await check('a tip while no overlay is connected waits, is delivered on connect, else fails after the window', async () => {@@@ -83,20 +84,21 @@ const { boot, check, done } = require('./helpers/app');await new Promise((ok) => setTimeout(ok, 50));const r2 = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 12 } });t.clock.offset += 11 * 60 * 1000;-- const n = domain.overlays.sweepFailed();++ const n = await domain.overlays.sweepFailed();t.clock.offset = 0;assert.ok(n >= 1);-- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r2.json.interaction.id);++ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r2.json.interaction.id]);assert.strictEqual(d.status, 'failed');-- assert.strictEqual(t.outboxRows('tips.overlay.failed').filter((e) => e.payload.delivery_id === d.id).length, 1);++ assert.strictEqual((await t.outboxRows('tips.overlay.failed')).filter((e) => e.payload.delivery_id === d.id).length, 1);// The payment is untouched by the overlay failure.-- assert.strictEqual(domain.interactions.get(r2.json.interaction.id).payment_state, 'settled');++ assert.strictEqual((await domain.interactions.get(t.db, r2.json.interaction.id)).payment_state, 'settled');});await check('simulation runs the full path with test = 1, no Billing call, never counted', async () => {++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;const before = {-- transfers: billing.calls.length, goal: domain.goals.present(domain.goals.get(goal.id)).current_amount,-- totals: domain.interactions.totals(alex.subject), events: t.outboxRows().length,++ transfers: billing.calls.length, goal: await goalNow(),++ totals: await domain.interactions.totals(t.db, alex.subject), events: (await t.outboxRows()).length,};const s = await t.sse(`/overlay/${secret}/events`);await s.waitFor((e) => e.event === 'hello');@@@ -114,13 +116,13 @@ const { boot, check, done } = require('./helpers/app');const job = t.adapters.test.jobs.find((j) => j.interaction.id === i.id);assert.ok(job && job.test === true, 'chat effect delivered to the test adapter, flagged test');assert.strictEqual(billing.calls.length, before.transfers, 'no Billing call');-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal, 'goal unchanged');-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);-- const after = domain.interactions.totals(alex.subject);++ assert.strictEqual(await goalNow(), before.goal, 'goal unchanged');++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);++ const after = await domain.interactions.totals(t.db, alex.subject);assert.strictEqual(after.settled_via_billing, before.totals.settled_via_billing);assert.strictEqual(after.interactions, before.totals.interactions);assert.strictEqual(after.simulations, before.totals.simulations + 1);-- assert.strictEqual(t.outboxRows().length, before.events, 'no durable events for a simulation');++ assert.strictEqual((await t.outboxRows()).length, before.events, 'no durable events for a simulation');const denied = await t.call('POST', '/api/v1/simulate', { user: viewer, body: { creator: 'alex' } });assert.strictEqual(denied.status, 403);});@@@ -82,7 +82,7 @@ const { boot, check, done } = require('./helpers/app');const loc = r.headers.get('location');assert.match(loc, /^\/receipts\/tint_/);const id = loc.split('/').pop();-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');// Double submit (same form nonce): the same interaction, no second transfer.const before = billing.transfers().length;const again = await post('/alex/tip', viewer, form);@@@ -106,7 +106,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(r2.status, 403, 'a token minted for someone else does not work');const d = await post('/dashboard/profile', alex, { csrf: 'nope', page_enabled: '0' });assert.strictEqual(d.status, 403);-- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, true);++ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, true);});await check('the form reports Billing\'s refusal in words; checkout redirects to the provider', async () => {@@@ -138,10 +138,10 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(g.status, 303);const sim = await post('/dashboard/simulate', alex, { csrf, idem: 'x', kind: 'tip', amount: '100', supporter_name: 'Test supporter', message: 'test' });assert.strictEqual(sim.status, 303);-- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 1").get(alex.subject).n, 1);-- const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String(domain.profiles.byHandle('alex').revision) });++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE creator_subject = $1 AND test', [alex.subject]), 1);++ const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String((await domain.profiles.byHandle(t.db, 'alex')).revision) });assert.strictEqual(off.status, 303);-- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, false, 'an unchecked box switches the page off');++ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, false, 'an unchecked box switches the page off');assert.strictEqual((await get('/alex')).status, 404);});@@@ -28,7 +28,7 @@ function payloadValidator(type) {const tip = (user, body) => t.call('POST', '/api/v1/checkout', { user, body: { creator: 'alex', ...body } });const alertOf = async (id) => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts.find((a) => a.interaction_id === id);const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);-- const eventsOf = (type, id) => t.outboxRows(type).filter((e) => e.subject.id === id);++ const eventsOf = async (type, id) => (await t.outboxRows(type)).filter((e) => e.subject.id === id);let anonId;await check('anonymous: "Anonymous" to the creator, overlays, chat and events; the supporter keeps their receipt', async () => {@@@ -46,19 +46,19 @@ function payloadValidator(type) {assert.strictEqual(a.data.supporter_name, 'Anonymous');assert.strictEqual(a.data.message, 'from nobody');assert.ok(!JSON.stringify(a.data).includes('Viewer Real Name') && !JSON.stringify(a.data).includes(viewer.subject));-- const g = t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(anonId);-- assert.strictEqual(JSON.parse(g.payload).by, 'Anonymous');++ const g = await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [anonId]);++ assert.strictEqual(g.payload.by, 'Anonymous');await domain.effects.drain();const [job] = jobsOf(anonId);assert.deepStrictEqual(job.supporter, { name: 'Anonymous', subject: null });assert.strictEqual(job.text, 'Anonymous tipped 120 Vibes: from nobody');-- const [ready] = eventsOf('tips.interaction.ready', anonId);++ const [ready] = await eventsOf('tips.interaction.ready', anonId);assert.strictEqual(ready.payload.supporter, null);assert.strictEqual(ready.payload.supporter_name, 'Anonymous');assert.ok(contracts.validate('tips.interaction.ready', ready.payload).valid);-- assert.ok(!JSON.stringify(t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');++ assert.ok(!JSON.stringify(await t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');for (const as of [{ user: alex }, { cap: ['tips.interaction.get'] }]) {const v = (await t.call('GET', `/api/v1/interactions/${anonId}`, as)).json.interaction;@@@ -80,7 +80,7 @@ function payloadValidator(type) {assert.strictEqual(a.amount, null);assert.strictEqual(a.amount_hidden, true);assert.strictEqual(a.supporter_name, 'Viewer');-- assert.strictEqual(JSON.parse(t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(hiddenId).payload).by, null,++ assert.strictEqual((await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [hiddenId])).payload.by, null,'the goal update does not pin the jump on them');await domain.effects.drain();const [job] = jobsOf(hiddenId);@@@ -88,7 +88,7 @@ function payloadValidator(type) {assert.strictEqual(job.interaction.amount, null);assert.strictEqual(job.privacy.hide_amount, true);assert.strictEqual((await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: alex })).json.interaction.amount, 222);-- assert.strictEqual(eventsOf('tips.interaction.ready', hiddenId)[0].payload.amount, 222);++ assert.strictEqual((await eventsOf('tips.interaction.ready', hiddenId))[0].payload.amount, 222);});await check('an overlay\'s minimum still applies to a hidden amount (without revealing it)', async () => {@@@ -183,10 +183,10 @@ function payloadValidator(type) {body: new URLSearchParams({ csrf, idem: 'form-privacy-00001', kind: 'tip', amount: '15', message: 'shh', pay_with: 'credit', supporter_name: 'Other', anonymous: '1', private_message: '1' }).toString(),});assert.strictEqual(res.status, 303);-- const i = domain.interactions.get(res.headers.get('location').split('/').pop());-- assert.strictEqual(i.anonymous, 1);-- assert.strictEqual(i.private_message, 1);-- assert.strictEqual(i.hide_amount, 0);++ const i = await domain.interactions.get(t.db, res.headers.get('location').split('/').pop());++ assert.strictEqual(i.anonymous, true);++ assert.strictEqual(i.private_message, true);++ assert.strictEqual(i.hide_amount, false);});await check('export: a supporter downloads their own tips, with messages and choices, nobody else\'s', async () => {@@@ -207,14 +207,15 @@ function payloadValidator(type) {});await check('erasure: the person goes, the money record stays; tips.interaction.erased redacts the earlier events', async () => {-- const totalsBefore = domain.interactions.totals(alex.subject);-- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;++ const totalsBefore = await domain.interactions.totals(t.db, alex.subject);++ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;++ const goalBefore = await goalNow();const buyer = t.network.newUser('buyer');billing.fund(buyer.subject, 1000);const kept = await tip(buyer, { amount: 150, pay_with: 'checkout', provider: 'stripe', message: 'pending one' });const done1 = await tip(buyer, { amount: 40, message: 'my secret words', supporter_name: 'Buyer Person' });-- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM api_idempotency WHERE key LIKE ?').get(`${buyer.subject}:%`).n >= 2);-- const unsent = eventsOf('tips.interaction.ready', done1.json.interaction.id)[0];++ assert.ok(await t.db.value('SELECT count(*) FROM api_idempotency WHERE key LIKE $1', [`${buyer.subject}:%`]) >= 2);++ const unsent = (await eventsOf('tips.interaction.ready', done1.json.interaction.id))[0];assert.strictEqual(unsent.payload.supporter.id, buyer.subject);const r = await t.call('POST', '/api/v1/me/erase', { user: buyer });@@@ -223,36 +224,36 @@ function payloadValidator(type) {assert.strictEqual(r.json.kept_pending, 1);assert.ok(r.json.stored_answers_removed >= 2);-- const i = domain.interactions.get(done1.json.interaction.id);++ const i = await domain.interactions.get(t.db, done1.json.interaction.id);assert.strictEqual(i.supporter_subject, null);assert.strictEqual(i.supporter_name, null);assert.strictEqual(i.message, null);assert.ok(i.erased_at);assert.strictEqual(i.amount, 40);assert.ok(i.billing_txn_id, 'the Billing reference stays');-- assert.strictEqual(domain.interactions.get(kept.json.interaction.id).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');++ assert.strictEqual((await domain.interactions.get(t.db, kept.json.interaction.id)).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');const a = await alertOf(i.id);assert.strictEqual(a.supporter_name, 'Anonymous');assert.strictEqual(a.message, null);-- const row = t.db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(i.id);++ const row = await t.db.maybe('SELECT * FROM tip_interactions WHERE id = $1', [i.id]);assert.ok(!JSON.stringify(row).includes('my secret words') && !JSON.stringify(row).includes('Buyer Person') && !JSON.stringify(row).includes(buyer.subject));-- const everything = JSON.stringify(t.db.prepare('SELECT * FROM paid_messages').all()) + JSON.stringify(t.db.prepare('SELECT payload FROM overlay_deliveries').all())-- + JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all());++ const everything = JSON.stringify(await t.db.many('SELECT * FROM paid_messages')) + JSON.stringify(await t.db.many('SELECT payload FROM overlay_deliveries'))++ + JSON.stringify(await t.db.many('SELECT * FROM api_idempotency'));assert.ok(!everything.includes('my secret words') && !everything.includes('Buyer Person'));-- const scrubbed = eventsOf('tips.interaction.ready', i.id)[0];++ const scrubbed = (await eventsOf('tips.interaction.ready', i.id))[0];assert.strictEqual(scrubbed.payload.supporter, null, 'the local outbox copy no longer names them');assert.strictEqual(scrubbed.payload.supporter_name, 'Anonymous');-- const [erased] = eventsOf('tips.interaction.erased', i.id);++ const [erased] = await eventsOf('tips.interaction.erased', i.id);assert.deepStrictEqual(erased.payload.redacts, { subject_type: 'interaction', subject_ids: [i.id] });const v = payloadValidator('tips.interaction.erased')(erased.payload);assert.ok(v.valid, JSON.stringify(v.errors));assert.ok(contracts.validate('events.event-envelope@1', erased).valid);-- const totalsAfter = domain.interactions.totals(alex.subject);++ const totalsAfter = await domain.interactions.totals(t.db, alex.subject);assert.strictEqual(totalsAfter.settled_via_billing, totalsBefore.settled_via_billing + 40, 'the erased tip still counts');assert.strictEqual(totalsAfter.settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');-- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 40, 'the goal keeps the money');++ assert.strictEqual(await goalNow(), goalBefore + 40, 'the goal keeps the money');const receipts = (await t.call('GET', '/api/v1/interactions', { user: buyer })).json.interactions;assert.deepStrictEqual(receipts.map((x) => x.id), [kept.json.interaction.id]);});@@@ -265,17 +266,17 @@ function payloadValidator(type) {const post = (form) => fetch(`${t.base}/receipts/erase`, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie }, body: new URLSearchParams(form).toString() });assert.strictEqual((await post({ csrf, idem: 'form-erase-000001' })).status, 403, 'no confirmation');assert.strictEqual((await post({ csrf: 'forged', idem: 'form-erase-000002', confirm: '1' })).status, 403);-- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n > 0);++ assert.ok(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]) > 0);const ok = await post({ csrf, idem: 'form-erase-000003', confirm: '1' });assert.strictEqual(ok.status, 303);assert.match(ok.headers.get('location'), /^\/receipts\?done=/);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n, 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]), 0);const lb = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.leaderboard;assert.ok(!lb.some((x) => x.name === 'Other'), 'an erased supporter leaves the leaderboard');});await check('every event is a valid envelope with a valid payload', async () => {-- for (const e of t.outboxRows()) {++ for (const e of await t.outboxRows()) {assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);const v = payloadValidator(e.event_type)(e.payload);assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)}`);nnew file mode 100644@@@ -0,0 +1,65 @@++'use strict';++// The hot list routes run a bounded number of queries, whatever the page size (no N+1): the receipts++// and creator lists (GET /api/v1/interactions, the hottest), goals, the supporters page and an++// overlay's /state. Counted on the database handle (db.stats().queries) around one request each.++const assert = require('assert');++const { boot, check, done } = require('./helpers/app');++++(async () => {++ const t = await boot();++ const { billing } = t;++ const alex = await t.creator('alex', { settings: { page: { supporters_page: true, supporters_messages: true, goal_supporters: true } } });++ const fans = ['ann', 'bob', 'cat', 'dan', 'eve'].map((n) => t.network.newUser(n));++ for (const f of fans) billing.fund(f.subject, 10_000);++ const tip = (f, body) => t.call('POST', '/api/v1/checkout', { user: f, body: { creator: 'alex', amount: 10, ...body } });++ const goals = [];++ for (const title of ['Desk', 'Chair', 'Lamp']) goals.push((await t.call('POST', '/api/v1/goals', { user: alex, body: { title, target_amount: 5000 } })).json.goal);++ // Several tips per supporter, messages, goal contributions: every list has many rows.++ for (let k = 0; k < 3; k++) for (const f of fans) assert.strictEqual((await tip(f, { message: `tip ${k} from ${f.username}`, goal_id: goals[k].id })).status, 201);++ const token = (await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'] } })).json.secret;++++ /** Queries one request runs. */++ async function queries(method, p, opts) {++ const before = t.db.stats().queries;++ const r = await t.call(method, p, opts);++ assert.strictEqual(r.status, 200, `${p}: ${r.text}`);++ return { n: t.db.stats().queries - before, r };++ }++++ await check('GET /api/v1/interactions (the creator\'s list, with effects): 3 queries for 1 row or 15', async () => {++ const one = await queries('GET', '/api/v1/interactions?as=creator&limit=1', { user: alex });++ const all = await queries('GET', '/api/v1/interactions?as=creator&limit=50', { user: alex });++ assert.strictEqual(one.r.json.interactions.length, 1);++ assert.strictEqual(all.r.json.interactions.length, 15);++ assert.ok(all.r.json.interactions.every((i) => Array.isArray(i.effects) && i.effects.length >= 1));++ assert.deepStrictEqual([one.n, all.n], [3, 3], 'the page, the creators\' filters, the effects');++ });++++ await check('GET /api/v1/interactions (a supporter\'s receipts; a service\'s view of a creator): bounded too', async () => {++ const mine = await queries('GET', '/api/v1/interactions', { user: fans[0] });++ assert.strictEqual(mine.r.json.interactions.length, 3);++ assert.strictEqual(mine.n, 2, 'the page and the filters (no effects for a supporter)');++ const svc = await queries('GET', '/api/v1/interactions?creator=alex&limit=50', { cap: ['tips.interaction.list'] });++ assert.strictEqual(svc.r.json.interactions.length, 15);++ assert.strictEqual(svc.n, 4, 'the creator, the page, the filters, the effects');++ });++++ await check('goals, the supporters page and an overlay\'s state: a fixed number of queries for any number of rows', async () => {++ const pub = await queries('GET', '/api/v1/goals?creator=alex', { token: null });++ assert.strictEqual(pub.r.json.goals.length, 3);++ assert.ok(pub.r.json.goals.every((g) => g.supporters.length === 5), 'each goal lists its supporters');++ assert.strictEqual(pub.n, 4, 'the profile, the goals, their totals, their supporters');++ const sup = await queries('GET', '/api/v1/profiles/alex/supporters', { token: null });++ assert.strictEqual(sup.r.json.leaderboard.length, 5);++ assert.strictEqual(sup.r.json.recent.length, 15);++ assert.strictEqual(sup.n, 3, 'the profile, the leaderboard, the recent messages');++ const before = t.db.stats().queries;++ const state = await (await fetch(`${t.base}/overlay/${token}/state`)).json();++ assert.strictEqual(state.alerts.length, 15);++ assert.strictEqual(state.goals.length, 3);++ assert.strictEqual(t.db.stats().queries - before, 4, 'the token, the goals, their totals, the alerts');++ });++++ await t.close();++ done();++})().catch((e) => { console.error(e); process.exit(1); });@@@ -30,13 +30,13 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(d.status, 200);assert.strictEqual(d.json.outcome, 'recorded', 'recorded on its own');}-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'pending', 'the paid message is still unpaid');-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM paid_messages WHERE interaction_id = ?').get(id).n, 0);-- const spoof = domain.interactions.byBillingTxn(cases[0].payload.transaction_id);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM paid_messages WHERE interaction_id = $1', [id]), 0);++ const spoof = await domain.interactions.byBillingTxn(t.db, cases[0].payload.transaction_id);assert.strictEqual(spoof.amount, 1);assert.strictEqual(spoof.origin, 'billing');-- assert.strictEqual(domain.interactions.totals(alex.subject).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');++ assert.strictEqual((await domain.interactions.totals(t.db, alex.subject)).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');});await check('overlay stream floods: a token opens a bounded number of streams at once', async () => {@@@ -83,7 +83,7 @@ const { boot, check, done } = require('./helpers/app');tts: { text: '<speak><break time="9s"/>hello world</speak>' },} });assert.strictEqual(r.status, 201, r.text);-- const i = domain.interactions.get(r.json.interaction.id);++ const i = await domain.interactions.get(t.db, r.json.interaction.id);assert.strictEqual(i.supporter_name, 'Viewer');assert.strictEqual(i.message, 'hi there');await domain.effects.drain();@@@ -92,6 +92,24 @@ const { boot, check, done } = require('./helpers/app');assert.ok(!/[]/.test(JSON.stringify(job)));});++ await check('text PostgreSQL cannot store (NUL, an unpaired surrogate) is cleaned, from people and from Billing events alike', async () => {++ const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 6, message: 'a\u0000b \ud83d', supporter_name: 'N\u0000ul' } });++ assert.strictEqual(r.status, 201, r.text);++ const i = await domain.interactions.get(t.db, r.json.interaction.id);++ assert.deepStrictEqual([i.message, i.supporter_name], ['ab �', 'Nul']);++ const tts = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'say \udc00 this\u0000' } } });++ assert.strictEqual(tts.status, 201, tts.text);++ const cfg = await t.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', template: '{name} \ud800 tipped' } });++ assert.strictEqual(cfg.status, 201, cfg.text);++ // A Billing event is redelivered until it is accepted: its text must never make it unstorable.++ const ev = billing.foreignDonation({ from: other.subject, to: alex.subject, amount: 9, message: 'from\u0000 billing \ud83d' });++ const d = await t.deliver(ev);++ assert.deepStrictEqual([d.status, d.json.outcome], [200, 'recorded']);++ assert.strictEqual((await domain.interactions.byBillingTxn(t.db, ev.payload.transaction_id)).message, 'from billing �');++ const cursor = Buffer.from(JSON.stringify(['1', 'x'])).toString('base64url');++ assert.strictEqual((await t.call('GET', `/api/v1/interactions?cursor=${cursor}`, { user: viewer })).status, 422, 'a cursor PostgreSQL could not read is refused, not a 500');++ });++await check('an overlay secret is shown once: an Idempotency-Key replay does not return it, the database never holds it', async () => {const first = await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { label: 'replay' }, key: 'overlay-key-0001' });assert.match(first.json.secret, /^tovl_/);@@@ -99,7 +117,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(replay.headers.get('idempotent-replayed'), 'true');assert.strictEqual(replay.json.token.id, first.json.token.id);assert.strictEqual(replay.json.secret, null);-- const everything = JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all()) + JSON.stringify(t.db.prepare('SELECT * FROM overlay_tokens').all());++ const everything = JSON.stringify(await t.db.many('SELECT * FROM api_idempotency')) + JSON.stringify(await t.db.many('SELECT * FROM overlay_tokens'));assert.ok(!everything.includes(first.json.secret));});@@@ -38,21 +38,21 @@ const { boot, check, done } = require('./helpers/app');// Same transaction republished under a new event id (a replay): still one interaction.const c = await t.deliver({ ...ev, event_id: billing.envelope('x', {}, 'x').event_id });assert.strictEqual(c.json.duplicate, false);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 1);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 1);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 1);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 1);});await check('a duplicate provider webhook (Billing settles once) → one logical interaction for a donation Tips did not start', async () => {// Billing turned one PowerChat site-routed tip into one transaction; Events may deliver it more than once.const ev = billing.foreignDonation({ from: null, to: alex.subject, amount: 300, provider: 'powerchat', message: 'from PowerChat' });for (let k = 0; k < 3; k++) await t.deliver(k === 2 ? { ...ev, event_id: billing.envelope('x', {}, 'x').event_id } : ev);-- const rows = t.db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').all(ev.payload.transaction_id);++ const rows = await t.db.many('SELECT * FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);assert.strictEqual(rows.length, 1);assert.strictEqual(rows[0].origin, 'billing');assert.strictEqual(rows[0].funding, 'provider');assert.strictEqual(rows[0].payment_state, 'settled');// Tips did not start it, so Tips does not announce it in chat a second time — overlay only.-- const effects = t.db.prepare('SELECT effect FROM interaction_effects WHERE interaction_id = ?').all(rows[0].id).map((e) => e.effect);++ const effects = (await t.db.many('SELECT effect FROM interaction_effects WHERE interaction_id = $1', [rows[0].id])).map((e) => e.effect);assert.deepStrictEqual(effects, ['overlay_alert']);assert.strictEqual(rows[0].delivery_state, 'delivered');});@@@ -60,16 +60,16 @@ const { boot, check, done } = require('./helpers/app');await check('delivery runs after settlement; a delivery failure never touches payment state', async () => {const test = t.adapters.test;await domain.effects.drain(); // the first tip delivers normally-- assert.strictEqual(domain.interactions.get(tipId).delivery_state, 'delivered');++ assert.strictEqual((await domain.interactions.get(t.db, tipId)).delivery_state, 'delivered');test.state.failWith = 'chat is down';test.state.permanent = true;const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 40 } });const id = r.json.interaction.id;await domain.effects.drain();-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'settled');assert.strictEqual(i.delivery_state, 'failed');-- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((e) => e.subject.id === id).length, 1);++ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((e) => e.subject.id === id).length, 1);test.state.failWith = null;assert.strictEqual(test.jobs.find((j) => j.interaction.id === tipId).text, 'Viewer tipped 250 Vibes: great stream');});@@@ -80,13 +80,13 @@ const { boot, check, done } = require('./helpers/app');const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 10 } });const id = r.json.interaction.id;await domain.effects.drain();-- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);++ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);assert.strictEqual(e.state, 'queued');assert.strictEqual(e.attempts, 1);test.state.failWith = null;t.clock.offset += 60_000;await domain.effects.drain();-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');t.clock.offset = 0;});@@@ -96,21 +96,21 @@ const { boot, check, done } = require('./helpers/app');const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 400 } });const id = r.json.interaction.id;await domain.effects.drain();-- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 400);++ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 400);const rev = billing.refund(r.json.interaction.payment.billing_txn_id);const d = await t.deliver(rev);assert.strictEqual(d.json.outcome, 'reversed');-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'reversed');assert.strictEqual(i.delivery_state, 'delivered');assert.strictEqual(i.reversed_bits, 400);-- assert.ok(t.db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE interaction_id = ? AND state = 'delivered'").get(id).n >= 2);-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0);++ assert.ok(await t.db.value("SELECT count(*) FROM interaction_effects WHERE interaction_id = $1 AND state = 'delivered'", [id]) >= 2);++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0);// The same reversal again changes nothing.const again = await t.deliver({ ...rev, event_id: billing.envelope('x', {}, 'x').event_id });assert.strictEqual(again.json.outcome, 'duplicate_reversal');-- assert.strictEqual(domain.interactions.get(id).reversed_bits, 400);++ assert.strictEqual((await domain.interactions.get(t.db, id)).reversed_bits, 400);await t.call('POST', `/api/v1/goals/${g.json.goal.id}/close`, { user: alex });});@@@ -118,12 +118,12 @@ const { boot, check, done } = require('./helpers/app');const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 20 } });const id = r.json.interaction.id;await t.deliver(billing.refund(r.json.interaction.payment.billing_txn_id));-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'reversed');assert.strictEqual(i.delivery_state, 'cancelled');await domain.effects.drain();assert.ok(!t.adapters.test.jobs.some((j) => j.interaction.id === id), 'nothing delivered after the reversal');-- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === id).length, 1);++ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === id).length, 1);});await check('insufficient credit: payment failed, nothing delivered, a replay of the key answers the same', async () => {@@@ -134,7 +134,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(r.json.details.interaction.payment.state, 'failed');const again = await t.call('POST', '/api/v1/checkout', { user: poor, body: { creator: 'alex', amount: 50 }, key: 'poor-key-00001' });assert.strictEqual(again.status, 409);-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(poor.subject).n, 1);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [poor.subject]), 1);});await check('self-tipping is refused by subject', async () => {@@@ -151,18 +151,18 @@ const { boot, check, done } = require('./helpers/app');assert.match(r.json.checkout_url, /^https:\/\/checkout\.stripe\.test\/pi_/);const id = r.json.interaction.id;assert.strictEqual(r.json.interaction.payment.state, 'pending');-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0, 'pending never counts');-- const intentId = domain.interactions.get(id).billing_intent_id;++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0, 'pending never counts');++ const intentId = (await domain.interactions.get(t.db, id)).billing_intent_id;assert.strictEqual(billing.calls.find((c) => c.url === '/api/v1/intents' && c.body.subject.id === buyer.subject).key, `tips:intent:${id}`);const purchase = billing.settlePurchase(intentId);const d = await t.deliver(purchase);assert.strictEqual(d.json.outcome, 'funded');await new Promise((ok) => setTimeout(ok, 100));await domain.interactions.processDueTransfers();-- const i = domain.interactions.get(id);++ const i = await domain.interactions.get(t.db, id);assert.strictEqual(i.payment_state, 'settled');assert.strictEqual(i.funding_txn_id, purchase.payload.transaction_id);-- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 300);++ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 300);// The purchase event again, and the transfer's own event: nothing new.assert.strictEqual((await t.deliver(purchase)).json.duplicate, true);const own = billing.events.find((e) => e.payload.metadata && e.payload.metadata.target && e.payload.metadata.target.id === id);@@@ -188,7 +188,7 @@ const { boot, check, done } = require('./helpers/app');t.clock.offset += 5000;await domain.interactions.processDueTransfers();t.clock.offset = 0;-- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');++ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');});await check('paid messages, TTS and media requests exist only after settlement', async () => {@@@ -205,17 +205,17 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(r.status, 201, r.text);ids.push(r.json.interaction.id);}-- const count = () => t.db.prepare('SELECT (SELECT COUNT(*) FROM paid_messages WHERE interaction_id IN (?, ?, ?)) + (SELECT COUNT(*) FROM paid_media_requests WHERE interaction_id IN (?, ?, ?)) AS n').get(...ids, ...ids).n;-- assert.strictEqual(count(), 0);-- for (const id of ids) await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));++ const count = () => t.db.value('SELECT (SELECT count(*) FROM paid_messages WHERE interaction_id = ANY($1)) + (SELECT count(*) FROM paid_media_requests WHERE interaction_id = ANY($1))', [ids]);++ assert.strictEqual(await count(), 0);++ for (const id of ids) await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));await new Promise((ok) => setTimeout(ok, 100));await domain.interactions.processDueTransfers();-- assert.strictEqual(count(), 3);-- const pm = t.db.prepare('SELECT * FROM paid_messages WHERE interaction_id = ?').get(ids[0]);++ assert.strictEqual(await count(), 3);++ const pm = await t.db.maybe('SELECT * FROM paid_messages WHERE interaction_id = $1', [ids[0]]);assert.strictEqual(pm.highlight_seconds, 60);await domain.effects.drain();-- assert.strictEqual(t.db.prepare('SELECT status FROM paid_messages WHERE interaction_id = ?').get(ids[1]).status, 'delivered');-- assert.strictEqual(t.db.prepare('SELECT status FROM paid_media_requests WHERE interaction_id = ?').get(ids[2]).status, 'accepted');++ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_messages WHERE interaction_id = $1', [ids[1]])).status, 'delivered');++ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_media_requests WHERE interaction_id = $1', [ids[2]])).status, 'accepted');const tts = t.adapters.test.jobs.find((j) => j.effect === 'tts' && j.interaction.id === ids[1]);assert.deepStrictEqual(tts.tts, { text: 'read me please', voice: 'amy' });});@@@ -230,7 +230,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(a.json.interaction.id, b.json.interaction.id);const denied = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.checkout.create'], body: { ...body, provider_ref: 'x2' } });assert.strictEqual(denied.status, 403);-- const totals = domain.interactions.totals(alex.subject);++ const totals = await domain.interactions.totals(t.db, alex.subject);assert.strictEqual(totals.external, 500);});@@@ -246,7 +246,7 @@ const { boot, check, done } = require('./helpers/app');assert.strictEqual(proxied.status, 404);const forged = await t.deliver({ ...ev, source: 'live' });assert.strictEqual(forged.json.outcome, 'ignored:source');-- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 0);++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 0);});await check('events: ready/goal events are relayed to OpenVibe.Events with the tips service token', async () => {nnew file mode 100644@@@ -0,0 +1,189 @@++'use strict';++// The one-time import (ADR-035; scripts/migrate-to-postgres.js): a SQLite file with the last SQLite++// release's schema (test/fixtures/sqlite-v3-schema.sql) and rows of every table goes through++// openvibe-sdk's importSqlite into the migrated schema: report.ok, every row counted and checksummed,++// values converted by column type (0/1 → boolean, ISO text → timestamptz, JSON text → jsonb), identities++// kept and continued, the SQLite file untouched; and the service then serves the imported data.++const assert = require('assert');++const crypto = require('crypto');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { execFileSync } = require('child_process');++const Database = require('better-sqlite3');++const { ids } = require('openvibe-contracts');++const { importSqlite } = require('openvibe-sdk/db');++const { testDb } = require('./helpers/db');++const { boot, check, done } = require('./helpers/app');++const { importOptions } = require('../scripts/migrate-to-postgres');++++const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-import-'));++const quiet = { log() {} };++const creator = ids.newId('user');++const supporter = ids.newId('user');++const T = (s) => `2026-09-${s}Z`; // an ISO time, as the SQLite release wrote them++++/** A SQLite file as the SQLite release left it: its schema and a few rows in every table. */++function fixture(file) {++ const s = new Database(file);++ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));++ const ins = (table, row) => s.prepare(`INSERT INTO ${table} (${Object.keys(row).join(', ')}) VALUES (${Object.keys(row).map(() => '?').join(', ')})`).run(...Object.values(row));++ ins('settings', { id: 1, schema_version: 3, created_at: T('01T00:00:00.000') });++ ins('creator_tip_profiles', {++ creator_subject: creator, handle: 'alex', display_name: 'Alex', page_enabled: 1, accepting: 1, tts_enabled: 0, media_requests_enabled: 1,++ revision: 4, created_at: T('01T10:00:00.000'), updated_at: T('02T11:30:00.250'),++ page_settings: '{"goal_amounts":false,"supporters_page":true}', filter_words: '["badword","two words"]', filter_action: 'hold', filter_links: 0,++ });++ const tip = (id, fields) => ins('tip_interactions', {++ id, creator_subject: creator, supporter_subject: supporter, supporter_name: 'Viewer', kind: 'tip', amount: 250, funding: 'credit', settlement: 'billing',++ payment_state: 'settled', delivery_state: 'delivered', created_at: T('03T12:00:00.000'), updated_at: T('03T12:00:01.000'), settled_at: T('03T12:00:00.500'), ...fields,++ });++ tip('tint_01', { billing_txn_id: 'txn_1', request: '{"goal_id":"tgoal_01"}', target: '{"service":"live","type":"stream","id":"42"}', message: 'great stream' });++ // Two things SQLite took and PostgreSQL does not: a NUL in text, an unpaired surrogate inside JSON.++ tip('tint_02', { message: 'nul\u0000here', request: '{"legacy_message":"cut \\ud83d","n":1}', billing_txn_id: 'txn_2', payment_state: 'reversed', reversal_txn_ids: '["txn_9"]', reversed_bits: 100, reversed_at: T('04T09:00:00.000'), anonymous: 1, hide_amount: 1, test: 0 });++ tip('tint_03', { kind: 'paid_message', amount: 500, message: 'hello chat', funding: 'checkout', payment_state: 'pending', delivery_state: 'awaiting_payment',++ settled_at: null, billing_intent_id: 'pi_1', checkout_url: 'https://checkout.example/pi_1', idempotency_key: 'api:usr:k1', transfer_due: 1, transfer_attempts: 2, next_transfer_at: 1790600000123 });++ tip('tint_04', { supporter_subject: null, supporter_name: null, erased_at: T('05T08:00:00.000'), moderation: 'hidden', moderated_by: 'filter', filtered: 1, test: 1, origin: 'import',++ settlement: 'imported', legacy_source: 'live:transactions:7', provider: 'powerchat', provider_ref: 'live-chat:3', request: '{}' });++ ins('tip_goals', { id: 'tgoal_01', creator_subject: creator, title: 'Desk', target_amount: 1000, status: 'active', opening_amount: 50, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000'), reached_at: null });++ ins('tip_goal_contributions', { id: 7, goal_id: 'tgoal_01', interaction_id: 'tint_01', amount: 250, reversed_amount: 0, created_at: T('03T12:00:00.600'), updated_at: T('03T12:00:00.600') });++ ins('tip_goal_contributions', { id: 12, goal_id: 'tgoal_01', interaction_id: 'tint_02', amount: 250, reversed_amount: 100, created_at: T('03T12:00:00.700'), updated_at: T('04T09:00:00.000') });++ ins('paid_messages', { id: 'tpm_01', interaction_id: 'tint_01', creator_subject: creator, kind: 'paid_message', text: 'hello', highlight_seconds: 30, status: 'delivered', chat_ref: 'null', test: 0, created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });++ ins('paid_media_requests', { id: 'tmr_01', interaction_id: 'tint_02', creator_subject: creator, url: 'https://youtu.be/x', provider: 'youtube', status: 'accepted', queue_ref: '{"queue_id":5}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });++ ins('overlay_configs', { id: 'tovc_01', creator_subject: creator, kind: 'alerts', name: 'Alerts', settings: '{"min_amount":1,"sound_url":null}', revision: 2, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000') });++ ins('overlay_tokens', { id: 'tovt_01', creator_subject: creator, token_hash: 'a'.repeat(64), scopes: '["alerts","goals"]', config_id: 'tovc_01', created_by: creator, created_at: T('02T00:00:00.000'), revoked_at: T('06T00:00:00.000') });++ ins('overlay_deliveries', { seq: 3, id: 'tovd_03', creator_subject: creator, kind: 'alert', interaction_id: 'tint_01', payload: '{"interaction_id":"tint_01","amount":250}', status: 'delivered', sends: 4, dedupe_key: 'alert:tint_01', created_at: T('03T12:00:01.000'), delivered_at: T('03T12:00:01.500'), hidden: 0 });++ ins('overlay_deliveries', { seq: 9, id: 'tovd_09', creator_subject: creator, kind: 'goal', goal_id: 'tgoal_01', payload: '{"goal":{"id":"tgoal_01"},"by":null}', status: 'pending', dedupe_key: 'goal:tgoal_01:r2', created_at: T('03T12:00:01.000'), hidden: 1 });++ ins('interaction_effects', { id: 4, interaction_id: 'tint_01', effect: 'chat_line', adapter: 'live-chat', state: 'delivered', attempts: 1, result: '{"ref":{"chat_message_id":1}}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });++ ins('interaction_effects', { id: 5, interaction_id: 'tint_01', effect: 'overlay_alert', adapter: 'overlay', state: 'delivered', result: 'null', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });++ ins('interaction_effects', { id: 6, interaction_id: 'tint_03', effect: 'paid_message', adapter: 'live-chat', state: 'queued', attempts: 2, next_attempt_at: 1790600000999, last_error: 'Live 503', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });++ ins('migration_maps', { id: 2, source: 'live', source_table: 'transactions', source_id: '7', target_type: 'interaction', target_id: 'tint_04', status: 'imported', run_id: 'timp_1', created_at: T('01T00:00:00.000'), updated_at: T('01T00:00:00.000') });++ ins('import_runs', { id: 'timp_1', source: 'live', dry_run: 0, started_at: T('01T00:00:00.000'), finished_at: T('01T00:00:05.000'), report: '{"counts":{"imported":1},"holds":[]}' });++ ins('tip_moderators', { creator_subject: creator, moderator_subject: supporter, name: 'Mod', added_by: 'invite:tmin_1', created_at: T('02T00:00:00.000') });++ ins('tip_moderator_invites', { id: 'tmin_1', creator_subject: creator, token_hash: 'b'.repeat(64), created_by: creator, created_at: T('02T00:00:00.000'), expires_at: T('09T00:00:00.000'), used_at: T('02T01:00:00.000'), used_by: supporter });++ ins('tip_moderation_log', { id: 3, creator_subject: creator, interaction_id: 'tint_04', action: 'hidden', by_role: 'moderator', actor: supporter, reason: 'abuse', created_at: T('05T08:00:00.000') });++ // A stored answer is replayed byte for byte: its text keeps its key order and spacing.++ ins('api_idempotency', { key: `${supporter}:tip-key-0001`, request_hash: 'h1', method: 'POST', path: '/api/v1/checkout', status: 201, response: '{"z":1,"a":{"b":2}}', created_at: T('03T12:00:00.000') });++ ins('event_outbox', { id: 1, event_id: 'evt_01', envelope: '{"event_type":"tips.interaction.ready","subject":{"type":"interaction","id":"tint_01"},"payload":{"supporter":null}}', created_at: 1790500000000, attempts: 1, sent_at: 1790500001000, seq: 17 });++ ins('event_outbox', { id: 2, event_id: 'evt_02', envelope: '{"event_type":"tips.goal.updated","subject":{"type":"goal","id":"tgoal_01"},"payload":{}}', created_at: 1790500002000, attempts: 3, next_attempt_at: 1790500999000, last_error: 'timeout' });++ ins('idempotency_receipts', { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });++ s.close();++ return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex');++}++++(async () => {++ const file = path.join(tmp, 'tips.db');++ const hash = fixture(file);++++ const imported = await testDb({ store: 'pglite' });++ const db = imported.db;++ let report;++ const options = await importOptions(db);++ const TABLES = options.tables;++ await check('the importer copies every table, verified by counts and checksums (report.ok)', async () => {++ report = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });++ assert.strictEqual(report.ok, true, JSON.stringify(report.problems));++ const rows = Object.fromEntries(report.tables.map((t) => [t.table, t.rows]));++ assert.deepStrictEqual(rows, {++ creator_tip_profiles: 1, tip_interactions: 4, tip_goals: 1, tip_goal_contributions: 2, paid_messages: 1, paid_media_requests: 1,++ overlay_configs: 1, overlay_tokens: 1, overlay_deliveries: 2, interaction_effects: 3, migration_maps: 1, import_runs: 1,++ tip_moderators: 1, tip_moderator_invites: 1, tip_moderation_log: 1, api_idempotency: 1, tips_event_outbox: 2, tips_event_inbox: 1,++ });++ assert.ok(report.tables.every((t) => /^[0-9a-f]{16}$/.test(t.checksum)), 'a checksum per table');++ const renamed = report.tables.filter((t) => t.source !== t.table).map((t) => `${t.source} → ${t.table}`).sort();++ assert.deepStrictEqual(renamed, ['event_outbox → tips_event_outbox', 'idempotency_receipts → tips_event_inbox']);++ assert.strictEqual(crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'), hash, 'the SQLite file is unchanged');++ });++++ await check('values are converted by column type', async () => {++ const p = await db.one('SELECT * FROM creator_tip_profiles');++ assert.deepStrictEqual([p.page_enabled, p.tts_enabled, p.filter_links, p.revision], [true, false, false, 4]);++ assert.deepStrictEqual(p.page_settings, { goal_amounts: false, supporters_page: true });++ assert.deepStrictEqual(p.filter_words, ['badword', 'two words']);++ assert.strictEqual(p.updated_at, T('02T11:30:00.250'), 'the same ISO time the API sent before');++ const i = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_01'");++ assert.deepStrictEqual(i.request, { goal_id: 'tgoal_01' });++ assert.deepStrictEqual(i.target, { service: 'live', type: 'stream', id: '42' });++ assert.deepStrictEqual([i.test, i.anonymous, i.transfer_due, i.amount], [false, false, false, 250]);++ assert.deepStrictEqual(i.reversal_txn_ids, []);++ const r = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_02'");++ assert.deepStrictEqual([r.anonymous, r.hide_amount, r.reversed_bits, r.reversal_txn_ids], [true, true, 100, ['txn_9']]);++ assert.strictEqual(r.message, 'nulhere', 'a NUL is dropped (PostgreSQL text cannot hold it)');++ assert.deepStrictEqual(r.request, { legacy_message: 'cut \ufffd', n: 1 }, 'an unpaired surrogate becomes U+FFFD (jsonb refuses it)');++ assert.deepStrictEqual(options.cleaned().sort((a, b) => a.column.localeCompare(b.column)),++ [{ column: 'tip_interactions.message', values: 1 }, { column: 'tip_interactions.request', values: 1 }], 'and the report says where');++ const pending = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_03'");++ assert.deepStrictEqual([pending.transfer_due, pending.transfer_attempts, pending.next_transfer_at, pending.settled_at], [true, 2, 1790600000123, null]);++ const erased = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_04'");++ assert.deepStrictEqual([erased.test, erased.filtered, erased.erased_at, erased.target], [true, true, T('05T08:00:00.000'), null]);++ assert.strictEqual(await db.value("SELECT chat_ref FROM paid_messages WHERE id = 'tpm_01'"), null, "JSON text 'null' → NULL (no reference)");++ assert.deepStrictEqual(await db.value("SELECT queue_ref FROM paid_media_requests WHERE id = 'tmr_01'"), { queue_id: 5 });++ assert.deepStrictEqual((await db.many('SELECT id, result FROM interaction_effects ORDER BY id')).map((e) => [e.id, e.result]),++ [[4, { ref: { chat_message_id: 1 } }], [5, null], [6, null]]);++ assert.deepStrictEqual(await db.value("SELECT scopes FROM overlay_tokens"), ['alerts', 'goals']);++ assert.deepStrictEqual((await db.many('SELECT seq, hidden, test FROM overlay_deliveries ORDER BY seq')).map((d) => [d.seq, d.hidden, d.test]), [[3, false, false], [9, true, false]]);++ assert.strictEqual(await db.value('SELECT response FROM api_idempotency'), '{"z":1,"a":{"b":2}}', 'a stored answer stays byte for byte');++ assert.strictEqual(await db.value('SELECT dry_run FROM import_runs'), false);++ const out = await db.many('SELECT id, envelope, sent_at, next_attempt_at FROM tips_event_outbox ORDER BY id');++ assert.deepStrictEqual(out.map((o) => [o.id, o.envelope.event_type, o.sent_at, o.next_attempt_at]), [[1, 'tips.interaction.ready', 1790500001000, 0], [2, 'tips.goal.updated', null, 1790500999000]]);++ assert.deepStrictEqual(await db.one('SELECT * FROM tips_event_inbox'), { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });++ });++++ await check('identities keep their values and continue after the imported maximum', async () => {++ const at = new Date().toISOString();++ assert.strictEqual(await db.value(`INSERT INTO overlay_deliveries (id, creator_subject, kind, payload, status, dedupe_key, created_at)++ VALUES ('tovd_new', $1, 'alert', '{}', 'pending', 'alert:new', $2) RETURNING seq`, [creator, at]), 10, 'the SSE ids go on from 9');++ assert.strictEqual(await db.value(`INSERT INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at)++ VALUES ('tint_02', 'chat_line', 'test', 'queued', $1, $1) RETURNING id`, [at]), 7);++ assert.strictEqual(await db.value(`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)++ VALUES ('tgoal_01', 'tint_03', 5, $1, $1) RETURNING id`, [at]), 13);++ assert.strictEqual(await db.value("INSERT INTO tips_event_outbox (event_id, envelope, created_at) VALUES ('evt_03', '{}', 1) RETURNING id"), 3);++ });++++ await check('a rehearsal can be repeated (truncate), and a value the checksum would miss is caught', async () => {++ const again = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });++ assert.strictEqual(again.ok, true, JSON.stringify(again.problems));++ // Without the 'null' mapping, JSON null and SQL NULL differ: the verification says so.++ const bad = await importSqlite({ sqlite: file, db, truncate: true, tables: { ...TABLES, paid_messages: {} }, only: ['paid_messages', 'tip_interactions', 'creator_tip_profiles'], log: quiet });++ assert.strictEqual(bad.ok, false);++ assert.match(bad.problems.map((x) => `${x.table}: ${x.problem}`).join('\n'), /paid_messages: verification failed/);++ await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });++ });++++ await check('the service serves the imported data as it did from SQLite', async () => {++ const t = await boot({ db });++ const who = { subject: supporter, username: 'viewer' };++ const r = await t.call('GET', '/api/v1/interactions', { user: who });++ assert.strictEqual(r.status, 200, r.text);++ const byId = Object.fromEntries(r.json.interactions.map((i) => [i.id, i]));++ assert.deepStrictEqual(Object.keys(byId).sort(), ['tint_01', 'tint_02', 'tint_03']);++ assert.strictEqual(byId.tint_01.created_at, T('03T12:00:00.000'));++ assert.strictEqual(byId.tint_01.payment.settled_at, T('03T12:00:00.500'));++ assert.deepStrictEqual(byId.tint_02.payment.reversal_txn_ids, ['txn_9']);++ assert.strictEqual(byId.tint_02.public.supporter_name, 'Anonymous');++ const goal = (await t.call('GET', '/api/v1/goals/tgoal_01', { user: { subject: creator, username: 'alex' } })).json.goal;++ assert.strictEqual(goal.current_amount, 50 + 250 + 150, 'opening amount plus what the contributions keep');++ const replay = await t.call('POST', '/api/v1/checkout', { user: who, key: 'tip-key-0001', body: {} });++ assert.strictEqual(replay.status, 422, 'a different request under a stored key is refused');++ assert.strictEqual(await t.db.value('SELECT count(*) FROM tips_event_inbox'), 1);++ await t.close();++ });++++ await check('scripts/migrate-to-postgres.js: OK exits 0; a column the schema lacks is a problem and exits 1', async () => {++ const out = execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', file], { encoding: 'utf8' });++ assert.match(out, /→ pglite \(\d+ ms\): OK/);++ assert.match(out, /tips_event_outbox\s+2 rows\s+[0-9a-f]{16}\s+\(from event_outbox\)/);++ assert.match(out, /cleaned: tip_interactions\.message: 1 value\(s\) with a NUL or an unpaired surrogate/);++ const extra = path.join(tmp, 'extra.db');++ fs.copyFileSync(file, extra);++ const s = new Database(extra); s.exec('ALTER TABLE tip_goals ADD COLUMN mystery TEXT'); s.close();++ let code = 0; let text = '';++ try { execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', extra], { encoding: 'utf8' }); } catch (e) { code = e.status; text = e.stdout; }++ assert.strictEqual(code, 1);++ assert.match(text, /problem: tip_goals: source columns with no target column .*mystery/);++ });++++ await imported.close();++ fs.rmSync(tmp, { recursive: true, force: true });++ done();++})().catch((e) => { console.error(e); process.exit(1); });
@@ -11,8 +11,22 @@ NODE_ENV=development
@@ -11,8 +11,22 @@ NODE_ENV=development
BASE_URL=http://localhost:4610
BASE_URL=http://localhost:4610
TRUST_PROXY=1
TRUST_PROXY=1
-# SQLite (one database per service). The systemd unit sets /var/lib/openvibe-tips/tips.db.
-TIPS_DB_PATH=./data/tips.db
+# ── Data: PostgreSQL + Valkey (ADR-035) ──────────────────────
+# In production OpenVibe.Host writes all four (sudo /opt/openvibe.host/roles/data/add-service.sh tips).
+# DATABASE_URL: the runtime role through PgBouncer (transaction mode; DML only). Unset in development:
+# an embedded PGlite database in data/pglite (one process). Tests need neither (PGlite, or the containers
+# of node_modules/openvibe-sdk/scripts/test-services.sh).
+DATABASE_URL=
+# DATABASE_DIRECT_URL: the owner role on a direct connection, for the migrations at boot and
+# scripts/migrate-to-postgres.js.
+DATABASE_DIRECT_URL=
+# VALKEY_URL: shared, never authoritative state (per-actor limits, overlay fan-out and stream slots).
+# Unset: kept in each process, right for one process only.
+VALKEY_URL=
+VALKEY_PREFIX=ov:tips:
+# The SQLite file of the release before PostgreSQL: read (never written) only by
+# scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.
+#TIPS_DB_PATH=./data/tips.db
# ── Identity: OpenVibe.Network ──────────────────────────────
# ── Identity: OpenVibe.Network ──────────────────────────────
OV_NETWORK_URL=https://openvibe.network
OV_NETWORK_URL=https://openvibe.network
@@ -28,7 +42,8 @@ OV_OAUTH_CLIENT_ID=tips
@@ -28,7 +42,8 @@ OV_OAUTH_CLIENT_ID=tips
OV_OAUTH_CLIENT_SECRET=
OV_OAUTH_CLIENT_SECRET=
#OV_OAUTH_REDIRECT_URI=https://openvibe.tips/auth/callback
#OV_OAUTH_REDIRECT_URI=https://openvibe.tips/auth/callback
#COOKIE_SECURE=true
#COOKIE_SECURE=true
-# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32).
+# Signs the anti-forgery tokens of the server-rendered forms (openssl rand -hex 32). Every process must
+# share it; unset, it is derived from OV_OAUTH_CLIENT_SECRET (and random per process without either).
TIPS_FORM_SECRET=
TIPS_FORM_SECRET=
# ── OpenVibe.Billing (money moves only there) ───────────────
# ── OpenVibe.Billing (money moves only there) ───────────────
@@ -9,6 +9,10 @@ jobs:
@@ -9,6 +9,10 @@ jobs:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
with:
with:
contracts-service: tips
contracts-service: tips
+ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers
+ # (openvibe-sdk scripts/test-services.sh). npm test runs every file on PGlite and
+ # test/integration.test.js on the containers; test:pg runs the whole suite through PgBouncer and Valkey.
+ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
security:
security:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
@@ -4,7 +4,9 @@
@@ -4,7 +4,9 @@
**Status:** alpha — runtime built and tested against stubs (roadmap Wave 9). Deployed internally, not
**Status:** alpha — runtime built and tested against stubs (roadmap Wave 9). Deployed internally, not
launched: running on the host since 2026-09-23 on `127.0.0.1:4610` only, holding the Live import and no
launched: running on the host since 2026-09-23 on `127.0.0.1:4610` only, holding the Live import and no
-creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).
+creator profile yet, with no public route. OpenVibe.Live remains where tips happen until the Billing cutover (see *What waits*).
+Data: PostgreSQL through PgBouncer, and Valkey for shared state ([ADR-035](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-035-postgresql-and-valkey.md));
+production runs the SQLite release until the switch in *Deploy*.
**Domain:** `openvibe.tips` (keeps its OpenVibe.Sites placeholder until the launch rule below holds)
**Domain:** `openvibe.tips` (keeps its OpenVibe.Sites placeholder until the launch rule below holds)
**Port / service id:** 4610 / `tips`
**Port / service id:** 4610 / `tips`
**Decision:** [ADR-012](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-012-economic-classification.md) — Tips never moves money; Billing does.
**Decision:** [ADR-012](https://github.com/OpenVibers/OpenVibe.Contracts/blob/main/docs/adr/ADR-012-economic-classification.md) — Tips never moves money; Billing does.
@@ -21,8 +23,8 @@ from *interaction delivered*.
@@ -21,8 +23,8 @@ from *interaction delivered*.
- `tip_interactions`, `creator_tip_profiles`, `tip_goals`, `tip_goal_contributions`, `paid_messages`,
- `tip_interactions`, `creator_tip_profiles`, `tip_goals`, `tip_goal_contributions`, `paid_messages`,
`paid_media_requests`, `overlay_configs`, `overlay_deliveries`, `interaction_effects`, `migration_maps`
`paid_media_requests`, `overlay_configs`, `overlay_deliveries`, `interaction_effects`, `migration_maps`
- (its own SQLite, `server/db.js`) — interaction state and **references** to Billing transaction ids,
- never a mutable cash balance
+ (its own PostgreSQL database `ov_tips`, schema in [migrations/](migrations/), access in `server/db.js`) —
+ interaction state and **references** to Billing transaction ids, never a mutable cash balance
- revocable scoped overlay tokens (`overlay_tokens`, hashed at rest; never creator cookies)
- revocable scoped overlay tokens (`overlay_tokens`, hashed at rest; never creator cookies)
## Does not own
## Does not own
@@ -34,6 +36,10 @@ from *interaction delivered*.
@@ -34,6 +36,10 @@ from *interaction delivered*.
## Depends on
## Depends on
+- **PostgreSQL 18 through PgBouncer (transaction mode) and Valkey 9** — OpenVibe.Host's data role
+ (`roles/data/add-service.sh tips`: database `ov_tips`, owner and runtime roles, a Valkey user on
+ `ov:tips:*`). PostgreSQL is required (the system of record); Valkey holds only what the processes share
+ (per-actor limit counters, overlay fan-out, overlay stream slots) and its loss changes no answer about money
- **OpenVibe.Billing** — `POST /api/v1/intents` (checkout), `POST /api/v1/transfers` (a tip from credit),
- **OpenVibe.Billing** — `POST /api/v1/intents` (checkout), `POST /api/v1/transfers` (a tip from credit),
`billing.transaction.settled|reversed` events, and `billing.receipt.external` (a tip on a creator's own
`billing.transaction.settled|reversed` events, and `billing.receipt.external` (a tip on a creator's own
PowerChat, once Billing receives the PowerChat webhook)
PowerChat, once Billing receives the PowerChat webhook)
@@ -42,8 +48,9 @@ from *interaction delivered*.
@@ -42,8 +48,9 @@ from *interaction delivered*.
- **OpenVibe.Network** — SSO for people, service tokens, JWKS, identity resolve (importer)
- **OpenVibe.Network** — SSO for people, service tokens, JWKS, identity resolve (importer)
- **OpenVibe.Live** — chat delivery through `/internal/tips/deliveries` (deployed in Live since `f11f809`; [docs/live-patch.diff](docs/live-patch.diff) is the original patch) until OpenVibe.Chat exposes paid messages; overlay consumer
- **OpenVibe.Live** — chat delivery through `/internal/tips/deliveries` (deployed in Live since `f11f809`; [docs/live-patch.diff](docs/live-patch.diff) is the original patch) until OpenVibe.Chat exposes paid messages; overlay consumer
- **OpenVibe.Shared** v1.25.0 (app icon, SSR footer, noscript nav, release manifest, legal pages) and
- **OpenVibe.Shared** v1.25.0 (app icon, SSR footer, noscript nav, release manifest, legal pages) and
- the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.12.0 (service tokens,
- outbox and inbox, per-actor limits), pinned by release tarball
+ the Network's `navbar.js`; **openvibe-contracts** v0.49.0 and **openvibe-sdk** v0.15.0 (service tokens,
+ the async data layer `openvibe-sdk/db`, the PostgreSQL outbox and inbox, Valkey, pub/sub and the shared
+ limit store), pinned by release tarball; `pg` and `iovalkey` (drivers), PGlite for tests
## Capabilities
## Capabilities
@@ -66,10 +73,14 @@ Called elsewhere, as the service principal `tips`:
@@ -66,10 +73,14 @@ Called elsewhere, as the service principal `tips`:
```bash
```bash
fnm exec --using=22.22.1 npm install
fnm exec --using=22.22.1 npm install
cp .env.example .env # OV_OAUTH_CLIENT_SECRET, TIPS_FORM_SECRET, TIPS_EVENTS_SECRET, …
cp .env.example .env # OV_OAUTH_CLIENT_SECRET, TIPS_FORM_SECRET, TIPS_EVENTS_SECRET, …
-npm run dev # http://localhost:4610
-npm test # every test/*.test.js: stub Network/Billing/Events/Live, temp DBs, random ports
+npm run dev # http://localhost:4610 (without DATABASE_URL: an embedded PGlite database in data/pglite)
+npm test # every test/*.test.js: stub Network/Billing/Events/Live, PGlite, random ports
+eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" # PostgreSQL 18 + PgBouncer + Valkey 9 containers
+npm test # … now also test/integration.test.js on the containers
+npm run test:pg # every test file through PgBouncer and Valkey
npm run subscribe # create the billing.transaction.* and billing.receipt.* subscriptions in OpenVibe.Events
npm run subscribe # create the billing.transaction.* and billing.receipt.* subscriptions in OpenVibe.Events
node scripts/import-live.js --live-db <live snapshot> --billing-db <billing snapshot> [--dry-run] [--json]
node scripts/import-live.js --live-db <live snapshot> --billing-db <billing snapshot> [--dry-run] [--json]
+npm run migrate-to-postgres -- [--sqlite <copy>] [--pglite] [--json] # the one-time SQLite import (Deploy)
```
```
Production (deployed, loopback only): `/opt/openvibe.tips`, env `/etc/openvibe/tips.env`, unit
Production (deployed, loopback only): `/opt/openvibe.tips`, env `/etc/openvibe/tips.env`, unit
@@ -101,7 +112,7 @@ delivery record; a reversal before delivery cancels the queued effects (`tips.in
@@ -101,7 +112,7 @@ delivery record; a reversal before delivery cancels the queued effects (`tips.in
| `external` | nothing — a tip on the creator's own PowerChat (ADR-012 EXTERNAL) | `POST /api/v1/interactions/external`, or Billing's `billing.receipt.external` (origin `billing-external`: Tips posts the chat line — `… tipped N Vibes: msg (PowerChat)` — plus overlay alert and goal, as Live's webhook did); once per (provider, provider_ref); never in Billing totals |
| `external` | nothing — a tip on the creator's own PowerChat (ADR-012 EXTERNAL) | `POST /api/v1/interactions/external`, or Billing's `billing.receipt.external` (origin `billing-external`: Tips posts the chat line — `… tipped N Vibes: msg (PowerChat)` — plus overlay alert and goal, as Live's webhook did); once per (provider, provider_ref); never in Billing totals |
| `none` (simulation) | nothing | at once, `test = 1`: full effect path, no goal contribution, no durable event, excluded from totals |
| `none` (simulation) | nothing | at once, `test = 1`: full effect path, no goal contribution, no durable event, excluded from totals |
-**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same SQLite
+**Exactly one logical interaction:** the inbox claims `(consumer, event_id)` in the same serializable
transaction as the change, so a redelivered event does nothing; `billing_txn_id` is UNIQUE, so the same
transaction as the change, so a redelivered event does nothing; `billing_txn_id` is UNIQUE, so the same
Billing transaction under another event id (republish, replay) is still one interaction; API calls carry
Billing transaction under another event id (republish, replay) is still one interaction; API calls carry
`Idempotency-Key` (stored responses + the interaction's own unique key); the no-JS form carries a nonce.
`Idempotency-Key` (stored responses + the interaction's own unique key); the no-JS form carries a nonce.
@@ -117,9 +128,35 @@ from Live (whose `current_amount` cannot be traced to individual donations).
@@ -117,9 +128,35 @@ from Live (whose `current_amount` cannot be traced to individual donations).
minus what Billing took back) — the number that must equal Billing's books — with `external` and
minus what Billing took back) — the number that must equal Billing's books — with `external` and
`simulations` reported apart.
`simulations` reported apart.
+**Data** (ADR-035; `server/db.js`, [migrations/](migrations/)):
+- Everything is async on `openvibe-sdk/db`: a function that touches the database takes the query handle
+ first (`db`, or the transaction's `t`), and after-commit work (overlay pushes, worker kicks) hangs on
+ the transaction (`t.after`). The schema is `migrations/NNNN_*.sql` (expand/migrate/contract), applied
+ at boot with the owner role; the service then serves through PgBouncer with no session state.
+- **Money paths are SERIALIZABLE with row locks** (ADR-007 amendment 2026-09-24, rule 3): a request and
+ its unpaid-checkout limit, settlement (paid message, media request, goal contribution), payment failure,
+ funding, reversal, the Billing inbox, and what cancels or releases what a payment bought (moderation's
+ hide and restore, the effects worker's writes). Locks are taken in one order: a creator's profile row
+ (settings changes and per-creator limits), the interaction row, its goals by id, the creator's overlay
+ stream (an advisory lock, so a creator's deliveries commit in seq order), then the rest. Serialization
+ failures and deadlocks are retried by the data layer.
+- Text is made storable on the way in: PostgreSQL keeps no NUL character, and jsonb no unpaired surrogate
+ (SQLite kept both), so they are dropped or replaced (U+FFFD) in what people and Billing's events send.
+- **Background work runs in every process** and claims its rows: due effects and due transfers with a
+ lease (`FOR UPDATE SKIP LOCKED`, the lease in `next_attempt_at` / `next_transfer_at`), the overlay
+ window sweep with `SKIP LOCKED`, the outbox relay by the SDK's lease; prunes are idempotent deletes.
+- **Shared state in Valkey** (none of it authoritative): per-actor limit counters (the SDK's shared limit
+ store), overlay fan-out (`overlay:<creator>` on pub/sub: a stream in one process hears a tip, a config
+ change, a retraction or a revocation made in another), overlay stream slots (a sorted set per creator,
+ refreshed by heartbeats). Without `VALKEY_URL` each process keeps its own, which is right for one process.
+- Every query shape has an index (`test/migrations.test.js` checks the plans), lists page by keyset, and a
+ list costs a fixed number of queries whatever its size (`test/query-budget.test.js`).
+
## API
## API
-`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH. Services present a Network
+`/api/v1`, problem+json errors, `Idempotency-Key` on every POST/PATCH (claimed before the handler runs:
+the same key twice at once runs once, the other answered `409 idempotency.in_progress` with `Retry-After`
+until the first answer is stored and replayed). Services present a Network
service token (audience `openvibe.tips`, one capability per route); people present their Network user
service token (audience `openvibe.tips`, one capability per route); people present their Network user
token as a Bearer and act on their own things only. The API never reads cookies.
token as a Bearer and act on their own things only. The API never reads cookies.
@@ -218,8 +255,8 @@ toward its goal and the creator's totals.
@@ -218,8 +255,8 @@ toward its goal and the creator's totals.
## Events
## Events
-Produced through the openvibe-sdk outbox (table `event_outbox`, source `tips`, same transaction as the
-change; relayed when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,
+Produced through the openvibe-sdk outbox (table `tips_event_outbox`, source `tips`, same transaction as
+the change; relayed by any process, rows claimed with a lease, when `EVENTS_URL` and the client secret are set): `tips.interaction.ready`,
`tips.interaction.failed`, `tips.interaction.cancelled`, `tips.goal.updated`, `tips.overlay.delivered`,
`tips.interaction.failed`, `tips.interaction.cancelled`, `tips.goal.updated`, `tips.overlay.delivered`,
`tips.overlay.failed`, and two whose schemas were proposed in [docs/events-proposal/](docs/events-proposal/)
`tips.overlay.failed`, and two whose schemas were proposed in [docs/events-proposal/](docs/events-proposal/)
and are in openvibe-contracts since v0.32.0: `tips.interaction.erased` (a supporter's erasure; `{ interaction_id,
and are in openvibe-contracts since v0.32.0: `tips.interaction.erased` (a supporter's erasure; `{ interaction_id,
@@ -319,6 +356,7 @@ service counts as its principal (`svc:live`), a person as `user:usr_…`, anyone
@@ -319,6 +356,7 @@ service counts as its principal (`svc:live`), a person as `user:usr_…`, anyone
Never limited: `/api/health`, `/api/ready`, `/release.json`, `/metrics`, the overlays (bounded by
Never limited: `/api/health`, `/api/ready`, `/release.json`, `/metrics`, the overlays (bounded by
`TIPS_OVERLAY_MAX_STREAMS`) and the signed Events deliveries at `/internal/events`. `test/actor-limits.test.js`.
`TIPS_OVERLAY_MAX_STREAMS`) and the signed Events deliveries at `/internal/events`. `test/actor-limits.test.js`.
+With `VALKEY_URL` every process counts one caller together (`test/integration.test.js`).
## Import and reconciliation
## Import and reconciliation
@@ -347,6 +385,8 @@ a copy of Billing's database for the links and totals:
@@ -347,6 +385,8 @@ a copy of Billing's database for the links and totals:
| simulation never counted; token revocation immediate; goals from settled only; reversal keeps the delivery record | `test/overlays.test.js`, `test/api.test.js`, `test/settlement.test.js` |
| simulation never counted; token revocation immediate; goals from settled only; reversal keeps the delivery record | `test/overlays.test.js`, `test/api.test.js`, `test/settlement.test.js` |
| an EXTERNAL PowerChat tip Billing announced is celebrated once: Live chat line, overlay, goal; never Billing money | `test/external.test.js` (redelivery and republish, same key as `POST /interactions/external`, Tips and Live goal ids, anonymous, test receipts, malformed payloads) |
| an EXTERNAL PowerChat tip Billing announced is celebrated once: Live chat line, overlay, goal; never Billing money | `test/external.test.js` (redelivery and republish, same key as `POST /interactions/external`, Tips and Live goal ids, anonymous, test receipts, malformed payloads) |
| paid messages are filtered before they are shown or read; the creator and their moderators hide and show them without touching the money | `test/moderation.test.js` (mask, hold and release, invisible/full-width evasion, TTS text, invitation links, hide cancels queued chat/TTS and retracts the overlay live/replay/state, restore, pending payments, who may moderate, event payloads) |
| paid messages are filtered before they are shown or read; the creator and their moderators hide and show them without touching the money | `test/moderation.test.js` (mask, hold and release, invisible/full-width evasion, TTS text, invitation links, hide cancels queued chat/TTS and retracts the overlay live/replay/state, restore, pending payments, who may moderate, event payloads) |
+| the suite passes on PostgreSQL behind PgBouncer in transaction mode; one settlement or reversal raced across two processes counts once; the unpaid-checkout limit holds under concurrency; leased work runs once; flushing Valkey changes no money answer (ADR-007 2026-09-24, ADR-035) | `npm run test:pg`; `test/integration.test.js` (two app instances on one database and one Valkey) |
+| the SQLite data moves whole: every table and column, verified by counts and checksums | `test/sqlite-import.test.js`, `test/migrations.test.js` (no column left behind) |
| a supporter's privacy holds everywhere; the creator chooses what public pages show; export and erasure keep the books reconciled | `test/privacy.test.js` (anonymous / hidden amount / private message across API, overlays, chat jobs, pages and events; goal and supporters page settings; export; erasure scrubs rows, overlay payloads, stored answers and unsent events, emits `tips.interaction.erased`, totals still equal Billing) |
| a supporter's privacy holds everywhere; the creator chooses what public pages show; export and erasure keep the books reconciled | `test/privacy.test.js` (anonymous / hidden amount / private message across API, overlays, chat jobs, pages and events; goal and supporters page settings; export; erasure scrubs rows, overlay payloads, stored answers and unsent events, emits `tips.interaction.erased`, totals still equal Billing) |
## What waits
## What waits
@@ -375,8 +415,25 @@ The unit is `openvibe-tips.service` on `127.0.0.1:4610`, the env file `/etc/open
@@ -375,8 +415,25 @@ The unit is `openvibe-tips.service` on `127.0.0.1:4610`, the env file `/etc/open
`/var/lib/openvibe-tips`. The vhost [deploy/nginx/openvibe.tips.conf](deploy/nginx/openvibe.tips.conf)
`/var/lib/openvibe-tips`. The vhost [deploy/nginx/openvibe.tips.conf](deploy/nginx/openvibe.tips.conf)
waits for the launch: `openvibe.tips` serves the Sites placeholder.
waits for the launch: `openvibe.tips` serves the Sites placeholder.
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
-restart; afterwards `sudo ovhost rollback tips --to <sha>`. Nothing blocks a rollback: the schema
-code only adds tables and columns.
+restart; afterwards `sudo ovhost rollback tips --to <sha>`. Migrations are expand-only until a
+`contract` migration, which waits out the 7-day N-1 window.
+
+**The switch from SQLite to PostgreSQL** (once; the procedure is the SDK's
+[migrating-to-postgresql.md](https://github.com/OpenVibers/OpenVibe.SDK/blob/main/docs/migrating-to-postgresql.md), section 6):
+1. `sudo /opt/openvibe.host/roles/data/add-service.sh tips` — database `ov_tips`, its roles and Valkey
+ user; it writes `DATABASE_URL`, `DATABASE_DIRECT_URL`, `VALKEY_URL`, `VALKEY_PREFIX` into
+ `/etc/openvibe/tips.env`.
+2. Rehearse on a copy of `/var/lib/openvibe-tips/tips.db` (`sqlite3 … ".backup …"` as the service user):
+ `npm run migrate-to-postgres -- --sqlite <copy> --pglite` must end OK.
+3. Merge the PostgreSQL release; do not deploy it yet.
+4. `sudo systemctl stop openvibe-tips` (the write freeze), then back up `tips.db`.
+5. From the new release's directory, as the service user with the service's environment:
+ `node scripts/migrate-to-postgres.js` — it migrates as the owner, imports every table with
+ `truncate`, verifies counts and checksums, changes nothing in the SQLite file, and exits 1 unless OK.
+6. `sudo ovhost deploy tips`; check `/api/ready` shows `db` `store: postgresql` and `valkey` ok, the
+ dashboard, a receipt and an overlay, and a flat error rate.
+7. Keep `tips.db` read-only for 7 days (the rollback: the previous release reads it, losing the writes
+ made since the switch unless they are replayed), then `ovhost archive push` it and delete it.
## Launch rule
## Launch rule
@@ -385,8 +442,8 @@ This repository does not make the product real, and the domain keeps its placeho
@@ -385,8 +442,8 @@ This repository does not make the product real, and the domain keeps its placeho
exist here (plan §12.12):
exist here (plan §12.12):
1. an owning runtime with health/readiness endpoints and observability — ✔ `/api/health`, `/api/ready`
1. an owning runtime with health/readiness endpoints and observability — ✔ `/api/health`, `/api/ready`
- (openvibe-shared/ready: the database required; Network key, Billing and Events optional, so
- their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals by
+ (openvibe-shared/ready: the database required and reporting the store that answered; Valkey, the
+ Network key, Billing and Events optional, so their loss degrades rather than fails), `/metrics` for direct loopback callers (golden signals by
route template, pending effects and overlay deliveries, outbox backlog);
route template, pending effects and overlay deliveries, outbox backlog);
2. canonical identity/auth integration (Network subjects, scoped service principals) — ✔; the `tips`
2. canonical identity/auth integration (Network subjects, scoped service principals) — ✔; the `tips`
principal is provisioned on the host;
principal is provisioned on the host;
@@ -8,6 +8,7 @@
@@ -8,6 +8,7 @@
"authoritative": false,
"authoritative": false,
"deployed": true,
"deployed": true,
"deployment": "Deployed internally, not launched: deployed with `sudo ovhost deploy tips`, /opt/openvibe.tips, unit openvibe-tips, 127.0.0.1:4610 loopback only; https://openvibe.tips still serves the OpenVibe.Sites placeholder. Principal tips; Events subscriptions billing.transaction.* and billing.receipt.* -> /internal/events (secret rotated 2026-09-24); TIPS_CHAT_ADAPTER is empty (no chat side effects).",
"deployment": "Deployed internally, not launched: deployed with `sudo ovhost deploy tips`, /opt/openvibe.tips, unit openvibe-tips, 127.0.0.1:4610 loopback only; https://openvibe.tips still serves the OpenVibe.Sites placeholder. Principal tips; Events subscriptions billing.transaction.* and billing.receipt.* -> /internal/events (secret rotated 2026-09-24); TIPS_CHAT_ADAPTER is empty (no chat side effects).",
+ "database": "PostgreSQL 18 through PgBouncer (transaction mode) as the system of record, Valkey 9 for shared state (per-actor limits, overlay fan-out, overlay stream slots), ADR-035: database ov_tips from OpenVibe.Host's data role, schema in migrations/ (0001_initial, expand), async on openvibe-sdk/db v0.15.0, money paths serializable with row locks, PostgreSQL outbox and inbox. The deployed release still runs on SQLite (/var/lib/openvibe-tips/tips.db) until the switch in README Deploy: scripts/migrate-to-postgres.js imports it verified by counts and checksums (rehearsed on the SQLite databases the previous release makes: all tables and columns, report.ok), and the file stays read-only for the 7-day N-1 window.",
"contracts": "openvibe-contracts v0.49.0",
"contracts": "openvibe-contracts v0.49.0",
"notes": "Runtime, API, SSR pages, overlays, the Billing settlement consumer, events outbox, Live import and reconciliation are built, tested and deployed, together with /metrics (loopback only), truthful /api/ready (Billing and Events checks), supporter privacy (anonymous, hidden amount, private message), goal and supporter-page settings, export and erasure (tips.interaction.erased), moderation (word filter, hide/show, invited moderators, tips.interaction.moderated) and per-actor limits. Capabilities (18 tips.*) and payload schemas are in openvibe-contracts. On 2026-09-24 production held the Live import (3 import runs, the last 2026-09-23 18:31 UTC: 139 interactions, 4 goals, 143 migration maps); 0 creator profiles and 0 overlay tokens (nobody had set up the product). No billing.* event had arrived: Billing runs in shadow. A written internal threat review (docs/threat-review.md) with its fixes and test/security.test.js. Not done: an independent security review; the openvibe.tips launch. Live stays the place tips happen until the Billing cutover (blocked on the owner re-pointing the PowerChat webhook) and the launch rule in README.md holds.",
"notes": "Runtime, API, SSR pages, overlays, the Billing settlement consumer, events outbox, Live import and reconciliation are built, tested and deployed, together with /metrics (loopback only), truthful /api/ready (Billing and Events checks), supporter privacy (anonymous, hidden amount, private message), goal and supporter-page settings, export and erasure (tips.interaction.erased), moderation (word filter, hide/show, invited moderators, tips.interaction.moderated) and per-actor limits. Capabilities (18 tips.*) and payload schemas are in openvibe-contracts. On 2026-09-24 production held the Live import (3 import runs, the last 2026-09-23 18:31 UTC: 139 interactions, 4 goals, 143 migration maps); 0 creator profiles and 0 overlay tokens (nobody had set up the product). No billing.* event had arrived: Billing runs in shadow. A written internal threat review (docs/threat-review.md) with its fixes and test/security.test.js. Not done: an independent security review; the openvibe.tips launch. Live stays the place tips happen until the Billing cutover (blocked on the owner re-pointing the PowerChat webhook) and the launch rule in README.md holds.",
"updated": "2026-09-28",
"updated": "2026-09-28",
@@ -22,7 +23,8 @@
@@ -22,7 +23,8 @@
"Live import and reconciliation against Billing",
"Live import and reconciliation against Billing",
"server-rendered creator pages useful without JavaScript",
"server-rendered creator pages useful without JavaScript",
"/api/v1 by capability",
"/api/v1 by capability",
- "SDK outbox and inbox",
- "per-actor limits (openvibe-sdk/limits)"
+ "per-actor limits (openvibe-sdk/limits)",
+ "PostgreSQL + Valkey data tier (ADR-035): serializable money paths, leased background work, shared limits and overlay fan-out",
+ "SDK outbox and inbox on PostgreSQL"
]
]
}
}
@@ -3,8 +3,11 @@ Description=OpenVibe.Tips — creator support: tips, goals, paid messages, TTS a
@@ -3,8 +3,11 @@ Description=OpenVibe.Tips — creator support: tips, goals, paid messages, TTS a
# Network signs the tokens Tips verifies (its key is fetched at boot and retried); Billing moves
# Network signs the tokens Tips verifies (its key is fetched at boot and retried); Billing moves
# the money; Events carries Billing's settlements in and Tips' events out. Tips starts without any
# the money; Events carries Billing's settlements in and Tips' events out. Tips starts without any
# of them: pages and overlays keep working, payments wait.
# of them: pages and overlays keep working, payments wait.
-After=network.target
-Wants=openvibe-network.service openvibe-billing.service
+# Its own data (ADR-035, OpenVibe.Host's data role): PostgreSQL (migrations at boot on the direct
+# connection, then PgBouncer) is required, the process exits and is restarted until it answers;
+# Valkey is shared state only (without it everything shared is counted per process).
+After=network.target postgresql.service pgbouncer.service valkey-server.service
+Wants=openvibe-network.service openvibe-billing.service postgresql.service pgbouncer.service valkey-server.service
[Service]
[Service]
Type=simple
Type=simple
@@ -19,7 +22,9 @@ Environment=PATH=/usr/local/bin:/usr/bin:/bin
@@ -19,7 +22,9 @@ Environment=PATH=/usr/local/bin:/usr/bin:/bin
ExecStart=/usr/bin/env node /opt/openvibe.tips/server/index.js
ExecStart=/usr/bin/env node /opt/openvibe.tips/server/index.js
Restart=always
Restart=always
RestartSec=5
RestartSec=5
-# Tips' own database (ADR-007: one database per service); the code tree stays read-only.
+# DATABASE_URL, DATABASE_DIRECT_URL, VALKEY_URL and VALKEY_PREFIX come from the env file
+# (roles/data/add-service.sh tips). The state directory keeps the SQLite file of the release before
+# PostgreSQL: read-only rollback for the 7-day N-1 window, read by scripts/migrate-to-postgres.js.
StateDirectory=openvibe-tips
StateDirectory=openvibe-tips
StateDirectoryMode=0700
StateDirectoryMode=0700
Environment=TIPS_DB_PATH=/var/lib/openvibe-tips/tips.db
Environment=TIPS_DB_PATH=/var/lib/openvibe-tips/tips.db
new file mode 100644
new file mode 100644
@@ -0,0 +1,365 @@
@@ -0,0 +1,365 @@
+-- phase: expand
+--
+-- OpenVibe.Tips on PostgreSQL (ADR-035): the SQLite schema of the last SQLite release (server/db.js,
+-- schema v3 with its added columns; the SDK outbox and inbox) in PostgreSQL types, so
+-- scripts/migrate-to-postgres.js can import the SQLite file as it is.
+--
+-- ISO-8601 text times -> timestamptz (the data layer returns the same ISO strings the API sent)
+-- epoch-ms integers -> bigint (next_transfer_at, next_attempt_at, the outbox's times)
+-- 0/1 integers -> boolean
+-- JSON text -> jsonb, except api_idempotency.response (below)
+-- amounts (bits/cents) -> bigint; SUM() of a bigint is numeric, so queries cast sums back ::bigint
+-- AUTOINCREMENT -> bigint GENERATED ALWAYS AS IDENTITY (the importer keeps the values)
+--
+-- Not carried over: the SQLite `settings` table (one row, schema_version 3): ov_migrations records the
+-- schema now, and readiness asks the database itself (db.ready()).
+--
+-- Every query shape the code runs has an index; the comment above each index names the queries.
+-- Locks are always taken in one order: a creator's profile row, tip_interactions, tip_goals by id, the
+-- creator's overlay stream (an advisory lock), then the rest (server/domain/index.js).
+
+-- ── Creator profiles ────────────────────────────────────────────────────────────────────────────
+CREATE TABLE creator_tip_profiles (
+ creator_subject text PRIMARY KEY, -- usr_…
+ handle text NOT NULL UNIQUE, -- lower-case Network username (projection)
+ display_name text NOT NULL,
+ avatar_url text,
+ headline text,
+ page_enabled boolean NOT NULL DEFAULT false, -- public page + indexing
+ accepting boolean NOT NULL DEFAULT true, -- new tips accepted
+ min_amount bigint NOT NULL DEFAULT 1,
+ paid_message_min bigint NOT NULL DEFAULT 100,
+ tts_enabled boolean NOT NULL DEFAULT false,
+ tts_min_amount bigint NOT NULL DEFAULT 100,
+ tts_max_chars integer NOT NULL DEFAULT 200,
+ tts_voice text NOT NULL DEFAULT 'gary',
+ media_requests_enabled boolean NOT NULL DEFAULT false,
+ media_request_min bigint NOT NULL DEFAULT 25,
+ media_max_seconds integer NOT NULL DEFAULT 600,
+ revision integer NOT NULL DEFAULT 1,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL,
+ page_settings jsonb NOT NULL DEFAULT '{}', -- what the public goal / supporters pages show
+ filter_words jsonb NOT NULL DEFAULT '[]', -- the creator's blocklist (filter.js)
+ filter_action text NOT NULL DEFAULT 'mask', -- mask | hold
+ filter_links boolean NOT NULL DEFAULT true -- links in paid messages shown as [link]
+);
+-- Home page and sitemap: switched-on pages, newest first / by handle.
+CREATE INDEX creator_tip_profiles_enabled ON creator_tip_profiles (updated_at DESC) WHERE page_enabled;
+
+-- ── Interactions ────────────────────────────────────────────────────────────────────────────────
+CREATE TABLE tip_interactions (
+ id text PRIMARY KEY, -- tint_<ULID>
+ creator_subject text NOT NULL,
+ supporter_subject text, -- null: external/anonymous/unmapped/erased
+ supporter_name text, -- display projection at the time
+ kind text NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
+ amount bigint NOT NULL CHECK (amount > 0),
+ currency text NOT NULL DEFAULT 'vibes-bits',
+ amount_cents bigint, -- money that arrived (provider/external)
+ message text,
+ request jsonb NOT NULL DEFAULT '{}', -- tts text/voice, media url, goal_id, highlight
+ funding text NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
+ settlement text NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
+ payment_state text NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
+ delivery_state text NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
+ test boolean NOT NULL DEFAULT false, -- simulation or Billing test money: never counted
+ billing_txn_id text UNIQUE, -- the Billing transaction that settled it
+ billing_intent_id text UNIQUE, -- checkout: the Billing payment intent
+ funding_txn_id text UNIQUE, -- checkout: the purchase that funded it
+ reversal_txn_ids jsonb NOT NULL DEFAULT '[]',
+ reversed_bits bigint NOT NULL DEFAULT 0, -- bits Billing took back from the creator
+ checkout_url text,
+ checkout_ref text,
+ provider text,
+ provider_ref text,
+ legacy_source text, -- 'live:transactions:<id>' etc.
+ idempotency_key text UNIQUE,
+ failure text,
+ origin text NOT NULL DEFAULT 'tips', -- tips | billing | external | billing-external | import
+ transfer_due boolean NOT NULL DEFAULT false, -- a Billing transfer still has to be (re)tried
+ transfer_attempts integer NOT NULL DEFAULT 0,
+ next_transfer_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the lease of the transfer job
+ target jsonb, -- EntityRef (e.g. a Live stream)
+ created_at timestamptz NOT NULL,
+ settled_at timestamptz,
+ reversed_at timestamptz,
+ delivered_at timestamptz,
+ updated_at timestamptz NOT NULL,
+ anonymous boolean NOT NULL DEFAULT false, -- "Anonymous" to everyone but the supporter
+ hide_amount boolean NOT NULL DEFAULT false, -- amount left out of overlays, chat, public pages
+ private_message boolean NOT NULL DEFAULT false, -- a tip's message is for the creator only
+ erased_at timestamptz, -- the supporter erased their data from it
+ moderation text NOT NULL DEFAULT 'visible', -- visible | held | hidden
+ moderated_at timestamptz,
+ moderated_by text, -- 'filter', usr_… or svc:…
+ filtered boolean NOT NULL DEFAULT false, -- the word filter matched it at settlement
+ UNIQUE (provider, provider_ref),
+ UNIQUE (legacy_source)
+);
+-- A creator's interactions, newest first, keyset-paged (list, dashboard, moderation queue "all"); totals.
+CREATE INDEX tip_interactions_creator ON tip_interactions (creator_subject, created_at, id);
+-- A supporter's receipts, newest first, keyset-paged; export; erasure.
+CREATE INDEX tip_interactions_supporter ON tip_interactions (supporter_subject, created_at, id) WHERE supporter_subject IS NOT NULL;
+-- The moderation queue by state, keyset-paged; the dashboard's held count.
+CREATE INDEX tip_interactions_moderation ON tip_interactions (creator_subject, moderation, created_at, id);
+-- Unpaid checkouts one supporter holds (the pending-checkout limit).
+CREATE INDEX tip_interactions_open_checkouts ON tip_interactions (supporter_subject, created_at)
+ WHERE funding = 'checkout' AND payment_state = 'pending';
+-- Due Billing transfers, claimed with a lease (FOR UPDATE SKIP LOCKED).
+CREATE INDEX tip_interactions_transfer_due ON tip_interactions (next_transfer_at) WHERE payment_state = 'pending' AND transfer_due;
+-- The public supporters page: the leaderboard (settled or reversed, public name and amount).
+CREATE INDEX tip_interactions_leaderboard ON tip_interactions (creator_subject, supporter_subject)
+ WHERE supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed') AND NOT anonymous
+ AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible';
+-- The public supporters page: recent public messages.
+CREATE INDEX tip_interactions_recent_public ON tip_interactions (creator_subject, settled_at, id)
+ WHERE payment_state = 'settled' AND message IS NOT NULL;
+
+-- ── Goals ───────────────────────────────────────────────────────────────────────────────────────
+CREATE TABLE tip_goals (
+ id text PRIMARY KEY, -- tgoal_<ULID>
+ creator_subject text NOT NULL,
+ title text NOT NULL,
+ description text,
+ target_amount bigint NOT NULL CHECK (target_amount > 0),
+ currency text NOT NULL DEFAULT 'vibes-bits',
+ image_url text,
+ status text NOT NULL CHECK (status IN ('active', 'closed')),
+ sort_order integer NOT NULL DEFAULT 0,
+ reached_at timestamptz,
+ closed_at timestamptz,
+ opening_amount bigint NOT NULL DEFAULT 0, -- carried over from Live's goal column
+ legacy_source text UNIQUE,
+ revision integer NOT NULL DEFAULT 1,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL
+);
+-- A creator's goals (by status), in the creator's order; active count; the goal a tip counts toward.
+CREATE INDEX tip_goals_creator ON tip_goals (creator_subject, status, sort_order, created_at);
+
+CREATE TABLE tip_goal_contributions (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ goal_id text NOT NULL REFERENCES tip_goals(id),
+ interaction_id text NOT NULL REFERENCES tip_interactions(id),
+ amount bigint NOT NULL CHECK (amount > 0),
+ reversed_amount bigint NOT NULL DEFAULT 0,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL,
+ UNIQUE (goal_id, interaction_id)
+);
+-- Goal totals (goal_id = ANY), the latest contributions of a goal (goal_id, id DESC).
+CREATE INDEX tip_goal_contributions_goal ON tip_goal_contributions (goal_id, id);
+-- A reversal's contributions; a supporter's export.
+CREATE INDEX tip_goal_contributions_interaction ON tip_goal_contributions (interaction_id);
+
+-- ── What settlement creates ─────────────────────────────────────────────────────────────────────
+CREATE TABLE paid_messages (
+ id text PRIMARY KEY, -- tpm_<ULID>
+ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),
+ creator_subject text NOT NULL,
+ kind text NOT NULL CHECK (kind IN ('paid_message', 'tts')),
+ text text NOT NULL,
+ voice text,
+ highlight_seconds integer NOT NULL DEFAULT 0,
+ status text NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
+ chat_ref jsonb, -- the reference the chat adapter returned
+ test boolean NOT NULL DEFAULT false,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL
+);
+
+CREATE TABLE paid_media_requests (
+ id text PRIMARY KEY, -- tmr_<ULID>
+ interaction_id text NOT NULL UNIQUE REFERENCES tip_interactions(id),
+ creator_subject text NOT NULL,
+ url text NOT NULL,
+ provider text,
+ status text NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
+ queue_ref jsonb, -- the reference returned by the queue owner
+ test boolean NOT NULL DEFAULT false,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL
+);
+
+-- ── Overlays ────────────────────────────────────────────────────────────────────────────────────
+CREATE TABLE overlay_configs (
+ id text PRIMARY KEY, -- tovc_<ULID>
+ creator_subject text NOT NULL,
+ kind text NOT NULL CHECK (kind IN ('alerts', 'goal')),
+ name text NOT NULL,
+ settings jsonb NOT NULL DEFAULT '{}',
+ goal_id text,
+ revision integer NOT NULL DEFAULT 1,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL
+);
+-- A creator's configs in creation order.
+CREATE INDEX overlay_configs_creator ON overlay_configs (creator_subject, created_at);
+
+CREATE TABLE overlay_tokens (
+ id text PRIMARY KEY, -- tovt_<ULID>
+ creator_subject text NOT NULL,
+ token_hash text NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
+ scopes jsonb NOT NULL, -- ["alerts","goals"]
+ config_id text,
+ label text,
+ created_by text NOT NULL,
+ created_at timestamptz NOT NULL,
+ last_used_at timestamptz,
+ revoked_at timestamptz
+);
+-- A creator's tokens (listing, the active count).
+CREATE INDEX overlay_tokens_creator ON overlay_tokens (creator_subject, created_at);
+
+CREATE TABLE overlay_deliveries (
+ seq bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, -- the SSE event id (monotonic)
+ id text NOT NULL UNIQUE, -- tovd_<ULID>
+ creator_subject text NOT NULL,
+ kind text NOT NULL CHECK (kind IN ('alert', 'goal')),
+ interaction_id text,
+ goal_id text,
+ payload jsonb NOT NULL,
+ test boolean NOT NULL DEFAULT false,
+ status text NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
+ delivered_at timestamptz,
+ failed_at timestamptz,
+ sends integer NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
+ dedupe_key text NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
+ created_at timestamptz NOT NULL,
+ hidden boolean NOT NULL DEFAULT false -- retracted: never sent or replayed again
+);
+-- An overlay's stream: a creator's rows after a seq (push, replay, pending on connect, /state, max seq).
+CREATE INDEX overlay_deliveries_creator ON overlay_deliveries (creator_subject, seq);
+-- The delivery window sweep and the pending gauge.
+CREATE INDEX overlay_deliveries_pending ON overlay_deliveries (seq) WHERE status = 'pending';
+-- An interaction's deliveries (erasure rewrite, moderation retract/restore).
+CREATE INDEX overlay_deliveries_interaction ON overlay_deliveries (interaction_id) WHERE interaction_id IS NOT NULL;
+
+-- ── Delivery effects (the effects worker claims due rows with a lease) ──────────────────────────
+CREATE TABLE interaction_effects (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ interaction_id text NOT NULL REFERENCES tip_interactions(id),
+ effect text NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
+ adapter text NOT NULL,
+ state text NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
+ attempts integer NOT NULL DEFAULT 0,
+ next_attempt_at bigint NOT NULL DEFAULT 0, -- epoch ms; also the worker's lease
+ last_error text,
+ result jsonb,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL,
+ UNIQUE (interaction_id, effect)
+);
+-- Due effects (the worker's claim) and the pending/due gauges.
+CREATE INDEX interaction_effects_due ON interaction_effects (next_attempt_at, id) WHERE state = 'queued';
+
+-- ── Live import bookkeeping ─────────────────────────────────────────────────────────────────────
+CREATE TABLE migration_maps (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ source text NOT NULL, -- 'live'
+ source_table text NOT NULL,
+ source_id text NOT NULL,
+ target_type text, -- 'interaction' | 'goal' | 'refund'
+ target_id text,
+ status text NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
+ reason text,
+ billing_txn_id text,
+ run_id text NOT NULL,
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL,
+ UNIQUE (source, source_table, source_id)
+);
+
+CREATE TABLE import_runs (
+ id text PRIMARY KEY,
+ source text NOT NULL,
+ dry_run boolean NOT NULL,
+ started_at timestamptz NOT NULL,
+ finished_at timestamptz,
+ report jsonb
+);
+
+-- ── Moderation ──────────────────────────────────────────────────────────────────────────────────
+CREATE TABLE tip_moderators (
+ creator_subject text NOT NULL,
+ moderator_subject text NOT NULL,
+ name text,
+ added_by text NOT NULL, -- the creator, a service, or 'invite:<id>'
+ created_at timestamptz NOT NULL,
+ removed_at timestamptz,
+ PRIMARY KEY (creator_subject, moderator_subject)
+);
+-- The creators a person moderates.
+CREATE INDEX tip_moderators_moderator ON tip_moderators (moderator_subject);
+
+CREATE TABLE tip_moderator_invites (
+ id text PRIMARY KEY, -- tmin_<ULID>
+ creator_subject text NOT NULL,
+ token_hash text NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
+ created_by text NOT NULL,
+ created_at timestamptz NOT NULL,
+ expires_at timestamptz NOT NULL,
+ used_at timestamptz,
+ used_by text,
+ revoked_at timestamptz
+);
+-- A creator's open invitations (listing and the open count).
+CREATE INDEX tip_moderator_invites_creator ON tip_moderator_invites (creator_subject, created_at);
+
+CREATE TABLE tip_moderation_log (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ creator_subject text NOT NULL,
+ interaction_id text NOT NULL,
+ action text NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
+ by_role text NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
+ actor text, -- usr_… / svc:… ; null for the filter
+ reason text,
+ created_at timestamptz NOT NULL
+);
+-- A creator's log, newest first.
+CREATE INDEX tip_moderation_log_creator ON tip_moderation_log (creator_subject, id);
+
+-- ── Stored API answers (Idempotency-Key) ────────────────────────────────────────────────────────
+-- response stays text: a replay returns the stored answer byte for byte, and jsonb would reorder keys.
+-- A supporter's erasure deletes theirs by key prefix or by their subject in the answer: a scan of at
+-- most a week of answers (pruned daily), for a rare action.
+CREATE TABLE api_idempotency (
+ key text PRIMARY KEY, -- <principal>:<Idempotency-Key>
+ request_hash text NOT NULL,
+ method text NOT NULL,
+ path text NOT NULL,
+ status integer NOT NULL,
+ response text NOT NULL,
+ created_at timestamptz NOT NULL
+);
+-- The weekly prune.
+CREATE INDEX api_idempotency_created ON api_idempotency (created_at);
+
+-- ── Events: the openvibe-sdk outbox and inbox (outboxSchema('tips_event_outbox'), inboxSchema('tips_event_inbox')) ──
+CREATE TABLE IF NOT EXISTS tips_event_outbox (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ event_id text NOT NULL UNIQUE,
+ envelope jsonb NOT NULL,
+ traceparent text,
+ created_at bigint NOT NULL,
+ attempts integer NOT NULL DEFAULT 0,
+ next_attempt_at bigint NOT NULL DEFAULT 0,
+ sent_at bigint,
+ seq bigint,
+ rejected_at bigint,
+ last_error text
+);
+CREATE INDEX IF NOT EXISTS tips_event_outbox_due ON tips_event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;
+CREATE INDEX IF NOT EXISTS tips_event_outbox_sent ON tips_event_outbox (sent_at) WHERE sent_at IS NOT NULL;
+-- Tips' own queries on it: the rejected count (/api/ready), and an erasure scrubbing the local copies
+-- of an interaction's events.
+CREATE INDEX tips_event_outbox_rejected ON tips_event_outbox (id) WHERE rejected_at IS NOT NULL;
+CREATE INDEX tips_event_outbox_interaction ON tips_event_outbox ((envelope->'subject'->>'id')) WHERE envelope->'subject'->>'type' = 'interaction';
+
+CREATE TABLE IF NOT EXISTS tips_event_inbox (
+ consumer text NOT NULL,
+ event_id text NOT NULL,
+ processed_at bigint NOT NULL,
+ PRIMARY KEY (consumer, event_id)
+);
@@ -13,15 +13,33 @@
@@ -13,15 +13,33 @@
"cookie-parser": "^1.4.7",
"cookie-parser": "^1.4.7",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
+ "iovalkey": "^0.4.0",
"jsonwebtoken": "^9.0.2",
"jsonwebtoken": "^9.0.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
+ "pg": "^8.23.0"
+ },
+ "devDependencies": {
+ "@electric-sql/pglite": "^0.5.8"
},
},
"engines": {
"engines": {
"node": ">=22.0.0"
"node": ">=22.0.0"
}
}
},
},
+ "node_modules/@electric-sql/pglite": {
+ "version": "0.5.8",
+ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",
+ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",
+ "devOptional": true,
+ "license": "Apache-2.0"
+ },
+ "node_modules/@iovalkey/commands": {
+ "version": "0.1.0",
+ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",
+ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",
+ "license": "MIT"
+ },
"node_modules/accepts": {
"node_modules/accepts": {
"version": "1.3.8",
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
"resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz",
@@ -223,6 +241,15 @@
@@ -223,6 +241,15 @@
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/cluster-key-slot": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
+ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/content-disposition": {
"node_modules/content-disposition": {
"version": "0.5.4",
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@ -305,6 +332,15 @@
@@ -305,6 +332,15 @@
"node": ">=4.0.0"
"node": ">=4.0.0"
}
}
},
},
+ "node_modules/denque": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
+ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10"
+ }
+ },
"node_modules/depd": {
"node_modules/depd": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -714,6 +750,49 @@
@@ -714,6 +750,49 @@
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/iovalkey": {
+ "version": "0.4.0",
+ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
+ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",
+ "license": "MIT",
+ "dependencies": {
+ "@iovalkey/commands": "^0.1.0",
+ "cluster-key-slot": "^1.1.0",
+ "debug": "^4.3.4",
+ "denque": "^2.1.0",
+ "lodash.defaults": "^4.2.0",
+ "lodash.isarguments": "^3.1.0",
+ "redis-errors": "^1.2.0",
+ "redis-parser": "^3.0.0",
+ "standard-as-callback": "^2.1.0"
+ },
+ "engines": {
+ "node": ">=18.12.0"
+ }
+ },
+ "node_modules/iovalkey/node_modules/debug": {
+ "version": "4.4.3",
+ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
+ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
+ "license": "MIT",
+ "dependencies": {
+ "ms": "^2.1.3"
+ },
+ "engines": {
+ "node": ">=6.0"
+ },
+ "peerDependenciesMeta": {
+ "supports-color": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/iovalkey/node_modules/ms": {
+ "version": "2.1.3",
+ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
+ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
+ "license": "MIT"
+ },
"node_modules/ipaddr.js": {
"node_modules/ipaddr.js": {
"version": "1.9.1",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -778,12 +857,24 @@
@@ -778,12 +857,24 @@
"safe-buffer": "^5.0.1"
"safe-buffer": "^5.0.1"
}
}
},
},
+ "node_modules/lodash.defaults": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
+ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
+ "license": "MIT"
+ },
"node_modules/lodash.includes": {
"node_modules/lodash.includes": {
"version": "4.3.0",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/lodash.isarguments": {
+ "version": "3.1.0",
+ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
+ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
+ "license": "MIT"
+ },
"node_modules/lodash.isboolean": {
"node_modules/lodash.isboolean": {
"version": "3.0.3",
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
@@ -999,12 +1090,32 @@
@@ -999,12 +1090,32 @@
}
}
},
},
"node_modules/openvibe-sdk": {
"node_modules/openvibe-sdk": {
- "version": "0.12.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "integrity": "sha512-iG1h905O1f7y/PnP8xLA6GCVzXEBDfSaBj2tc1ucNreWg2sT/AJWosb8NW0WRMK5WS4sIQ6tMsprmDLj/fjjeQ==",
+ "version": "0.15.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
+ "integrity": "sha512-5ZcqEZxRQjT15erqtEdJ4js3roSijUFiQZPbGIHpaOVFRJSI45ZCEgTBb9ByKNRdUNBl4ub9Vb1dF2CcCVJjrg==",
"license": "MIT",
"license": "MIT",
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
+ },
+ "peerDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
+ "better-sqlite3": ">=11",
+ "iovalkey": "^0.4.0",
+ "pg": "^8.23.0"
+ },
+ "peerDependenciesMeta": {
+ "@electric-sql/pglite": {
+ "optional": true
+ },
+ "better-sqlite3": {
+ "optional": true
+ },
+ "iovalkey": {
+ "optional": true
+ },
+ "pg": {
+ "optional": true
+ }
}
}
},
},
"node_modules/openvibe-shared": {
"node_modules/openvibe-shared": {
@@ -1031,6 +1142,134 @@
@@ -1031,6 +1142,134 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/pg": {
+ "version": "8.23.0",
+ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
+ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-connection-string": "^2.14.0",
+ "pg-pool": "^3.14.0",
+ "pg-protocol": "^1.16.0",
+ "pg-types": "2.2.0",
+ "pgpass": "1.0.5"
+ },
+ "engines": {
+ "node": ">= 16.0.0"
+ },
+ "optionalDependencies": {
+ "pg-cloudflare": "^1.4.0"
+ },
+ "peerDependencies": {
+ "pg-native": ">=3.0.1"
+ },
+ "peerDependenciesMeta": {
+ "pg-native": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/pg-cloudflare": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
+ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
+ "license": "MIT",
+ "optional": true
+ },
+ "node_modules/pg-connection-string": {
+ "version": "2.14.0",
+ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
+ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-int8": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
+ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
+ "license": "ISC",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/pg-pool": {
+ "version": "3.14.0",
+ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
+ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
+ "license": "MIT",
+ "peerDependencies": {
+ "pg": ">=8.0"
+ }
+ },
+ "node_modules/pg-protocol": {
+ "version": "1.16.0",
+ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
+ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-types": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
+ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-int8": "1.0.1",
+ "postgres-array": "~2.0.0",
+ "postgres-bytea": "~1.0.0",
+ "postgres-date": "~1.0.4",
+ "postgres-interval": "^1.1.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/pgpass": {
+ "version": "1.0.5",
+ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
+ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
+ "license": "MIT",
+ "dependencies": {
+ "split2": "^4.1.0"
+ }
+ },
+ "node_modules/postgres-array": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
+ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/postgres-bytea": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
+ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-date": {
+ "version": "1.0.7",
+ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
+ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-interval": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
+ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
+ "license": "MIT",
+ "dependencies": {
+ "xtend": "^4.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/prebuild-install": {
"node_modules/prebuild-install": {
"version": "7.1.3",
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -1154,6 +1393,27 @@
@@ -1154,6 +1393,27 @@
"node": ">= 6"
"node": ">= 6"
}
}
},
},
+ "node_modules/redis-errors": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
+ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/redis-parser": {
+ "version": "3.0.0",
+ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
+ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
+ "license": "MIT",
+ "dependencies": {
+ "redis-errors": "^1.0.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
"node_modules/require-from-string": {
"node_modules/require-from-string": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@ -1369,6 +1629,21 @@
@@ -1369,6 +1629,21 @@
"simple-concat": "^1.0.0"
"simple-concat": "^1.0.0"
}
}
},
},
+ "node_modules/split2": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+ "license": "ISC",
+ "engines": {
+ "node": ">= 10.x"
+ }
+ },
+ "node_modules/standard-as-callback": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
+ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
+ "license": "MIT"
+ },
"node_modules/statuses": {
"node_modules/statuses": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -1496,6 +1771,15 @@
@@ -1496,6 +1771,15 @@
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
"license": "ISC"
+ },
+ "node_modules/xtend": {
+ "version": "4.0.2",
+ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.4"
+ }
}
}
}
}
}
}
@@ -8,10 +8,17 @@
@@ -8,10 +8,17 @@
"start": "node server/index.js",
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"test": "node test/run.js",
"test": "node test/run.js",
+ "test:pg": "TIPS_TEST_STORE=pg node test/run.js",
"subscribe": "node scripts/subscribe.js",
"subscribe": "node scripts/subscribe.js",
- "import-live": "node scripts/import-live.js"
+ "import-live": "node scripts/import-live.js",
+ "migrate-to-postgres": "node scripts/migrate-to-postgres.js"
},
},
- "keywords": ["openvibe", "tips", "overlays", "creator"],
+ "keywords": [
+ "openvibe",
+ "tips",
+ "overlays",
+ "creator"
+ ],
"author": "OpenVibers",
"author": "OpenVibers",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"repository": "github:OpenVibers/OpenVibe.Tips",
"repository": "github:OpenVibers/OpenVibe.Tips",
@@ -20,12 +27,17 @@
@@ -20,12 +27,17 @@
"cookie-parser": "^1.4.7",
"cookie-parser": "^1.4.7",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
+ "iovalkey": "^0.4.0",
"jsonwebtoken": "^9.0.2",
"jsonwebtoken": "^9.0.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.49.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.15.0",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
+ "pg": "^8.23.0"
},
},
"engines": {
"engines": {
"node": ">=22.0.0"
"node": ">=22.0.0"
+ },
+ "devDependencies": {
+ "@electric-sql/pglite": "^0.5.8"
}
}
}
}
@@ -12,11 +12,15 @@
@@ -12,11 +12,15 @@
*
*
* With --billing-db the report ends with the reconciliation: per creator, Tips' settled total must
* With --billing-db the report ends with the reconciliation: per creator, Tips' settled total must
* equal Billing's; the exit code is 1 when it does not. Safe to re-run.
* equal Billing's; the exit code is 1 when it does not. Safe to re-run.
+ *
+ * Tips' side is its PostgreSQL database (DATABASE_URL; migrations applied with DATABASE_DIRECT_URL
+ * first). The two snapshots are other services' SQLite files, which is why this operator tool, and
+ * scripts/migrate-to-postgres.js, are the only code here that loads better-sqlite3.
*/
*/
const path = require('path');
const path = require('path');
const Database = require('better-sqlite3');
const Database = require('better-sqlite3');
const { loadConfig } = require('../server/config');
const { loadConfig } = require('../server/config');
-const { openDb } = require('../server/db');
+const { openDb, migrate } = require('../server/db');
const { createIdentity } = require('../server/network');
const { createIdentity } = require('../server/network');
const { createTipsOutbox } = require('../server/events/outbox');
const { createTipsOutbox } = require('../server/events/outbox');
const { createDomain } = require('../server/domain');
const { createDomain } = require('../server/domain');
@@ -30,10 +34,11 @@ async function main() {
@@ -30,10 +34,11 @@ async function main() {
const livePath = opt('live-db');
const livePath = opt('live-db');
if (!livePath) { console.error('usage: import-live.js --live-db <snapshot> [--billing-db <snapshot>] [--dry-run] [--json]'); process.exit(2); }
if (!livePath) { console.error('usage: import-live.js --live-db <snapshot> [--billing-db <snapshot>] [--dry-run] [--json]'); process.exit(2); }
const config = loadConfig();
const config = loadConfig();
- for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.dbPath)) throw new Error(`${p} is the Tips database`);
+ for (const p of [livePath, opt('billing-db')].filter(Boolean)) if (path.resolve(p) === path.resolve(config.sqlitePath)) throw new Error(`${p} is Tips' own (pre-PostgreSQL) database`);
const live = new Database(livePath, { readonly: true, fileMustExist: true });
const live = new Database(livePath, { readonly: true, fileMustExist: true });
const bdb = opt('billing-db') ? new Database(opt('billing-db'), { readonly: true, fileMustExist: true }) : null;
const bdb = opt('billing-db') ? new Database(opt('billing-db'), { readonly: true, fileMustExist: true }) : null;
- const db = openDb(config.dbPath);
+ const db = openDb(config);
+ await migrate(config, { serving: db });
const outbox = createTipsOutbox({ db, config });
const outbox = createTipsOutbox({ db, config });
const domain = createDomain({ db, config, outbox, billing: null, adapters: {}, log: console });
const domain = createDomain({ db, config, outbox, billing: null, adapters: {}, log: console });
const identity = createIdentity(config);
const identity = createIdentity(config);
@@ -55,7 +60,7 @@ async function main() {
@@ -55,7 +60,7 @@ async function main() {
}
}
live.close();
live.close();
if (bdb) bdb.close();
if (bdb) bdb.close();
- db.close();
+ await db.close();
if (report.reconciliation && !report.reconciliation.ok) process.exit(1);
if (report.reconciliation && !report.reconciliation.ok) process.exit(1);
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,142 @@
@@ -0,0 +1,142 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * The one-time move of Tips' SQLite database into its PostgreSQL schema (ADR-035; the procedure is
+ * openvibe-sdk docs/migrating-to-postgresql.md, section 6).
+ *
+ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
+ *
+ * 1. applies migrations/ as the owner (DATABASE_DIRECT_URL, a direct connection);
+ * 2. copies every table with openvibe-sdk's importSqlite (batched, parents first, identities kept and
+ * their sequences advanced) into emptied tables (truncate: a rehearsal can be repeated);
+ * 3. verifies each table's row count and a checksum of every row on both sides, prints the report,
+ * and exits 1 unless report.ok.
+ *
+ * The SQLite file (TIPS_DB_PATH, config.sqlitePath; --sqlite for a copy) is opened read-only: nothing in
+ * it changes. --pglite imports into an in-memory PostgreSQL instead (a rehearsal with nothing to set up;
+ * the report is all it leaves). --json prints the whole report.
+ *
+ * In production it runs once, from the new release's directory, as the service user with the service's
+ * environment, while the service is stopped (the write freeze), before the PostgreSQL release starts.
+ *
+ * What does not map one to one:
+ * settings not carried: the SQLite bookkeeping row (schema_version 3); ov_migrations
+ * records the schema now (migrations/0001_initial.sql)
+ * event_outbox → tips_event_outbox (the SDK outbox, outboxSchema('tips_event_outbox'))
+ * idempotency_receipts → tips_event_inbox (the SDK inbox, inboxSchema('tips_event_inbox'))
+ * JSON text 'null' → SQL NULL in the reference columns (interaction_effects.result,
+ * paid_messages.chat_ref, paid_media_requests.queue_ref): the SQLite code
+ * wrote JSON.stringify(null) when an adapter returned no reference, the
+ * PostgreSQL code writes NULL, and both mean "no reference"
+ * NUL, unpaired surrogate PostgreSQL stores no NUL character in text or jsonb and no unpaired surrogate
+ * in jsonb (SQLite took both; the code strips them now): a NUL is dropped and an
+ * unpaired surrogate becomes U+FFFD, in text and inside JSON values alike. The
+ * report lists every column where that happened and how many values changed
+ * No column is dropped.
+ */
+const path = require('path');
+const { createDb, importSqlite } = require('openvibe-sdk/db');
+const { loadConfig } = require('../server/config');
+const { migrate, MIGRATIONS } = require('../server/db');
+
+const args = process.argv.slice(2);
+const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };
+const flag = (name) => args.includes(`--${name}`);
+
+/** A row map turning the JSON text 'null' into SQL NULL for these columns. */
+const jsonNullToSql = (...cols) => (row) => {
+ for (const c of cols) if (row[c] === 'null') row[c] = null;
+ return row;
+};
+
+const TABLES = {
+ tips_event_outbox: { from: 'event_outbox' },
+ tips_event_inbox: { from: 'idempotency_receipts' },
+ interaction_effects: { map: jsonNullToSql('result') },
+ paid_messages: { map: jsonNullToSql('chat_ref') },
+ paid_media_requests: { map: jsonNullToSql('queue_ref') },
+};
+
+const cleanText = (v) => v.toWellFormed().replace(/\u0000/g, '');
+function cleanJson(v) {
+ if (typeof v === 'string') return cleanText(v);
+ if (Array.isArray(v)) return v.map(cleanJson);
+ if (v && typeof v === 'object') return Object.fromEntries(Object.entries(v).map(([k, x]) => [cleanText(k), cleanJson(x)]));
+ return v;
+}
+
+/**
+ * importSqlite's `tables` for this schema: TABLES, and for every table a map that makes its text and
+ * JSON values storable (see above). `cleaned` counts the distinct values changed per column (the
+ * verification sees the same mapped values on both sides, so this is the record of what changed).
+ */
+async function importOptions(db) {
+ const cols = await db.many(`SELECT table_name, column_name, udt_name FROM information_schema.columns
+ WHERE table_schema = current_schema() AND table_name <> 'ov_migrations' AND udt_name IN ('text', 'varchar', 'json', 'jsonb')`);
+ const byTable = new Map();
+ for (const c of cols) { if (!byTable.has(c.table_name)) byTable.set(c.table_name, []); byTable.get(c.table_name).push(c); }
+ const cleaned = new Map(); // "table.column" → Set of the original values changed
+ const note = (t, c, v) => { const k = `${t}.${c}`; if (!cleaned.has(k)) cleaned.set(k, new Set()); cleaned.get(k).add(v); };
+ const tables = {};
+ for (const [table, list] of byTable) {
+ const base = TABLES[table] || {};
+ tables[table] = {
+ ...base,
+ map(row) {
+ const r = base.map ? base.map(row) : row;
+ if (!r) return r;
+ for (const { column_name: c, udt_name: udt } of list) {
+ const v = r[c];
+ if (typeof v !== 'string') continue;
+ if (udt === 'json' || udt === 'jsonb') {
+ let parsed;
+ try { parsed = JSON.parse(v); } catch { continue; } // left for the importer to report
+ const fixed = JSON.stringify(cleanJson(parsed));
+ if (fixed !== JSON.stringify(parsed)) { note(table, c, v); r[c] = fixed; }
+ } else {
+ const fixed = cleanText(v);
+ if (fixed !== v) { note(table, c, v); r[c] = fixed; }
+ }
+ }
+ return r;
+ },
+ };
+ }
+ return { tables, cleaned: () => [...cleaned].map(([column, values]) => ({ column, values: values.size })) };
+}
+
+async function main() {
+ const config = loadConfig();
+ const sqlite = path.resolve(opt('sqlite') || config.sqlitePath);
+ const quiet = { log() {}, warn: console.warn, error: console.error };
+ let owner;
+ if (flag('pglite')) {
+ owner = createDb({ pglite: true, service: 'tips-import' });
+ await owner.migrate({ dir: MIGRATIONS, log: quiet });
+ } else {
+ await migrate(config, { log: quiet });
+ owner = createDb({ url: config.db.directUrl, service: 'tips-import', max: 2 });
+ }
+ try {
+ const t0 = Date.now();
+ const options = await importOptions(owner);
+ const report = await importSqlite({ sqlite, db: owner, truncate: true, tables: options.tables, log: quiet });
+ const cleaned = options.cleaned();
+ if (flag('json')) console.log(JSON.stringify({ sqlite, into: owner.store, ...report, cleaned }, null, 2));
+ else {
+ console.log(`import ${sqlite} → ${owner.store} (${Date.now() - t0} ms): ${report.ok ? 'OK' : 'PROBLEMS'}`);
+ for (const t of report.tables) console.log(` ${t.table.padEnd(24)} ${String(t.rows).padStart(8)} rows ${t.checksum || '-'}${t.source !== t.table ? ` (from ${t.source})` : ''}`);
+ for (const c of cleaned) console.log(` cleaned: ${c.column}: ${c.values} value(s) with a NUL or an unpaired surrogate`);
+ for (const p of report.problems) console.log(` problem: ${p.table}: ${p.problem}`);
+ }
+ return report.ok ? 0 : 1;
+ } finally {
+ await owner.close();
+ }
+}
+
+if (require.main === module) {
+ main().then((code) => process.exit(code), (e) => { console.error(`migrate-to-postgres failed: ${e.message}`); process.exit(1); });
+}
+
+module.exports = { TABLES, importOptions };
@@ -8,13 +8,16 @@
@@ -8,13 +8,16 @@
* problem+json `rate_limited` with Retry-After before it does any work (before the idempotency store,
* problem+json `rate_limited` with Retry-After before it does any work (before the idempotency store,
* Billing or chat delivery); the refusal is logged once and counted in tips_rate_limited_total{limit,window}.
* Billing or chat delivery); the refusal is logged once and counted in tips_rate_limited_total{limit,window}.
* Reads get TIPS_LIMITS_MINUTE / TIPS_LIMITS_HOUR (120 and 3000); writes and payments set their own,
* Reads get TIPS_LIMITS_MINUTE / TIPS_LIMITS_HOUR (120 and 3000); writes and payments set their own,
- * tighter numbers in api/v1.js. Counters live in this process: a restart forgets them.
+ * tighter numbers in api/v1.js. With VALKEY_URL the counters are shared by every process and host
+ * (openvibe-sdk createValkeyLimitStore: one atomic script per request); if Valkey fails, a request is
+ * counted by its process's own counters instead, so an outage neither opens nor closes the gates.
+ * Without Valkey they live in this process (one process only).
*
*
* Never limited: /api/health, /api/ready, /release.json, /metrics, the overlays (a scoped token each,
* Never limited: /api/health, /api/ready, /release.json, /metrics, the overlays (a scoped token each,
* capped by TIPS_OVERLAY_MAX_STREAMS) and the signed Events deliveries at /internal/events (Billing's
* capped by TIPS_OVERLAY_MAX_STREAMS) and the signed Events deliveries at /internal/events (Billing's
* settlements: Events pushes at its own pace, and a 429 would only make it retry and fall behind).
* settlements: Events pushes at its own pace, and a 429 would only make it retry and fall behind).
*/
*/
-const { createActorLimiter, defaultActor } = require('openvibe-sdk/limits');
+const { createActorLimiter, createValkeyLimitStore, defaultActor } = require('openvibe-sdk/limits');
function actor(req) {
function actor(req) {
const p = req.principal;
const p = req.principal;
@@ -24,7 +27,7 @@ function actor(req) {
@@ -24,7 +27,7 @@ function actor(req) {
}
}
/** limits(name, own) middleware for one app, plus limits.reads(name): the defaults on GET/HEAD. */
/** limits(name, own) middleware for one app, plus limits.reads(name): the defaults on GET/HEAD. */
-function createActorLimits({ config, now = () => Date.now(), registry = null, log = console }) {
+function createActorLimits({ config, valkey = null, now = () => Date.now(), registry = null, log = console }) {
const refused = registry
const refused = registry
? registry.counter({ name: 'tips_rate_limited_total', help: 'Requests refused 429 by a per-actor limit, by limit name and window', labelNames: ['limit', 'window'] })
? registry.counter({ name: 'tips_rate_limited_total', help: 'Requests refused 429 by a per-actor limit, by limit name and window', labelNames: ['limit', 'window'] })
: null;
: null;
@@ -32,6 +35,8 @@ function createActorLimits({ config, now = () => Date.now(), registry = null, lo
@@ -32,6 +35,8 @@ function createActorLimits({ config, now = () => Date.now(), registry = null, lo
limits: { minute: config.actorLimits.minute, hour: config.actorLimits.hour },
limits: { minute: config.actorLimits.minute, hour: config.actorLimits.hour },
actor,
actor,
now,
now,
+ store: createValkeyLimitStore(valkey),
+ log,
onLimited(e) {
onLimited(e) {
// The actor is a principal, a subject id or an address, never a token.
// The actor is a principal, a subject id or an address, never a token.
log.warn(`[Tips] limit ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);
log.warn(`[Tips] limit ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);
@@ -6,12 +6,21 @@
@@ -6,12 +6,21 @@
* same key and request returns it verbatim (Idempotent-Replayed: true) and does nothing; the same
* same key and request returns it verbatim (Idempotent-Replayed: true) and does nothing; the same
* key with another request is 422 idempotency.key_reused. Refusals are not stored.
* key with another request is 422 idempotency.key_reused. Refusals are not stored.
*
*
+ * The key is claimed before the handler runs (a row with status 0), so the same key sent twice at once,
+ * to one process or two, runs the handler once: the other request is answered 409
+ * idempotency.in_progress with Retry-After, and a retry after the first finished gets the stored answer.
+ * A 2xx answer is written into the claim before it is sent; any other outcome releases the claim. A
+ * claim older than a minute (its process died mid-request) may be taken over.
+ *
* The scoped key also becomes the interaction's idempotency_key (UNIQUE), the second line of defence.
* The scoped key also becomes the interaction's idempotency_key (UNIQUE), the second line of defence.
*/
*/
const crypto = require('crypto');
const crypto = require('crypto');
const { http } = require('openvibe-contracts');
const { http } = require('openvibe-contracts');
+const { sql } = require('openvibe-sdk/db');
const KEY_RE = /^[A-Za-z0-9._:-]{8,200}$/;
const KEY_RE = /^[A-Za-z0-9._:-]{8,200}$/;
+const CLAIM_MS = 60_000;
+const PENDING = 0; // api_idempotency.status of a claim whose answer is not in yet
function stable(v) {
function stable(v) {
if (Array.isArray(v)) return `[${v.map(stable).join(',')}]`;
if (Array.isArray(v)) return `[${v.map(stable).join(',')}]`;
@@ -21,31 +30,62 @@ function stable(v) {
@@ -21,31 +30,62 @@ function stable(v) {
const principalKey = (p) => (p.kind === 'service' ? p.sub : p.kind === 'user' ? p.subject : 'anonymous');
const principalKey = (p) => (p.kind === 'service' ? p.sub : p.kind === 'user' ? p.subject : 'anonymous');
-function idempotent(db, now) {
- return function idempotencyKey(req, res, next) {
+function idempotent(db, now, log = console) {
+ const iso = (ms) => new Date(ms).toISOString();
+ return async function idempotencyKey(req, res, next) {
const key = req.get('Idempotency-Key');
const key = req.get('Idempotency-Key');
if (!key || !KEY_RE.test(key)) {
if (!key || !KEY_RE.test(key)) {
return http.sendProblem(res, 400, 'idempotency.key_required', { detail: 'mutating calls need an Idempotency-Key header (8-200 of A-Z a-z 0-9 . _ : -)', ctx: req.ov });
return http.sendProblem(res, 400, 'idempotency.key_required', { detail: 'mutating calls need an Idempotency-Key header (8-200 of A-Z a-z 0-9 . _ : -)', ctx: req.ov });
}
}
const scoped = `${principalKey(req.principal)}:${key}`;
const scoped = `${principalKey(req.principal)}:${key}`;
const hash = crypto.createHash('sha256').update(`${req.method} ${req.baseUrl}${req.path}\n${stable(req.body || {})}`).digest('hex');
const hash = crypto.createHash('sha256').update(`${req.method} ${req.baseUrl}${req.path}\n${stable(req.body || {})}`).digest('hex');
- const row = db.prepare('SELECT * FROM api_idempotency WHERE key = ?').get(scoped);
- if (row) {
- if (row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });
+ const path = `${req.baseUrl}${req.path}`;
+ let claimed;
+ let row = null;
+ try {
+ const at = now();
+ claimed = await db.maybe(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
+ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${PENDING}, '', ${iso(at)})
+ ON CONFLICT (key) DO UPDATE SET request_hash = excluded.request_hash, method = excluded.method, path = excluded.path, created_at = excluded.created_at
+ WHERE api_idempotency.status = ${PENDING} AND api_idempotency.created_at < ${iso(at - CLAIM_MS)}
+ RETURNING key`);
+ if (!claimed) row = await db.maybe(sql`SELECT request_hash, status, response FROM api_idempotency WHERE key = ${scoped}`);
+ } catch (e) { return next(e); }
+ if (!claimed) {
+ if (row && row.request_hash !== hash) return http.sendProblem(res, 422, 'idempotency.key_reused', { detail: 'this Idempotency-Key was used for a different request', ctx: req.ov });
+ if (!row || row.status === PENDING) {
+ res.setHeader('Retry-After', '1');
+ return http.sendProblem(res, 409, 'idempotency.in_progress', { detail: 'a request with this Idempotency-Key is still being handled; retry shortly for its answer', ctx: req.ov });
+ }
res.setHeader('Idempotent-Replayed', 'true');
res.setHeader('Idempotent-Replayed', 'true');
- return res.status(row.status).json(JSON.parse(row.response));
+ // The stored text is the answer as first sent (jsonb would reorder its keys).
+ return res.status(row.status).type('application/json').send(row.response);
}
}
req.idempotencyKey = `api:${scoped}`;
req.idempotencyKey = `api:${scoped}`;
+ let answered = false;
+ const release = () => {
+ if (answered) return;
+ answered = true;
+ db.exec(sql`DELETE FROM api_idempotency WHERE key = ${scoped} AND status = ${PENDING} AND request_hash = ${hash}`)
+ .catch((e) => log.warn(`[Tips] Idempotency-Key claim not released: ${e.message}`));
+ };
+ res.on('finish', release);
+ res.on('close', release);
const json = res.json.bind(res);
const json = res.json.bind(res);
res.json = (body) => {
res.json = (body) => {
- if (res.statusCode >= 200 && res.statusCode < 300) {
- // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the
- // replay gets that instead, so no secret is kept in plain text here.
- const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;
- db.prepare('INSERT OR IGNORE INTO api_idempotency (key, request_hash, method, path, status, response, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')
- .run(scoped, hash, req.method, `${req.baseUrl}${req.path}`, res.statusCode, JSON.stringify(stored), new Date(now()).toISOString());
- }
- return json(body);
+ if (answered || res.statusCode < 200 || res.statusCode >= 300) return json(body);
+ answered = true;
+ // A handler whose answer carries a secret (an overlay token) sets res.locals.storedBody: the
+ // replay gets that instead, so no secret is kept in plain text here. An upsert: an erasure
+ // during the request may have removed the claim.
+ const stored = res.locals.storedBody !== undefined ? res.locals.storedBody : body;
+ db.exec(sql`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
+ VALUES (${scoped}, ${hash}, ${req.method}, ${path}, ${res.statusCode}, ${JSON.stringify(stored)}, ${iso(now())})
+ ON CONFLICT (key) DO UPDATE SET status = excluded.status, response = excluded.response, created_at = excluded.created_at
+ WHERE api_idempotency.request_hash = excluded.request_hash`)
+ .catch((e) => log.warn(`[Tips] stored answer for an Idempotency-Key not kept: ${e.message}`))
+ .then(() => json(body));
+ return res;
};
};
return next();
return next();
};
};
@@ -57,7 +97,7 @@ function idempotent(db, now) {
@@ -57,7 +97,7 @@ function idempotent(db, now) {
*/
*/
const RETAIN_MS = 7 * 24 * 3600 * 1000;
const RETAIN_MS = 7 * 24 * 3600 * 1000;
function pruneAnswers(db, now = Date.now(), retainMs = RETAIN_MS) {
function pruneAnswers(db, now = Date.now(), retainMs = RETAIN_MS) {
- return db.prepare('DELETE FROM api_idempotency WHERE created_at < ?').run(new Date(now - retainMs).toISOString()).changes;
+ return db.exec(sql`DELETE FROM api_idempotency WHERE created_at < ${new Date(now - retainMs).toISOString()}`);
}
}
module.exports = { idempotent, stable, principalKey, pruneAnswers, RETAIN_MS };
module.exports = { idempotent, stable, principalKey, pruneAnswers, RETAIN_MS };
@@ -40,7 +40,7 @@
@@ -40,7 +40,7 @@
*/
*/
const express = require('express');
const express = require('express');
const { http } = require('openvibe-contracts');
const { http } = require('openvibe-contracts');
-const { TipsError, fail, userSubject, displayName } = require('../util');
+const { TipsError, fail, userSubject, displayName, inputError } = require('../util');
const { idempotent } = require('./idempotency');
const { idempotent } = require('./idempotency');
const CAP = {
const CAP = {
@@ -67,7 +67,7 @@ const CAP = {
@@ -67,7 +67,7 @@ const CAP = {
function v1Router({ domain, apiAuth, limits }) {
function v1Router({ domain, apiAuth, limits }) {
const r = express.Router();
const r = express.Router();
const { db, profiles, goals, interactions, overlays } = domain;
const { db, profiles, goals, interactions, overlays } = domain;
- const idem = idempotent(db, domain.now);
+ const idem = idempotent(db, domain.now, domain.log);
// Per-actor limits (api/actor-limits.js), after the credential and before any work: every read takes
// Per-actor limits (api/actor-limits.js), after the credential and before any work: every read takes
// the defaults; each write below names its own, tighter numbers.
// the defaults; each write below names its own, tighter numbers.
@@ -86,40 +86,40 @@ function v1Router({ domain, apiAuth, limits }) {
@@ -86,40 +86,40 @@ function v1Router({ domain, apiAuth, limits }) {
return null;
return null;
}
}
/** The creator a call is about: body/query `creator` (subject or handle), or the signed-in user. */
/** The creator a call is about: body/query `creator` (subject or handle), or the signed-in user. */
- function creatorOf(req, value, { mustExist = true } = {}) {
+ async function creatorOf(req, value, { mustExist = true } = {}) {
const v = value != null && value !== '' ? value : (req.principal.kind === 'user' ? req.principal.subject : null);
const v = value != null && value !== '' ? value : (req.principal.kind === 'user' ? req.principal.subject : null);
if (!v) fail(422, 'tips.invalid_input', 'creator is required');
if (!v) fail(422, 'tips.invalid_input', 'creator is required');
const ref = typeof v === 'object' ? userSubject(v, 'creator') : String(v);
const ref = typeof v === 'object' ? userSubject(v, 'creator') : String(v);
- const profile = ref === 'me' && req.principal.kind === 'user' ? profiles.bySubject(req.principal.subject) : profiles.resolve(ref);
+ const profile = ref === 'me' && req.principal.kind === 'user' ? await profiles.bySubject(db, req.principal.subject) : await profiles.resolve(db, ref);
if (!profile && mustExist) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
if (!profile && mustExist) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
return profile;
return profile;
}
}
const write = (handler) => [idem, wrap(handler)];
const write = (handler) => [idem, wrap(handler)];
// ── Profiles ─────────────────────────────────────────────
// ── Profiles ─────────────────────────────────────────────
- r.get('/profiles/:creator', wrap((req, res) => {
- const p = creatorOf(req, req.params.creator);
+ r.get('/profiles/:creator', wrap(async (req, res) => {
+ const p = await creatorOf(req, req.params.creator);
const who = req.principal;
const who = req.principal;
const mine = who.kind === 'user' && who.subject === p.creator_subject;
const mine = who.kind === 'user' && who.subject === p.creator_subject;
const svc = who.kind === 'service' && apiAuth.granted(who, CAP.profileGet);
const svc = who.kind === 'service' && apiAuth.granted(who, CAP.profileGet);
// A page that is switched off does not exist for anyone but its creator and granted services.
// A page that is switched off does not exist for anyone but its creator and granted services.
if (!p.page_enabled && !mine && !svc) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
if (!p.page_enabled && !mine && !svc) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
const owner = mine || svc;
const owner = mine || svc;
- const active = goals.list(p.creator_subject, { status: 'active' });
- res.json({ profile: profiles.present(p, { owner }), goals: active.map((g) => (owner ? goals.present(g) : goals.publicGoal(g, p.page))) });
+ const active = await goals.list(db, p.creator_subject, { status: 'active' });
+ res.json({ profile: profiles.present(p, { owner }), goals: owner ? await goals.presentMany(db, active) : await goals.publicGoals(db, active, p) });
}));
}));
/** The supporters page: top supporters and recent public messages, as the creator chose to show them. */
/** The supporters page: top supporters and recent public messages, as the creator chose to show them. */
- r.get('/profiles/:creator/supporters', wrap((req, res) => {
- const p = creatorOf(req, req.params.creator);
+ r.get('/profiles/:creator/supporters', wrap(async (req, res) => {
+ const p = await creatorOf(req, req.params.creator);
const who = req.principal;
const who = req.principal;
const owner = (who.kind === 'user' && who.subject === p.creator_subject) || (who.kind === 'service' && apiAuth.granted(who, CAP.profileGet));
const owner = (who.kind === 'user' && who.subject === p.creator_subject) || (who.kind === 'service' && apiAuth.granted(who, CAP.profileGet));
if (!owner && (!p.page_enabled || !p.page.supporters_page)) fail(404, 'tips.supporters_not_public', 'this creator does not show their supporters');
if (!owner && (!p.page_enabled || !p.page.supporters_page)) fail(404, 'tips.supporters_not_public', 'this creator does not show their supporters');
const page = p.page;
const page = p.page;
res.json({
res.json({
creator: { type: 'user', id: p.creator_subject }, page,
creator: { type: 'user', id: p.creator_subject }, page,
- leaderboard: interactions.leaderboard(p.creator_subject).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),
- recent: page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,
+ leaderboard: (await interactions.leaderboard(db, p)).map((row) => (page.supporters_amounts ? row : { ...row, total: null })),
+ recent: page.supporters_messages ? (await interactions.recentPublic(db, p)).map((v) => (page.supporters_amounts ? v : { ...v, amount: null })) : null,
});
});
}));
}));
@@ -128,34 +128,34 @@ function v1Router({ domain, apiAuth, limits }) {
@@ -128,34 +128,34 @@ function v1Router({ domain, apiAuth, limits }) {
if (req.principal.kind !== 'user') fail(req.principal.kind === 'anonymous' ? 401 : 403, req.principal.kind === 'anonymous' ? 'token.missing' : 'tips.people_only', 'only the person themselves (a Network user token) can do this');
if (req.principal.kind !== 'user') fail(req.principal.kind === 'anonymous' ? 401 : 403, req.principal.kind === 'anonymous' ? 'token.missing' : 'tips.people_only', 'only the person themselves (a Network user token) can do this');
return req.principal.subject;
return req.principal.subject;
};
};
- r.get('/me/export', wrap((req, res) => res.json(interactions.exportFor(person(req)))));
+ r.get('/me/export', wrap(async (req, res) => res.json(await interactions.exportFor(person(req)))));
// A person erasing their own tips does it once; the network's account deletion arrives through
// A person erasing their own tips does it once; the network's account deletion arrives through
// /internal/events, which is never limited.
// /internal/events, which is never limited.
- r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write((req, res) => res.json(interactions.erase(person(req)))));
+ r.post('/me/erase', limits('tips.me.erase', { minute: 3, hour: 10 }), ...write(async (req, res) => res.json(await interactions.erase(person(req)))));
- r.patch('/profiles/:creator', edit, ...write((req, res) => {
+ r.patch('/profiles/:creator', edit, ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
let p;
let p;
if (req.params.creator === 'me') {
if (req.params.creator === 'me') {
if (req.principal.kind !== 'user') fail(401, 'token.missing', '`me` needs a user token');
if (req.principal.kind !== 'user') fail(401, 'token.missing', '`me` needs a user token');
- p = profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });
- } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !profiles.bySubject(req.params.creator)) {
+ p = await profiles.ensure(req.principal.subject, { username: req.principal.username, displayName: req.principal.name, avatarUrl: req.principal.avatar });
+ } else if (req.principal.kind === 'service' && /^usr_/.test(req.params.creator) && !(await profiles.bySubject(db, req.params.creator))) {
allow(req, CAP.profileUpdate);
allow(req, CAP.profileUpdate);
// A service opening a profile for a creator names the account projection it knows.
// A service opening a profile for a creator names the account projection it knows.
if (!b.handle) fail(422, 'tips.invalid_input', 'handle is required to create a profile');
if (!b.handle) fail(422, 'tips.invalid_input', 'handle is required to create a profile');
- p = profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });
+ p = await profiles.ensure(userSubject(req.params.creator, 'creator'), { username: b.handle, displayName: b.display_name || b.handle, avatarUrl: b.avatar_url });
} else {
} else {
- p = creatorOf(req, req.params.creator);
+ p = await creatorOf(req, req.params.creator);
}
}
allow(req, CAP.profileUpdate, p.creator_subject);
allow(req, CAP.profileUpdate, p.creator_subject);
- const out = profiles.update(p.creator_subject, b, { expectedRevision: b.revision });
+ const out = await profiles.update(p.creator_subject, b, { expectedRevision: b.revision });
res.json({ profile: profiles.present(out, { owner: true }) });
res.json({ profile: profiles.present(out, { owner: true }) });
}));
}));
- r.get('/profiles/:creator/totals', wrap((req, res) => {
- const p = creatorOf(req, req.params.creator);
+ r.get('/profiles/:creator/totals', wrap(async (req, res) => {
+ const p = await creatorOf(req, req.params.creator);
allow(req, CAP.interactionList, p.creator_subject);
allow(req, CAP.interactionList, p.creator_subject);
- res.json({ creator: { type: 'user', id: p.creator_subject }, totals: interactions.totals(p.creator_subject) });
+ res.json({ creator: { type: 'user', id: p.creator_subject }, totals: await interactions.totals(db, p.creator_subject) });
}));
}));
// ── Paid requests (checkout / credit) ────────────────────
// ── Paid requests (checkout / credit) ────────────────────
@@ -174,28 +174,27 @@ function v1Router({ domain, apiAuth, limits }) {
@@ -174,28 +174,27 @@ function v1Router({ domain, apiAuth, limits }) {
supporter = p.subject;
supporter = p.subject;
supporterName = displayName(b.supporter_name) || p.name || p.username;
supporterName = displayName(b.supporter_name) || p.name || p.username;
}
}
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
const funding = b.pay_with || 'credit';
const funding = b.pay_with || 'credit';
const tp = req.ov && req.ov.traceparent;
const tp = req.ov && req.ov.traceparent;
- const { interaction, replay } = interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });
+ const { interaction, replay } = await interactions.request(profile, { ...b, kind }, { supporter, supporterName, funding, idempotencyKey: req.idempotencyKey });
let i = interaction;
let i = interaction;
- let checkout = null;
+ const viewer = p.kind === 'service' ? 'service' : 'supporter';
if (!replay) {
if (!replay) {
if (funding === 'credit') {
if (funding === 'credit') {
const out = await interactions.transfer(i, { traceparent: tp });
const out = await interactions.transfer(i, { traceparent: tp });
- i = interactions.get(i.id);
+ i = await interactions.get(db, i.id);
if (out.refused) {
if (out.refused) {
- fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: interactions.present(i, { viewer: 'supporter' }) });
+ fail(out.refused === 'billing.insufficient_funds' ? 409 : 422, out.refused, out.detail || 'Billing refused the tip', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });
}
}
} else {
} else {
- checkout = await interactions.startCheckout(i, { provider: b.provider, traceparent: tp });
- i = checkout.interaction;
+ i = (await interactions.startCheckout(i, { provider: b.provider, traceparent: tp })).interaction;
}
}
} else if (i.payment_state === 'failed') {
} else if (i.payment_state === 'failed') {
- fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: interactions.present(i, { viewer: 'supporter' }) });
+ fail(409, 'tips.payment_failed', i.failure || 'the payment for this request failed', { interaction: await interactions.present(db, i, { viewer: 'supporter' }) });
}
}
res.status(replay ? 200 : 201).json({
res.status(replay ? 200 : 201).json({
- interaction: interactions.present(i, { viewer: p.kind === 'service' ? 'service' : 'supporter' }),
+ interaction: await interactions.present(db, i, { viewer }),
checkout_url: i.checkout_url || null, checkout_ref: i.checkout_ref || null,
checkout_url: i.checkout_url || null, checkout_ref: i.checkout_ref || null,
});
});
});
});
@@ -212,27 +211,29 @@ function v1Router({ domain, apiAuth, limits }) {
@@ -212,27 +211,29 @@ function v1Router({ domain, apiAuth, limits }) {
r.post('/media-requests', pays, ...paid('media_request', CAP.media));
r.post('/media-requests', pays, ...paid('media_request', CAP.media));
// ── Interactions ─────────────────────────────────────────
// ── Interactions ─────────────────────────────────────────
- r.get('/interactions', wrap((req, res) => {
+ // The hottest list: a keyset page of any size costs three queries (the page, the creators' filters,
+ // the effects) — test/query-budget.test.js holds it there.
+ r.get('/interactions', wrap(async (req, res) => {
const p = req.principal;
const p = req.principal;
const q = req.query;
const q = req.query;
let filter;
let filter;
if (p.kind === 'service') {
if (p.kind === 'service') {
allow(req, CAP.interactionList);
allow(req, CAP.interactionList);
filter = {
filter = {
- creator: q.creator ? (creatorOf(req, q.creator).creator_subject) : null,
+ creator: q.creator ? (await creatorOf(req, q.creator)).creator_subject : null,
supporter: q.supporter ? userSubject(String(q.supporter), 'supporter') : null,
supporter: q.supporter ? userSubject(String(q.supporter), 'supporter') : null,
};
};
if (!filter.creator && !filter.supporter) fail(422, 'tips.invalid_input', 'creator or supporter is required');
if (!filter.creator && !filter.supporter) fail(422, 'tips.invalid_input', 'creator or supporter is required');
} else if (p.kind === 'user') {
} else if (p.kind === 'user') {
filter = q.as === 'creator' ? { creator: p.subject } : { supporter: p.subject };
filter = q.as === 'creator' ? { creator: p.subject } : { supporter: p.subject };
} else fail(401, 'token.missing', 'sign in to see receipts');
} else fail(401, 'token.missing', 'sign in to see receipts');
- const out = interactions.list({ ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });
+ const out = await interactions.list(db, { ...filter, includeTest: q.include_test !== '0', cursor: q.cursor, limit: q.limit });
const viewer = p.kind === 'service' ? 'service' : (filter.creator ? 'owner' : 'supporter');
const viewer = p.kind === 'service' ? 'service' : (filter.creator ? 'owner' : 'supporter');
- res.json({ interactions: out.rows.map((i) => interactions.present(i, { viewer })), next_cursor: out.next_cursor });
+ res.json({ interactions: await interactions.presentMany(db, out.rows, { viewer }), next_cursor: out.next_cursor });
}));
}));
- r.get('/interactions/:id', wrap((req, res) => {
- const i = interactions.get(req.params.id);
+ r.get('/interactions/:id', wrap(async (req, res) => {
+ const i = await interactions.get(db, req.params.id);
const p = req.principal;
const p = req.principal;
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
let viewer;
let viewer;
@@ -241,164 +242,160 @@ function v1Router({ domain, apiAuth, limits }) {
@@ -241,164 +242,160 @@ function v1Router({ domain, apiAuth, limits }) {
else if (p.kind === 'user' && p.subject === i.supporter_subject) viewer = 'supporter';
else if (p.kind === 'user' && p.subject === i.supporter_subject) viewer = 'supporter';
else if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in to see this receipt');
else if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in to see this receipt');
else fail(404, 'tips.interaction_not_found', 'no such interaction');
else fail(404, 'tips.interaction_not_found', 'no such interaction');
- res.json({ interaction: interactions.present(i, { viewer }) });
+ res.json({ interaction: await interactions.present(db, i, { viewer }) });
}));
}));
// A service importing tips made elsewhere records one per tip, at most one a second.
// A service importing tips made elsewhere records one per tip, at most one a second.
- r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write((req, res) => {
+ r.post('/interactions/external', limits('tips.interaction.record', { minute: 60, hour: 1200 }), ...write(async (req, res) => {
allow(req, CAP.interactionRecord);
allow(req, CAP.interactionRecord);
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
- const out = interactions.recordExternal(profile, b);
- res.status(out.replay ? 200 : 201).json({ interaction: interactions.present(out.interaction, { viewer: 'service' }), duplicate: out.replay });
+ const profile = await creatorOf(req, b.creator);
+ const out = await interactions.recordExternal(profile, b);
+ res.status(out.replay ? 200 : 201).json({ interaction: await interactions.present(db, out.interaction, { viewer: 'service' }), duplicate: out.replay });
}));
}));
// ── Goals ────────────────────────────────────────────────
// ── Goals ────────────────────────────────────────────────
- function goalVisible(req, g) {
- const profile = profiles.bySubject(g.creator_subject);
- if (profile && profile.page_enabled) return true;
- const p = req.principal;
- return (p.kind === 'user' && p.subject === g.creator_subject) || (p.kind === 'service' && apiAuth.granted(p, CAP.goalUpdate));
- }
- const goalOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));
- r.get('/goals', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
- const sample = { creator_subject: profile.creator_subject };
- if (!goalVisible(req, sample)) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
+ const isOwner = (req, creator) => (req.principal.kind === 'user' && req.principal.subject === creator) || (req.principal.kind === 'service' && apiAuth.granted(req.principal, CAP.goalUpdate));
+ r.get('/goals', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
+ const full = isOwner(req, profile.creator_subject);
+ if (!profile.page_enabled && !full) fail(404, 'tips.creator_not_found', 'no tip profile for this creator');
const status = ['active', 'closed'].includes(req.query.status) ? req.query.status : undefined;
const status = ['active', 'closed'].includes(req.query.status) ? req.query.status : undefined;
- const full = goalOwner(req, profile.creator_subject);
- res.json({ goals: goals.list(profile.creator_subject, { status }).map((g) => (full ? goals.present(g) : goals.publicGoal(g, profile.page))) });
+ const list = await goals.list(db, profile.creator_subject, { status });
+ res.json({ goals: full ? await goals.presentMany(db, list) : await goals.publicGoals(db, list, profile) });
}));
}));
- r.get('/goals/:id', wrap((req, res) => {
- const g = goals.get(req.params.id);
- if (!g || !goalVisible(req, g)) fail(404, 'tips.goal_not_found', 'no such goal');
- if (!goalOwner(req, g.creator_subject)) return res.json({ goal: goals.publicGoal(g, profiles.bySubject(g.creator_subject).page) });
- res.json({ goal: goals.present(g, { contributions: true }) });
+ r.get('/goals/:id', wrap(async (req, res) => {
+ const g = await goals.get(db, req.params.id);
+ const profile = g && await profiles.bySubject(db, g.creator_subject);
+ const full = g && isOwner(req, g.creator_subject);
+ if (!g || (!(profile && profile.page_enabled) && !full)) fail(404, 'tips.goal_not_found', 'no such goal');
+ if (!full) return res.json({ goal: await goals.publicGoal(db, g, profile) });
+ res.json({ goal: await goals.present(db, g, { contributions: true }) });
}));
}));
- r.post('/goals', edit, ...write((req, res) => {
+ r.post('/goals', edit, ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
allow(req, CAP.goalCreate, profile.creator_subject);
allow(req, CAP.goalCreate, profile.creator_subject);
- res.status(201).json({ goal: goals.create(profile.creator_subject, b) });
+ res.status(201).json({ goal: await goals.create(profile.creator_subject, b) });
}));
}));
- const ownGoal = (req, cap) => {
- const g = goals.get(req.params.id);
+ const ownGoal = async (req, cap) => {
+ const g = await goals.get(db, req.params.id);
if (!g) fail(404, 'tips.goal_not_found', 'no such goal');
if (!g) fail(404, 'tips.goal_not_found', 'no such goal');
allow(req, cap, g.creator_subject);
allow(req, cap, g.creator_subject);
return g;
return g;
};
};
- r.patch('/goals/:id', edit, ...write((req, res) => res.json({ goal: goals.update(ownGoal(req, CAP.goalUpdate), req.body || {}) })));
- r.post('/goals/:id/close', edit, ...write((req, res) => res.json({ goal: goals.close(ownGoal(req, CAP.goalClose)) })));
+ r.patch('/goals/:id', edit, ...write(async (req, res) => res.json({ goal: await goals.update(await ownGoal(req, CAP.goalUpdate), req.body || {}) })));
+ r.post('/goals/:id/close', edit, ...write(async (req, res) => res.json({ goal: await goals.close(await ownGoal(req, CAP.goalClose)) })));
// ── Overlay tokens and configs ───────────────────────────
// ── Overlay tokens and configs ───────────────────────────
- r.get('/overlay-tokens', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
+ r.get('/overlay-tokens', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.tokenCreate, profile.creator_subject);
allow(req, CAP.tokenCreate, profile.creator_subject);
- res.json({ tokens: overlays.listTokens(profile.creator_subject).map(overlays.presentToken) });
+ res.json({ tokens: (await overlays.listTokens(db, profile.creator_subject)).map(overlays.presentToken) });
}));
}));
// A token opens an overlay's event stream: a creator mints a few for their scenes.
// A token opens an overlay's event stream: a creator mints a few for their scenes.
- r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write((req, res) => {
+ r.post('/overlay-tokens', limits('tips.overlay.token.create', { minute: 10, hour: 60 }), ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
allow(req, CAP.tokenCreate, profile.creator_subject);
allow(req, CAP.tokenCreate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
- const out = overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });
+ const out = await overlays.createToken(profile.creator_subject, { scopes: b.scopes, label: b.label, configId: b.config_id, createdBy: by });
// Shown once: only its hash is stored, and the stored answer for an Idempotency-Key replay has no
// Shown once: only its hash is stored, and the stored answer for an Idempotency-Key replay has no
// secret either (a caller who lost the answer revokes the token and creates another).
// secret either (a caller who lost the answer revokes the token and creates another).
res.locals.storedBody = { token: out.token, secret: null, overlay_url: null, events_url: null, note: 'the secret is shown once; revoke this token and create another' };
res.locals.storedBody = { token: out.token, secret: null, overlay_url: null, events_url: null, note: 'the secret is shown once; revoke this token and create another' };
res.status(201).json({ token: out.token, secret: out.secret, overlay_url: out.overlay_url, events_url: out.events_url });
res.status(201).json({ token: out.token, secret: out.secret, overlay_url: out.overlay_url, events_url: out.events_url });
}));
}));
- r.post('/overlay-tokens/:id/revoke', edit, ...write((req, res) => {
- const t = overlays.getToken(req.params.id);
+ r.post('/overlay-tokens/:id/revoke', edit, ...write(async (req, res) => {
+ const t = await overlays.getToken(db, req.params.id);
if (!t) fail(404, 'tips.overlay_token_not_found', 'no such overlay token');
if (!t) fail(404, 'tips.overlay_token_not_found', 'no such overlay token');
allow(req, CAP.tokenRevoke, t.creator_subject);
allow(req, CAP.tokenRevoke, t.creator_subject);
- res.json({ token: overlays.revokeToken(t) });
+ res.json({ token: await overlays.revokeToken(t) });
}));
}));
- r.get('/overlay-configs', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
+ r.get('/overlay-configs', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.configGet, profile.creator_subject);
allow(req, CAP.configGet, profile.creator_subject);
- res.json({ configs: overlays.listConfigs(profile.creator_subject).map(overlays.presentConfig) });
+ res.json({ configs: (await overlays.listConfigs(db, profile.creator_subject)).map(overlays.presentConfig) });
}));
}));
- r.get('/overlay-configs/:id', wrap((req, res) => {
- const c = overlays.getConfig(req.params.id);
+ r.get('/overlay-configs/:id', wrap(async (req, res) => {
+ const c = await overlays.getConfig(db, req.params.id);
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
allow(req, CAP.configGet, c.creator_subject);
allow(req, CAP.configGet, c.creator_subject);
res.json({ config: overlays.presentConfig(c) });
res.json({ config: overlays.presentConfig(c) });
}));
}));
- r.post('/overlay-configs', edit, ...write((req, res) => {
+ r.post('/overlay-configs', edit, ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
allow(req, CAP.configUpdate, profile.creator_subject);
allow(req, CAP.configUpdate, profile.creator_subject);
- res.status(201).json({ config: overlays.createConfig(profile.creator_subject, b) });
+ res.status(201).json({ config: await overlays.createConfig(profile.creator_subject, b) });
}));
}));
- r.patch('/overlay-configs/:id', edit, ...write((req, res) => {
- const c = overlays.getConfig(req.params.id);
+ r.patch('/overlay-configs/:id', edit, ...write(async (req, res) => {
+ const c = await overlays.getConfig(db, req.params.id);
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
if (!c) fail(404, 'tips.overlay_config_not_found', 'no such overlay config');
allow(req, CAP.configUpdate, c.creator_subject);
allow(req, CAP.configUpdate, c.creator_subject);
- res.json({ config: overlays.updateConfig(c, req.body || {}) });
+ res.json({ config: await overlays.updateConfig(c, req.body || {}) });
}));
}));
// ── Moderation (server/domain/moderation.js) ─────────────
// ── Moderation (server/domain/moderation.js) ─────────────
const { moderation } = domain;
const { moderation } = domain;
/** The creator, one of their moderators, or a service with tips.interaction.moderate. */
/** The creator, one of their moderators, or a service with tips.interaction.moderate. */
- function moderatorOf(req, creator, { hideExistence = false } = {}) {
+ async function moderatorOf(req, creator, { hideExistence = false } = {}) {
const p = req.principal;
const p = req.principal;
if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in (a Network user token) or present a service token');
if (p.kind === 'anonymous') fail(401, 'token.missing', 'sign in (a Network user token) or present a service token');
- const a = moderation.actorFor(p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));
+ const a = await moderation.actorFor(db, p, creator, p.kind === 'service' && apiAuth.granted(p, CAP.moderate));
if (a) return a;
if (a) return a;
if (p.kind === 'service') fail(403, 'capability.denied', `${CAP.moderate} not granted`);
if (p.kind === 'service') fail(403, 'capability.denied', `${CAP.moderate} not granted`);
if (hideExistence) fail(404, 'tips.interaction_not_found', 'no such interaction');
if (hideExistence) fail(404, 'tips.interaction_not_found', 'no such interaction');
fail(403, 'tips.forbidden', 'only the creator and their moderators moderate this page');
fail(403, 'tips.forbidden', 'only the creator and their moderators moderate this page');
return null;
return null;
}
}
- r.get('/moderation', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
- moderatorOf(req, profile.creator_subject);
- const out = moderation.queue(profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });
+ r.get('/moderation', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
+ await moderatorOf(req, profile.creator_subject);
+ const out = await moderation.queue(db, profile.creator_subject, { state: req.query.state || 'all', cursor: req.query.cursor, limit: req.query.limit });
res.json({ creator: { type: 'user', id: profile.creator_subject }, interactions: out.rows, next_cursor: out.next_cursor });
res.json({ creator: { type: 'user', id: profile.creator_subject }, interactions: out.rows, next_cursor: out.next_cursor });
}));
}));
- r.get('/moderation/log', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
+ r.get('/moderation/log', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.moderate, profile.creator_subject);
allow(req, CAP.moderate, profile.creator_subject);
- res.json({ log: moderation.log(profile.creator_subject) });
+ res.json({ log: await moderation.log(db, profile.creator_subject) });
}));
}));
for (const action of ['hide', 'restore']) {
for (const action of ['hide', 'restore']) {
- r.post(`/interactions/:id/${action}`, moderate, ...write((req, res) => {
- const i = interactions.get(req.params.id);
+ r.post(`/interactions/:id/${action}`, moderate, ...write(async (req, res) => {
+ const i = await interactions.get(db, req.params.id);
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
if (!i) fail(404, 'tips.interaction_not_found', 'no such interaction');
- const actor = moderatorOf(req, i.creator_subject, { hideExistence: true });
- const out = moderation[action](i, { ...actor, reason: (req.body || {}).reason });
- res.json({ interaction: moderation.present(out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });
+ const actor = await moderatorOf(req, i.creator_subject, { hideExistence: true });
+ const out = await moderation[action](i, { ...actor, reason: (req.body || {}).reason });
+ res.json({ interaction: await moderation.present(db, out.interaction), changed: out.changed, cancelled_effects: out.cancelled_effects || [] });
}));
}));
}
}
- r.get('/moderators', wrap((req, res) => {
- const profile = creatorOf(req, req.query.creator);
+ r.get('/moderators', wrap(async (req, res) => {
+ const profile = await creatorOf(req, req.query.creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
allow(req, CAP.profileUpdate, profile.creator_subject);
- res.json({ moderators: moderation.listModerators(profile.creator_subject).map(moderation.presentModerator) });
+ res.json({ moderators: (await moderation.listModerators(db, profile.creator_subject)).map(moderation.presentModerator) });
}));
}));
- r.post('/moderators', staffing, ...write((req, res) => {
+ r.post('/moderators', staffing, ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
allow(req, CAP.profileUpdate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
- res.status(201).json({ moderator: moderation.addModerator(profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });
+ res.status(201).json({ moderator: await moderation.addModerator(db, profile.creator_subject, b.moderator, { name: b.name, addedBy: by }) });
}));
}));
- r.post('/moderators/:subject/remove', staffing, ...write((req, res) => {
- const profile = creatorOf(req, (req.body || {}).creator);
+ r.post('/moderators/:subject/remove', staffing, ...write(async (req, res) => {
+ const profile = await creatorOf(req, (req.body || {}).creator);
allow(req, CAP.profileUpdate, profile.creator_subject);
allow(req, CAP.profileUpdate, profile.creator_subject);
- res.json(moderation.removeModerator(profile.creator_subject, req.params.subject));
+ res.json(await moderation.removeModerator(profile.creator_subject, req.params.subject));
}));
}));
// ── Simulation ───────────────────────────────────────────
// ── Simulation ───────────────────────────────────────────
// A simulated tip plays on the creator's overlays and chat like a real one.
// A simulated tip plays on the creator's overlays and chat like a real one.
- r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write((req, res) => {
+ r.post('/simulate', limits('tips.simulation.run', { minute: 10, hour: 100 }), ...write(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
- const profile = creatorOf(req, b.creator);
+ const profile = await creatorOf(req, b.creator);
allow(req, CAP.simulate, profile.creator_subject);
allow(req, CAP.simulate, profile.creator_subject);
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
const by = req.principal.kind === 'service' ? req.principal.sub : req.principal.subject;
- const i = interactions.simulate(profile, b, { by });
- res.status(201).json({ interaction: interactions.present(i, { viewer: 'owner' }) });
+ const i = await interactions.simulate(profile, b, { by });
+ res.status(201).json({ interaction: await interactions.present(db, i, { viewer: 'owner' }) });
}));
}));
return r;
return r;
@@ -416,6 +413,7 @@ function wrap(fn) {
@@ -416,6 +413,7 @@ function wrap(fn) {
function sendError(req, res, e) {
function sendError(req, res, e) {
if (res.headersSent) return;
if (res.headersSent) return;
+ e = inputError(e) || e;
if (e instanceof TipsError) {
if (e instanceof TipsError) {
return http.sendProblem(res, e.status, e.code, { detail: e.detail || e.message, ctx: req.ov, extra: e.extra && e.status < 500 ? { details: e.extra } : undefined });
return http.sendProblem(res, e.status, e.code, { detail: e.detail || e.message, ctx: req.ov, extra: e.extra && e.status < 500 ? { details: e.extra } : undefined });
}
}
@@ -11,12 +11,16 @@
@@ -11,12 +11,16 @@
* /overlay/:token[/events|/state] overlays (scoped revocable tokens)
* /overlay/:token[/events|/state] overlays (scoped revocable tokens)
* /, /dashboard, /receipts, /:handle … server-rendered pages
* /, /dashboard, /receipts, /:handle … server-rendered pages
*
*
- * createApp({ config, db, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.
+ * createApp({ config, db, valkey, registry, keys, billing, adapters, now, fetchImpl, log }) — everything injectable.
+ * `db` is an openvibe-sdk/db handle with the schema migrated (server/index.js migrates at boot; tests
+ * pass a PGlite one); `valkey` an openvibe-sdk/valkey handle or null (everything shared stays in this
+ * process).
*/
*/
const path = require('path');
const path = require('path');
const express = require('express');
const express = require('express');
const cookieParser = require('cookie-parser');
const cookieParser = require('cookie-parser');
const { http } = require('openvibe-contracts');
const { http } = require('openvibe-contracts');
+const { isLoopbackDirect } = require('openvibe-shared/metrics');
const { loadConfig } = require('./config');
const { loadConfig } = require('./config');
const { openDb } = require('./db');
const { openDb } = require('./db');
const { createKeyProvider, createUserAuth } = require('./network');
const { createKeyProvider, createUserAuth } = require('./network');
@@ -32,6 +36,7 @@ const { createSessionRoutes } = require('./web/session');
@@ -32,6 +36,7 @@ const { createSessionRoutes } = require('./web/session');
const { createWebRoutes } = require('./web/routes');
const { createWebRoutes } = require('./web/routes');
const { createLayout, assetVersion } = require('./web/layout');
const { createLayout, assetVersion } = require('./web/layout');
const pages = require('./web/pages');
const pages = require('./web/pages');
+const { inputError } = require('./util');
const { createTipsReadiness, registerTipsGauges } = require('./observability');
const { createTipsReadiness, registerTipsGauges } = require('./observability');
const VERSION = require('../package.json').version;
const VERSION = require('../package.json').version;
@@ -39,7 +44,8 @@ const PUBLIC_DIR = path.join(__dirname, '..', 'public');
@@ -39,7 +44,8 @@ const PUBLIC_DIR = path.join(__dirname, '..', 'public');
function createApp(opts = {}) {
function createApp(opts = {}) {
const config = opts.config || loadConfig();
const config = opts.config || loadConfig();
- const db = opts.db || openDb(config.dbPath);
+ const db = opts.db || openDb(config);
+ const valkey = opts.valkey || null;
const fetchImpl = opts.fetchImpl || globalThis.fetch;
const fetchImpl = opts.fetchImpl || globalThis.fetch;
const log = opts.log || console;
const log = opts.log || console;
const now = opts.now || (() => Date.now());
const now = opts.now || (() => Date.now());
@@ -48,7 +54,7 @@ function createApp(opts = {}) {
@@ -48,7 +54,7 @@ function createApp(opts = {}) {
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const adapters = opts.adapters || createAdapters(config, { fetchImpl });
const adapters = opts.adapters || createAdapters(config, { fetchImpl });
const outbox = opts.outbox || createTipsOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
const outbox = opts.outbox || createTipsOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
- const domain = createDomain({ db, config, outbox, billing, adapters, now, log });
+ const domain = createDomain({ db, config, outbox, billing, adapters, valkey, now, log });
const apiAuth = createApiAuth({ config, keys, userAuth });
const apiAuth = createApiAuth({ config, keys, userAuth });
const release = require('openvibe-shared/release').createRelease({ service: 'tips', root: path.join(__dirname, '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tips', root: path.join(__dirname, '..') });
const layout = createLayout({ config, release });
const layout = createLayout({ config, release });
@@ -58,10 +64,13 @@ function createApp(opts = {}) {
@@ -58,10 +64,13 @@ function createApp(opts = {}) {
app.set('trust proxy', config.trustProxy);
app.set('trust proxy', config.trustProxy);
// HTTP golden signals by route template, process metrics, release_info; GET /metrics answers
// HTTP golden signals by route template, process metrics, release_info; GET /metrics answers
// direct loopback callers only (Track O). An overlay's SSE stream is a session, not a request.
// direct loopback callers only (Track O). An overlay's SSE stream is a session, not a request.
+ // The Tips gauges are read from the database just before a direct scrape renders (one query).
+ const registry = opts.registry || require('openvibe-shared/metrics').createRegistry();
+ const gauges = registerTipsGauges(registry, { db, now, log });
+ app.get('/metrics', (req, res, next) => (isLoopbackDirect(req) ? gauges.refresh().then(() => next(), () => next()) : next()));
const metrics = require('openvibe-shared/metrics').instrument(app, {
const metrics = require('openvibe-shared/metrics').instrument(app, {
- service: 'tips', release: release.release, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),
+ service: 'tips', release: release.release, registry, skip: (req) => /^\/overlay\/[^/]+\/events$/.test(req.path),
});
});
- registerTipsGauges(metrics.registry, { db, outbox, now });
app.use(http.middleware());
app.use(http.middleware());
app.use((req, res, next) => {
app.use((req, res, next) => {
res.setHeader('X-Content-Type-Options', 'nosniff');
res.setHeader('X-Content-Type-Options', 'nosniff');
@@ -77,12 +86,12 @@ function createApp(opts = {}) {
@@ -77,12 +86,12 @@ function createApp(opts = {}) {
});
});
app.use(cookieParser());
app.use(cookieParser());
- app.get('/api/health', (req, res) => res.json({
- ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events: outbox.status(),
- }));
- // Readiness (openvibe-shared/ready): 503 only when the database fails; Network key, Billing and
- // Events are optional and degrade it (see observability.js).
- const readiness = createTipsReadiness({ db, keys, config, outbox, release: release.release, fetchImpl });
+ app.get('/api/health', (req, res, next) => outbox.status().then((events) => res.json({
+ ok: true, service: 'tips', version: VERSION, chat_adapter: config.chat.adapter, events,
+ }), next));
+ // Readiness (openvibe-shared/ready): 503 only when the database fails; Valkey, the Network key,
+ // Billing and Events are optional and degrade it (see observability.js).
+ const readiness = createTipsReadiness({ db, valkey, keys, config, outbox, release: release.release, fetchImpl });
app.get('/api/ready', readiness.handler);
app.get('/api/ready', readiness.handler);
// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports into /metrics.
// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports into /metrics.
release.mount(app, { registry: metrics.registry });
release.mount(app, { registry: metrics.registry });
@@ -94,7 +103,7 @@ function createApp(opts = {}) {
@@ -94,7 +103,7 @@ function createApp(opts = {}) {
? http.sendProblem(res, 404, 'not_found', { ctx: req.ov }) : next()), consumer.router);
? http.sendProblem(res, 404, 'not_found', { ctx: req.ov }) : next()), consumer.router);
// Per-actor limits on /api/v1 (api/actor-limits.js), counted once apiAuth resolved the caller.
// Per-actor limits on /api/v1 (api/actor-limits.js), counted once apiAuth resolved the caller.
// opts.limitsNow: the limiter's clock (tests).
// opts.limitsNow: the limiter's clock (tests).
- const limits = createActorLimits({ config, now: opts.limitsNow || now, registry: metrics.registry, log });
+ const limits = createActorLimits({ config, valkey, now: opts.limitsNow || now, registry: metrics.registry, log });
app.use('/api/v1', express.json({ limit: '64kb' }), (req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); }, apiAuth.middleware, v1Router({ domain, apiAuth, limits }));
app.use('/api/v1', express.json({ limit: '64kb' }), (req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); }, apiAuth.middleware, v1Router({ domain, apiAuth, limits }));
app.use('/auth', createSessionRoutes(config, userAuth, { fetchImpl }));
app.use('/auth', createSessionRoutes(config, userAuth, { fetchImpl }));
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'tips', service: 'tips', host: 'openvibe.tips', name: 'OpenVibe.Tips', profile: 'ugc' })); }
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'tips', service: 'tips', host: 'openvibe.tips', name: 'OpenVibe.Tips', profile: 'ugc' })); }
@@ -119,13 +128,18 @@ function createApp(opts = {}) {
@@ -119,13 +128,18 @@ function createApp(opts = {}) {
app.use((err, req, res, next) => {
app.use((err, req, res, next) => {
if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { ctx: req.ov });
if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { ctx: req.ov });
if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });
if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });
+ const refused = inputError(err);
+ if (refused && !res.headersSent) {
+ if (req.path.startsWith('/api/')) return http.sendProblem(res, 422, refused.code, { detail: refused.detail, ctx: req.ov });
+ return res.status(400).type('html').send(layout.page({ title: 'Not accepted', robots: 'noindex', body: pages.errorPage({ status: 400, title: 'That could not be saved', message: refused.detail }) }));
+ }
log.error('[Tips] unhandled error:', err);
log.error('[Tips] unhandled error:', err);
if (res.headersSent) return undefined;
if (res.headersSent) return undefined;
if (req.path.startsWith('/api/')) return http.sendProblem(res, 500, 'tips.internal', { ctx: req.ov });
if (req.path.startsWith('/api/')) return http.sendProblem(res, 500, 'tips.internal', { ctx: req.ov });
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
});
});
- Object.assign(app.locals, { config, db, domain, keys, outbox, adapters, billing, consumer, metrics });
+ Object.assign(app.locals, { config, db, valkey, domain, keys, outbox, adapters, billing, consumer, metrics });
return app;
return app;
}
}
@@ -8,6 +8,7 @@
@@ -8,6 +8,7 @@
* never prices or rates. Prices and the value of a bit are Billing's.
* never prices or rates. Prices and the value of a bit are Billing's.
*/
*/
require('dotenv').config();
require('dotenv').config();
+const crypto = require('crypto');
const int = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) ? n : d; };
const int = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) ? n : d; };
const bool = (v, d = false) => (v == null || v === '' ? d : ['1', 'true', 'yes', 'on'].includes(String(v).toLowerCase()));
const bool = (v, d = false) => (v == null || v === '' ? d : ['1', 'true', 'yes', 'on'].includes(String(v).toLowerCase()));
@@ -28,7 +29,23 @@ function loadConfig(env = process.env) {
@@ -28,7 +29,23 @@ function loadConfig(env = process.env) {
host: env.HOST || '127.0.0.1',
host: env.HOST || '127.0.0.1',
baseUrl,
baseUrl,
trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1,
trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1,
- dbPath: env.TIPS_DB_PATH || './data/tips.db',
+
+ // PostgreSQL (ADR-035; OpenVibe.Host's data role writes all four into /etc/openvibe/tips.env):
+ // DATABASE_URL is the pooled runtime role through PgBouncer (transaction mode), DATABASE_DIRECT_URL
+ // the owner role on a direct connection, for migrations at boot and the one-time import.
+ db: {
+ url: env.DATABASE_URL || '',
+ directUrl: env.DATABASE_DIRECT_URL || '',
+ },
+ // Valkey: shared, never authoritative (per-actor limit counters, overlay fan-out between
+ // processes, overlay stream slots). Unset: all of it stays in this process (one process only).
+ valkey: {
+ url: env.VALKEY_URL || '',
+ prefix: env.VALKEY_PREFIX || 'ov:tips:',
+ },
+ // The SQLite file of the release before PostgreSQL: read (never written) only by
+ // scripts/migrate-to-postgres.js. The systemd unit sets /var/lib/openvibe-tips/tips.db.
+ sqlitePath: env.TIPS_DB_PATH || './data/tips.db',
// Identity: service tokens and user tokens are RS256 JWTs signed by OpenVibe.Network.
// Identity: service tokens and user tokens are RS256 JWTs signed by OpenVibe.Network.
network: {
network: {
@@ -48,8 +65,11 @@ function loadConfig(env = process.env) {
@@ -48,8 +65,11 @@ function loadConfig(env = process.env) {
scope: env.OV_OAUTH_SCOPE || 'profile',
scope: env.OV_OAUTH_SCOPE || 'profile',
},
},
cookies: { secure: env.COOKIE_SECURE != null ? bool(env.COOKIE_SECURE) : isProduction },
cookies: { secure: env.COOKIE_SECURE != null ? bool(env.COOKIE_SECURE) : isProduction },
- // Signs the anti-forgery tokens of the server-rendered forms.
- formSecret: env.TIPS_FORM_SECRET || '',
+ // Signs the anti-forgery tokens of the server-rendered forms. Every process must use the same
+ // one (a form rendered by one process is posted to another), so without TIPS_FORM_SECRET it is
+ // derived from the client secret; only with neither is it per process (development).
+ formSecret: env.TIPS_FORM_SECRET
+ || (env.OV_OAUTH_CLIENT_SECRET ? crypto.createHmac('sha256', env.OV_OAUTH_CLIENT_SECRET).update('openvibe.tips form secret').digest('hex') : ''),
billing: {
billing: {
url: trim(env.BILLING_URL || 'http://127.0.0.1:4600'),
url: trim(env.BILLING_URL || 'http://127.0.0.1:4600'),
@@ -1,346 +1,62 @@
@@ -1,346 +1,62 @@
'use strict';
'use strict';
/**
/**
- * Tips' own SQLite database (WAL), created on boot, idempotent.
+ * Tips' own PostgreSQL database (ADR-035), through openvibe-sdk/db: async, pooled, one dialect.
*
*
- * The ten product tables (roadmap §15.11) store interaction state and REFERENCES to Billing
- * transactions — never a mutable cash balance. Amounts are integer vibes-bits as Billing counts
- * them; a creator's totals are always derived from settled interactions.
+ * The schema is migrations/NNNN_*.sql (expand/migrate/contract, ADR-028), applied at boot by
+ * migrate() on the owner's direct connection (DATABASE_DIRECT_URL); the service then serves on the
+ * pooled runtime role (DATABASE_URL, PgBouncer in transaction mode: no session state, 10 connections
+ * per process at most).
+ *
+ * The product tables (roadmap §15.11) store interaction state and REFERENCES to Billing transactions,
+ * never a mutable cash balance. Amounts are integer vibes-bits as Billing counts them; a creator's
+ * totals are always derived from settled interactions.
*
*
* creator_tip_profiles one per creator subject: page switch, minimums, TTS/media settings
* creator_tip_profiles one per creator subject: page switch, minimums, TTS/media settings
* tip_interactions one logical interaction; payment_state (mirrored from Billing, keyed by
* tip_interactions one logical interaction; payment_state (mirrored from Billing, keyed by
* Billing txn id) is separate from delivery_state (Tips' own effects)
* Billing txn id) is separate from delivery_state (Tips' own effects)
* tip_goals creator goals
* tip_goals creator goals
* tip_goal_contributions settled interactions counted toward a goal (never tests, never pending)
* tip_goal_contributions settled interactions counted toward a goal (never tests, never pending)
- * paid_messages highlighted paid chat messages and TTS requests — created on settlement
- * paid_media_requests paid media requests — created on settlement
+ * paid_messages highlighted paid chat messages and TTS requests, created on settlement
+ * paid_media_requests paid media requests, created on settlement
* overlay_configs alert / goal overlay settings
* overlay_configs alert / goal overlay settings
* overlay_deliveries what overlays were sent (monotonic seq per creator, replayable)
* overlay_deliveries what overlays were sent (monotonic seq per creator, replayable)
* interaction_effects the retryable delivery work of an interaction (chat line, TTS, media, overlay)
* interaction_effects the retryable delivery work of an interaction (chat line, TTS, media, overlay)
* migration_maps legacy (Live) row → Tips entity, with held/excluded reasons
* migration_maps legacy (Live) row → Tips entity, with held/excluded reasons
*
*
- * Supporting: overlay_tokens (hashed, scoped, revocable), api_idempotency, import_runs,
- * event_outbox + idempotency_receipts (openvibe-sdk outbox and inbox), settings. Columns added after
- * v1 are in COLUMNS below.
+ * Supporting: overlay_tokens (hashed, scoped, revocable), tip_moderators / _invites / _moderation_log,
+ * api_idempotency, import_runs, tips_event_outbox + tips_event_inbox (openvibe-sdk outbox and inbox).
+ *
+ * openDb(config) the serving handle: DATABASE_URL; in development without it, an embedded
+ * PGlite database in data/pglite (one process, nothing to install)
+ * migrate(config, {serving}) apply migrations/ with the owner role (DATABASE_DIRECT_URL), then close
+ * that connection; the embedded database migrates on its serving handle
*/
*/
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
-const Database = require('better-sqlite3');
-
-const SCHEMA = `
-CREATE TABLE IF NOT EXISTS settings (
- id INTEGER PRIMARY KEY CHECK (id = 1),
- schema_version INTEGER NOT NULL,
- created_at TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS creator_tip_profiles (
- creator_subject TEXT PRIMARY KEY, -- usr_…
- handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)
- display_name TEXT NOT NULL,
- avatar_url TEXT,
- headline TEXT,
- page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing
- accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted
- min_amount INTEGER NOT NULL DEFAULT 1,
- paid_message_min INTEGER NOT NULL DEFAULT 100,
- tts_enabled INTEGER NOT NULL DEFAULT 0,
- tts_min_amount INTEGER NOT NULL DEFAULT 100,
- tts_max_chars INTEGER NOT NULL DEFAULT 200,
- tts_voice TEXT NOT NULL DEFAULT 'gary',
- media_requests_enabled INTEGER NOT NULL DEFAULT 0,
- media_request_min INTEGER NOT NULL DEFAULT 25,
- media_max_seconds INTEGER NOT NULL DEFAULT 600,
- revision INTEGER NOT NULL DEFAULT 1,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS tip_interactions (
- id TEXT PRIMARY KEY, -- tint_<ULID>
- creator_subject TEXT NOT NULL,
- supporter_subject TEXT, -- null: external/anonymous/unmapped
- supporter_name TEXT, -- display projection at the time
- kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
- amount INTEGER NOT NULL CHECK (amount > 0),
- currency TEXT NOT NULL DEFAULT 'vibes-bits',
- amount_cents INTEGER, -- money that arrived (provider/external)
- message TEXT,
- request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target
- funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
- settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
- payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
- delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
- test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted
- billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it
- billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent
- funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it
- reversal_txn_ids TEXT NOT NULL DEFAULT '[]',
- reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator
- checkout_url TEXT,
- checkout_ref TEXT,
- provider TEXT,
- provider_ref TEXT,
- legacy_source TEXT, -- 'live:transactions:<id>' etc.
- idempotency_key TEXT UNIQUE,
- failure TEXT,
- origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import
- transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried
- transfer_attempts INTEGER NOT NULL DEFAULT 0,
- next_transfer_at INTEGER NOT NULL DEFAULT 0,
- target TEXT, -- JSON EntityRef (e.g. a Live stream)
- created_at TEXT NOT NULL,
- settled_at TEXT,
- reversed_at TEXT,
- delivered_at TEXT,
- updated_at TEXT NOT NULL,
- UNIQUE (provider, provider_ref),
- UNIQUE (legacy_source)
-);
-CREATE INDEX IF NOT EXISTS idx_tint_creator ON tip_interactions (creator_subject, created_at);
-CREATE INDEX IF NOT EXISTS idx_tint_supporter ON tip_interactions (supporter_subject, created_at);
-CREATE INDEX IF NOT EXISTS idx_tint_pending ON tip_interactions (payment_state, funding);
-
-CREATE TABLE IF NOT EXISTS tip_goals (
- id TEXT PRIMARY KEY, -- tgoal_<ULID>
- creator_subject TEXT NOT NULL,
- title TEXT NOT NULL,
- description TEXT,
- target_amount INTEGER NOT NULL CHECK (target_amount > 0),
- currency TEXT NOT NULL DEFAULT 'vibes-bits',
- image_url TEXT,
- status TEXT NOT NULL CHECK (status IN ('active', 'closed')),
- sort_order INTEGER NOT NULL DEFAULT 0,
- reached_at TEXT,
- closed_at TEXT,
- opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)
- legacy_source TEXT UNIQUE,
- revision INTEGER NOT NULL DEFAULT 1,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_goal_creator ON tip_goals (creator_subject, status);
-
-CREATE TABLE IF NOT EXISTS tip_goal_contributions (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- goal_id TEXT NOT NULL REFERENCES tip_goals(id),
- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
- amount INTEGER NOT NULL CHECK (amount > 0),
- reversed_amount INTEGER NOT NULL DEFAULT 0,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL,
- UNIQUE (goal_id, interaction_id)
-);
-
-CREATE TABLE IF NOT EXISTS paid_messages (
- id TEXT PRIMARY KEY, -- tpm_<ULID>
- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
- creator_subject TEXT NOT NULL,
- kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),
- text TEXT NOT NULL,
- voice TEXT,
- highlight_seconds INTEGER NOT NULL DEFAULT 0,
- status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
- chat_ref TEXT, -- JSON reference the chat adapter returned
- test INTEGER NOT NULL DEFAULT 0,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS paid_media_requests (
- id TEXT PRIMARY KEY, -- tmr_<ULID>
- interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
- creator_subject TEXT NOT NULL,
- url TEXT NOT NULL,
- provider TEXT,
- status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
- queue_ref TEXT, -- JSON reference returned by the queue owner
- test INTEGER NOT NULL DEFAULT 0,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS overlay_configs (
- id TEXT PRIMARY KEY, -- tovc_<ULID>
- creator_subject TEXT NOT NULL,
- kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),
- name TEXT NOT NULL,
- settings TEXT NOT NULL DEFAULT '{}',
- goal_id TEXT,
- revision INTEGER NOT NULL DEFAULT 1,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_ovc_creator ON overlay_configs (creator_subject);
-
-CREATE TABLE IF NOT EXISTS overlay_tokens (
- id TEXT PRIMARY KEY, -- tovt_<ULID>
- creator_subject TEXT NOT NULL,
- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
- scopes TEXT NOT NULL, -- JSON ["alerts","goals"]
- config_id TEXT,
- label TEXT,
- created_by TEXT NOT NULL,
- created_at TEXT NOT NULL,
- last_used_at TEXT,
- revoked_at TEXT
-);
-CREATE INDEX IF NOT EXISTS idx_ovt_creator ON overlay_tokens (creator_subject);
-
-CREATE TABLE IF NOT EXISTS overlay_deliveries (
- seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)
- id TEXT NOT NULL UNIQUE, -- tovd_<ULID>
- creator_subject TEXT NOT NULL,
- kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),
- interaction_id TEXT,
- goal_id TEXT,
- payload TEXT NOT NULL,
- test INTEGER NOT NULL DEFAULT 0,
- status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
- delivered_at TEXT,
- failed_at TEXT,
- sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
- dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
- created_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_ovd_creator ON overlay_deliveries (creator_subject, seq);
-CREATE INDEX IF NOT EXISTS idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';
-
-CREATE TABLE IF NOT EXISTS interaction_effects (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
- effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
- adapter TEXT NOT NULL,
- state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
- attempts INTEGER NOT NULL DEFAULT 0,
- next_attempt_at INTEGER NOT NULL DEFAULT 0,
- last_error TEXT,
- result TEXT,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL,
- UNIQUE (interaction_id, effect)
-);
-CREATE INDEX IF NOT EXISTS idx_effects_due ON interaction_effects (state, next_attempt_at);
+const { createDb } = require('openvibe-sdk/db');
-CREATE TABLE IF NOT EXISTS migration_maps (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- source TEXT NOT NULL, -- 'live'
- source_table TEXT NOT NULL,
- source_id TEXT NOT NULL,
- target_type TEXT, -- 'interaction' | 'goal'
- target_id TEXT,
- status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
- reason TEXT,
- billing_txn_id TEXT,
- run_id TEXT NOT NULL,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL,
- UNIQUE (source, source_table, source_id)
-);
+const MIGRATIONS = path.join(__dirname, '..', 'migrations');
+const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');
-CREATE TABLE IF NOT EXISTS import_runs (
- id TEXT PRIMARY KEY,
- source TEXT NOT NULL,
- dry_run INTEGER NOT NULL,
- started_at TEXT NOT NULL,
- finished_at TEXT,
- report TEXT
-);
-
-CREATE TABLE IF NOT EXISTS tip_moderators (
- creator_subject TEXT NOT NULL,
- moderator_subject TEXT NOT NULL,
- name TEXT, -- display name when they joined (for the creator's list)
- added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'
- created_at TEXT NOT NULL,
- removed_at TEXT,
- PRIMARY KEY (creator_subject, moderator_subject)
-);
-CREATE INDEX IF NOT EXISTS idx_tmod_moderator ON tip_moderators (moderator_subject);
-
-CREATE TABLE IF NOT EXISTS tip_moderator_invites (
- id TEXT PRIMARY KEY, -- tmin_<ULID>
- creator_subject TEXT NOT NULL,
- token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
- created_by TEXT NOT NULL,
- created_at TEXT NOT NULL,
- expires_at TEXT NOT NULL,
- used_at TEXT,
- used_by TEXT,
- revoked_at TEXT
-);
-
-CREATE TABLE IF NOT EXISTS tip_moderation_log (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- creator_subject TEXT NOT NULL,
- interaction_id TEXT NOT NULL,
- action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
- by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
- actor TEXT, -- usr_… / svc:… ; null for the filter
- reason TEXT,
- created_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);
-
-CREATE TABLE IF NOT EXISTS api_idempotency (
- key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>
- request_hash TEXT NOT NULL,
- method TEXT NOT NULL,
- path TEXT NOT NULL,
- status INTEGER NOT NULL,
- response TEXT NOT NULL,
- created_at TEXT NOT NULL
-);
-`;
-
-/**
- * Columns added after schema v1, for new and existing databases alike (ALTER TABLE … ADD COLUMN when
- * missing; idempotent, never rewrites a row).
- *
- * v2 privacy: what the supporter lets the public see of an interaction (server/domain/privacy.js),
- * their erasure, and what the creator's public goal and supporters pages show
- * v3 moderation (server/domain/moderation.js): an interaction's moderation state, the creator's
- * word filter, overlay alerts retracted by a moderator
- */
-const COLUMNS = [
- ['tip_interactions', 'anonymous', 'INTEGER NOT NULL DEFAULT 0'], // name shown as "Anonymous" to everyone but the supporter
- ['tip_interactions', 'hide_amount', 'INTEGER NOT NULL DEFAULT 0'], // amount left out of overlays, chat lines, public pages
- ['tip_interactions', 'private_message', 'INTEGER NOT NULL DEFAULT 0'], // a tip's message is for the creator only
- ['tip_interactions', 'erased_at', 'TEXT'], // the supporter erased their data from it
- ['creator_tip_profiles', 'page_settings', "TEXT NOT NULL DEFAULT '{}'"], // JSON: what the public goal / supporters pages show
- ['tip_interactions', 'moderation', "TEXT NOT NULL DEFAULT 'visible'"], // visible | held (the filter, awaiting review) | hidden (a moderator)
- ['tip_interactions', 'moderated_at', 'TEXT'],
- ['tip_interactions', 'moderated_by', 'TEXT'], // 'filter', usr_… or svc:…
- ['tip_interactions', 'filtered', 'INTEGER NOT NULL DEFAULT 0'], // the word filter matched it at settlement
- ['creator_tip_profiles', 'filter_words', "TEXT NOT NULL DEFAULT '[]'"], // JSON: the creator's blocklist (filter.js)
- ['creator_tip_profiles', 'filter_action', "TEXT NOT NULL DEFAULT 'mask'"],// mask | hold
- ['creator_tip_profiles', 'filter_links', 'INTEGER NOT NULL DEFAULT 1'], // links in paid messages shown as [link]
- ['overlay_deliveries', 'hidden', 'INTEGER NOT NULL DEFAULT 0'], // retracted: never sent or replayed again
-];
-
-const SCHEMA_VERSION = 3;
-
-function addColumns(db) {
- const have = new Map();
- for (const [table, name, def] of COLUMNS) {
- if (!have.has(table)) have.set(table, new Set(db.prepare(`PRAGMA table_info(${table})`).all().map((c) => c.name)));
- if (!have.get(table).has(name)) { db.exec(`ALTER TABLE ${table} ADD COLUMN ${name} ${def}`); have.get(table).add(name); }
+function openDb(config, { registry, log = console } = {}) {
+ if (!config.db.url && !config.isProduction) {
+ log.warn(`[Tips] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);
+ fs.mkdirSync(DEV_PGLITE, { recursive: true });
+ return createDb({ pglite: DEV_PGLITE, service: 'tips', registry, log });
}
}
+ return createDb({ url: config.db.url, service: 'tips', registry, log });
}
}
-function openDb(file) {
- if (file !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(file)), { recursive: true });
- const db = new Database(file);
- db.pragma('journal_mode = WAL');
- db.pragma('foreign_keys = ON');
- db.pragma('busy_timeout = 5000');
- db.exec(SCHEMA);
- db.transaction(() => addColumns(db))();
- db.prepare('INSERT OR IGNORE INTO settings (id, schema_version, created_at) VALUES (1, ?, ?)').run(SCHEMA_VERSION, new Date().toISOString());
- db.prepare('UPDATE settings SET schema_version = ? WHERE id = 1 AND schema_version < ?').run(SCHEMA_VERSION, SCHEMA_VERSION);
- return db;
+/**
+ * Apply pending migrations. Several processes starting together are safe: the SDK serialises runs
+ * with an advisory lock held on the owner's one direct session.
+ */
+async function migrate(config, { serving = null, log = console } = {}) {
+ if (serving && serving.store === 'pglite') return serving.migrate({ dir: MIGRATIONS, log });
+ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');
+ const owner = createDb({ url: config.db.directUrl, service: 'tips-migrate', max: 1, log });
+ try { return await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }
}
}
-module.exports = { openDb, SCHEMA_VERSION, COLUMNS };
+module.exports = { openDb, migrate, MIGRATIONS };
@@ -14,9 +14,21 @@
@@ -14,9 +14,21 @@
* is not sent, and the creator's word filter has starred blocked words out of the line and dropped
* is not sent, and the creator's word filter has starred blocked words out of the line and dropped
* them from the TTS text. `privacy` tells the receiving product what was withheld. An effect whose
* them from the TTS text. `privacy` tells the receiving product what was withheld. An effect whose
* interaction a moderator hid (or whose effect was cancelled meanwhile) is never delivered.
* interaction a moderator hid (or whose effect was cancelled meanwhile) is never delivered.
+ *
+ * Writes to an effect and what it delivers (paid message, media request, the interaction's delivery
+ * state) run SERIALIZABLE with the interaction row locked first, as the money paths do.
+ *
+ * Every process runs the worker; each batch of due effects is claimed with a lease (next_attempt_at
+ * moved past the batch's worst case, FOR UPDATE SKIP LOCKED), so no two processes deliver one effect
+ * at once, and a process that dies mid-batch leaves its effects to the others when the lease ends.
*/
*/
-const { iso, json } = require('../util');
-const { privacyOf, isAnonymous } = require('./privacy');
+const { sql } = require('openvibe-sdk/db');
+const { iso } = require('../util');
+const { privacyOf, isAnonymous, publicView } = require('./privacy');
+
+// A delivery takes up to 10 s (twice with a token refresh): 20 of them finish well inside the lease.
+const BATCH = 20;
+const LEASE_MS = 5 * 60_000;
function createEffects(ctx) {
function createEffects(ctx) {
const { db, config } = ctx;
const { db, config } = ctx;
@@ -24,8 +36,8 @@ function createEffects(ctx) {
@@ -24,8 +36,8 @@ function createEffects(ctx) {
let kickTimer = null;
let kickTimer = null;
function jobFor(effect, i, profile) {
function jobFor(effect, i, profile) {
- const req = json(i.request, {});
- const pv = ctx.view(i);
+ const req = i.request || {};
+ const pv = publicView(i, { filter: profile ? profile.filter : null });
const name = pv.supporter_name;
const name = pv.supporter_name;
const amount = pv.amount != null ? `${pv.amount.toLocaleString('en-US')} Vibes` : null;
const amount = pv.amount != null ? `${pv.amount.toLocaleString('en-US')} Vibes` : null;
const msg = pv.message ? `: ${pv.message}` : '';
const msg = pv.message ? `: ${pv.message}` : '';
@@ -39,7 +51,7 @@ function createEffects(ctx) {
@@ -39,7 +51,7 @@ function createEffects(ctx) {
privacy: privacyOf(i),
privacy: privacyOf(i),
// A tip on the creator's own PowerChat reads as Live's webhook wrote it: "… (PowerChat)".
// A tip on the creator's own PowerChat reads as Live's webhook wrote it: "… (PowerChat)".
text: `${name} ${amount ? `tipped ${amount}` : 'sent a tip'}${msg}${i.settlement === 'external' && i.provider === 'powerchat' ? ' (PowerChat)' : ''}`,
text: `${name} ${amount ? `tipped ${amount}` : 'sent a tip'}${msg}${i.settlement === 'external' && i.provider === 'powerchat' ? ' (PowerChat)' : ''}`,
- target: json(i.target, null),
+ target: i.target || null,
};
};
if (effect.effect === 'paid_message') job.highlight_seconds = req.highlight_seconds || 0;
if (effect.effect === 'paid_message') job.highlight_seconds = req.highlight_seconds || 0;
if (effect.effect === 'tts') job.tts = pv.tts;
if (effect.effect === 'tts') job.tts = pv.tts;
@@ -48,78 +60,92 @@ function createEffects(ctx) {
@@ -48,78 +60,92 @@ function createEffects(ctx) {
}
}
function markDone(effect, result) {
function markDone(effect, result) {
- ctx.tx(() => {
+ return ctx.tx(async (t) => {
+ await ctx.interactions.lock(t, effect.interaction_id);
const at = iso(ctx.now());
const at = iso(ctx.now());
- const r = db.prepare("UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ?, updated_at = ? WHERE id = ? AND state = 'queued'")
- .run(JSON.stringify(result || null), at, effect.id);
- if (!r.changes) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side
- const ref = JSON.stringify((result && result.ref) || null);
+ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'delivered', attempts = attempts + 1, last_error = NULL, result = ${sql.json(result || null)},
+ updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);
+ if (!n) return; // cancelled meanwhile (a reversal): the delivery stays recorded on the adapter's side
+ const ref = sql.json((result && result.ref) || null);
if (effect.effect === 'paid_message' || effect.effect === 'tts') {
if (effect.effect === 'paid_message' || effect.effect === 'tts') {
- db.prepare("UPDATE paid_messages SET status = 'delivered', chat_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);
+ await t.exec(sql`UPDATE paid_messages SET status = 'delivered', chat_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
}
}
if (effect.effect === 'media_request') {
if (effect.effect === 'media_request') {
- db.prepare("UPDATE paid_media_requests SET status = 'accepted', queue_ref = ?, updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(ref, at, effect.interaction_id);
+ await t.exec(sql`UPDATE paid_media_requests SET status = 'accepted', queue_ref = ${ref}, updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
}
}
- ctx.interactions.recomputeDelivery(effect.interaction_id);
- });
+ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);
+ }, ctx.MONEY);
}
}
- function markFailed(effect, err) {
- ctx.tx(() => {
+ async function markFailed(effect, err) {
+ await ctx.tx(async (t) => {
+ await ctx.interactions.lock(t, effect.interaction_id);
const at = iso(ctx.now());
const at = iso(ctx.now());
const attempts = effect.attempts + 1;
const attempts = effect.attempts + 1;
const final = err.permanent || attempts >= config.chat.maxAttempts;
const final = err.permanent || attempts >= config.chat.maxAttempts;
const backoff = config.chat.backoffMs[Math.min(attempts - 1, config.chat.backoffMs.length - 1)] || 60000;
const backoff = config.chat.backoffMs[Math.min(attempts - 1, config.chat.backoffMs.length - 1)] || 60000;
- db.prepare('UPDATE interaction_effects SET state = ?, attempts = ?, next_attempt_at = ?, last_error = ?, updated_at = ? WHERE id = ? AND state = \'queued\'')
- .run(final ? 'failed' : 'queued', attempts, ctx.now() + backoff, String(err.message || err).slice(0, 500), at, effect.id);
+ await t.exec(sql`UPDATE interaction_effects SET state = ${final ? 'failed' : 'queued'}, attempts = ${attempts}, next_attempt_at = ${ctx.now() + backoff},
+ last_error = ${String(err.message || err).slice(0, 500)}, updated_at = ${at} WHERE id = ${effect.id} AND state = 'queued'`);
if (final) {
if (final) {
- if (effect.effect === 'paid_message' || effect.effect === 'tts') db.prepare("UPDATE paid_messages SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);
- if (effect.effect === 'media_request') db.prepare("UPDATE paid_media_requests SET status = 'failed', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, effect.interaction_id);
- ctx.interactions.recomputeDelivery(effect.interaction_id);
+ if (effect.effect === 'paid_message' || effect.effect === 'tts') await t.exec(sql`UPDATE paid_messages SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
+ if (effect.effect === 'media_request') await t.exec(sql`UPDATE paid_media_requests SET status = 'failed', updated_at = ${at} WHERE interaction_id = ${effect.interaction_id} AND status = 'queued'`);
+ await ctx.interactions.recomputeDelivery(t, effect.interaction_id);
}
}
- });
+ }, ctx.MONEY);
ctx.outboxKick();
ctx.outboxKick();
}
}
async function runOne(effect) {
async function runOne(effect) {
- // Re-read: the batch was selected before earlier deliveries awaited; a hide or a reversal may
+ // Re-read: the batch was claimed before earlier deliveries awaited; a hide or a reversal may
// have cancelled this one since.
// have cancelled this one since.
- const current = db.prepare('SELECT state FROM interaction_effects WHERE id = ?').get(effect.id);
+ const current = await db.maybe(sql`SELECT state FROM interaction_effects WHERE id = ${effect.id}`);
if (!current || current.state !== 'queued') return 'cancelled';
if (!current || current.state !== 'queued') return 'cancelled';
- const i = ctx.interactions.get(effect.interaction_id);
+ const i = await ctx.interactions.get(db, effect.interaction_id);
if (!i || i.payment_state !== 'settled') {
if (!i || i.payment_state !== 'settled') {
// Reversed or failed before its turn: nothing to deliver.
// Reversed or failed before its turn: nothing to deliver.
- ctx.tx(() => ctx.interactions.cancelQueued(i, 'payment_not_settled'));
+ await ctx.tx(async (t) => { const locked = await ctx.interactions.lock(t, effect.interaction_id); if (locked) await ctx.interactions.cancelQueued(t, locked, 'payment_not_settled'); }, ctx.MONEY);
return 'cancelled';
return 'cancelled';
}
}
if (i.moderation !== 'visible') {
if (i.moderation !== 'visible') {
// Hidden or held: moderation.js cancels or releases these; never deliver one meanwhile.
// Hidden or held: moderation.js cancels or releases these; never deliver one meanwhile.
- db.prepare("UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE id = ? AND state = 'queued'").run(iso(ctx.now()), effect.id);
+ await db.exec(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${iso(ctx.now())}
+ WHERE id = ${effect.id} AND state = 'queued'`);
return 'cancelled';
return 'cancelled';
}
}
const adapter = ctx.adapters[effect.adapter];
const adapter = ctx.adapters[effect.adapter];
- if (!adapter) { markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }
+ if (!adapter) { await markFailed(effect, Object.assign(new Error(`no adapter ${effect.adapter} configured`), { permanent: true })); return 'failed'; }
+ let result;
try {
try {
- const result = await adapter.deliver(jobFor(effect, i, ctx.profiles.bySubject(i.creator_subject)));
- markDone(effect, result);
- return 'delivered';
+ result = await adapter.deliver(jobFor(effect, i, await ctx.profiles.bySubject(db, i.creator_subject)));
} catch (e) {
} catch (e) {
- markFailed(effect, e);
+ await markFailed(effect, e);
return 'retry';
return 'retry';
}
}
+ await markDone(effect, result);
+ return 'delivered';
+ }
+
+ /** Claim a batch of due effects (a lease each), oldest first. */
+ async function claim() {
+ const now = ctx.now();
+ const rows = await db.many(sql`UPDATE interaction_effects SET next_attempt_at = ${now + LEASE_MS}
+ WHERE id IN (SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now}
+ ORDER BY id LIMIT ${BATCH} FOR UPDATE SKIP LOCKED)
+ RETURNING *`);
+ return rows.sort((a, b) => a.id - b.id);
}
}
- /** Deliver every due effect once. */
+ /** Deliver every due effect once (this process's share of them). */
async function drain() {
async function drain() {
if (running) return { busy: true };
if (running) return { busy: true };
running = true;
running = true;
const out = { delivered: 0, retry: 0, failed: 0, cancelled: 0 };
const out = { delivered: 0, retry: 0, failed: 0, cancelled: 0 };
try {
try {
for (;;) {
for (;;) {
- const due = db.prepare("SELECT * FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ? ORDER BY id LIMIT 50").all(ctx.now());
+ const due = await claim();
if (!due.length) break;
if (!due.length) break;
for (const e of due) { const r = await runOne(e); out[r] = (out[r] || 0) + 1; }
for (const e of due) { const r = await runOne(e); out[r] = (out[r] || 0) + 1; }
- if (due.length < 50) break;
+ if (due.length < BATCH) break;
}
}
} finally { running = false; }
} finally { running = false; }
return out;
return out;
@@ -25,9 +25,9 @@ const INVISIBLE = /[----]/g;
@@ -25,9 +25,9 @@ const INVISIBLE = /[----]/g;
const COMBINING_RUN = /(\p{M}{2})\p{M}+/gu;
const COMBINING_RUN = /(\p{M}{2})\p{M}+/gu;
const URL_RE = /\b(?:https?:\/\/|www\.)[^\s<>"']+/gi;
const URL_RE = /\b(?:https?:\/\/|www\.)[^\s<>"']+/gi;
-/** Invisible characters and combining-mark runs out, NFKC. */
+/** Invisible and NUL characters and combining-mark runs out, unpaired surrogates replaced, NFKC. */
function clean(s) {
function clean(s) {
- return String(s == null ? '' : s).replace(INVISIBLE, '').normalize('NFKC').replace(COMBINING_RUN, '$1');
+ return String(s == null ? '' : s).toWellFormed().replace(INVISIBLE, '').replace(/\u0000/g, '').normalize('NFKC').replace(COMBINING_RUN, '$1');
}
}
/** The creator's input (a list, or text with one entry per line or comma) → a clean, bounded list. */
/** The creator's input (a list, or text with one entry per line or comma) → a clean, bounded list. */
@@ -16,75 +16,97 @@
@@ -16,75 +16,97 @@
* when goal_supporters is on, each supporter's own privacy applying (an anonymous one reads
* when goal_supporters is on, each supporter's own privacy applying (an anonymous one reads
* "Anonymous", a hidden amount is left out).
* "Anonymous", a hidden amount is left out).
*/
*/
+const { sql } = require('openvibe-sdk/db');
const { fail, iso, prefixedId, text, positiveInt } = require('../util');
const { fail, iso, prefixedId, text, positiveInt } = require('../util');
const { safeUrl } = require('./profiles');
const { safeUrl } = require('./profiles');
-const { publicName, shownName } = require('./privacy');
+const { publicName, shownName, publicView } = require('./privacy');
function createGoals(ctx) {
function createGoals(ctx) {
const { db } = ctx;
const { db } = ctx;
- const get = (id) => db.prepare('SELECT * FROM tip_goals WHERE id = ?').get(String(id || '')) || null;
- /** Settled contributions, plus the opening amount a goal imported from Live carried over. */
- const totalOf = (goalId) => {
- const r = db.prepare('SELECT COALESCE(SUM(amount - reversed_amount), 0) AS n, COUNT(*) AS c FROM tip_goal_contributions WHERE goal_id = ?').get(goalId);
- const g = db.prepare('SELECT opening_amount FROM tip_goals WHERE id = ?').get(goalId);
- return { n: r.n + ((g && g.opening_amount) || 0), c: r.c, contributions: r.n, opening: (g && g.opening_amount) || 0 };
- };
-
- function list(creator, { status } = {}) {
- const rows = status
- ? db.prepare('SELECT * FROM tip_goals WHERE creator_subject = ? AND status = ? ORDER BY sort_order, created_at').all(creator, status)
- : db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? ORDER BY status = 'active' DESC, sort_order, created_at DESC").all(creator);
- return rows;
+ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_goals WHERE id = ${String(id || '')}`);
+
+ /** Settled contributions per goal, in one query: Map(goal id → { n: sum, c: count }). */
+ async function totalsOf(q, goalIds) {
+ const out = new Map(goalIds.map((id) => [id, { n: 0, c: 0 }]));
+ if (!goalIds.length) return out;
+ const rows = await q.many(sql`SELECT goal_id, COALESCE(SUM(amount - reversed_amount), 0)::bigint AS n, count(*) AS c
+ FROM tip_goal_contributions WHERE goal_id = ANY(${goalIds}) GROUP BY goal_id`);
+ for (const r of rows) out.set(r.goal_id, { n: r.n, c: r.c });
+ return out;
+ }
+
+ function list(q, creator, { status } = {}) {
+ return status
+ ? q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = ${status} ORDER BY sort_order, created_at`)
+ : q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} ORDER BY status = 'active' DESC, sort_order, created_at DESC`);
+ }
+
+ /** The full view of each goal (the creator's, granted services', overlays'): one query for all totals. */
+ async function presentMany(q, goals) {
+ const totals = await totalsOf(q, goals.map((g) => g.id));
+ return goals.map((g) => {
+ const t = totals.get(g.id);
+ const n = t.n + (g.opening_amount || 0);
+ return {
+ id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,
+ target_amount: g.target_amount, current_amount: n, currency: g.currency, supporters_count: t.c,
+ carried_over_amount: g.opening_amount || 0,
+ percent: Math.min(100, Math.floor((n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,
+ image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,
+ created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,
+ };
+ });
}
}
- function present(g, { contributions = false } = {}) {
+ async function present(q, g, { contributions = false } = {}) {
if (!g) return null;
if (!g) return null;
- const t = totalOf(g.id);
- const out = {
- id: g.id, creator: { type: 'user', id: g.creator_subject }, title: g.title, description: g.description || null,
- target_amount: g.target_amount, current_amount: t.n, currency: g.currency, supporters_count: t.c,
- carried_over_amount: t.opening,
- percent: Math.min(100, Math.floor((t.n * 100) / g.target_amount)), reached: !!g.reached_at, reached_at: g.reached_at || null,
- image_url: g.image_url || null, status: g.status, sort_order: g.sort_order, revision: g.revision,
- created_at: g.created_at, updated_at: g.updated_at, closed_at: g.closed_at || null,
- };
+ const [out] = await presentMany(q, [g]);
if (contributions) {
if (contributions) {
// The creator's view: amounts always (their money), names as the supporter allowed.
// The creator's view: amounts always (their money), names as the supporter allowed.
- out.contributions = db.prepare(`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at
- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? ORDER BY c.id DESC LIMIT 200`).all(g.id)
+ out.contributions = (await q.many(sql`SELECT c.interaction_id, c.amount, c.reversed_amount, c.created_at, i.supporter_name, i.anonymous, i.erased_at
+ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ${g.id} ORDER BY c.id DESC LIMIT 200`))
.map(({ anonymous, erased_at: erasedAt, ...c }) => ({ ...c, supporter_name: publicName({ ...c, anonymous, erased_at: erasedAt }) }));
.map(({ anonymous, erased_at: erasedAt, ...c }) => ({ ...c, supporter_name: publicName({ ...c, anonymous, erased_at: erasedAt }) }));
}
}
return out;
return out;
}
}
- /** The public shape of a goal under the creator's page settings (`page`, profiles.js). */
- function publicGoal(g, page) {
- const out = present(g);
- if (!out) return null;
- if (!page.goal_amounts) Object.assign(out, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });
- if (page.goal_supporters) {
+ /**
+ * The public shape of goals under the creator's page settings (`profile.page`, profiles.js), with
+ * the creator's word filter (`profile.filter`) on supporter names: totals in one query, and the
+ * latest supporters of every goal in one more when the page shows them.
+ */
+ async function publicGoals(q, goals, profile) {
+ const page = profile.page;
+ const views = await presentMany(q, goals);
+ if (!page.goal_amounts) for (const v of views) Object.assign(v, { target_amount: null, current_amount: null, carried_over_amount: null, amounts_hidden: true });
+ if (page.goal_supporters && goals.length) {
// Moderated (held or hidden) interactions are left out; names go through the creator's filter.
// Moderated (held or hidden) interactions are left out; names go through the creator's filter.
- const settings = ctx.profiles.filterOf(g.creator_subject);
- out.supporters = db.prepare(`SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at
- FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id WHERE c.goal_id = ? AND c.amount > c.reversed_amount
- AND i.moderation = 'visible' ORDER BY c.id DESC LIMIT 10`).all(g.id)
- .map((c) => ({ name: shownName(c, settings), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));
+ const rows = await q.many(sql`SELECT gid AS goal_id, s.amount, s.created_at, s.supporter_name, s.anonymous, s.hide_amount, s.erased_at
+ FROM unnest(${goals.map((g) => g.id)}::text[]) AS gid
+ CROSS JOIN LATERAL (
+ SELECT c.amount - c.reversed_amount AS amount, c.created_at, i.supporter_name, i.anonymous, i.hide_amount, i.erased_at
+ FROM tip_goal_contributions c JOIN tip_interactions i ON i.id = c.interaction_id
+ WHERE c.goal_id = gid AND c.amount > c.reversed_amount AND i.moderation = 'visible'
+ ORDER BY c.id DESC LIMIT 10) s`);
+ for (const v of views) {
+ v.supporters = rows.filter((r) => r.goal_id === v.id)
+ .map((c) => ({ name: shownName(c, profile.filter), amount: page.goal_amounts && !c.hide_amount ? c.amount : null, at: c.created_at }));
+ }
}
}
- return out;
+ return views;
}
}
+ const publicGoal = async (q, g, profile) => (g ? (await publicGoals(q, [g], profile))[0] : null);
/** Inside a transaction: bump revision, emit the event, queue the overlay delivery. */
/** Inside a transaction: bump revision, emit the event, queue the overlay delivery. */
- function changed(goalId, reason, { interactionId = null, by = null } = {}) {
- const at = iso(ctx.now());
- db.prepare('UPDATE tip_goals SET revision = revision + 1, updated_at = ? WHERE id = ?').run(at, goalId);
- const g = get(goalId);
- const view = present(g);
- ctx.outbox.emit('tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {
+ async function changed(t, goalId, reason, { interactionId = null, by = null } = {}) {
+ const g = await t.one(sql`UPDATE tip_goals SET revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${goalId} RETURNING *`);
+ const [view] = await presentMany(t, [g]);
+ await ctx.outbox.emit(t, 'tips.goal.updated', { type: 'goal', id: g.id, revision: g.revision }, {
goal_id: g.id, creator: view.creator, title: g.title, target_amount: g.target_amount, current_amount: view.current_amount,
goal_id: g.id, creator: view.creator, title: g.title, target_amount: g.target_amount, current_amount: view.current_amount,
currency: g.currency, status: g.status, reached: view.reached, reason, interaction_id: interactionId,
currency: g.currency, status: g.status, reached: view.reached, reason, interaction_id: interactionId,
});
});
- ctx.overlays.addGoalDelivery(g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });
+ await ctx.overlays.addGoalDelivery(t, g.creator_subject, view, { reason, interactionId, by, dedupe: `goal:${g.id}:r${g.revision}` });
return view;
return view;
}
}
@@ -109,88 +131,103 @@ function createGoals(ctx) {
@@ -109,88 +131,103 @@ function createGoals(ctx) {
return out;
return out;
}
}
- function create(creator, input = {}) {
+ async function create(creator, input = {}) {
const f = fields(input, false);
const f = fields(input, false);
- const count = db.prepare("SELECT COUNT(*) AS n FROM tip_goals WHERE creator_subject = ? AND status = 'active'").get(creator).n;
- if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');
- return ctx.tx(() => {
+ return ctx.tx(async (t) => {
+ await ctx.lockCreator(t, creator);
+ const count = await t.value(sql`SELECT count(*) FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active'`);
+ if (count >= 20) fail(409, 'tips.too_many_goals', 'close a goal before adding another (20 active at most)');
const at = iso(ctx.now());
const at = iso(ctx.now());
const id = prefixedId('tgoal', ctx.now());
const id = prefixedId('tgoal', ctx.now());
- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)
- VALUES (?, ?, ?, ?, ?, ?, 'active', ?, ?, ?)`).run(id, creator, f.title, f.description || null, f.target_amount, f.image_url || null, f.sort_order || 0, at, at);
- return changed(id, 'created');
+ await t.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, description, target_amount, image_url, status, sort_order, created_at, updated_at)
+ VALUES (${id}, ${creator}, ${f.title}, ${f.description || null}, ${f.target_amount}, ${f.image_url || null}, 'active', ${f.sort_order || 0}, ${at}, ${at})`);
+ return changed(t, id, 'created');
});
});
}
}
- function update(goal, input = {}) {
+ /** An edit by the creator. The goal is locked, and its revision checked against the locked row. */
+ async function update(goal, input = {}) {
if (goal.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');
if (goal.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');
if (input.revision != null && Number(input.revision) !== goal.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${goal.revision}`);
if (input.revision != null && Number(input.revision) !== goal.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${goal.revision}`);
const f = fields(input, true);
const f = fields(input, true);
- const keys = Object.keys(f);
- if (!keys.length) return present(goal);
- return ctx.tx(() => {
- db.prepare(`UPDATE tip_goals SET ${keys.map((k) => `${k} = @${k}`).join(', ')} WHERE id = @id`).run({ ...f, id: goal.id });
- markReached(goal.id);
- return changed(goal.id, 'updated');
+ if (!Object.keys(f).length) return present(db, goal);
+ return ctx.tx(async (t) => {
+ const cur = await t.one(sql`SELECT * FROM tip_goals WHERE id = ${goal.id} FOR UPDATE`);
+ if (cur.status !== 'active') fail(409, 'tips.goal_closed', 'a closed goal cannot be edited');
+ if (input.revision != null && Number(input.revision) !== cur.revision) fail(409, 'tips.revision_conflict', `the goal is at revision ${cur.revision}`);
+ await t.exec(sql`UPDATE tip_goals SET ${sql.set(f)} WHERE id = ${goal.id}`);
+ await markReached(t, goal.id);
+ return changed(t, goal.id, 'updated');
});
});
}
}
- function close(goal) {
- if (goal.status === 'closed') return present(goal);
- return ctx.tx(() => {
- db.prepare("UPDATE tip_goals SET status = 'closed', closed_at = ? WHERE id = ?").run(iso(ctx.now()), goal.id);
- return changed(goal.id, 'closed');
+ async function close(goal) {
+ if (goal.status === 'closed') return present(db, goal);
+ return ctx.tx(async (t) => {
+ const n = await t.exec(sql`UPDATE tip_goals SET status = 'closed', closed_at = ${iso(ctx.now())} WHERE id = ${goal.id} AND status = 'active'`);
+ if (!n) return present(t, await get(t, goal.id));
+ return changed(t, goal.id, 'closed');
});
});
}
}
- function markReached(goalId) {
- const g = get(goalId);
- const reached = totalOf(goalId).n >= g.target_amount;
- if (reached && !g.reached_at) { db.prepare('UPDATE tip_goals SET reached_at = ? WHERE id = ?').run(iso(ctx.now()), goalId); return true; }
- if (!reached && g.reached_at) db.prepare('UPDATE tip_goals SET reached_at = NULL WHERE id = ?').run(goalId);
+ /** Inside a transaction: set or clear reached_at from the goal's total. True when reached just now. */
+ async function markReached(t, goalId) {
+ const g = await t.one(sql`SELECT g.target_amount, g.reached_at,
+ g.opening_amount + COALESCE((SELECT SUM(c.amount - c.reversed_amount) FROM tip_goal_contributions c WHERE c.goal_id = g.id), 0)::bigint AS total
+ FROM tip_goals g WHERE g.id = ${goalId}`);
+ const reached = g.total >= g.target_amount;
+ if (reached && !g.reached_at) { await t.exec(sql`UPDATE tip_goals SET reached_at = ${iso(ctx.now())} WHERE id = ${goalId}`); return true; }
+ if (!reached && g.reached_at) await t.exec(sql`UPDATE tip_goals SET reached_at = NULL WHERE id = ${goalId}`);
return false;
return false;
}
}
/** The goal an interaction counts toward, or null. */
/** The goal an interaction counts toward, or null. */
- function pick(creator, requestedGoalId) {
+ async function pick(q, creator, requestedGoalId) {
if (requestedGoalId) {
if (requestedGoalId) {
- const g = get(requestedGoalId);
+ const g = await get(q, requestedGoalId);
if (g && g.creator_subject === creator && g.status === 'active') return g;
if (g && g.creator_subject === creator && g.status === 'active') return g;
}
}
- const active = db.prepare("SELECT * FROM tip_goals WHERE creator_subject = ? AND status = 'active' LIMIT 2").all(creator);
+ const active = await q.many(sql`SELECT * FROM tip_goals WHERE creator_subject = ${creator} AND status = 'active' LIMIT 2`);
return active.length === 1 ? active[0] : null;
return active.length === 1 ? active[0] : null;
}
}
- /** Inside the settlement transaction. Simulations never get here. */
- function contribute(interaction, requestedGoalId) {
- const g = pick(interaction.creator_subject, requestedGoalId);
- if (!g) return null;
+ /**
+ * Inside the settlement transaction (the interaction row is locked already; the goal is locked
+ * next, keeping the lock order). Simulations never get here. `filter`: the creator's word filter.
+ */
+ async function contribute(t, interaction, requestedGoalId, filter) {
+ const picked = await pick(t, interaction.creator_subject, requestedGoalId);
+ if (!picked) return null;
+ const g = await t.one(sql`SELECT id, status FROM tip_goals WHERE id = ${picked.id} FOR UPDATE`);
+ if (g.status !== 'active') return null;
const at = iso(ctx.now());
const at = iso(ctx.now());
- const r = db.prepare(`INSERT OR IGNORE INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at) VALUES (?, ?, ?, ?, ?)`)
- .run(g.id, interaction.id, interaction.amount, at, at);
- if (!r.changes) return null;
- const reachedNow = markReached(g.id);
+ const n = await t.exec(sql`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)
+ VALUES (${g.id}, ${interaction.id}, ${interaction.amount}, ${at}, ${at}) ON CONFLICT DO NOTHING`);
+ if (!n) return null;
+ const reachedNow = await markReached(t, g.id);
// The overlay's goal update names the supporter as the alert does; not when the amount is hidden
// The overlay's goal update names the supporter as the alert does; not when the amount is hidden
// (the bar's jump would otherwise pin the amount on them), nor while moderation holds it.
// (the bar's jump would otherwise pin the amount on them), nor while moderation holds it.
- const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : ctx.view(interaction).supporter_name;
- return changed(g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });
+ const by = interaction.hide_amount || (interaction.moderation && interaction.moderation !== 'visible') ? null : publicView(interaction, { filter }).supporter_name;
+ return changed(t, g.id, reachedNow ? 'reached' : 'contribution', { interactionId: interaction.id, by });
}
}
/** Inside the reversal transaction: take `bits` back off every goal the interaction counted toward. */
/** Inside the reversal transaction: take `bits` back off every goal the interaction counted toward. */
- function reverse(interaction, bits) {
- const rows = db.prepare('SELECT * FROM tip_goal_contributions WHERE interaction_id = ?').all(interaction.id);
+ async function reverse(t, interaction, bits) {
+ const rows = await t.many(sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = ${interaction.id} ORDER BY goal_id`);
+ if (!rows.length) return [];
+ await t.many(sql`SELECT id FROM tip_goals WHERE id = ANY(${rows.map((c) => c.goal_id)}) ORDER BY id FOR UPDATE`);
const out = [];
const out = [];
for (const c of rows) {
for (const c of rows) {
const take = Math.min(bits, c.amount - c.reversed_amount);
const take = Math.min(bits, c.amount - c.reversed_amount);
if (take <= 0) continue;
if (take <= 0) continue;
- db.prepare('UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ?, updated_at = ? WHERE id = ?').run(take, iso(ctx.now()), c.id);
- markReached(c.goal_id);
- out.push(changed(c.goal_id, 'reversal', { interactionId: interaction.id }));
+ await t.exec(sql`UPDATE tip_goal_contributions SET reversed_amount = reversed_amount + ${take}, updated_at = ${iso(ctx.now())} WHERE id = ${c.id}`);
+ await markReached(t, c.goal_id);
+ out.push(await changed(t, c.goal_id, 'reversal', { interactionId: interaction.id }));
}
}
return out;
return out;
}
}
- return { get, list, present, publicGoal, create, update, close, contribute, reverse, pick, totalOf };
+ return { get, list, present, presentMany, publicGoal, publicGoals, create, update, close, contribute, reverse, pick, totalsOf };
}
}
module.exports = { createGoals };
module.exports = { createGoals };
@@ -3,12 +3,30 @@
@@ -3,12 +3,30 @@
/**
/**
* The Tips domain: one context shared by profiles, goals, interactions, overlays and effects.
* The Tips domain: one context shared by profiles, goals, interactions, overlays and effects.
*
*
- * ctx.tx(fn) run fn in one SQLite transaction; ctx.afterCommit(hook) queues work (overlay
- * pushes, worker kicks) that must only happen once the transaction committed
- * ctx.outbox.emit() durable event, inside the transaction (openvibe-sdk outbox)
- * ctx.view(i) the interaction's public view (privacy.js) through its creator's word filter
+ * Every function that touches the database takes the query handle first: `db` outside a transaction,
+ * or the transaction's `t` inside one (never `db` inside a transaction: that would be another
+ * connection, outside it). Operations that open their own transaction take no handle.
+ *
+ * ctx.tx(fn, { isolation }) run fn(t) in one transaction. t.after(hook) queues work (overlay pushes,
+ * worker and relay kicks) for after the commit: a retried attempt starts
+ * with no hooks, a rolled-back one runs none. Anything that changes money
+ * state (payment state, reversed bits, goal contributions, what a payment
+ * entitles: paid messages, media requests) runs { isolation: 'serializable' }
+ * (ADR-007 amendment 2026-09-24, rule 3); the SDK retries serialization
+ * failures and deadlocks.
+ * ctx.within(q, fn, opts) fn(q) when q is a transaction handle, else fn(t) in a new transaction
+ * ctx.afterCommit(q, hook) t.after(hook) inside a transaction; at once on `db` (already committed)
+ * ctx.lockCreator(t, s) lock a creator's profile row: settings changes and per-creator limits
+ * (20 active goals, 25 tokens, 50 moderators, 10 invitations) take it, so two
+ * requests at once count and write one after the other
+ * ctx.outbox.emit(t, …) a durable event, inside the transaction (openvibe-sdk outbox)
+ *
+ * Lock order (so two transactions never wait on each other in a cycle): a creator's profile row, then
+ * tip_interactions rows (SELECT … FOR UPDATE, several by id), then tip_goals rows by id, then the
+ * creator's overlay stream (an advisory lock taken before a delivery is inserted: overlays.js), then
+ * everything else (effects, paid messages and media requests, overlay deliveries, the outbox).
*/
*/
-const { publicView } = require('./privacy');
+const { sql } = require('openvibe-sdk/db');
const { createProfiles } = require('./profiles');
const { createProfiles } = require('./profiles');
const { createGoals } = require('./goals');
const { createGoals } = require('./goals');
const { createOverlays } = require('./overlays');
const { createOverlays } = require('./overlays');
@@ -16,23 +34,30 @@ const { createInteractions } = require('./interactions');
@@ -16,23 +34,30 @@ const { createInteractions } = require('./interactions');
const { createEffects } = require('./effects');
const { createEffects } = require('./effects');
const { createModeration } = require('./moderation');
const { createModeration } = require('./moderation');
-function createDomain({ db, config, outbox, billing, adapters, now = () => Date.now(), log = console }) {
- const ctx = { db, config, now, log, outbox, billing, adapters, _after: null };
+const MONEY = { isolation: 'serializable' };
+
+function createDomain({ db, config, outbox, billing, adapters, valkey = null, now = () => Date.now(), log = console }) {
+ const ctx = { db, config, now, log, outbox, billing, adapters, valkey, MONEY };
- ctx.tx = (fn) => {
- if (ctx._after) return db.transaction(fn)();
- const hooks = [];
- ctx._after = hooks;
- let result;
- try { result = db.transaction(fn)(); } finally { ctx._after = null; }
- for (const h of hooks) { try { h(); } catch (e) { log.warn('[Tips] after-commit:', e.message); } }
- return result;
+ const runHook = (h) => { Promise.resolve().then(h).catch((e) => log.warn('[Tips] after-commit:', e.message)); };
+ ctx.tx = async (fn, opts = {}) => {
+ let hooks = [];
+ const out = await db.tx(async (t) => {
+ const mine = []; // this attempt's hooks only: a retried or rolled-back attempt's never run
+ hooks = mine;
+ t.after = (hook) => { mine.push(hook); };
+ return fn(t);
+ }, opts);
+ hooks.forEach(runHook);
+ return out;
};
};
- ctx.afterCommit = (hook) => { if (ctx._after) ctx._after.push(hook); else hook(); };
+ const inTx = (q) => !!(q && typeof q.after === 'function');
+ ctx.within = (q, fn, opts) => (inTx(q) ? fn(q) : ctx.tx(fn, opts));
+ ctx.afterCommit = (q, hook) => { if (inTx(q)) q.after(hook); else runHook(hook); };
+ ctx.lockCreator = (t, subject) => t.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE creator_subject = ${subject} FOR UPDATE`);
ctx.outboxKick = () => { try { outbox.kick(); } catch { /* relay off */ } };
ctx.outboxKick = () => { try { outbox.kick(); } catch { /* relay off */ } };
ctx.profiles = createProfiles(ctx);
ctx.profiles = createProfiles(ctx);
- ctx.view = (i, opts = {}) => publicView(i, { ...opts, filter: ctx.profiles.filterOf(i.creator_subject) });
ctx.goals = createGoals(ctx);
ctx.goals = createGoals(ctx);
ctx.overlays = createOverlays(ctx);
ctx.overlays = createOverlays(ctx);
ctx.interactions = createInteractions(ctx);
ctx.interactions = createInteractions(ctx);
@@ -41,4 +66,4 @@ function createDomain({ db, config, outbox, billing, adapters, now = () => Date.
@@ -41,4 +66,4 @@ function createDomain({ db, config, outbox, billing, adapters, now = () => Date.
return ctx;
return ctx;
}
}
-module.exports = { createDomain };
+module.exports = { createDomain, MONEY };
@@ -32,15 +32,20 @@
@@ -32,15 +32,20 @@
* contribution, the overlay alert, the delivery effects and tips.interaction.ready. Delivery then
* contribution, the overlay alert, the delivery effects and tips.interaction.ready. Delivery then
* runs in the effects worker; its failure never touches payment_state.
* runs in the effects worker; its failure never touches payment_state.
*
*
+ * Every transaction that changes money state (a request and its pending-checkout limit, settlement,
+ * failure, funding, reversal) is SERIALIZABLE (ctx.MONEY) and locks the interaction row first
+ * (SELECT … FOR UPDATE), then the goals it counts toward: ADR-007 amendment 2026-09-24, rule 3.
+ *
* Privacy (privacy.js): the supporter's choices are stored with the interaction (anonymous,
* Privacy (privacy.js): the supporter's choices are stored with the interaction (anonymous,
* hide_amount, private_message); everything that leaves Tips for the public is publicView(), and
* hide_amount, private_message); everything that leaves Tips for the public is publicView(), and
* events and answers to anyone but the supporter never name an anonymous supporter. A supporter can
* events and answers to anyone but the supporter never name an anonymous supporter. A supporter can
* export their tips and erase their data from them (erase(): the money record stays, the person
* export their tips and erase their data from them (erase(): the money record stays, the person
* goes; tips.interaction.erased redacts the earlier events in OpenVibe.Events).
* goes; tips.interaction.erased redacts the earlier events in OpenVibe.Events).
*/
*/
-const { fail, iso, prefixedId, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { fail, iso, prefixedId, positiveInt, text, storable, userSubject, entityRef, displayName, readCursor, cursorOf, INVISIBLE } = require('../util');
const { VOICES } = require('./profiles');
const { VOICES } = require('./profiles');
-const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous } = require('./privacy');
+const { parsePrivacy, publicName, shownName, privacyOf, isAnonymous, publicView, ANONYMOUS } = require('./privacy');
const { BillingCallError } = require('../billing-client');
const { BillingCallError } = require('../billing-client');
const KINDS = ['tip', 'paid_message', 'tts', 'media_request'];
const KINDS = ['tip', 'paid_message', 'tts', 'media_request'];
@@ -53,18 +58,28 @@ const MAX_EXTERNAL_CENTS = 100_000_000; // Billing's per-receipt maximum
@@ -53,18 +58,28 @@ const MAX_EXTERNAL_CENTS = 100_000_000; // Billing's per-receipt maximum
const HIGHLIGHT = [[5000, 300], [1000, 120], [500, 60], [100, 30]];
const HIGHLIGHT = [[5000, 300], [1000, 120], [500, 60], [100, 30]];
const highlightFor = (bits) => (HIGHLIGHT.find(([min]) => bits >= min) || [0, 0])[1];
const highlightFor = (bits) => (HIGHLIGHT.find(([min]) => bits >= min) || [0, 0])[1];
const TRANSFER_BACKOFF = [2000, 10000, 30000, 60000, 300000, 600000];
const TRANSFER_BACKOFF = [2000, 10000, 30000, 60000, 300000, 600000];
+// Due transfers are claimed in small batches with a lease: a Billing call takes up to 10 s (twice
+// with a token refresh), so a batch finishes well inside the lease, and a process that dies with a
+// claimed batch leaves it to the others once the lease ends.
+const TRANSFER_BATCH = 10;
+const TRANSFER_LEASE_MS = 5 * 60_000;
+const COLUMNS = ['id', 'creator_subject', 'supporter_subject', 'supporter_name', 'kind', 'amount', 'amount_cents', 'message', 'request',
+ 'funding', 'settlement', 'payment_state', 'delivery_state', 'test', 'billing_txn_id', 'billing_intent_id', 'funding_txn_id', 'provider', 'provider_ref',
+ 'legacy_source', 'idempotency_key', 'origin', 'target', 'anonymous', 'hide_amount', 'private_message', 'created_at', 'updated_at'];
function createInteractions(ctx) {
function createInteractions(ctx) {
- const { db, config } = ctx;
+ const { db, config, MONEY } = ctx;
- const get = (id) => db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(String(id || '')) || null;
- const byBillingTxn = (txnId) => db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').get(txnId) || null;
+ const get = (q, id) => q.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')}`);
+ /** Inside a transaction: the interaction row, locked (the first lock any transaction takes). */
+ const lock = (t, id) => t.maybe(sql`SELECT * FROM tip_interactions WHERE id = ${String(id || '')} FOR UPDATE`);
+ const byBillingTxn = (q, txnId) => q.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${txnId}`);
// ── Input rules (the creator's settings) ─────────────────
// ── Input rules (the creator's settings) ─────────────────
// Plain words only: no control or invisible characters, no links, no markup a speech engine could
// Plain words only: no control or invisible characters, no links, no markup a speech engine could
// take as SSML.
// take as SSML.
function cleanTts(s) {
function cleanTts(s) {
- return String(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ')
+ return storable(s || '').replace(INVISIBLE, '').replace(/[\u0000-\u001f]/g, ' ').replace(/https?:\/\/\S+/gi, 'link').replace(/[<>]/g, ' ')
.replace(/\s+/g, ' ').trim();
.replace(/\s+/g, ' ').trim();
}
}
@@ -89,7 +104,7 @@ function createInteractions(ctx) {
@@ -89,7 +104,7 @@ function createInteractions(ctx) {
* Validate a request against the creator's profile. Returns the normalised fields:
* Validate a request against the creator's profile. Returns the normalised fields:
* { kind, amount, message, request: { goal_id, tts, media, highlight_seconds } }.
* { kind, amount, message, request: { goal_id, tts, media, highlight_seconds } }.
*/
*/
- function validate(profile, input, { simulation = false } = {}) {
+ async function validate(q, profile, input, { simulation = false } = {}) {
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip page');
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip page');
if (!profile.accepting && !simulation) fail(409, 'tips.not_accepting', `${profile.display_name} is not accepting tips right now`);
if (!profile.accepting && !simulation) fail(409, 'tips.not_accepting', `${profile.display_name} is not accepting tips right now`);
const kind = input.kind || 'tip';
const kind = input.kind || 'tip';
@@ -121,7 +136,7 @@ function createInteractions(ctx) {
@@ -121,7 +136,7 @@ function createInteractions(ctx) {
request.media = { ...mediaUrl(input.media && input.media.url), max_seconds: profile.media_max_seconds };
request.media = { ...mediaUrl(input.media && input.media.url), max_seconds: profile.media_max_seconds };
}
}
if (input.goal_id) {
if (input.goal_id) {
- const g = ctx.goals.get(input.goal_id);
+ const g = await ctx.goals.get(q, input.goal_id);
if (!g || g.creator_subject !== profile.creator_subject || g.status !== 'active') fail(422, 'tips.goal_not_found', 'that goal is not open on this page');
if (!g || g.creator_subject !== profile.creator_subject || g.status !== 'active') fail(422, 'tips.goal_not_found', 'that goal is not open on this page');
request.goal_id = g.id;
request.goal_id = g.id;
}
}
@@ -129,29 +144,27 @@ function createInteractions(ctx) {
@@ -129,29 +144,27 @@ function createInteractions(ctx) {
}
}
// ── Creation ─────────────────────────────────────────────
// ── Creation ─────────────────────────────────────────────
- function insert(row) {
+ async function insert(q, row) {
const at = iso(ctx.now());
const at = iso(ctx.now());
const r = {
const r = {
- supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: 0, billing_txn_id: null,
+ supporter_subject: null, supporter_name: null, amount_cents: null, message: null, request: {}, test: false, billing_txn_id: null,
billing_intent_id: null, funding_txn_id: null, provider: null, provider_ref: null, legacy_source: null, idempotency_key: null,
billing_intent_id: null, funding_txn_id: null, provider: null, provider_ref: null, legacy_source: null, idempotency_key: null,
- target: null, origin: 'tips', anonymous: 0, hide_amount: 0, private_message: 0, created_at: at, ...row,
+ target: null, origin: 'tips', anonymous: false, hide_amount: false, private_message: false, created_at: at, ...row,
+ };
+ const values = {
+ ...r, request: sql.json(r.request || {}), target: r.target ? sql.json(r.target) : null, test: !!r.test,
+ anonymous: !!r.anonymous, hide_amount: !!r.hide_amount, private_message: !!r.private_message, updated_at: r.created_at,
};
};
- db.prepare(`INSERT INTO tip_interactions (id, creator_subject, supporter_subject, supporter_name, kind, amount, amount_cents, message, request,
- funding, settlement, payment_state, delivery_state, test, billing_txn_id, billing_intent_id, funding_txn_id, provider, provider_ref,
- legacy_source, idempotency_key, origin, target, anonymous, hide_amount, private_message, created_at, updated_at)
- VALUES (@id, @creator_subject, @supporter_subject, @supporter_name, @kind, @amount, @amount_cents, @message, @request,
- @funding, @settlement, @payment_state, @delivery_state, @test, @billing_txn_id, @billing_intent_id, @funding_txn_id, @provider, @provider_ref,
- @legacy_source, @idempotency_key, @origin, @target, @anonymous, @hide_amount, @private_message, @created_at, @created_at)`)
- .run({ ...r, request: JSON.stringify(r.request || {}), target: r.target ? JSON.stringify(r.target) : null, anonymous: r.anonymous ? 1 : 0, hide_amount: r.hide_amount ? 1 : 0, private_message: r.private_message ? 1 : 0 });
- return get(r.id);
+ return q.one(sql`INSERT INTO tip_interactions ${sql.insert([values], COLUMNS)} RETURNING *`);
}
}
/**
/**
* A supporter's request (checkout or credit). Idempotent by `idempotencyKey` (the caller's key,
* A supporter's request (checkout or credit). Idempotent by `idempotencyKey` (the caller's key,
- * scoped to the caller): a repeat returns the interaction it created the first time.
+ * scoped to the caller): a repeat returns the interaction it created the first time. Serializable:
+ * the unpaid-checkout count and the new row are one decision.
*/
*/
- function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {
- const v = validate(profile, input);
+ async function request(profile, input, { supporter, supporterName, funding, idempotencyKey }) {
+ const v = await validate(db, profile, input);
const from = userSubject(supporter, 'supporter');
const from = userSubject(supporter, 'supporter');
if (from === profile.creator_subject) fail(422, 'tips.self_dealing', 'you cannot tip yourself');
if (from === profile.creator_subject) fail(422, 'tips.self_dealing', 'you cannot tip yourself');
if (!['credit', 'checkout'].includes(funding)) fail(422, 'tips.invalid_input', "pay_with must be 'credit' or 'checkout'");
if (!['credit', 'checkout'].includes(funding)) fail(422, 'tips.invalid_input', "pay_with must be 'credit' or 'checkout'");
@@ -161,25 +174,26 @@ function createInteractions(ctx) {
@@ -161,25 +174,26 @@ function createInteractions(ctx) {
const target = entityRef(input.target);
const target = entityRef(input.target);
const privacy = parsePrivacy(input.privacy, v.kind);
const privacy = parsePrivacy(input.privacy, v.kind);
if (!privacy.anonymous) checkName(profile, displayName(supporterName));
if (!privacy.anonymous) checkName(profile, displayName(supporterName));
- return ctx.tx(() => {
+ return ctx.tx(async (t) => {
if (idempotencyKey) {
if (idempotencyKey) {
- const prev = db.prepare('SELECT * FROM tip_interactions WHERE idempotency_key = ?').get(idempotencyKey);
+ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE idempotency_key = ${idempotencyKey}`);
if (prev) return { interaction: prev, replay: true };
if (prev) return { interaction: prev, replay: true };
}
}
if (funding === 'checkout') {
if (funding === 'checkout') {
// Each unpaid checkout is a Billing intent and a row here: bounded per supporter.
// Each unpaid checkout is a Billing intent and a row here: bounded per supporter.
const since = iso(ctx.now() - 24 * 3600 * 1000);
const since = iso(ctx.now() - 24 * 3600 * 1000);
- const open = db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ? AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= ?").get(from, since).n;
+ const open = await t.value(sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = ${from} AND funding = 'checkout'
+ AND payment_state = 'pending' AND created_at >= ${since}`);
if (open >= config.limits.pendingCheckouts) fail(429, 'tips.too_many_pending', `you have ${open} unpaid checkouts; finish or wait for them before starting another`);
if (open >= config.limits.pendingCheckouts) fail(429, 'tips.too_many_pending', `you have ${open} unpaid checkouts; finish or wait for them before starting another`);
}
}
- const i = insert({
+ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: from, supporter_name: displayName(supporterName) || 'Someone',
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: from, supporter_name: displayName(supporterName) || 'Someone',
kind: v.kind, amount: v.amount, message: v.message, request: v.request, funding, settlement: 'billing', payment_state: 'pending',
kind: v.kind, amount: v.amount, message: v.message, request: v.request, funding, settlement: 'billing', payment_state: 'pending',
delivery_state: 'awaiting_payment', idempotency_key: idempotencyKey || null, target,
delivery_state: 'awaiting_payment', idempotency_key: idempotencyKey || null, target,
...privacy, private_message: privacy.private_message && !!v.message,
...privacy, private_message: privacy.private_message && !!v.message,
});
});
return { interaction: i, replay: false };
return { interaction: i, replay: false };
- });
+ }, MONEY);
}
}
// ── Paying through Billing ───────────────────────────────
// ── Paying through Billing ───────────────────────────────
@@ -194,17 +208,18 @@ function createInteractions(ctx) {
@@ -194,17 +208,18 @@ function createInteractions(ctx) {
key: `tips:transfer:${i.id}`, traceparent,
key: `tips:transfer:${i.id}`, traceparent,
});
});
const txn = out.transaction;
const txn = out.transaction;
- ctx.tx(() => settleByTransaction(get(i.id), { id: txn.id, test: !!txn.test }));
+ await ctx.tx(async (t) => settleByTransaction(t, await lock(t, i.id), { id: txn.id, test: !!txn.test }), MONEY);
return { settled: true };
return { settled: true };
} catch (e) {
} catch (e) {
if (e instanceof BillingCallError && !e.retryable) {
if (e instanceof BillingCallError && !e.retryable) {
- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));
+ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);
return { settled: false, refused: e.code, detail: e.body && e.body.detail };
return { settled: false, refused: e.code, detail: e.body && e.body.detail };
}
}
- const cur = get(i.id);
- const n = cur.transfer_attempts + 1;
- db.prepare('UPDATE tip_interactions SET transfer_due = 1, transfer_attempts = ?, next_transfer_at = ?, failure = ?, updated_at = ? WHERE id = ? AND payment_state = \'pending\'')
- .run(n, ctx.now() + TRANSFER_BACKOFF[Math.min(n - 1, TRANSFER_BACKOFF.length - 1)], String(e.message).slice(0, 300), iso(ctx.now()), i.id);
+ // Due again after the attempt's backoff (TRANSFER_BACKOFF[attempts - 1], the last one repeating).
+ await db.exec(sql`UPDATE tip_interactions SET transfer_due = true, transfer_attempts = transfer_attempts + 1,
+ next_transfer_at = ${ctx.now()} + (${TRANSFER_BACKOFF}::bigint[])[LEAST(transfer_attempts + 1, ${TRANSFER_BACKOFF.length})],
+ failure = ${String(e.message).slice(0, 300)}, updated_at = ${iso(ctx.now())}
+ WHERE id = ${i.id} AND payment_state = 'pending'`);
ctx.log.warn(`[Tips] transfer for ${i.id} will be retried: ${e.message}`);
ctx.log.warn(`[Tips] transfer for ${i.id} will be retried: ${e.message}`);
return { settled: false, retrying: true };
return { settled: false, retrying: true };
}
}
@@ -222,7 +237,7 @@ function createInteractions(ctx) {
@@ -222,7 +237,7 @@ function createInteractions(ctx) {
});
});
} catch (e) {
} catch (e) {
if (e instanceof BillingCallError && !e.retryable) {
if (e instanceof BillingCallError && !e.retryable) {
- ctx.tx(() => failPayment(get(i.id), e.code || 'billing.refused', e.message));
+ await ctx.tx(async (t) => failPayment(t, await lock(t, i.id), e.code || 'billing.refused', e.message), MONEY);
fail(e.status === 409 ? 409 : 422, e.code || 'billing.refused', (e.body && e.body.detail) || 'Billing refused the checkout');
fail(e.status === 409 ? 409 : 422, e.code || 'billing.refused', (e.body && e.body.detail) || 'Billing refused the checkout');
}
}
fail(502, 'tips.billing_unavailable', 'checkout is unavailable right now; nothing was charged — try again shortly');
fail(502, 'tips.billing_unavailable', 'checkout is unavailable right now; nothing was charged — try again shortly');
@@ -235,63 +250,72 @@ function createInteractions(ctx) {
@@ -235,63 +250,72 @@ function createInteractions(ctx) {
}
}
// Pages link and redirect to it: only an https URL is ever kept.
// Pages link and redirect to it: only an https URL is ever kept.
if (url && !/^https:\/\/[^\s"'<>]+$/.test(url)) url = null;
if (url && !/^https:\/\/[^\s"'<>]+$/.test(url)) url = null;
- db.prepare('UPDATE tip_interactions SET billing_intent_id = ?, checkout_url = ?, checkout_ref = ?, provider = ?, updated_at = ? WHERE id = ?')
- .run(intent.id || null, url, ref, p, iso(ctx.now()), i.id);
- return { interaction: get(i.id), checkout_url: url, checkout_ref: ref, intent };
+ const row = await db.one(sql`UPDATE tip_interactions SET billing_intent_id = ${intent.id || null}, checkout_url = ${url}, checkout_ref = ${ref}, provider = ${p},
+ updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);
+ return { interaction: row, checkout_url: url, checkout_ref: ref, intent };
}
}
- /** Due transfers (credit retries and funded checkouts). Called by the jobs loop. */
+ /**
+ * Due transfers (credit retries and funded checkouts). Called by the jobs loop in every process:
+ * each batch is claimed with a lease (FOR UPDATE SKIP LOCKED), so no two processes try one at once.
+ */
async function processDueTransfers() {
async function processDueTransfers() {
- const rows = db.prepare("SELECT * FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due = 1 AND next_transfer_at <= ? ORDER BY next_transfer_at LIMIT 20").all(ctx.now());
+ const now = ctx.now();
+ const rows = await db.many(sql`UPDATE tip_interactions SET next_transfer_at = ${now + TRANSFER_LEASE_MS}
+ WHERE id IN (SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= ${now}
+ ORDER BY next_transfer_at LIMIT ${TRANSFER_BATCH} FOR UPDATE SKIP LOCKED)
+ RETURNING *`);
+ rows.sort((a, b) => (a.created_at < b.created_at ? -1 : 1));
let settled = 0;
let settled = 0;
for (const i of rows) { const r = await transfer(i); if (r.settled) settled++; }
for (const i of rows) { const r = await transfer(i); if (r.settled) settled++; }
return { attempted: rows.length, settled };
return { attempted: rows.length, settled };
}
}
// ── Settlement ───────────────────────────────────────────
// ── Settlement ───────────────────────────────────────────
- /** Inside a transaction: settle `i` by Billing transaction `txn` ({ id, test }). Idempotent. */
- function settleByTransaction(i, txn) {
+ /** Inside a MONEY transaction, `i` locked: settle it by Billing transaction `txn` ({ id, test }). Idempotent. */
+ async function settleByTransaction(t, i, txn) {
if (!i) return null;
if (!i) return null;
if (i.billing_txn_id === txn.id || i.payment_state === 'settled' || i.payment_state === 'reversed') return i;
if (i.billing_txn_id === txn.id || i.payment_state === 'settled' || i.payment_state === 'reversed') return i;
- const other = byBillingTxn(txn.id);
+ const other = await byBillingTxn(t, txn.id);
if (other && other.id !== i.id) return other;
if (other && other.id !== i.id) return other;
- db.prepare('UPDATE tip_interactions SET billing_txn_id = ?, test = MAX(test, ?), transfer_due = 0, failure = NULL WHERE id = ?').run(txn.id, txn.test ? 1 : 0, i.id);
- return settle(get(i.id));
+ const row = await t.one(sql`UPDATE tip_interactions SET billing_txn_id = ${txn.id}, test = test OR ${!!txn.test}, transfer_due = false, failure = NULL
+ WHERE id = ${i.id} RETURNING *`);
+ return settle(t, row);
}
}
- /** Inside a transaction: pending/failed → settled, and the whole effect path. */
- function settle(i) {
+ /** Inside a MONEY transaction, `i` locked (or created by it): pending/failed → settled, and the whole effect path. */
+ async function settle(t, i) {
const at = iso(ctx.now());
const at = iso(ctx.now());
const test = !!i.test;
const test = !!i.test;
- const req = json(i.request, {});
+ const req = i.request || {};
// A tip hidden by a moderator while its payment was pending will deliver nothing.
// A tip hidden by a moderator while its payment was pending will deliver nothing.
- db.prepare(`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,
- settled_at = ?, updated_at = ? WHERE id = ?`).run(at, at, i.id);
- i = get(i.id);
+ i = await t.one(sql`UPDATE tip_interactions SET payment_state = 'settled', delivery_state = CASE WHEN moderation = 'hidden' THEN 'cancelled' ELSE 'queued' END,
+ settled_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
if (i.kind === 'paid_message' || i.kind === 'tts') {
if (i.kind === 'paid_message' || i.kind === 'tts') {
- db.prepare(`INSERT OR IGNORE INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)
- VALUES (?, ?, ?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tpm', ctx.now()), i.id, i.creator_subject, i.kind,
- i.kind === 'tts' ? req.tts.text : (i.message || ''), i.kind === 'tts' ? req.tts.voice : null, req.highlight_seconds || 0, test ? 1 : 0, at, at);
+ await t.exec(sql`INSERT INTO paid_messages (id, interaction_id, creator_subject, kind, text, voice, highlight_seconds, status, test, created_at, updated_at)
+ VALUES (${prefixedId('tpm', ctx.now())}, ${i.id}, ${i.creator_subject}, ${i.kind}, ${i.kind === 'tts' ? req.tts.text : (i.message || '')},
+ ${i.kind === 'tts' ? req.tts.voice : null}, ${req.highlight_seconds || 0}, 'queued', ${test}, ${at}, ${at}) ON CONFLICT DO NOTHING`);
}
}
if (i.kind === 'media_request' && req.media) {
if (i.kind === 'media_request' && req.media) {
- db.prepare(`INSERT OR IGNORE INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)
- VALUES (?, ?, ?, ?, ?, 'queued', ?, ?, ?)`).run(prefixedId('tmr', ctx.now()), i.id, i.creator_subject, req.media.url, req.media.provider, test ? 1 : 0, at, at);
+ await t.exec(sql`INSERT INTO paid_media_requests (id, interaction_id, creator_subject, url, provider, status, test, created_at, updated_at)
+ VALUES (${prefixedId('tmr', ctx.now())}, ${i.id}, ${i.creator_subject}, ${req.media.url}, ${req.media.provider}, 'queued', ${test}, ${at}, ${at})
+ ON CONFLICT DO NOTHING`);
}
}
// The creator's word filter runs before anything is shown, posted or read (moderation.js): a
// The creator's word filter runs before anything is shown, posted or read (moderation.js): a
// match is masked, or held for review when the creator chose that. History is not screened.
// match is masked, or held for review when the creator chose that. History is not screened.
- const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i) : { hit: false };
- if (verdict.hit) ctx.moderation.markScreened(i, verdict);
- i = get(i.id);
- if (!test && i.origin !== 'import') ctx.goals.contribute(i, req.goal_id);
- if (i.origin !== 'import' && i.moderation === 'visible') release(i);
- recomputeDelivery(i.id);
+ const filter = await ctx.profiles.filterOf(t, i.creator_subject);
+ const verdict = i.origin !== 'import' && i.moderation === 'visible' ? ctx.moderation.screen(i, filter) : { hit: false };
+ if (verdict.hit) i = await ctx.moderation.markScreened(t, i, verdict);
+ if (!test && i.origin !== 'import') await ctx.goals.contribute(t, i, req.goal_id, filter);
+ if (i.origin !== 'import' && i.moderation === 'visible') await release(t, i, filter);
+ i = await recomputeDelivery(t, i.id);
if (!test && i.origin !== 'import') {
if (!test && i.origin !== 'import') {
- ctx.outbox.emit('tips.interaction.ready', { type: 'interaction', id: i.id }, summary(get(i.id)));
+ await ctx.outbox.emit(t, 'tips.interaction.ready', { type: 'interaction', id: i.id }, summary(i));
}
}
- if (verdict.hit) ctx.moderation.recordScreened(get(i.id), verdict);
- ctx.afterCommit(() => { ctx.effects.kick(); ctx.outboxKick(); });
- return get(i.id);
+ if (verdict.hit) await ctx.moderation.recordScreened(t, i, verdict);
+ ctx.afterCommit(t, () => { ctx.effects.kick(); ctx.outboxKick(); });
+ return i;
}
}
/**
/**
@@ -299,97 +323,90 @@ function createInteractions(ctx) {
@@ -299,97 +323,90 @@ function createInteractions(ctx) {
* deliveries. Settlement calls it, and moderation's restore() for one the filter held. Idempotent:
* deliveries. Settlement calls it, and moderation's restore() for one the filter held. Idempotent:
* nothing that exists (delivered or cancelled) is created again.
* nothing that exists (delivered or cancelled) is created again.
*/
*/
- function release(i) {
+ async function release(t, i, filter) {
const test = !!i.test;
const test = !!i.test;
- ctx.overlays.addAlert(i);
- addEffect(i, 'overlay_alert', 'overlay', 'delivered');
+ await ctx.overlays.addAlert(t, i, filter);
+ const at = iso(ctx.now());
+ const effects = [{ effect: 'overlay_alert', adapter: 'overlay', state: 'delivered' }];
// Chat effects: never for imports (history), never twice for a donation another product already
// Chat effects: never for imports (history), never twice for a donation another product already
// announced (origin billing), never to a real chat room for a simulation.
// announced (origin billing), never to a real chat room for a simulation.
const adapter = test ? 'test' : config.chat.adapter;
const adapter = test ? 'test' : config.chat.adapter;
if (ANNOUNCED_HERE.includes(i.origin) && adapter !== 'none') {
if (ANNOUNCED_HERE.includes(i.origin) && adapter !== 'none') {
- if (i.kind === 'tip') addEffect(i, 'chat_line', adapter);
- if (i.kind === 'paid_message') addEffect(i, 'paid_message', adapter);
- if (i.kind === 'tts') { addEffect(i, 'chat_line', adapter); addEffect(i, 'tts', adapter); }
- if (i.kind === 'media_request') addEffect(i, 'media_request', adapter);
+ const chat = { tip: ['chat_line'], paid_message: ['paid_message'], tts: ['chat_line', 'tts'], media_request: ['media_request'] }[i.kind] || [];
+ for (const effect of chat) effects.push({ effect, adapter, state: 'queued' });
}
}
- recomputeDelivery(i.id);
- ctx.afterCommit(() => ctx.effects.kick());
+ await t.exec(sql`INSERT INTO interaction_effects ${sql.insert(effects.map((e) => ({ interaction_id: i.id, ...e, created_at: at, updated_at: at })))}
+ ON CONFLICT DO NOTHING`);
+ await recomputeDelivery(t, i.id);
+ ctx.afterCommit(t, () => ctx.effects.kick());
}
}
- function addEffect(i, effect, adapter, state = 'queued') {
- const at = iso(ctx.now());
- db.prepare(`INSERT OR IGNORE INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?)`)
- .run(i.id, effect, adapter, state, at, at);
- }
-
- /** Inside a transaction: pending → failed (nothing was delivered; nothing moved in Billing). */
- function failPayment(i, code, detail) {
+ /** Inside a MONEY transaction, `i` locked: pending → failed (nothing was delivered; nothing moved in Billing). */
+ async function failPayment(t, i, code, detail) {
if (!i || i.payment_state !== 'pending') return i;
if (!i || i.payment_state !== 'pending') return i;
- const at = iso(ctx.now());
- db.prepare("UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = 0, failure = ?, updated_at = ? WHERE id = ?")
- .run(`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`, at, i.id);
- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_failed', code });
- return get(i.id);
+ const row = await t.one(sql`UPDATE tip_interactions SET payment_state = 'failed', delivery_state = 'cancelled', transfer_due = false,
+ failure = ${`${code}${detail ? `: ${String(detail).slice(0, 240)}` : ''}`}, updated_at = ${iso(ctx.now())} WHERE id = ${i.id} RETURNING *`);
+ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_failed', code });
+ return row;
}
}
- /** Billing settled a transaction. Inside the inbox transaction. */
- function onBillingSettled(p) {
+ /** Billing settled a transaction. Inside the inbox's MONEY transaction. */
+ async function onBillingSettled(t, p) {
const meta = p.metadata || {};
const meta = p.metadata || {};
const txn = { id: p.transaction_id, test: !!p.test };
const txn = { id: p.transaction_id, test: !!p.test };
if (!txn.id) return 'ignored:no_transaction';
if (!txn.id) return 'ignored:no_transaction';
if (p.type === 'donation') {
if (p.type === 'donation') {
- const known = byBillingTxn(txn.id);
- if (known) return 'duplicate_transaction';
+ if (await byBillingTxn(t, txn.id)) return 'duplicate_transaction';
const target = meta.target;
const target = meta.target;
if (target && target.service === 'tips' && target.type === 'interaction') {
if (target && target.service === 'tips' && target.type === 'interaction') {
- const i = get(target.id);
+ const i = await lock(t, target.id);
if (!i) return 'ignored:unknown_interaction';
if (!i) return 'ignored:unknown_interaction';
// Only the transfer Tips asked for settles the interaction: same amount, same creator, same
// Only the transfer Tips asked for settles the interaction: same amount, same creator, same
// supporter. Another service's donation that merely names it (a 1-bit transfer claiming a
// supporter. Another service's donation that merely names it (a 1-bit transfer claiming a
// 10,000-bit paid message) is recorded as what it is, a donation of its own amount.
// 10,000-bit paid message) is recorded as what it is, a donation of its own amount.
const matches = Number(meta.amount_bits) === i.amount && p.to_subject === i.creator_subject && (!i.supporter_subject || p.from_subject === i.supporter_subject);
const matches = Number(meta.amount_bits) === i.amount && p.to_subject === i.creator_subject && (!i.supporter_subject || p.from_subject === i.supporter_subject);
- if (matches) { settleByTransaction(i, txn); return 'settled'; }
+ if (matches) { await settleByTransaction(t, i, txn); return 'settled'; }
ctx.log.warn(`[Tips] donation ${txn.id} names interaction ${i.id} but does not match it (amount, creator or supporter); recorded on its own`);
ctx.log.warn(`[Tips] donation ${txn.id} names interaction ${i.id} but does not match it (amount, creator or supporter); recorded on its own`);
}
}
// A donation Tips did not start: record it once, by its transaction id.
// A donation Tips did not start: record it once, by its transaction id.
if (!p.to_subject) return 'ignored:no_recipient';
if (!p.to_subject) return 'ignored:no_recipient';
const amount = Number(meta.amount_bits) || 0;
const amount = Number(meta.amount_bits) || 0;
if (amount <= 0) return 'ignored:no_amount';
if (amount <= 0) return 'ignored:no_amount';
- const i = insert({
+ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: p.to_subject, supporter_subject: p.from_subject || null,
id: prefixedId('tint', ctx.now()), creator_subject: p.to_subject, supporter_subject: p.from_subject || null,
supporter_name: displayName(meta.donor_name) || null, anonymous: meta.anonymous === true, kind: meta.kind === 'paid_interaction' ? 'media_request' : 'tip', amount, amount_cents: meta.paid_cents || null,
supporter_name: displayName(meta.donor_name) || null, anonymous: meta.anonymous === true, kind: meta.kind === 'paid_interaction' ? 'media_request' : 'tip', amount, amount_cents: meta.paid_cents || null,
- message: meta.message ? String(meta.message).slice(0, config.limits.messageChars) : null,
+ message: meta.message ? storable(String(meta.message).slice(0, config.limits.messageChars)) : null,
funding: p.provider ? 'provider' : 'credit', settlement: 'billing', payment_state: 'pending', delivery_state: 'awaiting_payment',
funding: p.provider ? 'provider' : 'credit', settlement: 'billing', payment_state: 'pending', delivery_state: 'awaiting_payment',
- test: txn.test ? 1 : 0, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',
+ test: txn.test, billing_txn_id: txn.id, provider: p.provider || null, origin: 'billing',
target: meta.target && meta.target.service ? meta.target : null,
target: meta.target && meta.target.service ? meta.target : null,
});
});
- settle(i);
+ await settle(t, i);
return 'recorded';
return 'recorded';
}
}
if (p.type === 'purchase' && meta.intent_id) {
if (p.type === 'purchase' && meta.intent_id) {
- const i = db.prepare('SELECT * FROM tip_interactions WHERE billing_intent_id = ?').get(meta.intent_id);
+ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_intent_id = ${meta.intent_id} FOR UPDATE`);
if (!i) return 'ignored:not_a_tips_checkout';
if (!i) return 'ignored:not_a_tips_checkout';
if (i.funding_txn_id) return 'duplicate_funding';
if (i.funding_txn_id) return 'duplicate_funding';
- db.prepare("UPDATE tip_interactions SET funding_txn_id = ?, test = MAX(test, ?), transfer_due = CASE WHEN payment_state = 'pending' THEN 1 ELSE 0 END, next_transfer_at = 0, updated_at = ? WHERE id = ?")
- .run(txn.id, txn.test ? 1 : 0, iso(ctx.now()), i.id);
- ctx.afterCommit(() => { processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)); });
+ await t.exec(sql`UPDATE tip_interactions SET funding_txn_id = ${txn.id}, test = test OR ${txn.test}, transfer_due = (payment_state = 'pending'),
+ next_transfer_at = 0, updated_at = ${iso(ctx.now())} WHERE id = ${i.id}`);
+ ctx.afterCommit(t, () => processDueTransfers().catch((e) => ctx.log.warn('[Tips] transfer after funding:', e.message)));
return 'funded';
return 'funded';
}
}
return 'ignored';
return 'ignored';
}
}
- /** Billing reversed a transaction. Inside the inbox transaction. */
- function onBillingReversed(p) {
+ /** Billing reversed a transaction. Inside the inbox's MONEY transaction. */
+ async function onBillingReversed(t, p) {
const meta = p.metadata || {};
const meta = p.metadata || {};
const orig = p.reverses_txn;
const orig = p.reverses_txn;
if (!orig) return 'ignored:no_original';
if (!orig) return 'ignored:no_original';
- const i = byBillingTxn(orig);
+ const i = await t.maybe(sql`SELECT * FROM tip_interactions WHERE billing_txn_id = ${orig} FOR UPDATE`);
if (!i) {
if (!i) {
- const funded = db.prepare('SELECT * FROM tip_interactions WHERE funding_txn_id = ?').get(orig);
- if (funded && funded.payment_state === 'pending') { failPayment(funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }
+ const funded = await t.maybe(sql`SELECT * FROM tip_interactions WHERE funding_txn_id = ${orig} FOR UPDATE`);
+ if (funded && funded.payment_state === 'pending') { await failPayment(t, funded, 'billing.funding_reversed', 'the checkout payment was reversed before the tip was given'); return 'funding_reversed'; }
return 'ignored';
return 'ignored';
}
}
- const ids = json(i.reversal_txn_ids, []);
+ const ids = Array.isArray(i.reversal_txn_ids) ? [...i.reversal_txn_ids] : [];
if (ids.includes(p.transaction_id)) return 'duplicate_reversal';
if (ids.includes(p.transaction_id)) return 'duplicate_reversal';
ids.push(p.transaction_id);
ids.push(p.transaction_id);
// Bits Billing took back from the creator: a transfer refund moves them out of the payable. A
// Bits Billing took back from the creator: a transfer refund moves them out of the payable. A
@@ -397,52 +414,54 @@ function createInteractions(ctx) {
@@ -397,52 +414,54 @@ function createInteractions(ctx) {
const clawed = p.type === 'refund' && p.from_subject === i.creator_subject ? Math.max(0, Number(meta.amount_bits) || 0) : 0;
const clawed = p.type === 'refund' && p.from_subject === i.creator_subject ? Math.max(0, Number(meta.amount_bits) || 0) : 0;
const take = Math.min(clawed, i.amount - i.reversed_bits);
const take = Math.min(clawed, i.amount - i.reversed_bits);
const at = iso(ctx.now());
const at = iso(ctx.now());
- db.prepare("UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ?, reversed_bits = reversed_bits + ?, reversed_at = ?, updated_at = ? WHERE id = ?")
- .run(JSON.stringify(ids), take, at, at, i.id);
- if (take > 0 && !i.test) ctx.goals.reverse(i, take);
+ const reversed = await t.one(sql`UPDATE tip_interactions SET payment_state = 'reversed', reversal_txn_ids = ${sql.json(ids)}, reversed_bits = reversed_bits + ${take},
+ reversed_at = ${at}, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
+ if (take > 0 && !i.test) await ctx.goals.reverse(t, reversed, take);
// Undelivered effects are cancelled; a delivery that already happened stays on record.
// Undelivered effects are cancelled; a delivery that already happened stays on record.
- let cancelled = cancelQueued(get(i.id), 'payment_reversed');
- const now = get(i.id);
+ let cancelled = await cancelQueued(t, reversed, 'payment_reversed');
+ const now = await get(t, i.id);
if (!cancelled && now.moderation === 'held' && now.delivery_state === 'queued') {
if (!cancelled && now.moderation === 'held' && now.delivery_state === 'queued') {
// Held by the filter and never shown: it will not be released now.
// Held by the filter and never shown: it will not be released now.
- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);
- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
- if (!now.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason: 'payment_reversed' });
+ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
+ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
+ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
+ if (!now.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason: 'payment_reversed' });
cancelled = true;
cancelled = true;
}
}
return cancelled ? 'reversed_cancelled' : 'reversed';
return cancelled ? 'reversed_cancelled' : 'reversed';
}
}
- /** Inside a transaction: queued effects → cancelled. True when anything was cancelled. */
- function cancelQueued(i, reason) {
+ /** Inside a transaction, the interaction locked: queued effects → cancelled. True when anything was cancelled. */
+ async function cancelQueued(t, i, reason) {
const at = iso(ctx.now());
const at = iso(ctx.now());
- const r = db.prepare("UPDATE interaction_effects SET state = 'cancelled', updated_at = ? WHERE interaction_id = ? AND state = 'queued'").run(at, i.id);
- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, i.id);
- if (!r.changes) return false;
- db.prepare("UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ? WHERE id = ?").run(at, i.id);
- if (!i.test) ctx.outbox.emit('tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(get(i.id)), reason });
+ const n = await t.exec(sql`UPDATE interaction_effects SET state = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND state = 'queued'`);
+ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
+ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${i.id} AND status = 'queued'`);
+ if (!n) return false;
+ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = 'cancelled', updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
+ if (!i.test) await ctx.outbox.emit(t, 'tips.interaction.cancelled', { type: 'interaction', id: i.id }, { ...summary(row), reason });
return true;
return true;
}
}
- /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. */
- function recomputeDelivery(id) {
- const i = get(id);
+ /** Aggregate the effects into delivery_state (inside a transaction). Emits failed once. Returns the row. */
+ async function recomputeDelivery(t, id) {
+ const i = await get(t, id);
if (!i || i.delivery_state === 'cancelled' || i.delivery_state === 'awaiting_payment') return i;
if (!i || i.delivery_state === 'cancelled' || i.delivery_state === 'awaiting_payment') return i;
- const effects = db.prepare('SELECT state FROM interaction_effects WHERE interaction_id = ?').all(id).map((e) => e.state);
- if (i.moderation === 'held' && !effects.length) return i; // waiting for a moderator: still to deliver
+ const effects = await t.many(sql`SELECT effect, adapter, state, last_error FROM interaction_effects WHERE interaction_id = ${id} ORDER BY id`);
+ const states = effects.map((e) => e.state);
+ if (i.moderation === 'held' && !states.length) return i; // waiting for a moderator: still to deliver
let next = 'queued';
let next = 'queued';
- if (effects.includes('failed') && !effects.includes('queued')) next = 'failed';
- else if (!effects.includes('queued')) next = 'delivered';
+ if (states.includes('failed') && !states.includes('queued')) next = 'failed';
+ else if (!states.includes('queued')) next = 'delivered';
if (next === i.delivery_state) return i;
if (next === i.delivery_state) return i;
const at = iso(ctx.now());
const at = iso(ctx.now());
- db.prepare(`UPDATE tip_interactions SET delivery_state = ?, delivered_at = CASE WHEN ? = 'delivered' THEN ? ELSE delivered_at END, updated_at = ? WHERE id = ?`).run(next, next, at, at, id);
+ const row = await t.one(sql`UPDATE tip_interactions SET delivery_state = ${next},
+ delivered_at = CASE WHEN ${next === 'delivered'} THEN ${at}::timestamptz ELSE delivered_at END, updated_at = ${at} WHERE id = ${id} RETURNING *`);
if (next === 'failed' && !i.test) {
if (next === 'failed' && !i.test) {
- const failed = db.prepare("SELECT effect, adapter, last_error FROM interaction_effects WHERE interaction_id = ? AND state = 'failed'").all(id);
- ctx.outbox.emit('tips.interaction.failed', { type: 'interaction', id }, { ...summary(get(id)), failed_effects: failed });
+ const failed = effects.filter((e) => e.state === 'failed').map(({ effect, adapter, last_error: lastError }) => ({ effect, adapter, last_error: lastError }));
+ await ctx.outbox.emit(t, 'tips.interaction.failed', { type: 'interaction', id }, { ...summary(row), failed_effects: failed });
}
}
- return get(id);
+ return row;
}
}
/**
/**
@@ -450,12 +469,12 @@ function createInteractions(ctx) {
@@ -450,12 +469,12 @@ function createInteractions(ctx) {
* or a Live donation_goals id (links minted by Live), which the Live import mapped to a Tips goal.
* or a Live donation_goals id (links minted by Live), which the Live import mapped to a Tips goal.
* goals.contribute() still checks it is the creator's and active, else falls back to Live's rule.
* goals.contribute() still checks it is the creator's and active, else falls back to Live's rule.
*/
*/
- function goalFromPurpose(...refs) {
+ async function goalFromPurpose(q, ...refs) {
for (const v of refs) {
for (const v of refs) {
- const m = String(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);
+ const m = storable(v || '').match(/(?:^|[:_-])goal[:_-]?([A-Za-z0-9_]+)/i);
if (!m) continue;
if (!m) continue;
if (/^\d+$/.test(m[1])) {
if (/^\d+$/.test(m[1])) {
- const map = db.prepare("SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ? AND status = 'imported'").get(m[1]);
+ const map = await q.maybe(sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = ${m[1]} AND status = 'imported'`);
if (map && map.target_id) return map.target_id;
if (map && map.target_id) return map.target_id;
continue;
continue;
}
}
@@ -466,84 +485,84 @@ function createInteractions(ctx) {
@@ -466,84 +485,84 @@ function createInteractions(ctx) {
/**
/**
* Billing announced an EXTERNAL receipt (billing.receipt.external): a tip paid on the creator's own
* Billing announced an EXTERNAL receipt (billing.receipt.external): a tip paid on the creator's own
- * PowerChat. Inside the inbox transaction. Recorded once by (provider, provider event id) — the same
- * key POST /interactions/external uses — settled at once (the provider already confirmed it), and
- * announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.
+ * PowerChat. Inside the inbox's MONEY transaction. Recorded once by (provider, provider event id) —
+ * the same key POST /interactions/external uses — settled at once (the provider already confirmed
+ * it), and announced by Tips: chat line through the chat adapter, overlay alert, goal contribution.
*/
*/
- function onBillingExternal(p) {
+ async function onBillingExternal(t, p) {
const provider = String(p.provider || '').toLowerCase();
const provider = String(p.provider || '').toLowerCase();
- const ref = String(p.provider_event_id || '').trim();
+ const ref = storable(p.provider_event_id || '').trim();
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider) || !ref || ref.length > 200) return 'ignored:no_reference';
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider) || !ref || ref.length > 200) return 'ignored:no_reference';
const creator = p.streamer && p.streamer.type === 'user' && SUBJECT_RE.test(String(p.streamer.id || '')) ? p.streamer.id : null;
const creator = p.streamer && p.streamer.type === 'user' && SUBJECT_RE.test(String(p.streamer.id || '')) ? p.streamer.id : null;
if (!creator) return 'ignored:no_streamer';
if (!creator) return 'ignored:no_streamer';
const cents = Number(p.amount_cents);
const cents = Number(p.amount_cents);
if (!Number.isSafeInteger(cents) || cents < 1 || cents > MAX_EXTERNAL_CENTS) return 'ignored:no_amount';
if (!Number.isSafeInteger(cents) || cents < 1 || cents > MAX_EXTERNAL_CENTS) return 'ignored:no_amount';
- if (db.prepare('SELECT 1 FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) return 'duplicate_receipt';
+ if (await t.maybe(sql`SELECT 1 FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`)) return 'duplicate_receipt';
// Goals and the chat line count Vibes: the value Billing put on the money (1 bit = 1 cent today).
// Goals and the chat line count Vibes: the value Billing put on the money (1 bit = 1 cent today).
const bits = Number(p.value_bits);
const bits = Number(p.value_bits);
const amount = Math.min(Number.isSafeInteger(bits) && bits > 0 ? bits : cents, config.limits.maxBits);
const amount = Math.min(Number.isSafeInteger(bits) && bits > 0 ? bits : cents, config.limits.maxBits);
- const message = p.message ? String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars) || null : null;
- const goalId = goalFromPurpose(p.app_purpose, p.app_ref);
- const i = insert({
+ const message = p.message ? storable(String(p.message).replace(/\r\n?/g, '\n').trim().slice(0, config.limits.messageChars)) || null : null;
+ const goalId = await goalFromPurpose(t, p.app_purpose, p.app_ref);
+ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: creator, supporter_subject: null,
id: prefixedId('tint', ctx.now()), creator_subject: creator, supporter_subject: null,
supporter_name: p.anonymous ? 'Anonymous' : (displayName(p.donor_name) || 'Someone'), anonymous: !!p.anonymous, kind: 'tip', amount, amount_cents: cents,
supporter_name: p.anonymous ? 'Anonymous' : (displayName(p.donor_name) || 'Someone'), anonymous: !!p.anonymous, kind: 'tip', amount, amount_cents: cents,
message, request: goalId ? { goal_id: goalId } : {}, funding: 'external', settlement: 'external', payment_state: 'pending',
message, request: goalId ? { goal_id: goalId } : {}, funding: 'external', settlement: 'external', payment_state: 'pending',
- delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: p.test ? 1 : 0,
+ delivery_state: 'awaiting_payment', provider, provider_ref: ref, origin: 'billing-external', test: !!p.test,
});
});
- settle(i);
+ await settle(t, i);
return 'recorded_external';
return 'recorded_external';
}
}
// ── External and simulated ───────────────────────────────
// ── External and simulated ───────────────────────────────
/** A tip on the creator's own PowerChat (EXTERNAL, ADR-012): recorded once by provider ref. */
/** A tip on the creator's own PowerChat (EXTERNAL, ADR-012): recorded once by provider ref. */
- function recordExternal(profile, input) {
+ async function recordExternal(profile, input) {
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip profile');
if (!profile) fail(404, 'tips.creator_not_found', 'this creator has no tip profile');
const provider = String(input.provider || '').toLowerCase();
const provider = String(input.provider || '').toLowerCase();
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider)) fail(422, 'tips.invalid_input', 'provider is required');
if (!/^[a-z][a-z0-9_-]{1,39}$/.test(provider)) fail(422, 'tips.invalid_input', 'provider is required');
- const ref = String(input.provider_ref || '').trim();
+ const ref = storable(input.provider_ref || '').trim();
if (!ref || ref.length > 200) fail(422, 'tips.invalid_input', 'provider_ref (the provider event id) is required');
if (!ref || ref.length > 200) fail(422, 'tips.invalid_input', 'provider_ref (the provider event id) is required');
const cents = positiveInt(input.amount_cents, 'amount_cents', 100_000_000);
const cents = positiveInt(input.amount_cents, 'amount_cents', 100_000_000);
const privacy = parsePrivacy(input.privacy, 'tip');
const privacy = parsePrivacy(input.privacy, 'tip');
- return ctx.tx(() => {
- const prev = db.prepare('SELECT * FROM tip_interactions WHERE provider = ? AND provider_ref = ?').get(provider, ref);
+ return ctx.tx(async (t) => {
+ const prev = await t.maybe(sql`SELECT * FROM tip_interactions WHERE provider = ${provider} AND provider_ref = ${ref}`);
if (prev) return { interaction: prev, replay: true };
if (prev) return { interaction: prev, replay: true };
- const i = insert({
+ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: input.supporter ? userSubject(input.supporter, 'supporter') : null,
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: input.supporter ? userSubject(input.supporter, 'supporter') : null,
supporter_name: displayName(input.supporter_name) || 'Someone', kind: 'tip', amount: cents, amount_cents: cents,
supporter_name: displayName(input.supporter_name) || 'Someone', kind: 'tip', amount: cents, amount_cents: cents,
message: text(input.message, 'message', 500), request: input.goal_id ? { goal_id: String(input.goal_id) } : {},
message: text(input.message, 'message', 500), request: input.goal_id ? { goal_id: String(input.goal_id) } : {},
funding: 'external', settlement: 'external', payment_state: 'pending', delivery_state: 'awaiting_payment',
funding: 'external', settlement: 'external', payment_state: 'pending', delivery_state: 'awaiting_payment',
// origin external: the provider (or Live's webhook) already announced it in chat; with
// origin external: the provider (or Live's webhook) already announced it in chat; with
// announce: true Tips delivers the chat line itself.
// announce: true Tips delivers the chat line itself.
- provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: input.test ? 1 : 0,
+ provider, provider_ref: ref, origin: input.announce === true ? 'tips' : 'external', test: !!input.test,
...privacy, private_message: privacy.private_message && !!input.message,
...privacy, private_message: privacy.private_message && !!input.message,
});
});
- return { interaction: settle(i), replay: false };
- });
+ return { interaction: await settle(t, i), replay: false };
+ }, MONEY);
}
}
- /** The full effect path with test = 1 and no Billing call. */
- function simulate(profile, input, { by }) {
- const v = validate(profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });
+ /** The full effect path with test = true and no Billing call. */
+ async function simulate(profile, input, { by }) {
+ const v = await validate(db, profile, { ...input, amount: input.amount || Math.max(profile.min_amount, 100) }, { simulation: true });
const privacy = parsePrivacy(input.privacy, v.kind);
const privacy = parsePrivacy(input.privacy, v.kind);
- return ctx.tx(() => {
- const i = insert({
+ return ctx.tx(async (t) => {
+ const i = await insert(t, {
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: null,
id: prefixedId('tint', ctx.now()), creator_subject: profile.creator_subject, supporter_subject: null,
supporter_name: displayName(input.supporter_name) || 'Test supporter', kind: v.kind, amount: v.amount, message: v.message,
supporter_name: displayName(input.supporter_name) || 'Test supporter', kind: v.kind, amount: v.amount, message: v.message,
request: { ...v.request, simulated_by: by }, funding: 'none', settlement: 'simulated', payment_state: 'pending',
request: { ...v.request, simulated_by: by }, funding: 'none', settlement: 'simulated', payment_state: 'pending',
- delivery_state: 'awaiting_payment', test: 1, ...privacy, private_message: privacy.private_message && !!v.message,
+ delivery_state: 'awaiting_payment', test: true, ...privacy, private_message: privacy.private_message && !!v.message,
});
});
- const out = settle(i);
+ const out = await settle(t, i);
// Show the goal widget moving without counting anything: a test goal delivery with the
// Show the goal widget moving without counting anything: a test goal delivery with the
// would-be total. No contribution row is written.
// would-be total. No contribution row is written.
- const g = ctx.goals.pick(profile.creator_subject, v.request.goal_id);
+ const g = await ctx.goals.pick(t, profile.creator_subject, v.request.goal_id);
if (g) {
if (g) {
- const view = ctx.goals.present(g);
+ const [view] = await ctx.goals.presentMany(t, [g]);
const would = { ...view, current_amount: view.current_amount + v.amount, percent: Math.min(100, Math.floor(((view.current_amount + v.amount) * 100) / view.target_amount)) };
const would = { ...view, current_amount: view.current_amount + v.amount, percent: Math.min(100, Math.floor(((view.current_amount + v.amount) * 100) / view.target_amount)) };
- const shown = get(i.id);
- ctx.overlays.addGoalDelivery(profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: shown.hide_amount || shown.moderation !== 'visible' ? null : ctx.view(shown).supporter_name, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });
+ const by2 = out.hide_amount || out.moderation !== 'visible' ? null : publicView(out, { filter: profile.filter }).supporter_name;
+ await ctx.overlays.addGoalDelivery(t, profile.creator_subject, would, { reason: 'simulation', interactionId: i.id, by: by2, dedupe: `goal:${g.id}:sim:${i.id}`, test: true });
}
}
return out;
return out;
- });
+ }, MONEY);
}
}
// ── Reads ────────────────────────────────────────────────
// ── Reads ────────────────────────────────────────────────
@@ -557,14 +576,8 @@ function createInteractions(ctx) {
@@ -557,14 +576,8 @@ function createInteractions(ctx) {
};
};
}
}
- /**
- * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or
- * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;
- * `public` is what may be shown to the public, `privacy` the supporter's choices.
- */
- function present(i, { viewer = 'owner' } = {}) {
- if (!i) return null;
- const req = json(i.request, {});
+ function shape(i, viewer, filter, effects) {
+ const req = i.request || {};
const own = viewer === 'supporter';
const own = viewer === 'supporter';
const named = own || !isAnonymous(i);
const named = own || !isAnonymous(i);
const out = {
const out = {
@@ -572,15 +585,15 @@ function createInteractions(ctx) {
@@ -572,15 +585,15 @@ function createInteractions(ctx) {
supporter_name: named ? (i.supporter_name || null) : publicName(i), kind: i.kind, amount: i.amount, currency: i.currency, message: i.message || null,
supporter_name: named ? (i.supporter_name || null) : publicName(i), kind: i.kind, amount: i.amount, currency: i.currency, message: i.message || null,
tts: req.tts || null, media: req.media ? { url: req.media.url, provider: req.media.provider } : null, goal_id: req.goal_id || null,
tts: req.tts || null, media: req.media ? { url: req.media.url, provider: req.media.provider } : null, goal_id: req.goal_id || null,
funding: i.funding, settlement: i.settlement, origin: i.origin, test: !!i.test,
funding: i.funding, settlement: i.settlement, origin: i.origin, test: !!i.test,
- privacy: privacyOf(i), public: ctx.view(i), erased_at: i.erased_at || null,
+ privacy: privacyOf(i), public: publicView(i, { filter }), erased_at: i.erased_at || null,
moderation: { state: i.moderation, filtered: !!i.filtered, at: i.moderated_at || null },
moderation: { state: i.moderation, filtered: !!i.filtered, at: i.moderated_at || null },
- payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: json(i.reversal_txn_ids, []), reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },
+ payment: { state: i.payment_state, billing_txn_id: i.billing_txn_id || null, reversal_txn_ids: i.reversal_txn_ids || [], reversed_amount: i.reversed_bits, failure: i.failure || null, settled_at: i.settled_at || null, reversed_at: i.reversed_at || null },
delivery: { state: i.delivery_state, delivered_at: i.delivered_at || null },
delivery: { state: i.delivery_state, delivered_at: i.delivered_at || null },
created_at: i.created_at, updated_at: i.updated_at,
created_at: i.created_at, updated_at: i.updated_at,
};
};
if (i.funding === 'checkout') out.checkout = { url: i.checkout_url || null, ref: i.checkout_ref || null, provider: i.provider || null, intent_id: i.billing_intent_id || null };
if (i.funding === 'checkout') out.checkout = { url: i.checkout_url || null, ref: i.checkout_ref || null, provider: i.provider || null, intent_id: i.billing_intent_id || null };
if (viewer === 'owner' || viewer === 'service') {
if (viewer === 'owner' || viewer === 'service') {
- out.effects = db.prepare('SELECT effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);
+ out.effects = effects;
out.provider = i.provider || null;
out.provider = i.provider || null;
out.provider_ref = i.provider_ref || null;
out.provider_ref = i.provider_ref || null;
out.legacy_source = i.legacy_source || null;
out.legacy_source = i.legacy_source || null;
@@ -588,55 +601,83 @@ function createInteractions(ctx) {
@@ -588,55 +601,83 @@ function createInteractions(ctx) {
return out;
return out;
}
}
- /** Cursor-paged list, newest first. filter: { creator?, supporter?, includeTest? } */
- function list({ creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {
+ /**
+ * viewer: 'supporter' (their own receipt: everything they chose), 'owner' (the creator) or
+ * 'service'. Anyone but the supporter gets no subject and "Anonymous" for an anonymous supporter;
+ * `public` is what may be shown to the public, `privacy` the supporter's choices. A page of any
+ * size costs two queries: the creators' filters, and (owner, service) every row's effects.
+ */
+ async function presentMany(q, rows, { viewer = 'owner' } = {}) {
+ if (!rows.length) return [];
+ const filters = await ctx.profiles.filtersOf(q, rows.map((i) => i.creator_subject));
+ const effects = new Map();
+ if (viewer === 'owner' || viewer === 'service') {
+ const list = await q.many(sql`SELECT interaction_id, effect, adapter, state, attempts, last_error, updated_at FROM interaction_effects
+ WHERE interaction_id = ANY(${rows.map((i) => i.id)}) ORDER BY interaction_id, id`);
+ for (const { interaction_id: id, ...e } of list) { if (!effects.has(id)) effects.set(id, []); effects.get(id).push(e); }
+ }
+ return rows.map((i) => shape(i, viewer, filters.get(i.creator_subject), effects.get(i.id) || []));
+ }
+ const present = async (q, i, opts) => (i ? (await presentMany(q, [i], opts))[0] : null);
+
+ /** A keyset page, newest first. filter: { creator?, supporter?, includeTest? } (one of the first two). */
+ async function list(q, { creator, supporter, includeTest = true, cursor, limit = 50 } = {}) {
const n = Math.min(200, Math.max(1, Number(limit) || 50));
const n = Math.min(200, Math.max(1, Number(limit) || 50));
- let cur = null;
- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }
- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE (@c IS NULL OR creator_subject = @c) AND (@s IS NULL OR supporter_subject = @s)
- AND (@t = 1 OR test = 0) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))
- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator || null, s: supporter || null, t: includeTest ? 1 : 0, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });
+ const cur = cursor ? readCursor(cursor) : null;
+ const where = [];
+ if (creator) where.push(sql`creator_subject = ${creator}`);
+ if (supporter) where.push(sql`supporter_subject = ${supporter}`);
+ if (!where.length) fail(422, 'tips.invalid_input', 'creator or supporter is required');
+ if (!includeTest) where.push(sql`NOT test`);
+ if (cur) where.push(sql`(created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)`);
+ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE ${sql.join(where, sql` AND `)} ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);
const page = rows.slice(0, n);
const page = rows.slice(0, n);
- const next = rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null;
- return { rows: page, next_cursor: next };
+ return { rows: page, next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };
}
}
// ── The public supporters page ───────────────────────────
// ── The public supporters page ───────────────────────────
- // Only settled, non-test interactions of supporters who kept both their name and their amount
- // public count toward the leaderboard (a hidden amount must not be inferable from a rank).
- const leaderboardWhere = (t) => `${t}.creator_subject = @c AND ${t}.supporter_subject IS NOT NULL AND ${t}.test = 0
- AND ${t}.payment_state IN ('settled', 'reversed') AND ${t}.anonymous = 0 AND ${t}.hide_amount = 0 AND ${t}.erased_at IS NULL
- AND ${t}.moderation = 'visible'`;
-
- /** Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). */
- function leaderboard(creator, { limit = 20 } = {}) {
- const rows = db.prepare(`SELECT t.supporter_subject AS s, SUM(t.amount - t.reversed_bits) AS total, COUNT(*) AS tips, MIN(t.created_at) AS first_at,
- (SELECT j.supporter_name FROM tip_interactions j WHERE ${leaderboardWhere('j')} AND j.supporter_subject = t.supporter_subject
- ORDER BY j.created_at DESC LIMIT 1) AS name
- FROM tip_interactions t WHERE ${leaderboardWhere('t')}
- GROUP BY t.supporter_subject HAVING total > 0 ORDER BY total DESC, first_at LIMIT @n`).all({ c: creator, n: Math.min(100, Math.max(1, limit)) });
- const settings = ctx.profiles.filterOf(creator);
- return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, settings), total: r.total, tips: r.tips }));
+ /**
+ * Top supporters: [{ rank, name, total, tips }] (name = the latest name they tipped under). Only
+ * settled, non-test interactions of supporters who kept both their name and their amount public
+ * count (a hidden amount must not be inferable from a rank).
+ */
+ async function leaderboard(q, profile, { limit = 20 } = {}) {
+ const rows = await q.many(sql`SELECT supporter_subject AS s, SUM(amount - reversed_bits)::bigint AS total, count(*) AS tips, MIN(created_at) AS first_at,
+ (array_agg(supporter_name ORDER BY created_at DESC))[1] AS name
+ FROM tip_interactions
+ WHERE creator_subject = ${profile.creator_subject} AND supporter_subject IS NOT NULL AND NOT test AND payment_state IN ('settled', 'reversed')
+ AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible'
+ GROUP BY supporter_subject HAVING SUM(amount - reversed_bits) > 0
+ ORDER BY total DESC, first_at LIMIT ${Math.min(100, Math.max(1, limit))}`);
+ return rows.map((r, k) => ({ rank: k + 1, name: shownName({ supporter_name: r.name }, profile.filter), total: r.total, tips: r.tips }));
}
}
/** Recent settled interactions with a public message, as publicView(). */
/** Recent settled interactions with a public message, as publicView(). */
- function recentPublic(creator, { limit = 20 } = {}) {
- return db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND payment_state = 'settled' AND private_message = 0
- AND erased_at IS NULL AND moderation = 'visible' AND message IS NOT NULL ORDER BY settled_at DESC, id DESC LIMIT ?`).all(creator, Math.min(100, Math.max(1, limit)))
- .map((i) => ctx.view(i));
+ async function recentPublic(q, profile, { limit = 20 } = {}) {
+ return (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${profile.creator_subject} AND payment_state = 'settled' AND message IS NOT NULL
+ AND NOT test AND NOT private_message AND erased_at IS NULL AND moderation = 'visible' ORDER BY settled_at DESC, id DESC LIMIT ${Math.min(100, Math.max(1, limit))}`))
+ .map((i) => publicView(i, { filter: profile.filter }));
}
}
// ── A supporter's own data ───────────────────────────────
// ── A supporter's own data ───────────────────────────────
- /** Everything Tips holds about a supporter's tips (their own view), for a download. */
- function exportFor(subject) {
- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ? ORDER BY created_at').all(subject);
+ /** Everything Tips holds about a supporter's tips (their own view), for a download: five queries. */
+ async function exportFor(subject) {
+ const rows = await db.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY created_at`);
+ const ids = rows.map((i) => i.id);
+ const views = await presentMany(db, rows, { viewer: 'supporter' });
+ const group = (list) => { const m = new Map(); for (const { interaction_id: id, ...r } of list) { if (!m.has(id)) m.set(id, []); m.get(id).push(r); } return m; };
+ const [contributions, paid, media] = ids.length ? await Promise.all([
+ db.many(sql`SELECT interaction_id, goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ANY(${ids}) ORDER BY id`),
+ db.many(sql`SELECT interaction_id, kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ANY(${ids})`),
+ db.many(sql`SELECT interaction_id, url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ANY(${ids})`),
+ ]).then((r) => r.map(group)) : [new Map(), new Map(), new Map()];
return {
return {
service: 'tips', subject: { type: 'user', id: subject }, exported_at: iso(ctx.now()),
service: 'tips', subject: { type: 'user', id: subject }, exported_at: iso(ctx.now()),
- interactions: rows.map((i) => ({
- ...present(i, { viewer: 'supporter' }),
- goal_contributions: db.prepare('SELECT goal_id, amount, reversed_amount, created_at FROM tip_goal_contributions WHERE interaction_id = ?').all(i.id),
- paid_message: db.prepare('SELECT kind, text, voice, highlight_seconds, status, created_at, updated_at FROM paid_messages WHERE interaction_id = ?').get(i.id) || null,
- media_request: db.prepare('SELECT url, provider, status, created_at, updated_at FROM paid_media_requests WHERE interaction_id = ?').get(i.id) || null,
+ interactions: views.map((v) => ({
+ ...v,
+ goal_contributions: contributions.get(v.id) || [],
+ paid_message: (paid.get(v.id) || [])[0] || null,
+ media_request: (media.get(v.id) || [])[0] || null,
})),
})),
not_included: 'Payments, balances and refunds are OpenVibe.Billing\'s records; ask Billing for them.',
not_included: 'Payments, balances and refunds are OpenVibe.Billing\'s records; ask Billing for them.',
};
};
@@ -649,74 +690,67 @@ function createInteractions(ctx) {
@@ -649,74 +690,67 @@ function createInteractions(ctx) {
* answers, overlay payloads and the unsent events that named them. Each erased interaction emits
* answers, overlay payloads and the unsent events that named them. Each erased interaction emits
* tips.interaction.erased, whose `redacts` has OpenVibe.Events tombstone the earlier events about
* tips.interaction.erased, whose `redacts` has OpenVibe.Events tombstone the earlier events about
* it. Interactions still waiting for their payment are kept (the payment needs the supporter) and
* it. Interactions still waiting for their payment are kept (the payment needs the supporter) and
- * counted in kept_pending.
+ * counted in kept_pending. The supporter's rows are locked in id order first; the writes are one
+ * statement per table.
*/
*/
function erase(subject) {
function erase(subject) {
- return ctx.tx(() => {
- const rows = db.prepare('SELECT * FROM tip_interactions WHERE supporter_subject = ?').all(subject);
+ return ctx.tx(async (t) => {
+ const rows = await t.many(sql`SELECT * FROM tip_interactions WHERE supporter_subject = ${subject} ORDER BY id FOR UPDATE`);
const at = iso(ctx.now());
const at = iso(ctx.now());
- let erased = 0;
- for (const i of rows) {
- if (i.payment_state === 'pending') continue;
- const req = json(i.request, {});
- const keep = {};
- if (req.goal_id) keep.goal_id = req.goal_id;
- if (req.highlight_seconds) keep.highlight_seconds = req.highlight_seconds;
- db.prepare(`UPDATE tip_interactions SET supporter_subject = NULL, supporter_name = NULL, anonymous = 1, message = NULL, request = ?,
- idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ?, updated_at = ? WHERE id = ?`)
- .run(JSON.stringify(keep), at, at, i.id);
- db.prepare("UPDATE paid_messages SET text = '', voice = NULL, updated_at = ? WHERE interaction_id = ?").run(at, i.id);
- db.prepare("UPDATE paid_media_requests SET url = '', updated_at = ? WHERE interaction_id = ?").run(at, i.id);
- const now = get(i.id);
- ctx.overlays.refreshInteraction(now);
- scrubOutbox(now);
- if (!i.test) {
- ctx.outbox.emit('tips.interaction.erased', { type: 'interaction', id: i.id }, {
+ const done = rows.filter((i) => i.payment_state !== 'pending');
+ if (done.length) {
+ const keep = done.map((i) => {
+ const req = i.request || {};
+ const k = {};
+ if (req.goal_id) k.goal_id = req.goal_id;
+ if (req.highlight_seconds) k.highlight_seconds = req.highlight_seconds;
+ return { id: i.id, request: k };
+ });
+ const ids = done.map((i) => i.id);
+ const erased = await t.many(sql`UPDATE tip_interactions x SET supporter_subject = NULL, supporter_name = NULL, anonymous = true, message = NULL,
+ request = v.request, idempotency_key = NULL, checkout_url = NULL, checkout_ref = NULL, erased_at = ${at}, updated_at = ${at}
+ FROM jsonb_to_recordset(${sql.json(keep)}) AS v(id text, request jsonb) WHERE x.id = v.id RETURNING x.*`);
+ await t.exec(sql`UPDATE paid_messages SET text = '', voice = NULL, updated_at = ${at} WHERE interaction_id = ANY(${ids})`);
+ await t.exec(sql`UPDATE paid_media_requests SET url = '', updated_at = ${at} WHERE interaction_id = ANY(${ids})`);
+ await ctx.overlays.refreshInteractions(t, erased, await ctx.profiles.filtersOf(t, erased.map((i) => i.creator_subject)));
+ // Events about them still in the local outbox lose the supporter's name.
+ await t.exec(sql`UPDATE tips_event_outbox
+ SET envelope = jsonb_set(jsonb_set(envelope, '{payload,supporter}', 'null'::jsonb), '{payload,supporter_name}', ${sql.json(ANONYMOUS)})
+ WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY(${ids}) AND envelope->'payload' ? 'supporter'`);
+ for (const i of done) {
+ if (i.test) continue;
+ await ctx.outbox.emit(t, 'tips.interaction.erased', { type: 'interaction', id: i.id }, {
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, erased_at: at,
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, erased_at: at,
redacts: { subject_type: 'interaction', subject_ids: [i.id] },
redacts: { subject_type: 'interaction', subject_ids: [i.id] },
});
});
}
}
- erased++;
+ t.after(ctx.outboxKick);
}
}
// Stored API answers (Idempotency-Key replays) carry messages and the subject.
// Stored API answers (Idempotency-Key replays) carry messages and the subject.
- const prefix = `${subject}:`;
- const answers = db.prepare('DELETE FROM api_idempotency WHERE substr(key, 1, ?) = ? OR instr(response, ?) > 0').run(prefix.length, prefix, subject).changes;
- if (erased) ctx.afterCommit(() => ctx.outboxKick());
- return { erased, kept_pending: rows.length - erased, stored_answers_removed: answers };
+ const answers = await t.exec(sql`DELETE FROM api_idempotency WHERE starts_with(key, ${`${subject}:`}) OR strpos(response, ${subject}) > 0`);
+ return { erased: done.length, kept_pending: rows.length - done.length, stored_answers_removed: answers };
});
});
}
}
- /** Events about an erased interaction that are still in the local outbox lose the supporter's name. */
- function scrubOutbox(i) {
- const rows = db.prepare("SELECT id, envelope FROM event_outbox WHERE json_extract(envelope, '$.subject.type') = 'interaction' AND json_extract(envelope, '$.subject.id') = ?").all(i.id);
- for (const r of rows) {
- const env = JSON.parse(r.envelope);
- if (!env.payload || !('supporter' in env.payload)) continue;
- env.payload.supporter = null;
- env.payload.supporter_name = publicName(i);
- db.prepare('UPDATE event_outbox SET envelope = ? WHERE id = ?').run(JSON.stringify(env), r.id);
- }
- }
-
/**
/**
* A creator's totals, derived from settled interactions. `billing` is what must equal Billing's
* A creator's totals, derived from settled interactions. `billing` is what must equal Billing's
* books (settled through Billing or imported from Live's ledger, minus what Billing took back);
* books (settled through Billing or imported from Live's ledger, minus what Billing took back);
* external and test interactions are reported apart and never mixed in.
* external and test interactions are reported apart and never mixed in.
*/
*/
- function totals(creator) {
- const r = db.prepare(`SELECT
- COALESCE(SUM(CASE WHEN test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') THEN amount - reversed_bits END), 0) AS billing,
- COALESCE(SUM(CASE WHEN test = 0 AND settlement = 'external' AND payment_state = 'settled' THEN amount END), 0) AS external,
- COALESCE(SUM(CASE WHEN test = 0 AND payment_state IN ('settled', 'reversed') THEN 1 END), 0) AS count,
- COALESCE(SUM(CASE WHEN test = 1 THEN 1 END), 0) AS test_count,
- COALESCE(SUM(CASE WHEN payment_state = 'pending' THEN 1 END), 0) AS pending_count
- FROM tip_interactions WHERE creator_subject = ?`).get(creator);
+ async function totals(q, creator) {
+ const r = await q.one(sql`SELECT
+ COALESCE(SUM(amount - reversed_bits) FILTER (WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed')), 0)::bigint AS billing,
+ COALESCE(SUM(amount) FILTER (WHERE NOT test AND settlement = 'external' AND payment_state = 'settled'), 0)::bigint AS external,
+ count(*) FILTER (WHERE NOT test AND payment_state IN ('settled', 'reversed')) AS count,
+ count(*) FILTER (WHERE test) AS test_count,
+ count(*) FILTER (WHERE payment_state = 'pending') AS pending_count
+ FROM tip_interactions WHERE creator_subject = ${creator}`);
return { currency: 'vibes-bits', settled_via_billing: r.billing, external: r.external, interactions: r.count, simulations: r.test_count, pending: r.pending_count };
return { currency: 'vibes-bits', settled_via_billing: r.billing, external: r.external, interactions: r.count, simulations: r.test_count, pending: r.pending_count };
}
}
return {
return {
- get, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,
- onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, list, totals, insert, KINDS,
+ get, lock, byBillingTxn, validate, request, transfer, startCheckout, processDueTransfers, settleByTransaction, settle, failPayment,
+ onBillingSettled, onBillingReversed, onBillingExternal, cancelQueued, recomputeDelivery, recordExternal, simulate, summary, present, presentMany, list, totals, insert, KINDS,
leaderboard, recentPublic, exportFor, erase, release,
leaderboard, recentPublic, exportFor, erase, release,
};
};
}
}
@@ -18,9 +18,15 @@
@@ -18,9 +18,15 @@
* tips.interaction.moderate. Every outcome is recorded in tip_moderation_log and published as
* tips.interaction.moderate. Every outcome is recorded in tip_moderation_log and published as
* tips.interaction.moderated (never for simulations): { interaction_id, creator, action, by,
* tips.interaction.moderated (never for simulations): { interaction_id, creator, action, by,
* moderation_state, cancelled_effects }. The payload names no supporter, message or moderator.
* moderation_state, cancelled_effects }. The payload names no supporter, message or moderator.
+ *
+ * hide() and restore() cancel or release what a payment bought (paid messages, TTS, media requests), so
+ * they run SERIALIZABLE and lock the interaction row first (the domain's lock order): two moderators
+ * acting at once apply one after the other.
*/
*/
const crypto = require('crypto');
const crypto = require('crypto');
-const { fail, iso, prefixedId, sha256, userSubject, displayName, text } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { fail, iso, prefixedId, sha256, userSubject, displayName, text, readCursor, cursorOf } = require('../util');
+const { publicView } = require('./privacy');
const filter = require('./filter');
const filter = require('./filter');
const INVITE_RE = /^tmin_[A-Za-z0-9_-]{43}$/;
const INVITE_RE = /^tmin_[A-Za-z0-9_-]{43}$/;
@@ -32,139 +38,150 @@ function createModeration(ctx) {
@@ -32,139 +38,150 @@ function createModeration(ctx) {
const { db } = ctx;
const { db } = ctx;
// ── Moderators ───────────────────────────────────────────
// ── Moderators ───────────────────────────────────────────
- const isModerator = (creator, subject) => !!(subject && db.prepare('SELECT 1 FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').get(creator, subject));
- const listModerators = (creator) => db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL ORDER BY created_at').all(creator);
+ const isModerator = async (q, creator, subject) => !!(subject && await q.maybe(sql`SELECT 1 FROM tip_moderators
+ WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`));
+ const listModerators = (q, creator) => q.many(sql`SELECT * FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL ORDER BY created_at`);
/** Creators whose interactions this person moderates (with their handles). */
/** Creators whose interactions this person moderates (with their handles). */
- const moderatedBy = (subject) => db.prepare(`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m
- JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ? AND m.removed_at IS NULL ORDER BY p.handle`).all(subject);
+ const moderatedBy = (q, subject) => q.many(sql`SELECT p.creator_subject, p.handle, p.display_name FROM tip_moderators m
+ JOIN creator_tip_profiles p ON p.creator_subject = m.creator_subject WHERE m.moderator_subject = ${subject} AND m.removed_at IS NULL ORDER BY p.handle`);
- function addModerator(creator, moderator, { name, addedBy }) {
+ /** In q's transaction, or a new one: the creator's row is locked while the limit is counted. */
+ function addModerator(q, creator, moderator, { name, addedBy }) {
const subject = userSubject(moderator, 'moderator');
const subject = userSubject(moderator, 'moderator');
if (subject === creator) fail(422, 'tips.invalid_input', 'the creator moderates their own page already');
if (subject === creator) fail(422, 'tips.invalid_input', 'the creator moderates their own page already');
- const count = db.prepare('SELECT COUNT(*) AS n FROM tip_moderators WHERE creator_subject = ? AND removed_at IS NULL').get(creator).n;
- if (!isModerator(creator, subject) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);
- db.prepare(`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at) VALUES (?, ?, ?, ?, ?)
- ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, name), added_by = excluded.added_by`)
- .run(creator, subject, displayName(name), addedBy, iso(ctx.now()));
- return presentModerator(db.prepare('SELECT * FROM tip_moderators WHERE creator_subject = ? AND moderator_subject = ?').get(creator, subject));
+ return ctx.within(q, async (t) => {
+ await ctx.lockCreator(t, creator);
+ const count = await t.value(sql`SELECT count(*) FROM tip_moderators WHERE creator_subject = ${creator} AND removed_at IS NULL`);
+ if (!(await isModerator(t, creator, subject)) && count >= MAX_MODERATORS) fail(409, 'tips.too_many_moderators', `remove a moderator first (${MAX_MODERATORS} at most)`);
+ return presentModerator(await t.one(sql`INSERT INTO tip_moderators (creator_subject, moderator_subject, name, added_by, created_at)
+ VALUES (${creator}, ${subject}, ${displayName(name)}, ${addedBy}, ${iso(ctx.now())})
+ ON CONFLICT (creator_subject, moderator_subject) DO UPDATE SET removed_at = NULL, name = COALESCE(excluded.name, tip_moderators.name), added_by = excluded.added_by
+ RETURNING *`));
+ });
}
}
- function removeModerator(creator, moderator) {
+ async function removeModerator(creator, moderator) {
const subject = userSubject(moderator, 'moderator');
const subject = userSubject(moderator, 'moderator');
- const r = db.prepare('UPDATE tip_moderators SET removed_at = ? WHERE creator_subject = ? AND moderator_subject = ? AND removed_at IS NULL').run(iso(ctx.now()), creator, subject);
- return { removed: r.changes > 0 };
+ const n = await db.exec(sql`UPDATE tip_moderators SET removed_at = ${iso(ctx.now())} WHERE creator_subject = ${creator} AND moderator_subject = ${subject} AND removed_at IS NULL`);
+ return { removed: n > 0 };
}
}
const presentModerator = (m) => (m ? { moderator: { type: 'user', id: m.moderator_subject }, name: m.name || null, added_at: m.created_at } : null);
const presentModerator = (m) => (m ? { moderator: { type: 'user', id: m.moderator_subject }, name: m.name || null, added_at: m.created_at } : null);
// ── Invitations (a show-once link the creator hands to a moderator) ──
// ── Invitations (a show-once link the creator hands to a moderator) ──
- function createInvite(creator, { createdBy }) {
- const open = db.prepare('SELECT COUNT(*) AS n FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ?').get(creator, iso(ctx.now())).n;
- if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');
+ async function createInvite(creator, { createdBy }) {
const secret = `tmin_${crypto.randomBytes(32).toString('base64url')}`;
const secret = `tmin_${crypto.randomBytes(32).toString('base64url')}`;
const id = prefixedId('tmin', ctx.now());
const id = prefixedId('tmin', ctx.now());
const at = ctx.now();
const at = ctx.now();
- db.prepare('INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at) VALUES (?, ?, ?, ?, ?, ?)')
- .run(id, creator, sha256(secret), createdBy, iso(at), iso(at + INVITE_TTL_MS));
+ await ctx.tx(async (t) => {
+ await ctx.lockCreator(t, creator);
+ const open = await t.value(sql`SELECT count(*) FROM tip_moderator_invites WHERE creator_subject = ${creator} AND used_at IS NULL AND revoked_at IS NULL
+ AND expires_at > ${iso(at)}`);
+ if (open >= 10) fail(409, 'tips.too_many_invites', 'revoke an open invitation first (10 at most)');
+ await t.exec(sql`INSERT INTO tip_moderator_invites (id, creator_subject, token_hash, created_by, created_at, expires_at)
+ VALUES (${id}, ${creator}, ${sha256(secret)}, ${createdBy}, ${iso(at)}, ${iso(at + INVITE_TTL_MS)})`);
+ });
return { id, url: `${ctx.config.baseUrl}/moderate/invite/${secret}`, expires_at: iso(at + INVITE_TTL_MS) };
return { id, url: `${ctx.config.baseUrl}/moderate/invite/${secret}`, expires_at: iso(at + INVITE_TTL_MS) };
}
}
/** The open invitation for a presented secret, or null (unknown, used, revoked or expired). */
/** The open invitation for a presented secret, or null (unknown, used, revoked or expired). */
- function findInvite(secret) {
+ async function findInvite(q, secret) {
if (!INVITE_RE.test(String(secret || ''))) return null;
if (!INVITE_RE.test(String(secret || ''))) return null;
- const row = db.prepare('SELECT * FROM tip_moderator_invites WHERE token_hash = ?').get(sha256(secret));
- if (!row || row.used_at || row.revoked_at || row.expires_at <= iso(ctx.now())) return null;
+ const row = await q.maybe(sql`SELECT * FROM tip_moderator_invites WHERE token_hash = ${sha256(secret)}`);
+ if (!row || row.used_at || row.revoked_at || Date.parse(row.expires_at) <= ctx.now()) return null;
return row;
return row;
}
}
+ /** Accept once: the invitation is taken by a conditional update, so two people at once cannot both use it. */
function acceptInvite(secret, { subject, name }) {
function acceptInvite(secret, { subject, name }) {
- return ctx.tx(() => {
- const inv = findInvite(secret);
+ return ctx.tx(async (t) => {
+ const inv = await findInvite(t, secret);
if (!inv) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');
if (!inv) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');
if (inv.creator_subject === subject) fail(422, 'tips.invalid_input', 'this is your own page');
if (inv.creator_subject === subject) fail(422, 'tips.invalid_input', 'this is your own page');
- db.prepare('UPDATE tip_moderator_invites SET used_at = ?, used_by = ? WHERE id = ? AND used_at IS NULL').run(iso(ctx.now()), subject, inv.id);
- return { creator: inv.creator_subject, moderator: addModerator(inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };
+ const n = await t.exec(sql`UPDATE tip_moderator_invites SET used_at = ${iso(ctx.now())}, used_by = ${subject} WHERE id = ${inv.id} AND used_at IS NULL AND revoked_at IS NULL`);
+ if (!n) fail(404, 'tips.invite_not_found', 'this invitation is not valid (it may have been used, revoked or expired)');
+ return { creator: inv.creator_subject, moderator: await addModerator(t, inv.creator_subject, subject, { name, addedBy: `invite:${inv.id}` }) };
});
});
}
}
- const openInvites = (creator) => db.prepare('SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ? ORDER BY created_at').all(creator, iso(ctx.now()));
- function revokeInvite(creator, id) {
- return { revoked: db.prepare('UPDATE tip_moderator_invites SET revoked_at = ? WHERE id = ? AND creator_subject = ? AND used_at IS NULL AND revoked_at IS NULL').run(iso(ctx.now()), String(id || ''), creator).changes > 0 };
+ const openInvites = (q, creator) => q.many(sql`SELECT id, created_at, expires_at FROM tip_moderator_invites WHERE creator_subject = ${creator}
+ AND used_at IS NULL AND revoked_at IS NULL AND expires_at > ${iso(ctx.now())} ORDER BY created_at`);
+ async function revokeInvite(creator, id) {
+ const n = await db.exec(sql`UPDATE tip_moderator_invites SET revoked_at = ${iso(ctx.now())} WHERE id = ${String(id || '')} AND creator_subject = ${creator}
+ AND used_at IS NULL AND revoked_at IS NULL`);
+ return { revoked: n > 0 };
}
}
// ── The filter at settlement ─────────────────────────────
// ── The filter at settlement ─────────────────────────────
/**
/**
- * Inside the settlement transaction, before anything is shown: does the creator's filter match the
- * name or what will be shown or read? Returns { hit, hold }.
+ * Before anything is shown or read: does the creator's filter (`settings`, profiles.filterOf) match
+ * the name or what will be shown or read? Returns { hit, hold }.
*/
*/
- function screen(i) {
- const settings = ctx.profiles.filterOf(i.creator_subject);
+ function screen(i, settings) {
if (!settings || !settings.words.length) return { hit: false, hold: false };
if (!settings || !settings.words.length) return { hit: false, hold: false };
- const req = JSON.parse(i.request || '{}');
+ const req = i.request || {};
const texts = [i.anonymous ? null : i.supporter_name, i.private_message ? null : i.message, req.tts && req.tts.text];
const texts = [i.anonymous ? null : i.supporter_name, i.private_message ? null : i.message, req.tts && req.tts.text];
const hit = filter.hits(settings, ...texts);
const hit = filter.hits(settings, ...texts);
return { hit, hold: hit && settings.action === 'hold' };
return { hit, hold: hit && settings.action === 'hold' };
}
}
// ── Outcomes ─────────────────────────────────────────────
// ── Outcomes ─────────────────────────────────────────────
- function record(i, action, { role, actor = null, reason = null, cancelled = [] }) {
- db.prepare('INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at) VALUES (?, ?, ?, ?, ?, ?, ?)')
- .run(i.creator_subject, i.id, action, role, actor, reason, iso(ctx.now()));
+ async function record(t, i, action, { role, actor = null, reason = null, cancelled = [] }) {
+ await t.exec(sql`INSERT INTO tip_moderation_log (creator_subject, interaction_id, action, by_role, actor, reason, created_at)
+ VALUES (${i.creator_subject}, ${i.id}, ${action}, ${role}, ${actor}, ${reason}, ${iso(ctx.now())})`);
if (!i.test) {
if (!i.test) {
- const now = ctx.interactions.get(i.id);
- ctx.outbox.emit('tips.interaction.moderated', { type: 'interaction', id: i.id }, {
+ await ctx.outbox.emit(t, 'tips.interaction.moderated', { type: 'interaction', id: i.id }, {
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, action, by: role,
interaction_id: i.id, creator: { type: 'user', id: i.creator_subject }, action, by: role,
- moderation_state: now.moderation, cancelled_effects: cancelled,
+ moderation_state: i.moderation, cancelled_effects: cancelled,
});
});
- ctx.afterCommit(() => ctx.outboxKick());
+ ctx.afterCommit(t, () => ctx.outboxKick());
}
}
}
}
/**
/**
* Settlement found a filter match (inside its transaction). markScreened() sets the state before
* Settlement found a filter match (inside its transaction). markScreened() sets the state before
- * anything is shown (held, or only filtered = masked); recordScreened() logs and publishes it once
- * the settlement's own event is out.
+ * anything is shown (held, or only filtered = masked) and returns the row; recordScreened() logs and
+ * publishes it once the settlement's own event is out.
*/
*/
- function markScreened(i, verdict) {
+ function markScreened(t, i, verdict) {
const at = iso(ctx.now());
const at = iso(ctx.now());
- if (verdict.hold) db.prepare("UPDATE tip_interactions SET filtered = 1, moderation = 'held', moderated_at = ?, moderated_by = 'filter', updated_at = ? WHERE id = ?").run(at, at, i.id);
- else db.prepare('UPDATE tip_interactions SET filtered = 1, updated_at = ? WHERE id = ?').run(at, i.id);
+ return verdict.hold
+ ? t.one(sql`UPDATE tip_interactions SET filtered = true, moderation = 'held', moderated_at = ${at}, moderated_by = 'filter', updated_at = ${at} WHERE id = ${i.id} RETURNING *`)
+ : t.one(sql`UPDATE tip_interactions SET filtered = true, updated_at = ${at} WHERE id = ${i.id} RETURNING *`);
}
}
- function recordScreened(i, verdict) {
- record(i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });
+ function recordScreened(t, i, verdict) {
+ return record(t, i, verdict.hold ? 'held' : 'filtered', { role: 'filter', reason: 'blocked_words' });
}
}
- /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else 403. */
- function actorFor(principal, creator, granted) {
+ /** Who is acting: { role, actor } for the creator, a moderator or a granted service; else null. */
+ async function actorFor(q, principal, creator, granted) {
if (principal.kind === 'service' && granted) return { role: 'service', actor: principal.sub };
if (principal.kind === 'service' && granted) return { role: 'service', actor: principal.sub };
if (principal.kind === 'user' && principal.subject === creator) return { role: 'creator', actor: principal.subject };
if (principal.kind === 'user' && principal.subject === creator) return { role: 'creator', actor: principal.subject };
- if (principal.kind === 'user' && isModerator(creator, principal.subject)) return { role: 'moderator', actor: principal.subject };
+ if (principal.kind === 'user' && await isModerator(q, creator, principal.subject)) return { role: 'moderator', actor: principal.subject };
return null;
return null;
}
}
/** Hide from overlays, chat/TTS still to come and public pages. The money is untouched. */
/** Hide from overlays, chat/TTS still to come and public pages. The money is untouched. */
function hide(i, { role, actor, reason }) {
function hide(i, { role, actor, reason }) {
const note = text(reason, 'reason', 200);
const note = text(reason, 'reason', 200);
- return ctx.tx(() => {
- const cur = ctx.interactions.get(i.id);
+ return ctx.tx(async (t) => {
+ const cur = await ctx.interactions.lock(t, i.id);
if (cur.moderation === 'hidden') return { interaction: cur, changed: false };
if (cur.moderation === 'hidden') return { interaction: cur, changed: false };
const at = iso(ctx.now());
const at = iso(ctx.now());
- const cancelled = db.prepare(`SELECT effect FROM interaction_effects WHERE interaction_id = ? AND state = 'queued' AND effect IN (${CHAT_EFFECTS.map(() => '?').join(',')})`)
- .all(cur.id, ...CHAT_EFFECTS).map((e) => e.effect);
+ const cancelled = (await t.many(sql`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ${at}
+ WHERE interaction_id = ${cur.id} AND state = 'queued' RETURNING effect`)).map((e) => e.effect).filter((e) => CHAT_EFFECTS.includes(e));
if (cancelled.length) {
if (cancelled.length) {
- db.prepare(`UPDATE interaction_effects SET state = 'cancelled', last_error = 'hidden by moderation', updated_at = ? WHERE interaction_id = ? AND state = 'queued'`).run(at, cur.id);
- db.prepare("UPDATE paid_messages SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);
- db.prepare("UPDATE paid_media_requests SET status = 'cancelled', updated_at = ? WHERE interaction_id = ? AND status = 'queued'").run(at, cur.id);
+ await t.exec(sql`UPDATE paid_messages SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);
+ await t.exec(sql`UPDATE paid_media_requests SET status = 'cancelled', updated_at = ${at} WHERE interaction_id = ${cur.id} AND status = 'queued'`);
}
}
const undelivered = cancelled.length > 0 || cur.moderation === 'held' || cur.payment_state === 'pending';
const undelivered = cancelled.length > 0 || cur.moderation === 'held' || cur.payment_state === 'pending';
- db.prepare(`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ?, moderated_by = ?,
- delivery_state = CASE WHEN ? AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END, updated_at = ? WHERE id = ?`)
- .run(at, actor || role, undelivered ? 1 : 0, at, cur.id);
- const now = ctx.interactions.get(cur.id);
- ctx.overlays.retract(now);
- record(now, 'hidden', { role, actor, reason: note, cancelled });
- return { interaction: ctx.interactions.get(cur.id), changed: true, cancelled_effects: cancelled };
- });
+ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'hidden', moderated_at = ${at}, moderated_by = ${actor || role},
+ delivery_state = CASE WHEN ${undelivered} AND delivery_state IN ('queued', 'awaiting_payment') THEN 'cancelled' ELSE delivery_state END,
+ updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);
+ await ctx.overlays.retract(t, now, await ctx.profiles.filterOf(t, now.creator_subject));
+ await record(t, now, 'hidden', { role, actor, reason: note, cancelled });
+ return { interaction: now, changed: true, cancelled_effects: cancelled };
+ }, ctx.MONEY);
}
}
/**
/**
@@ -174,26 +191,26 @@ function createModeration(ctx) {
@@ -174,26 +191,26 @@ function createModeration(ctx) {
*/
*/
function restore(i, { role, actor, reason }) {
function restore(i, { role, actor, reason }) {
const note = text(reason, 'reason', 200);
const note = text(reason, 'reason', 200);
- return ctx.tx(() => {
- const cur = ctx.interactions.get(i.id);
+ return ctx.tx(async (t) => {
+ const cur = await ctx.interactions.lock(t, i.id);
if (cur.moderation === 'visible') return { interaction: cur, changed: false };
if (cur.moderation === 'visible') return { interaction: cur, changed: false };
const at = iso(ctx.now());
const at = iso(ctx.now());
// Never shown (held, or hidden before it was): released now, its delivery starting over.
// Never shown (held, or hidden before it was): released now, its delivery starting over.
- const neverShown = !db.prepare("SELECT 1 FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").get(cur.id);
+ const neverShown = !(await t.maybe(sql`SELECT 1 FROM overlay_deliveries WHERE interaction_id = ${cur.id} AND kind = 'alert'`));
const release = cur.payment_state === 'settled' && cur.origin !== 'import' && neverShown;
const release = cur.payment_state === 'settled' && cur.origin !== 'import' && neverShown;
- db.prepare(`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ?, moderated_by = ?,
- delivery_state = CASE WHEN ? THEN 'queued' ELSE delivery_state END, updated_at = ? WHERE id = ?`).run(at, actor || role, release ? 1 : 0, at, cur.id);
- const now = ctx.interactions.get(cur.id);
- ctx.overlays.unretract(now);
- if (release) ctx.interactions.release(now);
- record(now, 'restored', { role, actor, reason: note });
- return { interaction: ctx.interactions.get(cur.id), changed: true };
- });
+ const now = await t.one(sql`UPDATE tip_interactions SET moderation = 'visible', moderated_at = ${at}, moderated_by = ${actor || role},
+ delivery_state = CASE WHEN ${release} THEN 'queued' ELSE delivery_state END, updated_at = ${at} WHERE id = ${cur.id} RETURNING *`);
+ const settings = await ctx.profiles.filterOf(t, now.creator_subject);
+ await ctx.overlays.unretract(t, now, settings);
+ if (release) await ctx.interactions.release(t, now, settings);
+ await record(t, now, 'restored', { role, actor, reason: note });
+ return { interaction: await ctx.interactions.get(t, cur.id), changed: true };
+ }, ctx.MONEY);
}
}
/** The moderators' view: what was written (not private things), and what the public sees. */
/** The moderators' view: what was written (not private things), and what the public sees. */
- function present(i) {
- const req = JSON.parse(i.request || '{}');
+ function shape(i, settings) {
+ const req = i.request || {};
return {
return {
id: i.id, kind: i.kind, created_at: i.created_at, settled_at: i.settled_at || null, payment_state: i.payment_state, test: !!i.test,
id: i.id, kind: i.kind, created_at: i.created_at, settled_at: i.settled_at || null, payment_state: i.payment_state, test: !!i.test,
moderation: { state: i.moderation, at: i.moderated_at || null, filtered: !!i.filtered },
moderation: { state: i.moderation, at: i.moderated_at || null, filtered: !!i.filtered },
@@ -202,29 +219,34 @@ function createModeration(ctx) {
@@ -202,29 +219,34 @@ function createModeration(ctx) {
message: i.private_message || i.erased_at ? null : (i.message || null),
message: i.private_message || i.erased_at ? null : (i.message || null),
tts_text: req.tts && !i.erased_at ? req.tts.text : null,
tts_text: req.tts && !i.erased_at ? req.tts.text : null,
media_url: req.media && !i.erased_at ? req.media.url : null,
media_url: req.media && !i.erased_at ? req.media.url : null,
- public: ctx.view(i),
+ public: publicView(i, { filter: settings }),
};
};
}
}
+ const present = async (q, i) => shape(i, await ctx.profiles.filterOf(q, i.creator_subject));
- /** A creator's interactions for review, newest first. state: held | hidden | visible | all. */
- function queue(creator, { state = 'all', cursor, limit = 50 } = {}) {
+ /** A creator's interactions for review, newest first, keyset-paged. state: held | hidden | visible | all. */
+ async function queue(q, creator, { state = 'all', cursor, limit = 50 } = {}) {
if (!['held', 'hidden', 'visible', 'all'].includes(state)) fail(422, 'tips.invalid_input', 'state must be held, hidden, visible or all');
if (!['held', 'hidden', 'visible', 'all'].includes(state)) fail(422, 'tips.invalid_input', 'state must be held, hidden, visible or all');
const n = Math.min(200, Math.max(1, Number(limit) || 50));
const n = Math.min(200, Math.max(1, Number(limit) || 50));
- let cur = null;
- if (cursor) { try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { fail(422, 'tips.invalid_input', 'bad cursor'); } }
- const rows = db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = @c AND payment_state IN ('settled', 'reversed', 'pending')
- AND (@s = 'all' OR moderation = @s) AND (@ca IS NULL OR created_at < @ca OR (created_at = @ca AND id < @ci))
- ORDER BY created_at DESC, id DESC LIMIT @n`).all({ c: creator, s: state, ca: cur ? cur[0] : null, ci: cur ? cur[1] : null, n: n + 1 });
+ const cur = cursor ? readCursor(cursor) : null;
+ const rows = await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND payment_state IN ('settled', 'reversed', 'pending')
+ ${state === 'all' ? sql`` : sql`AND moderation = ${state}`} ${cur ? sql`AND (created_at, id) < (${cur[0]}::timestamptz, ${cur[1]}::text)` : sql``}
+ ORDER BY created_at DESC, id DESC LIMIT ${n + 1}`);
const page = rows.slice(0, n);
const page = rows.slice(0, n);
- return { rows: page.map(present), next_cursor: rows.length > n ? Buffer.from(JSON.stringify([page[n - 1].created_at, page[n - 1].id])).toString('base64url') : null };
+ const settings = await ctx.profiles.filterOf(q, creator);
+ return { rows: page.map((i) => shape(i, settings)), next_cursor: rows.length > n ? cursorOf(page[n - 1]) : null };
}
}
- const log = (creator, limit = 100) => db.prepare('SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log WHERE creator_subject = ? ORDER BY id DESC LIMIT ?').all(creator, limit);
+ const log = (q, creator, limit = 100) => q.many(sql`SELECT interaction_id, action, by_role, actor, reason, created_at FROM tip_moderation_log
+ WHERE creator_subject = ${creator} ORDER BY id DESC LIMIT ${limit}`);
+
+ /** Held paid messages waiting for review (the dashboard's count). */
+ const heldCount = (q, creator) => q.value(sql`SELECT count(*) FROM tip_interactions WHERE creator_subject = ${creator} AND moderation = 'held' AND payment_state = 'settled'`);
return {
return {
isModerator, listModerators, moderatedBy, addModerator, removeModerator, presentModerator,
isModerator, listModerators, moderatedBy, addModerator, removeModerator, presentModerator,
createInvite, findInvite, acceptInvite, openInvites, revokeInvite,
createInvite, findInvite, acceptInvite, openInvites, revokeInvite,
- screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log,
+ screen, markScreened, recordScreened, actorFor, hide, restore, present, queue, log, heldCount,
};
};
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,223 @@
@@ -0,0 +1,223 @@
+'use strict';
+
+/**
+ * The overlays' live side: open SSE streams, fan-out between processes, and the per-token stream cap.
+ *
+ * Streams are sockets of one process; what they must hear can happen in any process (a tip settled by
+ * another process, a moderator's hide, a config change, a revocation). Each process subscribes, per
+ * creator with an open stream here, to `overlay:<creator>` on openvibe-sdk/pubsub (Valkey, or in-process
+ * without it) and acts on what arrives:
+ *
+ * { t: 'new' } push the creator's new deliveries (read from the database)
+ * { t: 'retract', interaction_id, rows } tell alert streams to drop an alert a moderator hid
+ * { t: 'config', config } a changed overlay config, to the streams using it
+ * { t: 'revoke', token_id } close the token's streams at once
+ *
+ * Nothing authoritative travels here: a lost message costs a push that the next one (or a reconnect's
+ * replay from the database) makes up.
+ *
+ * Stream slots (TIPS_OVERLAY_MAX_STREAMS per token) are counted across processes in a Valkey sorted set
+ * per creator (member <token>|<stream>, score = expiry); a stream refreshes its slot on every heartbeat,
+ * so a dead process's slots lapse after three missed heartbeats. `connected(creator)` counts the live
+ * slots. Without Valkey, or when it fails, the slots are counted in this process.
+ */
+const crypto = require('crypto');
+const { createPubSub } = require('openvibe-sdk/pubsub');
+
+// Drop lapsed slots, count the token's, and take one when under the cap, atomically.
+// KEYS[1] the creator's set; ARGV: now, '<token>|', max, expiry, member, ttl (ms).
+const ACQUIRE = `
+redis.call('ZREMRANGEBYSCORE', KEYS[1], '-inf', ARGV[1])
+local n = 0
+for _, m in ipairs(redis.call('ZRANGE', KEYS[1], 0, -1)) do
+ if string.sub(m, 1, #ARGV[2]) == ARGV[2] then n = n + 1 end
+end
+if n >= tonumber(ARGV[3]) then return 0 end
+redis.call('ZADD', KEYS[1], ARGV[4], ARGV[5])
+redis.call('PEXPIRE', KEYS[1], ARGV[6])
+return 1`;
+
+function memorySlots(now) {
+ const byCreator = new Map(); // creator → Map(member → expiry)
+ const live = (creator) => {
+ const m = byCreator.get(creator);
+ if (!m) return new Map();
+ for (const [k, exp] of m) if (exp <= now()) m.delete(k);
+ if (!m.size) byCreator.delete(creator);
+ return m;
+ };
+ return {
+ async acquire(creator, tokenId, member, max, ttlMs) {
+ const m = live(creator);
+ if ([...m.keys()].filter((k) => k.startsWith(`${tokenId}|`)).length >= max) return false;
+ if (!byCreator.has(creator)) byCreator.set(creator, m);
+ m.set(member, now() + ttlMs);
+ return true;
+ },
+ async refresh(creator, member, ttlMs) { const m = byCreator.get(creator); if (m && m.has(member)) m.set(member, now() + ttlMs); },
+ async release(creator, member) { const m = byCreator.get(creator); if (m) { m.delete(member); if (!m.size) byCreator.delete(creator); } },
+ async count(creator) { return live(creator).size; },
+ };
+}
+
+function valkeySlots(valkey, now) {
+ const c = valkey.client;
+ if (typeof c.tipsOverlayAcquire !== 'function') c.defineCommand('tipsOverlayAcquire', { numberOfKeys: 1, lua: ACQUIRE });
+ const key = (creator) => valkey.key('overlay', 'streams', creator);
+ return {
+ async acquire(creator, tokenId, member, max, ttlMs) {
+ const t = now();
+ return (await c.tipsOverlayAcquire(key(creator), String(t), `${tokenId}|`, String(max), String(t + ttlMs), member, String(ttlMs))) === 1;
+ },
+ async refresh(creator, member, ttlMs) { await c.multi().zadd(key(creator), 'XX', now() + ttlMs, member).pexpire(key(creator), ttlMs).exec(); },
+ async release(creator, member) { await c.zrem(key(creator), member); },
+ async count(creator) { return c.zcount(key(creator), now(), '+inf'); },
+ };
+}
+
+/**
+ * @param {object} o
+ * @param {object|null} o.valkey openvibe-sdk/valkey handle, or null (in-process)
+ * @param {(client) => Promise<void>} o.pushNew read and write a client's new deliveries
+ * @param {number} o.heartbeatMs
+ */
+function createOverlayHub({ valkey = null, pushNew, heartbeatMs, now = () => Date.now(), log = console }) {
+ const ps = createPubSub({ valkey, log });
+ const local = memorySlots(now);
+ const shared = valkey ? valkeySlots(valkey, now) : null;
+ const ttlMs = heartbeatMs * 3;
+ let warned = false;
+ /** The shared slot store; on a Valkey error this call counts in the process instead. */
+ const slots = async (op, ...args) => {
+ if (shared) {
+ try { return await shared[op](...args); } catch (e) {
+ if (!warned) { warned = true; log.warn(`[Tips] overlay stream slots: Valkey failed (${e.message}); counting in this process`); }
+ }
+ }
+ return local[op](...args);
+ };
+ const creators = new Map(); // creator → { clients: Set, off: Promise<() => Promise> }
+ let nextClient = 1;
+
+ function write(client, event, id, data) {
+ if (client.closed) return false;
+ try {
+ client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
+ return true;
+ } catch { return false; }
+ }
+
+ /** Run fn for a client after whatever it is already doing (its pushes never interleave). */
+ function queue(client, fn) {
+ client.chain = client.chain.then(() => (client.closed ? null : fn())).catch((e) => log.warn('[Tips] overlay push:', e.message));
+ return client.chain;
+ }
+ /** New deliveries for this client: coalesced, so a burst of notifications reads once. */
+ function schedulePush(client) {
+ if (client.pushQueued) return;
+ client.pushQueued = true;
+ queue(client, () => { client.pushQueued = false; return pushNew(client); });
+ }
+
+ function onMessage(creator, msg) {
+ const entry = creators.get(creator);
+ if (!entry || !msg) return;
+ for (const c of [...entry.clients]) {
+ if (msg.t === 'new') schedulePush(c);
+ else if (msg.t === 'retract' && c.scopes.includes('alerts')) {
+ queue(c, () => { for (const r of msg.rows || []) write(c, 'retract', null, { interaction_id: msg.interaction_id, delivery_id: r.id, seq: r.seq }); });
+ } else if (msg.t === 'config' && msg.config && c.configId === msg.config.id) {
+ queue(c, () => { write(c, 'config', null, { config: msg.config }); });
+ } else if (msg.t === 'revoke' && c.tokenId === msg.token_id) {
+ write(c, 'revoked', null, { reason: 'this overlay token was revoked' });
+ detach(c);
+ }
+ }
+ }
+
+ // Through Valkey every process hears it, this one included. If publishing fails, this process's own
+ // streams still hear it (the others catch up from the database on their next push or reconnect).
+ const publish = (creator, msg) => ps.publish(`overlay:${creator}`, msg).then(() => undefined, (e) => {
+ log.warn('[Tips] overlay fan-out:', e.message);
+ onMessage(creator, JSON.parse(JSON.stringify(msg)));
+ });
+
+ /** Subscribe for a creator's messages; a failure is retried while the creator has streams here. */
+ function subscribe(creator, entry) {
+ entry.off = ps.subscribe(`overlay:${creator}`, (msg) => onMessage(creator, msg)).catch((e) => {
+ log.warn('[Tips] overlay subscribe:', e.message);
+ const retry = setTimeout(() => { if (creators.get(creator) === entry) subscribe(creator, entry); }, 5000);
+ if (retry.unref) retry.unref();
+ return async () => {};
+ });
+ }
+
+ /** A stream place for this token, or null past the cap (the route answers 429). */
+ async function reserve(token, max) {
+ const slot = { creator: token.creator_subject, member: `${token.id}|${crypto.randomBytes(8).toString('hex')}` };
+ return (await slots('acquire', slot.creator, token.id, slot.member, max, ttlMs)) ? slot : null;
+ }
+
+ /**
+ * Register a stream (its slot reserved) and run `start(client)` (hello, replay) before any push.
+ * The subscription is in place before start reads, so nothing committed meanwhile is missed.
+ */
+ function attach(token, res, slot, { minAmount = 0, configId = null }, start) {
+ const client = {
+ id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId,
+ minAmount, res, lastSeq: 0, closed: false, slot, chain: Promise.resolve(), pushQueued: false,
+ };
+ let entry = creators.get(client.creator);
+ if (!entry) {
+ entry = { clients: new Set(), off: null };
+ creators.set(client.creator, entry);
+ subscribe(client.creator, entry);
+ }
+ entry.clients.add(client);
+ res.on('close', () => detach(client));
+ if (res.destroyed) { detach(client); return client; } // gone before it was attached
+ client.beat = setInterval(() => {
+ if (!write(client, 'ping', null, { t: Date.now() })) { detach(client); return; }
+ slots('refresh', slot.creator, slot.member, ttlMs).catch(() => {});
+ }, heartbeatMs);
+ if (client.beat.unref) client.beat.unref();
+ queue(client, async () => { await entry.off; await start(client); });
+ schedulePush(client);
+ return client;
+ }
+
+ function detach(client) {
+ if (client.closed) return;
+ client.closed = true;
+ clearInterval(client.beat);
+ slots('release', client.slot.creator, client.slot.member).catch(() => {});
+ const entry = creators.get(client.creator);
+ if (entry) {
+ entry.clients.delete(client);
+ if (!entry.clients.size) {
+ creators.delete(client.creator);
+ entry.off.then((off) => off()).catch(() => {});
+ }
+ }
+ try { client.res.end(); } catch { /* already gone */ }
+ }
+
+ return {
+ write,
+ attach,
+ reserve,
+ /** Give a reserved slot back (the stream never started). */
+ release: (slot) => slots('release', slot.creator, slot.member).catch(() => {}),
+ /** Tell every process's streams of this creator: new deliveries are in the database. */
+ notify: (creator) => publish(creator, { t: 'new' }),
+ retract: (creator, interactionId, rows) => publish(creator, { t: 'retract', interaction_id: interactionId, rows }),
+ config: (creator, config) => publish(creator, { t: 'config', config }),
+ revoke: (creator, tokenId) => publish(creator, { t: 'revoke', token_id: tokenId }),
+ /** Open streams of this creator's overlays, in every process. */
+ connected: (creator) => slots('count', creator),
+ closeAll() { for (const entry of [...creators.values()]) for (const c of [...entry.clients]) detach(c); },
+ async close() { this.closeAll(); await ps.close(); },
+ };
+}
+
+module.exports = { createOverlayHub };
@@ -1,16 +1,17 @@
@@ -1,16 +1,17 @@
'use strict';
'use strict';
/**
/**
- * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE hub.
+ * Overlays: scoped, revocable tokens; configs; deliveries; the live SSE streams (overlay-hub.js).
*
*
* Tokens. `tovl_<43 base64url chars>` shown ONCE at creation; only its SHA-256 is stored. A token
* Tokens. `tovl_<43 base64url chars>` shown ONCE at creation; only its SHA-256 is stored. A token
* carries scopes (alerts, goals), optionally a config, and is never a creator cookie: an OBS
* carries scopes (alerts, goals), optionally a config, and is never a creator cookie: an OBS
* browser source holds it in its URL and nothing else. Revocation is immediate: the row is marked
* browser source holds it in its URL and nothing else. Revocation is immediate: the row is marked
- * and every open stream of that token is closed in the same call.
+ * and every open stream of that token, in any process, is told and closed.
*
*
* Deliveries. Every alert (a settled interaction) and goal change becomes one overlay_deliveries
* Deliveries. Every alert (a settled interaction) and goal change becomes one overlay_deliveries
* row with a monotonic seq (the SSE event id). The first time a row is written to any overlay it
* row with a monotonic seq (the SSE event id). The first time a row is written to any overlay it
- * becomes `delivered` and emits tips.overlay.delivered; a row no overlay received within the
+ * becomes `delivered` and emits tips.overlay.delivered (recorded just before the bytes are written);
+ * a row no overlay received within the
* delivery window becomes `failed` (tips.overlay.failed). A reconnect with Last-Event-ID
* delivery window becomes `failed` (tips.overlay.failed). A reconnect with Last-Event-ID
* REPLAYS rows after that id: replay only writes bytes to a socket — it never charges, never
* REPLAYS rows after that id: replay only writes bytes to a socket — it never charges, never
* re-counts a goal, never emits another event. Simulated (test) rows are flagged and emit nothing.
* re-counts a goal, never emits another event. Simulated (test) rows are flagged and emit nothing.
@@ -22,62 +23,70 @@
@@ -22,62 +23,70 @@
* sent, replayed or listed again — and tells connected overlays to drop it (`retract` event).
* sent, replayed or listed again — and tells connected overlays to drop it (`retract` event).
*/
*/
const crypto = require('crypto');
const crypto = require('crypto');
-const { fail, iso, prefixedId, sha256, json, text } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { fail, iso, prefixedId, sha256, text } = require('../util');
const { safeUrl } = require('./profiles');
const { safeUrl } = require('./profiles');
-const { isHidden } = require('./privacy');
+const { isHidden, publicView } = require('./privacy');
+const { createOverlayHub } = require('./overlay-hub');
const SCOPES = ['alerts', 'goals'];
const SCOPES = ['alerts', 'goals'];
+const STREAM_LOCK = 4610; // the advisory-lock namespace of a creator's overlay stream (Tips' port)
const TOKEN_RE = /^tovl_[A-Za-z0-9_-]{43}$/;
const TOKEN_RE = /^tovl_[A-Za-z0-9_-]{43}$/;
+// A delivery row with the interaction's real amount (an overlay's minimum applies to a hidden amount too).
+const DELIVERY_COLUMNS = sql`d.*, i.amount AS interaction_amount`;
+const DELIVERY_FROM = sql`overlay_deliveries d LEFT JOIN tip_interactions i ON i.id = d.interaction_id`;
function createOverlays(ctx) {
function createOverlays(ctx) {
const { db } = ctx;
const { db } = ctx;
- const clients = new Map(); // creator subject → Set<client>
- let nextClient = 1;
+ const hub = createOverlayHub({ valkey: ctx.valkey, pushNew, heartbeatMs: ctx.config.overlays.heartbeatMs, log: ctx.log });
// ── Tokens ───────────────────────────────────────────────
// ── Tokens ───────────────────────────────────────────────
- function createToken(creator, { scopes, label, configId, createdBy }) {
+ async function createToken(creator, { scopes, label, configId, createdBy }) {
const list = Array.isArray(scopes) && scopes.length ? [...new Set(scopes.map(String))] : SCOPES;
const list = Array.isArray(scopes) && scopes.length ? [...new Set(scopes.map(String))] : SCOPES;
if (list.some((s) => !SCOPES.includes(s))) fail(422, 'tips.invalid_input', `scopes must be among ${SCOPES.join(', ')}`);
if (list.some((s) => !SCOPES.includes(s))) fail(422, 'tips.invalid_input', `scopes must be among ${SCOPES.join(', ')}`);
if (configId) {
if (configId) {
- const c = getConfig(configId);
+ const c = await getConfig(db, configId);
if (!c || c.creator_subject !== creator) fail(404, 'tips.overlay_config_not_found', 'no such overlay config for this creator');
if (!c || c.creator_subject !== creator) fail(404, 'tips.overlay_config_not_found', 'no such overlay config for this creator');
}
}
- const active = db.prepare('SELECT COUNT(*) AS n FROM overlay_tokens WHERE creator_subject = ? AND revoked_at IS NULL').get(creator).n;
- if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');
const secret = `tovl_${crypto.randomBytes(32).toString('base64url')}`;
const secret = `tovl_${crypto.randomBytes(32).toString('base64url')}`;
- const id = prefixedId('tovt', ctx.now());
- db.prepare(`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)
- VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, sha256(secret), JSON.stringify(list), configId || null, text(label, 'label', 80), createdBy, iso(ctx.now()));
- return { token: presentToken(getToken(id)), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };
+ const row = await ctx.tx(async (t) => {
+ await ctx.lockCreator(t, creator);
+ const active = await t.value(sql`SELECT count(*) FROM overlay_tokens WHERE creator_subject = ${creator} AND revoked_at IS NULL`);
+ if (active >= 25) fail(409, 'tips.too_many_tokens', 'revoke an overlay token before creating another (25 active at most)');
+ return t.one(sql`INSERT INTO overlay_tokens (id, creator_subject, token_hash, scopes, config_id, label, created_by, created_at)
+ VALUES (${prefixedId('tovt', ctx.now())}, ${creator}, ${sha256(secret)}, ${sql.json(list)}, ${configId || null}, ${text(label, 'label', 80)}, ${createdBy}, ${iso(ctx.now())})
+ RETURNING *`);
+ });
+ return { token: presentToken(row), secret, overlay_url: `${ctx.config.baseUrl}/overlay/${secret}`, events_url: `${ctx.config.baseUrl}/overlay/${secret}/events` };
}
}
- const getToken = (id) => db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(String(id || '')) || null;
- const listTokens = (creator) => db.prepare('SELECT * FROM overlay_tokens WHERE creator_subject = ? ORDER BY revoked_at IS NOT NULL, created_at DESC').all(creator);
+ const getToken = (q, id) => q.maybe(sql`SELECT * FROM overlay_tokens WHERE id = ${String(id || '')}`);
+ const listTokens = (q, creator) => q.many(sql`SELECT * FROM overlay_tokens WHERE creator_subject = ${creator} ORDER BY revoked_at IS NOT NULL, created_at DESC`);
/** The active token row for a presented secret, or null (unknown, malformed or revoked). */
/** The active token row for a presented secret, or null (unknown, malformed or revoked). */
- function authenticate(secret) {
+ async function authenticate(q, secret) {
if (!TOKEN_RE.test(String(secret || ''))) return null;
if (!TOKEN_RE.test(String(secret || ''))) return null;
- const row = db.prepare('SELECT * FROM overlay_tokens WHERE token_hash = ?').get(sha256(secret));
+ const row = await q.maybe(sql`SELECT * FROM overlay_tokens WHERE token_hash = ${sha256(secret)}`);
if (!row || row.revoked_at) return null;
if (!row || row.revoked_at) return null;
- return { ...row, scopes: json(row.scopes, []) };
+ return row;
}
}
- function revokeToken(row) {
- if (!row.revoked_at) db.prepare('UPDATE overlay_tokens SET revoked_at = ? WHERE id = ? AND revoked_at IS NULL').run(iso(ctx.now()), row.id);
- closeToken(row.id);
- return presentToken(getToken(row.id));
+ async function revokeToken(row) {
+ const out = await db.maybe(sql`UPDATE overlay_tokens SET revoked_at = ${iso(ctx.now())} WHERE id = ${row.id} AND revoked_at IS NULL RETURNING *`);
+ await hub.revoke(row.creator_subject, row.id);
+ return presentToken(out || await getToken(db, row.id));
}
}
function presentToken(t) {
function presentToken(t) {
if (!t) return null;
if (!t) return null;
return {
return {
- id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: json(t.scopes, []), label: t.label || null, config_id: t.config_id || null,
+ id: t.id, creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes, label: t.label || null, config_id: t.config_id || null,
created_at: t.created_at, last_used_at: t.last_used_at || null, revoked_at: t.revoked_at || null, active: !t.revoked_at,
created_at: t.created_at, last_used_at: t.last_used_at || null, revoked_at: t.revoked_at || null, active: !t.revoked_at,
};
};
}
}
// ── Configs ──────────────────────────────────────────────
// ── Configs ──────────────────────────────────────────────
- const getConfig = (id) => db.prepare('SELECT * FROM overlay_configs WHERE id = ?').get(String(id || '')) || null;
- const listConfigs = (creator) => db.prepare('SELECT * FROM overlay_configs WHERE creator_subject = ? ORDER BY created_at').all(creator);
+ const getConfig = (q, id) => q.maybe(sql`SELECT * FROM overlay_configs WHERE id = ${String(id || '')}`);
+ const listConfigs = (q, creator) => q.many(sql`SELECT * FROM overlay_configs WHERE creator_subject = ${creator} ORDER BY created_at`);
function settingsOf(kind, input = {}, base = {}) {
function settingsOf(kind, input = {}, base = {}) {
const s = { ...base };
const s = { ...base };
@@ -110,236 +119,248 @@ function createOverlays(ctx) {
@@ -110,236 +119,248 @@ function createOverlays(ctx) {
goal: { show_amounts: true, show_percent: true },
goal: { show_amounts: true, show_percent: true },
};
};
- function createConfig(creator, input = {}) {
+ async function createConfig(creator, input = {}) {
const kind = input.kind === 'goal' ? 'goal' : input.kind === 'alerts' ? 'alerts' : fail(422, 'tips.invalid_input', "kind must be 'alerts' or 'goal'");
const kind = input.kind === 'goal' ? 'goal' : input.kind === 'alerts' ? 'alerts' : fail(422, 'tips.invalid_input', "kind must be 'alerts' or 'goal'");
const name = text(input.name, 'name', 80) || (kind === 'alerts' ? 'Alerts' : 'Goal');
const name = text(input.name, 'name', 80) || (kind === 'alerts' ? 'Alerts' : 'Goal');
let goalId = null;
let goalId = null;
if (kind === 'goal' && input.goal_id) {
if (kind === 'goal' && input.goal_id) {
- const g = ctx.goals.get(input.goal_id);
+ const g = await ctx.goals.get(db, input.goal_id);
if (!g || g.creator_subject !== creator) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
if (!g || g.creator_subject !== creator) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
goalId = g.id;
goalId = g.id;
}
}
- const id = prefixedId('tovc', ctx.now());
const at = iso(ctx.now());
const at = iso(ctx.now());
- db.prepare(`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`)
- .run(id, creator, kind, name, JSON.stringify(settingsOf(kind, input, DEFAULTS[kind])), goalId, at, at);
- return presentConfig(getConfig(id));
+ const row = await db.one(sql`INSERT INTO overlay_configs (id, creator_subject, kind, name, settings, goal_id, created_at, updated_at)
+ VALUES (${prefixedId('tovc', ctx.now())}, ${creator}, ${kind}, ${name}, ${sql.json(settingsOf(kind, input, DEFAULTS[kind]))}, ${goalId}, ${at}, ${at}) RETURNING *`);
+ return presentConfig(row);
}
}
- function updateConfig(c, input = {}) {
- if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);
- const settings = settingsOf(c.kind, input, json(c.settings, {}));
+ /**
+ * A settings change on the locked row (two at once apply one after the other); with `revision` it
+ * applies only to that revision (else 409).
+ */
+ function updateConfig(config, input = {}) {
+ return ctx.tx(async (t) => {
+ const c = await t.one(sql`SELECT * FROM overlay_configs WHERE id = ${config.id} FOR UPDATE`);
+ if (input.revision != null && Number(input.revision) !== c.revision) fail(409, 'tips.revision_conflict', `the config is at revision ${c.revision}`);
+ return changeConfig(t, c, input);
+ });
+ }
+ async function changeConfig(t, c, input) {
+ const settings = settingsOf(c.kind, input, c.settings || {});
let goalId = c.goal_id;
let goalId = c.goal_id;
if (c.kind === 'goal' && input.goal_id !== undefined) {
if (c.kind === 'goal' && input.goal_id !== undefined) {
if (!input.goal_id) goalId = null;
if (!input.goal_id) goalId = null;
else {
else {
- const g = ctx.goals.get(input.goal_id);
+ const g = await ctx.goals.get(t, input.goal_id);
if (!g || g.creator_subject !== c.creator_subject) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
if (!g || g.creator_subject !== c.creator_subject) fail(404, 'tips.goal_not_found', 'no such goal for this creator');
goalId = g.id;
goalId = g.id;
}
}
}
}
const name = input.name !== undefined ? (text(input.name, 'name', 80) || c.name) : c.name;
const name = input.name !== undefined ? (text(input.name, 'name', 80) || c.name) : c.name;
- db.prepare('UPDATE overlay_configs SET name = ?, settings = ?, goal_id = ?, revision = revision + 1, updated_at = ? WHERE id = ?')
- .run(name, JSON.stringify(settings), goalId, iso(ctx.now()), c.id);
- const out = presentConfig(getConfig(c.id));
- // Open overlays using this config pick the change up at once.
- for (const cl of clients.get(c.creator_subject) || []) if (cl.configId === c.id) write(cl, 'config', null, { config: out });
+ const out = presentConfig(await t.one(sql`UPDATE overlay_configs SET name = ${name}, settings = ${sql.json(settings)}, goal_id = ${goalId},
+ revision = revision + 1, updated_at = ${iso(ctx.now())} WHERE id = ${c.id} RETURNING *`));
+ // Open overlays using this config, in any process, pick the change up once it is committed.
+ t.after(() => hub.config(c.creator_subject, out));
return out;
return out;
}
}
function presentConfig(c) {
function presentConfig(c) {
if (!c) return null;
if (!c) return null;
- return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: json(c.settings, {}), goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };
+ return { id: c.id, creator: { type: 'user', id: c.creator_subject }, kind: c.kind, name: c.name, settings: c.settings || {}, goal_id: c.goal_id || null, revision: c.revision, created_at: c.created_at, updated_at: c.updated_at };
}
}
// ── Deliveries ───────────────────────────────────────────
// ── Deliveries ───────────────────────────────────────────
- function insertDelivery(creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {
- const id = prefixedId('tovd', ctx.now());
- const r = db.prepare(`INSERT OR IGNORE INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)
- VALUES (?, ?, ?, ?, ?, ?, ?, 'pending', ?, ?)`).run(id, creator, kind, interactionId, goalId, JSON.stringify(payload), test ? 1 : 0, dedupe, iso(ctx.now()));
- if (r.changes) ctx.afterCommit(() => notify(creator));
- return r.changes ? id : null;
+ /**
+ * Inside a transaction. Open overlays hear of it once the transaction committed.
+ *
+ * A row's seq comes from its identity when it is inserted, but other processes see it only at commit.
+ * Two transactions of one creator committing out of order (seq 101 before 100) would let a stream move
+ * past 100 before 100 exists for it, and never send it. So a creator's deliveries are inserted under
+ * a transaction-scoped advisory lock (allowed through PgBouncer): the next insert for that creator
+ * waits for the previous transaction to end, and a creator's seqs become visible in order.
+ */
+ async function insertDelivery(t, creator, kind, { interactionId = null, goalId = null, payload, test = false, dedupe }) {
+ await t.query(sql`SELECT pg_advisory_xact_lock(${STREAM_LOCK}, hashtext(${creator}))`);
+ const row = await t.maybe(sql`INSERT INTO overlay_deliveries (id, creator_subject, kind, interaction_id, goal_id, payload, test, status, dedupe_key, created_at)
+ VALUES (${prefixedId('tovd', ctx.now())}, ${creator}, ${kind}, ${interactionId}, ${goalId}, ${sql.json(payload)}, ${!!test}, 'pending', ${dedupe}, ${iso(ctx.now())})
+ ON CONFLICT DO NOTHING RETURNING id`);
+ if (row) ctx.afterCommit(t, () => hub.notify(creator));
+ return row ? row.id : null;
}
}
- /** Inside the settlement transaction (or a simulation). */
- function addAlert(interaction) {
- const payload = ctx.view(interaction, { at: iso(ctx.now()) });
- return insertDelivery(interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });
+ /** Inside the settlement transaction (or a simulation). `filter`: the creator's word filter. */
+ function addAlert(t, interaction, filter) {
+ const payload = publicView(interaction, { at: iso(ctx.now()), filter });
+ return insertDelivery(t, interaction.creator_subject, 'alert', { interactionId: interaction.id, payload, test: interaction.test, dedupe: `alert:${interaction.id}` });
}
}
/**
/**
- * Inside a transaction: the interaction's public view changed. Its alert payload is rewritten, and
- * the goal updates it caused no longer name anyone.
+ * Inside a transaction: these interactions' public view changed (an erasure, a moderation). Their
+ * alert payloads are rewritten and the goal updates they caused no longer name anyone, in one read
+ * and one write. `filters`: Map(creator → the creator's word filter).
*/
*/
- function refreshInteraction(interaction) {
- const view = ctx.view(interaction);
- for (const row of db.prepare('SELECT seq, kind, payload FROM overlay_deliveries WHERE interaction_id = ?').all(interaction.id)) {
- const old = json(row.payload, {});
- const payload = row.kind === 'alert' ? ctx.view(interaction, { at: old.at }) : { ...old, by: interaction.hide_amount || isHidden(interaction) ? null : view.supporter_name };
- db.prepare('UPDATE overlay_deliveries SET payload = ? WHERE seq = ?').run(JSON.stringify(payload), row.seq);
- }
+ async function refreshInteractions(t, interactions, filters) {
+ if (!interactions.length) return;
+ const byId = new Map(interactions.map((i) => [i.id, i]));
+ const rows = await t.many(sql`SELECT seq, kind, payload, interaction_id FROM overlay_deliveries WHERE interaction_id = ANY(${[...byId.keys()]})`);
+ if (!rows.length) return;
+ const updates = rows.map((row) => {
+ const i = byId.get(row.interaction_id);
+ const filter = filters.get(i.creator_subject) || null;
+ const old = row.payload || {};
+ const payload = row.kind === 'alert' ? publicView(i, { at: old.at, filter })
+ : { ...old, by: i.hide_amount || isHidden(i) ? null : publicView(i, { filter }).supporter_name };
+ return { seq: row.seq, payload };
+ });
+ await t.exec(sql`UPDATE overlay_deliveries d SET payload = v.payload
+ FROM jsonb_to_recordset(${sql.json(updates)}) AS v(seq bigint, payload jsonb) WHERE d.seq = v.seq`);
}
}
+ const refreshInteraction = (t, interaction, filter) => refreshInteractions(t, [interaction], new Map([[interaction.creator_subject, filter]]));
/** Inside a transaction: a moderator hid the interaction. Its alert is never sent or replayed again. */
/** Inside a transaction: a moderator hid the interaction. Its alert is never sent or replayed again. */
- function retract(interaction) {
- db.prepare("UPDATE overlay_deliveries SET hidden = 1 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);
- refreshInteraction(interaction);
- const rows = db.prepare("SELECT id, seq FROM overlay_deliveries WHERE interaction_id = ? AND kind = 'alert'").all(interaction.id);
- ctx.afterCommit(() => {
- for (const c of clients.get(interaction.creator_subject) || []) {
- if (!c.scopes.includes('alerts')) continue;
- for (const r of rows) write(c, 'retract', null, { interaction_id: interaction.id, delivery_id: r.id, seq: r.seq });
- }
- });
+ async function retract(t, interaction, filter) {
+ const rows = await t.many(sql`UPDATE overlay_deliveries SET hidden = true WHERE interaction_id = ${interaction.id} AND kind = 'alert' RETURNING id, seq`);
+ await refreshInteraction(t, interaction, filter);
+ if (rows.length) ctx.afterCommit(t, () => hub.retract(interaction.creator_subject, interaction.id, rows.map((r) => ({ id: r.id, seq: r.seq }))));
}
}
/** Inside a transaction: shown again. Back in /state and replays; open overlays are not re-alerted. */
/** Inside a transaction: shown again. Back in /state and replays; open overlays are not re-alerted. */
- function unretract(interaction) {
- db.prepare("UPDATE overlay_deliveries SET hidden = 0 WHERE interaction_id = ? AND kind = 'alert'").run(interaction.id);
- refreshInteraction(interaction);
+ async function unretract(t, interaction, filter) {
+ await t.exec(sql`UPDATE overlay_deliveries SET hidden = false WHERE interaction_id = ${interaction.id} AND kind = 'alert'`);
+ await refreshInteraction(t, interaction, filter);
}
}
- function addGoalDelivery(creator, goalView, { reason, interactionId, by, dedupe, test = false }) {
- return insertDelivery(creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });
+ function addGoalDelivery(t, creator, goalView, { reason, interactionId, by, dedupe, test = false }) {
+ return insertDelivery(t, creator, 'goal', { interactionId, goalId: goalView.id, payload: { goal: goalView, reason, by: by || null, test }, test, dedupe });
}
}
- /** First write of a pending row: delivered + the event (never for tests). */
- function markDelivered(row) {
- ctx.tx(() => {
- const r = db.prepare("UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);
- if (r.changes && !row.test) {
- ctx.outbox.emit('tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {
+ /**
+ * About to write these rows to an overlay: count the sends, and the first write of a pending row
+ * makes it delivered + the event (never for tests). One transaction for the batch.
+ */
+ async function recordSends(rows) {
+ const pending = rows.filter((r) => r.status === 'pending').map((r) => r.seq);
+ await ctx.tx(async (t) => {
+ await t.exec(sql`UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ANY(${rows.map((r) => r.seq)})`);
+ if (!pending.length) return;
+ const done = await t.many(sql`UPDATE overlay_deliveries SET status = 'delivered', delivered_at = ${iso(ctx.now())}
+ WHERE seq = ANY(${pending}) AND status = 'pending' RETURNING id, creator_subject, kind, interaction_id, goal_id, test`);
+ for (const row of done) {
+ if (row.test) continue;
+ await ctx.outbox.emit(t, 'tips.overlay.delivered', { type: 'overlay_delivery', id: row.id }, {
delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id, goal_id: row.goal_id,
delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id, goal_id: row.goal_id,
});
});
}
}
+ if (done.some((r) => !r.test)) t.after(ctx.outboxKick);
});
});
- ctx.outboxKick();
}
}
- /** Pending rows older than the window → failed (tips.overlay.failed). Returns how many. */
- function sweepFailed() {
+ /**
+ * Pending rows older than the window → failed (tips.overlay.failed). Returns how many. Rows are
+ * claimed with SKIP LOCKED, so processes sweeping at once never fail a row twice.
+ */
+ async function sweepFailed() {
const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE status = 'pending' AND hidden = 0 AND created_at < ? ORDER BY seq LIMIT 500").all(cutoff);
- if (!rows.length) return 0;
- ctx.tx(() => {
+ const n = await ctx.tx(async (t) => {
+ const rows = await t.many(sql`UPDATE overlay_deliveries SET status = 'failed', failed_at = ${iso(ctx.now())}
+ WHERE seq IN (SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < ${cutoff}
+ ORDER BY seq LIMIT 500 FOR UPDATE SKIP LOCKED)
+ RETURNING id, seq, creator_subject, kind, interaction_id, goal_id, test`);
+ rows.sort((a, b) => a.seq - b.seq);
for (const row of rows) {
for (const row of rows) {
- const r = db.prepare("UPDATE overlay_deliveries SET status = 'failed', failed_at = ? WHERE seq = ? AND status = 'pending'").run(iso(ctx.now()), row.seq);
- if (r.changes && !row.test) {
- ctx.outbox.emit('tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {
- delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,
- goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',
- });
- }
+ if (row.test) continue;
+ await ctx.outbox.emit(t, 'tips.overlay.failed', { type: 'overlay_delivery', id: row.id }, {
+ delivery_id: row.id, creator: { type: 'user', id: row.creator_subject }, kind: row.kind, interaction_id: row.interaction_id,
+ goal_id: row.goal_id, reason: 'no overlay showed it within the delivery window',
+ });
}
}
+ return rows.length;
});
});
- ctx.outboxKick();
- return rows.length;
+ if (n) ctx.outboxKick();
+ return n;
}
}
- // ── SSE hub ──────────────────────────────────────────────
+ // ── Streams (overlay-hub.js) ─────────────────────────────
const scopeOf = (kind) => (kind === 'alert' ? 'alerts' : 'goals');
const scopeOf = (kind) => (kind === 'alert' ? 'alerts' : 'goals');
- function write(client, event, id, data) {
- if (client.closed) return false;
- try {
- client.res.write(`${id != null ? `id: ${id}\n` : ''}event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
- return true;
- } catch { return false; }
- }
-
- function send(client, row) {
- client.lastSeq = Math.max(client.lastSeq, row.seq);
- if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) return;
- const payload = json(row.payload, {});
- if (row.kind === 'alert' && client.minAmount && !row.test) {
- // A hidden amount is not in the payload; the threshold still applies to the real one.
- const amount = payload.amount != null ? payload.amount : (db.prepare('SELECT amount FROM tip_interactions WHERE id = ?').get(row.interaction_id) || {}).amount;
- if (amount < client.minAmount) return;
+ /** Write these rows to a client (scopes, retractions and its minimum applied), after recording them. */
+ async function deliver(client, rows) {
+ const shown = [];
+ for (const row of rows) {
+ client.lastSeq = Math.max(client.lastSeq, row.seq);
+ if (row.hidden || !client.scopes.includes(scopeOf(row.kind))) continue;
+ const payload = row.payload || {};
+ if (row.kind === 'alert' && client.minAmount && !row.test) {
+ // A hidden amount is not in the payload; the threshold still applies to the real one.
+ const amount = payload.amount != null ? payload.amount : row.interaction_amount;
+ if (amount < client.minAmount) continue;
+ }
+ shown.push({ row, payload });
}
}
- if (!write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload })) return;
- db.prepare('UPDATE overlay_deliveries SET sends = sends + 1 WHERE seq = ?').run(row.seq);
- if (row.status === 'pending') markDelivered(row);
+ if (!shown.length || client.closed) return;
+ await recordSends(shown.map((x) => x.row));
+ for (const { row, payload } of shown) hub.write(client, row.kind, row.seq, { delivery_id: row.id, seq: row.seq, ...payload });
}
}
- function pushNew(client) {
- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq LIMIT 200").all(client.creator, client.lastSeq);
- for (const row of rows) send(client, row);
+ async function pushNew(client) {
+ const rows = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
+ WHERE d.creator_subject = ${client.creator} AND d.seq > ${client.lastSeq} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq LIMIT 200`);
+ await deliver(client, rows);
}
}
- function notify(creator) {
- for (const c of clients.get(creator) || []) pushNew(c);
- }
+ /** A stream place for this token (TIPS_OVERLAY_MAX_STREAMS across processes), or null: answer 429. */
+ const reserveStream = (token) => hub.reserve(token, ctx.config.overlays.maxStreamsPerToken);
+ const releaseStream = (slot) => hub.release(slot);
/**
/**
- * Attach an SSE response. lastEventId (Last-Event-ID): replay what came after it (bounded).
- * Without it: what is still pending within the window, then everything new.
+ * Attach an SSE response (its slot from reserveStream). lastEventId (Last-Event-ID): replay what
+ * came after it (bounded). Without it: what is still pending within the window, then everything new.
*/
*/
- function attach(token, res, { lastEventId, config } = {}) {
- const settings = config ? json(config.settings, {}) : {};
- const client = {
- id: nextClient++, creator: token.creator_subject, tokenId: token.id, scopes: token.scopes, configId: token.config_id || null,
- minAmount: config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0, res, lastSeq: 0, closed: false,
- };
- if (!clients.has(client.creator)) clients.set(client.creator, new Set());
- clients.get(client.creator).add(client);
- db.prepare('UPDATE overlay_tokens SET last_used_at = ? WHERE id = ?').run(iso(ctx.now()), token.id);
-
- const maxSeq = db.prepare('SELECT COALESCE(MAX(seq), 0) AS n FROM overlay_deliveries WHERE creator_subject = ?').get(client.creator).n;
- write(client, 'hello', null, {
- creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,
- goals: client.scopes.includes('goals') ? ctx.goals.list(client.creator, { status: 'active' }).map((g) => ctx.goals.present(g)) : [],
- resumed_from: lastEventId != null ? lastEventId : null,
+ function attach(token, res, slot, { lastEventId, config } = {}) {
+ const settings = config ? config.settings || {} : {};
+ const minAmount = config && config.kind === 'alerts' ? Number(settings.min_amount) || 0 : 0;
+ return hub.attach(token, res, slot, { minAmount, configId: token.config_id || null }, async (client) => {
+ await db.exec(sql`UPDATE overlay_tokens SET last_used_at = ${iso(ctx.now())} WHERE id = ${token.id}`);
+ const maxSeq = await db.value(sql`SELECT COALESCE(MAX(seq), 0) FROM overlay_deliveries WHERE creator_subject = ${client.creator}`);
+ const goals = client.scopes.includes('goals') ? await ctx.goals.presentMany(db, await ctx.goals.list(db, client.creator, { status: 'active' })) : [];
+ hub.write(client, 'hello', null, {
+ creator: { type: 'user', id: client.creator }, scopes: client.scopes, config: config ? presentConfig(config) : null,
+ goals, resumed_from: lastEventId != null ? lastEventId : null,
+ });
+ const last = Number(lastEventId);
+ if (lastEventId != null && Number.isInteger(last) && last >= 0) {
+ const limit = ctx.config.overlays.replayLimit;
+ const rows = (await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
+ WHERE d.creator_subject = ${client.creator} AND d.seq > ${last} AND d.status <> 'failed' AND NOT d.hidden ORDER BY d.seq DESC LIMIT ${limit}`)).reverse();
+ client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;
+ await deliver(client, rows);
+ client.lastSeq = Math.max(client.lastSeq, maxSeq);
+ } else {
+ const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
+ const pending = await db.many(sql`SELECT ${DELIVERY_COLUMNS} FROM ${DELIVERY_FROM}
+ WHERE d.creator_subject = ${client.creator} AND d.status = 'pending' AND NOT d.hidden AND d.created_at >= ${cutoff} ORDER BY d.seq LIMIT 100`);
+ await deliver(client, pending);
+ client.lastSeq = Math.max(client.lastSeq, maxSeq);
+ }
});
});
- const last = Number(lastEventId);
- if (lastEventId != null && Number.isInteger(last) && last >= 0) {
- const limit = ctx.config.overlays.replayLimit;
- const rows = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND seq > ? AND status != 'failed' AND hidden = 0 ORDER BY seq DESC LIMIT ?").all(client.creator, last, limit).reverse();
- client.lastSeq = rows.length ? rows[0].seq - 1 : maxSeq;
- for (const row of rows) send(client, row);
- client.lastSeq = Math.max(client.lastSeq, maxSeq);
- } else {
- const cutoff = iso(ctx.now() - ctx.config.overlays.ttlMs);
- const pending = db.prepare("SELECT * FROM overlay_deliveries WHERE creator_subject = ? AND status = 'pending' AND hidden = 0 AND created_at >= ? ORDER BY seq LIMIT 100").all(client.creator, cutoff);
- for (const row of pending) send(client, row);
- client.lastSeq = Math.max(client.lastSeq, maxSeq);
- }
- pushNew(client);
- const beat = setInterval(() => { if (!write(client, 'ping', null, { t: Date.now() })) detach(client); }, ctx.config.overlays.heartbeatMs);
- if (beat.unref) beat.unref();
- client.beat = beat;
- res.on('close', () => detach(client));
- return client;
}
}
- function detach(client) {
- if (client.closed) return;
- client.closed = true;
- clearInterval(client.beat);
- const set = clients.get(client.creator);
- if (set) { set.delete(client); if (!set.size) clients.delete(client.creator); }
- try { client.res.end(); } catch { /* already gone */ }
+ function recentDeliveries(q, creator, limit = 50) {
+ return q.many(sql`SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at
+ FROM overlay_deliveries WHERE creator_subject = ${creator} ORDER BY seq DESC LIMIT ${limit}`);
}
}
- function closeToken(tokenId) {
- for (const set of clients.values()) {
- for (const c of [...set]) {
- if (c.tokenId === tokenId) { write(c, 'revoked', null, { reason: 'this overlay token was revoked' }); detach(c); }
- }
- }
- }
-
- function connected(creator) { return (clients.get(creator) || new Set()).size; }
- function streamsFor(token) { let n = 0; for (const c of clients.get(token.creator_subject) || []) if (c.tokenId === token.id) n++; return n; }
- function closeAll() { for (const set of clients.values()) for (const c of [...set]) detach(c); }
-
- function recentDeliveries(creator, limit = 50) {
- return db.prepare('SELECT seq, id, kind, interaction_id, goal_id, test, status, hidden, sends, created_at, delivered_at, failed_at FROM overlay_deliveries WHERE creator_subject = ? ORDER BY seq DESC LIMIT ?').all(creator, limit);
+ /** The newest alerts of a creator, for GET /overlay/:token/state. */
+ async function recentAlerts(q, creator) {
+ return (await q.many(sql`SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ${creator} AND kind = 'alert' AND NOT hidden
+ ORDER BY seq DESC LIMIT 20`)).map((d) => ({ seq: d.seq, ...(d.payload || {}), test: !!d.test, created_at: d.created_at }));
}
}
return {
return {
SCOPES, createToken, getToken, listTokens, authenticate, revokeToken, presentToken,
SCOPES, createToken, getToken, listTokens, authenticate, revokeToken, presentToken,
getConfig, listConfigs, createConfig, updateConfig, presentConfig,
getConfig, listConfigs, createConfig, updateConfig, presentConfig,
- addAlert, refreshInteraction, retract, unretract, addGoalDelivery, sweepFailed, attach, detach, closeToken, notify, connected, streamsFor, closeAll, recentDeliveries,
+ addAlert, refreshInteraction, refreshInteractions, retract, unretract, addGoalDelivery, sweepFailed, reserveStream, releaseStream, attach,
+ connected: hub.connected, closeAll: () => hub.closeAll(), close: () => hub.close(), recentDeliveries, recentAlerts,
};
};
}
}
@@ -25,7 +25,7 @@
@@ -25,7 +25,7 @@
* word filter (filter.js) stars blocked words out of the name and message and drops them from what
* word filter (filter.js) stars blocked words out of the name and message and drops them from what
* text-to-speech reads.
* text-to-speech reads.
*/
*/
-const { fail, json } = require('../util');
+const { fail } = require('../util');
const filter = require('./filter');
const filter = require('./filter');
const ANONYMOUS = 'Anonymous';
const ANONYMOUS = 'Anonymous';
@@ -64,10 +64,11 @@ function shownName(i, settings) {
@@ -64,10 +64,11 @@ function shownName(i, settings) {
/**
/**
* The public shape of an interaction. `at` is kept from an earlier overlay payload when given, so a
* The public shape of an interaction. `at` is kept from an earlier overlay payload when given, so a
- * rewritten alert keeps its time; `filter` is the creator's filter settings.
+ * rewritten alert keeps its time; `filter` is the creator's filter settings (profiles.filterOf, or a
+ * profile's `filter`), fetched once per creator by the caller.
*/
*/
function publicView(i, { at, filter: settings = null } = {}) {
function publicView(i, { at, filter: settings = null } = {}) {
- const req = json(i.request, {});
+ const req = i.request || {};
const privateMsg = !!i.private_message || !!i.erased_at;
const privateMsg = !!i.private_message || !!i.erased_at;
const base = {
const base = {
interaction_id: i.id,
interaction_id: i.id,
@@ -19,7 +19,8 @@
@@ -19,7 +19,8 @@
* filter_words / filter_action / filter_links are the creator's word filter for paid messages
* filter_words / filter_action / filter_links are the creator's word filter for paid messages
* (filter.js; `filter` in the API): filterOf(subject) is what every public view applies.
* (filter.js; `filter` in the API): filterOf(subject) is what every public view applies.
*/
*/
-const { fail, iso, text, json } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { fail, iso, text } = require('../util');
const filterLib = require('./filter');
const filterLib = require('./filter');
const HANDLE_RE = /^[a-z0-9][a-z0-9_.-]{0,39}$/;
const HANDLE_RE = /^[a-z0-9][a-z0-9_.-]{0,39}$/;
@@ -36,55 +37,70 @@ function normalizeHandle(v) {
@@ -36,55 +37,70 @@ function normalizeHandle(v) {
const PAGE_DEFAULTS = Object.freeze({ goal_amounts: true, goal_supporters: false, supporters_page: false, supporters_amounts: false, supporters_messages: false });
const PAGE_DEFAULTS = Object.freeze({ goal_amounts: true, goal_supporters: false, supporters_page: false, supporters_amounts: false, supporters_messages: false });
-function pageSettings(raw) {
- const stored = json(raw, {});
+/** page_settings (jsonb) → every setting, defaults filled in. */
+function pageSettings(stored) {
+ const s = stored && typeof stored === 'object' ? stored : {};
const out = {};
const out = {};
- for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof stored[k] === 'boolean' ? stored[k] : PAGE_DEFAULTS[k];
+ for (const k of Object.keys(PAGE_DEFAULTS)) out[k] = typeof s[k] === 'boolean' ? s[k] : PAGE_DEFAULTS[k];
return out;
return out;
}
}
+/** The creator's filter settings for publicView() from a profile row (filter_* columns). */
+const filterOfRow = (row) => ({
+ words: Array.isArray(row.filter_words) ? row.filter_words : [],
+ action: row.filter_action === 'hold' ? 'hold' : 'mask',
+ links: row.filter_links !== false,
+});
+
function parse(row) {
function parse(row) {
if (!row) return null;
if (!row) return null;
- const b = (k) => !!row[k];
- return {
- ...row, page_enabled: b('page_enabled'), accepting: b('accepting'), tts_enabled: b('tts_enabled'), media_requests_enabled: b('media_requests_enabled'),
- page: pageSettings(row.page_settings),
- filter: { words: json(row.filter_words, []), action: filterLib.ACTIONS.includes(row.filter_action) ? row.filter_action : 'mask', links: row.filter_links !== 0 },
- };
+ return { ...row, page: pageSettings(row.page_settings), filter: filterOfRow(row) };
}
}
+const HANDLE_TAKEN = 'creator_tip_profiles_handle_key';
+
function createProfiles(ctx) {
function createProfiles(ctx) {
const { db } = ctx;
const { db } = ctx;
- const bySubject = (s) => parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE creator_subject = ?').get(s));
- const byHandle = (h) => { const n = normalizeHandle(h); return n ? parse(db.prepare('SELECT * FROM creator_tip_profiles WHERE handle = ?').get(n)) : null; };
+ const bySubject = async (q, s) => parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(s || '')}`));
+ const byHandle = async (q, h) => { const n = normalizeHandle(h); return n ? parse(await q.maybe(sql`SELECT * FROM creator_tip_profiles WHERE handle = ${n}`)) : null; };
/** usr_… or a handle (with or without @). */
/** usr_… or a handle (with or without @). */
- const resolve = (ref) => (/^usr_/.test(String(ref || '')) ? bySubject(String(ref)) : byHandle(ref));
+ const resolve = (q, ref) => (/^usr_/.test(String(ref || '')) ? bySubject(q, String(ref)) : byHandle(q, ref));
/** A handle not held by another creator: the username, else username-<suffix of the subject>. */
/** A handle not held by another creator: the username, else username-<suffix of the subject>. */
- function freeHandle(subject, username) {
+ async function freeHandle(q, subject, username) {
let h = normalizeHandle(username) || `creator-${subject.slice(-8).toLowerCase()}`;
let h = normalizeHandle(username) || `creator-${subject.slice(-8).toLowerCase()}`;
- const holder = db.prepare('SELECT creator_subject FROM creator_tip_profiles WHERE handle = ?').get(h);
+ const holder = await q.maybe(sql`SELECT creator_subject FROM creator_tip_profiles WHERE handle = ${h}`);
if (holder && holder.creator_subject !== subject) h = `${h.slice(0, 30)}-${subject.slice(-6).toLowerCase()}`;
if (holder && holder.creator_subject !== subject) h = `${h.slice(0, 30)}-${subject.slice(-6).toLowerCase()}`;
return h;
return h;
}
}
- /** Create or refresh the profile from the creator's own Network account (claims). */
- function ensure(subject, { username, displayName, avatarUrl } = {}) {
+ /**
+ * Create or refresh the profile from the creator's own Network account (claims). Two first visits
+ * at once create it once; a handle another creator took meanwhile is looked for again.
+ */
+ async function ensure(subject, claims = {}, attempt = 0) {
+ try { return await ensureOnce(subject, claims); } catch (e) {
+ if (e.code === '23505' && e.constraint === HANDLE_TAKEN && attempt < 2) return ensure(subject, claims, attempt + 1);
+ throw e;
+ }
+ }
+ async function ensureOnce(subject, { username, displayName, avatarUrl } = {}) {
const at = iso(ctx.now());
const at = iso(ctx.now());
- const existing = bySubject(subject);
+ const existing = await bySubject(db, subject);
const name = String(displayName || username || 'Creator').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80) || 'Creator';
const name = String(displayName || username || 'Creator').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80) || 'Creator';
if (!existing) {
if (!existing) {
- db.prepare(`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)
- VALUES (?, ?, ?, ?, ?, ?)`).run(subject, freeHandle(subject, username), name, safeUrl(avatarUrl), at, at);
- return bySubject(subject);
+ const row = await db.maybe(sql`INSERT INTO creator_tip_profiles (creator_subject, handle, display_name, avatar_url, created_at, updated_at)
+ VALUES (${subject}, ${await freeHandle(db, subject, username)}, ${name}, ${safeUrl(avatarUrl)}, ${at}, ${at})
+ ON CONFLICT (creator_subject) DO NOTHING RETURNING *`);
+ return row ? parse(row) : bySubject(db, subject);
}
}
- const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? freeHandle(subject, username) : existing.handle;
+ const handle = normalizeHandle(username) && normalizeHandle(username) !== existing.handle ? await freeHandle(db, subject, username) : existing.handle;
if (handle !== existing.handle || name !== existing.display_name || (safeUrl(avatarUrl) || null) !== (existing.avatar_url || null)) {
if (handle !== existing.handle || name !== existing.display_name || (safeUrl(avatarUrl) || null) !== (existing.avatar_url || null)) {
- db.prepare('UPDATE creator_tip_profiles SET handle = ?, display_name = ?, avatar_url = ?, updated_at = ? WHERE creator_subject = ?')
- .run(handle, name, safeUrl(avatarUrl), at, subject);
+ return parse(await db.one(sql`UPDATE creator_tip_profiles SET handle = ${handle}, display_name = ${name}, avatar_url = ${safeUrl(avatarUrl)}, updated_at = ${at}
+ WHERE creator_subject = ${subject} RETURNING *`));
}
}
- return bySubject(subject);
+ return existing;
}
}
const LIMITS = {
const LIMITS = {
@@ -92,13 +108,25 @@ function createProfiles(ctx) {
@@ -92,13 +108,25 @@ function createProfiles(ctx) {
media_request_min: [1, 1_000_000], media_max_seconds: [10, 3 * 3600],
media_request_min: [1, 1_000_000], media_max_seconds: [10, 3 * 3600],
};
};
const BOOLS = ['page_enabled', 'accepting', 'tts_enabled', 'media_requests_enabled'];
const BOOLS = ['page_enabled', 'accepting', 'tts_enabled', 'media_requests_enabled'];
+ const on = (v) => v === true || v === 'on' || v === '1' || v === 1;
+ /**
+ * A settings change, on the locked row (two changes at once apply one after the other, neither lost).
+ * With expectedRevision it applies only to that revision (else 409).
+ */
function update(subject, patch = {}, { expectedRevision } = {}) {
function update(subject, patch = {}, { expectedRevision } = {}) {
- const p = bySubject(subject);
- if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');
- if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);
+ return ctx.tx(async (t) => {
+ const p = parse(await t.maybe(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ${String(subject || '')} FOR UPDATE`));
+ if (!p) fail(404, 'tips.profile_not_found', 'no tip profile for this creator');
+ if (expectedRevision != null && Number(expectedRevision) !== p.revision) fail(409, 'tips.revision_conflict', `the profile is at revision ${p.revision}`);
+ return change(t, p, patch);
+ });
+ }
+
+ async function change(t, p, patch) {
+ const subject = p.creator_subject;
const set = {};
const set = {};
- for (const k of BOOLS) if (patch[k] !== undefined) set[k] = patch[k] === true || patch[k] === 'on' || patch[k] === '1' || patch[k] === 1 ? 1 : 0;
+ for (const k of BOOLS) if (patch[k] !== undefined) set[k] = on(patch[k]);
for (const [k, [lo, hi]] of Object.entries(LIMITS)) {
for (const [k, [lo, hi]] of Object.entries(LIMITS)) {
if (patch[k] === undefined || patch[k] === '') continue;
if (patch[k] === undefined || patch[k] === '') continue;
const n = Number(patch[k]);
const n = Number(patch[k]);
@@ -120,31 +148,29 @@ function createProfiles(ctx) {
@@ -120,31 +148,29 @@ function createProfiles(ctx) {
const unknown = Object.keys(patch.page).filter((k) => !(k in PAGE_DEFAULTS));
const unknown = Object.keys(patch.page).filter((k) => !(k in PAGE_DEFAULTS));
if (unknown.length) fail(422, 'tips.invalid_input', `page has no setting ${unknown[0]}`);
if (unknown.length) fail(422, 'tips.invalid_input', `page has no setting ${unknown[0]}`);
const next = { ...p.page };
const next = { ...p.page };
- for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = patch.page[k] === true || patch.page[k] === 'on' || patch.page[k] === '1' || patch.page[k] === 1;
- set.page_settings = JSON.stringify(next);
+ for (const k of Object.keys(PAGE_DEFAULTS)) if (patch.page[k] !== undefined) next[k] = on(patch.page[k]);
+ set.page_settings = sql.json(next);
}
}
if (patch.filter !== undefined) {
if (patch.filter !== undefined) {
const f = patch.filter;
const f = patch.filter;
if (!f || typeof f !== 'object' || Array.isArray(f)) fail(422, 'tips.invalid_input', 'filter must be an object { words, action, links }');
if (!f || typeof f !== 'object' || Array.isArray(f)) fail(422, 'tips.invalid_input', 'filter must be an object { words, action, links }');
const unknown = Object.keys(f).filter((k) => !['words', 'action', 'links'].includes(k));
const unknown = Object.keys(f).filter((k) => !['words', 'action', 'links'].includes(k));
if (unknown.length) fail(422, 'tips.invalid_input', `filter has no setting ${unknown[0]}`);
if (unknown.length) fail(422, 'tips.invalid_input', `filter has no setting ${unknown[0]}`);
- if (f.words !== undefined) set.filter_words = JSON.stringify(filterLib.normalizeWords(f.words));
+ if (f.words !== undefined) set.filter_words = sql.json(filterLib.normalizeWords(f.words));
if (f.action !== undefined) {
if (f.action !== undefined) {
if (!filterLib.ACTIONS.includes(f.action)) fail(422, 'tips.invalid_input', `filter.action must be one of ${filterLib.ACTIONS.join(', ')}`);
if (!filterLib.ACTIONS.includes(f.action)) fail(422, 'tips.invalid_input', `filter.action must be one of ${filterLib.ACTIONS.join(', ')}`);
set.filter_action = f.action;
set.filter_action = f.action;
}
}
- if (f.links !== undefined) set.filter_links = f.links === true || f.links === 'on' || f.links === '1' || f.links === 1 ? 1 : 0;
+ if (f.links !== undefined) set.filter_links = on(f.links);
}
}
if (patch.display_name !== undefined) {
if (patch.display_name !== undefined) {
const n = text(patch.display_name, 'display_name', 80);
const n = text(patch.display_name, 'display_name', 80);
if (!n) fail(422, 'tips.invalid_input', 'display_name cannot be empty');
if (!n) fail(422, 'tips.invalid_input', 'display_name cannot be empty');
set.display_name = n.replace(/[<>]/g, '');
set.display_name = n.replace(/[<>]/g, '');
}
}
- const keys = Object.keys(set);
- if (!keys.length) return p;
- db.prepare(`UPDATE creator_tip_profiles SET ${keys.map((k) => `${k} = @${k}`).join(', ')}, revision = revision + 1, updated_at = @at WHERE creator_subject = @s`)
- .run({ ...set, at: iso(ctx.now()), s: subject });
- return bySubject(subject);
+ if (!Object.keys(set).length) return p;
+ return parse(await t.one(sql`UPDATE creator_tip_profiles SET ${sql.set({ ...set, updated_at: iso(ctx.now()) })}, revision = revision + 1
+ WHERE creator_subject = ${subject} RETURNING *`));
}
}
/** Public shape; `owner` adds the settings only the creator sees. */
/** Public shape; `owner` adds the settings only the creator sees. */
@@ -165,13 +191,31 @@ function createProfiles(ctx) {
@@ -165,13 +191,31 @@ function createProfiles(ctx) {
}
}
/** The creator's filter settings for publicView() (null when they have no profile). */
/** The creator's filter settings for publicView() (null when they have no profile). */
- function filterOf(subject) {
- const row = db.prepare('SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ?').get(subject);
- if (!row) return null;
- return { words: json(row.filter_words, []), action: row.filter_action === 'hold' ? 'hold' : 'mask', links: row.filter_links !== 0 };
+ async function filterOf(q, subject) {
+ const row = await q.maybe(sql`SELECT filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ${subject}`);
+ return row ? filterOfRow(row) : null;
+ }
+ /** Several creators' profiles in one query: Map(subject → profile | null). */
+ async function bySubjects(q, subjects) {
+ const list = [...new Set(subjects)];
+ const out = new Map(list.map((s) => [s, null]));
+ if (!list.length) return out;
+ for (const row of await q.many(sql`SELECT * FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) out.set(row.creator_subject, parse(row));
+ return out;
+ }
+
+ /** Several creators' filters in one query: Map(subject → settings | null). */
+ async function filtersOf(q, subjects) {
+ const list = [...new Set(subjects)];
+ const out = new Map(list.map((s) => [s, null]));
+ if (!list.length) return out;
+ for (const row of await q.many(sql`SELECT creator_subject, filter_words, filter_action, filter_links FROM creator_tip_profiles WHERE creator_subject = ANY(${list})`)) {
+ out.set(row.creator_subject, filterOfRow(row));
+ }
+ return out;
}
}
- return { bySubject, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf };
+ return { bySubject, bySubjects, byHandle, resolve, ensure, update, present, normalizeHandle, filterOf, filtersOf };
}
}
function safeUrl(u) {
function safeUrl(u) {
@@ -14,41 +14,41 @@
@@ -14,41 +14,41 @@
* chat line, overlay alert, goal. Billing sends it only once it is the
* chat line, overlay alert, goal. Billing sends it only once it is the
* money authority (while Live is, Live's webhook announces the tip)
* money authority (while Live is, Live's webhook announces the tip)
*
*
- * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same SQLite
+ * Exactly once, twice over: the openvibe-sdk inbox claims (consumer, event_id) in the same
* transaction as the change, so a redelivered event does nothing; and the Billing transaction id is
* transaction as the change, so a redelivered event does nothing; and the Billing transaction id is
* UNIQUE on tip_interactions, so the same transaction arriving under another event id (a replay, a
* UNIQUE on tip_interactions, so the same transaction arriving under another event id (a replay, a
- * republish) still yields one logical interaction.
+ * republish) still yields one logical interaction. That transaction moves money state, so it is
+ * SERIALIZABLE (domain MONEY): two deliveries racing in two processes settle once.
*
*
* The signature (X-OpenVibe-Signature, HMAC-SHA256 of the raw body with TIPS_EVENTS_SECRET) is
* The signature (X-OpenVibe-Signature, HMAC-SHA256 of the raw body with TIPS_EVENTS_SECRET) is
* verified with openvibe-sdk's parseDelivery. Only events whose source is `billing` are applied.
* verified with openvibe-sdk's parseDelivery. Only events whose source is `billing` are applied.
*/
*/
const express = require('express');
const express = require('express');
const { http } = require('openvibe-contracts');
const { http } = require('openvibe-contracts');
-const { parseDelivery, createInbox } = require('openvibe-sdk/events');
+const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');
const CONSUMER = 'tips-billing';
const CONSUMER = 'tips-billing';
+const TABLE = 'tips_event_inbox'; // migrations/0001_initial.sql (inboxSchema)
function consumerRouter({ domain, config, log = console }) {
function consumerRouter({ domain, config, log = console }) {
const router = express.Router();
const router = express.Router();
- const inbox = createInbox(domain.db, { now: domain.now });
- inbox.ensureSchema();
+ // The inbox's transaction is the domain's MONEY one (serializable, with after-commit hooks).
+ const inbox = createPgInbox({ tx: (fn) => domain.tx(fn, domain.MONEY), maybe: (...a) => domain.db.maybe(...a) }, { table: TABLE, now: domain.now });
/** Apply one envelope (also used by tests and the replay tool). Returns { duplicate, outcome }. */
/** Apply one envelope (also used by tests and the replay tool). Returns { duplicate, outcome }. */
- function apply(event) {
- return domain.tx(() => {
- const r = inbox.once(CONSUMER, event.event_id, () => {
- if (event.source !== 'billing') return 'ignored:source';
- const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};
- if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(payload);
- if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(payload);
- if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(payload);
- return 'ignored:type';
- });
- return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };
+ async function apply(event) {
+ const r = await inbox.once(CONSUMER, event.event_id, async (t) => {
+ if (event.source !== 'billing') return 'ignored:source';
+ const payload = event.payload && typeof event.payload === 'object' ? event.payload : {};
+ if (event.event_type === 'billing.transaction.settled') return domain.interactions.onBillingSettled(t, payload);
+ if (event.event_type === 'billing.transaction.reversed') return domain.interactions.onBillingReversed(t, payload);
+ if (event.event_type === 'billing.receipt.external') return domain.interactions.onBillingExternal(t, payload);
+ return 'ignored:type';
});
});
+ return r.duplicate ? { duplicate: true, outcome: null } : { duplicate: false, outcome: r.result };
}
}
- router.post('/events', express.raw({ type: () => true, limit: '256kb' }), (req, res) => {
+ router.post('/events', express.raw({ type: () => true, limit: '256kb' }), async (req, res) => {
const secrets = config.events.webhookSecrets;
const secrets = config.events.webhookSecrets;
if (!secrets.length) return http.sendProblem(res, 503, 'tips.webhook_disabled', { detail: 'TIPS_EVENTS_SECRET is not set', ctx: req.ov });
if (!secrets.length) return http.sendProblem(res, 503, 'tips.webhook_disabled', { detail: 'TIPS_EVENTS_SECRET is not set', ctx: req.ov });
const raw = Buffer.isBuffer(req.body) ? req.body : Buffer.alloc(0);
const raw = Buffer.isBuffer(req.body) ? req.body : Buffer.alloc(0);
@@ -62,7 +62,7 @@ function consumerRouter({ domain, config, log = console }) {
@@ -62,7 +62,7 @@ function consumerRouter({ domain, config, log = console }) {
}
}
let out;
let out;
try {
try {
- out = apply(event);
+ out = await apply(event);
} catch (e) {
} catch (e) {
// Not acknowledged: Events retries it, and the inbox claim rolled back with the change.
// Not acknowledged: Events retries it, and the inbox claim rolled back with the change.
log.error(`[Tips] event ${event.event_id} (${event.event_type}) failed:`, e.message);
log.error(`[Tips] event ${event.event_id} (${event.event_type}) failed:`, e.message);
@@ -10,16 +10,18 @@
@@ -10,16 +10,18 @@
* tips.overlay.delivered an overlay delivery reached at least one overlay (first time only)
* tips.overlay.delivered an overlay delivery reached at least one overlay (first time only)
* tips.overlay.failed no overlay received it within the delivery window
* tips.overlay.failed no overlay received it within the delivery window
*
*
- * enqueue() runs inside the SQLite transaction that makes the change, so an event exists if and
- * only if its change committed. Simulations (test) never produce events. The relay publishes with
+ * emit(t, …) runs inside the transaction that makes the change (its handle `t`), so an event exists if
+ * and only if its change committed. Simulations (test) never produce events. The relay publishes with
* Tips' service token (events.event.publish, audience openvibe.events) only when EVENTS_URL and
* Tips' service token (events.event.publish, audience openvibe.events) only when EVENTS_URL and
- * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in event_outbox.
+ * OV_OAUTH_CLIENT_SECRET are set; otherwise rows wait in tips_event_outbox. Any number of processes
+ * relay the one table: the SDK claims due rows with a lease (FOR UPDATE SKIP LOCKED).
*/
*/
const { createClient } = require('openvibe-sdk/core');
const { createClient } = require('openvibe-sdk/core');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
-const { createEventsClient, createOutbox } = require('openvibe-sdk/events');
+const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
const ACTOR = { type: 'service', id: 'tips' };
const ACTOR = { type: 'service', id: 'tips' };
+const TABLE = 'tips_event_outbox'; // migrations/0001_initial.sql (outboxSchema)
function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
const enabled = !!(config.events.url && config.oauth.clientSecret);
const enabled = !!(config.events.url && config.oauth.clientSecret);
@@ -35,8 +37,9 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -35,8 +37,9 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
}
}
const events = createEventsClient(createClient(clientOpts), { source: 'tips' });
const events = createEventsClient(createClient(clientOpts), { source: 'tips' });
let lastError = null;
let lastError = null;
- const outbox = createOutbox(db, {
+ const outbox = createPgOutbox(db, {
events,
events,
+ table: TABLE,
intervalMs: config.events.intervalMs,
intervalMs: config.events.intervalMs,
now,
now,
onError: (err) => {
onError: (err) => {
@@ -45,11 +48,10 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -45,11 +48,10 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
lastError = msg;
lastError = msg;
},
},
});
});
- outbox.ensureSchema();
- /** Inside the caller's transaction. */
- function emit(eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {
- return outbox.enqueue({ event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });
+ /** Inside the caller's transaction `t`. */
+ function emit(t, eventType, subject, payload, { visibility = 'internal', priority = 'important', traceparent } = {}) {
+ return outbox.enqueue(t, { event_type: eventType, actor: ACTOR, subject, payload, visibility, priority }, { traceparent });
}
}
return {
return {
@@ -59,8 +61,8 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -59,8 +61,8 @@ function createTipsOutbox({ db, config, fetchImpl, now, log = console }) {
start() { if (enabled) outbox.start(); },
start() { if (enabled) outbox.start(); },
stop: () => outbox.stop(),
stop: () => outbox.stop(),
kick() { if (enabled) outbox.kick(); },
kick() { if (enabled) outbox.kick(); },
- status: () => ({ enabled, pending: outbox.pending(), rejected: outbox.rejected(), last_error: lastError }),
+ status: async () => ({ enabled, pending: await outbox.pending(), rejected: await outbox.rejected(), last_error: lastError }),
};
};
}
}
-module.exports = { createTipsOutbox };
+module.exports = { createTipsOutbox, TABLE };
@@ -32,8 +32,13 @@
@@ -32,8 +32,13 @@
* Idempotent: every row is keyed by legacy_source / migration_maps (live, table, id); a re-run changes
* Idempotent: every row is keyed by legacy_source / migration_maps (live, table, id); a re-run changes
* nothing, and a later snapshot only adds what is new or releases what was held. --dry-run does it all
* nothing, and a later snapshot only adds what is new or releases what was held. --dry-run does it all
* inside a transaction that is rolled back.
* inside a transaction that is rolled back.
+ *
+ * The snapshots are other services' SQLite files, read synchronously (better-sqlite3 handles passed in
+ * by scripts/import-live.js); everything written goes to Tips' PostgreSQL in one SERIALIZABLE
+ * transaction (it records settled money and applies refunds), whose report starts over on a retry.
*/
*/
-const { iso, prefixedId, json } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { iso, prefixedId, json, storable } = require('../util');
const liveTs = (s) => {
const liveTs = (s) => {
if (!s) return null;
if (!s) return null;
@@ -41,8 +46,9 @@ const liveTs = (s) => {
@@ -41,8 +46,9 @@ const liveTs = (s) => {
const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);
const t = Date.parse(str.includes('T') ? str : `${str.replace(' ', 'T')}Z`);
return Number.isFinite(t) ? new Date(t).toISOString() : null;
return Number.isFinite(t) ? new Date(t).toISOString() : null;
};
};
-const tableExists = (db, name) => !!db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);
-const columns = (db, t) => (tableExists(db, t) ? db.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);
+// The snapshot (SQLite, read-only) side.
+const tableExists = (snap, name) => !!snap.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name);
+const columns = (snap, t) => (tableExists(snap, t) ? snap.prepare(`PRAGMA table_info(${t})`).all().map((c) => c.name) : []);
class DryRun extends Error {}
class DryRun extends Error {}
@@ -79,15 +85,15 @@ function billingSnapshotSource(bdb) {
@@ -79,15 +85,15 @@ function billingSnapshotSource(bdb) {
}
}
async function importLive(domain, { live, billingSource = null, resolveLiveUsers, dryRun = false, log = console }) {
async function importLive(domain, { live, billingSource = null, resolveLiveUsers, dryRun = false, log = console }) {
- const { db } = domain;
const started = domain.now();
const started = domain.now();
const runId = prefixedId('timp', started);
const runId = prefixedId('timp', started);
const at = iso(started);
const at = iso(started);
const report = {
const report = {
run_id: runId, dry_run: !!dryRun, started_at: at, source: 'openvibe-live snapshot', billing_source: billingSource ? billingSource.name : null,
run_id: runId, dry_run: !!dryRun, started_at: at, source: 'openvibe-live snapshot', billing_source: billingSource ? billingSource.name : null,
- counts: { imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 },
- holds: [], excluded: [], unlinked: [], reconciliation: null,
+ counts: {}, holds: [], excluded: [], unlinked: [], reconciliation: null,
};
};
+ const fresh = () => ({ imported: 0, held: 0, excluded: 0, refunds_applied: 0, external: 0, goals: 0, unchanged: 0, released: 0, linked_to_billing: 0, unlinked: 0 });
+ Object.assign(report.counts, fresh());
// ── Read the snapshot ────────────────────────────────────
// ── Read the snapshot ────────────────────────────────────
const txns = tableExists(live, 'transactions') ? live.prepare("SELECT * FROM transactions WHERE type IN ('donation', 'refund') ORDER BY id").all() : [];
const txns = tableExists(live, 'transactions') ? live.prepare("SELECT * FROM transactions WHERE type IN ('donation', 'refund') ORDER BY id").all() : [];
@@ -114,25 +120,36 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -114,25 +120,36 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
for (const g of goalRows) if (g.user_id) ids.add(String(g.user_id));
for (const g of goalRows) if (g.user_id) ids.add(String(g.user_id));
const map = ids.size ? await resolveLiveUsers([...ids]) : new Map();
const map = ids.size ? await resolveLiveUsers([...ids]) : new Map();
const subjectOf = (liveId) => { const v = liveId == null ? null : map.get(String(liveId)); return v ? (typeof v === 'string' ? v : v.subject) : null; };
const subjectOf = (liveId) => { const v = liveId == null ? null : map.get(String(liveId)); return v ? (typeof v === 'string' ? v : v.subject) : null; };
- const nameOf = (liveId) => { const u = users.get(String(liveId)); return u ? (u.display_name || u.username) : null; };
+ const nameOf = (liveId) => { const u = users.get(String(liveId)); return u && (u.display_name || u.username) ? storable(u.display_name || u.username) : null; };
report.counts.identities = { resolved: [...ids].filter((i) => subjectOf(i)).length, unmapped: [...ids].filter((i) => !subjectOf(i)).length };
report.counts.identities = { resolved: [...ids].filter((i) => subjectOf(i)).length, unmapped: [...ids].filter((i) => !subjectOf(i)).length };
- const mapRow = db.prepare('SELECT * FROM migration_maps WHERE source = ? AND source_table = ? AND source_id = ?');
- const upsertMap = db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)
- VALUES ('live', @table, @id, @target_type, @target_id, @status, @reason, @billing, @run, @at, @at)
- ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,
- reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);
- const record = (table, id, fields) => upsertMap.run({ table, id: String(id), target_type: null, target_id: null, reason: null, billing: null, run: runId, at, ...fields });
+ let q = null; // the import's transaction
+ const mapRow = (_source, table, id) => q.maybe(sql`SELECT * FROM migration_maps WHERE source = 'live' AND source_table = ${table} AND source_id = ${id}`);
+ const record = (table, id, fields) => {
+ const f = { target_type: null, target_id: null, reason: null, billing: null, ...fields };
+ return q.exec(sql`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, reason, billing_txn_id, run_id, created_at, updated_at)
+ VALUES ('live', ${table}, ${String(id)}, ${f.target_type}, ${f.target_id}, ${f.status}, ${f.reason}, ${f.billing}, ${runId}, ${at}, ${at})
+ ON CONFLICT (source, source_table, source_id) DO UPDATE SET target_type = excluded.target_type, target_id = excluded.target_id, status = excluded.status,
+ reason = excluded.reason, billing_txn_id = COALESCE(excluded.billing_txn_id, migration_maps.billing_txn_id), run_id = excluded.run_id, updated_at = excluded.updated_at`);
+ };
- const run = () => {
+ const run = async (t0) => {
+ q = t0;
+ // A retried attempt (serialization failure) reports from scratch.
+ Object.assign(report.counts, fresh(), { live: report.counts.live, identities: report.counts.identities });
+ report.holds = []; report.excluded = []; report.unlinked = [];
// ── 2. donations ─────────────────────────────────────
// ── 2. donations ─────────────────────────────────────
for (const t of txns.filter((x) => x.type === 'donation')) {
for (const t of txns.filter((x) => x.type === 'donation')) {
- const prev = mapRow.get('live', 'transactions', String(t.id));
+ const prev = await mapRow('live', 'transactions', String(t.id));
if (prev && prev.status !== 'held') {
if (prev && prev.status !== 'held') {
// Re-link to Billing when an earlier run had no Billing transaction for it yet.
// Re-link to Billing when an earlier run had no Billing transaction for it yet.
if (prev.status === 'imported' && !prev.billing_txn_id && billingSource) {
if (prev.status === 'imported' && !prev.billing_txn_id && billingSource) {
const b = billingSource.linkLiveTxn(t.id);
const b = billingSource.linkLiveTxn(t.id);
- if (b) { db.prepare('UPDATE tip_interactions SET billing_txn_id = ? WHERE id = ? AND billing_txn_id IS NULL').run(b, prev.target_id); record('transactions', t.id, { ...pick(prev), billing: b }); report.counts.linked_to_billing++; }
+ if (b) {
+ await q.exec(sql`UPDATE tip_interactions SET billing_txn_id = ${b} WHERE id = ${prev.target_id} AND billing_txn_id IS NULL`);
+ await record('transactions', t.id, { ...pick(prev), billing: b });
+ report.counts.linked_to_billing++;
+ }
}
}
report.counts.unchanged++;
report.counts.unchanged++;
continue;
continue;
@@ -141,33 +158,33 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -141,33 +158,33 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
const creator = subjectOf(t.to_user_id);
const creator = subjectOf(t.to_user_id);
if (t.status !== 'completed' || amount <= 0) {
if (t.status !== 'completed' || amount <= 0) {
const reason = amount <= 0 ? 'non-positive amount' : `status ${t.status}: no balance moved on Live`;
const reason = amount <= 0 ? 'non-positive amount' : `status ${t.status}: no balance moved on Live`;
- record('transactions', t.id, { status: 'excluded', reason });
+ await record('transactions', t.id, { status: 'excluded', reason });
report.counts.excluded++; report.excluded.push({ live_txn: t.id, reason });
report.counts.excluded++; report.excluded.push({ live_txn: t.id, reason });
continue;
continue;
}
}
if (!creator) {
if (!creator) {
const reason = `creator (Live user ${t.to_user_id}) has no Network subject yet`;
const reason = `creator (Live user ${t.to_user_id}) has no Network subject yet`;
- record('transactions', t.id, { status: 'held', reason });
+ await record('transactions', t.id, { status: 'held', reason });
report.counts.held++; report.holds.push({ live_txn: t.id, live_creator: t.to_user_id, amount });
report.counts.held++; report.holds.push({ live_txn: t.id, live_creator: t.to_user_id, amount });
continue;
continue;
}
}
const created = liveTs(t.created_at) || at;
const created = liveTs(t.created_at) || at;
const test = !!(resetAt && created < resetAt);
const test = !!(resetAt && created < resetAt);
const billingTxn = billingSource ? billingSource.linkLiveTxn(t.id) : null;
const billingTxn = billingSource ? billingSource.linkLiveTxn(t.id) : null;
- const message = t.message ? String(t.message).slice(0, 500) : null;
+ const message = t.message ? storable(String(t.message).slice(0, 500)) : null;
const media = /^Media request:/i.test(message || '');
const media = /^Media request:/i.test(message || '');
const site = /^PowerChat tip via site account/i.test(message || '');
const site = /^PowerChat tip via site account/i.test(message || '');
- const i = domain.interactions.insert({
+ const i = await domain.interactions.insert(q, {
id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_subject: subjectOf(t.from_user_id),
id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_subject: subjectOf(t.from_user_id),
supporter_name: nameOf(t.from_user_id) || (site ? 'PowerChat supporter' : null), kind: media ? 'media_request' : 'tip', amount,
supporter_name: nameOf(t.from_user_id) || (site ? 'PowerChat supporter' : null), kind: media ? 'media_request' : 'tip', amount,
message: media || site ? null : message, request: { legacy_message: message, live_stream_id: t.stream_id || null },
message: media || site ? null : message, request: { legacy_message: message, live_stream_id: t.stream_id || null },
funding: site ? 'provider' : 'credit', settlement: 'imported', payment_state: 'settled', delivery_state: 'delivered',
funding: site ? 'provider' : 'credit', settlement: 'imported', payment_state: 'settled', delivery_state: 'delivered',
- test: test ? 1 : 0, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,
+ test, billing_txn_id: billingTxn, provider: site ? 'powerchat' : null, legacy_source: `live:transactions:${t.id}`,
origin: 'import', created_at: created,
origin: 'import', created_at: created,
target: t.stream_id ? { service: 'live', type: 'stream', id: String(t.stream_id) } : null,
target: t.stream_id ? { service: 'live', type: 'stream', id: String(t.stream_id) } : null,
});
});
- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);
- record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });
+ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);
+ await record('transactions', t.id, { status: 'imported', target_type: 'interaction', target_id: i.id, billing: billingTxn, reason: prev ? 'released from hold' : null });
if (prev) report.counts.released++;
if (prev) report.counts.released++;
report.counts.imported++;
report.counts.imported++;
if (billingTxn) report.counts.linked_to_billing++;
if (billingTxn) report.counts.linked_to_billing++;
@@ -176,15 +193,17 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -176,15 +193,17 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
// ── 3. refunds (streamer → donor) ────────────────────
// ── 3. refunds (streamer → donor) ────────────────────
for (const t of txns.filter((x) => x.type === 'refund')) {
for (const t of txns.filter((x) => x.type === 'refund')) {
- const prev = mapRow.get('live', 'transactions', String(t.id));
+ const prev = await mapRow('live', 'transactions', String(t.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const amount = Math.round(Number(t.amount) || 0);
const amount = Math.round(Number(t.amount) || 0);
- if (t.status !== 'completed' || amount <= 0) { record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }
+ if (t.status !== 'completed' || amount <= 0) { await record('transactions', t.id, { status: 'excluded', reason: `refund with status ${t.status}` }); report.counts.excluded++; continue; }
const creator = subjectOf(t.from_user_id);
const creator = subjectOf(t.from_user_id);
const donor = subjectOf(t.to_user_id);
const donor = subjectOf(t.to_user_id);
const created = liveTs(t.created_at) || at;
const created = liveTs(t.created_at) || at;
- const candidates = creator ? db.prepare(`SELECT * FROM tip_interactions WHERE creator_subject = ? AND settlement = 'imported' AND legacy_source LIKE 'live:transactions:%'
- AND (supporter_subject IS ? ) AND created_at <= ? AND amount - reversed_bits > 0 ORDER BY created_at DESC`).all(creator, donor, created) : [];
+ // The donations a refund may apply to, locked in id order (the domain's lock order).
+ const candidates = creator ? (await q.many(sql`SELECT * FROM tip_interactions WHERE creator_subject = ${creator} AND settlement = 'imported'
+ AND starts_with(legacy_source, 'live:transactions:') AND supporter_subject IS NOT DISTINCT FROM ${donor}::text AND created_at <= ${created}
+ AND amount - reversed_bits > 0 ORDER BY id FOR UPDATE`)).sort((a, b) => (a.created_at < b.created_at ? 1 : a.created_at > b.created_at ? -1 : 0)) : [];
// Plan first; apply only when the whole refund can be placed.
// Plan first; apply only when the whole refund can be placed.
let left = amount;
let left = amount;
const plan = [];
const plan = [];
@@ -195,17 +214,18 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -195,17 +214,18 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
left -= take;
left -= take;
}
}
if (left > 0) {
if (left > 0) {
- record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });
+ await record('transactions', t.id, { status: 'held', reason: `refund of ${amount} from Live user ${t.from_user_id} to ${t.to_user_id}: no earlier imported donation left to apply it to` });
report.counts.held++; report.holds.push({ live_txn: t.id, refund: true, amount });
report.counts.held++; report.holds.push({ live_txn: t.id, refund: true, amount });
continue;
continue;
}
}
const applied = [];
const applied = [];
for (const { c, take } of plan) {
for (const { c, take } of plan) {
- const ids = json(c.reversal_txn_ids, []); ids.push(`live:transactions:${t.id}`);
- db.prepare("UPDATE tip_interactions SET reversed_bits = reversed_bits + ?, payment_state = 'reversed', reversed_at = ?, reversal_txn_ids = ? WHERE id = ?").run(take, created, JSON.stringify(ids), c.id);
+ const ids = [...(c.reversal_txn_ids || []), `live:transactions:${t.id}`];
+ await q.exec(sql`UPDATE tip_interactions SET reversed_bits = reversed_bits + ${take}, payment_state = 'reversed', reversed_at = ${created},
+ reversal_txn_ids = ${sql.json(ids)} WHERE id = ${c.id}`);
applied.push({ interaction: c.id, bits: take });
applied.push({ interaction: c.id, bits: take });
}
}
- record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });
+ await record('transactions', t.id, { status: 'imported', target_type: 'refund', target_id: applied.map((a) => a.interaction).join(','), billing: billingSource ? billingSource.linkLiveTxn(t.id) : null });
report.counts.refunds_applied++;
report.counts.refunds_applied++;
}
}
@@ -215,7 +235,7 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -215,7 +235,7 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
for (const c of chatDonations) {
for (const c of chatDonations) {
const m = json(c.metadata, {});
const m = json(c.metadata, {});
if (m.source !== 'powerchat' || m.kind !== 'donation') continue;
if (m.source !== 'powerchat' || m.kind !== 'donation') continue;
- const prev = mapRow.get('live', 'chat_messages', String(c.id));
+ const prev = await mapRow('live', 'chat_messages', String(c.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const amount = Math.round(Number(m.amount) || 0);
const amount = Math.round(Number(m.amount) || 0);
const created = liveTs(c.created_at) || at;
const created = liveTs(c.created_at) || at;
@@ -223,53 +243,55 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -223,53 +243,55 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
&& Math.abs(Date.parse(liveTs(t.created_at) || at) - Date.parse(created)) <= 10 * 60 * 1000);
&& Math.abs(Date.parse(liveTs(t.created_at) || at) - Date.parse(created)) <= 10 * 60 * 1000);
if (twin) {
if (twin) {
usedSite.add(twin.id);
usedSite.add(twin.id);
- record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });
+ await record('chat_messages', c.id, { status: 'excluded', reason: `celebration of site-routed tip (Live transaction ${twin.id})` });
report.counts.excluded++;
report.counts.excluded++;
continue;
continue;
}
}
- if (amount <= 0) { record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }
+ if (amount <= 0) { await record('chat_messages', c.id, { status: 'excluded', reason: 'no amount' }); report.counts.excluded++; continue; }
const creator = subjectOf(c.channel_user_id);
const creator = subjectOf(c.channel_user_id);
if (!creator) {
if (!creator) {
- record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });
+ await record('chat_messages', c.id, { status: 'held', reason: `creator (Live user ${c.channel_user_id}) has no Network subject yet` });
report.counts.held++; report.holds.push({ live_chat_message: c.id, live_creator: c.channel_user_id, amount });
report.counts.held++; report.holds.push({ live_chat_message: c.id, live_creator: c.channel_user_id, amount });
continue;
continue;
}
}
- const i = domain.interactions.insert({
- id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? String(m.username).slice(0, 80) : 'PowerChat supporter',
- kind: 'tip', amount, amount_cents: amount, message: m.message ? String(m.message).slice(0, 500) : null, funding: 'external', settlement: 'external',
- payment_state: 'settled', delivery_state: 'delivered', test: resetAt && created < resetAt ? 1 : 0, provider: 'powerchat',
+ const i = await domain.interactions.insert(q, {
+ id: prefixedId('tint', Date.parse(created)), creator_subject: creator, supporter_name: m.username ? storable(String(m.username).slice(0, 80)) : 'PowerChat supporter',
+ kind: 'tip', amount, amount_cents: amount, message: m.message ? storable(String(m.message).slice(0, 500)) : null, funding: 'external', settlement: 'external',
+ payment_state: 'settled', delivery_state: 'delivered', test: !!(resetAt && created < resetAt), provider: 'powerchat',
provider_ref: `live-chat:${c.id}`, legacy_source: `live:chat_messages:${c.id}`, origin: 'import', created_at: created,
provider_ref: `live-chat:${c.id}`, legacy_source: `live:chat_messages:${c.id}`, origin: 'import', created_at: created,
});
});
- db.prepare('UPDATE tip_interactions SET settled_at = ?, delivered_at = ? WHERE id = ?').run(created, created, i.id);
- record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });
+ await q.exec(sql`UPDATE tip_interactions SET settled_at = ${created}, delivered_at = ${created} WHERE id = ${i.id}`);
+ await record('chat_messages', c.id, { status: 'imported', target_type: 'interaction', target_id: i.id });
report.counts.external++;
report.counts.external++;
}
}
// ── 5. goals ─────────────────────────────────────────
// ── 5. goals ─────────────────────────────────────────
for (const g of goalRows) {
for (const g of goalRows) {
- const prev = mapRow.get('live', 'donation_goals', String(g.id));
+ const prev = await mapRow('live', 'donation_goals', String(g.id));
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
if (prev && prev.status !== 'held') { report.counts.unchanged++; continue; }
const creator = subjectOf(g.user_id);
const creator = subjectOf(g.user_id);
- if (!creator) { record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }
+ if (!creator) { await record('donation_goals', g.id, { status: 'held', reason: `creator (Live user ${g.user_id}) has no Network subject yet` }); report.counts.held++; continue; }
const target = Math.round(Number(g.target_amount) || 0);
const target = Math.round(Number(g.target_amount) || 0);
- if (target <= 0 || !g.title) { record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }
+ if (target <= 0 || !g.title) { await record('donation_goals', g.id, { status: 'excluded', reason: 'no title or target' }); report.counts.excluded++; continue; }
const created = liveTs(g.created_at) || at;
const created = liveTs(g.created_at) || at;
const id = prefixedId('tgoal', Date.parse(created));
const id = prefixedId('tgoal', Date.parse(created));
- db.prepare(`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)
- VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(id, creator, String(g.title).slice(0, 120), target,
- /^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null, g.is_active ? 'active' : 'closed', Number(g.sort_order) || 0,
- liveTs(g.reached_at), g.is_active ? null : at, Math.max(0, Math.round(Number(g.current_amount) || 0)), `live:donation_goals:${g.id}`, created, at);
- record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });
+ await q.exec(sql`INSERT INTO tip_goals (id, creator_subject, title, target_amount, image_url, status, sort_order, reached_at, closed_at, opening_amount, legacy_source, created_at, updated_at)
+ VALUES (${id}, ${creator}, ${storable(String(g.title).slice(0, 120))}, ${target}, ${/^https:\/\//.test(String(g.image_url || '')) ? g.image_url : null},
+ ${g.is_active ? 'active' : 'closed'}, ${Number(g.sort_order) || 0}, ${liveTs(g.reached_at)}, ${g.is_active ? null : at},
+ ${Math.max(0, Math.round(Number(g.current_amount) || 0))}, ${`live:donation_goals:${g.id}`}, ${created}, ${at})`);
+ await record('donation_goals', g.id, { status: 'imported', target_type: 'goal', target_id: id, reason: `opening_amount ${Math.round(Number(g.current_amount) || 0)} carried over from Live's current_amount` });
report.counts.goals++;
report.counts.goals++;
}
}
- if (dryRun) { report.reconciliation = reconcile(domain, billingSource); throw new DryRun(); }
+ if (dryRun) { report.reconciliation = await reconcile(domain, billingSource, q); throw new DryRun(); }
};
};
- try { db.transaction(run)(); } catch (e) { if (!(e instanceof DryRun)) throw e; }
- if (!dryRun) report.reconciliation = reconcile(domain, billingSource);
+ // Serializable while the service writes too: a conflict retries the whole run (idempotent), up to 10 times.
+ try { await domain.tx(run, { ...domain.MONEY, retries: 10 }); } catch (e) { if (!(e instanceof DryRun)) throw e; }
+ if (!dryRun) report.reconciliation = await reconcile(domain, billingSource);
report.finished_at = iso(domain.now());
report.finished_at = iso(domain.now());
- db.prepare('INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report) VALUES (?, ?, ?, ?, ?, ?)').run(runId, 'live', dryRun ? 1 : 0, report.started_at, report.finished_at, JSON.stringify(report));
+ await domain.db.exec(sql`INSERT INTO import_runs (id, source, dry_run, started_at, finished_at, report)
+ VALUES (${runId}, 'live', ${!!dryRun}, ${report.started_at}, ${report.finished_at}, ${sql.json(report)})`);
if (log && log.log) log.log(`[Tips] import ${runId}${dryRun ? ' (dry run, rolled back)' : ''}: ${report.counts.imported} tips, ${report.counts.external} external, ${report.counts.goals} goals, ${report.counts.held} held, ${report.counts.excluded} excluded; reconciliation ${report.reconciliation ? (report.reconciliation.ok ? 'OK' : 'MISMATCH') : 'skipped (no Billing source)'}`);
if (log && log.log) log.log(`[Tips] import ${runId}${dryRun ? ' (dry run, rolled back)' : ''}: ${report.counts.imported} tips, ${report.counts.external} external, ${report.counts.goals} goals, ${report.counts.held} held, ${report.counts.excluded} excluded; reconciliation ${report.reconciliation ? (report.reconciliation.ok ? 'OK' : 'MISMATCH') : 'skipped (no Billing source)'}`);
return report;
return report;
}
}
@@ -277,11 +299,11 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
@@ -277,11 +299,11 @@ async function importLive(domain, { live, billingSource = null, resolveLiveUsers
function pick(row) { return { status: row.status, target_type: row.target_type, target_id: row.target_id, reason: row.reason }; }
function pick(row) { return { status: row.status, target_type: row.target_type, target_id: row.target_id, reason: row.reason }; }
/** Per-creator totals: Tips (settled via Billing or imported, non-test, minus refunds) vs Billing. */
/** Per-creator totals: Tips (settled via Billing or imported, non-test, minus refunds) vs Billing. */
-function reconcile(domain, billingSource) {
+async function reconcile(domain, billingSource, q = domain.db) {
if (!billingSource) return null;
if (!billingSource) return null;
const billing = billingSource.creatorTotals();
const billing = billingSource.creatorTotals();
- const tips = new Map(domain.db.prepare(`SELECT creator_subject AS s, SUM(amount - reversed_bits) AS n FROM tip_interactions
- WHERE test = 0 AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`).all().map((r) => [r.s, r.n]));
+ const tips = new Map((await q.many(sql`SELECT creator_subject AS s, SUM(amount - reversed_bits)::bigint AS n FROM tip_interactions
+ WHERE NOT test AND settlement IN ('billing', 'imported') AND payment_state IN ('settled', 'reversed') GROUP BY creator_subject`)).map((r) => [r.s, r.n]));
const creators = [...new Set([...billing.keys(), ...tips.keys()])].sort();
const creators = [...new Set([...billing.keys(), ...tips.keys()])].sort();
const rows = creators.map((s) => ({ creator: s, tips: tips.get(s) || 0, billing: billing.get(s) || 0, difference: (tips.get(s) || 0) - (billing.get(s) || 0) }));
const rows = creators.map((s) => ({ creator: s, tips: tips.get(s) || 0, billing: billing.get(s) || 0, difference: (tips.get(s) || 0) - (billing.get(s) || 0) }));
const mismatches = rows.filter((r) => r.difference !== 0);
const mismatches = rows.filter((r) => r.difference !== 0);
@@ -4,44 +4,73 @@
@@ -4,44 +4,73 @@
* OpenVibe.Tips — process entry. `node server/index.js`
* OpenVibe.Tips — process entry. `node server/index.js`
* Listens on PORT (4610) behind nginx (openvibe.tips); see deploy/.
* Listens on PORT (4610) behind nginx (openvibe.tips); see deploy/.
*
*
- * Background jobs (TIPS_JOBS=off disables them): the effects worker (chat delivery with retries),
- * due Billing transfers (funded checkouts, retries after Billing was unreachable), the overlay
- * delivery window (pending → failed), stored API answers older than a week, and the events outbox
- * relay when EVENTS_URL is set.
+ * Boot (ADR-035): apply migrations/ with the owner role (DATABASE_DIRECT_URL, one direct connection,
+ * closed afterwards), then serve on the pooled runtime role (DATABASE_URL, through PgBouncer) with
+ * Valkey (VALKEY_URL) for what the processes share. Several processes may start together: the
+ * migration run is serialised by an advisory lock.
+ *
+ * Background jobs (TIPS_JOBS=off disables them), in every process: the effects worker (chat delivery
+ * with retries), due Billing transfers (funded checkouts, retries after Billing was unreachable), the
+ * overlay delivery window (pending → failed), stored API answers and sent events older than a week,
+ * and the events outbox relay when EVENTS_URL is set. Running them in several processes is safe: the
+ * effects, the transfers, the sweep and the relay claim their rows (FOR UPDATE SKIP LOCKED, with a
+ * lease), and the prunes are idempotent deletes.
*/
*/
const { loadConfig } = require('./config');
const { loadConfig } = require('./config');
+const { openDb, migrate } = require('./db');
const { createApp } = require('./app');
const { createApp } = require('./app');
+const { createValkey } = require('openvibe-sdk/valkey');
+const { createRegistry } = require('openvibe-shared/metrics');
-const config = loadConfig();
-const app = createApp({ config });
-const { domain, keys, outbox } = app.locals;
-keys.start();
+async function main() {
+ const config = loadConfig();
+ const registry = createRegistry();
+ const db = openDb(config, { registry });
+ const m = await migrate(config, { serving: db });
+ if (m.held.length) console.warn(`[Tips] migrations held: ${m.held.map((h) => `${h.id} (${h.reason})`).join('; ')}`);
+ const valkey = createValkey({ url: config.valkey.url, prefix: config.valkey.prefix });
-const timers = [];
-if (config.jobs.enabled) {
- const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };
- every(config.jobs.intervalMs, () => domain.effects.drain());
- every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());
- every(30_000, () => domain.overlays.sweepFailed());
- every(6 * 3600 * 1000, () => outbox.outbox.prune());
- every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(domain.db));
- outbox.start();
-}
+ const app = createApp({ config, db, valkey, registry });
+ const { domain, keys, outbox } = app.locals;
+ keys.start();
-const server = app.listen(config.port, config.host, () => {
- console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (db ${config.dbPath})`);
- console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);
-});
-server.keepAliveTimeout = 65_000;
+ const timers = [];
+ if (config.jobs.enabled) {
+ const every = (ms, fn) => { const t = setInterval(() => { Promise.resolve().then(fn).catch((e) => console.warn('[Tips] job:', e.message)); }, ms); t.unref(); timers.push(t); };
+ every(config.jobs.intervalMs, () => domain.effects.drain());
+ every(Math.max(config.jobs.intervalMs, 5000), () => domain.interactions.processDueTransfers());
+ every(30_000, () => domain.overlays.sweepFailed());
+ every(6 * 3600 * 1000, () => outbox.outbox.prune());
+ every(6 * 3600 * 1000, () => require('./api/idempotency').pruneAnswers(db));
+ outbox.start();
+ }
-function shutdown(signal) {
- console.log(`[Tips] ${signal} — closing`);
- timers.forEach(clearInterval);
- domain.overlays.closeAll();
- outbox.stop();
- keys.stop();
- server.close(() => { try { domain.db.close(); } catch { /* */ } process.exit(0); });
- setTimeout(() => process.exit(0), 5000).unref();
+ const server = app.listen(config.port, config.host, () => {
+ console.log(`[Tips] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (store ${db.store}; valkey ${valkey ? 'on' : 'off: per process'})`);
+ console.log(`[Tips] billing ${config.billing.url}; chat adapter ${config.chat.adapter}; events relay ${outbox.enabled ? `→ ${config.events.url}` : 'off (outbox accumulates)'}; billing events ${config.events.webhookSecrets.length ? 'accepted' : 'not accepted (TIPS_EVENTS_SECRET unset)'}`);
+ });
+ server.keepAliveTimeout = 65_000;
+
+ let closing = false;
+ async function shutdown(signal) {
+ if (closing) return;
+ closing = true;
+ console.log(`[Tips] ${signal} — closing`);
+ setTimeout(() => process.exit(0), 5000).unref();
+ timers.forEach(clearInterval);
+ keys.stop();
+ await domain.overlays.close().catch(() => {});
+ await outbox.stop();
+ await new Promise((resolve) => server.close(resolve));
+ await db.close().catch(() => {});
+ if (valkey) await valkey.close().catch(() => {});
+ process.exit(0);
+ }
+ process.on('SIGTERM', () => { shutdown('SIGTERM'); });
+ process.on('SIGINT', () => { shutdown('SIGINT'); });
}
}
-process.on('SIGTERM', () => shutdown('SIGTERM'));
-process.on('SIGINT', () => shutdown('SIGINT'));
+
+main().catch((e) => {
+ console.error(`[Tips] could not start: ${e.message}`);
+ process.exit(1);
+});
@@ -3,7 +3,12 @@
@@ -3,7 +3,12 @@
* Track O: truthful readiness for GET /api/ready and the Tips gauges on GET /metrics
* Track O: truthful readiness for GET /api/ready and the Tips gauges on GET /metrics
* (openvibe-shared/ready and openvibe-shared/metrics).
* (openvibe-shared/ready and openvibe-shared/metrics).
*
*
- * db required a real query on Tips' SQLite (the settings row is there and answers)
+ * db required a real round trip to PostgreSQL through the pool (db.ready(): the store
+ * that answered — postgresql, or pglite in development and tests — and the
+ * pool's state)
+ * valkey optional a real PING (valkey.ready()). Without it (VALKEY_URL unset: skipped, or
+ * down: degraded) per-actor limits, overlay fan-out and stream slots are
+ * counted in each process, which is right for one process only
* network_jwks optional the Network signing key has loaded. Without it pages, overlays and the
* network_jwks optional the Network signing key has loaded. Without it pages, overlays and the
* signed Billing webhook still work, but no service or user token can be
* signed Billing webhook still work, but no service or user token can be
* verified (API calls and sign-in fail), so it degrades rather than fails
* verified (API calls and sign-in fail), so it degrades rather than fails
@@ -13,12 +18,16 @@
@@ -13,12 +18,16 @@
* EVENTS_URL or the client secret unset) tips.* events wait in the outbox
* EVENTS_URL or the client secret unset) tips.* events wait in the outbox
*
*
* Gauges: pending deliveries (chat/TTS/media effects waiting for their adapter, overlay alerts no
* Gauges: pending deliveries (chat/TTS/media effects waiting for their adapter, overlay alerts no
- * overlay has shown yet) and the events outbox backlog. Counts only, never subjects.
+ * overlay has shown yet) and the events outbox backlog. Counts only, never subjects. They are read in
+ * one query just before a direct scrape renders (refresh()); a scrape whose read failed leaves them
+ * out rather than reporting stale or invented values.
*/
*/
-const { createReadiness } = require('openvibe-shared/ready');
+const { sql } = require('openvibe-sdk/db');
+const { createReadiness, skip } = require('openvibe-shared/ready');
+const { TABLE: OUTBOX } = require('./events/outbox');
const PING_TTL_MS = 15_000;
const PING_TTL_MS = 15_000;
-// interaction_effects.effect (the CHECK constraint in db.js): each reported, 0 when none waits.
+// interaction_effects.effect (the CHECK constraint in migrations/0001_initial.sql): each reported, 0 when none waits.
const EFFECTS = ['chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert'];
const EFFECTS = ['chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert'];
function probe(url, fetchImpl) {
function probe(url, fetchImpl) {
@@ -29,7 +38,7 @@ function probe(url, fetchImpl) {
@@ -29,7 +38,7 @@ function probe(url, fetchImpl) {
};
};
}
}
-function createTipsReadiness({ db, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {
+function createTipsReadiness({ db, valkey = null, keys, config, outbox, release = null, fetchImpl = globalThis.fetch }) {
const events = outbox.enabled
const events = outbox.enabled
? probe(`${config.events.url}/api/health`, fetchImpl)
? probe(`${config.events.url}/api/health`, fetchImpl)
: () => 'events relay off (EVENTS_URL or OV_OAUTH_CLIENT_SECRET unset): tips.* events wait in the outbox';
: () => 'events relay off (EVENTS_URL or OV_OAUTH_CLIENT_SECRET unset): tips.* events wait in the outbox';
@@ -37,45 +46,56 @@ function createTipsReadiness({ db, keys, config, outbox, release = null, fetchIm
@@ -37,45 +46,56 @@ function createTipsReadiness({ db, keys, config, outbox, release = null, fetchIm
service: 'tips',
service: 'tips',
release,
release,
checks: [
checks: [
- { name: 'db', required: true, check: () => (db.prepare('SELECT id FROM settings WHERE id = 1').get() ? true : 'database has no settings row') },
+ { name: 'db', required: true, check: () => db.ready() },
+ { name: 'valkey', required: false, check: () => (valkey ? valkey.ready() : skip('VALKEY_URL unset: limits, overlay fan-out and stream slots are per process (one process only)')) },
{ name: 'network_jwks', required: false, check: () => (keys.get() ? true : 'Network signing key not loaded yet: tokens cannot be verified') },
{ name: 'network_jwks', required: false, check: () => (keys.get() ? true : 'Network signing key not loaded yet: tokens cannot be verified') },
{ name: 'billing', required: false, cacheMs: PING_TTL_MS, timeoutMs: 2500, check: probe(`${config.billing.url}/api/health`, fetchImpl) },
{ name: 'billing', required: false, cacheMs: PING_TTL_MS, timeoutMs: 2500, check: probe(`${config.billing.url}/api/health`, fetchImpl) },
{ name: 'events', required: false, cacheMs: outbox.enabled ? PING_TTL_MS : 0, timeoutMs: 2500, check: events },
{ name: 'events', required: false, cacheMs: outbox.enabled ? PING_TTL_MS : 0, timeoutMs: 2500, check: events },
],
],
- details: (body) => ({
- events_outbox: body.checks.db.status === 'ok' ? outbox.status() : null,
- billing_events_accepted: config.events.webhookSecrets.length > 0,
- pending_deliveries: body.checks.db.status === 'ok' ? pendingDeliveries(db) : null,
- }),
+ details: async (body) => (body.checks.db.status === 'ok'
+ ? { events_outbox: await outbox.status(), billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: await pendingDeliveries(db) }
+ : { events_outbox: null, billing_events_accepted: config.events.webhookSecrets.length > 0, pending_deliveries: null }),
});
});
}
}
-function pendingDeliveries(db) {
- const effects = {};
- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) effects[r.effect] = r.n;
- const overlay = db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n;
- return { effects, overlay };
+async function pendingDeliveries(db) {
+ const r = await db.one(sql`SELECT
+ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,
+ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay`);
+ return { effects: r.effects, overlay: r.overlay };
}
}
-/** Tips gauges on the openvibe-shared/metrics registry. */
-function registerTipsGauges(registry, { db, outbox, now = () => Date.now() }) {
+/** Tips gauges on the openvibe-shared/metrics registry; refresh() reads them all in one query. */
+function registerTipsGauges(registry, { db, now = () => Date.now(), log = console }) {
+ let snap = null;
+ const read = (f) => () => (snap ? f(snap) : undefined);
registry.gauge({
registry.gauge({
name: 'tips_effects_pending', help: 'Chat, TTS, media and alert effects waiting for their delivery adapter, by effect', labelNames: ['effect'],
name: 'tips_effects_pending', help: 'Chat, TTS, media and alert effects waiting for their delivery adapter, by effect', labelNames: ['effect'],
- collect: () => {
- const n = Object.fromEntries(EFFECTS.map((e) => [e, 0]));
- for (const r of db.prepare("SELECT effect, COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect").all()) n[r.effect] = r.n;
- return Object.entries(n).map(([effect, value]) => ({ labels: { effect }, value }));
- },
+ collect: read((s) => EFFECTS.map((effect) => ({ labels: { effect }, value: Number(s.effects[effect] || 0) }))),
});
});
registry.gauge({
registry.gauge({
name: 'tips_effects_due', help: 'Queued effects whose next attempt is due now (a growing number means the worker is behind)',
name: 'tips_effects_due', help: 'Queued effects whose next attempt is due now (a growing number means the worker is behind)',
- collect: () => db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ?").get(now()).n,
- });
- registry.gauge({
- name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet',
- collect: () => db.prepare("SELECT COUNT(*) AS n FROM overlay_deliveries WHERE status = 'pending'").get().n,
+ collect: read((s) => s.due),
});
});
- registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: () => outbox.status().pending });
+ registry.gauge({ name: 'tips_overlay_deliveries_pending', help: 'Overlay alerts and goal updates no overlay has shown yet', collect: read((s) => s.overlay) });
+ registry.gauge({ name: 'tips_outbox_pending', help: 'tips.* events waiting in the outbox', collect: read((s) => s.outbox) });
+ return {
+ async refresh() {
+ try {
+ snap = await Promise.race([
+ db.one(sql`SELECT
+ (SELECT COALESCE(jsonb_object_agg(effect, n), '{}') FROM (SELECT effect, count(*) AS n FROM interaction_effects WHERE state = 'queued' GROUP BY effect) e) AS effects,
+ (SELECT count(*) FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= ${now()}) AS due,
+ (SELECT count(*) FROM overlay_deliveries WHERE status = 'pending') AS overlay,
+ (SELECT count(*) FROM ${sql.ident(OUTBOX)} WHERE sent_at IS NULL AND rejected_at IS NULL) AS outbox`),
+ new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 2000).unref()),
+ ]);
+ } catch (e) {
+ snap = null;
+ log.warn(`[Tips] gauges not read: ${e.message}`);
+ }
+ },
+ };
}
}
module.exports = { createTipsReadiness, registerTipsGauges };
module.exports = { createTipsReadiness, registerTipsGauges };
@@ -19,6 +19,7 @@ function fail(status, code, detail, extra) { throw new TipsError(status, code, d
@@ -19,6 +19,7 @@ function fail(status, code, detail, extra) { throw new TipsError(status, code, d
const prefixedId = (prefix, ms = Date.now()) => `${prefix}_${ids.ulid(ms)}`;
const prefixedId = (prefix, ms = Date.now()) => `${prefix}_${ids.ulid(ms)}`;
const iso = (ms) => new Date(ms).toISOString();
const iso = (ms) => new Date(ms).toISOString();
const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');
const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');
+/** JSON text (a SQLite snapshot's column) → value, or `d`. PostgreSQL jsonb columns come back parsed. */
const json = (v, d) => { if (v == null || v === '') return d; try { return JSON.parse(v); } catch { return d; } };
const json = (v, d) => { if (v == null || v === '') return d; try { return JSON.parse(v); } catch { return d; } };
function positiveInt(v, field, max) {
function positiveInt(v, field, max) {
@@ -32,10 +33,16 @@ function positiveInt(v, field, max) {
@@ -32,10 +33,16 @@ function positiveInt(v, field, max) {
// from the creator's filter and turn text around on stream (docs/threat-review.md).
// from the creator's filter and turn text around on stream (docs/threat-review.md).
const INVISIBLE = /[\u00AD\u200B-\u200F\u202A-\u202E\u2060-\u2064\u2066-\u2069\uFEFF]/g;
const INVISIBLE = /[\u00AD\u200B-\u200F\u202A-\u202E\u2060-\u2064\u2066-\u2069\uFEFF]/g;
-/** Optional trimmed text: null when empty, refused when longer than max. Invisible characters are dropped. */
+// PostgreSQL stores no NUL character in text or jsonb, and jsonb refuses an unpaired surrogate (SQLite
+// took both): every text from outside goes through storable() before it reaches the database.
+const NUL = /\u0000/g;
+/** A string PostgreSQL can store: NUL characters dropped, unpaired surrogates replaced by U+FFFD. */
+const storable = (v) => String(v).toWellFormed().replace(NUL, '');
+
+/** Optional trimmed text: null when empty, refused when longer than max. Invisible and NUL characters are dropped. */
function text(v, field, max) {
function text(v, field, max) {
if (v == null) return null;
if (v == null) return null;
- const s = String(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();
+ const s = storable(v).replace(INVISIBLE, '').replace(/\r\n?/g, '\n').trim();
if (!s) return null;
if (!s) return null;
if (s.length > max) fail(422, 'tips.text_too_long', `${field} must be at most ${max} characters`);
if (s.length > max) fail(422, 'tips.text_too_long', `${field} must be at most ${max} characters`);
return s;
return s;
@@ -53,14 +60,36 @@ function userSubject(v, field = 'subject') {
@@ -53,14 +60,36 @@ function userSubject(v, field = 'subject') {
function entityRef(v, field = 'target') {
function entityRef(v, field = 'target') {
if (v == null) return null;
if (v == null) return null;
if (!validate('common.entity-ref@1', v).valid) fail(422, 'tips.invalid_input', `${field} must be an EntityRef ({ service, type, id })`);
if (!validate('common.entity-ref@1', v).valid) fail(422, 'tips.invalid_input', `${field} must be an EntityRef ({ service, type, id })`);
- return { service: v.service, type: v.type, id: String(v.id) };
+ return { service: v.service, type: v.type, id: storable(v.id) };
}
}
/** The display name a supporter chose, kept short, without markup characters. */
/** The display name a supporter chose, kept short, without markup characters. */
function displayName(v) {
function displayName(v) {
if (v == null) return null;
if (v == null) return null;
- const s = String(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80);
+ const s = storable(v).replace(INVISIBLE, '').replace(/[<>\u0000-\u001f]/g, '').trim().slice(0, 80).toWellFormed();
return s || null;
return s || null;
}
}
-module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, userSubject, entityRef, displayName, INVISIBLE };
+/** A keyset cursor: base64url JSON [created_at ISO, id] → the pair, or 422; cursorOf(row) makes one. */
+const ISO_RE = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(\.\d{1,6})?Z$/;
+function readCursor(cursor) {
+ let cur = null;
+ try { cur = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8')); } catch { /* refused below */ }
+ if (!Array.isArray(cur) || typeof cur[0] !== 'string' || !ISO_RE.test(cur[0]) || typeof cur[1] !== 'string' || cur[1].length > 64) fail(422, 'tips.invalid_input', 'bad cursor');
+ return cur;
+}
+const cursorOf = (row) => Buffer.from(JSON.stringify([row.created_at, row.id])).toString('base64url');
+
+/**
+ * A database refusal the caller caused (text PostgreSQL cannot store: a NUL character, a malformed
+ * sequence) → the TipsError to answer with, else null (a real failure: 500).
+ */
+function inputError(e) {
+ const code = e && (e.code || (e.cause && e.cause.code));
+ // 22021/22P05: a character the encoding or jsonb refuses; 22P02, 22007, 22008: a value that is not
+ // what its column takes (an unreadable number, JSON text or time).
+ return ['22021', '22P05', '22P02', '22007', '22008'].includes(code)
+ ? new TipsError(422, 'tips.invalid_input', 'the request carries a value that cannot be stored (such as a NUL character)') : null;
+}
+
+module.exports = { TipsError, fail, prefixedId, iso, sha256, json, positiveInt, text, storable, userSubject, entityRef, displayName, inputError, readCursor, cursorOf, INVISIBLE };
new file mode 100644
new file mode 100644
@@ -0,0 +1,31 @@
@@ -0,0 +1,31 @@
+'use strict';
+
+/**
+ * Express 4 ignores a rejected promise, so an async handler's error would never reach the error
+ * handler (the request would hang). A router made here wraps every handler it is given: a returned
+ * promise that rejects is passed to next(err). Error handlers (four arguments) are left as they are.
+ *
+ * const r = asyncRouter(express.Router());
+ * r.get('/x', async (req, res) => { res.json(await something()); });
+ */
+const METHODS = ['all', 'get', 'post', 'put', 'patch', 'delete', 'head', 'options', 'use'];
+
+function catching(fn) {
+ if (typeof fn !== 'function' || fn.length >= 4) return fn;
+ return function asyncHandler(req, res, next) {
+ let out;
+ try { out = fn(req, res, next); } catch (e) { return next(e); }
+ if (out && typeof out.then === 'function') out.then(undefined, next);
+ return undefined;
+ };
+}
+
+function asyncRouter(router) {
+ for (const m of METHODS) {
+ const orig = router[m].bind(router);
+ router[m] = (...args) => orig(...args.map((a) => (Array.isArray(a) ? a.map(catching) : catching(a))));
+ }
+ return router;
+}
+
+module.exports = { asyncRouter, catching };
@@ -25,8 +25,10 @@ const crypto = require('crypto');
@@ -25,8 +25,10 @@ const crypto = require('crypto');
const ovServe = require('openvibe-shared/serve');
const ovServe = require('openvibe-shared/serve');
const frame = require('openvibe-shared/frame');
const frame = require('openvibe-shared/frame');
const express = require('express');
const express = require('express');
-const { TipsError, json: parseJson } = require('../util');
+const { sql } = require('openvibe-sdk/db');
+const { TipsError } = require('../util');
const { viewerMiddleware } = require('./session');
const { viewerMiddleware } = require('./session');
+const { asyncRouter } = require('./async-router');
const pages = require('./pages');
const pages = require('./pages');
const { esc, asset } = require('./layout');
const { esc, asset } = require('./layout');
const { PAGE_DEFAULTS } = require('../domain/profiles');
const { PAGE_DEFAULTS } = require('../domain/profiles');
@@ -37,7 +39,8 @@ const MOD_FLASH = new Set(['Hidden', 'Shown', 'You are now a moderator']);
@@ -37,7 +39,8 @@ const MOD_FLASH = new Set(['Hidden', 'Shown', 'You are now a moderator']);
const RECEIPT_FLASH = new Set(['Your data was erased from your tips']);
const RECEIPT_FLASH = new Set(['Your data was erased from your tips']);
function createWebRoutes({ domain, config, layout, userAuth }) {
function createWebRoutes({ domain, config, layout, userAuth }) {
- const r = express.Router();
+ // Every handler below may be async: its errors reach the app's error handler (the 500 page).
+ const r = asyncRouter(express.Router());
const { profiles, goals, interactions, overlays, moderation, db } = domain;
const { profiles, goals, interactions, overlays, moderation, db } = domain;
const withViewer = viewerMiddleware(userAuth);
const withViewer = viewerMiddleware(userAuth);
const form = express.urlencoded({ extended: false, limit: '32kb' });
const form = express.urlencoded({ extended: false, limit: '32kb' });
@@ -64,62 +67,66 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -64,62 +67,66 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
const bool = (v) => v === '1' || v === 'on' || v === true;
const bool = (v) => v === '1' || v === 'on' || v === true;
// ── Home, robots, sitemap ────────────────────────────────
// ── Home, robots, sitemap ────────────────────────────────
- r.get('/', withViewer, (req, res) => {
- const creators = db.prepare(`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY updated_at DESC LIMIT 24`).all();
+ r.get('/', withViewer, async (req, res) => {
+ const creators = await db.many(sql`SELECT handle, display_name, avatar_url FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`);
html(res, pageFor(req, { active: 'home', canonicalPath: '/', body: pages.home({ creators }) }));
html(res, pageFor(req, { active: 'home', canonicalPath: '/', body: pages.home({ creators }) }));
});
});
// What shipped on OpenVibe.Tips: the shared update log every OpenVibe site has.
// What shipped on OpenVibe.Tips: the shared update log every OpenVibe site has.
r.get('/updates', withViewer, (req, res) => html(res, pageFor(req, { canonicalPath: '/updates', title: 'What shipped on OpenVibe.Tips', body: frame.updatesBody({ service: 'tips', siteName: 'OpenVibe.Tips' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` })));
r.get('/updates', withViewer, (req, res) => html(res, pageFor(req, { canonicalPath: '/updates', title: 'What shipped on OpenVibe.Tips', body: frame.updatesBody({ service: 'tips', siteName: 'OpenVibe.Tips' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` })));
r.get('/robots.txt', (req, res) => res.type('text/plain').set('Cache-Control', 'public, max-age=3600').send(
r.get('/robots.txt', (req, res) => res.type('text/plain').set('Cache-Control', 'public, max-age=3600').send(
`User-agent: *\nAllow: /\nDisallow: /dashboard\nDisallow: /receipts\nDisallow: /overlay/\nDisallow: /moderate/\nDisallow: /auth/\nDisallow: /api/\nDisallow: /internal/\nSitemap: ${config.baseUrl}/sitemap.xml\n`));
`User-agent: *\nAllow: /\nDisallow: /dashboard\nDisallow: /receipts\nDisallow: /overlay/\nDisallow: /moderate/\nDisallow: /auth/\nDisallow: /api/\nDisallow: /internal/\nSitemap: ${config.baseUrl}/sitemap.xml\n`));
- r.get('/sitemap.xml', (req, res) => {
- const rows = db.prepare('SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled = 1 ORDER BY handle').all();
+ r.get('/sitemap.xml', async (req, res) => {
+ const rows = await db.many(sql`SELECT handle, updated_at FROM creator_tip_profiles WHERE page_enabled ORDER BY handle`);
const urls = [`<url><loc>${esc(config.baseUrl)}/</loc></url>`, ...rows.map((p) => `<url><loc>${esc(`${config.baseUrl}/${p.handle}`)}</loc><lastmod>${esc(p.updated_at.slice(0, 10))}</lastmod></url>`)];
const urls = [`<url><loc>${esc(config.baseUrl)}/</loc></url>`, ...rows.map((p) => `<url><loc>${esc(`${config.baseUrl}/${p.handle}`)}</loc><lastmod>${esc(p.updated_at.slice(0, 10))}</lastmod></url>`)];
res.type('application/xml').set('Cache-Control', 'public, max-age=900').send(`<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls.join('')}</urlset>\n`);
res.type('application/xml').set('Cache-Control', 'public, max-age=900').send(`<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls.join('')}</urlset>\n`);
});
});
// ── Receipts ─────────────────────────────────────────────
// ── Receipts ─────────────────────────────────────────────
- const decorate = (i) => {
- const p = profiles.bySubject(i.creator_subject);
- return { ...interactions.present(i, { viewer: 'supporter' }), creator_handle: p && p.handle, creator_name: p && p.display_name };
- };
- r.get('/receipts', withViewer, (req, res) => {
+ /** The supporter's view of each receipt, with its creator's handle and name (one query for all creators). */
+ async function decorate(rows) {
+ const [views, byCreator] = await Promise.all([
+ interactions.presentMany(db, rows, { viewer: 'supporter' }),
+ profiles.bySubjects(db, rows.map((i) => i.creator_subject)),
+ ]);
+ return views.map((v, k) => { const p = byCreator.get(rows[k].creator_subject); return { ...v, creator_handle: p && p.handle, creator_name: p && p.display_name }; });
+ }
+ r.get('/receipts', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
- const out = interactions.list({ supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });
+ const out = await interactions.list(db, { supporter: req.viewer.subject, cursor: req.query.cursor, limit: 50 });
const flash = RECEIPT_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
const flash = RECEIPT_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
- html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: out.rows.map(decorate), next: out.next_cursor, flash }) }));
+ return html(res, pageFor(req, { title: 'Receipts', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts', body: pages.receiptsPage({ rows: await decorate(out.rows), next: out.next_cursor, flash }) }));
});
});
// The supporter's own data: a download, and erasure (privacy.js, interactions.erase()).
// The supporter's own data: a download, and erasure (privacy.js, interactions.erase()).
- r.get('/receipts/export', withViewer, (req, res) => {
+ r.get('/receipts/export', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts')}`);
- const day = new Date(domain.now()).toISOString().slice(0, 10);
- res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${day}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })
- .type('application/json').send(`${JSON.stringify(interactions.exportFor(req.viewer.subject), null, 2)}\n`);
+ const today = new Date(domain.now()).toISOString().slice(0, 10);
+ return res.status(200).set({ 'Cache-Control': 'no-store', 'Content-Disposition': `attachment; filename="openvibe-tips-${today}.json"`, 'X-Robots-Tag': 'noindex, nofollow' })
+ .type('application/json').send(`${JSON.stringify(await interactions.exportFor(req.viewer.subject), null, 2)}\n`);
});
});
- function renderErase(req, res, status = 200) {
- const rows = db.prepare("SELECT payment_state FROM tip_interactions WHERE supporter_subject = ?").all(req.viewer.subject);
+ async function renderErase(req, res, status = 200) {
+ const c = await db.one(sql`SELECT count(*) AS n, count(*) FILTER (WHERE payment_state = 'pending') AS pending FROM tip_interactions WHERE supporter_subject = ${req.viewer.subject}`);
html(res, pageFor(req, {
html(res, pageFor(req, {
title: 'Erase your data', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts/erase',
title: 'Erase your data', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: '/receipts/erase',
- body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: rows.length, pending: rows.filter((x) => x.payment_state === 'pending').length }),
+ body: pages.erasePage({ csrf: csrfFor(req.viewer.subject), idem: idem(), count: c.n, pending: c.pending }),
}), status);
}), status);
}
}
- r.get('/receipts/erase', withViewer, (req, res) => {
+ r.get('/receipts/erase', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
- renderErase(req, res);
+ return renderErase(req, res);
});
});
- r.post('/receipts/erase', withViewer, form, (req, res) => {
+ r.post('/receipts/erase', withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent('/receipts/erase')}`);
if (!csrfOk(req) || !bool(req.body.confirm)) return renderErase(req, res, 403);
if (!csrfOk(req) || !bool(req.body.confirm)) return renderErase(req, res, 403);
- interactions.erase(req.viewer.subject);
+ await interactions.erase(req.viewer.subject);
return res.redirect(303, `/receipts?done=${encodeURIComponent('Your data was erased from your tips')}`);
return res.redirect(303, `/receipts?done=${encodeURIComponent('Your data was erased from your tips')}`);
});
});
- r.get('/receipts/:id', withViewer, (req, res) => {
+ r.get('/receipts/:id', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
- const i = interactions.get(req.params.id);
+ const i = await interactions.get(db, req.params.id);
const as = i && (i.supporter_subject === req.viewer.subject ? 'supporter' : i.creator_subject === req.viewer.subject ? 'creator' : null);
const as = i && (i.supporter_subject === req.viewer.subject ? 'supporter' : i.creator_subject === req.viewer.subject ? 'creator' : null);
if (!i || !as) return notFound(req, res);
if (!i || !as) return notFound(req, res);
- const view = interactions.present(i, { viewer: as === 'creator' ? 'owner' : 'supporter' });
- html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile: profiles.bySubject(i.creator_subject), as, cancelled: req.query.cancelled === '1' }) }));
+ const [view, profile] = await Promise.all([interactions.present(db, i, { viewer: as === 'creator' ? 'owner' : 'supporter' }), profiles.bySubject(db, i.creator_subject)]);
+ return html(res, pageFor(req, { title: 'Receipt', active: 'receipts', robots: 'noindex,nofollow', canonicalPath: `/receipts/${i.id}`, body: pages.receiptPage({ i: view, profile, as, cancelled: req.query.cancelled === '1' }) }));
});
});
// ── Dashboard ────────────────────────────────────────────
// ── Dashboard ────────────────────────────────────────────
@@ -128,42 +135,45 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -128,42 +135,45 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
const claims = req.viewerClaims || {};
const claims = req.viewerClaims || {};
return profiles.ensure(v.subject, { username: v.username, displayName: v.name, avatarUrl: claims.avatar_url });
return profiles.ensure(v.subject, { username: v.username, displayName: v.name, avatarUrl: claims.avatar_url });
}
}
- function ensureAlertsConfig(subject) {
- if (!overlays.listConfigs(subject).some((c) => c.kind === 'alerts')) overlays.createConfig(subject, { kind: 'alerts', name: 'Alerts' });
- }
- function renderDashboard(req, res, { flash, error, status = 200 } = {}) {
- const p = ensureProfile(req);
- ensureAlertsConfig(p.creator_subject);
- const recent = interactions.list({ creator: p.creator_subject, limit: 20 }).rows.map((i) => interactions.present(i, { viewer: 'owner' }));
+ async function renderDashboard(req, res, { flash, error, status = 200 } = {}) {
+ const p = await ensureProfile(req);
+ let configs = await overlays.listConfigs(db, p.creator_subject);
+ if (!configs.some((c) => c.kind === 'alerts')) {
+ await overlays.createConfig(p.creator_subject, { kind: 'alerts', name: 'Alerts' });
+ configs = await overlays.listConfigs(db, p.creator_subject);
+ }
+ const s = p.creator_subject;
+ const [recentRows, goalRows, tokens, totals, deliveries, connected, held, moderators, invites, log, moderating] = await Promise.all([
+ interactions.list(db, { creator: s, limit: 20 }).then((out) => interactions.presentMany(db, out.rows, { viewer: 'owner' })),
+ goals.list(db, s).then((list) => goals.presentMany(db, list)),
+ overlays.listTokens(db, s), interactions.totals(db, s), overlays.recentDeliveries(db, s, 20), overlays.connected(s),
+ moderation.heldCount(db, s), moderation.listModerators(db, s), moderation.openInvites(db, s), moderation.log(db, s, 20), moderation.moderatedBy(db, req.viewer.subject),
+ ]);
html(res, pageFor(req, {
html(res, pageFor(req, {
title: 'Dashboard', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard',
title: 'Dashboard', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard',
body: pages.dashboard({
body: pages.dashboard({
- profile: p, goals: goals.list(p.creator_subject).map((g) => goals.present(g)), tokens: overlays.listTokens(p.creator_subject).map(overlays.presentToken),
- configs: overlays.listConfigs(p.creator_subject).map(overlays.presentConfig), totals: interactions.totals(p.creator_subject), recent,
- deliveries: overlays.recentDeliveries(p.creator_subject, 20), csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,
- connected: overlays.connected(p.creator_subject), chatAdapter: config.chat.adapter,
- mod: {
- held: db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND moderation = 'held' AND payment_state = 'settled'").get(p.creator_subject).n,
- moderators: moderation.listModerators(p.creator_subject).map(moderation.presentModerator), invites: moderation.openInvites(p.creator_subject),
- log: moderation.log(p.creator_subject, 20), moderating: moderation.moderatedBy(req.viewer.subject),
- },
+ profile: p, goals: goalRows, tokens: tokens.map(overlays.presentToken),
+ configs: configs.map(overlays.presentConfig), totals, recent: recentRows,
+ deliveries, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error,
+ connected, chatAdapter: config.chat.adapter,
+ mod: { held, moderators: moderators.map(moderation.presentModerator), invites, log, moderating },
}),
}),
}), status);
}), status);
}
}
- r.get('/dashboard', withViewer, (req, res) => {
+ r.get('/dashboard', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
// Only our own confirmations are shown: a crafted ?done= link cannot put words on the page.
// Only our own confirmations are shown: a crafted ?done= link cannot put words on the page.
- renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });
+ return renderDashboard(req, res, { flash: FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null });
});
});
/** A dashboard form action: sign-in + anti-forgery, errors re-render the dashboard. */
/** A dashboard form action: sign-in + anti-forgery, errors re-render the dashboard. */
function action(path, fn) {
function action(path, fn) {
- r.post(path, withViewer, form, (req, res) => {
+ r.post(path, withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent('/dashboard')}`);
if (!csrfOk(req)) return renderDashboard(req, res, { error: 'That form expired. Please try again.', status: 403 });
if (!csrfOk(req)) return renderDashboard(req, res, { error: 'That form expired. Please try again.', status: 403 });
try {
try {
- const p = ensureProfile(req);
- const out = fn(req, p);
+ const p = await ensureProfile(req);
+ const out = await fn(req, p);
if (out && out.render) return out.render(res);
if (out && out.render) return out.render(res);
return res.redirect(303, `/dashboard?done=${encodeURIComponent((out && out.done) || 'Saved')}`);
return res.redirect(303, `/dashboard?done=${encodeURIComponent((out && out.done) || 'Saved')}`);
} catch (e) {
} catch (e) {
@@ -172,9 +182,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -172,9 +182,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}
}
});
});
}
}
- action('/dashboard/profile', (req, p) => {
+ action('/dashboard/profile', async (req, p) => {
const b = req.body;
const b = req.body;
- profiles.update(p.creator_subject, {
+ await profiles.update(p.creator_subject, {
page_enabled: bool(b.page_enabled), accepting: bool(b.accepting), tts_enabled: bool(b.tts_enabled), media_requests_enabled: bool(b.media_requests_enabled),
page_enabled: bool(b.page_enabled), accepting: bool(b.accepting), tts_enabled: bool(b.tts_enabled), media_requests_enabled: bool(b.media_requests_enabled),
headline: b.headline, min_amount: b.min_amount, paid_message_min: b.paid_message_min, tts_min_amount: b.tts_min_amount, tts_max_chars: b.tts_max_chars,
headline: b.headline, min_amount: b.min_amount, paid_message_min: b.paid_message_min, tts_min_amount: b.tts_min_amount, tts_max_chars: b.tts_max_chars,
tts_voice: b.tts_voice, media_request_min: b.media_request_min, media_max_seconds: b.media_max_seconds,
tts_voice: b.tts_voice, media_request_min: b.media_request_min, media_max_seconds: b.media_max_seconds,
@@ -182,46 +192,46 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -182,46 +192,46 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}, { expectedRevision: b.revision });
}, { expectedRevision: b.revision });
return { done: 'Page settings saved' };
return { done: 'Page settings saved' };
});
});
- action('/dashboard/goals', (req, p) => { goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });
- const ownGoal = (req, p) => {
- const g = goals.get(req.params.id);
+ action('/dashboard/goals', async (req, p) => { await goals.create(p.creator_subject, req.body); return { done: 'Goal added' }; });
+ const ownGoal = async (req, p) => {
+ const g = await goals.get(db, req.params.id);
if (!g || g.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.goal_not_found', 'no such goal');
if (!g || g.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.goal_not_found', 'no such goal');
return g;
return g;
};
};
- action('/dashboard/goals/:id', (req, p) => { goals.update(ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });
- action('/dashboard/goals/:id/close', (req, p) => { goals.close(ownGoal(req, p)); return { done: 'Goal closed' }; });
- action('/dashboard/overlay-tokens', (req, p) => {
+ action('/dashboard/goals/:id', async (req, p) => { await goals.update(await ownGoal(req, p), { title: req.body.title, target_amount: req.body.target_amount, revision: req.body.revision }); return { done: 'Goal updated' }; });
+ action('/dashboard/goals/:id/close', async (req, p) => { await goals.close(await ownGoal(req, p)); return { done: 'Goal closed' }; });
+ action('/dashboard/overlay-tokens', async (req, p) => {
const scopes = [bool(req.body.scope_alerts) && 'alerts', bool(req.body.scope_goals) && 'goals'].filter(Boolean);
const scopes = [bool(req.body.scope_alerts) && 'alerts', bool(req.body.scope_goals) && 'goals'].filter(Boolean);
- const alertCfg = overlays.listConfigs(p.creator_subject).find((c) => c.kind === 'alerts');
- const out = overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });
+ const alertCfg = (await overlays.listConfigs(db, p.creator_subject)).find((c) => c.kind === 'alerts');
+ const out = await overlays.createToken(p.creator_subject, { scopes, label: req.body.label, configId: alertCfg ? alertCfg.id : null, createdBy: req.viewer.subject });
return { render: (res) => html(res, pageFor(req, { title: 'Overlay link', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.tokenCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
return { render: (res) => html(res, pageFor(req, { title: 'Overlay link', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.tokenCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
});
});
- action('/dashboard/overlay-tokens/:id/revoke', (req, p) => {
- const t = overlays.getToken(req.params.id);
+ action('/dashboard/overlay-tokens/:id/revoke', async (req, p) => {
+ const t = await overlays.getToken(db, req.params.id);
if (!t || t.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_token_not_found', 'no such overlay token');
if (!t || t.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_token_not_found', 'no such overlay token');
- overlays.revokeToken(t);
+ await overlays.revokeToken(t);
return { done: 'Overlay link revoked' };
return { done: 'Overlay link revoked' };
});
});
- action('/dashboard/overlay-configs/:id', (req, p) => {
- const c = overlays.getConfig(req.params.id);
+ action('/dashboard/overlay-configs/:id', async (req, p) => {
+ const c = await overlays.getConfig(db, req.params.id);
if (!c || c.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_config_not_found', 'no such overlay config');
if (!c || c.creator_subject !== p.creator_subject) throw new TipsError(404, 'tips.overlay_config_not_found', 'no such overlay config');
const b = req.body;
const b = req.body;
- overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });
+ await overlays.updateConfig(c, { ...b, show_message: bool(b.show_message), speak_message: bool(b.speak_message), show_amount: bool(b.show_amount) });
return { done: 'Alert settings saved' };
return { done: 'Alert settings saved' };
});
});
- action('/dashboard/filter', (req, p) => {
- profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });
+ action('/dashboard/filter', async (req, p) => {
+ await profiles.update(p.creator_subject, { filter: { words: req.body.words || '', action: req.body.action, links: bool(req.body.links) } });
return { done: 'Filter saved' };
return { done: 'Filter saved' };
});
});
- action('/dashboard/moderators/:subject/remove', (req, p) => { moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });
- action('/dashboard/moderator-invites', (req, p) => {
- const out = moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });
+ action('/dashboard/moderators/:subject/remove', async (req, p) => { await moderation.removeModerator(p.creator_subject, req.params.subject); return { done: 'Moderator removed' }; });
+ action('/dashboard/moderator-invites', async (req, p) => {
+ const out = await moderation.createInvite(p.creator_subject, { createdBy: req.viewer.subject });
return { render: (res) => html(res, pageFor(req, { title: 'Moderator invitation', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.inviteCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
return { render: (res) => html(res, pageFor(req, { title: 'Moderator invitation', active: 'dashboard', robots: 'noindex,nofollow', canonicalPath: '/dashboard', body: pages.inviteCreated({ out }) }), 201, { 'Cache-Control': 'no-store' }) };
});
});
- action('/dashboard/moderator-invites/:id/revoke', (req, p) => { moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });
- action('/dashboard/simulate', (req, p) => {
+ action('/dashboard/moderator-invites/:id/revoke', async (req, p) => { await moderation.revokeInvite(p.creator_subject, req.params.id); return { done: 'Invitation revoked' }; });
+ action('/dashboard/simulate', async (req, p) => {
const b = req.body;
const b = req.body;
- interactions.simulate(p, {
+ await interactions.simulate(p, {
kind: b.kind, amount: b.amount, message: b.message, supporter_name: b.supporter_name,
kind: b.kind, amount: b.amount, message: b.message, supporter_name: b.supporter_name,
tts: b.kind === 'tts' ? { text: b.message } : undefined, media: b.kind === 'media_request' ? { url: b.media_url } : undefined,
tts: b.kind === 'tts' ? { text: b.message } : undefined, media: b.kind === 'media_request' ? { url: b.media_url } : undefined,
}, { by: req.viewer.subject });
}, { by: req.viewer.subject });
@@ -230,13 +240,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -230,13 +240,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
// ── Overlays (token in the path; no cookies involved) ────
// ── Overlays (token in the path; no cookies involved) ────
const overlayHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
const overlayHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
- function tokenOr404(req, res) {
- const t = overlays.authenticate(req.params.token);
+ async function tokenOr404(req, res) {
+ const t = await overlays.authenticate(db, req.params.token);
if (!t) { res.status(404).set(overlayHeaders).type('text/plain').send('This overlay link is not valid (it may have been revoked).'); return null; }
if (!t) { res.status(404).set(overlayHeaders).type('text/plain').send('This overlay link is not valid (it may have been revoked).'); return null; }
return t;
return t;
}
}
- r.get('/overlay/:token', (req, res) => {
- const t = tokenOr404(req, res);
+ r.get('/overlay/:token', async (req, res) => {
+ const t = await tokenOr404(req, res);
if (!t) return;
if (!t) return;
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/html; charset=utf-8' }).send(`<!DOCTYPE html>
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/html; charset=utf-8' }).send(`<!DOCTYPE html>
<html lang="en"><head><meta charset="utf-8"><meta name="robots" content="noindex,nofollow"><meta name="referrer" content="no-referrer">
<html lang="en"><head><meta charset="utf-8"><meta name="robots" content="noindex,nofollow"><meta name="referrer" content="no-referrer">
@@ -246,59 +256,59 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -246,59 +256,59 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
<noscript><p class="nojs">This overlay needs JavaScript (OBS Browser Sources have it on).</p></noscript>
<noscript><p class="nojs">This overlay needs JavaScript (OBS Browser Sources have it on).</p></noscript>
<script src="${asset('js/overlay.js')}" defer></script></body></html>`);
<script src="${asset('js/overlay.js')}" defer></script></body></html>`);
});
});
- r.get('/overlay/:token/events', (req, res) => {
- const t = tokenOr404(req, res);
+ r.get('/overlay/:token/events', async (req, res) => {
+ const t = await tokenOr404(req, res);
if (!t) return;
if (!t) return;
- if (overlays.streamsFor(t) >= config.overlays.maxStreamsPerToken) {
+ // A bounded number of open streams per token, counted across every process (overlay-hub.js).
+ const slot = await overlays.reserveStream(t);
+ if (!slot) {
res.status(429).set({ ...overlayHeaders, 'Retry-After': '30' }).type('text/plain').send('Too many open streams for this overlay link.');
res.status(429).set({ ...overlayHeaders, 'Retry-After': '30' }).type('text/plain').send('Too many open streams for this overlay link.');
return;
return;
}
}
+ let overlayConfig = null;
+ try { overlayConfig = t.config_id ? await overlays.getConfig(db, t.config_id) : null; } catch (e) { overlays.releaseStream(slot); throw e; }
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/event-stream; charset=utf-8', Connection: 'keep-alive', 'X-Accel-Buffering': 'no' });
res.status(200).set({ ...overlayHeaders, 'Content-Type': 'text/event-stream; charset=utf-8', Connection: 'keep-alive', 'X-Accel-Buffering': 'no' });
res.flushHeaders();
res.flushHeaders();
res.write('retry: 3000\n\n');
res.write('retry: 3000\n\n');
- const overlayConfig = t.config_id ? overlays.getConfig(t.config_id) : null;
const lastEventId = req.get('last-event-id') != null ? req.get('last-event-id') : req.query.last_event_id;
const lastEventId = req.get('last-event-id') != null ? req.get('last-event-id') : req.query.last_event_id;
- overlays.attach(t, res, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });
+ overlays.attach(t, res, slot, { lastEventId: lastEventId != null && lastEventId !== '' ? lastEventId : undefined, config: overlayConfig });
});
});
- r.get('/overlay/:token/state', (req, res) => {
- const t = tokenOr404(req, res);
+ r.get('/overlay/:token/state', async (req, res) => {
+ const t = await tokenOr404(req, res);
if (!t) return;
if (!t) return;
const out = { creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes };
const out = { creator: { type: 'user', id: t.creator_subject }, scopes: t.scopes };
- if (t.scopes.includes('goals')) out.goals = goals.list(t.creator_subject, { status: 'active' }).map((g) => goals.present(g));
- if (t.scopes.includes('alerts')) {
- out.alerts = db.prepare("SELECT seq, payload, test, created_at FROM overlay_deliveries WHERE creator_subject = ? AND kind = 'alert' AND hidden = 0 ORDER BY seq DESC LIMIT 20")
- .all(t.creator_subject).map((d) => ({ seq: d.seq, ...parseJson(d.payload, {}), test: !!d.test, created_at: d.created_at }));
- }
+ if (t.scopes.includes('goals')) out.goals = await goals.presentMany(db, await goals.list(db, t.creator_subject, { status: 'active' }));
+ if (t.scopes.includes('alerts')) out.alerts = await overlays.recentAlerts(db, t.creator_subject);
res.set(overlayHeaders).json(out);
res.set(overlayHeaders).json(out);
});
});
// ── Moderation (the creator and their moderators) ─────────
// ── Moderation (the creator and their moderators) ─────────
// Invitation links carry a secret in the path, like overlay links: no-store, no referrer, noindex.
// Invitation links carry a secret in the path, like overlay links: no-store, no referrer, noindex.
const inviteHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
const inviteHeaders = { 'Cache-Control': 'no-store', 'Referrer-Policy': 'no-referrer', 'X-Robots-Tag': 'noindex, nofollow' };
- function inviteOr404(req, res) {
- const inv = moderation.findInvite(req.params.token);
- const profile = inv && profiles.bySubject(inv.creator_subject);
+ async function inviteOr404(req, res) {
+ const inv = await moderation.findInvite(db, req.params.token);
+ const profile = inv && await profiles.bySubject(db, inv.creator_subject);
if (!inv || !profile) {
if (!inv || !profile) {
html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 404, title: 'This invitation is not valid', message: 'It may have been used, revoked or expired. Ask the creator for a new one.' }) }), 404, inviteHeaders);
html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 404, title: 'This invitation is not valid', message: 'It may have been used, revoked or expired. Ask the creator for a new one.' }) }), 404, inviteHeaders);
return null;
return null;
}
}
return { inv, profile };
return { inv, profile };
}
}
- r.get('/moderate/invite/:token', withViewer, (req, res) => {
+ r.get('/moderate/invite/:token', withViewer, async (req, res) => {
if (!req.viewer) return res.set(inviteHeaders).redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
if (!req.viewer) return res.set(inviteHeaders).redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
- const found = inviteOr404(req, res);
+ const found = await inviteOr404(req, res);
if (!found) return undefined;
if (!found) return undefined;
return html(res, pageFor(req, {
return html(res, pageFor(req, {
title: `Moderate ${found.profile.display_name}`, robots: 'noindex,nofollow', canonicalPath: '/',
title: `Moderate ${found.profile.display_name}`, robots: 'noindex,nofollow', canonicalPath: '/',
body: pages.invitePage({ profile: found.profile, csrf: csrfFor(req.viewer.subject), idem: idem(), secret: req.params.token, isSelf: found.profile.creator_subject === req.viewer.subject }),
body: pages.invitePage({ profile: found.profile, csrf: csrfFor(req.viewer.subject), idem: idem(), secret: req.params.token, isSelf: found.profile.creator_subject === req.viewer.subject }),
}), 200, inviteHeaders);
}), 200, inviteHeaders);
});
});
- r.post('/moderate/invite/:token', withViewer, form, (req, res) => {
+ r.post('/moderate/invite/:token', withViewer, form, async (req, res) => {
if (!req.viewer) return res.set(inviteHeaders).redirect(303, '/auth/login');
if (!req.viewer) return res.set(inviteHeaders).redirect(303, '/auth/login');
if (!csrfOk(req)) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 403, title: 'That form expired', message: 'Open the invitation link again.' }) }), 403, inviteHeaders);
if (!csrfOk(req)) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: 403, title: 'That form expired', message: 'Open the invitation link again.' }) }), 403, inviteHeaders);
try {
try {
- const out = moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });
- const profile = profiles.bySubject(out.creator);
+ const out = await moderation.acceptInvite(req.params.token, { subject: req.viewer.subject, name: req.viewer.name || req.viewer.username });
+ const profile = await profiles.bySubject(db, out.creator);
return res.set(inviteHeaders).redirect(303, `/moderate/${encodeURIComponent(profile.handle)}?done=${encodeURIComponent('You are now a moderator')}`);
return res.set(inviteHeaders).redirect(303, `/moderate/${encodeURIComponent(profile.handle)}?done=${encodeURIComponent('You are now a moderator')}`);
} catch (e) {
} catch (e) {
if (e instanceof TipsError) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: e.status, title: 'This invitation is not valid', message: e.detail || e.message }) }), e.status, inviteHeaders);
if (e instanceof TipsError) return html(res, pageFor(req, { title: 'Invitation', robots: 'noindex,nofollow', body: pages.errorPage({ status: e.status, title: 'This invitation is not valid', message: e.detail || e.message }) }), e.status, inviteHeaders);
@@ -307,36 +317,35 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -307,36 +317,35 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
});
});
/** The creator or one of their moderators, for /moderate/:handle; else null (the page is a 404). */
/** The creator or one of their moderators, for /moderate/:handle; else null (the page is a 404). */
- function modAccess(req) {
- const p = profiles.byHandle(req.params.handle);
+ async function modAccess(req) {
+ const p = await profiles.byHandle(db, req.params.handle);
if (!p || !req.viewer) return { p };
if (!p || !req.viewer) return { p };
- const actor = moderation.actorFor(req.viewer, p.creator_subject, false);
- return { p, actor };
+ return { p, actor: await moderation.actorFor(db, req.viewer, p.creator_subject, false) };
}
}
- function renderModerate(req, res, p, { error, status = 200 } = {}) {
+ async function renderModerate(req, res, p, { error, status = 200 } = {}) {
const state = ['held', 'hidden', 'visible', 'all'].includes(req.query.state) ? req.query.state : 'all';
const state = ['held', 'hidden', 'visible', 'all'].includes(req.query.state) ? req.query.state : 'all';
- const out = moderation.queue(p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });
+ const out = await moderation.queue(db, p.creator_subject, { state, cursor: req.query.cursor, limit: 50 });
const flash = MOD_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
const flash = MOD_FLASH.has(String(req.query.done || '')) ? String(req.query.done) : null;
html(res, pageFor(req, {
html(res, pageFor(req, {
title: `Paid messages — ${p.display_name}`, robots: 'noindex,nofollow', canonicalPath: `/moderate/${p.handle}`,
title: `Paid messages — ${p.display_name}`, robots: 'noindex,nofollow', canonicalPath: `/moderate/${p.handle}`,
body: pages.moderatePage({ profile: p, rows: out.rows, state, next: out.next_cursor, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error, isCreator: req.viewer.subject === p.creator_subject }),
body: pages.moderatePage({ profile: p, rows: out.rows, state, next: out.next_cursor, csrf: csrfFor(req.viewer.subject), idem: idem(), flash, error, isCreator: req.viewer.subject === p.creator_subject }),
}), status);
}), status);
}
}
- r.get('/moderate/:handle', withViewer, (req, res) => {
+ r.get('/moderate/:handle', withViewer, async (req, res) => {
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
if (!req.viewer) return res.redirect(`/auth/login?next=${encodeURIComponent(req.originalUrl)}`);
- const { p, actor } = modAccess(req);
+ const { p, actor } = await modAccess(req);
if (!p || !actor) return notFound(req, res);
if (!p || !actor) return notFound(req, res);
return renderModerate(req, res, p);
return renderModerate(req, res, p);
});
});
- r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, (req, res) => {
+ r.post('/moderate/:handle/:id/:act(hide|restore)', withViewer, form, async (req, res) => {
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/moderate/${req.params.handle}`)}`);
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/moderate/${req.params.handle}`)}`);
- const { p, actor } = modAccess(req);
+ const { p, actor } = await modAccess(req);
if (!p || !actor) return notFound(req, res);
if (!p || !actor) return notFound(req, res);
if (!csrfOk(req)) return renderModerate(req, res, p, { error: 'That form expired. Please try again.', status: 403 });
if (!csrfOk(req)) return renderModerate(req, res, p, { error: 'That form expired. Please try again.', status: 403 });
- const i = interactions.get(req.params.id);
+ const i = await interactions.get(db, req.params.id);
if (!i || i.creator_subject !== p.creator_subject) return notFound(req, res);
if (!i || i.creator_subject !== p.creator_subject) return notFound(req, res);
try {
try {
- moderation[req.params.act](i, { ...actor, reason: req.body.reason });
+ await moderation[req.params.act](i, { ...actor, reason: req.body.reason });
} catch (e) {
} catch (e) {
if (e instanceof TipsError) return renderModerate(req, res, p, { error: e.detail || e.message, status: 400 });
if (e instanceof TipsError) return renderModerate(req, res, p, { error: e.detail || e.message, status: 400 });
throw e;
throw e;
@@ -346,13 +355,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -346,13 +355,13 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
// ── Creator pages (last: /:handle catches everything else) ──
// ── Creator pages (last: /:handle catches everything else) ──
const providers = config.billing.providers;
const providers = config.billing.providers;
- function creatorView(req, res, { values, error, status = 200 } = {}) {
- const p = profiles.byHandle(req.params.handle);
+ async function creatorView(req, res, { values, error, status = 200 } = {}) {
+ const p = await profiles.byHandle(db, req.params.handle);
const owner = !!(p && req.viewer && req.viewer.subject === p.creator_subject);
const owner = !!(p && req.viewer && req.viewer.subject === p.creator_subject);
if (!p || (!p.page_enabled && !owner)) return notFound(req, res);
if (!p || (!p.page_enabled && !owner)) return notFound(req, res);
if (p.handle !== String(req.params.handle)) return res.redirect(301, `/${p.handle}`); // @name, Name → the canonical path
if (p.handle !== String(req.params.handle)) return res.redirect(301, `/${p.handle}`); // @name, Name → the canonical path
// The page shows what the public sees, for the creator too (a preview); the dashboard has the rest.
// The page shows what the public sees, for the creator too (a preview); the dashboard has the rest.
- const list = goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page));
+ const list = await goals.publicGoals(db, await goals.list(db, p.creator_subject), p);
const csrf = req.viewer ? csrfFor(req.viewer.subject) : '';
const csrf = req.viewer ? csrfFor(req.viewer.subject) : '';
const jsonLd = p.page_enabled ? [{ '@context': 'https://schema.org', '@type': 'ProfilePage', name: `${p.display_name} on ${'OpenVibe.Tips'}`, url: `${config.baseUrl}/${p.handle}`, mainEntity: { '@type': 'Person', name: p.display_name, alternateName: p.handle, image: p.avatar_url || undefined } }] : [];
const jsonLd = p.page_enabled ? [{ '@context': 'https://schema.org', '@type': 'ProfilePage', name: `${p.display_name} on ${'OpenVibe.Tips'}`, url: `${config.baseUrl}/${p.handle}`, mainEntity: { '@type': 'Person', name: p.display_name, alternateName: p.handle, image: p.avatar_url || undefined } }] : [];
return html(res, pageFor(req, {
return html(res, pageFor(req, {
@@ -363,33 +372,34 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -363,33 +372,34 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
}), status, p.page_enabled ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
}), status, p.page_enabled ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
}
}
r.get('/:handle', withViewer, (req, res, next) => (profiles.normalizeHandle(req.params.handle) ? creatorView(req, res) : next()));
r.get('/:handle', withViewer, (req, res, next) => (profiles.normalizeHandle(req.params.handle) ? creatorView(req, res) : next()));
- r.get('/:handle/goals', withViewer, (req, res, next) => {
- const p = profiles.byHandle(req.params.handle);
+ r.get('/:handle/goals', withViewer, async (req, res, next) => {
+ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
if (!p) return next();
const owner = req.viewer && req.viewer.subject === p.creator_subject;
const owner = req.viewer && req.viewer.subject === p.creator_subject;
if (!p.page_enabled && !owner) return notFound(req, res);
if (!p.page_enabled && !owner) return notFound(req, res);
- html(res, pageFor(req, {
+ return html(res, pageFor(req, {
title: `${p.display_name} — goals`, canonicalPath: `/${p.handle}/goals`, robots: p.page_enabled ? 'index,follow' : 'noindex,nofollow',
title: `${p.display_name} — goals`, canonicalPath: `/${p.handle}/goals`, robots: p.page_enabled ? 'index,follow' : 'noindex,nofollow',
- body: pages.goalsPage({ profile: p, goals: goals.list(p.creator_subject).map((g) => goals.publicGoal(g, p.page)) }),
+ body: pages.goalsPage({ profile: p, goals: await goals.publicGoals(db, await goals.list(db, p.creator_subject), p) }),
}));
}));
});
});
- r.get('/:handle/supporters', withViewer, (req, res, next) => {
- const p = profiles.byHandle(req.params.handle);
+ r.get('/:handle/supporters', withViewer, async (req, res, next) => {
+ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
if (!p) return next();
const owner = req.viewer && req.viewer.subject === p.creator_subject;
const owner = req.viewer && req.viewer.subject === p.creator_subject;
// Off unless the creator shows it; the creator gets a preview while it is off.
// Off unless the creator shows it; the creator gets a preview while it is off.
if ((!p.page_enabled || !p.page.supporters_page) && !owner) return notFound(req, res);
if ((!p.page_enabled || !p.page.supporters_page) && !owner) return notFound(req, res);
- const leaderboard = interactions.leaderboard(p.creator_subject).map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));
- const recent = p.page.supporters_messages ? interactions.recentPublic(p.creator_subject).map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;
+ const [top, recentRows] = await Promise.all([interactions.leaderboard(db, p), p.page.supporters_messages ? interactions.recentPublic(db, p) : null]);
+ const leaderboard = top.map((row) => (p.page.supporters_amounts ? row : { ...row, total: null }));
+ const recent = recentRows ? recentRows.map((v) => (p.page.supporters_amounts ? v : { ...v, amount: null })) : null;
const indexable = p.page_enabled && p.page.supporters_page;
const indexable = p.page_enabled && p.page.supporters_page;
- html(res, pageFor(req, {
+ return html(res, pageFor(req, {
title: `${p.display_name} — supporters`, canonicalPath: `/${p.handle}/supporters`, robots: indexable ? 'index,follow' : 'noindex,nofollow',
title: `${p.display_name} — supporters`, canonicalPath: `/${p.handle}/supporters`, robots: indexable ? 'index,follow' : 'noindex,nofollow',
body: pages.supportersPage({ profile: p, leaderboard, recent }),
body: pages.supportersPage({ profile: p, leaderboard, recent }),
}), 200, indexable ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
}), 200, indexable ? {} : { 'X-Robots-Tag': 'noindex, nofollow' });
});
});
r.post('/:handle/tip', withViewer, form, async (req, res, next) => {
r.post('/:handle/tip', withViewer, form, async (req, res, next) => {
- const p = profiles.byHandle(req.params.handle);
+ const p = await profiles.byHandle(db, req.params.handle);
if (!p) return next();
if (!p) return next();
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/${p.handle}`)}`);
if (!req.viewer) return res.redirect(303, `/auth/login?next=${encodeURIComponent(`/${p.handle}`)}`);
const b = req.body || {};
const b = req.body || {};
@@ -409,7 +419,7 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -409,7 +419,7 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
privacy: { anonymous: values.anonymous, hide_amount: values.hide_amount, private_message: values.private_message },
privacy: { anonymous: values.anonymous, hide_amount: values.hide_amount, private_message: values.private_message },
};
};
const funding = b.pay_with === 'checkout' ? 'checkout' : 'credit';
const funding = b.pay_with === 'checkout' ? 'checkout' : 'credit';
- const { interaction, replay } = interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });
+ const { interaction, replay } = await interactions.request(p, input, { supporter: req.viewer.subject, supporterName: b.supporter_name || req.viewer.name, funding, idempotencyKey: key });
let i = interaction;
let i = interaction;
if (!replay) {
if (!replay) {
if (funding === 'credit') {
if (funding === 'credit') {
@@ -17,6 +17,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -17,6 +17,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(h.json.service, 'tips');
assert.strictEqual(h.json.service, 'tips');
const r = await t.call('GET', '/api/ready', { token: null });
const r = await t.call('GET', '/api/ready', { token: null });
assert.strictEqual(r.json.ready, true);
assert.strictEqual(r.json.ready, true);
+ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered: pglite here, postgresql on the containers');
const rel = await t.call('GET', '/release.json', { token: null });
const rel = await t.call('GET', '/release.json', { token: null });
assert.strictEqual(rel.json.service, 'tips');
assert.strictEqual(rel.json.service, 'tips');
assert.deepStrictEqual(require('openvibe-contracts').validate('registry.release-manifest@1', rel.json).errors, []);
assert.deepStrictEqual(require('openvibe-contracts').validate('registry.release-manifest@1', rel.json).errors, []);
@@ -107,6 +108,38 @@ const { boot, check, done } = require('./helpers/app');
@@ -107,6 +108,38 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(none.status, 400);
assert.strictEqual(none.status, 400);
});
});
+ await check('an Idempotency-Key is claimed before the handler runs: once at a time, a stale claim taken over, refusals released', async () => {
+ const goals = () => t.db.value('SELECT count(*) FROM tip_goals WHERE creator_subject = $1', [alex.subject]);
+ const before = await goals();
+ // Two at once: the handler runs once; the other answer is "in progress" or the replay.
+ const body = { title: 'Once only', target_amount: 70 };
+ const both = await Promise.all([1, 2].map(() => t.call('POST', '/api/v1/goals', { user: alex, body, key: 'goal-key-race-01' })));
+ assert.ok(both.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(both.map((r) => r.status)));
+ assert.strictEqual(new Set(both.filter((r) => r.status === 201).map((r) => r.json.goal.id)).size, 1);
+ assert.strictEqual(await goals(), before + 1);
+ // A claim still in progress (as another process holds it): 409 with Retry-After.
+ const { stable } = require('../server/api/idempotency');
+ const hashOf = (b) => require('crypto').createHash('sha256').update(`POST /api/v1/goals\n${stable(b)}`).digest('hex');
+ const claim = (key, b, ageMs) => t.db.exec(`INSERT INTO api_idempotency (key, request_hash, method, path, status, response, created_at)
+ VALUES ($1, $2, 'POST', '/api/v1/goals', 0, '', $3)`, [`${alex.subject}:${key}`, hashOf(b), new Date(Date.now() - ageMs).toISOString()]);
+ await claim('goal-key-busy-0001', { title: 'Busy', target_amount: 5 }, 1000);
+ const busy = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Busy', target_amount: 5 }, key: 'goal-key-busy-0001' });
+ assert.deepStrictEqual([busy.status, busy.json.code, busy.headers.get('retry-after')], [409, 'idempotency.in_progress', '1']);
+ // A claim left by a process that died over a minute ago is taken over.
+ await claim('goal-key-stale-001', { title: 'Stale', target_amount: 5 }, 120_000);
+ const taken = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });
+ assert.strictEqual(taken.status, 201, taken.text);
+ const replay = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Stale', target_amount: 5 }, key: 'goal-key-stale-001' });
+ assert.deepStrictEqual([replay.headers.get('idempotent-replayed'), replay.json.goal.id], ['true', taken.json.goal.id]);
+ // A refusal is not stored: its claim is released, and a retry of the refused request runs again.
+ const bad = await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' });
+ assert.strictEqual(bad.status, 422);
+ await new Promise((ok) => setTimeout(ok, 50));
+ assert.strictEqual(await t.db.value("SELECT count(*) FROM api_idempotency WHERE key LIKE '%goal-key-refused-1'"), 0, 'refusals are not stored');
+ assert.strictEqual((await t.call('POST', '/api/v1/goals', { user: alex, body: { title: '' }, key: 'goal-key-refused-1' })).status, 422);
+ for (const id of [taken.json.goal.id, both.find((r) => r.status === 201).json.goal.id]) await t.call('POST', `/api/v1/goals/${id}/close`, { user: alex });
+ });
+
await check('receipts: supporters see their own, creators theirs, nobody else', async () => {
await check('receipts: supporters see their own, creators theirs, nobody else', async () => {
const mine = await t.call('GET', '/api/v1/interactions', { user: viewer });
const mine = await t.call('GET', '/api/v1/interactions', { user: viewer });
assert.ok(mine.json.interactions.length >= 2);
assert.ok(mine.json.interactions.length >= 2);
@@ -158,7 +191,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -158,7 +191,7 @@ const { boot, check, done } = require('./helpers/app');
await check('totals: derived from settled interactions, never from a stored balance', async () => {
await check('totals: derived from settled interactions, never from a stored balance', async () => {
const r = await t.call('GET', '/api/v1/profiles/alex/totals', { user: alex });
const r = await t.call('GET', '/api/v1/profiles/alex/totals', { user: alex });
- const sum = t.db.prepare("SELECT SUM(amount - reversed_bits) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 0 AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')").get(alex.subject).n;
+ const sum = await t.db.value("SELECT SUM(amount - reversed_bits)::bigint FROM tip_interactions WHERE creator_subject = $1 AND NOT test AND settlement IN ('billing','imported') AND payment_state IN ('settled','reversed')", [alex.subject]);
assert.strictEqual(r.json.totals.settled_via_billing, sum);
assert.strictEqual(r.json.totals.settled_via_billing, sum);
assert.strictEqual(r.json.totals.settled_via_billing, billing.payable.get(alex.subject), 'equals what Billing holds for the creator');
assert.strictEqual(r.json.totals.settled_via_billing, billing.payable.get(alex.subject), 'equals what Billing holds for the creator');
assert.strictEqual((await t.call('GET', '/api/v1/profiles/alex/totals', { user: viewer })).status, 403);
assert.strictEqual((await t.call('GET', '/api/v1/profiles/alex/totals', { user: viewer })).status, 403);
@@ -32,15 +32,15 @@ const TYPES = ['tips.interaction.ready', 'tips.interaction.failed', 'tips.intera
@@ -32,15 +32,15 @@ const TYPES = ['tips.interaction.ready', 'tips.interaction.failed', 'tips.intera
await domain.effects.drain();
await domain.effects.drain();
live.state.fail = null;
live.state.fail = null;
t.clock.offset += 11 * 60 * 1000;
t.clock.offset += 11 * 60 * 1000;
- domain.overlays.sweepFailed();
+ await domain.overlays.sweepFailed();
t.clock.offset = 0;
t.clock.offset = 0;
// A reversal before delivery (tips.interaction.cancelled) and one after it.
// A reversal before delivery (tips.interaction.cancelled) and one after it.
const pending = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'read me' } } });
const pending = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'read me' } } });
- const txn = domain.interactions.get(pending.json.interaction.id).billing_txn_id;
+ const txn = (await domain.interactions.get(t.db, pending.json.interaction.id)).billing_txn_id;
await t.deliver(billing.refund(txn));
await t.deliver(billing.refund(txn));
- await t.deliver(billing.refund(domain.interactions.get(tip.json.interaction.id).billing_txn_id, 100));
+ await t.deliver(billing.refund((await domain.interactions.get(t.db, tip.json.interaction.id)).billing_txn_id, 100));
- const rows = t.outboxRows();
+ const rows = await t.outboxRows();
for (const type of TYPES) assert.ok(rows.some((e) => e.event_type === type), `no ${type} was produced`);
for (const type of TYPES) assert.ok(rows.some((e) => e.event_type === type), `no ${type} was produced`);
for (const e of rows) {
for (const e of rows) {
const env = contracts.validate('events.event-envelope@1', e);
const env = contracts.validate('events.event-envelope@1', e);
@@ -24,8 +24,8 @@ const { boot, check, done } = require('./helpers/app');
@@ -24,8 +24,8 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(d.job.test, false);
assert.strictEqual(d.job.test, false);
const grant = t.network.grants.find((g) => g.audience === 'openvibe.live');
const grant = t.network.grants.find((g) => g.audience === 'openvibe.live');
assert.strictEqual(grant.scope, 'live.tips_delivery.write');
assert.strictEqual(grant.scope, 'live.tips_delivery.write');
- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(r.json.interaction.id);
- assert.deepStrictEqual(JSON.parse(e.result).ref, { chat_message_id: 1 });
+ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [r.json.interaction.id]);
+ assert.deepStrictEqual(e.result.ref, { chat_message_id: 1 }, 'jsonb comes back parsed');
});
});
await check('TTS goes out as its own effect with the text and voice', async () => {
await check('TTS goes out as its own effect with the text and voice', async () => {
@@ -41,17 +41,17 @@ const { boot, check, done } = require('./helpers/app');
@@ -41,17 +41,17 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 7 } });
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 7 } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await domain.effects.drain();
await domain.effects.drain();
- let e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
+ let e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'queued');
assert.strictEqual(e.state, 'queued');
assert.match(e.last_error, /Live 503/);
assert.match(e.last_error, /Live 503/);
- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
+ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
live.state.fail = null;
live.state.fail = null;
t.clock.offset += 5000;
t.clock.offset += 5000;
await domain.effects.drain();
await domain.effects.drain();
t.clock.offset = 0;
t.clock.offset = 0;
- e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
+ e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'delivered');
assert.strictEqual(e.state, 'delivered');
- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
+ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
});
});
await check('Live refuses (422): the effect fails at once, tips.interaction.failed, payment untouched', async () => {
await check('Live refuses (422): the effect fails at once, tips.interaction.failed, payment untouched', async () => {
@@ -60,17 +60,17 @@ const { boot, check, done } = require('./helpers/app');
@@ -60,17 +60,17 @@ const { boot, check, done } = require('./helpers/app');
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await domain.effects.drain();
await domain.effects.drain();
live.state.fail = null;
live.state.fail = null;
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.delivery_state, 'failed');
assert.strictEqual(i.delivery_state, 'failed');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
- assert.strictEqual(billing.payable.get(t.domain.profiles.byHandle('alex').creator_subject) >= 8, true);
- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((ev) => ev.subject.id === id).length, 1);
+ assert.strictEqual(billing.payable.get((await t.domain.profiles.byHandle(t.db, 'alex')).creator_subject) >= 8, true);
+ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((ev) => ev.subject.id === id).length, 1);
});
});
await check('simulations never reach Live: their chat effects stay in the test adapter', async () => {
await check('simulations never reach Live: their chat effects stay in the test adapter', async () => {
- const alex = t.domain.profiles.byHandle('alex');
+ const alex = await t.domain.profiles.byHandle(t.db, 'alex');
const before = live.deliveries.length;
const before = live.deliveries.length;
- const sim = domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });
+ const sim = await domain.interactions.simulate(alex, { kind: 'tip', amount: 100 }, { by: alex.creator_subject });
await domain.effects.drain();
await domain.effects.drain();
assert.strictEqual(live.deliveries.length, before);
assert.strictEqual(live.deliveries.length, before);
assert.ok(t.adapters.test.jobs.some((j) => j.interaction.id === sim.id && j.test));
assert.ok(t.adapters.test.jobs.some((j) => j.interaction.id === sim.id && j.test));
@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');
},
},
};
};
};
};
- const byRef = (ref) => t.db.prepare("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = ?").get(ref);
+ const byRef = (ref) => t.db.maybe("SELECT * FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = $1", [ref]);
let first;
let first;
await check('an EXTERNAL receipt is recorded, settled and announced in Live chat as Live\'s webhook wrote it', async () => {
await check('an EXTERNAL receipt is recorded, settled and announced in Live chat as Live\'s webhook wrote it', async () => {
@@ -37,7 +37,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -37,7 +37,7 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.deliver(first);
const r = await t.deliver(first);
assert.strictEqual(r.status, 200, JSON.stringify(r.json));
assert.strictEqual(r.status, 200, JSON.stringify(r.json));
assert.strictEqual(r.json.outcome, 'recorded_external');
assert.strictEqual(r.json.outcome, 'recorded_external');
- const i = byRef('ext-1');
+ const i = await byRef('ext-1');
assert.deepStrictEqual([i.creator_subject, i.kind, i.amount, i.amount_cents, i.supporter_name, i.message], [alex.subject, 'tip', 1234, 1234, 'Generous Fan', 'love the stream']);
assert.deepStrictEqual([i.creator_subject, i.kind, i.amount, i.amount_cents, i.supporter_name, i.message], [alex.subject, 'tip', 1234, 1234, 'Generous Fan', 'love the stream']);
assert.deepStrictEqual([i.funding, i.settlement, i.origin, i.payment_state, i.billing_txn_id], ['external', 'external', 'billing-external', 'settled', null]);
assert.deepStrictEqual([i.funding, i.settlement, i.origin, i.payment_state, i.billing_txn_id], ['external', 'external', 'billing-external', 'settled', null]);
await domain.effects.drain();
await domain.effects.drain();
@@ -46,12 +46,12 @@ const { boot, check, done } = require('./helpers/app');
@@ -46,12 +46,12 @@ const { boot, check, done } = require('./helpers/app');
assert.deepStrictEqual([d[0].key, d[0].job.effect, d[0].job.test], [`${i.id}:chat_line`, 'chat_line', false]);
assert.deepStrictEqual([d[0].key, d[0].job.effect, d[0].job.test], [`${i.id}:chat_line`, 'chat_line', false]);
assert.strictEqual(d[0].job.text, 'Generous Fan tipped 1,234 Vibes: love the stream (PowerChat)');
assert.strictEqual(d[0].job.text, 'Generous Fan tipped 1,234 Vibes: love the stream (PowerChat)');
assert.deepStrictEqual(d[0].job.creator, { type: 'user', id: alex.subject, handle: 'alex' });
assert.deepStrictEqual(d[0].job.creator, { type: 'user', id: alex.subject, handle: 'alex' });
- const effects = t.db.prepare('SELECT effect, state FROM interaction_effects WHERE interaction_id = ? ORDER BY id').all(i.id);
+ const effects = await t.db.many('SELECT effect, state FROM interaction_effects WHERE interaction_id = $1 ORDER BY id', [i.id]);
assert.deepStrictEqual(effects, [{ effect: 'overlay_alert', state: 'delivered' }, { effect: 'chat_line', state: 'delivered' }]);
assert.deepStrictEqual(effects, [{ effect: 'overlay_alert', state: 'delivered' }, { effect: 'chat_line', state: 'delivered' }]);
- const goal = domain.goals.present(domain.goals.get(g.json.goal.id));
+ const goal = await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id));
assert.strictEqual(goal.current_amount, 1234, 'the sole active goal advances, as on Live');
assert.strictEqual(goal.current_amount, 1234, 'the sole active goal advances, as on Live');
- assert.ok(t.outboxRows('tips.goal.updated').some((e) => e.payload.interaction_id === i.id));
- assert.ok(t.outboxRows('tips.interaction.ready').some((e) => e.subject.id === i.id));
+ assert.ok((await t.outboxRows('tips.goal.updated')).some((e) => e.payload.interaction_id === i.id));
+ assert.ok((await t.outboxRows('tips.interaction.ready')).some((e) => e.subject.id === i.id));
});
});
await check('a redelivery or a republish of the same payment announces nothing more', async () => {
await check('a redelivery or a republish of the same payment announces nothing more', async () => {
@@ -61,12 +61,13 @@ const { boot, check, done } = require('./helpers/app');
@@ -61,12 +61,13 @@ const { boot, check, done } = require('./helpers/app');
const viaApi = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.interaction.record'], body: { creator: 'alex', provider: 'powerchat', provider_ref: 'ext-1', amount_cents: 1234 } });
const viaApi = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.interaction.record'], body: { creator: 'alex', provider: 'powerchat', provider_ref: 'ext-1', amount_cents: 1234 } });
assert.strictEqual(viaApi.json.duplicate, true, 'the same key as POST /interactions/external');
assert.strictEqual(viaApi.json.duplicate, true, 'the same key as POST /interactions/external');
await domain.effects.drain();
await domain.effects.drain();
- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE provider_ref = 'ext-1'").get().n, 1);
- assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === byRef('ext-1').id).length, 1);
+ assert.strictEqual(await t.db.value("SELECT count(*) FROM tip_interactions WHERE provider_ref = 'ext-1'"), 1);
+ const one = await byRef('ext-1');
+ assert.strictEqual(live.deliveries.filter((x) => x.job.interaction.id === one.id).length, 1);
});
});
await check('EXTERNAL money stays out of the Billing total', async () => {
await check('EXTERNAL money stays out of the Billing total', async () => {
- const totals = domain.interactions.totals(alex.subject);
+ const totals = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totals.external, 1234);
assert.strictEqual(totals.external, 1234);
assert.strictEqual(totals.settled_via_billing, 0);
assert.strictEqual(totals.settled_via_billing, 0);
});
});
@@ -75,19 +76,19 @@ const { boot, check, done } = require('./helpers/app');
@@ -75,19 +76,19 @@ const { boot, check, done } = require('./helpers/app');
const a = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 900 } })).json.goal;
const a = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 900 } })).json.goal;
const b = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Chair', target_amount: 900 } })).json.goal;
const b = (await t.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Chair', target_amount: 900 } })).json.goal;
await t.deliver(receipt({ provider_event_id: 'ext-goal-tips', amount_cents: 100, value_bits: 100, app_purpose: `goal:${a.id}` }));
await t.deliver(receipt({ provider_event_id: 'ext-goal-tips', amount_cents: 100, value_bits: 100, app_purpose: `goal:${a.id}` }));
- t.db.prepare(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)
- VALUES ('live', 'donation_goals', '12', 'goal', ?, 'imported', 'imp_test', ?, ?)`).run(b.id, new Date().toISOString(), new Date().toISOString());
+ await t.db.exec(`INSERT INTO migration_maps (source, source_table, source_id, target_type, target_id, status, run_id, created_at, updated_at)
+ VALUES ('live', 'donation_goals', '12', 'goal', $1, 'imported', 'imp_test', $2, $2)`, [b.id, new Date().toISOString()]);
await t.deliver(receipt({ provider_event_id: 'ext-goal-live', amount_cents: 200, value_bits: 200, app_purpose: 'goal:12' }));
await t.deliver(receipt({ provider_event_id: 'ext-goal-live', amount_cents: 200, value_bits: 200, app_purpose: 'goal:12' }));
await t.deliver(receipt({ provider_event_id: 'ext-goal-none', amount_cents: 300, value_bits: 300 }));
await t.deliver(receipt({ provider_event_id: 'ext-goal-none', amount_cents: 300, value_bits: 300 }));
- assert.strictEqual(domain.goals.present(domain.goals.get(a.id)).current_amount, 100);
- assert.strictEqual(domain.goals.present(domain.goals.get(b.id)).current_amount, 200);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(byRef('ext-goal-none').id).n, 0, 'several active goals and no pick: none (Live\'s rule)');
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, a.id))).current_amount, 100);
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, b.id))).current_amount, 200);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [(await byRef('ext-goal-none')).id]), 0, 'several active goals and no pick: none (Live\'s rule)');
});
});
await check('anonymous: no donor name; a creator with no tip page is still announced', async () => {
await check('anonymous: no donor name; a creator with no tip page is still announced', async () => {
const bob = t.network.newUser('bob');
const bob = t.network.newUser('bob');
await t.deliver(receipt({ provider_event_id: 'ext-anon', donor_name: null, anonymous: true, message: 'hi' }, { streamer: bob.subject }));
await t.deliver(receipt({ provider_event_id: 'ext-anon', donor_name: null, anonymous: true, message: 'hi' }, { streamer: bob.subject }));
- const i = byRef('ext-anon');
+ const i = await byRef('ext-anon');
assert.deepStrictEqual([i.creator_subject, i.supporter_name], [bob.subject, 'Anonymous']);
assert.deepStrictEqual([i.creator_subject, i.supporter_name], [bob.subject, 'Anonymous']);
await domain.effects.drain();
await domain.effects.drain();
const d = live.deliveries.find((x) => x.job.interaction.id === i.id);
const d = live.deliveries.find((x) => x.job.interaction.id === i.id);
@@ -99,20 +100,20 @@ const { boot, check, done } = require('./helpers/app');
@@ -99,20 +100,20 @@ const { boot, check, done } = require('./helpers/app');
const before = live.deliveries.length;
const before = live.deliveries.length;
await t.deliver(receipt({ provider_event_id: 'ext-test', test: true }));
await t.deliver(receipt({ provider_event_id: 'ext-test', test: true }));
await domain.effects.drain();
await domain.effects.drain();
- const i = byRef('ext-test');
- assert.strictEqual(i.test, 1);
+ const i = await byRef('ext-test');
+ assert.strictEqual(i.test, true);
assert.strictEqual(live.deliveries.length, before);
assert.strictEqual(live.deliveries.length, before);
assert.strictEqual(t.adapters.test.jobs.filter((j) => j.interaction.id === i.id).length, 1);
assert.strictEqual(t.adapters.test.jobs.filter((j) => j.interaction.id === i.id).length, 1);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);
});
});
await check('malformed or foreign receipts are ignored, not recorded', async () => {
await check('malformed or foreign receipts are ignored, not recorded', async () => {
- const n = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;
+ const n = await t.db.value('SELECT count(*) FROM tip_interactions');
assert.strictEqual((await t.deliver(receipt({ streamer: { type: 'user', id: '42' } }))).json.outcome, 'ignored:no_streamer');
assert.strictEqual((await t.deliver(receipt({ streamer: { type: 'user', id: '42' } }))).json.outcome, 'ignored:no_streamer');
assert.strictEqual((await t.deliver(receipt({ amount_cents: 1e20 }))).json.outcome, 'ignored:no_amount');
assert.strictEqual((await t.deliver(receipt({ amount_cents: 1e20 }))).json.outcome, 'ignored:no_amount');
assert.strictEqual((await t.deliver(receipt({ provider_event_id: '' }))).json.outcome, 'ignored:no_reference');
assert.strictEqual((await t.deliver(receipt({ provider_event_id: '' }))).json.outcome, 'ignored:no_reference');
assert.strictEqual((await t.deliver({ ...receipt(), source: 'live' })).json.outcome, 'ignored:source');
assert.strictEqual((await t.deliver({ ...receipt(), source: 'live' })).json.outcome, 'ignored:source');
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, n);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), n);
});
});
await t.close();
await t.close();
new file mode 100644
new file mode 100644
@@ -0,0 +1,301 @@
@@ -0,0 +1,301 @@
+-- The SQLite schema of the last SQLite release of OpenVibe.Tips (schema v3: server/db.js with its added
+-- columns, plus the openvibe-sdk outbox and inbox), as sqlite_master holds it in a database that release made.
+-- test/sqlite-import.test.js builds its fixture from it. Never edit: it is the source side of the one-time import.
+
+CREATE TABLE settings (
+ id INTEGER PRIMARY KEY CHECK (id = 1),
+ schema_version INTEGER NOT NULL,
+ created_at TEXT NOT NULL
+);
+
+CREATE TABLE creator_tip_profiles (
+ creator_subject TEXT PRIMARY KEY, -- usr_…
+ handle TEXT NOT NULL UNIQUE, -- lower-case Network username (projection)
+ display_name TEXT NOT NULL,
+ avatar_url TEXT,
+ headline TEXT,
+ page_enabled INTEGER NOT NULL DEFAULT 0, -- public page + indexing
+ accepting INTEGER NOT NULL DEFAULT 1, -- new tips accepted
+ min_amount INTEGER NOT NULL DEFAULT 1,
+ paid_message_min INTEGER NOT NULL DEFAULT 100,
+ tts_enabled INTEGER NOT NULL DEFAULT 0,
+ tts_min_amount INTEGER NOT NULL DEFAULT 100,
+ tts_max_chars INTEGER NOT NULL DEFAULT 200,
+ tts_voice TEXT NOT NULL DEFAULT 'gary',
+ media_requests_enabled INTEGER NOT NULL DEFAULT 0,
+ media_request_min INTEGER NOT NULL DEFAULT 25,
+ media_max_seconds INTEGER NOT NULL DEFAULT 600,
+ revision INTEGER NOT NULL DEFAULT 1,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+, page_settings TEXT NOT NULL DEFAULT '{}', filter_words TEXT NOT NULL DEFAULT '[]', filter_action TEXT NOT NULL DEFAULT 'mask', filter_links INTEGER NOT NULL DEFAULT 1);
+
+CREATE TABLE tip_interactions (
+ id TEXT PRIMARY KEY, -- tint_<ULID>
+ creator_subject TEXT NOT NULL,
+ supporter_subject TEXT, -- null: external/anonymous/unmapped
+ supporter_name TEXT, -- display projection at the time
+ kind TEXT NOT NULL CHECK (kind IN ('tip', 'paid_message', 'tts', 'media_request')),
+ amount INTEGER NOT NULL CHECK (amount > 0),
+ currency TEXT NOT NULL DEFAULT 'vibes-bits',
+ amount_cents INTEGER, -- money that arrived (provider/external)
+ message TEXT,
+ request TEXT NOT NULL DEFAULT '{}', -- JSON: tts text/voice, media url, goal_id, target
+ funding TEXT NOT NULL CHECK (funding IN ('credit', 'checkout', 'provider', 'external', 'none')),
+ settlement TEXT NOT NULL CHECK (settlement IN ('billing', 'external', 'simulated', 'imported')),
+ payment_state TEXT NOT NULL CHECK (payment_state IN ('pending', 'settled', 'reversed', 'failed')),
+ delivery_state TEXT NOT NULL CHECK (delivery_state IN ('awaiting_payment', 'queued', 'delivered', 'failed', 'cancelled')),
+ test INTEGER NOT NULL DEFAULT 0, -- simulation or Billing test money: never counted
+ billing_txn_id TEXT UNIQUE, -- the Billing transaction that settled it
+ billing_intent_id TEXT UNIQUE, -- checkout: the Billing payment intent
+ funding_txn_id TEXT UNIQUE, -- checkout: the purchase that funded it
+ reversal_txn_ids TEXT NOT NULL DEFAULT '[]',
+ reversed_bits INTEGER NOT NULL DEFAULT 0, -- bits Billing took back from the creator
+ checkout_url TEXT,
+ checkout_ref TEXT,
+ provider TEXT,
+ provider_ref TEXT,
+ legacy_source TEXT, -- 'live:transactions:<id>' etc.
+ idempotency_key TEXT UNIQUE,
+ failure TEXT,
+ origin TEXT NOT NULL DEFAULT 'tips', -- tips | billing (settled elsewhere) | external | billing-external (announced by Billing, delivered here) | import
+ transfer_due INTEGER NOT NULL DEFAULT 0, -- a Billing transfer still has to be (re)tried
+ transfer_attempts INTEGER NOT NULL DEFAULT 0,
+ next_transfer_at INTEGER NOT NULL DEFAULT 0,
+ target TEXT, -- JSON EntityRef (e.g. a Live stream)
+ created_at TEXT NOT NULL,
+ settled_at TEXT,
+ reversed_at TEXT,
+ delivered_at TEXT,
+ updated_at TEXT NOT NULL, anonymous INTEGER NOT NULL DEFAULT 0, hide_amount INTEGER NOT NULL DEFAULT 0, private_message INTEGER NOT NULL DEFAULT 0, erased_at TEXT, moderation TEXT NOT NULL DEFAULT 'visible', moderated_at TEXT, moderated_by TEXT, filtered INTEGER NOT NULL DEFAULT 0,
+ UNIQUE (provider, provider_ref),
+ UNIQUE (legacy_source)
+);
+
+CREATE TABLE tip_goals (
+ id TEXT PRIMARY KEY, -- tgoal_<ULID>
+ creator_subject TEXT NOT NULL,
+ title TEXT NOT NULL,
+ description TEXT,
+ target_amount INTEGER NOT NULL CHECK (target_amount > 0),
+ currency TEXT NOT NULL DEFAULT 'vibes-bits',
+ image_url TEXT,
+ status TEXT NOT NULL CHECK (status IN ('active', 'closed')),
+ sort_order INTEGER NOT NULL DEFAULT 0,
+ reached_at TEXT,
+ closed_at TEXT,
+ opening_amount INTEGER NOT NULL DEFAULT 0, -- carried over from Live's goal column (not tied to tips)
+ legacy_source TEXT UNIQUE,
+ revision INTEGER NOT NULL DEFAULT 1,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+CREATE TABLE tip_goal_contributions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ goal_id TEXT NOT NULL REFERENCES tip_goals(id),
+ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
+ amount INTEGER NOT NULL CHECK (amount > 0),
+ reversed_amount INTEGER NOT NULL DEFAULT 0,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL,
+ UNIQUE (goal_id, interaction_id)
+);
+
+CREATE TABLE paid_messages (
+ id TEXT PRIMARY KEY, -- tpm_<ULID>
+ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
+ creator_subject TEXT NOT NULL,
+ kind TEXT NOT NULL CHECK (kind IN ('paid_message', 'tts')),
+ text TEXT NOT NULL,
+ voice TEXT,
+ highlight_seconds INTEGER NOT NULL DEFAULT 0,
+ status TEXT NOT NULL CHECK (status IN ('queued', 'delivered', 'failed', 'cancelled')),
+ chat_ref TEXT, -- JSON reference the chat adapter returned
+ test INTEGER NOT NULL DEFAULT 0,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+CREATE TABLE paid_media_requests (
+ id TEXT PRIMARY KEY, -- tmr_<ULID>
+ interaction_id TEXT NOT NULL UNIQUE REFERENCES tip_interactions(id),
+ creator_subject TEXT NOT NULL,
+ url TEXT NOT NULL,
+ provider TEXT,
+ status TEXT NOT NULL CHECK (status IN ('queued', 'accepted', 'played', 'skipped', 'failed', 'cancelled', 'refunded')),
+ queue_ref TEXT, -- JSON reference returned by the queue owner
+ test INTEGER NOT NULL DEFAULT 0,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+CREATE TABLE overlay_configs (
+ id TEXT PRIMARY KEY, -- tovc_<ULID>
+ creator_subject TEXT NOT NULL,
+ kind TEXT NOT NULL CHECK (kind IN ('alerts', 'goal')),
+ name TEXT NOT NULL,
+ settings TEXT NOT NULL DEFAULT '{}',
+ goal_id TEXT,
+ revision INTEGER NOT NULL DEFAULT 1,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL
+);
+
+CREATE TABLE overlay_tokens (
+ id TEXT PRIMARY KEY, -- tovt_<ULID>
+ creator_subject TEXT NOT NULL,
+ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the secret; the secret is shown once
+ scopes TEXT NOT NULL, -- JSON ["alerts","goals"]
+ config_id TEXT,
+ label TEXT,
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ last_used_at TEXT,
+ revoked_at TEXT
+);
+
+CREATE TABLE overlay_deliveries (
+ seq INTEGER PRIMARY KEY AUTOINCREMENT, -- the SSE event id (monotonic)
+ id TEXT NOT NULL UNIQUE, -- tovd_<ULID>
+ creator_subject TEXT NOT NULL,
+ kind TEXT NOT NULL CHECK (kind IN ('alert', 'goal')),
+ interaction_id TEXT,
+ goal_id TEXT,
+ payload TEXT NOT NULL,
+ test INTEGER NOT NULL DEFAULT 0,
+ status TEXT NOT NULL CHECK (status IN ('pending', 'delivered', 'failed')),
+ delivered_at TEXT,
+ failed_at TEXT,
+ sends INTEGER NOT NULL DEFAULT 0, -- times written to an overlay (replays included)
+ dedupe_key TEXT NOT NULL UNIQUE, -- alert:<interaction> | goal:<goal>:<interaction|revision>
+ created_at TEXT NOT NULL
+, hidden INTEGER NOT NULL DEFAULT 0);
+
+CREATE TABLE interaction_effects (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ interaction_id TEXT NOT NULL REFERENCES tip_interactions(id),
+ effect TEXT NOT NULL CHECK (effect IN ('chat_line', 'paid_message', 'tts', 'media_request', 'overlay_alert')),
+ adapter TEXT NOT NULL,
+ state TEXT NOT NULL CHECK (state IN ('queued', 'delivered', 'failed', 'cancelled')),
+ attempts INTEGER NOT NULL DEFAULT 0,
+ next_attempt_at INTEGER NOT NULL DEFAULT 0,
+ last_error TEXT,
+ result TEXT,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL,
+ UNIQUE (interaction_id, effect)
+);
+
+CREATE TABLE migration_maps (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ source TEXT NOT NULL, -- 'live'
+ source_table TEXT NOT NULL,
+ source_id TEXT NOT NULL,
+ target_type TEXT, -- 'interaction' | 'goal'
+ target_id TEXT,
+ status TEXT NOT NULL CHECK (status IN ('imported', 'held', 'excluded')),
+ reason TEXT,
+ billing_txn_id TEXT,
+ run_id TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ updated_at TEXT NOT NULL,
+ UNIQUE (source, source_table, source_id)
+);
+
+CREATE TABLE import_runs (
+ id TEXT PRIMARY KEY,
+ source TEXT NOT NULL,
+ dry_run INTEGER NOT NULL,
+ started_at TEXT NOT NULL,
+ finished_at TEXT,
+ report TEXT
+);
+
+CREATE TABLE tip_moderators (
+ creator_subject TEXT NOT NULL,
+ moderator_subject TEXT NOT NULL,
+ name TEXT, -- display name when they joined (for the creator's list)
+ added_by TEXT NOT NULL, -- the creator, a service, or 'invite:<id>'
+ created_at TEXT NOT NULL,
+ removed_at TEXT,
+ PRIMARY KEY (creator_subject, moderator_subject)
+);
+
+CREATE TABLE tip_moderator_invites (
+ id TEXT PRIMARY KEY, -- tmin_<ULID>
+ creator_subject TEXT NOT NULL,
+ token_hash TEXT NOT NULL UNIQUE, -- sha256 of the link's secret (shown once)
+ created_by TEXT NOT NULL,
+ created_at TEXT NOT NULL,
+ expires_at TEXT NOT NULL,
+ used_at TEXT,
+ used_by TEXT,
+ revoked_at TEXT
+);
+
+CREATE TABLE tip_moderation_log (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ creator_subject TEXT NOT NULL,
+ interaction_id TEXT NOT NULL,
+ action TEXT NOT NULL CHECK (action IN ('held', 'filtered', 'hidden', 'restored')),
+ by_role TEXT NOT NULL CHECK (by_role IN ('filter', 'creator', 'moderator', 'service')),
+ actor TEXT, -- usr_… / svc:… ; null for the filter
+ reason TEXT,
+ created_at TEXT NOT NULL
+);
+
+CREATE TABLE api_idempotency (
+ key TEXT PRIMARY KEY, -- <principal>:<Idempotency-Key>
+ request_hash TEXT NOT NULL,
+ method TEXT NOT NULL,
+ path TEXT NOT NULL,
+ status INTEGER NOT NULL,
+ response TEXT NOT NULL,
+ created_at TEXT NOT NULL
+);
+
+CREATE TABLE event_outbox (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ event_id TEXT NOT NULL UNIQUE,
+ envelope TEXT NOT NULL,
+ traceparent TEXT,
+ created_at INTEGER NOT NULL,
+ attempts INTEGER NOT NULL DEFAULT 0,
+ next_attempt_at INTEGER NOT NULL DEFAULT 0,
+ sent_at INTEGER,
+ seq INTEGER,
+ rejected_at INTEGER,
+ last_error TEXT
+ );
+
+CREATE TABLE idempotency_receipts (
+ consumer TEXT NOT NULL,
+ event_id TEXT NOT NULL,
+ processed_at INTEGER NOT NULL,
+ PRIMARY KEY (consumer, event_id)
+ );
+
+CREATE INDEX idx_tint_creator ON tip_interactions (creator_subject, created_at);
+
+CREATE INDEX idx_tint_supporter ON tip_interactions (supporter_subject, created_at);
+
+CREATE INDEX idx_tint_pending ON tip_interactions (payment_state, funding);
+
+CREATE INDEX idx_goal_creator ON tip_goals (creator_subject, status);
+
+CREATE INDEX idx_ovc_creator ON overlay_configs (creator_subject);
+
+CREATE INDEX idx_ovt_creator ON overlay_tokens (creator_subject);
+
+CREATE INDEX idx_ovd_creator ON overlay_deliveries (creator_subject, seq);
+
+CREATE INDEX idx_ovd_pending ON overlay_deliveries (seq) WHERE status = 'pending';
+
+CREATE INDEX idx_effects_due ON interaction_effects (state, next_attempt_at);
+
+CREATE INDEX idx_tmod_moderator ON tip_moderators (moderator_subject);
+
+CREATE INDEX idx_tmodlog_creator ON tip_moderation_log (creator_subject, id);
+
+CREATE INDEX idx_event_outbox_due ON event_outbox(sent_at, rejected_at, next_attempt_at);
@@ -1,8 +1,13 @@
@@ -1,8 +1,13 @@
'use strict';
'use strict';
/**
/**
- * Boots Tips against the stubs on a random port with a temp database. Jobs are off: tests drive the
+ * Boots Tips against the stubs on a random port with a migrated database of its own (helpers/db.js:
+ * PGlite, or the containers with opts.store 'pg' / TIPS_TEST_STORE=pg). Jobs are off: tests drive the
* effects worker and transfer retries explicitly (domain.effects.drain(), processDueTransfers()).
* effects worker and transfer retries explicitly (domain.effects.drain(), processDueTransfers()).
- * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow).
+ * opts.env overrides the environment; opts.appOpts is passed to createApp (e.g. limitsNow);
+ * opts.valkey an openvibe-sdk/valkey handle (default: the containers' Valkey with TIPS_TEST_STORE=pg,
+ * else none: everything shared stays in the process);
+ * opts.db an existing handle (a second process on the same database); opts.share another instance
+ * whose stubs (Network, Billing, Events, Live) this one uses.
*
*
* t.call(method, path, { body, user, cap, sub, key }) user: a stub user → Bearer user JWT;
* t.call(method, path, { body, user, cap, sub, key }) user: a stub user → Bearer user JWT;
* otherwise a service token with `cap`
* otherwise a service token with `cap`
@@ -16,18 +21,20 @@ const http = require('http');
@@ -16,18 +21,20 @@ const http = require('http');
const crypto = require('crypto');
const crypto = require('crypto');
const { signDelivery, signDeliveryHeaders } = require('openvibe-sdk/events');
const { signDelivery, signDeliveryHeaders } = require('openvibe-sdk/events');
const { startNetwork, startBilling, startEvents, startLive } = require('./stubs');
const { startNetwork, startBilling, startEvents, startLive } = require('./stubs');
+const { testDb, testValkey } = require('./db');
const EVENTS_SECRET = 'e'.repeat(48);
const EVENTS_SECRET = 'e'.repeat(48);
async function boot(opts = {}) {
async function boot(opts = {}) {
- const network = await startNetwork();
- const billing = await startBilling(network);
- const events = await startEvents();
- const live = opts.live ? await startLive(network) : null;
+ // opts.share: another booted instance whose stubs this one uses (a second process of one service).
+ const shared = opts.share || null;
+ const network = shared ? shared.network : await startNetwork();
+ const billing = shared ? shared.billing : await startBilling(network);
+ const events = shared ? shared.events : await startEvents();
+ const live = shared ? shared.live : (opts.live ? await startLive(network) : null);
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-test-'));
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-test-'));
const env = {
const env = {
NODE_ENV: 'test',
NODE_ENV: 'test',
- TIPS_DB_PATH: path.join(dir, 'tips.db'),
BASE_URL: 'http://tips.test',
BASE_URL: 'http://tips.test',
OV_NETWORK_URL: network.url,
OV_NETWORK_URL: network.url,
OV_NETWORK_INTERNAL_URL: network.url,
OV_NETWORK_INTERNAL_URL: network.url,
@@ -53,7 +60,10 @@ async function boot(opts = {}) {
@@ -53,7 +60,10 @@ async function boot(opts = {}) {
const clock = { offset: 0 };
const clock = { offset: 0 };
const logs = [];
const logs = [];
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')) };
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')) };
- const app = createApp({ config, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });
+ const store = opts.db ? { db: opts.db, store: opts.db.store, close: async () => {} } : await testDb({ store: opts.store });
+ const ownValkey = opts.valkey === undefined && process.env.TIPS_TEST_STORE === 'pg' ? testValkey() : null;
+ const valkey = opts.valkey || ownValkey;
+ const app = createApp({ config, db: store.db, valkey, now: () => Date.now() + clock.offset, log, ...(opts.appOpts || {}) });
await app.locals.keys.load();
await app.locals.keys.load();
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;
@@ -135,17 +145,20 @@ async function boot(opts = {}) {
@@ -135,17 +145,20 @@ async function boot(opts = {}) {
return u;
return u;
}
}
- const outboxRows = (type) => domain.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope)).filter((e) => !type || e.event_type === type);
+ /** The events in the outbox (envelopes, oldest first), of one type or all. */
+ const outboxRows = async (type) => (await domain.db.many('SELECT envelope FROM tips_event_outbox ORDER BY id')).map((r) => r.envelope).filter((e) => !type || e.event_type === type);
return {
return {
- app, base, call, deliver, sse, creator, domain, db: domain.db, config, clock, network, billing, events, live, logs, dir, outboxRows,
+ app, base, call, deliver, sse, creator, domain, db: domain.db, store, valkey, config, clock, network, billing, events, live, logs, dir, outboxRows,
adapters: app.locals.adapters,
adapters: app.locals.adapters,
close: async () => {
close: async () => {
- domain.overlays.closeAll();
+ await domain.overlays.close();
+ server.closeAllConnections();
await new Promise((r) => server.close(r));
await new Promise((r) => server.close(r));
await app.locals.outbox.stop();
await app.locals.outbox.stop();
- await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);
- try { domain.db.close(); } catch { /* */ }
+ if (!shared) await Promise.all([network.close(), billing.close(), events.close(), live && live.close()]);
+ await store.close();
+ if (ownValkey) await ownValkey.close();
fs.rmSync(dir, { recursive: true, force: true });
fs.rmSync(dir, { recursive: true, force: true });
},
},
};
};
new file mode 100644
new file mode 100644
@@ -0,0 +1,85 @@
@@ -0,0 +1,85 @@
+'use strict';
+/**
+ * A migrated database for one test run (ADR-035).
+ *
+ * store 'pglite' (the default): real PostgreSQL in-process, migrated (openvibe-sdk/db { pglite }).
+ * store 'pg': the production-shaped containers (openvibe-sdk scripts/test-services.sh up), when
+ * OV_TEST_PG_URL (PgBouncer, transaction mode) and OV_TEST_PG_DIRECT_URL are set. The run gets roles
+ * and a schema of its own, shaped as OpenVibe.Host's roles/data/add-service.sh makes them: an owner
+ * role that migrates on the direct connection, and a runtime role (DML only, statement_timeout 15 s,
+ * lock_timeout 5 s) that serves through PgBouncer. Other suites sharing the database never see them;
+ * close() drops them.
+ *
+ * TIPS_TEST_STORE=pg runs every test file on the containers instead of PGlite.
+ *
+ * const { db, close } = await testDb();
+ * testValkey() the containers' Valkey (OV_TEST_VALKEY_URL) under a prefix of its own, or null
+ */
+const crypto = require('crypto');
+const { createDb } = require('openvibe-sdk/db');
+const { createValkey } = require('openvibe-sdk/valkey');
+const { MIGRATIONS } = require('../../server/db');
+
+const quiet = { log() {}, warn() {}, error: (...a) => console.error(...a) };
+const pgAvailable = () => !!(process.env.OV_TEST_PG_URL && process.env.OV_TEST_PG_DIRECT_URL);
+const valkeyAvailable = () => !!process.env.OV_TEST_VALKEY_URL;
+
+async function testDb({ store = process.env.TIPS_TEST_STORE || 'pglite', max = 4 } = {}) {
+ if (store !== 'pg') {
+ const db = createDb({ pglite: true, service: 'tips-test', log: quiet });
+ await db.migrate({ dir: MIGRATIONS, log: quiet });
+ return { db, store: 'pglite', close: () => db.close().catch(() => {}) };
+ }
+ if (!pgAvailable()) throw new Error('store pg needs OV_TEST_PG_URL and OV_TEST_PG_DIRECT_URL (openvibe-sdk scripts/test-services.sh up)');
+ const name = `tips_t${process.pid}_${crypto.randomBytes(4).toString('hex')}`;
+ const owner = `${name}_owner`;
+ const pw = crypto.randomBytes(16).toString('hex');
+ const su = createDb({ url: process.env.OV_TEST_PG_DIRECT_URL, service: 'tips-test-admin', max: 1, log: quiet });
+ const database = await su.value('SELECT current_database()');
+ for (const stmt of [
+ `CREATE ROLE ${owner} LOGIN PASSWORD '${pw}'`,
+ `CREATE ROLE ${name} LOGIN PASSWORD '${pw}'`,
+ `GRANT CONNECT ON DATABASE ${database} TO ${owner}, ${name}`,
+ `CREATE SCHEMA ${name} AUTHORIZATION ${owner}`,
+ `ALTER ROLE ${owner} SET search_path = ${name}`,
+ `ALTER ROLE ${name} SET search_path = ${name}`,
+ `ALTER ROLE ${name} SET statement_timeout = '15s'`,
+ `ALTER ROLE ${name} SET lock_timeout = '5s'`,
+ `GRANT USAGE ON SCHEMA ${name} TO ${name}`,
+ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ${name}`,
+ `ALTER DEFAULT PRIVILEGES FOR ROLE ${owner} IN SCHEMA ${name} GRANT USAGE, SELECT, UPDATE ON SEQUENCES TO ${name}`,
+ ]) await su.query(stmt);
+ const as = (url, user) => { const u = new URL(url); u.username = user; u.password = pw; return u.toString(); };
+ async function drop() {
+ try {
+ // PgBouncer keeps its server connections after the client pool closes: end them here, or
+ // every run would leave its roles' idle connections holding the server's slots.
+ await su.query('SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE usename = ANY($1)', [[name, owner]]);
+ await su.query(`DROP SCHEMA IF EXISTS ${name} CASCADE`);
+ for (const r of [name, owner]) { await su.query(`DROP OWNED BY ${r}`); await su.query(`DROP ROLE ${r}`); }
+ } finally { await su.close(); }
+ }
+ let db;
+ try {
+ const ownerDb = createDb({ url: as(process.env.OV_TEST_PG_DIRECT_URL, owner), service: 'tips-test-migrate', max: 1, log: quiet });
+ try { await ownerDb.migrate({ dir: MIGRATIONS, log: quiet }); } finally { await ownerDb.close(); }
+ db = createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet });
+ } catch (e) { await drop().catch(() => {}); throw e; } // a failed setup leaves nothing behind
+ return {
+ db, store: 'postgresql', schema: name,
+ /** Another pooled handle on the same database (a second process). */
+ open: (o = {}) => createDb({ url: as(process.env.OV_TEST_PG_URL, name), service: 'tips-test', max, log: quiet, ...o }),
+ async close() {
+ await db.close().catch(() => {});
+ await drop();
+ },
+ };
+}
+
+/** The containers' Valkey under a prefix no other run uses (ov:tips-test:<random>:), or null without it. */
+function testValkey() {
+ if (!valkeyAvailable()) return null;
+ return createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: `ov:tips-test:${crypto.randomBytes(4).toString('hex')}:`, log: quiet });
+}
+
+module.exports = { testDb, testValkey, pgAvailable, valkeyAvailable };
@@ -81,61 +81,61 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
@@ -81,61 +81,61 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
const r = await run({ dryRun: true });
const r = await run({ dryRun: true });
assert.strictEqual(r.dry_run, true);
assert.strictEqual(r.dry_run, true);
assert.ok(r.counts.imported > 0);
assert.ok(r.counts.imported > 0);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 0);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM migration_maps').get().n, 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM migration_maps'), 0);
});
});
let report;
let report;
await check('donations import settled and linked to Billing; failed excluded; unmapped creators held; test era flagged', async () => {
await check('donations import settled and linked to Billing; failed excluded; unmapped creators held; test era flagged', async () => {
report = await run();
report = await run();
- const byLegacy = (id) => t.db.prepare('SELECT * FROM tip_interactions WHERE legacy_source = ?').get(`live:transactions:${id}`);
- const two = byLegacy(2);
+ const byLegacy = (id) => t.db.maybe('SELECT * FROM tip_interactions WHERE legacy_source = $1', [`live:transactions:${id}`]);
+ const two = await byLegacy(2);
assert.strictEqual(two.payment_state, 'settled');
assert.strictEqual(two.payment_state, 'settled');
assert.strictEqual(two.settlement, 'imported');
assert.strictEqual(two.settlement, 'imported');
assert.strictEqual(two.billing_txn_id, 'txn_import_2');
assert.strictEqual(two.billing_txn_id, 'txn_import_2');
assert.strictEqual(two.supporter_subject, subjects[2]);
assert.strictEqual(two.supporter_subject, subjects[2]);
assert.strictEqual(two.message, 'nice');
assert.strictEqual(two.message, 'nice');
assert.strictEqual(two.created_at, '2026-02-01T10:00:00.000Z');
assert.strictEqual(two.created_at, '2026-02-01T10:00:00.000Z');
- assert.strictEqual(byLegacy(1).test, 1, 'before stats_vibes_reset_at');
- const ghost = byLegacy(3);
+ assert.strictEqual((await byLegacy(1)).test, true, 'before stats_vibes_reset_at');
+ const ghost = await byLegacy(3);
assert.strictEqual(ghost.supporter_subject, null);
assert.strictEqual(ghost.supporter_subject, null);
assert.strictEqual(ghost.supporter_name, 'Ghost');
assert.strictEqual(ghost.supporter_name, 'Ghost');
- assert.strictEqual(byLegacy(6).kind, 'media_request');
- assert.strictEqual(byLegacy(8).funding, 'provider');
- assert.strictEqual(byLegacy(4), undefined, 'held, not dropped');
- const held = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get();
+ assert.strictEqual((await byLegacy(6)).kind, 'media_request');
+ assert.strictEqual((await byLegacy(8)).funding, 'provider');
+ assert.strictEqual(await byLegacy(4), null, 'held, not dropped');
+ const held = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'");
assert.strictEqual(held.status, 'held');
assert.strictEqual(held.status, 'held');
- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'").get();
+ const excluded = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'transactions' AND source_id = '5'");
assert.strictEqual(excluded.status, 'excluded');
assert.strictEqual(excluded.status, 'excluded');
assert.match(excluded.reason, /status failed/);
assert.match(excluded.reason, /status failed/);
assert.strictEqual(report.counts.linked_to_billing, report.counts.imported);
assert.strictEqual(report.counts.linked_to_billing, report.counts.imported);
// Imports are history: no effects, no overlay alerts, no events.
// Imports are history: no effects, no overlay alerts, no events.
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM interaction_effects').get().n, 0);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM overlay_deliveries').get().n, 0);
- assert.strictEqual(t.outboxRows('tips.interaction.ready').length, 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM interaction_effects'), 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM overlay_deliveries'), 0);
+ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).length, 0);
});
});
await check('the refund nets out against the donation it undoes', async () => {
await check('the refund nets out against the donation it undoes', async () => {
- const six = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'").get();
+ const six = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:6'");
assert.strictEqual(six.reversed_bits, 250);
assert.strictEqual(six.reversed_bits, 250);
assert.strictEqual(six.payment_state, 'reversed');
assert.strictEqual(six.payment_state, 'reversed');
assert.strictEqual(report.counts.refunds_applied, 1);
assert.strictEqual(report.counts.refunds_applied, 1);
});
});
await check('external PowerChat tips: the direct one is imported as EXTERNAL; the site-routed celebration is not counted twice', async () => {
await check('external PowerChat tips: the direct one is imported as EXTERNAL; the site-routed celebration is not counted twice', async () => {
- const ext = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'").get();
+ const ext = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:chat_messages:2'");
assert.strictEqual(ext.settlement, 'external');
assert.strictEqual(ext.settlement, 'external');
assert.strictEqual(ext.amount, 700);
assert.strictEqual(ext.amount, 700);
assert.strictEqual(ext.supporter_name, 'Direct');
assert.strictEqual(ext.supporter_name, 'Direct');
- const twin = t.db.prepare("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'").get();
+ const twin = await t.db.maybe("SELECT * FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '1'");
assert.strictEqual(twin.status, 'excluded');
assert.strictEqual(twin.status, 'excluded');
assert.match(twin.reason, /site-routed tip \(Live transaction 8\)/);
assert.match(twin.reason, /site-routed tip \(Live transaction 8\)/);
- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'").get().n, 0, 'on-site donation lines are not PowerChat');
+ assert.strictEqual(await t.db.value("SELECT count(*) FROM migration_maps WHERE source_table = 'chat_messages' AND source_id = '3'"), 0, 'on-site donation lines are not PowerChat');
});
});
await check('goals import with Live\'s current amount carried over (not invented contributions)', async () => {
await check('goals import with Live\'s current amount carried over (not invented contributions)', async () => {
- const g = t.db.prepare("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'").get();
- const view = domain.goals.present(g);
+ const g = await t.db.maybe("SELECT * FROM tip_goals WHERE legacy_source = 'live:donation_goals:1'");
+ const view = await domain.goals.present(t.db, g);
assert.strictEqual(view.current_amount, 1200);
assert.strictEqual(view.current_amount, 1200);
assert.strictEqual(view.carried_over_amount, 1200);
assert.strictEqual(view.carried_over_amount, 1200);
assert.strictEqual(view.supporters_count, 0);
assert.strictEqual(view.supporters_count, 0);
@@ -154,34 +154,34 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
@@ -154,34 +154,34 @@ function billingFixture(file, subjects, { withNative = false } = {}) {
const evTest = t.billing.envelope('billing.transaction.settled', { transaction_id: 'txn_native_test', type: 'donation', test: true, from_subject: subjects[2], to_subject: subjects[1], provider: null, metadata: { kind: 'donation', amount_bits: 999 } }, 'txn_native_test');
const evTest = t.billing.envelope('billing.transaction.settled', { transaction_id: 'txn_native_test', type: 'donation', test: true, from_subject: subjects[2], to_subject: subjects[1], provider: null, metadata: { kind: 'donation', amount_bits: 999 } }, 'txn_native_test');
await t.deliver(evTest);
await t.deliver(evTest);
const { reconcile } = require('../server/importer/live');
const { reconcile } = require('../server/importer/live');
- const r = reconcile(domain, billingSnapshotSource(bdb));
+ const r = await reconcile(domain, billingSnapshotSource(bdb));
assert.strictEqual(r.ok, true, JSON.stringify(r.mismatches));
assert.strictEqual(r.ok, true, JSON.stringify(r.mismatches));
- assert.strictEqual(domain.interactions.totals(subjects[1]).settled_via_billing, 740);
- assert.strictEqual(domain.interactions.totals(subjects[1]).external, 700);
+ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).settled_via_billing, 740);
+ assert.strictEqual((await domain.interactions.totals(t.db, subjects[1])).external, 700);
});
});
await check('a re-run changes nothing; a newly mapped creator is released from hold', async () => {
await check('a re-run changes nothing; a newly mapped creator is released from hold', async () => {
- const before = t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n;
+ const before = await t.db.value('SELECT count(*) FROM tip_interactions');
const again = await run();
const again = await run();
assert.strictEqual(again.counts.imported, 0);
assert.strictEqual(again.counts.imported, 0);
assert.strictEqual(again.counts.goals, 0);
assert.strictEqual(again.counts.goals, 0);
assert.ok(again.counts.unchanged > 0);
assert.ok(again.counts.unchanged > 0);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before);
const heldUser = t.network.newUser('held', 4);
const heldUser = t.network.newUser('held', 4);
const r = await run();
const r = await run();
assert.strictEqual(r.counts.released, 1);
assert.strictEqual(r.counts.released, 1);
assert.strictEqual(r.counts.goals, 1);
assert.strictEqual(r.counts.goals, 1);
- const four = t.db.prepare("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'").get();
+ const four = await t.db.maybe("SELECT * FROM tip_interactions WHERE legacy_source = 'live:transactions:4'");
assert.strictEqual(four.creator_subject, heldUser.subject);
assert.strictEqual(four.creator_subject, heldUser.subject);
assert.strictEqual(four.billing_txn_id, 'txn_import_4');
assert.strictEqual(four.billing_txn_id, 'txn_import_4');
- assert.strictEqual(t.db.prepare("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'").get().status, 'imported');
+ assert.strictEqual(await t.db.value("SELECT status FROM migration_maps WHERE source_table = 'transactions' AND source_id = '4'"), 'imported');
});
});
await check('every source row is imported, excluded with a reason, or held — none silently dropped', async () => {
await check('every source row is imported, excluded with a reason, or held — none silently dropped', async () => {
const tipRows = live.prepare("SELECT id FROM transactions WHERE type IN ('donation', 'refund')").all().map((x) => String(x.id));
const tipRows = live.prepare("SELECT id FROM transactions WHERE type IN ('donation', 'refund')").all().map((x) => String(x.id));
- const mapped = new Set(t.db.prepare("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'").all().map((x) => x.source_id));
+ const mapped = new Set((await t.db.many("SELECT source_id FROM migration_maps WHERE source_table = 'transactions'")).map((x) => x.source_id));
assert.deepStrictEqual(tipRows.filter((id) => !mapped.has(id)), []);
assert.deepStrictEqual(tipRows.filter((id) => !mapped.has(id)), []);
- const excluded = t.db.prepare("SELECT * FROM migration_maps WHERE status = 'excluded'").all();
+ const excluded = await t.db.many("SELECT * FROM migration_maps WHERE status = 'excluded'");
assert.ok(excluded.every((x) => x.reason));
assert.ok(excluded.every((x) => x.reason));
});
});
new file mode 100644
new file mode 100644
@@ -0,0 +1,237 @@
@@ -0,0 +1,237 @@
+'use strict';
+// Tips on the production-shaped data tier (ADR-035; the tests of ADR-007's 2026-09-24 amendment):
+// PostgreSQL 18 behind PgBouncer in transaction mode and Valkey 9, from
+// `eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)"`. Two Tips processes (two app
+// instances, each with its own pool and Valkey connection) share one database, one Valkey and the same
+// Billing and Network:
+// - money races: one settlement or one reversal delivered many times at once, to both processes,
+// counts once; the unpaid-checkout limit holds under concurrent requests (serializable)
+// - work claimed with a lease: two effects workers and two transfer retries at once act once each
+// - per-process state in Valkey: per-actor limits, overlay fan-out (a stream on one process hears
+// tips, config changes and revocations made on the other), overlay stream slots
+// - flushing Valkey changes no money answer
+// Without the containers it prints why it skipped; the rest of the suite runs on PGlite.
+const assert = require('assert');
+const { ids } = require('openvibe-contracts');
+const { createValkey } = require('openvibe-sdk/valkey');
+const { testDb, testValkey, pgAvailable, valkeyAvailable } = require('./helpers/db');
+const { boot, check, done } = require('./helpers/app');
+
+if (!pgAvailable()) {
+ console.log('integration (PgBouncer + Valkey): skipped (OV_TEST_PG_URL not set; eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)")');
+ process.exit(0);
+}
+
+const LIMIT_CLOCK = Date.UTC(2026, 8, 28, 12, 0, 10); // one fixed minute for both processes' limiters
+
+(async () => {
+ const store = await testDb({ store: 'pg', max: 6 });
+ const vA = testValkey();
+ const vB = vA ? createValkey({ url: process.env.OV_TEST_VALKEY_URL, prefix: vA.prefix, log: { warn() {} } }) : null;
+ const env = { TIPS_OVERLAY_MAX_STREAMS: '2', TIPS_MAX_PENDING_CHECKOUTS: '3' };
+ const appOpts = { limitsNow: () => LIMIT_CLOCK };
+ const A = await boot({ db: store.db, valkey: vA, env, appOpts });
+ const dbB = store.open();
+ const B = await boot({ db: dbB, valkey: vB, env, appOpts, share: A });
+ const { billing } = A;
+ const db = store.db;
+ const alex = await A.creator('alex');
+ const fan = A.network.newUser('fan');
+ billing.fund(fan.subject, 1_000_000);
+ const goal = (await A.call('POST', '/api/v1/goals', { user: alex, body: { title: 'Desk', target_amount: 100_000 } })).json.goal;
+ const fresh = (e) => ({ ...e, event_id: ids.newId('event') });
+ const both = (n, fn) => Promise.all(Array.from({ length: n }, (_, k) => fn(k % 2 ? B : A, k)));
+
+ await check('the store is PostgreSQL through PgBouncer; Valkey answers', async () => {
+ const r = await A.call('GET', '/api/ready', { token: null });
+ assert.strictEqual(r.status, 200, r.text);
+ assert.strictEqual(r.json.checks.db.detail.store, 'postgresql');
+ assert.strictEqual(r.json.checks.valkey.status, vA ? 'ok' : 'skipped');
+ });
+
+ await check('one Billing settlement delivered 6 times at once, to both processes, settles once', async () => {
+ const ev = billing.foreignDonation({ from: fan.subject, to: alex.subject, amount: 300, provider: 'powerchat' });
+ const out = await both(6, (t) => t.deliver(fresh(ev)));
+ assert.ok(out.every((r) => r.status === 200), JSON.stringify(out.map((r) => [r.status, r.json])));
+ assert.deepStrictEqual(out.map((r) => r.json.outcome).sort(), ['duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'duplicate_transaction', 'recorded']);
+ const rows = await db.many('SELECT id FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);
+ assert.strictEqual(rows.length, 1);
+ assert.strictEqual(await db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [rows[0].id]), 1);
+ assert.strictEqual((await A.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === rows[0].id).length, 1);
+ assert.strictEqual(await db.value("SELECT count(*) FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [rows[0].id]), 1);
+ });
+
+ await check('one reversal delivered 6 times at once takes the bits back once, from the goal too', async () => {
+ const r = await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 400, goal_id: goal.id } });
+ assert.strictEqual(r.status, 201, r.text);
+ const id = r.json.interaction.id;
+ const before = (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount;
+ const rev = billing.refund(r.json.interaction.payment.billing_txn_id);
+ const out = await both(6, (t) => t.deliver(fresh(rev)));
+ assert.ok(out.every((x) => x.status === 200), JSON.stringify(out.map((x) => [x.status, x.json])));
+ assert.strictEqual(out.filter((x) => x.json.outcome === 'duplicate_reversal').length, 5);
+ const i = await db.one('SELECT reversed_bits, reversal_txn_ids, payment_state FROM tip_interactions WHERE id = $1', [id]);
+ assert.deepStrictEqual([i.reversed_bits, i.reversal_txn_ids.length, i.payment_state], [400, 1, 'reversed']);
+ assert.strictEqual((await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount, before - 400);
+ assert.strictEqual(await A.domain.interactions.totals(db, alex.subject).then((x) => x.settled_via_billing), billing.payable.get(alex.subject), 'reconciles with Billing');
+ });
+
+ await check('one Idempotency-Key sent at once to both processes runs once: one interaction, one Billing transfer', async () => {
+ const key = `race-${Date.now()}-key`;
+ const body = { creator: 'alex', amount: 33, message: 'once' };
+ const out = await Promise.all([A, B, A, B].map((t) => t.call('POST', '/api/v1/checkout', { user: fan, body, key })));
+ // The first runs; the others are told it is in progress (409, Retry-After) or get its stored answer.
+ assert.ok(out.every((r) => r.status === 201 || (r.status === 409 && r.json.code === 'idempotency.in_progress')), JSON.stringify(out.map((r) => [r.status, r.json && r.json.code])));
+ const ok = out.filter((r) => r.status === 201);
+ assert.ok(ok.length >= 1);
+ assert.strictEqual(new Set(ok.map((r) => r.json.interaction.id)).size, 1, 'every answer names the one interaction');
+ const id = ok[0].json.interaction.id;
+ assert.strictEqual(billing.transfers().filter((c) => c.key === `tips:transfer:${id}`).length, 1);
+ assert.strictEqual(await db.value('SELECT count(*) FROM tip_interactions WHERE idempotency_key = $1', [`api:${fan.subject}:${key}`]), 1);
+ const later = await B.call('POST', '/api/v1/checkout', { user: fan, body, key });
+ assert.deepStrictEqual([later.status, later.headers.get('idempotent-replayed'), later.json.interaction.id], [201, 'true', id]);
+ });
+
+ await check('a creator\'s overlay deliveries become visible in seq order, however their transactions interleave', async () => {
+ // T1 inserts a delivery and holds its transaction open; T2 (the other process) inserts one for the
+ // same creator. T2 waits for T1, so seq order is commit order and no stream skips a seq.
+ const view = { id: goal.id };
+ let release;
+ const hold = new Promise((ok) => { release = ok; });
+ let t1Seq = null; let t2Done = false;
+ const t1 = A.domain.tx(async (t) => {
+ await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-1-${Date.now()}` });
+ t1Seq = await t.value('SELECT max(seq) FROM overlay_deliveries');
+ await hold;
+ });
+ while (t1Seq === null) await new Promise((ok) => setTimeout(ok, 10));
+ const t2 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-2-${Date.now()}` })).then(() => { t2Done = true; });
+ await new Promise((ok) => setTimeout(ok, 300));
+ const waited = !t2Done;
+ release(); // before any assertion, so a failure reports instead of holding the transaction
+ await Promise.all([t1, t2]);
+ assert.strictEqual(waited, true, 'the second insert waits for the first transaction');
+ const [first, second] = (await db.many("SELECT seq FROM overlay_deliveries WHERE dedupe_key LIKE 'seq-order-%' ORDER BY seq")).map((r) => r.seq);
+ assert.strictEqual(first, t1Seq);
+ assert.ok(second > first);
+ // Another creator is not held up by it.
+ const other = A.network.newUser('other-creator');
+ let otherDone = false;
+ let release2;
+ const hold2 = new Promise((ok) => { release2 = ok; });
+ const t3 = A.domain.tx(async (t) => { await A.domain.overlays.addGoalDelivery(t, alex.subject, view, { reason: 'test', dedupe: `seq-order-3-${Date.now()}` }); await hold2; });
+ await new Promise((ok) => setTimeout(ok, 50));
+ const t4 = B.domain.tx((t) => B.domain.overlays.addGoalDelivery(t, other.subject, view, { reason: 'test', dedupe: `seq-order-4-${Date.now()}` })).then(() => { otherDone = true; });
+ await Promise.race([t4, new Promise((ok) => setTimeout(ok, 2000))]);
+ const unblocked = otherDone;
+ release2();
+ await Promise.all([t3, t4]);
+ assert.strictEqual(unblocked, true, 'another creator\'s insert does not wait');
+ });
+
+ await check('the unpaid-checkout limit (3) holds for 6 concurrent requests on two processes', async () => {
+ const buyer = A.network.newUser('buyer');
+ const out = await both(6, (t, k) => t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 100 + k, pay_with: 'checkout', provider: 'stripe' } }));
+ assert.deepStrictEqual(out.map((r) => r.status).sort(), [201, 201, 201, 429, 429, 429], JSON.stringify(out.map((r) => r.json && r.json.code)));
+ assert.ok(out.filter((r) => r.status === 429).every((r) => r.json.code === 'tips.too_many_pending'));
+ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1 AND payment_state = 'pending'", [buyer.subject]), 3);
+ });
+
+ await check('two effects workers draining at once deliver each chat line once (leased claims)', async () => {
+ const ids6 = [];
+ for (let k = 0; k < 6; k++) ids6.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 10 + k, message: `m${k}` } })).json.interaction.id);
+ await Promise.all([A.domain.effects.drain(), B.domain.effects.drain(), A.domain.effects.drain(), B.domain.effects.drain()]);
+ const jobs = [...A.adapters.test.jobs, ...B.adapters.test.jobs].filter((j) => ids6.includes(j.interaction.id));
+ const per = new Map();
+ for (const j of jobs) per.set(j.delivery_id, (per.get(j.delivery_id) || 0) + 1);
+ assert.strictEqual(per.size, 6);
+ assert.ok([...per.values()].every((n) => n === 1), JSON.stringify([...per]));
+ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND delivery_state = 'delivered'", [ids6]), 6);
+ });
+
+ await check('two processes retrying due transfers at once call Billing once per tip', async () => {
+ billing.state.down = true;
+ const pending = [];
+ for (let k = 0; k < 4; k++) pending.push((await A.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 20 + k } })).json.interaction.id);
+ billing.state.down = false;
+ const mark = billing.calls.length;
+ A.clock.offset = B.clock.offset = 60_000;
+ await Promise.all([A.domain.interactions.processDueTransfers(), B.domain.interactions.processDueTransfers(), A.domain.interactions.processDueTransfers()]);
+ A.clock.offset = B.clock.offset = 0;
+ const calls = billing.calls.slice(mark).filter((c) => c.url === '/api/v1/transfers');
+ assert.deepStrictEqual(calls.map((c) => c.key).sort(), pending.map((id) => `tips:transfer:${id}`).sort(), 'one call each');
+ assert.strictEqual(await db.value("SELECT count(*) FROM tip_interactions WHERE id = ANY($1) AND payment_state = 'settled'", [pending]), 4);
+ });
+
+ if (!valkeyAvailable()) {
+ console.log('integration, shared state in Valkey: skipped (OV_TEST_VALKEY_URL not set)');
+ } else {
+ await check('per-actor limits are counted across processes (POST /me/erase: 3 a minute a person)', async () => {
+ const person = A.network.newUser('eraser');
+ const codes = [];
+ for (const t of [A, B, A, B]) codes.push((await t.call('POST', '/api/v1/me/erase', { user: person })).status);
+ assert.deepStrictEqual(codes, [200, 200, 200, 429]);
+ });
+
+ let secret;
+ let cfg;
+ await check('a stream on process A hears a tip, a config change and a revocation made on process B', async () => {
+ cfg = (await B.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', name: 'Main' } })).json.config;
+ const tok = (await B.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'], config_id: cfg.id } })).json;
+ secret = tok.secret;
+ const s = await A.sse(`/overlay/${secret}/events`);
+ await s.waitFor((e) => e.event === 'hello');
+ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 77, message: 'from B' } });
+ const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === r.json.interaction.id);
+ assert.strictEqual(alert.data.amount, 77);
+ await B.call('PATCH', `/api/v1/overlay-configs/${cfg.id}`, { user: alex, body: { duration_ms: 4000 } });
+ const c = await s.waitFor((e) => e.event === 'config');
+ assert.strictEqual(c.data.config.settings.duration_ms, 4000);
+ await B.call('POST', `/api/v1/overlay-tokens/${tok.token.id}/revoke`, { user: alex });
+ assert.ok(await s.waitFor((e) => e.event === 'revoked'));
+ await new Promise((ok) => setTimeout(ok, 100));
+ assert.strictEqual(s.ended, true);
+ });
+
+ await check('overlay stream slots (2 a token) are counted across processes', async () => {
+ const tok = (await A.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts'] } })).json.secret;
+ const s1 = await A.sse(`/overlay/${tok}/events`);
+ const s2 = await B.sse(`/overlay/${tok}/events`);
+ assert.deepStrictEqual([s1.statusCode, s2.statusCode], [200, 200]);
+ assert.strictEqual((await fetch(`${A.base}/overlay/${tok}/events`)).status, 429, 'the third stream, on either process');
+ assert.strictEqual(await B.domain.overlays.connected(alex.subject), 2);
+ s1.close();
+ await new Promise((ok) => setTimeout(ok, 150));
+ const s3 = await B.sse(`/overlay/${tok}/events`);
+ assert.strictEqual(s3.statusCode, 200, 'a closed stream frees its place for the other process');
+ s2.close(); s3.close();
+ });
+
+ await check('flushing Valkey changes no money answer', async () => {
+ const snapshot = async () => ({
+ totals: await A.domain.interactions.totals(db, alex.subject),
+ goal: (await A.domain.goals.present(db, await A.domain.goals.get(db, goal.id))).current_amount,
+ receipts: (await B.call('GET', '/api/v1/interactions?limit=200', { user: fan })).json.interactions.map((i) => [i.id, i.payment.state, i.amount]),
+ });
+ const before = await snapshot();
+ let cursor = '0'; let deleted = 0;
+ do {
+ const [next, keys] = await vA.client.scan(cursor, 'MATCH', `${vA.prefix}*`, 'COUNT', 500);
+ cursor = next;
+ if (keys.length) deleted += await vA.client.del(...keys);
+ } while (cursor !== '0');
+ assert.ok(deleted > 0, 'there was shared state to lose');
+ assert.deepStrictEqual(await snapshot(), before);
+ assert.strictEqual(before.totals.settled_via_billing, billing.payable.get(alex.subject), 'and it still reconciles with Billing');
+ const r = await B.call('POST', '/api/v1/checkout', { user: fan, body: { creator: 'alex', amount: 5 } });
+ assert.strictEqual(r.json.interaction.payment.state, 'settled', 'tips still settle');
+ });
+ }
+
+ await B.close();
+ await A.close();
+ await dbB.close();
+ await store.close();
+ if (vA) { await vA.close(); await vB.close(); }
+ done();
+})().catch((e) => { console.error(e); process.exit(1); });
new file mode 100644
new file mode 100644
@@ -0,0 +1,161 @@
@@ -0,0 +1,161 @@
+'use strict';
+// migrations/ (ADR-035, ADR-028): the files parse with their phase, apply in order, and apply again
+// without change; the schema carries every column the SQLite release had (nothing lost in the import);
+// every query shape the code runs can be served by an index; and no runtime file loads better-sqlite3.
+const assert = require('assert');
+const fs = require('fs');
+const path = require('path');
+const Database = require('better-sqlite3');
+const { createDb, sql } = require('openvibe-sdk/db');
+const { MIGRATIONS } = require('../server/db');
+const { TABLES } = require('../scripts/migrate-to-postgres');
+const { check, done } = require('./helpers/app');
+
+const quiet = { log() {} };
+const ROOT = path.join(__dirname, '..');
+
+/** The migration files as the SDK reads them: NNNN_name.sql with a phase header (and after: for a contract). */
+function parse(dir) {
+ return fs.readdirSync(dir).filter((f) => f.endsWith('.sql')).sort().map((file) => {
+ const m = /^(\d{4,})_([a-z0-9_-]+)\.sql$/.exec(file);
+ assert.ok(m, `${file} is named NNNN_description.sql`);
+ const head = fs.readFileSync(path.join(dir, file), 'utf8').split('\n').slice(0, 12).join('\n');
+ const phase = (/^--\s*phase:\s*(expand|migrate|contract)\s*$/m.exec(head) || [])[1];
+ assert.ok(phase, `${file} has a phase header`);
+ return { id: m[1], name: m[2], file, phase, after: (/^--\s*after:\s*(\d{4,})\s*$/m.exec(head) || [])[1] || null, transaction: !/^--\s*no-transaction\s*$/m.test(head) };
+ });
+}
+
+(async () => {
+ const db = createDb({ pglite: true, service: 'tips-test', log: { warn() {}, error: console.error } });
+
+ await check('the files parse: numbered, each with its phase; 0001 is the initial expand', async () => {
+ const files = parse(MIGRATIONS);
+ assert.ok(files.length >= 1);
+ assert.deepStrictEqual([files[0].id, files[0].name, files[0].phase, files[0].transaction], ['0001', 'initial', 'expand', true]);
+ assert.deepStrictEqual(files.map((f) => f.id), [...files.map((f) => f.id)].sort(), 'applied in order');
+ for (const f of files) if (f.phase === 'contract') assert.ok(f.after, `${f.file} names its expand`);
+ });
+
+ await check('they apply once, in order, and a second run changes nothing', async () => {
+ const first = await db.migrate({ dir: MIGRATIONS, log: quiet });
+ assert.deepStrictEqual(first.applied.map((m) => m.id), parse(MIGRATIONS).map((m) => m.id));
+ const schema = async () => db.many(sql`SELECT table_name, column_name, data_type, is_nullable, column_default FROM information_schema.columns
+ WHERE table_schema = 'public' ORDER BY table_name, ordinal_position`);
+ const before = await schema();
+ const second = await db.migrate({ dir: MIGRATIONS, log: quiet });
+ assert.deepStrictEqual([second.applied, second.held], [[], []]);
+ assert.deepStrictEqual(await schema(), before, 'the schema is the same after a second run');
+ assert.strictEqual(await db.value(sql`SELECT count(*) FROM ov_migrations`), parse(MIGRATIONS).length);
+ });
+
+ await check('every table has a primary key; money and time columns have their types', async () => {
+ const noKey = await db.many(sql`SELECT c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
+ WHERE n.nspname = 'public' AND c.relkind = 'r' AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.oid AND i.indisprimary)`);
+ assert.deepStrictEqual(noKey, []);
+ const type = async (t, c) => db.value(sql`SELECT data_type FROM information_schema.columns WHERE table_schema = 'public' AND table_name = ${t} AND column_name = ${c}`);
+ for (const [t, c] of [['tip_interactions', 'amount'], ['tip_interactions', 'reversed_bits'], ['tip_goals', 'target_amount'], ['tip_goal_contributions', 'amount'], ['tip_interactions', 'next_transfer_at'], ['interaction_effects', 'next_attempt_at']]) {
+ assert.strictEqual(await type(t, c), 'bigint', `${t}.${c}`);
+ }
+ for (const [t, c] of [['tip_interactions', 'created_at'], ['tip_interactions', 'settled_at'], ['overlay_deliveries', 'created_at'], ['api_idempotency', 'created_at']]) {
+ assert.strictEqual(await type(t, c), 'timestamp with time zone', `${t}.${c}`);
+ }
+ for (const [t, c] of [['tip_interactions', 'test'], ['tip_interactions', 'transfer_due'], ['creator_tip_profiles', 'page_enabled'], ['overlay_deliveries', 'hidden']]) {
+ assert.strictEqual(await type(t, c), 'boolean', `${t}.${c}`);
+ }
+ for (const [t, c] of [['tip_interactions', 'request'], ['overlay_deliveries', 'payload'], ['tips_event_outbox', 'envelope'], ['overlay_tokens', 'scopes']]) {
+ assert.strictEqual(await type(t, c), 'jsonb', `${t}.${c}`);
+ }
+ assert.strictEqual(await type('api_idempotency', 'response'), 'text', 'replayed byte for byte');
+ });
+
+ await check('nothing the SQLite release stored is left behind: every source table and column has a target', async () => {
+ const s = new Database(':memory:');
+ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));
+ const from = (target) => (TABLES[target] && TABLES[target].from) || target;
+ const targets = new Map();
+ for (const r of await db.many(sql`SELECT table_name, column_name FROM information_schema.columns WHERE table_schema = 'public' AND table_name <> 'ov_migrations'`)) {
+ const src = from(r.table_name);
+ if (!targets.has(src)) targets.set(src, new Set());
+ targets.get(src).add(r.column_name);
+ }
+ const missing = [];
+ for (const { name } of s.prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'").all()) {
+ if (name === 'settings') continue; // not carried on purpose: ov_migrations records the schema (0001_initial.sql)
+ if (!targets.has(name)) { missing.push(name); continue; }
+ for (const c of s.prepare(`PRAGMA table_info(${name})`).all()) if (!targets.get(name).has(c.name)) missing.push(`${name}.${c.name}`);
+ }
+ s.close();
+ assert.deepStrictEqual(missing, [], 'no table or column dropped (dropColumns stays empty)');
+ assert.ok(Object.values(TABLES).every((o) => !o.dropColumns), 'no dropColumns');
+ });
+
+ await check('every query shape the code runs can be served by an index (no sequential scan of its table)', async () => {
+ // With sequential scans switched off for the plan, a shape that has no usable index still plans a
+ // Seq Scan (at a prohibitive cost): that is what this looks for.
+ const shapes = {
+ 'interactions of a creator, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND (created_at, id) < (now(), 'x') ORDER BY created_at DESC, id DESC LIMIT 51`],
+ 'receipts of a supporter, a keyset page': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE supporter_subject = 's' AND NOT test ORDER BY created_at DESC, id DESC LIMIT 51`],
+ 'the moderation queue by state': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state IN ('settled', 'reversed', 'pending') AND moderation = 'held' ORDER BY created_at DESC, id DESC LIMIT 51`],
+ 'unpaid checkouts of a supporter': ['tip_interactions', sql`SELECT count(*) FROM tip_interactions WHERE supporter_subject = 's' AND funding = 'checkout' AND payment_state = 'pending' AND created_at >= now()`],
+ 'due transfers (the lease claim)': ['tip_interactions', sql`SELECT id FROM tip_interactions WHERE payment_state = 'pending' AND transfer_due AND next_transfer_at <= 5 ORDER BY next_transfer_at LIMIT 10 FOR UPDATE SKIP LOCKED`],
+ 'the supporters leaderboard': ['tip_interactions', sql`SELECT supporter_subject, SUM(amount - reversed_bits) FROM tip_interactions WHERE creator_subject = 'c' AND supporter_subject IS NOT NULL AND NOT test
+ AND payment_state IN ('settled', 'reversed') AND NOT anonymous AND NOT hide_amount AND erased_at IS NULL AND moderation = 'visible' GROUP BY supporter_subject`],
+ 'recent public messages': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE creator_subject = 'c' AND payment_state = 'settled' AND message IS NOT NULL AND NOT test ORDER BY settled_at DESC, id DESC LIMIT 20`],
+ 'an interaction by Billing transaction': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_txn_id = 't'`],
+ 'an interaction by checkout intent': ['tip_interactions', sql`SELECT * FROM tip_interactions WHERE billing_intent_id = 'i'`],
+ 'an interaction by provider reference': ['tip_interactions', sql`SELECT 1 FROM tip_interactions WHERE provider = 'powerchat' AND provider_ref = 'r'`],
+ 'goals of a creator': ['tip_goals', sql`SELECT * FROM tip_goals WHERE creator_subject = 'c' AND status = 'active' ORDER BY sort_order, created_at`],
+ 'goal totals': ['tip_goal_contributions', sql`SELECT goal_id, SUM(amount - reversed_amount) FROM tip_goal_contributions WHERE goal_id = ANY('{a,b}'::text[]) GROUP BY goal_id`],
+ 'the latest supporters of a goal': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE goal_id = 'g' ORDER BY id DESC LIMIT 10`],
+ 'the contributions of an interaction': ['tip_goal_contributions', sql`SELECT * FROM tip_goal_contributions WHERE interaction_id = 'i'`],
+ 'due effects (the lease claim)': ['interaction_effects', sql`SELECT id FROM interaction_effects WHERE state = 'queued' AND next_attempt_at <= 5 ORDER BY id LIMIT 20 FOR UPDATE SKIP LOCKED`],
+ 'the effects of a page of interactions': ['interaction_effects', sql`SELECT * FROM interaction_effects WHERE interaction_id = ANY('{a,b}'::text[]) ORDER BY interaction_id, id`],
+ 'an overlay stream after a seq': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE creator_subject = 'c' AND seq > 5 AND status <> 'failed' AND NOT hidden ORDER BY seq LIMIT 200`],
+ 'the delivery window sweep': ['overlay_deliveries', sql`SELECT seq FROM overlay_deliveries WHERE status = 'pending' AND NOT hidden AND created_at < now() ORDER BY seq LIMIT 500`],
+ 'the deliveries of an interaction': ['overlay_deliveries', sql`SELECT * FROM overlay_deliveries WHERE interaction_id = 'i'`],
+ 'overlay tokens of a creator': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE creator_subject = 'c' ORDER BY revoked_at IS NOT NULL, created_at DESC`],
+ 'an overlay token by its secret': ['overlay_tokens', sql`SELECT * FROM overlay_tokens WHERE token_hash = 'h'`],
+ 'overlay configs of a creator': ['overlay_configs', sql`SELECT * FROM overlay_configs WHERE creator_subject = 'c' ORDER BY created_at`],
+ 'a creator by handle': ['creator_tip_profiles', sql`SELECT * FROM creator_tip_profiles WHERE handle = 'h'`],
+ 'switched-on pages (home, sitemap)': ['creator_tip_profiles', sql`SELECT handle FROM creator_tip_profiles WHERE page_enabled ORDER BY updated_at DESC LIMIT 24`],
+ 'the creators a person moderates': ['tip_moderators', sql`SELECT * FROM tip_moderators WHERE moderator_subject = 's' AND removed_at IS NULL`],
+ 'open invitations of a creator': ['tip_moderator_invites', sql`SELECT * FROM tip_moderator_invites WHERE creator_subject = 'c' AND used_at IS NULL ORDER BY created_at`],
+ 'the moderation log of a creator': ['tip_moderation_log', sql`SELECT * FROM tip_moderation_log WHERE creator_subject = 'c' ORDER BY id DESC LIMIT 100`],
+ 'the weekly prune of stored answers': ['api_idempotency', sql`DELETE FROM api_idempotency WHERE created_at < now()`],
+ 'an erasure scrubbing outbox copies': ['tips_event_outbox', sql`SELECT id FROM tips_event_outbox WHERE envelope->'subject'->>'type' = 'interaction' AND envelope->'subject'->>'id' = ANY('{a}'::text[])`],
+ 'the rejected-events count': ['tips_event_outbox', sql`SELECT count(*) FROM tips_event_outbox WHERE rejected_at IS NOT NULL`],
+ 'a Live goal through the import map': ['migration_maps', sql`SELECT target_id FROM migration_maps WHERE source = 'live' AND source_table = 'donation_goals' AND source_id = '12'`],
+ };
+ const scans = [];
+ for (const [name, [table, q]] of Object.entries(shapes)) {
+ const plan = await db.tx(async (t) => {
+ await t.query('SET LOCAL enable_seqscan = off');
+ return t.value(sql`EXPLAIN (FORMAT JSON) ${q}`);
+ });
+ const nodes = [];
+ const walk = (n) => { nodes.push(n); (n.Plans || []).forEach(walk); };
+ walk(plan[0].Plan);
+ if (nodes.some((n) => n['Node Type'] === 'Seq Scan' && n['Relation Name'] === table)) scans.push(name);
+ }
+ assert.deepStrictEqual(scans, []);
+ });
+
+ await check('no runtime file loads better-sqlite3 (only the import tools may)', async () => {
+ const offenders = [];
+ const walk = (dir) => {
+ for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
+ const p = path.join(dir, e.name);
+ if (e.isDirectory()) walk(p);
+ else if (e.name.endsWith('.js') && /require\(['"]better-sqlite3['"]\)|\.prepare\(|db\.transaction\(/.test(fs.readFileSync(p, 'utf8').replace(/(live|bdb|snap)\.prepare\(/g, ''))) offenders.push(path.relative(ROOT, p));
+ }
+ };
+ walk(path.join(ROOT, 'server'));
+ assert.deepStrictEqual(offenders, []);
+ const loaders = fs.readdirSync(path.join(ROOT, 'scripts')).filter((f) => /require\(['"]better-sqlite3['"]\)/.test(fs.readFileSync(path.join(ROOT, 'scripts', f), 'utf8')));
+ assert.deepStrictEqual(loaders, ['import-live.js'], "the Live import reads Live's and Billing's snapshot files; migrate-to-postgres.js reads Tips' through importSqlite");
+ });
+
+ await db.close();
+ done();
+})().catch((e) => { console.error(e); process.exit(1); });
@@ -29,7 +29,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -29,7 +29,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const pay = (kind, body) => t.call('POST', { tip: '/api/v1/checkout', paid_message: '/api/v1/paid-messages', tts: '/api/v1/tts-requests' }[kind], { user: viewer, body: { creator: 'alex', ...body } });
const pay = (kind, body) => t.call('POST', { tip: '/api/v1/checkout', paid_message: '/api/v1/paid-messages', tts: '/api/v1/tts-requests' }[kind], { user: viewer, body: { creator: 'alex', ...body } });
const state = async () => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts;
const state = async () => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts;
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
- const moderated = (id) => t.outboxRows('tips.interaction.moderated').filter((e) => e.subject.id === id).map((e) => e.payload);
+ const moderated = async (id) => (await t.outboxRows('tips.interaction.moderated')).filter((e) => e.subject.id === id).map((e) => e.payload);
const cookie = (u) => `ov_token=${t.network.signUser(u)}`;
const cookie = (u) => `ov_token=${t.network.signUser(u)}`;
const get = (p, u) => fetch(t.base + p, { headers: u ? { Cookie: cookie(u) } : {}, redirect: 'manual' }).then(async (r) => ({ status: r.status, headers: r.headers, text: await r.text() }));
const get = (p, u) => fetch(t.base + p, { headers: u ? { Cookie: cookie(u) } : {}, redirect: 'manual' }).then(async (r) => ({ status: r.status, headers: r.headers, text: await r.text() }));
const post = (p, u, form) => fetch(t.base + p, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie(u) }, body: new URLSearchParams(form).toString() })
const post = (p, u, form) => fetch(t.base + p, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie(u) }, body: new URLSearchParams(form).toString() })
@@ -62,7 +62,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -62,7 +62,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.match(own.message, /BADWORD/, 'the creator reads the original');
assert.match(own.message, /BADWORD/, 'the creator reads the original');
assert.strictEqual(own.moderation.filtered, true);
assert.strictEqual(own.moderation.filtered, true);
assert.strictEqual(own.moderation.state, 'visible');
assert.strictEqual(own.moderation.state, 'visible');
- const [ev] = moderated(id);
+ const [ev] = await moderated(id);
assert.deepStrictEqual(ev, { interaction_id: id, creator: { type: 'user', id: alex.subject }, action: 'filtered', by: 'filter', moderation_state: 'visible', cancelled_effects: [] });
assert.deepStrictEqual(ev, { interaction_id: id, creator: { type: 'user', id: alex.subject }, action: 'filtered', by: 'filter', moderation_state: 'visible', cancelled_effects: [] });
const tts = await pay('tts', { amount: 150, tts: { text: 'say badword now please' } });
const tts = await pay('tts', { amount: 150, tts: { text: 'say badword now please' } });
await domain.effects.drain();
await domain.effects.drain();
@@ -74,7 +74,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -74,7 +74,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
let heldId;
let heldId;
await check('hold: nothing is shown, posted or read until someone reviews it; the money counts at once', async () => {
await check('hold: nothing is shown, posted or read until someone reviews it; the money counts at once', async () => {
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { action: 'hold' } } });
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { action: 'hold' } } });
- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
+ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
+ const goalBefore = await goalNow();
const s = await t.sse(`/overlay/${secret}/events`);
const s = await t.sse(`/overlay/${secret}/events`);
await s.waitFor((e) => e.event === 'hello');
await s.waitFor((e) => e.event === 'hello');
const r = await pay('tts', { amount: 300, supporter_name: 'Viewer', tts: { text: 'read this badword aloud' } });
const r = await pay('tts', { amount: 300, supporter_name: 'Viewer', tts: { text: 'read this badword aloud' } });
@@ -85,13 +86,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -85,13 +86,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
s.close();
s.close();
assert.ok(!(await state()).some((x) => x.interaction_id === heldId));
assert.ok(!(await state()).some((x) => x.interaction_id === heldId));
assert.strictEqual(jobsOf(heldId).length, 0, 'no chat line, no TTS');
assert.strictEqual(jobsOf(heldId).length, 0, 'no chat line, no TTS');
- const i = domain.interactions.get(heldId);
+ const i = await domain.interactions.get(t.db, heldId);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.moderation, 'held');
assert.strictEqual(i.moderation, 'held');
assert.strictEqual(i.delivery_state, 'queued', 'waiting for review, not "delivered"');
assert.strictEqual(i.delivery_state, 'queued', 'waiting for review, not "delivered"');
- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 300);
- assert.strictEqual(t.outboxRows('tips.interaction.ready').filter((e) => e.subject.id === heldId).length, 1);
- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);
+ assert.strictEqual(await goalNow(), goalBefore + 300);
+ assert.strictEqual((await t.outboxRows('tips.interaction.ready')).filter((e) => e.subject.id === heldId).length, 1);
+ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held']]);
const q = await t.call('GET', '/api/v1/moderation?creator=alex&state=held', { user: alex });
const q = await t.call('GET', '/api/v1/moderation?creator=alex&state=held', { user: alex });
assert.deepStrictEqual(q.json.interactions.map((x) => x.id), [heldId]);
assert.deepStrictEqual(q.json.interactions.map((x) => x.id), [heldId]);
assert.strictEqual(q.json.interactions[0].tts_text, 'read this badword aloud', 'reviewers read what was written');
assert.strictEqual(q.json.interactions[0].tts_text, 'read this badword aloud', 'reviewers read what was written');
@@ -108,7 +109,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -108,7 +109,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const url = inv.text.match(/value="(http:\/\/tips\.test\/moderate\/invite\/tmin_[^"]+)"/)[1];
const url = inv.text.match(/value="(http:\/\/tips\.test\/moderate\/invite\/tmin_[^"]+)"/)[1];
const path = url.replace('http://tips.test', '');
const path = url.replace('http://tips.test', '');
assert.ok(!(await get('/dashboard', alex)).text.includes(url), 'shown once');
assert.ok(!(await get('/dashboard', alex)).text.includes(url), 'shown once');
- assert.ok(!JSON.stringify(t.db.prepare('SELECT * FROM tip_moderator_invites').all()).includes(path.split('/').pop()), 'stored hashed');
+ assert.ok(!JSON.stringify(await t.db.many('SELECT * FROM tip_moderator_invites')).includes(path.split('/').pop()), 'stored hashed');
assert.match((await get(path)).headers.get('location'), /^\/auth\/login/);
assert.match((await get(path)).headers.get('location'), /^\/auth\/login/);
const page = await get(path, mod);
const page = await get(path, mod);
assert.strictEqual(page.status, 200);
assert.strictEqual(page.status, 200);
@@ -139,10 +140,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -139,10 +140,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.strictEqual(alert.data.tts.text, 'read this aloud', 'blocked words stay out of what is read');
assert.strictEqual(alert.data.tts.text, 'read this aloud', 'blocked words stay out of what is read');
await domain.effects.drain();
await domain.effects.drain();
assert.deepStrictEqual(jobsOf(heldId).map((j) => j.effect).sort(), ['chat_line', 'tts']);
assert.deepStrictEqual(jobsOf(heldId).map((j) => j.effect).sort(), ['chat_line', 'tts']);
- const i = domain.interactions.get(heldId);
+ const i = await domain.interactions.get(t.db, heldId);
assert.strictEqual(i.moderation, 'visible');
assert.strictEqual(i.moderation, 'visible');
assert.strictEqual(i.delivery_state, 'delivered');
assert.strictEqual(i.delivery_state, 'delivered');
- assert.deepStrictEqual(moderated(heldId).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);
+ assert.deepStrictEqual((await moderated(heldId)).map((e) => [e.action, e.by, e.moderation_state]), [['held', 'filter', 'held'], ['restored', 'moderator', 'visible']]);
});
});
let hiddenId;
let hiddenId;
@@ -154,7 +155,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -154,7 +155,8 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r = await pay('tts', { amount: 500, supporter_name: 'Viewer', message: 'nasty but not blocked', tts: { text: 'nasty but not blocked' } });
const r = await pay('tts', { amount: 500, supporter_name: 'Viewer', message: 'nasty but not blocked', tts: { text: 'nasty but not blocked' } });
hiddenId = r.json.interaction.id;
hiddenId = r.json.interaction.id;
const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === hiddenId);
const alert = await s.waitFor((e) => e.event === 'alert' && e.data.interaction_id === hiddenId);
- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
+ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
+ const goalBefore = await goalNow();
const h = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: mod, body: { reason: 'abuse' } });
const h = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: mod, body: { reason: 'abuse' } });
assert.strictEqual(h.status, 200, h.text);
assert.strictEqual(h.status, 200, h.text);
assert.strictEqual(h.json.changed, true);
assert.strictEqual(h.json.changed, true);
@@ -164,11 +166,11 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -164,11 +166,11 @@ const validModerated = payloadValidator('tips.interaction.moderated');
s.close();
s.close();
await domain.effects.drain();
await domain.effects.drain();
assert.strictEqual(jobsOf(hiddenId).length, 0, 'nothing was posted or read');
assert.strictEqual(jobsOf(hiddenId).length, 0, 'nothing was posted or read');
- const i = domain.interactions.get(hiddenId);
+ const i = await domain.interactions.get(t.db, hiddenId);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'cancelled');
assert.strictEqual(i.delivery_state, 'cancelled');
assert.strictEqual(billing.payable.get(alex.subject), payableBefore + 500, 'the payment stands');
assert.strictEqual(billing.payable.get(alex.subject), payableBefore + 500, 'the payment stands');
- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore, 'the goal keeps it');
+ assert.strictEqual(await goalNow(), goalBefore, 'the goal keeps it');
assert.ok(!(await state()).some((x) => x.interaction_id === hiddenId));
assert.ok(!(await state()).some((x) => x.interaction_id === hiddenId));
const replay = await t.sse(`/overlay/${secret}/events`, { lastEventId: alert.id - 1 });
const replay = await t.sse(`/overlay/${secret}/events`, { lastEventId: alert.id - 1 });
await replay.waitFor((e) => e.event === 'hello');
await replay.waitFor((e) => e.event === 'hello');
@@ -177,13 +179,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -177,13 +179,13 @@ const validModerated = payloadValidator('tips.interaction.moderated');
replay.close();
replay.close();
const sup = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json;
const sup = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json;
assert.ok(!sup.recent.some((v) => v.interaction_id === hiddenId));
assert.ok(!sup.recent.some((v) => v.interaction_id === hiddenId));
- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');
- const [ev] = moderated(hiddenId);
+ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === hiddenId).length, 0, 'not a payment cancellation');
+ const [ev] = await moderated(hiddenId);
assert.deepStrictEqual({ ...ev, cancelled_effects: ev.cancelled_effects.sort() }, { interaction_id: hiddenId, creator: { type: 'user', id: alex.subject }, action: 'hidden', by: 'moderator', moderation_state: 'hidden', cancelled_effects: ['chat_line', 'tts'] });
assert.deepStrictEqual({ ...ev, cancelled_effects: ev.cancelled_effects.sort() }, { interaction_id: hiddenId, creator: { type: 'user', id: alex.subject }, action: 'hidden', by: 'moderator', moderation_state: 'hidden', cancelled_effects: ['chat_line', 'tts'] });
assert.ok(!JSON.stringify(ev).includes('abuse') && !JSON.stringify(ev).includes(mod.subject), 'the event carries no note and no moderator');
assert.ok(!JSON.stringify(ev).includes('abuse') && !JSON.stringify(ev).includes(mod.subject), 'the event carries no note and no moderator');
const again = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: alex });
const again = await t.call('POST', `/api/v1/interactions/${hiddenId}/hide`, { user: alex });
assert.strictEqual(again.json.changed, false);
assert.strictEqual(again.json.changed, false);
- assert.strictEqual(moderated(hiddenId).length, 1, 'hiding twice publishes once');
+ assert.strictEqual((await moderated(hiddenId)).length, 1, 'hiding twice publishes once');
const receipt = (await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: viewer })).json.interaction;
const receipt = (await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: viewer })).json.interaction;
assert.strictEqual(receipt.moderation.state, 'hidden');
assert.strictEqual(receipt.moderation.state, 'hidden');
assert.strictEqual(receipt.payment.state, 'settled');
assert.strictEqual(receipt.payment.state, 'settled');
@@ -196,7 +198,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -196,7 +198,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
await domain.effects.drain();
await domain.effects.drain();
assert.strictEqual(jobsOf(hiddenId).length, 0, 'no late TTS');
assert.strictEqual(jobsOf(hiddenId).length, 0, 'no late TTS');
assert.ok((await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.recent.some((v) => v.interaction_id === hiddenId));
assert.ok((await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.recent.some((v) => v.interaction_id === hiddenId));
- assert.deepStrictEqual(moderated(hiddenId).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);
+ assert.deepStrictEqual((await moderated(hiddenId)).map((e) => [e.action, e.by]), [['hidden', 'moderator'], ['restored', 'creator']]);
});
});
await check('who may moderate: the creator, their moderators, services with tips.interaction.moderate; nobody else', async () => {
await check('who may moderate: the creator, their moderators, services with tips.interaction.moderate; nobody else', async () => {
@@ -208,7 +210,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -208,7 +210,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.get'] })).status, 403);
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.get'] })).status, 403);
const svc = await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.moderate'], sub: 'svc:chat' });
const svc = await t.call('POST', `/api/v1/interactions/${id}/hide`, { cap: ['tips.interaction.moderate'], sub: 'svc:chat' });
assert.strictEqual(svc.status, 200, svc.text);
assert.strictEqual(svc.status, 200, svc.text);
- assert.strictEqual(moderated(id)[0].by, 'service');
+ assert.strictEqual((await moderated(id))[0].by, 'service');
assert.strictEqual((await get('/moderate/alex', stranger)).status, 404);
assert.strictEqual((await get('/moderate/alex', stranger)).status, 404);
assert.strictEqual((await t.call('GET', '/api/v1/moderation?creator=alex', { user: stranger })).status, 403);
assert.strictEqual((await t.call('GET', '/api/v1/moderation?creator=alex', { user: stranger })).status, 403);
const log = await t.call('GET', '/api/v1/moderation/log?creator=alex', { user: alex });
const log = await t.call('GET', '/api/v1/moderation/log?creator=alex', { user: alex });
@@ -227,10 +229,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -227,10 +229,10 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r = await t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 150, message: 'later', pay_with: 'checkout', provider: 'stripe' } });
const r = await t.call('POST', '/api/v1/checkout', { user: buyer, body: { creator: 'alex', amount: 150, message: 'later', pay_with: 'checkout', provider: 'stripe' } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { user: alex })).status, 200);
assert.strictEqual((await t.call('POST', `/api/v1/interactions/${id}/hide`, { user: alex })).status, 200);
- await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));
+ await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));
await new Promise((ok) => setTimeout(ok, 50));
await new Promise((ok) => setTimeout(ok, 50));
await domain.interactions.processDueTransfers();
await domain.interactions.processDueTransfers();
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'cancelled');
assert.strictEqual(i.delivery_state, 'cancelled');
await domain.effects.drain();
await domain.effects.drain();
@@ -242,7 +244,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -242,7 +244,7 @@ const validModerated = payloadValidator('tips.interaction.moderated');
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { words: ['badword'], action: 'hold' } } });
await t.call('PATCH', '/api/v1/profiles/me', { user: alex, body: { filter: { words: ['badword'], action: 'hold' } } });
const r = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'a badword test' } });
const r = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'a badword test' } });
assert.strictEqual(r.json.interaction.moderation.state, 'held');
assert.strictEqual(r.json.interaction.moderation.state, 'held');
- assert.strictEqual(moderated(r.json.interaction.id).length, 0);
+ assert.strictEqual((await moderated(r.json.interaction.id)).length, 0);
const shown = await t.call('POST', `/api/v1/interactions/${r.json.interaction.id}/restore`, { user: alex });
const shown = await t.call('POST', `/api/v1/interactions/${r.json.interaction.id}/restore`, { user: alex });
assert.strictEqual(shown.json.interaction.moderation.state, 'visible');
assert.strictEqual(shown.json.interaction.moderation.state, 'visible');
assert.strictEqual((await state()).find((x) => x.interaction_id === r.json.interaction.id).message, 'a ******* test');
assert.strictEqual((await state()).find((x) => x.interaction_id === r.json.interaction.id).message, 'a ******* test');
@@ -250,16 +252,16 @@ const validModerated = payloadValidator('tips.interaction.moderated');
@@ -250,16 +252,16 @@ const validModerated = payloadValidator('tips.interaction.moderated');
const r2 = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'another badword' } });
const r2 = await t.call('POST', '/api/v1/simulate', { user: alex, body: { kind: 'paid_message', amount: 200, message: 'another badword' } });
const id2 = r2.json.interaction.id;
const id2 = r2.json.interaction.id;
await t.call('POST', `/api/v1/interactions/${id2}/hide`, { user: alex });
await t.call('POST', `/api/v1/interactions/${id2}/hide`, { user: alex });
- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'cancelled');
+ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'cancelled');
await t.call('POST', `/api/v1/interactions/${id2}/restore`, { user: alex });
await t.call('POST', `/api/v1/interactions/${id2}/restore`, { user: alex });
await domain.effects.drain();
await domain.effects.drain();
- assert.strictEqual(domain.interactions.get(id2).delivery_state, 'delivered');
+ assert.strictEqual((await domain.interactions.get(t.db, id2)).delivery_state, 'delivered');
assert.strictEqual(jobsOf(id2).length, 1);
assert.strictEqual(jobsOf(id2).length, 1);
assert.ok((await state()).some((x) => x.interaction_id === id2));
assert.ok((await state()).some((x) => x.interaction_id === id2));
});
});
await check('every event is a valid envelope with a valid payload', async () => {
await check('every event is a valid envelope with a valid payload', async () => {
- for (const e of t.outboxRows()) {
+ for (const e of await t.outboxRows()) {
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
const v = e.event_type === 'tips.interaction.moderated' ? validModerated(e.payload) : payloadValidator(e.event_type)(e.payload);
const v = e.event_type === 'tips.interaction.moderated' ? validModerated(e.payload) : payloadValidator(e.event_type)(e.payload);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)} ${JSON.stringify(e.payload)}`);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)} ${JSON.stringify(e.payload)}`);
@@ -1,6 +1,7 @@
@@ -1,6 +1,7 @@
'use strict';
'use strict';
// Track O: GET /metrics answers direct loopback callers only, labels requests by route template and
// Track O: GET /metrics answers direct loopback callers only, labels requests by route template and
-// carries the Tips gauges; /api/ready is 503 only when the database fails, and a missing Billing,
+// carries the Tips gauges; /api/ready is 503 only when the database fails, reports the store that
+// answered (db.ready()) and Valkey (valkey.ready(), skipped without VALKEY_URL), and a missing Billing,
// Events or Network key degrades it instead.
// Events or Network key degrades it instead.
const assert = require('assert');
const assert = require('assert');
const nodeHttp = require('http');
const nodeHttp = require('http');
@@ -21,13 +22,17 @@ function get(base, p, headers = {}) {
@@ -21,13 +22,17 @@ function get(base, p, headers = {}) {
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.json.status, 'ready');
assert.strictEqual(r.json.status, 'ready');
assert.strictEqual(r.json.service, 'tips');
assert.strictEqual(r.json.service, 'tips');
- assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'network_jwks', 'billing', 'events']);
+ assert.deepStrictEqual(Object.keys(r.json.checks), ['db', 'valkey', 'network_jwks', 'billing', 'events']);
for (const [name, c] of Object.entries(r.json.checks)) {
for (const [name, c] of Object.entries(r.json.checks)) {
- assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);
+ // Without Valkey: skipped with its reason, never claimed "ok"; with it, a real PING.
+ if (name === 'valkey' && !t.valkey) { assert.strictEqual(c.status, 'skipped', JSON.stringify(c)); assert.match(c.reason, /VALKEY_URL unset/); assert.deepStrictEqual(r.json.skipped, ['valkey']); }
+ else assert.strictEqual(c.status, 'ok', `${name}: ${c.error}`);
assert.strictEqual(c.required, name === 'db', name);
assert.strictEqual(c.required, name === 'db', name);
assert.strictEqual(typeof c.latency_ms, 'number');
assert.strictEqual(typeof c.latency_ms, 'number');
assert.ok(Date.parse(c.checked_at));
assert.ok(Date.parse(c.checked_at));
}
}
+ assert.strictEqual(r.json.checks.db.detail.store, t.store.store, 'the store that answered, from a real round trip');
+ assert.ok(r.json.checks.db.detail.pool && typeof r.json.checks.db.detail.pool.total === 'number');
assert.ok(r.json.events_outbox && r.json.pending_deliveries);
assert.ok(r.json.events_outbox && r.json.pending_deliveries);
assert.strictEqual(r.headers.get('cache-control'), 'no-store');
assert.strictEqual(r.headers.get('cache-control'), 'no-store');
});
});
@@ -80,7 +85,7 @@ function get(base, p, headers = {}) {
@@ -80,7 +85,7 @@ function get(base, p, headers = {}) {
await check('a broken database makes the service unready (503); /metrics still answers', async () => {
await check('a broken database makes the service unready (503); /metrics still answers', async () => {
const d = await boot();
const d = await boot();
- d.db.close();
+ await d.db.close();
const r = await d.call('GET', '/api/ready', { token: null });
const r = await d.call('GET', '/api/ready', { token: null });
assert.strictEqual(r.status, 503, r.text);
assert.strictEqual(r.status, 503, r.text);
assert.strictEqual(r.json.ready, false);
assert.strictEqual(r.json.ready, false);
@@ -21,7 +21,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -21,7 +21,7 @@ const { boot, check, done } = require('./helpers/app');
tokenId = r.json.token.id;
tokenId = r.json.token.id;
assert.match(secret, /^tovl_[A-Za-z0-9_-]{43}$/);
assert.match(secret, /^tovl_[A-Za-z0-9_-]{43}$/);
assert.strictEqual(r.json.overlay_url, `http://tips.test/overlay/${secret}`);
assert.strictEqual(r.json.overlay_url, `http://tips.test/overlay/${secret}`);
- const row = t.db.prepare('SELECT * FROM overlay_tokens WHERE id = ?').get(tokenId);
+ const row = await t.db.maybe('SELECT * FROM overlay_tokens WHERE id = $1', [tokenId]);
assert.notStrictEqual(row.token_hash, secret);
assert.notStrictEqual(row.token_hash, secret);
assert.ok(!JSON.stringify(row).includes(secret), 'the secret is not stored');
assert.ok(!JSON.stringify(row).includes(secret), 'the secret is not stored');
const list = await t.call('GET', '/api/v1/overlay-tokens', { user: alex });
const list = await t.call('GET', '/api/v1/overlay-tokens', { user: alex });
@@ -48,16 +48,17 @@ const { boot, check, done } = require('./helpers/app');
@@ -48,16 +48,17 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(g.data.goal.id, goal.id);
assert.strictEqual(g.data.goal.id, goal.id);
firstAlertSeq = alert.id;
firstAlertSeq = alert.id;
s.close();
s.close();
- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r.json.interaction.id);
+ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r.json.interaction.id]);
assert.strictEqual(d.status, 'delivered');
assert.strictEqual(d.status, 'delivered');
- assert.strictEqual(t.outboxRows('tips.overlay.delivered').filter((e) => e.payload.delivery_id === d.id).length, 1);
+ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).filter((e) => e.payload.delivery_id === d.id).length, 1);
});
});
await check('overlay replay (Last-Event-ID) resends but never charges, re-counts or re-emits', async () => {
await check('overlay replay (Last-Event-ID) resends but never charges, re-counts or re-emits', async () => {
+ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
const before = {
const before = {
- transfers: billing.transfers().length, contributions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n,
- goal: domain.goals.present(domain.goals.get(goal.id)).current_amount, delivered: t.outboxRows('tips.overlay.delivered').length,
- interactions: t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, payable: billing.payable.get(alex.subject),
+ transfers: billing.transfers().length, contributions: await t.db.value('SELECT count(*) FROM tip_goal_contributions'),
+ goal: await goalNow(), delivered: (await t.outboxRows('tips.overlay.delivered')).length,
+ interactions: await t.db.value('SELECT count(*) FROM tip_interactions'), payable: billing.payable.get(alex.subject),
};
};
for (let k = 0; k < 3; k++) {
for (let k = 0; k < 3; k++) {
const s = await t.sse(`/overlay/${secret}/events`, { lastEventId: firstAlertSeq - 1 });
const s = await t.sse(`/overlay/${secret}/events`, { lastEventId: firstAlertSeq - 1 });
@@ -66,12 +67,12 @@ const { boot, check, done } = require('./helpers/app');
@@ -66,12 +67,12 @@ const { boot, check, done } = require('./helpers/app');
s.close();
s.close();
}
}
assert.strictEqual(billing.transfers().length, before.transfers);
assert.strictEqual(billing.transfers().length, before.transfers);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions').get().n, before.contributions);
- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal);
- assert.strictEqual(t.outboxRows('tips.overlay.delivered').length, before.delivered);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, before.interactions);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions'), before.contributions);
+ assert.strictEqual(await goalNow(), before.goal);
+ assert.strictEqual((await t.outboxRows('tips.overlay.delivered')).length, before.delivered);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), before.interactions);
assert.strictEqual(billing.payable.get(alex.subject), before.payable);
assert.strictEqual(billing.payable.get(alex.subject), before.payable);
- assert.ok(t.db.prepare('SELECT sends FROM overlay_deliveries WHERE seq = ?').get(firstAlertSeq).sends >= 4);
+ assert.ok(await t.db.value('SELECT sends FROM overlay_deliveries WHERE seq = $1', [firstAlertSeq]) >= 4);
});
});
await check('a tip while no overlay is connected waits, is delivered on connect, else fails after the window', async () => {
await check('a tip while no overlay is connected waits, is delivered on connect, else fails after the window', async () => {
@@ -83,20 +84,21 @@ const { boot, check, done } = require('./helpers/app');
@@ -83,20 +84,21 @@ const { boot, check, done } = require('./helpers/app');
await new Promise((ok) => setTimeout(ok, 50));
await new Promise((ok) => setTimeout(ok, 50));
const r2 = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 12 } });
const r2 = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 12 } });
t.clock.offset += 11 * 60 * 1000;
t.clock.offset += 11 * 60 * 1000;
- const n = domain.overlays.sweepFailed();
+ const n = await domain.overlays.sweepFailed();
t.clock.offset = 0;
t.clock.offset = 0;
assert.ok(n >= 1);
assert.ok(n >= 1);
- const d = t.db.prepare("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = ?").get(r2.json.interaction.id);
+ const d = await t.db.maybe("SELECT * FROM overlay_deliveries WHERE kind = 'alert' AND interaction_id = $1", [r2.json.interaction.id]);
assert.strictEqual(d.status, 'failed');
assert.strictEqual(d.status, 'failed');
- assert.strictEqual(t.outboxRows('tips.overlay.failed').filter((e) => e.payload.delivery_id === d.id).length, 1);
+ assert.strictEqual((await t.outboxRows('tips.overlay.failed')).filter((e) => e.payload.delivery_id === d.id).length, 1);
// The payment is untouched by the overlay failure.
// The payment is untouched by the overlay failure.
- assert.strictEqual(domain.interactions.get(r2.json.interaction.id).payment_state, 'settled');
+ assert.strictEqual((await domain.interactions.get(t.db, r2.json.interaction.id)).payment_state, 'settled');
});
});
await check('simulation runs the full path with test = 1, no Billing call, never counted', async () => {
await check('simulation runs the full path with test = 1, no Billing call, never counted', async () => {
+ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
const before = {
const before = {
- transfers: billing.calls.length, goal: domain.goals.present(domain.goals.get(goal.id)).current_amount,
- totals: domain.interactions.totals(alex.subject), events: t.outboxRows().length,
+ transfers: billing.calls.length, goal: await goalNow(),
+ totals: await domain.interactions.totals(t.db, alex.subject), events: (await t.outboxRows()).length,
};
};
const s = await t.sse(`/overlay/${secret}/events`);
const s = await t.sse(`/overlay/${secret}/events`);
await s.waitFor((e) => e.event === 'hello');
await s.waitFor((e) => e.event === 'hello');
@@ -114,13 +116,13 @@ const { boot, check, done } = require('./helpers/app');
@@ -114,13 +116,13 @@ const { boot, check, done } = require('./helpers/app');
const job = t.adapters.test.jobs.find((j) => j.interaction.id === i.id);
const job = t.adapters.test.jobs.find((j) => j.interaction.id === i.id);
assert.ok(job && job.test === true, 'chat effect delivered to the test adapter, flagged test');
assert.ok(job && job.test === true, 'chat effect delivered to the test adapter, flagged test');
assert.strictEqual(billing.calls.length, before.transfers, 'no Billing call');
assert.strictEqual(billing.calls.length, before.transfers, 'no Billing call');
- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, before.goal, 'goal unchanged');
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_goal_contributions WHERE interaction_id = ?').get(i.id).n, 0);
- const after = domain.interactions.totals(alex.subject);
+ assert.strictEqual(await goalNow(), before.goal, 'goal unchanged');
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_goal_contributions WHERE interaction_id = $1', [i.id]), 0);
+ const after = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(after.settled_via_billing, before.totals.settled_via_billing);
assert.strictEqual(after.settled_via_billing, before.totals.settled_via_billing);
assert.strictEqual(after.interactions, before.totals.interactions);
assert.strictEqual(after.interactions, before.totals.interactions);
assert.strictEqual(after.simulations, before.totals.simulations + 1);
assert.strictEqual(after.simulations, before.totals.simulations + 1);
- assert.strictEqual(t.outboxRows().length, before.events, 'no durable events for a simulation');
+ assert.strictEqual((await t.outboxRows()).length, before.events, 'no durable events for a simulation');
const denied = await t.call('POST', '/api/v1/simulate', { user: viewer, body: { creator: 'alex' } });
const denied = await t.call('POST', '/api/v1/simulate', { user: viewer, body: { creator: 'alex' } });
assert.strictEqual(denied.status, 403);
assert.strictEqual(denied.status, 403);
});
});
@@ -82,7 +82,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -82,7 +82,7 @@ const { boot, check, done } = require('./helpers/app');
const loc = r.headers.get('location');
const loc = r.headers.get('location');
assert.match(loc, /^\/receipts\/tint_/);
assert.match(loc, /^\/receipts\/tint_/);
const id = loc.split('/').pop();
const id = loc.split('/').pop();
- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
+ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
// Double submit (same form nonce): the same interaction, no second transfer.
// Double submit (same form nonce): the same interaction, no second transfer.
const before = billing.transfers().length;
const before = billing.transfers().length;
const again = await post('/alex/tip', viewer, form);
const again = await post('/alex/tip', viewer, form);
@@ -106,7 +106,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -106,7 +106,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r2.status, 403, 'a token minted for someone else does not work');
assert.strictEqual(r2.status, 403, 'a token minted for someone else does not work');
const d = await post('/dashboard/profile', alex, { csrf: 'nope', page_enabled: '0' });
const d = await post('/dashboard/profile', alex, { csrf: 'nope', page_enabled: '0' });
assert.strictEqual(d.status, 403);
assert.strictEqual(d.status, 403);
- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, true);
+ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, true);
});
});
await check('the form reports Billing\'s refusal in words; checkout redirects to the provider', async () => {
await check('the form reports Billing\'s refusal in words; checkout redirects to the provider', async () => {
@@ -138,10 +138,10 @@ const { boot, check, done } = require('./helpers/app');
@@ -138,10 +138,10 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(g.status, 303);
assert.strictEqual(g.status, 303);
const sim = await post('/dashboard/simulate', alex, { csrf, idem: 'x', kind: 'tip', amount: '100', supporter_name: 'Test supporter', message: 'test' });
const sim = await post('/dashboard/simulate', alex, { csrf, idem: 'x', kind: 'tip', amount: '100', supporter_name: 'Test supporter', message: 'test' });
assert.strictEqual(sim.status, 303);
assert.strictEqual(sim.status, 303);
- assert.strictEqual(t.db.prepare("SELECT COUNT(*) AS n FROM tip_interactions WHERE creator_subject = ? AND test = 1").get(alex.subject).n, 1);
- const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String(domain.profiles.byHandle('alex').revision) });
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE creator_subject = $1 AND test', [alex.subject]), 1);
+ const off = await post('/dashboard/profile', alex, { csrf, idem: 'x', accepting: '1', revision: String((await domain.profiles.byHandle(t.db, 'alex')).revision) });
assert.strictEqual(off.status, 303);
assert.strictEqual(off.status, 303);
- assert.strictEqual(domain.profiles.byHandle('alex').page_enabled, false, 'an unchecked box switches the page off');
+ assert.strictEqual((await domain.profiles.byHandle(t.db, 'alex')).page_enabled, false, 'an unchecked box switches the page off');
assert.strictEqual((await get('/alex')).status, 404);
assert.strictEqual((await get('/alex')).status, 404);
});
});
@@ -28,7 +28,7 @@ function payloadValidator(type) {
@@ -28,7 +28,7 @@ function payloadValidator(type) {
const tip = (user, body) => t.call('POST', '/api/v1/checkout', { user, body: { creator: 'alex', ...body } });
const tip = (user, body) => t.call('POST', '/api/v1/checkout', { user, body: { creator: 'alex', ...body } });
const alertOf = async (id) => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts.find((a) => a.interaction_id === id);
const alertOf = async (id) => (await t.call('GET', `/overlay/${secret}/state`, { token: null })).json.alerts.find((a) => a.interaction_id === id);
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
const jobsOf = (id) => t.adapters.test.jobs.filter((j) => j.interaction.id === id);
- const eventsOf = (type, id) => t.outboxRows(type).filter((e) => e.subject.id === id);
+ const eventsOf = async (type, id) => (await t.outboxRows(type)).filter((e) => e.subject.id === id);
let anonId;
let anonId;
await check('anonymous: "Anonymous" to the creator, overlays, chat and events; the supporter keeps their receipt', async () => {
await check('anonymous: "Anonymous" to the creator, overlays, chat and events; the supporter keeps their receipt', async () => {
@@ -46,19 +46,19 @@ function payloadValidator(type) {
@@ -46,19 +46,19 @@ function payloadValidator(type) {
assert.strictEqual(a.data.supporter_name, 'Anonymous');
assert.strictEqual(a.data.supporter_name, 'Anonymous');
assert.strictEqual(a.data.message, 'from nobody');
assert.strictEqual(a.data.message, 'from nobody');
assert.ok(!JSON.stringify(a.data).includes('Viewer Real Name') && !JSON.stringify(a.data).includes(viewer.subject));
assert.ok(!JSON.stringify(a.data).includes('Viewer Real Name') && !JSON.stringify(a.data).includes(viewer.subject));
- const g = t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(anonId);
- assert.strictEqual(JSON.parse(g.payload).by, 'Anonymous');
+ const g = await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [anonId]);
+ assert.strictEqual(g.payload.by, 'Anonymous');
await domain.effects.drain();
await domain.effects.drain();
const [job] = jobsOf(anonId);
const [job] = jobsOf(anonId);
assert.deepStrictEqual(job.supporter, { name: 'Anonymous', subject: null });
assert.deepStrictEqual(job.supporter, { name: 'Anonymous', subject: null });
assert.strictEqual(job.text, 'Anonymous tipped 120 Vibes: from nobody');
assert.strictEqual(job.text, 'Anonymous tipped 120 Vibes: from nobody');
- const [ready] = eventsOf('tips.interaction.ready', anonId);
+ const [ready] = await eventsOf('tips.interaction.ready', anonId);
assert.strictEqual(ready.payload.supporter, null);
assert.strictEqual(ready.payload.supporter, null);
assert.strictEqual(ready.payload.supporter_name, 'Anonymous');
assert.strictEqual(ready.payload.supporter_name, 'Anonymous');
assert.ok(contracts.validate('tips.interaction.ready', ready.payload).valid);
assert.ok(contracts.validate('tips.interaction.ready', ready.payload).valid);
- assert.ok(!JSON.stringify(t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');
+ assert.ok(!JSON.stringify(await t.outboxRows()).includes('Viewer Real Name'), 'no event anywhere carries the name');
for (const as of [{ user: alex }, { cap: ['tips.interaction.get'] }]) {
for (const as of [{ user: alex }, { cap: ['tips.interaction.get'] }]) {
const v = (await t.call('GET', `/api/v1/interactions/${anonId}`, as)).json.interaction;
const v = (await t.call('GET', `/api/v1/interactions/${anonId}`, as)).json.interaction;
@@ -80,7 +80,7 @@ function payloadValidator(type) {
@@ -80,7 +80,7 @@ function payloadValidator(type) {
assert.strictEqual(a.amount, null);
assert.strictEqual(a.amount, null);
assert.strictEqual(a.amount_hidden, true);
assert.strictEqual(a.amount_hidden, true);
assert.strictEqual(a.supporter_name, 'Viewer');
assert.strictEqual(a.supporter_name, 'Viewer');
- assert.strictEqual(JSON.parse(t.db.prepare("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = ?").get(hiddenId).payload).by, null,
+ assert.strictEqual((await t.db.maybe("SELECT payload FROM overlay_deliveries WHERE kind = 'goal' AND interaction_id = $1", [hiddenId])).payload.by, null,
'the goal update does not pin the jump on them');
'the goal update does not pin the jump on them');
await domain.effects.drain();
await domain.effects.drain();
const [job] = jobsOf(hiddenId);
const [job] = jobsOf(hiddenId);
@@ -88,7 +88,7 @@ function payloadValidator(type) {
@@ -88,7 +88,7 @@ function payloadValidator(type) {
assert.strictEqual(job.interaction.amount, null);
assert.strictEqual(job.interaction.amount, null);
assert.strictEqual(job.privacy.hide_amount, true);
assert.strictEqual(job.privacy.hide_amount, true);
assert.strictEqual((await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: alex })).json.interaction.amount, 222);
assert.strictEqual((await t.call('GET', `/api/v1/interactions/${hiddenId}`, { user: alex })).json.interaction.amount, 222);
- assert.strictEqual(eventsOf('tips.interaction.ready', hiddenId)[0].payload.amount, 222);
+ assert.strictEqual((await eventsOf('tips.interaction.ready', hiddenId))[0].payload.amount, 222);
});
});
await check('an overlay\'s minimum still applies to a hidden amount (without revealing it)', async () => {
await check('an overlay\'s minimum still applies to a hidden amount (without revealing it)', async () => {
@@ -183,10 +183,10 @@ function payloadValidator(type) {
@@ -183,10 +183,10 @@ function payloadValidator(type) {
body: new URLSearchParams({ csrf, idem: 'form-privacy-00001', kind: 'tip', amount: '15', message: 'shh', pay_with: 'credit', supporter_name: 'Other', anonymous: '1', private_message: '1' }).toString(),
body: new URLSearchParams({ csrf, idem: 'form-privacy-00001', kind: 'tip', amount: '15', message: 'shh', pay_with: 'credit', supporter_name: 'Other', anonymous: '1', private_message: '1' }).toString(),
});
});
assert.strictEqual(res.status, 303);
assert.strictEqual(res.status, 303);
- const i = domain.interactions.get(res.headers.get('location').split('/').pop());
- assert.strictEqual(i.anonymous, 1);
- assert.strictEqual(i.private_message, 1);
- assert.strictEqual(i.hide_amount, 0);
+ const i = await domain.interactions.get(t.db, res.headers.get('location').split('/').pop());
+ assert.strictEqual(i.anonymous, true);
+ assert.strictEqual(i.private_message, true);
+ assert.strictEqual(i.hide_amount, false);
});
});
await check('export: a supporter downloads their own tips, with messages and choices, nobody else\'s', async () => {
await check('export: a supporter downloads their own tips, with messages and choices, nobody else\'s', async () => {
@@ -207,14 +207,15 @@ function payloadValidator(type) {
@@ -207,14 +207,15 @@ function payloadValidator(type) {
});
});
await check('erasure: the person goes, the money record stays; tips.interaction.erased redacts the earlier events', async () => {
await check('erasure: the person goes, the money record stays; tips.interaction.erased redacts the earlier events', async () => {
- const totalsBefore = domain.interactions.totals(alex.subject);
- const goalBefore = domain.goals.present(domain.goals.get(goal.id)).current_amount;
+ const totalsBefore = await domain.interactions.totals(t.db, alex.subject);
+ const goalNow = async () => (await domain.goals.present(t.db, await domain.goals.get(t.db, goal.id))).current_amount;
+ const goalBefore = await goalNow();
const buyer = t.network.newUser('buyer');
const buyer = t.network.newUser('buyer');
billing.fund(buyer.subject, 1000);
billing.fund(buyer.subject, 1000);
const kept = await tip(buyer, { amount: 150, pay_with: 'checkout', provider: 'stripe', message: 'pending one' });
const kept = await tip(buyer, { amount: 150, pay_with: 'checkout', provider: 'stripe', message: 'pending one' });
const done1 = await tip(buyer, { amount: 40, message: 'my secret words', supporter_name: 'Buyer Person' });
const done1 = await tip(buyer, { amount: 40, message: 'my secret words', supporter_name: 'Buyer Person' });
- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM api_idempotency WHERE key LIKE ?').get(`${buyer.subject}:%`).n >= 2);
- const unsent = eventsOf('tips.interaction.ready', done1.json.interaction.id)[0];
+ assert.ok(await t.db.value('SELECT count(*) FROM api_idempotency WHERE key LIKE $1', [`${buyer.subject}:%`]) >= 2);
+ const unsent = (await eventsOf('tips.interaction.ready', done1.json.interaction.id))[0];
assert.strictEqual(unsent.payload.supporter.id, buyer.subject);
assert.strictEqual(unsent.payload.supporter.id, buyer.subject);
const r = await t.call('POST', '/api/v1/me/erase', { user: buyer });
const r = await t.call('POST', '/api/v1/me/erase', { user: buyer });
@@ -223,36 +224,36 @@ function payloadValidator(type) {
@@ -223,36 +224,36 @@ function payloadValidator(type) {
assert.strictEqual(r.json.kept_pending, 1);
assert.strictEqual(r.json.kept_pending, 1);
assert.ok(r.json.stored_answers_removed >= 2);
assert.ok(r.json.stored_answers_removed >= 2);
- const i = domain.interactions.get(done1.json.interaction.id);
+ const i = await domain.interactions.get(t.db, done1.json.interaction.id);
assert.strictEqual(i.supporter_subject, null);
assert.strictEqual(i.supporter_subject, null);
assert.strictEqual(i.supporter_name, null);
assert.strictEqual(i.supporter_name, null);
assert.strictEqual(i.message, null);
assert.strictEqual(i.message, null);
assert.ok(i.erased_at);
assert.ok(i.erased_at);
assert.strictEqual(i.amount, 40);
assert.strictEqual(i.amount, 40);
assert.ok(i.billing_txn_id, 'the Billing reference stays');
assert.ok(i.billing_txn_id, 'the Billing reference stays');
- assert.strictEqual(domain.interactions.get(kept.json.interaction.id).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');
+ assert.strictEqual((await domain.interactions.get(t.db, kept.json.interaction.id)).supporter_subject, buyer.subject, 'a pending payment keeps its supporter');
const a = await alertOf(i.id);
const a = await alertOf(i.id);
assert.strictEqual(a.supporter_name, 'Anonymous');
assert.strictEqual(a.supporter_name, 'Anonymous');
assert.strictEqual(a.message, null);
assert.strictEqual(a.message, null);
- const row = t.db.prepare('SELECT * FROM tip_interactions WHERE id = ?').get(i.id);
+ const row = await t.db.maybe('SELECT * FROM tip_interactions WHERE id = $1', [i.id]);
assert.ok(!JSON.stringify(row).includes('my secret words') && !JSON.stringify(row).includes('Buyer Person') && !JSON.stringify(row).includes(buyer.subject));
assert.ok(!JSON.stringify(row).includes('my secret words') && !JSON.stringify(row).includes('Buyer Person') && !JSON.stringify(row).includes(buyer.subject));
- const everything = JSON.stringify(t.db.prepare('SELECT * FROM paid_messages').all()) + JSON.stringify(t.db.prepare('SELECT payload FROM overlay_deliveries').all())
- + JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all());
+ const everything = JSON.stringify(await t.db.many('SELECT * FROM paid_messages')) + JSON.stringify(await t.db.many('SELECT payload FROM overlay_deliveries'))
+ + JSON.stringify(await t.db.many('SELECT * FROM api_idempotency'));
assert.ok(!everything.includes('my secret words') && !everything.includes('Buyer Person'));
assert.ok(!everything.includes('my secret words') && !everything.includes('Buyer Person'));
- const scrubbed = eventsOf('tips.interaction.ready', i.id)[0];
+ const scrubbed = (await eventsOf('tips.interaction.ready', i.id))[0];
assert.strictEqual(scrubbed.payload.supporter, null, 'the local outbox copy no longer names them');
assert.strictEqual(scrubbed.payload.supporter, null, 'the local outbox copy no longer names them');
assert.strictEqual(scrubbed.payload.supporter_name, 'Anonymous');
assert.strictEqual(scrubbed.payload.supporter_name, 'Anonymous');
- const [erased] = eventsOf('tips.interaction.erased', i.id);
+ const [erased] = await eventsOf('tips.interaction.erased', i.id);
assert.deepStrictEqual(erased.payload.redacts, { subject_type: 'interaction', subject_ids: [i.id] });
assert.deepStrictEqual(erased.payload.redacts, { subject_type: 'interaction', subject_ids: [i.id] });
const v = payloadValidator('tips.interaction.erased')(erased.payload);
const v = payloadValidator('tips.interaction.erased')(erased.payload);
assert.ok(v.valid, JSON.stringify(v.errors));
assert.ok(v.valid, JSON.stringify(v.errors));
assert.ok(contracts.validate('events.event-envelope@1', erased).valid);
assert.ok(contracts.validate('events.event-envelope@1', erased).valid);
- const totalsAfter = domain.interactions.totals(alex.subject);
+ const totalsAfter = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totalsAfter.settled_via_billing, totalsBefore.settled_via_billing + 40, 'the erased tip still counts');
assert.strictEqual(totalsAfter.settled_via_billing, totalsBefore.settled_via_billing + 40, 'the erased tip still counts');
assert.strictEqual(totalsAfter.settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
assert.strictEqual(totalsAfter.settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
- assert.strictEqual(domain.goals.present(domain.goals.get(goal.id)).current_amount, goalBefore + 40, 'the goal keeps the money');
+ assert.strictEqual(await goalNow(), goalBefore + 40, 'the goal keeps the money');
const receipts = (await t.call('GET', '/api/v1/interactions', { user: buyer })).json.interactions;
const receipts = (await t.call('GET', '/api/v1/interactions', { user: buyer })).json.interactions;
assert.deepStrictEqual(receipts.map((x) => x.id), [kept.json.interaction.id]);
assert.deepStrictEqual(receipts.map((x) => x.id), [kept.json.interaction.id]);
});
});
@@ -265,17 +266,17 @@ function payloadValidator(type) {
@@ -265,17 +266,17 @@ function payloadValidator(type) {
const post = (form) => fetch(`${t.base}/receipts/erase`, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie }, body: new URLSearchParams(form).toString() });
const post = (form) => fetch(`${t.base}/receipts/erase`, { method: 'POST', redirect: 'manual', headers: { 'Content-Type': 'application/x-www-form-urlencoded', Cookie: cookie }, body: new URLSearchParams(form).toString() });
assert.strictEqual((await post({ csrf, idem: 'form-erase-000001' })).status, 403, 'no confirmation');
assert.strictEqual((await post({ csrf, idem: 'form-erase-000001' })).status, 403, 'no confirmation');
assert.strictEqual((await post({ csrf: 'forged', idem: 'form-erase-000002', confirm: '1' })).status, 403);
assert.strictEqual((await post({ csrf: 'forged', idem: 'form-erase-000002', confirm: '1' })).status, 403);
- assert.ok(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n > 0);
+ assert.ok(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]) > 0);
const ok = await post({ csrf, idem: 'form-erase-000003', confirm: '1' });
const ok = await post({ csrf, idem: 'form-erase-000003', confirm: '1' });
assert.strictEqual(ok.status, 303);
assert.strictEqual(ok.status, 303);
assert.match(ok.headers.get('location'), /^\/receipts\?done=/);
assert.match(ok.headers.get('location'), /^\/receipts\?done=/);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(other.subject).n, 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [other.subject]), 0);
const lb = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.leaderboard;
const lb = (await t.call('GET', '/api/v1/profiles/alex/supporters', { token: null })).json.leaderboard;
assert.ok(!lb.some((x) => x.name === 'Other'), 'an erased supporter leaves the leaderboard');
assert.ok(!lb.some((x) => x.name === 'Other'), 'an erased supporter leaves the leaderboard');
});
});
await check('every event is a valid envelope with a valid payload', async () => {
await check('every event is a valid envelope with a valid payload', async () => {
- for (const e of t.outboxRows()) {
+ for (const e of await t.outboxRows()) {
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
assert.ok(contracts.validate('events.event-envelope@1', e).valid, e.event_type);
const v = payloadValidator(e.event_type)(e.payload);
const v = payloadValidator(e.event_type)(e.payload);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)}`);
assert.ok(v.valid, `${e.event_type}: ${JSON.stringify(v.errors)}`);
new file mode 100644
new file mode 100644
@@ -0,0 +1,65 @@
@@ -0,0 +1,65 @@
+'use strict';
+// The hot list routes run a bounded number of queries, whatever the page size (no N+1): the receipts
+// and creator lists (GET /api/v1/interactions, the hottest), goals, the supporters page and an
+// overlay's /state. Counted on the database handle (db.stats().queries) around one request each.
+const assert = require('assert');
+const { boot, check, done } = require('./helpers/app');
+
+(async () => {
+ const t = await boot();
+ const { billing } = t;
+ const alex = await t.creator('alex', { settings: { page: { supporters_page: true, supporters_messages: true, goal_supporters: true } } });
+ const fans = ['ann', 'bob', 'cat', 'dan', 'eve'].map((n) => t.network.newUser(n));
+ for (const f of fans) billing.fund(f.subject, 10_000);
+ const tip = (f, body) => t.call('POST', '/api/v1/checkout', { user: f, body: { creator: 'alex', amount: 10, ...body } });
+ const goals = [];
+ for (const title of ['Desk', 'Chair', 'Lamp']) goals.push((await t.call('POST', '/api/v1/goals', { user: alex, body: { title, target_amount: 5000 } })).json.goal);
+ // Several tips per supporter, messages, goal contributions: every list has many rows.
+ for (let k = 0; k < 3; k++) for (const f of fans) assert.strictEqual((await tip(f, { message: `tip ${k} from ${f.username}`, goal_id: goals[k].id })).status, 201);
+ const token = (await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { scopes: ['alerts', 'goals'] } })).json.secret;
+
+ /** Queries one request runs. */
+ async function queries(method, p, opts) {
+ const before = t.db.stats().queries;
+ const r = await t.call(method, p, opts);
+ assert.strictEqual(r.status, 200, `${p}: ${r.text}`);
+ return { n: t.db.stats().queries - before, r };
+ }
+
+ await check('GET /api/v1/interactions (the creator\'s list, with effects): 3 queries for 1 row or 15', async () => {
+ const one = await queries('GET', '/api/v1/interactions?as=creator&limit=1', { user: alex });
+ const all = await queries('GET', '/api/v1/interactions?as=creator&limit=50', { user: alex });
+ assert.strictEqual(one.r.json.interactions.length, 1);
+ assert.strictEqual(all.r.json.interactions.length, 15);
+ assert.ok(all.r.json.interactions.every((i) => Array.isArray(i.effects) && i.effects.length >= 1));
+ assert.deepStrictEqual([one.n, all.n], [3, 3], 'the page, the creators\' filters, the effects');
+ });
+
+ await check('GET /api/v1/interactions (a supporter\'s receipts; a service\'s view of a creator): bounded too', async () => {
+ const mine = await queries('GET', '/api/v1/interactions', { user: fans[0] });
+ assert.strictEqual(mine.r.json.interactions.length, 3);
+ assert.strictEqual(mine.n, 2, 'the page and the filters (no effects for a supporter)');
+ const svc = await queries('GET', '/api/v1/interactions?creator=alex&limit=50', { cap: ['tips.interaction.list'] });
+ assert.strictEqual(svc.r.json.interactions.length, 15);
+ assert.strictEqual(svc.n, 4, 'the creator, the page, the filters, the effects');
+ });
+
+ await check('goals, the supporters page and an overlay\'s state: a fixed number of queries for any number of rows', async () => {
+ const pub = await queries('GET', '/api/v1/goals?creator=alex', { token: null });
+ assert.strictEqual(pub.r.json.goals.length, 3);
+ assert.ok(pub.r.json.goals.every((g) => g.supporters.length === 5), 'each goal lists its supporters');
+ assert.strictEqual(pub.n, 4, 'the profile, the goals, their totals, their supporters');
+ const sup = await queries('GET', '/api/v1/profiles/alex/supporters', { token: null });
+ assert.strictEqual(sup.r.json.leaderboard.length, 5);
+ assert.strictEqual(sup.r.json.recent.length, 15);
+ assert.strictEqual(sup.n, 3, 'the profile, the leaderboard, the recent messages');
+ const before = t.db.stats().queries;
+ const state = await (await fetch(`${t.base}/overlay/${token}/state`)).json();
+ assert.strictEqual(state.alerts.length, 15);
+ assert.strictEqual(state.goals.length, 3);
+ assert.strictEqual(t.db.stats().queries - before, 4, 'the token, the goals, their totals, the alerts');
+ });
+
+ await t.close();
+ done();
+})().catch((e) => { console.error(e); process.exit(1); });
@@ -30,13 +30,13 @@ const { boot, check, done } = require('./helpers/app');
@@ -30,13 +30,13 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(d.status, 200);
assert.strictEqual(d.status, 200);
assert.strictEqual(d.json.outcome, 'recorded', 'recorded on its own');
assert.strictEqual(d.json.outcome, 'recorded', 'recorded on its own');
}
}
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'pending', 'the paid message is still unpaid');
assert.strictEqual(i.payment_state, 'pending', 'the paid message is still unpaid');
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM paid_messages WHERE interaction_id = ?').get(id).n, 0);
- const spoof = domain.interactions.byBillingTxn(cases[0].payload.transaction_id);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM paid_messages WHERE interaction_id = $1', [id]), 0);
+ const spoof = await domain.interactions.byBillingTxn(t.db, cases[0].payload.transaction_id);
assert.strictEqual(spoof.amount, 1);
assert.strictEqual(spoof.amount, 1);
assert.strictEqual(spoof.origin, 'billing');
assert.strictEqual(spoof.origin, 'billing');
- assert.strictEqual(domain.interactions.totals(alex.subject).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
+ assert.strictEqual((await domain.interactions.totals(t.db, alex.subject)).settled_via_billing, billing.payable.get(alex.subject), 'still reconciles to Billing');
});
});
await check('overlay stream floods: a token opens a bounded number of streams at once', async () => {
await check('overlay stream floods: a token opens a bounded number of streams at once', async () => {
@@ -83,7 +83,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -83,7 +83,7 @@ const { boot, check, done } = require('./helpers/app');
tts: { text: '<speak><break time="9s"/>hello world</speak>' },
tts: { text: '<speak><break time="9s"/>hello world</speak>' },
} });
} });
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.status, 201, r.text);
- const i = domain.interactions.get(r.json.interaction.id);
+ const i = await domain.interactions.get(t.db, r.json.interaction.id);
assert.strictEqual(i.supporter_name, 'Viewer');
assert.strictEqual(i.supporter_name, 'Viewer');
assert.strictEqual(i.message, 'hi there');
assert.strictEqual(i.message, 'hi there');
await domain.effects.drain();
await domain.effects.drain();
@@ -92,6 +92,24 @@ const { boot, check, done } = require('./helpers/app');
@@ -92,6 +92,24 @@ const { boot, check, done } = require('./helpers/app');
assert.ok(!/[]/.test(JSON.stringify(job)));
assert.ok(!/[]/.test(JSON.stringify(job)));
});
});
+ await check('text PostgreSQL cannot store (NUL, an unpaired surrogate) is cleaned, from people and from Billing events alike', async () => {
+ const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 6, message: 'a\u0000b \ud83d', supporter_name: 'N\u0000ul' } });
+ assert.strictEqual(r.status, 201, r.text);
+ const i = await domain.interactions.get(t.db, r.json.interaction.id);
+ assert.deepStrictEqual([i.message, i.supporter_name], ['ab �', 'Nul']);
+ const tts = await t.call('POST', '/api/v1/tts-requests', { user: viewer, body: { creator: 'alex', amount: 150, tts: { text: 'say \udc00 this\u0000' } } });
+ assert.strictEqual(tts.status, 201, tts.text);
+ const cfg = await t.call('POST', '/api/v1/overlay-configs', { user: alex, body: { kind: 'alerts', template: '{name} \ud800 tipped' } });
+ assert.strictEqual(cfg.status, 201, cfg.text);
+ // A Billing event is redelivered until it is accepted: its text must never make it unstorable.
+ const ev = billing.foreignDonation({ from: other.subject, to: alex.subject, amount: 9, message: 'from\u0000 billing \ud83d' });
+ const d = await t.deliver(ev);
+ assert.deepStrictEqual([d.status, d.json.outcome], [200, 'recorded']);
+ assert.strictEqual((await domain.interactions.byBillingTxn(t.db, ev.payload.transaction_id)).message, 'from billing �');
+ const cursor = Buffer.from(JSON.stringify(['1', 'x'])).toString('base64url');
+ assert.strictEqual((await t.call('GET', `/api/v1/interactions?cursor=${cursor}`, { user: viewer })).status, 422, 'a cursor PostgreSQL could not read is refused, not a 500');
+ });
+
await check('an overlay secret is shown once: an Idempotency-Key replay does not return it, the database never holds it', async () => {
await check('an overlay secret is shown once: an Idempotency-Key replay does not return it, the database never holds it', async () => {
const first = await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { label: 'replay' }, key: 'overlay-key-0001' });
const first = await t.call('POST', '/api/v1/overlay-tokens', { user: alex, body: { label: 'replay' }, key: 'overlay-key-0001' });
assert.match(first.json.secret, /^tovl_/);
assert.match(first.json.secret, /^tovl_/);
@@ -99,7 +117,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -99,7 +117,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(replay.headers.get('idempotent-replayed'), 'true');
assert.strictEqual(replay.headers.get('idempotent-replayed'), 'true');
assert.strictEqual(replay.json.token.id, first.json.token.id);
assert.strictEqual(replay.json.token.id, first.json.token.id);
assert.strictEqual(replay.json.secret, null);
assert.strictEqual(replay.json.secret, null);
- const everything = JSON.stringify(t.db.prepare('SELECT * FROM api_idempotency').all()) + JSON.stringify(t.db.prepare('SELECT * FROM overlay_tokens').all());
+ const everything = JSON.stringify(await t.db.many('SELECT * FROM api_idempotency')) + JSON.stringify(await t.db.many('SELECT * FROM overlay_tokens'));
assert.ok(!everything.includes(first.json.secret));
assert.ok(!everything.includes(first.json.secret));
});
});
@@ -38,21 +38,21 @@ const { boot, check, done } = require('./helpers/app');
@@ -38,21 +38,21 @@ const { boot, check, done } = require('./helpers/app');
// Same transaction republished under a new event id (a replay): still one interaction.
// Same transaction republished under a new event id (a replay): still one interaction.
const c = await t.deliver({ ...ev, event_id: billing.envelope('x', {}, 'x').event_id });
const c = await t.deliver({ ...ev, event_id: billing.envelope('x', {}, 'x').event_id });
assert.strictEqual(c.json.duplicate, false);
assert.strictEqual(c.json.duplicate, false);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 1);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions').get().n, 1);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 1);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions'), 1);
});
});
await check('a duplicate provider webhook (Billing settles once) → one logical interaction for a donation Tips did not start', async () => {
await check('a duplicate provider webhook (Billing settles once) → one logical interaction for a donation Tips did not start', async () => {
// Billing turned one PowerChat site-routed tip into one transaction; Events may deliver it more than once.
// Billing turned one PowerChat site-routed tip into one transaction; Events may deliver it more than once.
const ev = billing.foreignDonation({ from: null, to: alex.subject, amount: 300, provider: 'powerchat', message: 'from PowerChat' });
const ev = billing.foreignDonation({ from: null, to: alex.subject, amount: 300, provider: 'powerchat', message: 'from PowerChat' });
for (let k = 0; k < 3; k++) await t.deliver(k === 2 ? { ...ev, event_id: billing.envelope('x', {}, 'x').event_id } : ev);
for (let k = 0; k < 3; k++) await t.deliver(k === 2 ? { ...ev, event_id: billing.envelope('x', {}, 'x').event_id } : ev);
- const rows = t.db.prepare('SELECT * FROM tip_interactions WHERE billing_txn_id = ?').all(ev.payload.transaction_id);
+ const rows = await t.db.many('SELECT * FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]);
assert.strictEqual(rows.length, 1);
assert.strictEqual(rows.length, 1);
assert.strictEqual(rows[0].origin, 'billing');
assert.strictEqual(rows[0].origin, 'billing');
assert.strictEqual(rows[0].funding, 'provider');
assert.strictEqual(rows[0].funding, 'provider');
assert.strictEqual(rows[0].payment_state, 'settled');
assert.strictEqual(rows[0].payment_state, 'settled');
// Tips did not start it, so Tips does not announce it in chat a second time — overlay only.
// Tips did not start it, so Tips does not announce it in chat a second time — overlay only.
- const effects = t.db.prepare('SELECT effect FROM interaction_effects WHERE interaction_id = ?').all(rows[0].id).map((e) => e.effect);
+ const effects = (await t.db.many('SELECT effect FROM interaction_effects WHERE interaction_id = $1', [rows[0].id])).map((e) => e.effect);
assert.deepStrictEqual(effects, ['overlay_alert']);
assert.deepStrictEqual(effects, ['overlay_alert']);
assert.strictEqual(rows[0].delivery_state, 'delivered');
assert.strictEqual(rows[0].delivery_state, 'delivered');
});
});
@@ -60,16 +60,16 @@ const { boot, check, done } = require('./helpers/app');
@@ -60,16 +60,16 @@ const { boot, check, done } = require('./helpers/app');
await check('delivery runs after settlement; a delivery failure never touches payment state', async () => {
await check('delivery runs after settlement; a delivery failure never touches payment state', async () => {
const test = t.adapters.test;
const test = t.adapters.test;
await domain.effects.drain(); // the first tip delivers normally
await domain.effects.drain(); // the first tip delivers normally
- assert.strictEqual(domain.interactions.get(tipId).delivery_state, 'delivered');
+ assert.strictEqual((await domain.interactions.get(t.db, tipId)).delivery_state, 'delivered');
test.state.failWith = 'chat is down';
test.state.failWith = 'chat is down';
test.state.permanent = true;
test.state.permanent = true;
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 40 } });
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 40 } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await domain.effects.drain();
await domain.effects.drain();
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.delivery_state, 'failed');
assert.strictEqual(i.delivery_state, 'failed');
- assert.strictEqual(t.outboxRows('tips.interaction.failed').filter((e) => e.subject.id === id).length, 1);
+ assert.strictEqual((await t.outboxRows('tips.interaction.failed')).filter((e) => e.subject.id === id).length, 1);
test.state.failWith = null;
test.state.failWith = null;
assert.strictEqual(test.jobs.find((j) => j.interaction.id === tipId).text, 'Viewer tipped 250 Vibes: great stream');
assert.strictEqual(test.jobs.find((j) => j.interaction.id === tipId).text, 'Viewer tipped 250 Vibes: great stream');
});
});
@@ -80,13 +80,13 @@ const { boot, check, done } = require('./helpers/app');
@@ -80,13 +80,13 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 10 } });
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: alex.subject, amount: 10 } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await domain.effects.drain();
await domain.effects.drain();
- const e = t.db.prepare("SELECT * FROM interaction_effects WHERE interaction_id = ? AND effect = 'chat_line'").get(id);
+ const e = await t.db.maybe("SELECT * FROM interaction_effects WHERE interaction_id = $1 AND effect = 'chat_line'", [id]);
assert.strictEqual(e.state, 'queued');
assert.strictEqual(e.state, 'queued');
assert.strictEqual(e.attempts, 1);
assert.strictEqual(e.attempts, 1);
test.state.failWith = null;
test.state.failWith = null;
t.clock.offset += 60_000;
t.clock.offset += 60_000;
await domain.effects.drain();
await domain.effects.drain();
- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
+ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
t.clock.offset = 0;
t.clock.offset = 0;
});
});
@@ -96,21 +96,21 @@ const { boot, check, done } = require('./helpers/app');
@@ -96,21 +96,21 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 400 } });
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 400 } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await domain.effects.drain();
await domain.effects.drain();
- assert.strictEqual(domain.interactions.get(id).delivery_state, 'delivered');
- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 400);
+ assert.strictEqual((await domain.interactions.get(t.db, id)).delivery_state, 'delivered');
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 400);
const rev = billing.refund(r.json.interaction.payment.billing_txn_id);
const rev = billing.refund(r.json.interaction.payment.billing_txn_id);
const d = await t.deliver(rev);
const d = await t.deliver(rev);
assert.strictEqual(d.json.outcome, 'reversed');
assert.strictEqual(d.json.outcome, 'reversed');
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.delivery_state, 'delivered');
assert.strictEqual(i.delivery_state, 'delivered');
assert.strictEqual(i.reversed_bits, 400);
assert.strictEqual(i.reversed_bits, 400);
- assert.ok(t.db.prepare("SELECT COUNT(*) AS n FROM interaction_effects WHERE interaction_id = ? AND state = 'delivered'").get(id).n >= 2);
- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0);
+ assert.ok(await t.db.value("SELECT count(*) FROM interaction_effects WHERE interaction_id = $1 AND state = 'delivered'", [id]) >= 2);
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0);
// The same reversal again changes nothing.
// The same reversal again changes nothing.
const again = await t.deliver({ ...rev, event_id: billing.envelope('x', {}, 'x').event_id });
const again = await t.deliver({ ...rev, event_id: billing.envelope('x', {}, 'x').event_id });
assert.strictEqual(again.json.outcome, 'duplicate_reversal');
assert.strictEqual(again.json.outcome, 'duplicate_reversal');
- assert.strictEqual(domain.interactions.get(id).reversed_bits, 400);
+ assert.strictEqual((await domain.interactions.get(t.db, id)).reversed_bits, 400);
await t.call('POST', `/api/v1/goals/${g.json.goal.id}/close`, { user: alex });
await t.call('POST', `/api/v1/goals/${g.json.goal.id}/close`, { user: alex });
});
});
@@ -118,12 +118,12 @@ const { boot, check, done } = require('./helpers/app');
@@ -118,12 +118,12 @@ const { boot, check, done } = require('./helpers/app');
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 20 } });
const r = await t.call('POST', '/api/v1/checkout', { user: viewer, body: { creator: 'alex', amount: 20 } });
const id = r.json.interaction.id;
const id = r.json.interaction.id;
await t.deliver(billing.refund(r.json.interaction.payment.billing_txn_id));
await t.deliver(billing.refund(r.json.interaction.payment.billing_txn_id));
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.payment_state, 'reversed');
assert.strictEqual(i.delivery_state, 'cancelled');
assert.strictEqual(i.delivery_state, 'cancelled');
await domain.effects.drain();
await domain.effects.drain();
assert.ok(!t.adapters.test.jobs.some((j) => j.interaction.id === id), 'nothing delivered after the reversal');
assert.ok(!t.adapters.test.jobs.some((j) => j.interaction.id === id), 'nothing delivered after the reversal');
- assert.strictEqual(t.outboxRows('tips.interaction.cancelled').filter((e) => e.subject.id === id).length, 1);
+ assert.strictEqual((await t.outboxRows('tips.interaction.cancelled')).filter((e) => e.subject.id === id).length, 1);
});
});
await check('insufficient credit: payment failed, nothing delivered, a replay of the key answers the same', async () => {
await check('insufficient credit: payment failed, nothing delivered, a replay of the key answers the same', async () => {
@@ -134,7 +134,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -134,7 +134,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r.json.details.interaction.payment.state, 'failed');
assert.strictEqual(r.json.details.interaction.payment.state, 'failed');
const again = await t.call('POST', '/api/v1/checkout', { user: poor, body: { creator: 'alex', amount: 50 }, key: 'poor-key-00001' });
const again = await t.call('POST', '/api/v1/checkout', { user: poor, body: { creator: 'alex', amount: 50 }, key: 'poor-key-00001' });
assert.strictEqual(again.status, 409);
assert.strictEqual(again.status, 409);
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE supporter_subject = ?').get(poor.subject).n, 1);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE supporter_subject = $1', [poor.subject]), 1);
});
});
await check('self-tipping is refused by subject', async () => {
await check('self-tipping is refused by subject', async () => {
@@ -151,18 +151,18 @@ const { boot, check, done } = require('./helpers/app');
@@ -151,18 +151,18 @@ const { boot, check, done } = require('./helpers/app');
assert.match(r.json.checkout_url, /^https:\/\/checkout\.stripe\.test\/pi_/);
assert.match(r.json.checkout_url, /^https:\/\/checkout\.stripe\.test\/pi_/);
const id = r.json.interaction.id;
const id = r.json.interaction.id;
assert.strictEqual(r.json.interaction.payment.state, 'pending');
assert.strictEqual(r.json.interaction.payment.state, 'pending');
- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 0, 'pending never counts');
- const intentId = domain.interactions.get(id).billing_intent_id;
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 0, 'pending never counts');
+ const intentId = (await domain.interactions.get(t.db, id)).billing_intent_id;
assert.strictEqual(billing.calls.find((c) => c.url === '/api/v1/intents' && c.body.subject.id === buyer.subject).key, `tips:intent:${id}`);
assert.strictEqual(billing.calls.find((c) => c.url === '/api/v1/intents' && c.body.subject.id === buyer.subject).key, `tips:intent:${id}`);
const purchase = billing.settlePurchase(intentId);
const purchase = billing.settlePurchase(intentId);
const d = await t.deliver(purchase);
const d = await t.deliver(purchase);
assert.strictEqual(d.json.outcome, 'funded');
assert.strictEqual(d.json.outcome, 'funded');
await new Promise((ok) => setTimeout(ok, 100));
await new Promise((ok) => setTimeout(ok, 100));
await domain.interactions.processDueTransfers();
await domain.interactions.processDueTransfers();
- const i = domain.interactions.get(id);
+ const i = await domain.interactions.get(t.db, id);
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.payment_state, 'settled');
assert.strictEqual(i.funding_txn_id, purchase.payload.transaction_id);
assert.strictEqual(i.funding_txn_id, purchase.payload.transaction_id);
- assert.strictEqual(domain.goals.present(domain.goals.get(g.json.goal.id)).current_amount, 300);
+ assert.strictEqual((await domain.goals.present(t.db, await domain.goals.get(t.db, g.json.goal.id))).current_amount, 300);
// The purchase event again, and the transfer's own event: nothing new.
// The purchase event again, and the transfer's own event: nothing new.
assert.strictEqual((await t.deliver(purchase)).json.duplicate, true);
assert.strictEqual((await t.deliver(purchase)).json.duplicate, true);
const own = billing.events.find((e) => e.payload.metadata && e.payload.metadata.target && e.payload.metadata.target.id === id);
const own = billing.events.find((e) => e.payload.metadata && e.payload.metadata.target && e.payload.metadata.target.id === id);
@@ -188,7 +188,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -188,7 +188,7 @@ const { boot, check, done } = require('./helpers/app');
t.clock.offset += 5000;
t.clock.offset += 5000;
await domain.interactions.processDueTransfers();
await domain.interactions.processDueTransfers();
t.clock.offset = 0;
t.clock.offset = 0;
- assert.strictEqual(domain.interactions.get(id).payment_state, 'settled');
+ assert.strictEqual((await domain.interactions.get(t.db, id)).payment_state, 'settled');
});
});
await check('paid messages, TTS and media requests exist only after settlement', async () => {
await check('paid messages, TTS and media requests exist only after settlement', async () => {
@@ -205,17 +205,17 @@ const { boot, check, done } = require('./helpers/app');
@@ -205,17 +205,17 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.status, 201, r.text);
ids.push(r.json.interaction.id);
ids.push(r.json.interaction.id);
}
}
- const count = () => t.db.prepare('SELECT (SELECT COUNT(*) FROM paid_messages WHERE interaction_id IN (?, ?, ?)) + (SELECT COUNT(*) FROM paid_media_requests WHERE interaction_id IN (?, ?, ?)) AS n').get(...ids, ...ids).n;
- assert.strictEqual(count(), 0);
- for (const id of ids) await t.deliver(billing.settlePurchase(domain.interactions.get(id).billing_intent_id));
+ const count = () => t.db.value('SELECT (SELECT count(*) FROM paid_messages WHERE interaction_id = ANY($1)) + (SELECT count(*) FROM paid_media_requests WHERE interaction_id = ANY($1))', [ids]);
+ assert.strictEqual(await count(), 0);
+ for (const id of ids) await t.deliver(billing.settlePurchase((await domain.interactions.get(t.db, id)).billing_intent_id));
await new Promise((ok) => setTimeout(ok, 100));
await new Promise((ok) => setTimeout(ok, 100));
await domain.interactions.processDueTransfers();
await domain.interactions.processDueTransfers();
- assert.strictEqual(count(), 3);
- const pm = t.db.prepare('SELECT * FROM paid_messages WHERE interaction_id = ?').get(ids[0]);
+ assert.strictEqual(await count(), 3);
+ const pm = await t.db.maybe('SELECT * FROM paid_messages WHERE interaction_id = $1', [ids[0]]);
assert.strictEqual(pm.highlight_seconds, 60);
assert.strictEqual(pm.highlight_seconds, 60);
await domain.effects.drain();
await domain.effects.drain();
- assert.strictEqual(t.db.prepare('SELECT status FROM paid_messages WHERE interaction_id = ?').get(ids[1]).status, 'delivered');
- assert.strictEqual(t.db.prepare('SELECT status FROM paid_media_requests WHERE interaction_id = ?').get(ids[2]).status, 'accepted');
+ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_messages WHERE interaction_id = $1', [ids[1]])).status, 'delivered');
+ assert.strictEqual((await t.db.maybe('SELECT status FROM paid_media_requests WHERE interaction_id = $1', [ids[2]])).status, 'accepted');
const tts = t.adapters.test.jobs.find((j) => j.effect === 'tts' && j.interaction.id === ids[1]);
const tts = t.adapters.test.jobs.find((j) => j.effect === 'tts' && j.interaction.id === ids[1]);
assert.deepStrictEqual(tts.tts, { text: 'read me please', voice: 'amy' });
assert.deepStrictEqual(tts.tts, { text: 'read me please', voice: 'amy' });
});
});
@@ -230,7 +230,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -230,7 +230,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(a.json.interaction.id, b.json.interaction.id);
assert.strictEqual(a.json.interaction.id, b.json.interaction.id);
const denied = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.checkout.create'], body: { ...body, provider_ref: 'x2' } });
const denied = await t.call('POST', '/api/v1/interactions/external', { cap: ['tips.checkout.create'], body: { ...body, provider_ref: 'x2' } });
assert.strictEqual(denied.status, 403);
assert.strictEqual(denied.status, 403);
- const totals = domain.interactions.totals(alex.subject);
+ const totals = await domain.interactions.totals(t.db, alex.subject);
assert.strictEqual(totals.external, 500);
assert.strictEqual(totals.external, 500);
});
});
@@ -246,7 +246,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -246,7 +246,7 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(proxied.status, 404);
assert.strictEqual(proxied.status, 404);
const forged = await t.deliver({ ...ev, source: 'live' });
const forged = await t.deliver({ ...ev, source: 'live' });
assert.strictEqual(forged.json.outcome, 'ignored:source');
assert.strictEqual(forged.json.outcome, 'ignored:source');
- assert.strictEqual(t.db.prepare('SELECT COUNT(*) AS n FROM tip_interactions WHERE billing_txn_id = ?').get(ev.payload.transaction_id).n, 0);
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tip_interactions WHERE billing_txn_id = $1', [ev.payload.transaction_id]), 0);
});
});
await check('events: ready/goal events are relayed to OpenVibe.Events with the tips service token', async () => {
await check('events: ready/goal events are relayed to OpenVibe.Events with the tips service token', async () => {
new file mode 100644
new file mode 100644
@@ -0,0 +1,189 @@
@@ -0,0 +1,189 @@
+'use strict';
+// The one-time import (ADR-035; scripts/migrate-to-postgres.js): a SQLite file with the last SQLite
+// release's schema (test/fixtures/sqlite-v3-schema.sql) and rows of every table goes through
+// openvibe-sdk's importSqlite into the migrated schema: report.ok, every row counted and checksummed,
+// values converted by column type (0/1 → boolean, ISO text → timestamptz, JSON text → jsonb), identities
+// kept and continued, the SQLite file untouched; and the service then serves the imported data.
+const assert = require('assert');
+const crypto = require('crypto');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { execFileSync } = require('child_process');
+const Database = require('better-sqlite3');
+const { ids } = require('openvibe-contracts');
+const { importSqlite } = require('openvibe-sdk/db');
+const { testDb } = require('./helpers/db');
+const { boot, check, done } = require('./helpers/app');
+const { importOptions } = require('../scripts/migrate-to-postgres');
+
+const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tips-import-'));
+const quiet = { log() {} };
+const creator = ids.newId('user');
+const supporter = ids.newId('user');
+const T = (s) => `2026-09-${s}Z`; // an ISO time, as the SQLite release wrote them
+
+/** A SQLite file as the SQLite release left it: its schema and a few rows in every table. */
+function fixture(file) {
+ const s = new Database(file);
+ s.exec(fs.readFileSync(path.join(__dirname, 'fixtures', 'sqlite-v3-schema.sql'), 'utf8'));
+ const ins = (table, row) => s.prepare(`INSERT INTO ${table} (${Object.keys(row).join(', ')}) VALUES (${Object.keys(row).map(() => '?').join(', ')})`).run(...Object.values(row));
+ ins('settings', { id: 1, schema_version: 3, created_at: T('01T00:00:00.000') });
+ ins('creator_tip_profiles', {
+ creator_subject: creator, handle: 'alex', display_name: 'Alex', page_enabled: 1, accepting: 1, tts_enabled: 0, media_requests_enabled: 1,
+ revision: 4, created_at: T('01T10:00:00.000'), updated_at: T('02T11:30:00.250'),
+ page_settings: '{"goal_amounts":false,"supporters_page":true}', filter_words: '["badword","two words"]', filter_action: 'hold', filter_links: 0,
+ });
+ const tip = (id, fields) => ins('tip_interactions', {
+ id, creator_subject: creator, supporter_subject: supporter, supporter_name: 'Viewer', kind: 'tip', amount: 250, funding: 'credit', settlement: 'billing',
+ payment_state: 'settled', delivery_state: 'delivered', created_at: T('03T12:00:00.000'), updated_at: T('03T12:00:01.000'), settled_at: T('03T12:00:00.500'), ...fields,
+ });
+ tip('tint_01', { billing_txn_id: 'txn_1', request: '{"goal_id":"tgoal_01"}', target: '{"service":"live","type":"stream","id":"42"}', message: 'great stream' });
+ // Two things SQLite took and PostgreSQL does not: a NUL in text, an unpaired surrogate inside JSON.
+ tip('tint_02', { message: 'nul\u0000here', request: '{"legacy_message":"cut \\ud83d","n":1}', billing_txn_id: 'txn_2', payment_state: 'reversed', reversal_txn_ids: '["txn_9"]', reversed_bits: 100, reversed_at: T('04T09:00:00.000'), anonymous: 1, hide_amount: 1, test: 0 });
+ tip('tint_03', { kind: 'paid_message', amount: 500, message: 'hello chat', funding: 'checkout', payment_state: 'pending', delivery_state: 'awaiting_payment',
+ settled_at: null, billing_intent_id: 'pi_1', checkout_url: 'https://checkout.example/pi_1', idempotency_key: 'api:usr:k1', transfer_due: 1, transfer_attempts: 2, next_transfer_at: 1790600000123 });
+ tip('tint_04', { supporter_subject: null, supporter_name: null, erased_at: T('05T08:00:00.000'), moderation: 'hidden', moderated_by: 'filter', filtered: 1, test: 1, origin: 'import',
+ settlement: 'imported', legacy_source: 'live:transactions:7', provider: 'powerchat', provider_ref: 'live-chat:3', request: '{}' });
+ ins('tip_goals', { id: 'tgoal_01', creator_subject: creator, title: 'Desk', target_amount: 1000, status: 'active', opening_amount: 50, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000'), reached_at: null });
+ ins('tip_goal_contributions', { id: 7, goal_id: 'tgoal_01', interaction_id: 'tint_01', amount: 250, reversed_amount: 0, created_at: T('03T12:00:00.600'), updated_at: T('03T12:00:00.600') });
+ ins('tip_goal_contributions', { id: 12, goal_id: 'tgoal_01', interaction_id: 'tint_02', amount: 250, reversed_amount: 100, created_at: T('03T12:00:00.700'), updated_at: T('04T09:00:00.000') });
+ ins('paid_messages', { id: 'tpm_01', interaction_id: 'tint_01', creator_subject: creator, kind: 'paid_message', text: 'hello', highlight_seconds: 30, status: 'delivered', chat_ref: 'null', test: 0, created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
+ ins('paid_media_requests', { id: 'tmr_01', interaction_id: 'tint_02', creator_subject: creator, url: 'https://youtu.be/x', provider: 'youtube', status: 'accepted', queue_ref: '{"queue_id":5}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
+ ins('overlay_configs', { id: 'tovc_01', creator_subject: creator, kind: 'alerts', name: 'Alerts', settings: '{"min_amount":1,"sound_url":null}', revision: 2, created_at: T('02T00:00:00.000'), updated_at: T('02T00:00:00.000') });
+ ins('overlay_tokens', { id: 'tovt_01', creator_subject: creator, token_hash: 'a'.repeat(64), scopes: '["alerts","goals"]', config_id: 'tovc_01', created_by: creator, created_at: T('02T00:00:00.000'), revoked_at: T('06T00:00:00.000') });
+ ins('overlay_deliveries', { seq: 3, id: 'tovd_03', creator_subject: creator, kind: 'alert', interaction_id: 'tint_01', payload: '{"interaction_id":"tint_01","amount":250}', status: 'delivered', sends: 4, dedupe_key: 'alert:tint_01', created_at: T('03T12:00:01.000'), delivered_at: T('03T12:00:01.500'), hidden: 0 });
+ ins('overlay_deliveries', { seq: 9, id: 'tovd_09', creator_subject: creator, kind: 'goal', goal_id: 'tgoal_01', payload: '{"goal":{"id":"tgoal_01"},"by":null}', status: 'pending', dedupe_key: 'goal:tgoal_01:r2', created_at: T('03T12:00:01.000'), hidden: 1 });
+ ins('interaction_effects', { id: 4, interaction_id: 'tint_01', effect: 'chat_line', adapter: 'live-chat', state: 'delivered', attempts: 1, result: '{"ref":{"chat_message_id":1}}', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:02.000') });
+ ins('interaction_effects', { id: 5, interaction_id: 'tint_01', effect: 'overlay_alert', adapter: 'overlay', state: 'delivered', result: 'null', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });
+ ins('interaction_effects', { id: 6, interaction_id: 'tint_03', effect: 'paid_message', adapter: 'live-chat', state: 'queued', attempts: 2, next_attempt_at: 1790600000999, last_error: 'Live 503', created_at: T('03T12:00:01.000'), updated_at: T('03T12:00:01.000') });
+ ins('migration_maps', { id: 2, source: 'live', source_table: 'transactions', source_id: '7', target_type: 'interaction', target_id: 'tint_04', status: 'imported', run_id: 'timp_1', created_at: T('01T00:00:00.000'), updated_at: T('01T00:00:00.000') });
+ ins('import_runs', { id: 'timp_1', source: 'live', dry_run: 0, started_at: T('01T00:00:00.000'), finished_at: T('01T00:00:05.000'), report: '{"counts":{"imported":1},"holds":[]}' });
+ ins('tip_moderators', { creator_subject: creator, moderator_subject: supporter, name: 'Mod', added_by: 'invite:tmin_1', created_at: T('02T00:00:00.000') });
+ ins('tip_moderator_invites', { id: 'tmin_1', creator_subject: creator, token_hash: 'b'.repeat(64), created_by: creator, created_at: T('02T00:00:00.000'), expires_at: T('09T00:00:00.000'), used_at: T('02T01:00:00.000'), used_by: supporter });
+ ins('tip_moderation_log', { id: 3, creator_subject: creator, interaction_id: 'tint_04', action: 'hidden', by_role: 'moderator', actor: supporter, reason: 'abuse', created_at: T('05T08:00:00.000') });
+ // A stored answer is replayed byte for byte: its text keeps its key order and spacing.
+ ins('api_idempotency', { key: `${supporter}:tip-key-0001`, request_hash: 'h1', method: 'POST', path: '/api/v1/checkout', status: 201, response: '{"z":1,"a":{"b":2}}', created_at: T('03T12:00:00.000') });
+ ins('event_outbox', { id: 1, event_id: 'evt_01', envelope: '{"event_type":"tips.interaction.ready","subject":{"type":"interaction","id":"tint_01"},"payload":{"supporter":null}}', created_at: 1790500000000, attempts: 1, sent_at: 1790500001000, seq: 17 });
+ ins('event_outbox', { id: 2, event_id: 'evt_02', envelope: '{"event_type":"tips.goal.updated","subject":{"type":"goal","id":"tgoal_01"},"payload":{}}', created_at: 1790500002000, attempts: 3, next_attempt_at: 1790500999000, last_error: 'timeout' });
+ ins('idempotency_receipts', { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });
+ s.close();
+ return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex');
+}
+
+(async () => {
+ const file = path.join(tmp, 'tips.db');
+ const hash = fixture(file);
+
+ const imported = await testDb({ store: 'pglite' });
+ const db = imported.db;
+ let report;
+ const options = await importOptions(db);
+ const TABLES = options.tables;
+ await check('the importer copies every table, verified by counts and checksums (report.ok)', async () => {
+ report = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
+ assert.strictEqual(report.ok, true, JSON.stringify(report.problems));
+ const rows = Object.fromEntries(report.tables.map((t) => [t.table, t.rows]));
+ assert.deepStrictEqual(rows, {
+ creator_tip_profiles: 1, tip_interactions: 4, tip_goals: 1, tip_goal_contributions: 2, paid_messages: 1, paid_media_requests: 1,
+ overlay_configs: 1, overlay_tokens: 1, overlay_deliveries: 2, interaction_effects: 3, migration_maps: 1, import_runs: 1,
+ tip_moderators: 1, tip_moderator_invites: 1, tip_moderation_log: 1, api_idempotency: 1, tips_event_outbox: 2, tips_event_inbox: 1,
+ });
+ assert.ok(report.tables.every((t) => /^[0-9a-f]{16}$/.test(t.checksum)), 'a checksum per table');
+ const renamed = report.tables.filter((t) => t.source !== t.table).map((t) => `${t.source} → ${t.table}`).sort();
+ assert.deepStrictEqual(renamed, ['event_outbox → tips_event_outbox', 'idempotency_receipts → tips_event_inbox']);
+ assert.strictEqual(crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex'), hash, 'the SQLite file is unchanged');
+ });
+
+ await check('values are converted by column type', async () => {
+ const p = await db.one('SELECT * FROM creator_tip_profiles');
+ assert.deepStrictEqual([p.page_enabled, p.tts_enabled, p.filter_links, p.revision], [true, false, false, 4]);
+ assert.deepStrictEqual(p.page_settings, { goal_amounts: false, supporters_page: true });
+ assert.deepStrictEqual(p.filter_words, ['badword', 'two words']);
+ assert.strictEqual(p.updated_at, T('02T11:30:00.250'), 'the same ISO time the API sent before');
+ const i = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_01'");
+ assert.deepStrictEqual(i.request, { goal_id: 'tgoal_01' });
+ assert.deepStrictEqual(i.target, { service: 'live', type: 'stream', id: '42' });
+ assert.deepStrictEqual([i.test, i.anonymous, i.transfer_due, i.amount], [false, false, false, 250]);
+ assert.deepStrictEqual(i.reversal_txn_ids, []);
+ const r = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_02'");
+ assert.deepStrictEqual([r.anonymous, r.hide_amount, r.reversed_bits, r.reversal_txn_ids], [true, true, 100, ['txn_9']]);
+ assert.strictEqual(r.message, 'nulhere', 'a NUL is dropped (PostgreSQL text cannot hold it)');
+ assert.deepStrictEqual(r.request, { legacy_message: 'cut \ufffd', n: 1 }, 'an unpaired surrogate becomes U+FFFD (jsonb refuses it)');
+ assert.deepStrictEqual(options.cleaned().sort((a, b) => a.column.localeCompare(b.column)),
+ [{ column: 'tip_interactions.message', values: 1 }, { column: 'tip_interactions.request', values: 1 }], 'and the report says where');
+ const pending = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_03'");
+ assert.deepStrictEqual([pending.transfer_due, pending.transfer_attempts, pending.next_transfer_at, pending.settled_at], [true, 2, 1790600000123, null]);
+ const erased = await db.one("SELECT * FROM tip_interactions WHERE id = 'tint_04'");
+ assert.deepStrictEqual([erased.test, erased.filtered, erased.erased_at, erased.target], [true, true, T('05T08:00:00.000'), null]);
+ assert.strictEqual(await db.value("SELECT chat_ref FROM paid_messages WHERE id = 'tpm_01'"), null, "JSON text 'null' → NULL (no reference)");
+ assert.deepStrictEqual(await db.value("SELECT queue_ref FROM paid_media_requests WHERE id = 'tmr_01'"), { queue_id: 5 });
+ assert.deepStrictEqual((await db.many('SELECT id, result FROM interaction_effects ORDER BY id')).map((e) => [e.id, e.result]),
+ [[4, { ref: { chat_message_id: 1 } }], [5, null], [6, null]]);
+ assert.deepStrictEqual(await db.value("SELECT scopes FROM overlay_tokens"), ['alerts', 'goals']);
+ assert.deepStrictEqual((await db.many('SELECT seq, hidden, test FROM overlay_deliveries ORDER BY seq')).map((d) => [d.seq, d.hidden, d.test]), [[3, false, false], [9, true, false]]);
+ assert.strictEqual(await db.value('SELECT response FROM api_idempotency'), '{"z":1,"a":{"b":2}}', 'a stored answer stays byte for byte');
+ assert.strictEqual(await db.value('SELECT dry_run FROM import_runs'), false);
+ const out = await db.many('SELECT id, envelope, sent_at, next_attempt_at FROM tips_event_outbox ORDER BY id');
+ assert.deepStrictEqual(out.map((o) => [o.id, o.envelope.event_type, o.sent_at, o.next_attempt_at]), [[1, 'tips.interaction.ready', 1790500001000, 0], [2, 'tips.goal.updated', null, 1790500999000]]);
+ assert.deepStrictEqual(await db.one('SELECT * FROM tips_event_inbox'), { consumer: 'tips-billing', event_id: 'evt_billing_1', processed_at: 1790500000500 });
+ });
+
+ await check('identities keep their values and continue after the imported maximum', async () => {
+ const at = new Date().toISOString();
+ assert.strictEqual(await db.value(`INSERT INTO overlay_deliveries (id, creator_subject, kind, payload, status, dedupe_key, created_at)
+ VALUES ('tovd_new', $1, 'alert', '{}', 'pending', 'alert:new', $2) RETURNING seq`, [creator, at]), 10, 'the SSE ids go on from 9');
+ assert.strictEqual(await db.value(`INSERT INTO interaction_effects (interaction_id, effect, adapter, state, created_at, updated_at)
+ VALUES ('tint_02', 'chat_line', 'test', 'queued', $1, $1) RETURNING id`, [at]), 7);
+ assert.strictEqual(await db.value(`INSERT INTO tip_goal_contributions (goal_id, interaction_id, amount, created_at, updated_at)
+ VALUES ('tgoal_01', 'tint_03', 5, $1, $1) RETURNING id`, [at]), 13);
+ assert.strictEqual(await db.value("INSERT INTO tips_event_outbox (event_id, envelope, created_at) VALUES ('evt_03', '{}', 1) RETURNING id"), 3);
+ });
+
+ await check('a rehearsal can be repeated (truncate), and a value the checksum would miss is caught', async () => {
+ const again = await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
+ assert.strictEqual(again.ok, true, JSON.stringify(again.problems));
+ // Without the 'null' mapping, JSON null and SQL NULL differ: the verification says so.
+ const bad = await importSqlite({ sqlite: file, db, truncate: true, tables: { ...TABLES, paid_messages: {} }, only: ['paid_messages', 'tip_interactions', 'creator_tip_profiles'], log: quiet });
+ assert.strictEqual(bad.ok, false);
+ assert.match(bad.problems.map((x) => `${x.table}: ${x.problem}`).join('\n'), /paid_messages: verification failed/);
+ await importSqlite({ sqlite: file, db, truncate: true, tables: TABLES, log: quiet });
+ });
+
+ await check('the service serves the imported data as it did from SQLite', async () => {
+ const t = await boot({ db });
+ const who = { subject: supporter, username: 'viewer' };
+ const r = await t.call('GET', '/api/v1/interactions', { user: who });
+ assert.strictEqual(r.status, 200, r.text);
+ const byId = Object.fromEntries(r.json.interactions.map((i) => [i.id, i]));
+ assert.deepStrictEqual(Object.keys(byId).sort(), ['tint_01', 'tint_02', 'tint_03']);
+ assert.strictEqual(byId.tint_01.created_at, T('03T12:00:00.000'));
+ assert.strictEqual(byId.tint_01.payment.settled_at, T('03T12:00:00.500'));
+ assert.deepStrictEqual(byId.tint_02.payment.reversal_txn_ids, ['txn_9']);
+ assert.strictEqual(byId.tint_02.public.supporter_name, 'Anonymous');
+ const goal = (await t.call('GET', '/api/v1/goals/tgoal_01', { user: { subject: creator, username: 'alex' } })).json.goal;
+ assert.strictEqual(goal.current_amount, 50 + 250 + 150, 'opening amount plus what the contributions keep');
+ const replay = await t.call('POST', '/api/v1/checkout', { user: who, key: 'tip-key-0001', body: {} });
+ assert.strictEqual(replay.status, 422, 'a different request under a stored key is refused');
+ assert.strictEqual(await t.db.value('SELECT count(*) FROM tips_event_inbox'), 1);
+ await t.close();
+ });
+
+ await check('scripts/migrate-to-postgres.js: OK exits 0; a column the schema lacks is a problem and exits 1', async () => {
+ const out = execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', file], { encoding: 'utf8' });
+ assert.match(out, /→ pglite \(\d+ ms\): OK/);
+ assert.match(out, /tips_event_outbox\s+2 rows\s+[0-9a-f]{16}\s+\(from event_outbox\)/);
+ assert.match(out, /cleaned: tip_interactions\.message: 1 value\(s\) with a NUL or an unpaired surrogate/);
+ const extra = path.join(tmp, 'extra.db');
+ fs.copyFileSync(file, extra);
+ const s = new Database(extra); s.exec('ALTER TABLE tip_goals ADD COLUMN mystery TEXT'); s.close();
+ let code = 0; let text = '';
+ try { execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'migrate-to-postgres.js'), '--pglite', '--sqlite', extra], { encoding: 'utf8' }); } catch (e) { code = e.status; text = e.stdout; }
+ assert.strictEqual(code, 1);
+ assert.match(text, /problem: tip_goals: source columns with no target column .*mystery/);
+ });
+
+ await imported.close();
+ fs.rmSync(tmp, { recursive: true, force: true });
+ done();
+})().catch((e) => { console.error(e); process.exit(1); });