openvibers/openvibe.billing Commits
Retry failed credit renewals in a pastdue grace before ending them (#12)
Changes
@@@ -77,6 +77,9 @@ BILLING_SESSION_SECRET=#BILLING_MIN_CASHOUT_BITS=500#BILLING_ESCROW_DAYS=14#BILLING_STRIPE_GRACE_DAYS=3++# Days a failed credit renewal stays past_due (retried by the hourly sweep on the same key) before it ends.++# 0 (default) ends it at once; turn on only after VIP reads past_due (docs/renewal-grace-cutover.md).++#BILLING_RENEWAL_GRACE_DAYS=0# ── Providers: each adapter is ENABLED only when its secrets are set ──# PowerChat (the only rail carrying money in production)@@@ -200,7 +200,7 @@ are not stored. Errors are RFC 9457 problem+json. People are SubjectRefs `{ "typ| `POST /api/v1/subscriptions` | `billing.subscription.manage` | start/renew a period `{subscriber, streamer, source: credit, auto_renew?}` (share credited every period); `source: receipt` also needs `billing.ledger.admin` || `POST /api/v1/subscriptions/:id/cancel` | `billing.subscription.manage` | cancel at period end; Stripe is cancelled at Stripe first || `GET /api/v1/subscriptions[?subscriber&streamer&status]`, `GET …/:id` | `billing.entitlement.check` | read |--| `GET /api/v1/entitlements/:subject[?streamer=]` | `billing.entitlement.check` | `{active, expires_at, subscription}` — needs nothing but Billing |++| `GET /api/v1/entitlements/:subject[?streamer=]` | `billing.entitlement.check` | `{active, expires_at, grace_until, subscription}` — needs nothing but Billing; `grace_until` is set only while the subscription is `past_due` (never active) || `GET /api/v1/balances/:subject` | `billing.balance.read` | `{credit, payable, pending_payouts, promo_bits, payable_value_cents}` (`promo_bits`: unused free allowance, never transferable) || `GET /api/v1/transactions?subject=&cursor=&limit=`, `GET …/:id` | `billing.balance.read` | history (cursor paging), one transaction with `reversed_by` || `POST /api/v1/usage` | `billing.usage.record` | store a `platform.usage-sample@1` reading (charges nothing here: the rating sweep does, see Rates; accepted while frozen); idempotent by the reading's `idempotency_key`: same money fields (`service`, `project`, `subject`, `resource`, `provider`, `operation`, `quantity`, `unit`, `at`; `at` as an instant, absent = null) → 200 with the first stored row, unchanged (`Idempotent-Replayed: true`), different money fields → 409 `billing.usage_key_reused`; placement and correlation fields (`node`, `cell`, `region`, `route_epoch`, `trace_id`, `source`, `cost_estimate`, `free_allowance_used`, `vibes_charged`) never cause a 409; no `Idempotency-Key` header |@@@ -338,6 +338,15 @@ Reversing a subscription payment revokes the periods it granted.pruned, failed ones are kept. Results: the console, `GET /api/v1/admin/reconciliations`, and `/metrics`(`billing_reconciliation_ok`, `…_failed_checks`, `…_last_run_timestamp_seconds`); a failed scheduled runis also logged.++- **Renewals** run in the hourly subscription sweep (`BILLING_SWEEP_INTERVAL_MS`, not while frozen; on demand++ `POST /api/v1/admin/sweep`): a credit subscription whose period ended is charged once from the subscriber's++ credit (never the promo allowance) under `renew:<sub>:<period end>` — `…:<k>` after `k` charges under that++ period's earlier keys were refunded or charged back — so a retried, replayed or concurrent sweep charges and++ credits the creator share once. A renewal the credit cannot pay ends the subscription at once++ (`renewal_insufficient_credit`) unless `BILLING_RENEWAL_GRACE_DAYS` > 0 (default 0): then it is `past_due`++ until the period end + the grace (`grace_until`; no access meanwhile, event reason `renewal_failed`), every++ later sweep retries the same key (a top-up is picked up; a cancel ends it), and past `grace_until` it expires++ (`grace_ended`). Turning the grace on: [docs/renewal-grace-cutover.md](docs/renewal-grace-cutover.md).- **Freeze** before any risky change: the console's Freeze page, `POST /api/v1/admin/freeze {"on": true, "reason": "…"}`, or on thehost `node scripts/freeze.js on "<reason>"` (same switch; after `off` the running service processes theheld webhooks on its next retry tick).@@@ -356,7 +365,8 @@ Reversing a subscription payment revokes the periods it granted.random ports. What they prove: the journal balances per currency and refuses edits(`ledger.test.js`); a receipt settles once whatever retries carry it, and reversals claw back only whatcan be (`webhooks.test.js`, `providers.test.js`, `stripe.test.js`); subscriptions and entitlements--(`subscriptions.test.js`); EXTERNAL receipts are announced once and only under `billing`++(`subscriptions.test.js`); a renewal is one charge per period, a failed one is retried on its key in the++grace and then ended, a promo allowance never pays it (`renewal-grace.test.js`); EXTERNAL receipts are announced once and only under `billing`(`external.test.js`); the import reconciles (`import.test.js`); freeze, reconciliation and review(`operations.test.js`); the cutover sequence (`cutover.test.js`); the staff console and its staff map(`console.test.js`, `staff-map.test.js`); the public policy (`policy.test.js`); the vhost keeps `/api/v1`@@@ -16,6 +16,7 @@"provider receipts stored before processing, settled once (PowerChat, Stripe, PayPal, CCBill, NOWPayments adapters)","intents, transfers, refunds, recycle, cashouts with escrow","subscriptions and entitlements",++ "renewal grace: a failed credit renewal is past_due and retried on its key until grace_until (BILLING_RENEWAL_GRACE_DAYS, off by default)","EXTERNAL PowerChat receipts (billing.receipt.external under BILLING_AUTHORITY=billing)","economy freeze","hourly reconciliation, ledger against receipts",nnew file mode 100644@@@ -0,0 +1,117 @@++# Cutover runbook — renewal grace (OpenVibe.Billing, plan T5 step 11, s2)++++A credit renewal the subscriber cannot pay used to end the subscription at once (`expired`, reason++`renewal_insufficient_credit`). With this change it can instead move to **`past_due`** for++`BILLING_RENEWAL_GRACE_DAYS` past the period end: the hourly sweep retries the same charge under the **same++idempotency key** until it is paid (`active` again, reason `renewed`, one charge, one creator share) or++`grace_until` passes (`expired`, reason `grace_ended`). **The grace ships off** (`BILLING_RENEWAL_GRACE_DAYS`++defaults to `0`), so the deploy changes no behavior: no row becomes `past_due` until the grace is turned on.++++Also in this change, whatever the grace: the renewal key is `renew:<sub>:<period end>` as before, and becomes++`renew:<sub>:<period end>:<k>` only after `k` charges under that period's earlier keys were refunded or charged back++(a reversal that brings `current_period_end` back to a period already renewed once no longer replays the reversed++charge as a "renewal" with no money and no period). Existing keys and their replays are unchanged.++++No new route, capability or provider change. Events: `billing.entitlement.changed` gains reasons++`renewal_failed` and `grace_ended`, `subscription.status` `past_due`, and top-level `grace_until` /++`renewal_period_end` — all in `openvibe-contracts` v0.94.2 (Contracts PR #31). `GET /api/v1/entitlements/…` and++the subscription reads carry `grace_until` (and `renewal_failed_at`, `renewal_attempts` on the subscription).++++## What the migration does++++`migrations/0005_renewal_grace.sql` (`phase: expand`), in one transaction — additive and idempotent:++++```sql++ALTER TABLE subscriptions DROP CONSTRAINT IF EXISTS subscriptions_status_check;++ALTER TABLE subscriptions ADD CONSTRAINT subscriptions_status_check CHECK (status IN ('active', 'past_due', 'canceled', 'expired'));++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS grace_until text COLLATE "C";++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_failed_at text COLLATE "C";++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_attempts bigint NOT NULL DEFAULT 0;++CREATE INDEX IF NOT EXISTS subscriptions_due_idx ON subscriptions (status, current_period_end);++```++++The CHECK only widens (every existing row passes); the columns are nullable or defaulted, so the old code runs++against the new schema unchanged.++++## Backup++++Before the merge, take the ledger snapshot any data change requires (PR #6's rule):++++```sh++sudo ovhost backup --all --logical++```++++Note the stamp: the logical `pg_dump` lands at++`/var/backups/openvibe/billing/<stamp>/billing.dump` (`root:root`, 0600). Keep it until the deploy is verified;++it is the way back if the restore ever has to be used. Confirm it exists and is non-empty before deploying:++++```sh++sudo ls -l /var/backups/openvibe/billing/<stamp>/billing.dump++```++++## Rehearsal++++The harness records `ov rehearse OpenVibe.Billing 12` itself. `seed: none` keeps the base empty of fixtures++(0005 needs no rows — it only widens a CHECK and adds columns), so the rehearsal builds main's migrations,++applies 0005, runs the migrations a second time (nothing must apply), then checks the widened CHECK, the three++new columns and the sweep index.++++```rehearse++seed: none++node -e 'const a=require("assert"),{createDb}=require("openvibe-sdk/db");const db=createDb({url:process.env.DATABASE_DIRECT_URL,service:"rehearsal",max:1});(async()=>{const c=await db.prepare("SELECT pg_get_constraintdef(oid) AS d FROM pg_constraint WHERE conrelid = ?::regclass AND conname = ?").get("subscriptions","subscriptions_status_check");for(const k of ["active","past_due","canceled","expired"])a.ok(c.d.includes(k),k);const cols=await db.prepare("SELECT column_name AS n, is_nullable AS nullable, column_default AS def FROM information_schema.columns WHERE table_name = ?").all("subscriptions");const by=Object.fromEntries(cols.map((r)=>[r.n,r]));a.strictEqual(by.grace_until.nullable,"YES");a.strictEqual(by.renewal_failed_at.nullable,"YES");a.strictEqual(by.renewal_attempts.nullable,"NO");a.ok(String(by.renewal_attempts.def).includes("0"),by.renewal_attempts.def);const i=await db.prepare("SELECT indexname AS n FROM pg_indexes WHERE tablename = ? AND indexname = ?").get("subscriptions","subscriptions_due_idx");a.ok(i&&i.n,"subscriptions_due_idx");console.log("0005: subscriptions_status_check widened to past_due, grace_until/renewal_failed_at/renewal_attempts added, subscriptions_due_idx present")})().finally(()=>db.close()).catch((e)=>{console.error(e);process.exitCode=1})'++```++++By hand on the restored snapshot (a scratch role, never production), after++`pg_restore` into a throwaway database and running the migrations with `openDb` as in PR #6's++runbook:++++```sql++SELECT pg_get_constraintdef(oid) FROM pg_constraint++ WHERE conrelid = 'subscriptions'::regclass AND conname = 'subscriptions_status_check';++SELECT id, name, phase FROM ov_migrations WHERE id = '0005'; -- 0005 | renewal_grace | expand++SELECT COUNT(*) FROM subscriptions WHERE status = 'past_due'; -- 0 (the grace is off)++```++++and compare `node scripts/reconcile.js` before and after: the same account, transaction and entry counts, every++check ok.++++## Verification++++After the deploy, on production (read-only):++++1. `/api/ready` returns 200 with `db: "ok"`; the boot log has the line++ `migrations: applied 0005_renewal_grace (expand, NNms)`; `ov_migrations` has the `0005` row.++2. `\d subscriptions` lists `grace_until`, `renewal_failed_at` (nullable text) and `renewal_attempts`++ (`bigint NOT NULL DEFAULT 0`), and `subscriptions_due_idx`.++3. `SELECT count(*) FROM subscriptions WHERE status = 'past_due'` is **0**: the grace is still off++ (`BILLING_RENEWAL_GRACE_DAYS` unset or `0` in `/etc/openvibe/billing.env`), so the deploy changed no++ behavior. An unpaid renewal still ends at once (`expired`, reason `renewal_insufficient_credit`).++4. `node scripts/reconcile.js` shows the same totals as before the deploy, every check ok.++++## Order++++1. **Contracts** — v0.94.2 (`past_due`, `renewal_failed`, `grace_ended`, `grace_until`) is released (s1).++2. **Backup** — the snapshot above, and confirm it.++3. **Billing, grace 0** — merge and deploy this change through its pipeline. Leave `BILLING_RENEWAL_GRACE_DAYS`++ unset (= 0). Verify with the checks above. Renewals behave as before.++4. **VIP consumes `past_due`** — VIP (plan s4) treats `subscription.status: past_due` as not entitled, reads++ `grace_until` for its "renew your membership" notice and records each period's charge. Do not go further until++ VIP with that change is deployed.++5. **Turn the grace on** — plan s5 sets the default to 3; on a host before that, set++ `BILLING_RENEWAL_GRACE_DAYS=3` in `/etc/openvibe/billing.env` and restart through the deploy pipeline. Check++ after the next sweep: the log line `subscription sweep: … past due …` and, for a failed renewal,++ `SELECT id, status, grace_until, renewal_attempts FROM subscriptions WHERE status = 'past_due'`.++++## Rollback++++- **Turn the grace off:** set `BILLING_RENEWAL_GRACE_DAYS=0` (or unset it) and restart. The next sweep retries each++ `past_due` subscription once more on its key — a subscriber who topped up renews, with one charge — and ends++ the rest (`expired`, reason `grace_ended`). Nothing else changes. This is the way back for behavior; no++ schema restore is needed.++- **Roll the code back:** turn the grace off first and wait for one sweep, so no row is `past_due` (the old code++ neither renews nor ends a `past_due` row). Then deploy the previous release (`sudo ovhost rollback billing`);++ **the migration stays**: never drop the columns or narrow the CHECK (old code ignores them).++- **Restore the snapshot (last resort):** stop the unit, `pg_restore` the `billing.dump` from the stamp above++ over `ov_billing`, then start the previous release. Only if the schema or data is actually wrong; the migrate is++ additive, so rollback without restore is the normal path.++- A renewal charged under a `…:<k>` key is an ordinary `subscription` transaction; rolling back does not touch it.nnew file mode 100644@@@ -0,0 +1,20 @@++-- phase: expand++-- Renewal grace (plan T5 step 11, s2): a credit renewal the subscriber cannot pay moves the subscription to `past_due`++-- until grace_until, and the renewal sweep retries the charge under the SAME idempotency key until it is paid++-- (active again) or grace_until passes (expired, reason grace_ended). Off by default: BILLING_RENEWAL_GRACE_DAYS=0++-- keeps the old behavior (expire at once, reason renewal_insufficient_credit), so no row becomes past_due until the++-- grace is turned on. Additive and idempotent: the one ALTER of an existing CHECK widens subscriptions_status_check++-- (the name PostgreSQL gave 0001's inline CHECK on subscriptions.status) by 'past_due'; the rest are new columns, two++-- nullable and one with a default. Rollback is BILLING_RENEWAL_GRACE_DAYS=0 (the sweep expires the past_due rows);++-- the columns and the widened CHECK stay (docs/renewal-grace-cutover.md).++++ALTER TABLE subscriptions DROP CONSTRAINT IF EXISTS subscriptions_status_check;++ALTER TABLE subscriptions ADD CONSTRAINT subscriptions_status_check CHECK (status IN ('active', 'past_due', 'canceled', 'expired'));++++-- past_due only: when the sweep stops retrying and ends the subscription (the failed period's end + the grace days).++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS grace_until text COLLATE "C";++-- The last failed renewal charge, and how many failed for the current period (both reset by a paid period).++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_failed_at text COLLATE "C";++ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_attempts bigint NOT NULL DEFAULT 0;++++CREATE INDEX IF NOT EXISTS subscriptions_due_idx ON subscriptions (status, current_period_end);@@@ -111,6 +111,9 @@ function loadConfig(env = process.env) {minCashoutBits: int(env.BILLING_MIN_CASHOUT_BITS, 500),escrowDays: num(env.BILLING_ESCROW_DAYS, 14),stripeGraceDays: num(env.BILLING_STRIPE_GRACE_DAYS, 3),++ // A failed credit renewal stays past_due (retried by the sweep) this many days past the period end;++ // 0 = end it at once (renewal_insufficient_credit).++ renewalGraceDays: Math.max(0, num(env.BILLING_RENEWAL_GRACE_DAYS, 0)),},// Provider adapters: each is enabled only when its secrets are set.@@@ -35,7 +35,7 @@ const { runScheduled } = require('./reconcile');every(config.jobs.sweepIntervalMs, async () => {if (await isFrozen(ctx.db)) return;const out = await subscriptions.sweep(ctx);-- if (out.renewed.length || out.expired.length || out.canceled.length) console.log(`[Billing] subscription sweep: ${out.renewed.length} renewed, ${out.expired.length} expired, ${out.canceled.length} canceled`);++ if (out.renewed.length || out.expired.length || out.canceled.length || out.past_due.length) console.log(`[Billing] subscription sweep: ${out.renewed.length} renewed, ${out.past_due.length} past due, ${out.expired.length} expired, ${out.canceled.length} canceled`);});if (config.jobs.reconcileIntervalMs > 0) {const reconcileNow = async () => await runScheduled(ctx, { keepDays: config.jobs.reconcileKeepDays });@@@ -15,6 +15,11 @@** Entitlement truth is the entitlements table: one row per granted period; a subject is* entitled to a streamer while a non-revoked row covers now. It needs nothing from Live.++ *++ * A credit renewal the subscriber cannot pay leaves the subscription past_due for++ * BILLING_RENEWAL_GRACE_DAYS (0 = end it at once): no period covers the grace, so the entitlement++ * is not active, and the sweep retries the same charge under the same key until it is paid or++ * grace_until passes.*/const { post, getTxn, requireFunds, iso, prefixedId, money } = require('../ledger');const { enqueue } = require('../outbox');@@@ -24,7 +29,7 @@ const intents = require('./intents');const KIND = 'channel_subscription';--function parse(row) { return row ? { ...row, auto_renew: !!row.auto_renew, cancel_at_period_end: !!row.cancel_at_period_end } : null; }++function parse(row) { return row ? { ...row, auto_renew: !!row.auto_renew, cancel_at_period_end: !!row.cancel_at_period_end, renewal_attempts: Number(row.renewal_attempts) || 0 } : null; }async function find(db, id) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE id = ?').get(id)); }async function findPair(db, subscriber, streamer) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE subscriber = ? AND streamer = ?').get(subscriber, streamer)); }async function findByProviderRef(db, provider, ref) { return ref ? parse(await db.prepare('SELECT * FROM subscriptions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) : null; }@@@ -35,7 +40,8 @@ function present(s) {id: s.id, subscriber: { type: 'user', id: s.subscriber }, streamer: { type: 'user', id: s.streamer }, tier: s.tier,provider: s.provider, provider_ref: s.provider_ref || null, route: s.route || null, status: s.status,auto_renew: !!s.auto_renew, cancel_at_period_end: !!s.cancel_at_period_end, price_cents: s.price_cents,-- current_period_end: s.current_period_end, created_at: s.created_at, updated_at: s.updated_at,++ current_period_end: s.current_period_end, grace_until: s.grace_until || null, renewal_failed_at: s.renewal_failed_at || null,++ renewal_attempts: Number(s.renewal_attempts) || 0, created_at: s.created_at, updated_at: s.updated_at,};}@@@ -58,6 +64,8 @@ async function entitlement(db, subject, streamer, ms) {return {subject: { type: 'user', id: subject }, streamer: { type: 'user', id: streamer }, kind: KIND,active: !!expires, expires_at: expires || null,++ // past_due: when the sweep stops retrying the renewal. The grace grants no access (active stays false).++ grace_until: sub && sub.status === 'past_due' ? sub.grace_until || null : null,subscription: sub ? { id: sub.id, status: sub.status, auto_renew: sub.auto_renew, cancel_at_period_end: sub.cancel_at_period_end, provider: sub.provider } : null,};}@@@ -122,7 +130,7 @@ async function pay(ctx, input) {const running = prior && prior.current_period_end && Date.parse(prior.current_period_end) > ms ? Date.parse(prior.current_period_end) : ms;const start = running;const end = r && r.periodEnd ? Math.max(Date.parse(r.periodEnd), start + 1) : start + rates.subPeriodDays * 86_400_000;-- const renewal = !!(prior && (prior.status === 'active' || input.renewal));++ const renewal = !!(prior && (prior.status === 'active' || prior.status === 'past_due' || input.renewal));const { txn } = await post(ctx, {type: 'subscription',@@@ -143,7 +151,8 @@ async function pay(ctx, input) {let sub;if (prior) {await db.prepare(`UPDATE subscriptions SET provider = ?, provider_ref = COALESCE(?, provider_ref), route = ?, status = 'active', auto_renew = ?,-- cancel_at_period_end = 0, price_cents = ?, current_period_end = ?, updated_at = ? WHERE id = ?`)++ cancel_at_period_end = 0, price_cents = ?, current_period_end = ?, grace_until = NULL, renewal_failed_at = NULL, renewal_attempts = 0,++ updated_at = ? WHERE id = ?`).run(provider, r ? r.providerRef || null : null, route, autoRenew, priceCents, iso(end), nowIso, prior.id);sub = await find(db, prior.id);} else {@@@ -180,7 +189,8 @@ async function cancel(ctx, { id, actor }, adapters) {const { db } = ctx;const sub = await find(db, id);if (!sub) fail(404, 'billing.subscription_not_found', `no subscription ${id}`);-- if (sub.status !== 'active') return { subscription: sub, provider_sync: 'not_needed', replay: true };++ // past_due: cancelling stops the retries; the next sweep ends it.++ if (sub.status !== 'active' && sub.status !== 'past_due') return { subscription: sub, provider_sync: 'not_needed', replay: true };let providerSync = 'not_needed';if (sub.provider === 'stripe') {const stripe = adapters && adapters.stripe;@@@ -200,19 +210,62 @@ async function cancel(ctx, { id, actor }, adapters) {return { subscription: await find(db, id), provider_sync: providerSync, replay: false };}--async function setStatus(ctx, id, status, reason) {++/**++ * End (or reactivate) a subscription and say so. `extra` joins the event payload. With `periodEnd`, the change applies++ * only while the row still has that period end (a renewal paid meanwhile wins).++ */++async function setStatus(ctx, id, status, reason, { extra, periodEnd } = {}) {const { db } = ctx;const sub = await find(db, id);if (!sub || sub.status === status) return sub;-- await db.prepare('UPDATE subscriptions SET status = ?, auto_renew = CASE WHEN ? = \'active\' THEN auto_renew ELSE 0 END, updated_at = ? WHERE id = ?')-- .run(status, status, iso(ctx.now()), id);++ const r = await db.prepare(`UPDATE subscriptions SET status = ?, auto_renew = CASE WHEN ? = 'active' THEN auto_renew ELSE 0 END, grace_until = NULL, updated_at = ?++ WHERE id = ? AND status = ?${periodEnd ? ' AND current_period_end = ?' : ''}`)++ .run(status, status, iso(ctx.now()), id, sub.status, ...(periodEnd ? [periodEnd] : []));++ if (!r.changes) return await find(db, id);await enqueue(ctx, {event_type: 'billing.entitlement.changed', subject: { type: 'entitlement', id: `${sub.subscriber}:${KIND}:${sub.streamer}` },-- payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: reason || status },++ payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: reason || status, ...(extra || {}) },});return await find(db, id);}++/**++ * A renewal charge failed: past_due until `graceUntil` (kept from the first failure of the period). The event goes out++ * once, when the subscription turns past_due; a failed retry only counts the attempt. No-op if the row changed since++ * the sweep read it (a renewal paid, the subscription ended, or a concurrent sweep marked it first).++ */++async function markPastDue(ctx, sub, graceUntil) {++ const { db } = ctx;++ return await money(db, async () => {++ const at = iso(ctx.now());++ const r = await db.prepare(`UPDATE subscriptions SET status = 'past_due', grace_until = ?, renewal_failed_at = ?, renewal_attempts = renewal_attempts + 1, updated_at = ?++ WHERE id = ? AND status = ? AND current_period_end = ?`).run(graceUntil, at, at, sub.id, sub.status, sub.current_period_end);++ if (!r.changes || sub.status === 'past_due') return !!r.changes;++ await enqueue(ctx, {++ event_type: 'billing.entitlement.changed', subject: { type: 'entitlement', id: `${sub.subscriber}:${KIND}:${sub.streamer}` },++ payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: 'renewal_failed', grace_until: graceUntil, renewal_period_end: sub.current_period_end },++ });++ return true;++ });++}++++/**++ * The renewal key of the period that ends at the subscription's current_period_end: renew:<sub>:<period end>, then++ * renew:<sub>:<period end>:<k> once k charges under the earlier keys were refunded or charged back (a reversal can++ * bring current_period_end back to an end already renewed once; replaying that key would "renew" with no charge and++ * no period). An unreversed charge under a key is that renewal: the key is returned and pay() replays it.++ */++async function renewalKey(db, sub) {++ const base = `renew:${sub.id}:${sub.current_period_end}`;++ for (let k = 0; ; k++) {++ const key = k ? `${base}:${k}` : base;++ const txn = await db.prepare('SELECT id FROM transactions WHERE idempotency_key = ?').get(key);++ if (!txn) return key;++ const reversed = await db.prepare("SELECT 1 AS x FROM transactions WHERE reverses_txn = ? AND type IN ('refund', 'chargeback') LIMIT 1").get(txn.id);++ if (!reversed) return key;++ }++}++/** Revoke the periods a reversed payment granted. Returns the number of periods revoked. */async function revokeForTxn(ctx, txnId, reason) {const { db } = ctx;@@@ -237,18 +290,22 @@ async function revokeForTxn(ctx, txnId, reason) {}/**-- * Renewal sweep (hourly job). For every active subscription whose period ended:++ * Renewal sweep (hourly job). For every active or past_due subscription whose period ended:* - Stripe: Stripe renews itself (invoice.paid); expire only after the grace window;* - cancel-at-period-end / no auto-renew: end it;-- * - otherwise renew from the subscriber's credit (deterministic key per period), or end it-- * when the credit does not cover the price.++ * - past_due past grace_until: end it (grace_ended);++ * - otherwise renew from the subscriber's credit (deterministic key per period, renewalKey), or,++ * when the credit does not cover the price, past_due until the period end + the renewal grace++ * (retried on the same key by every later sweep), or end it at once without a grace.++ * Canceled and expired subscriptions are never charged.*/async function sweep(ctx, { limit = 200 } = {}) {const { db, rates } = ctx;-- const out = { renewed: [], expired: [], canceled: [], skipped: 0 };++ const out = { renewed: [], expired: [], canceled: [], past_due: [], skipped: 0 };const now = ctx.now();-- const due = (await db.prepare("SELECT * FROM subscriptions WHERE status = 'active' AND current_period_end IS NOT NULL AND current_period_end <= ? ORDER BY current_period_end LIMIT ?")-- .all(iso(now), limit)).map(parse);++ const graceMs = Math.max(0, Number(rates.renewalGraceDays) || 0) * 86_400_000;++ const due = (await db.prepare(`SELECT * FROM subscriptions WHERE status IN ('active', 'past_due') AND current_period_end IS NOT NULL AND current_period_end <= ?++ ORDER BY current_period_end LIMIT ?`).all(iso(now), limit)).map(parse);for (const sub of due) {const end = Date.parse(sub.current_period_end);if (sub.provider === 'stripe') {@@@ -259,16 +316,26 @@ async function sweep(ctx, { limit = 200 } = {}) {const st = sub.cancel_at_period_end ? 'canceled' : 'expired';await setStatus(ctx, sub.id, st, st); (st === 'canceled' ? out.canceled : out.expired).push(sub.id); continue;}++ const ended = { periodEnd: sub.current_period_end, extra: { renewal_period_end: sub.current_period_end } };++ if (sub.status === 'past_due' && (!sub.grace_until || now >= Date.parse(sub.grace_until))) {++ await setStatus(ctx, sub.id, 'expired', 'grace_ended', ended); out.expired.push(sub.id); continue;++ }try {-- await pay(ctx, {++ const paid = await pay(ctx, {subscriber: sub.subscriber, streamer: sub.streamer, source: 'credit', priceCents: sub.price_cents || rates.subPriceCents,-- autoRenew: true, renewal: true, idempotencyKey: `renew:${sub.id}:${sub.current_period_end}`,++ autoRenew: true, renewal: true, idempotencyKey: await renewalKey(db, sub),actor: { principal: 'svc:billing', job: 'renewal-sweep' },});-- out.renewed.push(sub.id);++ // A replay means a concurrent sweep already renewed this period: it reports it, this one skips it.++ if (paid.replay) out.skipped++; else out.renewed.push(sub.id);} catch (e) {if (e.code !== 'billing.insufficient_funds') throw e;-- await setStatus(ctx, sub.id, 'expired', 'renewal_insufficient_credit'); out.expired.push(sub.id);++ const graceUntil = sub.grace_until || iso(end + graceMs);++ if (graceMs > 0 && now < Date.parse(graceUntil)) {++ if (await markPastDue(ctx, sub, graceUntil)) out.past_due.push(sub.id);++ continue;++ }++ await setStatus(ctx, sub.id, 'expired', sub.status === 'past_due' ? 'grace_ended' : 'renewal_insufficient_credit', ended); out.expired.push(sub.id);}}return out;@@@ -283,4 +350,4 @@ async function list(db, { subscriber, streamer, status } = {}) {return (await db.prepare(`SELECT * FROM subscriptions ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY created_at DESC LIMIT 500`).all(...args)).map(parse);}--module.exports = { pay, cancel, sweep, entitlement, activeEntitlements, revokeForTxn, setStatus, find, findPair, findByProviderRef, list, present, KIND };++module.exports = { pay, cancel, sweep, renewalKey, entitlement, activeEntitlements, revokeForTxn, setStatus, find, findPair, findByProviderRef, list, present, KIND };nnew file mode 100644@@@ -0,0 +1,185 @@++'use strict';++/**++ * Renewal grace (plan T5 step 11, s2; money path): a credit renewal is one charge and one creator share per period++ * however often or concurrently the sweep runs; a renewal the subscriber cannot pay leaves the subscription past_due++ * (no access) and is retried under the SAME key until it is paid or the grace ends; a promo allowance never pays a++ * renewal; BILLING_RENEWAL_GRACE_DAYS=0 ends it at once; and a renewal reversed by a refund renews under a fresh key.++ */++const assert = require('assert');++const { boot, fund, check, done } = require('./helpers/app');++++const DAY = 86_400_000;++++(async () => {++ const t = await boot({ env: { BILLING_RENEWAL_GRACE_DAYS: '3' } });++ const subs = require('../server/ops/subscriptions');++ const { post, money } = require('../server/ledger');++ console.log('renewal grace');++++ const streamer = t.user(60);++ const sweep = async () => {++ const r = await t.call('POST', '/api/v1/admin/sweep', { key: null });++ assert.strictEqual(r.status, 200, r.text);++ return r.json;++ };++ const subscribe = async (u, bits) => {++ if (bits) await fund(t, u, bits);++ const r = await t.call('POST', '/api/v1/subscriptions', { body: { subscriber: u, streamer, source: 'credit' } });++ assert.strictEqual(r.status, 201, r.text);++ return r.json.subscription;++ };++ const row = async (id) => await subs.find(t.db, id);++ const renewals = async (id) => (await t.db.prepare("SELECT idempotency_key FROM transactions WHERE type = 'subscription' AND idempotency_key LIKE ? ORDER BY created_at").all(`renew:${id}:%`)).map((r) => r.idempotency_key);++ const changed = async (u) => (await t.db.prepare('SELECT event FROM outbox ORDER BY seq').all()).map((r) => JSON.parse(r.event))++ .filter((e) => e.event_type === 'billing.entitlement.changed' && e.subject.id.startsWith(`${u.id}:`)).map((e) => e.payload);++ const ent = async (u) => (await t.call('GET', `/api/v1/entitlements/${u.id}?streamer=${streamer.id}`)).json;++ const payable = async () => (await t.balances(streamer.id)).payable;++++ const [a, b, c, d, e] = [t.user(61), t.user(62), t.user(63), t.user(64), t.user(65)];++ const subA = await subscribe(a, 998); // can pay one renewal++ const subB = await subscribe(b, 499); // fails, tops up during the grace++ const subC = await subscribe(c, 499); // fails, never pays++ const subD = await subscribe(d, 499); // fails, cancels during the grace++ const subE = await subscribe(e, 499); // promo allowance only++ const P = subB.current_period_end;++ const graceUntil = (s) => new Date(Date.parse(s.current_period_end) + 3 * DAY).toISOString();++ let share = 5 * 349;++++ await check('a promo allowance stays out of renewals', async () => {++ t.clock.offset = 32 * DAY;++ const g = await t.call('POST', '/api/v1/admin/promo/grant', { body: { subject: e, bits: 1000, service: 'openvibe.tools' }, cap: ['billing.ledger.admin'] });++ assert.strictEqual(g.status, 201, g.text);++ assert.strictEqual(await payable(), share);++ });++++ await check('concurrent sweeps renew a paid-up subscription once: one charge, one creator share', async () => {++ const [s1, s2] = await Promise.all([sweep(), sweep()]);++ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subA.id]);++ assert.deepStrictEqual(await renewals(subA.id), [`renew:${subA.id}:${subA.current_period_end}`]);++ share += 349;++ assert.strictEqual(await payable(), share);++ assert.strictEqual((await t.balances(a.id)).credit, 0);++ const replay = await subs.pay(t.ctx, { subscriber: a, streamer, source: 'credit', renewal: true, idempotencyKey: `renew:${subA.id}:${subA.current_period_end}`, actor: { principal: 'test' } });++ assert.strictEqual(replay.replay, true);++ assert.strictEqual(await payable(), share, 'a replayed renewal credits no second share');++ assert.deepStrictEqual((await sweep()).renewed, [], 'the renewed period is not due again');++ });++++ await check('an unpaid renewal turns past_due with grace_until, grants no access and changes no money', async () => {++ for (const s of [subB, subC, subD, subE]) {++ const r = await row(s.id);++ assert.strictEqual(r.status, 'past_due', s.id);++ assert.strictEqual(r.grace_until, graceUntil(s));++ assert.ok(r.renewal_attempts >= 1); // the sweeps above retried it once or twice++ assert.strictEqual(r.auto_renew, true);++ assert.deepStrictEqual(await renewals(s.id), []);++ }++ const ev = (await changed(b)).filter((p) => p.reason === 'renewal_failed');++ assert.strictEqual(ev.length, 1, 'one renewal_failed event despite two concurrent sweeps');++ assert.deepStrictEqual([ev[0].active, ev[0].subscription.status, ev[0].grace_until, ev[0].renewal_period_end], [false, 'past_due', graceUntil(subB), P]);++ const en = await ent(b);++ assert.deepStrictEqual([en.active, en.subscription.status, en.grace_until], [false, 'past_due', graceUntil(subB)]);++ const got = (await t.call('GET', `/api/v1/subscriptions/${subB.id}`)).json;++ assert.strictEqual(got.subscription.grace_until, graceUntil(subB));++ assert.strictEqual(await payable(), share);++ const be = await t.balances(e.id);++ assert.deepStrictEqual([be.credit, be.promo_bits], [0, 1000], 'the promo allowance paid nothing');++ await t.assertReconciled('after failed renewals');++ });++++ await check('each sweep in the grace retries the same charge; a retry that fails again emits nothing', async () => {++ t.clock.offset = 33 * DAY;++ const before = (await changed(b)).length;++ const attempts = (await row(subB.id)).renewal_attempts;++ const s = await sweep();++ assert.deepStrictEqual(s.renewed, []);++ assert.strictEqual((await row(subB.id)).renewal_attempts, attempts + 1);++ assert.strictEqual((await row(subB.id)).grace_until, graceUntil(subB), 'the grace is not extended by a retry');++ assert.strictEqual((await changed(b)).length, before);++ assert.strictEqual(await subs.renewalKey(t.db, await row(subB.id)), `renew:${subB.id}:${P}`);++ });++++ await check('a top-up in the grace is picked up by the next sweep: active again, one charge, one share', async () => {++ await fund(t, b, 499);++ const c1 = await t.call('POST', `/api/v1/subscriptions/${subD.id}/cancel`, { cap: ['billing.subscription.manage'], body: {} });++ assert.strictEqual(c1.status, 200, c1.text);++ assert.strictEqual(c1.json.subscription.cancel_at_period_end, true);++ t.clock.offset = 33.5 * DAY;++ const [s1, s2] = await Promise.all([sweep(), sweep()]);++ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subB.id]);++ assert.deepStrictEqual(await renewals(subB.id), [`renew:${subB.id}:${P}`], 'the retried key is the period key');++ share += 349;++ assert.strictEqual(await payable(), share);++ const r = await row(subB.id);++ assert.deepStrictEqual([r.status, r.grace_until, r.renewal_failed_at, r.renewal_attempts], ['active', null, null, 0]);++ const en = await ent(b);++ assert.deepStrictEqual([en.active, en.grace_until], [true, null]);++ assert.strictEqual((await changed(b)).at(-1).reason, 'renewed');++ assert.strictEqual((await row(subD.id)).status, 'canceled', 'cancelling in the grace ends it, uncharged');++ assert.deepStrictEqual(await renewals(subD.id), []);++ });++++ await check('past grace_until the sweep ends it (grace_ended) and never charges it again', async () => {++ t.clock.offset = 35.1 * DAY;++ const s = await sweep();++ assert.deepStrictEqual(s.expired.sort(), [subC.id, subE.id].sort());++ for (const [u, sb] of [[c, subC], [e, subE]]) {++ assert.strictEqual((await row(sb.id)).status, 'expired');++ const last = (await changed(u)).at(-1);++ assert.deepStrictEqual([last.reason, last.active, last.subscription.status, last.renewal_period_end], ['grace_ended', false, 'expired', sb.current_period_end]);++ }++ await fund(t, c, 499);++ t.clock.offset = 36 * DAY;++ await sweep();++ assert.deepStrictEqual(await renewals(subC.id), [], 'an expired subscription is never charged');++ assert.strictEqual((await t.balances(c.id)).credit, 499);++ assert.deepStrictEqual(await renewals(subE.id), [], 'the promo allowance never paid a renewal');++ assert.strictEqual(await payable(), share);++ await t.assertReconciled('after the grace');++ });++++ await check('with BILLING_RENEWAL_GRACE_DAYS=0 an unpaid renewal ends at once (renewal_insufficient_credit)', async () => {++ t.ctx.rates.renewalGraceDays = 0;++ const f = t.user(66);++ const subF = await subscribe(f, 499);++ t.clock.offset = 36 * DAY + 32 * DAY;++ await sweep();++ const r = await row(subF.id);++ assert.deepStrictEqual([r.status, r.grace_until, r.renewal_attempts], ['expired', null, 0]);++ assert.strictEqual((await changed(f)).at(-1).reason, 'renewal_insufficient_credit');++ t.ctx.rates.renewalGraceDays = 3;++ });++++ await check('a renewal reversed by a refund renews under a fresh key, with a charge and a period', async () => {++ const g = t.user(67);++ const subG = await subscribe(g, 998);++ const PG = subG.current_period_end;++ t.clock.offset += 32 * DAY;++ await sweep();++ const [key0] = await renewals(subG.id);++ assert.strictEqual(key0, `renew:${subG.id}:${PG}`);++ const orig = await t.db.prepare('SELECT id FROM transactions WHERE idempotency_key = ?').get(key0);++ // A reversal that brings current_period_end back to PG (a credit-period refund, plan s3) with the subscription renewing.++ await money(t.db, async () => {++ await post(t.ctx, { type: 'refund', idempotencyKey: `test-refund:${orig.id}`, reversesTxn: orig.id, entries: [], actor: { principal: 'test' }, metadata: { cents: 0 } });++ await subs.revokeForTxn(t.ctx, orig.id, 'refund');++ });++ await t.db.prepare("UPDATE subscriptions SET status = 'active', auto_renew = 1 WHERE id = ?").run(subG.id);++ assert.strictEqual((await row(subG.id)).current_period_end, PG);++ assert.strictEqual(await subs.renewalKey(t.db, await row(subG.id)), `${key0}:1`);++ await fund(t, g, 499);++ const before = await payable();++ const [s1, s2] = await Promise.all([sweep(), sweep()]);++ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subG.id]);++ assert.deepStrictEqual(await renewals(subG.id), [key0, `${key0}:1`]);++ assert.strictEqual(await payable(), before + 349, 'one charge, one share');++ assert.strictEqual((await t.balances(g.id)).credit, 0);++ assert.strictEqual((await ent(g)).active, true);++ await sweep();++ assert.strictEqual((await renewals(subG.id)).length, 2);++ });++++ t.clock.offset = 0;++ await t.close();++ done();++})();
@@ -77,6 +77,9 @@ BILLING_SESSION_SECRET=
@@ -77,6 +77,9 @@ BILLING_SESSION_SECRET=
#BILLING_MIN_CASHOUT_BITS=500
#BILLING_MIN_CASHOUT_BITS=500
#BILLING_ESCROW_DAYS=14
#BILLING_ESCROW_DAYS=14
#BILLING_STRIPE_GRACE_DAYS=3
#BILLING_STRIPE_GRACE_DAYS=3
+# Days a failed credit renewal stays past_due (retried by the hourly sweep on the same key) before it ends.
+# 0 (default) ends it at once; turn on only after VIP reads past_due (docs/renewal-grace-cutover.md).
+#BILLING_RENEWAL_GRACE_DAYS=0
# ── Providers: each adapter is ENABLED only when its secrets are set ──
# ── Providers: each adapter is ENABLED only when its secrets are set ──
# PowerChat (the only rail carrying money in production)
# PowerChat (the only rail carrying money in production)
@@ -200,7 +200,7 @@ are not stored. Errors are RFC 9457 problem+json. People are SubjectRefs `{ "typ
@@ -200,7 +200,7 @@ are not stored. Errors are RFC 9457 problem+json. People are SubjectRefs `{ "typ
| `POST /api/v1/subscriptions` | `billing.subscription.manage` | start/renew a period `{subscriber, streamer, source: credit, auto_renew?}` (share credited every period); `source: receipt` also needs `billing.ledger.admin` |
| `POST /api/v1/subscriptions` | `billing.subscription.manage` | start/renew a period `{subscriber, streamer, source: credit, auto_renew?}` (share credited every period); `source: receipt` also needs `billing.ledger.admin` |
| `POST /api/v1/subscriptions/:id/cancel` | `billing.subscription.manage` | cancel at period end; Stripe is cancelled at Stripe first |
| `POST /api/v1/subscriptions/:id/cancel` | `billing.subscription.manage` | cancel at period end; Stripe is cancelled at Stripe first |
| `GET /api/v1/subscriptions[?subscriber&streamer&status]`, `GET …/:id` | `billing.entitlement.check` | read |
| `GET /api/v1/subscriptions[?subscriber&streamer&status]`, `GET …/:id` | `billing.entitlement.check` | read |
-| `GET /api/v1/entitlements/:subject[?streamer=]` | `billing.entitlement.check` | `{active, expires_at, subscription}` — needs nothing but Billing |
+| `GET /api/v1/entitlements/:subject[?streamer=]` | `billing.entitlement.check` | `{active, expires_at, grace_until, subscription}` — needs nothing but Billing; `grace_until` is set only while the subscription is `past_due` (never active) |
| `GET /api/v1/balances/:subject` | `billing.balance.read` | `{credit, payable, pending_payouts, promo_bits, payable_value_cents}` (`promo_bits`: unused free allowance, never transferable) |
| `GET /api/v1/balances/:subject` | `billing.balance.read` | `{credit, payable, pending_payouts, promo_bits, payable_value_cents}` (`promo_bits`: unused free allowance, never transferable) |
| `GET /api/v1/transactions?subject=&cursor=&limit=`, `GET …/:id` | `billing.balance.read` | history (cursor paging), one transaction with `reversed_by` |
| `GET /api/v1/transactions?subject=&cursor=&limit=`, `GET …/:id` | `billing.balance.read` | history (cursor paging), one transaction with `reversed_by` |
| `POST /api/v1/usage` | `billing.usage.record` | store a `platform.usage-sample@1` reading (charges nothing here: the rating sweep does, see Rates; accepted while frozen); idempotent by the reading's `idempotency_key`: same money fields (`service`, `project`, `subject`, `resource`, `provider`, `operation`, `quantity`, `unit`, `at`; `at` as an instant, absent = null) → 200 with the first stored row, unchanged (`Idempotent-Replayed: true`), different money fields → 409 `billing.usage_key_reused`; placement and correlation fields (`node`, `cell`, `region`, `route_epoch`, `trace_id`, `source`, `cost_estimate`, `free_allowance_used`, `vibes_charged`) never cause a 409; no `Idempotency-Key` header |
| `POST /api/v1/usage` | `billing.usage.record` | store a `platform.usage-sample@1` reading (charges nothing here: the rating sweep does, see Rates; accepted while frozen); idempotent by the reading's `idempotency_key`: same money fields (`service`, `project`, `subject`, `resource`, `provider`, `operation`, `quantity`, `unit`, `at`; `at` as an instant, absent = null) → 200 with the first stored row, unchanged (`Idempotent-Replayed: true`), different money fields → 409 `billing.usage_key_reused`; placement and correlation fields (`node`, `cell`, `region`, `route_epoch`, `trace_id`, `source`, `cost_estimate`, `free_allowance_used`, `vibes_charged`) never cause a 409; no `Idempotency-Key` header |
@@ -338,6 +338,15 @@ Reversing a subscription payment revokes the periods it granted.
@@ -338,6 +338,15 @@ Reversing a subscription payment revokes the periods it granted.
pruned, failed ones are kept. Results: the console, `GET /api/v1/admin/reconciliations`, and `/metrics`
pruned, failed ones are kept. Results: the console, `GET /api/v1/admin/reconciliations`, and `/metrics`
(`billing_reconciliation_ok`, `…_failed_checks`, `…_last_run_timestamp_seconds`); a failed scheduled run
(`billing_reconciliation_ok`, `…_failed_checks`, `…_last_run_timestamp_seconds`); a failed scheduled run
is also logged.
is also logged.
+- **Renewals** run in the hourly subscription sweep (`BILLING_SWEEP_INTERVAL_MS`, not while frozen; on demand
+ `POST /api/v1/admin/sweep`): a credit subscription whose period ended is charged once from the subscriber's
+ credit (never the promo allowance) under `renew:<sub>:<period end>` — `…:<k>` after `k` charges under that
+ period's earlier keys were refunded or charged back — so a retried, replayed or concurrent sweep charges and
+ credits the creator share once. A renewal the credit cannot pay ends the subscription at once
+ (`renewal_insufficient_credit`) unless `BILLING_RENEWAL_GRACE_DAYS` > 0 (default 0): then it is `past_due`
+ until the period end + the grace (`grace_until`; no access meanwhile, event reason `renewal_failed`), every
+ later sweep retries the same key (a top-up is picked up; a cancel ends it), and past `grace_until` it expires
+ (`grace_ended`). Turning the grace on: [docs/renewal-grace-cutover.md](docs/renewal-grace-cutover.md).
- **Freeze** before any risky change: the console's Freeze page, `POST /api/v1/admin/freeze {"on": true, "reason": "…"}`, or on the
- **Freeze** before any risky change: the console's Freeze page, `POST /api/v1/admin/freeze {"on": true, "reason": "…"}`, or on the
host `node scripts/freeze.js on "<reason>"` (same switch; after `off` the running service processes the
host `node scripts/freeze.js on "<reason>"` (same switch; after `off` the running service processes the
held webhooks on its next retry tick).
held webhooks on its next retry tick).
@@ -356,7 +365,8 @@ Reversing a subscription payment revokes the periods it granted.
@@ -356,7 +365,8 @@ Reversing a subscription payment revokes the periods it granted.
random ports. What they prove: the journal balances per currency and refuses edits
random ports. What they prove: the journal balances per currency and refuses edits
(`ledger.test.js`); a receipt settles once whatever retries carry it, and reversals claw back only what
(`ledger.test.js`); a receipt settles once whatever retries carry it, and reversals claw back only what
can be (`webhooks.test.js`, `providers.test.js`, `stripe.test.js`); subscriptions and entitlements
can be (`webhooks.test.js`, `providers.test.js`, `stripe.test.js`); subscriptions and entitlements
-(`subscriptions.test.js`); EXTERNAL receipts are announced once and only under `billing`
+(`subscriptions.test.js`); a renewal is one charge per period, a failed one is retried on its key in the
+grace and then ended, a promo allowance never pays it (`renewal-grace.test.js`); EXTERNAL receipts are announced once and only under `billing`
(`external.test.js`); the import reconciles (`import.test.js`); freeze, reconciliation and review
(`external.test.js`); the import reconciles (`import.test.js`); freeze, reconciliation and review
(`operations.test.js`); the cutover sequence (`cutover.test.js`); the staff console and its staff map
(`operations.test.js`); the cutover sequence (`cutover.test.js`); the staff console and its staff map
(`console.test.js`, `staff-map.test.js`); the public policy (`policy.test.js`); the vhost keeps `/api/v1`
(`console.test.js`, `staff-map.test.js`); the public policy (`policy.test.js`); the vhost keeps `/api/v1`
@@ -16,6 +16,7 @@
@@ -16,6 +16,7 @@
"provider receipts stored before processing, settled once (PowerChat, Stripe, PayPal, CCBill, NOWPayments adapters)",
"provider receipts stored before processing, settled once (PowerChat, Stripe, PayPal, CCBill, NOWPayments adapters)",
"intents, transfers, refunds, recycle, cashouts with escrow",
"intents, transfers, refunds, recycle, cashouts with escrow",
"subscriptions and entitlements",
"subscriptions and entitlements",
+ "renewal grace: a failed credit renewal is past_due and retried on its key until grace_until (BILLING_RENEWAL_GRACE_DAYS, off by default)",
"EXTERNAL PowerChat receipts (billing.receipt.external under BILLING_AUTHORITY=billing)",
"EXTERNAL PowerChat receipts (billing.receipt.external under BILLING_AUTHORITY=billing)",
"economy freeze",
"economy freeze",
"hourly reconciliation, ledger against receipts",
"hourly reconciliation, ledger against receipts",
new file mode 100644
new file mode 100644
@@ -0,0 +1,117 @@
@@ -0,0 +1,117 @@
+# Cutover runbook — renewal grace (OpenVibe.Billing, plan T5 step 11, s2)
+
+A credit renewal the subscriber cannot pay used to end the subscription at once (`expired`, reason
+`renewal_insufficient_credit`). With this change it can instead move to **`past_due`** for
+`BILLING_RENEWAL_GRACE_DAYS` past the period end: the hourly sweep retries the same charge under the **same
+idempotency key** until it is paid (`active` again, reason `renewed`, one charge, one creator share) or
+`grace_until` passes (`expired`, reason `grace_ended`). **The grace ships off** (`BILLING_RENEWAL_GRACE_DAYS`
+defaults to `0`), so the deploy changes no behavior: no row becomes `past_due` until the grace is turned on.
+
+Also in this change, whatever the grace: the renewal key is `renew:<sub>:<period end>` as before, and becomes
+`renew:<sub>:<period end>:<k>` only after `k` charges under that period's earlier keys were refunded or charged back
+(a reversal that brings `current_period_end` back to a period already renewed once no longer replays the reversed
+charge as a "renewal" with no money and no period). Existing keys and their replays are unchanged.
+
+No new route, capability or provider change. Events: `billing.entitlement.changed` gains reasons
+`renewal_failed` and `grace_ended`, `subscription.status` `past_due`, and top-level `grace_until` /
+`renewal_period_end` — all in `openvibe-contracts` v0.94.2 (Contracts PR #31). `GET /api/v1/entitlements/…` and
+the subscription reads carry `grace_until` (and `renewal_failed_at`, `renewal_attempts` on the subscription).
+
+## What the migration does
+
+`migrations/0005_renewal_grace.sql` (`phase: expand`), in one transaction — additive and idempotent:
+
+```sql
+ALTER TABLE subscriptions DROP CONSTRAINT IF EXISTS subscriptions_status_check;
+ALTER TABLE subscriptions ADD CONSTRAINT subscriptions_status_check CHECK (status IN ('active', 'past_due', 'canceled', 'expired'));
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS grace_until text COLLATE "C";
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_failed_at text COLLATE "C";
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_attempts bigint NOT NULL DEFAULT 0;
+CREATE INDEX IF NOT EXISTS subscriptions_due_idx ON subscriptions (status, current_period_end);
+```
+
+The CHECK only widens (every existing row passes); the columns are nullable or defaulted, so the old code runs
+against the new schema unchanged.
+
+## Backup
+
+Before the merge, take the ledger snapshot any data change requires (PR #6's rule):
+
+```sh
+sudo ovhost backup --all --logical
+```
+
+Note the stamp: the logical `pg_dump` lands at
+`/var/backups/openvibe/billing/<stamp>/billing.dump` (`root:root`, 0600). Keep it until the deploy is verified;
+it is the way back if the restore ever has to be used. Confirm it exists and is non-empty before deploying:
+
+```sh
+sudo ls -l /var/backups/openvibe/billing/<stamp>/billing.dump
+```
+
+## Rehearsal
+
+The harness records `ov rehearse OpenVibe.Billing 12` itself. `seed: none` keeps the base empty of fixtures
+(0005 needs no rows — it only widens a CHECK and adds columns), so the rehearsal builds main's migrations,
+applies 0005, runs the migrations a second time (nothing must apply), then checks the widened CHECK, the three
+new columns and the sweep index.
+
+```rehearse
+seed: none
+node -e 'const a=require("assert"),{createDb}=require("openvibe-sdk/db");const db=createDb({url:process.env.DATABASE_DIRECT_URL,service:"rehearsal",max:1});(async()=>{const c=await db.prepare("SELECT pg_get_constraintdef(oid) AS d FROM pg_constraint WHERE conrelid = ?::regclass AND conname = ?").get("subscriptions","subscriptions_status_check");for(const k of ["active","past_due","canceled","expired"])a.ok(c.d.includes(k),k);const cols=await db.prepare("SELECT column_name AS n, is_nullable AS nullable, column_default AS def FROM information_schema.columns WHERE table_name = ?").all("subscriptions");const by=Object.fromEntries(cols.map((r)=>[r.n,r]));a.strictEqual(by.grace_until.nullable,"YES");a.strictEqual(by.renewal_failed_at.nullable,"YES");a.strictEqual(by.renewal_attempts.nullable,"NO");a.ok(String(by.renewal_attempts.def).includes("0"),by.renewal_attempts.def);const i=await db.prepare("SELECT indexname AS n FROM pg_indexes WHERE tablename = ? AND indexname = ?").get("subscriptions","subscriptions_due_idx");a.ok(i&&i.n,"subscriptions_due_idx");console.log("0005: subscriptions_status_check widened to past_due, grace_until/renewal_failed_at/renewal_attempts added, subscriptions_due_idx present")})().finally(()=>db.close()).catch((e)=>{console.error(e);process.exitCode=1})'
+```
+
+By hand on the restored snapshot (a scratch role, never production), after
+`pg_restore` into a throwaway database and running the migrations with `openDb` as in PR #6's
+runbook:
+
+```sql
+SELECT pg_get_constraintdef(oid) FROM pg_constraint
+ WHERE conrelid = 'subscriptions'::regclass AND conname = 'subscriptions_status_check';
+SELECT id, name, phase FROM ov_migrations WHERE id = '0005'; -- 0005 | renewal_grace | expand
+SELECT COUNT(*) FROM subscriptions WHERE status = 'past_due'; -- 0 (the grace is off)
+```
+
+and compare `node scripts/reconcile.js` before and after: the same account, transaction and entry counts, every
+check ok.
+
+## Verification
+
+After the deploy, on production (read-only):
+
+1. `/api/ready` returns 200 with `db: "ok"`; the boot log has the line
+ `migrations: applied 0005_renewal_grace (expand, NNms)`; `ov_migrations` has the `0005` row.
+2. `\d subscriptions` lists `grace_until`, `renewal_failed_at` (nullable text) and `renewal_attempts`
+ (`bigint NOT NULL DEFAULT 0`), and `subscriptions_due_idx`.
+3. `SELECT count(*) FROM subscriptions WHERE status = 'past_due'` is **0**: the grace is still off
+ (`BILLING_RENEWAL_GRACE_DAYS` unset or `0` in `/etc/openvibe/billing.env`), so the deploy changed no
+ behavior. An unpaid renewal still ends at once (`expired`, reason `renewal_insufficient_credit`).
+4. `node scripts/reconcile.js` shows the same totals as before the deploy, every check ok.
+
+## Order
+
+1. **Contracts** — v0.94.2 (`past_due`, `renewal_failed`, `grace_ended`, `grace_until`) is released (s1).
+2. **Backup** — the snapshot above, and confirm it.
+3. **Billing, grace 0** — merge and deploy this change through its pipeline. Leave `BILLING_RENEWAL_GRACE_DAYS`
+ unset (= 0). Verify with the checks above. Renewals behave as before.
+4. **VIP consumes `past_due`** — VIP (plan s4) treats `subscription.status: past_due` as not entitled, reads
+ `grace_until` for its "renew your membership" notice and records each period's charge. Do not go further until
+ VIP with that change is deployed.
+5. **Turn the grace on** — plan s5 sets the default to 3; on a host before that, set
+ `BILLING_RENEWAL_GRACE_DAYS=3` in `/etc/openvibe/billing.env` and restart through the deploy pipeline. Check
+ after the next sweep: the log line `subscription sweep: … past due …` and, for a failed renewal,
+ `SELECT id, status, grace_until, renewal_attempts FROM subscriptions WHERE status = 'past_due'`.
+
+## Rollback
+
+- **Turn the grace off:** set `BILLING_RENEWAL_GRACE_DAYS=0` (or unset it) and restart. The next sweep retries each
+ `past_due` subscription once more on its key — a subscriber who topped up renews, with one charge — and ends
+ the rest (`expired`, reason `grace_ended`). Nothing else changes. This is the way back for behavior; no
+ schema restore is needed.
+- **Roll the code back:** turn the grace off first and wait for one sweep, so no row is `past_due` (the old code
+ neither renews nor ends a `past_due` row). Then deploy the previous release (`sudo ovhost rollback billing`);
+ **the migration stays**: never drop the columns or narrow the CHECK (old code ignores them).
+- **Restore the snapshot (last resort):** stop the unit, `pg_restore` the `billing.dump` from the stamp above
+ over `ov_billing`, then start the previous release. Only if the schema or data is actually wrong; the migrate is
+ additive, so rollback without restore is the normal path.
+- A renewal charged under a `…:<k>` key is an ordinary `subscription` transaction; rolling back does not touch it.
new file mode 100644
new file mode 100644
@@ -0,0 +1,20 @@
@@ -0,0 +1,20 @@
+-- phase: expand
+-- Renewal grace (plan T5 step 11, s2): a credit renewal the subscriber cannot pay moves the subscription to `past_due`
+-- until grace_until, and the renewal sweep retries the charge under the SAME idempotency key until it is paid
+-- (active again) or grace_until passes (expired, reason grace_ended). Off by default: BILLING_RENEWAL_GRACE_DAYS=0
+-- keeps the old behavior (expire at once, reason renewal_insufficient_credit), so no row becomes past_due until the
+-- grace is turned on. Additive and idempotent: the one ALTER of an existing CHECK widens subscriptions_status_check
+-- (the name PostgreSQL gave 0001's inline CHECK on subscriptions.status) by 'past_due'; the rest are new columns, two
+-- nullable and one with a default. Rollback is BILLING_RENEWAL_GRACE_DAYS=0 (the sweep expires the past_due rows);
+-- the columns and the widened CHECK stay (docs/renewal-grace-cutover.md).
+
+ALTER TABLE subscriptions DROP CONSTRAINT IF EXISTS subscriptions_status_check;
+ALTER TABLE subscriptions ADD CONSTRAINT subscriptions_status_check CHECK (status IN ('active', 'past_due', 'canceled', 'expired'));
+
+-- past_due only: when the sweep stops retrying and ends the subscription (the failed period's end + the grace days).
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS grace_until text COLLATE "C";
+-- The last failed renewal charge, and how many failed for the current period (both reset by a paid period).
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_failed_at text COLLATE "C";
+ALTER TABLE subscriptions ADD COLUMN IF NOT EXISTS renewal_attempts bigint NOT NULL DEFAULT 0;
+
+CREATE INDEX IF NOT EXISTS subscriptions_due_idx ON subscriptions (status, current_period_end);
@@ -111,6 +111,9 @@ function loadConfig(env = process.env) {
@@ -111,6 +111,9 @@ function loadConfig(env = process.env) {
minCashoutBits: int(env.BILLING_MIN_CASHOUT_BITS, 500),
minCashoutBits: int(env.BILLING_MIN_CASHOUT_BITS, 500),
escrowDays: num(env.BILLING_ESCROW_DAYS, 14),
escrowDays: num(env.BILLING_ESCROW_DAYS, 14),
stripeGraceDays: num(env.BILLING_STRIPE_GRACE_DAYS, 3),
stripeGraceDays: num(env.BILLING_STRIPE_GRACE_DAYS, 3),
+ // A failed credit renewal stays past_due (retried by the sweep) this many days past the period end;
+ // 0 = end it at once (renewal_insufficient_credit).
+ renewalGraceDays: Math.max(0, num(env.BILLING_RENEWAL_GRACE_DAYS, 0)),
},
},
// Provider adapters: each is enabled only when its secrets are set.
// Provider adapters: each is enabled only when its secrets are set.
@@ -35,7 +35,7 @@ const { runScheduled } = require('./reconcile');
@@ -35,7 +35,7 @@ const { runScheduled } = require('./reconcile');
every(config.jobs.sweepIntervalMs, async () => {
every(config.jobs.sweepIntervalMs, async () => {
if (await isFrozen(ctx.db)) return;
if (await isFrozen(ctx.db)) return;
const out = await subscriptions.sweep(ctx);
const out = await subscriptions.sweep(ctx);
- if (out.renewed.length || out.expired.length || out.canceled.length) console.log(`[Billing] subscription sweep: ${out.renewed.length} renewed, ${out.expired.length} expired, ${out.canceled.length} canceled`);
+ if (out.renewed.length || out.expired.length || out.canceled.length || out.past_due.length) console.log(`[Billing] subscription sweep: ${out.renewed.length} renewed, ${out.past_due.length} past due, ${out.expired.length} expired, ${out.canceled.length} canceled`);
});
});
if (config.jobs.reconcileIntervalMs > 0) {
if (config.jobs.reconcileIntervalMs > 0) {
const reconcileNow = async () => await runScheduled(ctx, { keepDays: config.jobs.reconcileKeepDays });
const reconcileNow = async () => await runScheduled(ctx, { keepDays: config.jobs.reconcileKeepDays });
@@ -15,6 +15,11 @@
@@ -15,6 +15,11 @@
*
*
* Entitlement truth is the entitlements table: one row per granted period; a subject is
* Entitlement truth is the entitlements table: one row per granted period; a subject is
* entitled to a streamer while a non-revoked row covers now. It needs nothing from Live.
* entitled to a streamer while a non-revoked row covers now. It needs nothing from Live.
+ *
+ * A credit renewal the subscriber cannot pay leaves the subscription past_due for
+ * BILLING_RENEWAL_GRACE_DAYS (0 = end it at once): no period covers the grace, so the entitlement
+ * is not active, and the sweep retries the same charge under the same key until it is paid or
+ * grace_until passes.
*/
*/
const { post, getTxn, requireFunds, iso, prefixedId, money } = require('../ledger');
const { post, getTxn, requireFunds, iso, prefixedId, money } = require('../ledger');
const { enqueue } = require('../outbox');
const { enqueue } = require('../outbox');
@@ -24,7 +29,7 @@ const intents = require('./intents');
@@ -24,7 +29,7 @@ const intents = require('./intents');
const KIND = 'channel_subscription';
const KIND = 'channel_subscription';
-function parse(row) { return row ? { ...row, auto_renew: !!row.auto_renew, cancel_at_period_end: !!row.cancel_at_period_end } : null; }
+function parse(row) { return row ? { ...row, auto_renew: !!row.auto_renew, cancel_at_period_end: !!row.cancel_at_period_end, renewal_attempts: Number(row.renewal_attempts) || 0 } : null; }
async function find(db, id) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE id = ?').get(id)); }
async function find(db, id) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE id = ?').get(id)); }
async function findPair(db, subscriber, streamer) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE subscriber = ? AND streamer = ?').get(subscriber, streamer)); }
async function findPair(db, subscriber, streamer) { return parse(await db.prepare('SELECT * FROM subscriptions WHERE subscriber = ? AND streamer = ?').get(subscriber, streamer)); }
async function findByProviderRef(db, provider, ref) { return ref ? parse(await db.prepare('SELECT * FROM subscriptions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) : null; }
async function findByProviderRef(db, provider, ref) { return ref ? parse(await db.prepare('SELECT * FROM subscriptions WHERE provider = ? AND provider_ref = ?').get(provider, ref)) : null; }
@@ -35,7 +40,8 @@ function present(s) {
@@ -35,7 +40,8 @@ function present(s) {
id: s.id, subscriber: { type: 'user', id: s.subscriber }, streamer: { type: 'user', id: s.streamer }, tier: s.tier,
id: s.id, subscriber: { type: 'user', id: s.subscriber }, streamer: { type: 'user', id: s.streamer }, tier: s.tier,
provider: s.provider, provider_ref: s.provider_ref || null, route: s.route || null, status: s.status,
provider: s.provider, provider_ref: s.provider_ref || null, route: s.route || null, status: s.status,
auto_renew: !!s.auto_renew, cancel_at_period_end: !!s.cancel_at_period_end, price_cents: s.price_cents,
auto_renew: !!s.auto_renew, cancel_at_period_end: !!s.cancel_at_period_end, price_cents: s.price_cents,
- current_period_end: s.current_period_end, created_at: s.created_at, updated_at: s.updated_at,
+ current_period_end: s.current_period_end, grace_until: s.grace_until || null, renewal_failed_at: s.renewal_failed_at || null,
+ renewal_attempts: Number(s.renewal_attempts) || 0, created_at: s.created_at, updated_at: s.updated_at,
};
};
}
}
@@ -58,6 +64,8 @@ async function entitlement(db, subject, streamer, ms) {
@@ -58,6 +64,8 @@ async function entitlement(db, subject, streamer, ms) {
return {
return {
subject: { type: 'user', id: subject }, streamer: { type: 'user', id: streamer }, kind: KIND,
subject: { type: 'user', id: subject }, streamer: { type: 'user', id: streamer }, kind: KIND,
active: !!expires, expires_at: expires || null,
active: !!expires, expires_at: expires || null,
+ // past_due: when the sweep stops retrying the renewal. The grace grants no access (active stays false).
+ grace_until: sub && sub.status === 'past_due' ? sub.grace_until || null : null,
subscription: sub ? { id: sub.id, status: sub.status, auto_renew: sub.auto_renew, cancel_at_period_end: sub.cancel_at_period_end, provider: sub.provider } : null,
subscription: sub ? { id: sub.id, status: sub.status, auto_renew: sub.auto_renew, cancel_at_period_end: sub.cancel_at_period_end, provider: sub.provider } : null,
};
};
}
}
@@ -122,7 +130,7 @@ async function pay(ctx, input) {
@@ -122,7 +130,7 @@ async function pay(ctx, input) {
const running = prior && prior.current_period_end && Date.parse(prior.current_period_end) > ms ? Date.parse(prior.current_period_end) : ms;
const running = prior && prior.current_period_end && Date.parse(prior.current_period_end) > ms ? Date.parse(prior.current_period_end) : ms;
const start = running;
const start = running;
const end = r && r.periodEnd ? Math.max(Date.parse(r.periodEnd), start + 1) : start + rates.subPeriodDays * 86_400_000;
const end = r && r.periodEnd ? Math.max(Date.parse(r.periodEnd), start + 1) : start + rates.subPeriodDays * 86_400_000;
- const renewal = !!(prior && (prior.status === 'active' || input.renewal));
+ const renewal = !!(prior && (prior.status === 'active' || prior.status === 'past_due' || input.renewal));
const { txn } = await post(ctx, {
const { txn } = await post(ctx, {
type: 'subscription',
type: 'subscription',
@@ -143,7 +151,8 @@ async function pay(ctx, input) {
@@ -143,7 +151,8 @@ async function pay(ctx, input) {
let sub;
let sub;
if (prior) {
if (prior) {
await db.prepare(`UPDATE subscriptions SET provider = ?, provider_ref = COALESCE(?, provider_ref), route = ?, status = 'active', auto_renew = ?,
await db.prepare(`UPDATE subscriptions SET provider = ?, provider_ref = COALESCE(?, provider_ref), route = ?, status = 'active', auto_renew = ?,
- cancel_at_period_end = 0, price_cents = ?, current_period_end = ?, updated_at = ? WHERE id = ?`)
+ cancel_at_period_end = 0, price_cents = ?, current_period_end = ?, grace_until = NULL, renewal_failed_at = NULL, renewal_attempts = 0,
+ updated_at = ? WHERE id = ?`)
.run(provider, r ? r.providerRef || null : null, route, autoRenew, priceCents, iso(end), nowIso, prior.id);
.run(provider, r ? r.providerRef || null : null, route, autoRenew, priceCents, iso(end), nowIso, prior.id);
sub = await find(db, prior.id);
sub = await find(db, prior.id);
} else {
} else {
@@ -180,7 +189,8 @@ async function cancel(ctx, { id, actor }, adapters) {
@@ -180,7 +189,8 @@ async function cancel(ctx, { id, actor }, adapters) {
const { db } = ctx;
const { db } = ctx;
const sub = await find(db, id);
const sub = await find(db, id);
if (!sub) fail(404, 'billing.subscription_not_found', `no subscription ${id}`);
if (!sub) fail(404, 'billing.subscription_not_found', `no subscription ${id}`);
- if (sub.status !== 'active') return { subscription: sub, provider_sync: 'not_needed', replay: true };
+ // past_due: cancelling stops the retries; the next sweep ends it.
+ if (sub.status !== 'active' && sub.status !== 'past_due') return { subscription: sub, provider_sync: 'not_needed', replay: true };
let providerSync = 'not_needed';
let providerSync = 'not_needed';
if (sub.provider === 'stripe') {
if (sub.provider === 'stripe') {
const stripe = adapters && adapters.stripe;
const stripe = adapters && adapters.stripe;
@@ -200,19 +210,62 @@ async function cancel(ctx, { id, actor }, adapters) {
@@ -200,19 +210,62 @@ async function cancel(ctx, { id, actor }, adapters) {
return { subscription: await find(db, id), provider_sync: providerSync, replay: false };
return { subscription: await find(db, id), provider_sync: providerSync, replay: false };
}
}
-async function setStatus(ctx, id, status, reason) {
+/**
+ * End (or reactivate) a subscription and say so. `extra` joins the event payload. With `periodEnd`, the change applies
+ * only while the row still has that period end (a renewal paid meanwhile wins).
+ */
+async function setStatus(ctx, id, status, reason, { extra, periodEnd } = {}) {
const { db } = ctx;
const { db } = ctx;
const sub = await find(db, id);
const sub = await find(db, id);
if (!sub || sub.status === status) return sub;
if (!sub || sub.status === status) return sub;
- await db.prepare('UPDATE subscriptions SET status = ?, auto_renew = CASE WHEN ? = \'active\' THEN auto_renew ELSE 0 END, updated_at = ? WHERE id = ?')
- .run(status, status, iso(ctx.now()), id);
+ const r = await db.prepare(`UPDATE subscriptions SET status = ?, auto_renew = CASE WHEN ? = 'active' THEN auto_renew ELSE 0 END, grace_until = NULL, updated_at = ?
+ WHERE id = ? AND status = ?${periodEnd ? ' AND current_period_end = ?' : ''}`)
+ .run(status, status, iso(ctx.now()), id, sub.status, ...(periodEnd ? [periodEnd] : []));
+ if (!r.changes) return await find(db, id);
await enqueue(ctx, {
await enqueue(ctx, {
event_type: 'billing.entitlement.changed', subject: { type: 'entitlement', id: `${sub.subscriber}:${KIND}:${sub.streamer}` },
event_type: 'billing.entitlement.changed', subject: { type: 'entitlement', id: `${sub.subscriber}:${KIND}:${sub.streamer}` },
- payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: reason || status },
+ payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: reason || status, ...(extra || {}) },
});
});
return await find(db, id);
return await find(db, id);
}
}
+/**
+ * A renewal charge failed: past_due until `graceUntil` (kept from the first failure of the period). The event goes out
+ * once, when the subscription turns past_due; a failed retry only counts the attempt. No-op if the row changed since
+ * the sweep read it (a renewal paid, the subscription ended, or a concurrent sweep marked it first).
+ */
+async function markPastDue(ctx, sub, graceUntil) {
+ const { db } = ctx;
+ return await money(db, async () => {
+ const at = iso(ctx.now());
+ const r = await db.prepare(`UPDATE subscriptions SET status = 'past_due', grace_until = ?, renewal_failed_at = ?, renewal_attempts = renewal_attempts + 1, updated_at = ?
+ WHERE id = ? AND status = ? AND current_period_end = ?`).run(graceUntil, at, at, sub.id, sub.status, sub.current_period_end);
+ if (!r.changes || sub.status === 'past_due') return !!r.changes;
+ await enqueue(ctx, {
+ event_type: 'billing.entitlement.changed', subject: { type: 'entitlement', id: `${sub.subscriber}:${KIND}:${sub.streamer}` },
+ payload: { ...await entitlement(db, sub.subscriber, sub.streamer, ctx.now()), reason: 'renewal_failed', grace_until: graceUntil, renewal_period_end: sub.current_period_end },
+ });
+ return true;
+ });
+}
+
+/**
+ * The renewal key of the period that ends at the subscription's current_period_end: renew:<sub>:<period end>, then
+ * renew:<sub>:<period end>:<k> once k charges under the earlier keys were refunded or charged back (a reversal can
+ * bring current_period_end back to an end already renewed once; replaying that key would "renew" with no charge and
+ * no period). An unreversed charge under a key is that renewal: the key is returned and pay() replays it.
+ */
+async function renewalKey(db, sub) {
+ const base = `renew:${sub.id}:${sub.current_period_end}`;
+ for (let k = 0; ; k++) {
+ const key = k ? `${base}:${k}` : base;
+ const txn = await db.prepare('SELECT id FROM transactions WHERE idempotency_key = ?').get(key);
+ if (!txn) return key;
+ const reversed = await db.prepare("SELECT 1 AS x FROM transactions WHERE reverses_txn = ? AND type IN ('refund', 'chargeback') LIMIT 1").get(txn.id);
+ if (!reversed) return key;
+ }
+}
+
/** Revoke the periods a reversed payment granted. Returns the number of periods revoked. */
/** Revoke the periods a reversed payment granted. Returns the number of periods revoked. */
async function revokeForTxn(ctx, txnId, reason) {
async function revokeForTxn(ctx, txnId, reason) {
const { db } = ctx;
const { db } = ctx;
@@ -237,18 +290,22 @@ async function revokeForTxn(ctx, txnId, reason) {
@@ -237,18 +290,22 @@ async function revokeForTxn(ctx, txnId, reason) {
}
}
/**
/**
- * Renewal sweep (hourly job). For every active subscription whose period ended:
+ * Renewal sweep (hourly job). For every active or past_due subscription whose period ended:
* - Stripe: Stripe renews itself (invoice.paid); expire only after the grace window;
* - Stripe: Stripe renews itself (invoice.paid); expire only after the grace window;
* - cancel-at-period-end / no auto-renew: end it;
* - cancel-at-period-end / no auto-renew: end it;
- * - otherwise renew from the subscriber's credit (deterministic key per period), or end it
- * when the credit does not cover the price.
+ * - past_due past grace_until: end it (grace_ended);
+ * - otherwise renew from the subscriber's credit (deterministic key per period, renewalKey), or,
+ * when the credit does not cover the price, past_due until the period end + the renewal grace
+ * (retried on the same key by every later sweep), or end it at once without a grace.
+ * Canceled and expired subscriptions are never charged.
*/
*/
async function sweep(ctx, { limit = 200 } = {}) {
async function sweep(ctx, { limit = 200 } = {}) {
const { db, rates } = ctx;
const { db, rates } = ctx;
- const out = { renewed: [], expired: [], canceled: [], skipped: 0 };
+ const out = { renewed: [], expired: [], canceled: [], past_due: [], skipped: 0 };
const now = ctx.now();
const now = ctx.now();
- const due = (await db.prepare("SELECT * FROM subscriptions WHERE status = 'active' AND current_period_end IS NOT NULL AND current_period_end <= ? ORDER BY current_period_end LIMIT ?")
- .all(iso(now), limit)).map(parse);
+ const graceMs = Math.max(0, Number(rates.renewalGraceDays) || 0) * 86_400_000;
+ const due = (await db.prepare(`SELECT * FROM subscriptions WHERE status IN ('active', 'past_due') AND current_period_end IS NOT NULL AND current_period_end <= ?
+ ORDER BY current_period_end LIMIT ?`).all(iso(now), limit)).map(parse);
for (const sub of due) {
for (const sub of due) {
const end = Date.parse(sub.current_period_end);
const end = Date.parse(sub.current_period_end);
if (sub.provider === 'stripe') {
if (sub.provider === 'stripe') {
@@ -259,16 +316,26 @@ async function sweep(ctx, { limit = 200 } = {}) {
@@ -259,16 +316,26 @@ async function sweep(ctx, { limit = 200 } = {}) {
const st = sub.cancel_at_period_end ? 'canceled' : 'expired';
const st = sub.cancel_at_period_end ? 'canceled' : 'expired';
await setStatus(ctx, sub.id, st, st); (st === 'canceled' ? out.canceled : out.expired).push(sub.id); continue;
await setStatus(ctx, sub.id, st, st); (st === 'canceled' ? out.canceled : out.expired).push(sub.id); continue;
}
}
+ const ended = { periodEnd: sub.current_period_end, extra: { renewal_period_end: sub.current_period_end } };
+ if (sub.status === 'past_due' && (!sub.grace_until || now >= Date.parse(sub.grace_until))) {
+ await setStatus(ctx, sub.id, 'expired', 'grace_ended', ended); out.expired.push(sub.id); continue;
+ }
try {
try {
- await pay(ctx, {
+ const paid = await pay(ctx, {
subscriber: sub.subscriber, streamer: sub.streamer, source: 'credit', priceCents: sub.price_cents || rates.subPriceCents,
subscriber: sub.subscriber, streamer: sub.streamer, source: 'credit', priceCents: sub.price_cents || rates.subPriceCents,
- autoRenew: true, renewal: true, idempotencyKey: `renew:${sub.id}:${sub.current_period_end}`,
+ autoRenew: true, renewal: true, idempotencyKey: await renewalKey(db, sub),
actor: { principal: 'svc:billing', job: 'renewal-sweep' },
actor: { principal: 'svc:billing', job: 'renewal-sweep' },
});
});
- out.renewed.push(sub.id);
+ // A replay means a concurrent sweep already renewed this period: it reports it, this one skips it.
+ if (paid.replay) out.skipped++; else out.renewed.push(sub.id);
} catch (e) {
} catch (e) {
if (e.code !== 'billing.insufficient_funds') throw e;
if (e.code !== 'billing.insufficient_funds') throw e;
- await setStatus(ctx, sub.id, 'expired', 'renewal_insufficient_credit'); out.expired.push(sub.id);
+ const graceUntil = sub.grace_until || iso(end + graceMs);
+ if (graceMs > 0 && now < Date.parse(graceUntil)) {
+ if (await markPastDue(ctx, sub, graceUntil)) out.past_due.push(sub.id);
+ continue;
+ }
+ await setStatus(ctx, sub.id, 'expired', sub.status === 'past_due' ? 'grace_ended' : 'renewal_insufficient_credit', ended); out.expired.push(sub.id);
}
}
}
}
return out;
return out;
@@ -283,4 +350,4 @@ async function list(db, { subscriber, streamer, status } = {}) {
@@ -283,4 +350,4 @@ async function list(db, { subscriber, streamer, status } = {}) {
return (await db.prepare(`SELECT * FROM subscriptions ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY created_at DESC LIMIT 500`).all(...args)).map(parse);
return (await db.prepare(`SELECT * FROM subscriptions ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY created_at DESC LIMIT 500`).all(...args)).map(parse);
}
}
-module.exports = { pay, cancel, sweep, entitlement, activeEntitlements, revokeForTxn, setStatus, find, findPair, findByProviderRef, list, present, KIND };
+module.exports = { pay, cancel, sweep, renewalKey, entitlement, activeEntitlements, revokeForTxn, setStatus, find, findPair, findByProviderRef, list, present, KIND };
new file mode 100644
new file mode 100644
@@ -0,0 +1,185 @@
@@ -0,0 +1,185 @@
+'use strict';
+/**
+ * Renewal grace (plan T5 step 11, s2; money path): a credit renewal is one charge and one creator share per period
+ * however often or concurrently the sweep runs; a renewal the subscriber cannot pay leaves the subscription past_due
+ * (no access) and is retried under the SAME key until it is paid or the grace ends; a promo allowance never pays a
+ * renewal; BILLING_RENEWAL_GRACE_DAYS=0 ends it at once; and a renewal reversed by a refund renews under a fresh key.
+ */
+const assert = require('assert');
+const { boot, fund, check, done } = require('./helpers/app');
+
+const DAY = 86_400_000;
+
+(async () => {
+ const t = await boot({ env: { BILLING_RENEWAL_GRACE_DAYS: '3' } });
+ const subs = require('../server/ops/subscriptions');
+ const { post, money } = require('../server/ledger');
+ console.log('renewal grace');
+
+ const streamer = t.user(60);
+ const sweep = async () => {
+ const r = await t.call('POST', '/api/v1/admin/sweep', { key: null });
+ assert.strictEqual(r.status, 200, r.text);
+ return r.json;
+ };
+ const subscribe = async (u, bits) => {
+ if (bits) await fund(t, u, bits);
+ const r = await t.call('POST', '/api/v1/subscriptions', { body: { subscriber: u, streamer, source: 'credit' } });
+ assert.strictEqual(r.status, 201, r.text);
+ return r.json.subscription;
+ };
+ const row = async (id) => await subs.find(t.db, id);
+ const renewals = async (id) => (await t.db.prepare("SELECT idempotency_key FROM transactions WHERE type = 'subscription' AND idempotency_key LIKE ? ORDER BY created_at").all(`renew:${id}:%`)).map((r) => r.idempotency_key);
+ const changed = async (u) => (await t.db.prepare('SELECT event FROM outbox ORDER BY seq').all()).map((r) => JSON.parse(r.event))
+ .filter((e) => e.event_type === 'billing.entitlement.changed' && e.subject.id.startsWith(`${u.id}:`)).map((e) => e.payload);
+ const ent = async (u) => (await t.call('GET', `/api/v1/entitlements/${u.id}?streamer=${streamer.id}`)).json;
+ const payable = async () => (await t.balances(streamer.id)).payable;
+
+ const [a, b, c, d, e] = [t.user(61), t.user(62), t.user(63), t.user(64), t.user(65)];
+ const subA = await subscribe(a, 998); // can pay one renewal
+ const subB = await subscribe(b, 499); // fails, tops up during the grace
+ const subC = await subscribe(c, 499); // fails, never pays
+ const subD = await subscribe(d, 499); // fails, cancels during the grace
+ const subE = await subscribe(e, 499); // promo allowance only
+ const P = subB.current_period_end;
+ const graceUntil = (s) => new Date(Date.parse(s.current_period_end) + 3 * DAY).toISOString();
+ let share = 5 * 349;
+
+ await check('a promo allowance stays out of renewals', async () => {
+ t.clock.offset = 32 * DAY;
+ const g = await t.call('POST', '/api/v1/admin/promo/grant', { body: { subject: e, bits: 1000, service: 'openvibe.tools' }, cap: ['billing.ledger.admin'] });
+ assert.strictEqual(g.status, 201, g.text);
+ assert.strictEqual(await payable(), share);
+ });
+
+ await check('concurrent sweeps renew a paid-up subscription once: one charge, one creator share', async () => {
+ const [s1, s2] = await Promise.all([sweep(), sweep()]);
+ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subA.id]);
+ assert.deepStrictEqual(await renewals(subA.id), [`renew:${subA.id}:${subA.current_period_end}`]);
+ share += 349;
+ assert.strictEqual(await payable(), share);
+ assert.strictEqual((await t.balances(a.id)).credit, 0);
+ const replay = await subs.pay(t.ctx, { subscriber: a, streamer, source: 'credit', renewal: true, idempotencyKey: `renew:${subA.id}:${subA.current_period_end}`, actor: { principal: 'test' } });
+ assert.strictEqual(replay.replay, true);
+ assert.strictEqual(await payable(), share, 'a replayed renewal credits no second share');
+ assert.deepStrictEqual((await sweep()).renewed, [], 'the renewed period is not due again');
+ });
+
+ await check('an unpaid renewal turns past_due with grace_until, grants no access and changes no money', async () => {
+ for (const s of [subB, subC, subD, subE]) {
+ const r = await row(s.id);
+ assert.strictEqual(r.status, 'past_due', s.id);
+ assert.strictEqual(r.grace_until, graceUntil(s));
+ assert.ok(r.renewal_attempts >= 1); // the sweeps above retried it once or twice
+ assert.strictEqual(r.auto_renew, true);
+ assert.deepStrictEqual(await renewals(s.id), []);
+ }
+ const ev = (await changed(b)).filter((p) => p.reason === 'renewal_failed');
+ assert.strictEqual(ev.length, 1, 'one renewal_failed event despite two concurrent sweeps');
+ assert.deepStrictEqual([ev[0].active, ev[0].subscription.status, ev[0].grace_until, ev[0].renewal_period_end], [false, 'past_due', graceUntil(subB), P]);
+ const en = await ent(b);
+ assert.deepStrictEqual([en.active, en.subscription.status, en.grace_until], [false, 'past_due', graceUntil(subB)]);
+ const got = (await t.call('GET', `/api/v1/subscriptions/${subB.id}`)).json;
+ assert.strictEqual(got.subscription.grace_until, graceUntil(subB));
+ assert.strictEqual(await payable(), share);
+ const be = await t.balances(e.id);
+ assert.deepStrictEqual([be.credit, be.promo_bits], [0, 1000], 'the promo allowance paid nothing');
+ await t.assertReconciled('after failed renewals');
+ });
+
+ await check('each sweep in the grace retries the same charge; a retry that fails again emits nothing', async () => {
+ t.clock.offset = 33 * DAY;
+ const before = (await changed(b)).length;
+ const attempts = (await row(subB.id)).renewal_attempts;
+ const s = await sweep();
+ assert.deepStrictEqual(s.renewed, []);
+ assert.strictEqual((await row(subB.id)).renewal_attempts, attempts + 1);
+ assert.strictEqual((await row(subB.id)).grace_until, graceUntil(subB), 'the grace is not extended by a retry');
+ assert.strictEqual((await changed(b)).length, before);
+ assert.strictEqual(await subs.renewalKey(t.db, await row(subB.id)), `renew:${subB.id}:${P}`);
+ });
+
+ await check('a top-up in the grace is picked up by the next sweep: active again, one charge, one share', async () => {
+ await fund(t, b, 499);
+ const c1 = await t.call('POST', `/api/v1/subscriptions/${subD.id}/cancel`, { cap: ['billing.subscription.manage'], body: {} });
+ assert.strictEqual(c1.status, 200, c1.text);
+ assert.strictEqual(c1.json.subscription.cancel_at_period_end, true);
+ t.clock.offset = 33.5 * DAY;
+ const [s1, s2] = await Promise.all([sweep(), sweep()]);
+ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subB.id]);
+ assert.deepStrictEqual(await renewals(subB.id), [`renew:${subB.id}:${P}`], 'the retried key is the period key');
+ share += 349;
+ assert.strictEqual(await payable(), share);
+ const r = await row(subB.id);
+ assert.deepStrictEqual([r.status, r.grace_until, r.renewal_failed_at, r.renewal_attempts], ['active', null, null, 0]);
+ const en = await ent(b);
+ assert.deepStrictEqual([en.active, en.grace_until], [true, null]);
+ assert.strictEqual((await changed(b)).at(-1).reason, 'renewed');
+ assert.strictEqual((await row(subD.id)).status, 'canceled', 'cancelling in the grace ends it, uncharged');
+ assert.deepStrictEqual(await renewals(subD.id), []);
+ });
+
+ await check('past grace_until the sweep ends it (grace_ended) and never charges it again', async () => {
+ t.clock.offset = 35.1 * DAY;
+ const s = await sweep();
+ assert.deepStrictEqual(s.expired.sort(), [subC.id, subE.id].sort());
+ for (const [u, sb] of [[c, subC], [e, subE]]) {
+ assert.strictEqual((await row(sb.id)).status, 'expired');
+ const last = (await changed(u)).at(-1);
+ assert.deepStrictEqual([last.reason, last.active, last.subscription.status, last.renewal_period_end], ['grace_ended', false, 'expired', sb.current_period_end]);
+ }
+ await fund(t, c, 499);
+ t.clock.offset = 36 * DAY;
+ await sweep();
+ assert.deepStrictEqual(await renewals(subC.id), [], 'an expired subscription is never charged');
+ assert.strictEqual((await t.balances(c.id)).credit, 499);
+ assert.deepStrictEqual(await renewals(subE.id), [], 'the promo allowance never paid a renewal');
+ assert.strictEqual(await payable(), share);
+ await t.assertReconciled('after the grace');
+ });
+
+ await check('with BILLING_RENEWAL_GRACE_DAYS=0 an unpaid renewal ends at once (renewal_insufficient_credit)', async () => {
+ t.ctx.rates.renewalGraceDays = 0;
+ const f = t.user(66);
+ const subF = await subscribe(f, 499);
+ t.clock.offset = 36 * DAY + 32 * DAY;
+ await sweep();
+ const r = await row(subF.id);
+ assert.deepStrictEqual([r.status, r.grace_until, r.renewal_attempts], ['expired', null, 0]);
+ assert.strictEqual((await changed(f)).at(-1).reason, 'renewal_insufficient_credit');
+ t.ctx.rates.renewalGraceDays = 3;
+ });
+
+ await check('a renewal reversed by a refund renews under a fresh key, with a charge and a period', async () => {
+ const g = t.user(67);
+ const subG = await subscribe(g, 998);
+ const PG = subG.current_period_end;
+ t.clock.offset += 32 * DAY;
+ await sweep();
+ const [key0] = await renewals(subG.id);
+ assert.strictEqual(key0, `renew:${subG.id}:${PG}`);
+ const orig = await t.db.prepare('SELECT id FROM transactions WHERE idempotency_key = ?').get(key0);
+ // A reversal that brings current_period_end back to PG (a credit-period refund, plan s3) with the subscription renewing.
+ await money(t.db, async () => {
+ await post(t.ctx, { type: 'refund', idempotencyKey: `test-refund:${orig.id}`, reversesTxn: orig.id, entries: [], actor: { principal: 'test' }, metadata: { cents: 0 } });
+ await subs.revokeForTxn(t.ctx, orig.id, 'refund');
+ });
+ await t.db.prepare("UPDATE subscriptions SET status = 'active', auto_renew = 1 WHERE id = ?").run(subG.id);
+ assert.strictEqual((await row(subG.id)).current_period_end, PG);
+ assert.strictEqual(await subs.renewalKey(t.db, await row(subG.id)), `${key0}:1`);
+ await fund(t, g, 499);
+ const before = await payable();
+ const [s1, s2] = await Promise.all([sweep(), sweep()]);
+ assert.deepStrictEqual([...s1.renewed, ...s2.renewed], [subG.id]);
+ assert.deepStrictEqual(await renewals(subG.id), [key0, `${key0}:1`]);
+ assert.strictEqual(await payable(), before + 349, 'one charge, one share');
+ assert.strictEqual((await t.balances(g.id)).credit, 0);
+ assert.strictEqual((await ent(g)).active, true);
+ await sweep();
+ assert.strictEqual((await renewals(subG.id)).length, 2);
+ });
+
+ t.clock.offset = 0;
+ await t.close();
+ done();
+})();