openvibers/openvibe.quest Commits
Quest: account export and deletion (ADR-033) — a person's quest progress, completions and badges are exported on network.account.export_requested and deleted whole on network.account.deleted (openvibe-sdk/account-data, SDK v0.36.0), the two account topics sharing the existing Events consumer and its boot subscriptions
Changes
@@@ -121,6 +121,31 @@ idempotent by key), so the OpenCoins tests need no real wallet.- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: quest` in CI) and with**OpenVibe.Services** before the first deploy.++## Account export and deletion++++A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its++part (ADR-033). Quest receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`++(loopback only) through the consumer it already runs — the two account topics are handed to++[server/identity/account-data.js](server/identity/account-data.js), which maps the tables, and the boot-time++subscriptions for them are created (alongside the catalog's events) by++[server/events-consumer.js](server/events-consumer.js):++++- **Exported:** the quests a person is partway through (`progress.json`), the quests they completed and their OpenCoins++ (`completions.json`) and the badges they earned (`badges.json`), pushed to `POST /internal/account-exports/:id/parts`++ with this service's own token. Nothing here is a secret — Quest stores no token, key or credential.++- **Erased:** all three tables hold the person's own rows and nothing anyone else's page hangs under them, so they are++ deleted whole and nothing is kept. Deleting a completion deletes its OpenCoins record here; the wallet credit at++ OpenVibe.Network is Network's own data. Quest then confirms with `POST /internal/account-deletions/:id/confirmations`++ and the counts.++- **Anonymized:** nothing. There is no row Quest keeps that was written by this person for another person to read.++++The delivery receipts (`quest_event_inbox`) are this service's own, not a person's rows, and are neither exported nor++erased.++++Environment: `QUEST_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer++503), `QUEST_EVENTS_URL` (or `EVENTS_URL`) is where the subscriptions are created at boot (off when unset), and++`QUEST_EVENTS_ENDPOINT` overrides the loopback endpoint; `QUEST_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.++## Security (threat notes)Reporting a vulnerability: [SECURITY.md](SECURITY.md).@@@ -7,11 +7,11 @@"port": "4980","domain": "openvibe.quest","contracts": "openvibe-contracts v0.115.0",-- "sdk": "openvibe-sdk v0.35.2",++ "sdk": "openvibe-sdk v0.36.0","shared": "openvibe-shared v2.14.0","packages": ["openvibe-contracts v0.115.0",-- "openvibe-sdk v0.35.2",++ "openvibe-sdk v0.36.0","openvibe-shared v2.14.0"],"summary": "OpenVibe.Quest: one quest log across every OpenVibe site. OpenVibe.Events delivers the domain events the sites already emit to POST /internal/events (signed v2, once-only through the openvibe-sdk inbox), the person taken from the field each event's own schema defines, and a progress count, a completion and a badge move because of it. 12 first-party quests in code, grouped by the site whose events each counts. Rewards are badges and OpenCoins — credited through OpenVibe.Network's wallet, off by default (QUEST_COINS), with per-person and network-wide daily caps, failures recorded and retried. Public API and server-rendered pages, complete without JavaScript. Not deployed; OpenCoins are off until Network grants the two capabilities.",@@@ -25,14 +25,15 @@"QUEST_COINS off by default (coins_state=skipped, the pages say the rewards start soon); per-person (50) and network-wide (5000) daily caps; a failure recorded with its reason and retried by a timer","GET /api/v1/quests, /quests/:id, /me/quests (a signed-in person's own log), /profiles/:subject/badges (id, name and date only)","Pages: home (the quests grouped by site, how progress works), a quest, your quest log (plain <progress> bars), the rules, the update log; robots, sitemap with every quest page, llms.txt, llms-full.txt",-- "Tests on temp PGlite databases with a mock Network that also serves the wallet: signature refusal, idempotent replay, counting to completion, badge once, coins off, coins on with caps and a retry, events with no subject ignored, /me requires sign-in, hostile titles escaped"++ "Tests on temp PGlite databases with a mock Network that also serves the wallet: signature refusal, idempotent replay, counting to completion, badge once, coins off, coins on with caps and a retry, events with no subject ignored, /me requires sign-in, hostile titles escaped",++ "Account export and deletion (ADR-033): the quest progress, completions and badges a person has are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted, with the counts confirmed back; the two account topics join the catalog's events in the boot subscriptions (server/identity/account-data.js)"],"notYet": ["A deployment: the service is not registered with OpenVibe.Services nor deployed by OpenVibe.Host, and its capabilities are not in openvibe-contracts yet (no quest.* manifest exists)","OpenCoins are off in production until OpenVibe.Network grants the quest client identity.subject.resolve and network.coins.credit","Community-made quests (streamers, communities, mods): they need moderation, which this release does not have","Repeatable quests and seasonal resets: every quest completes once per person",-- "Account deletion and merges (ADR-033, ADR-029): quest_progress, quest_completions and quest_badges rows are not removed or re-keyed when a Network account is deleted or merged",++ "Account merges (ADR-029): quest_progress, quest_completions and quest_badges rows are not re-keyed when a Network account is merged into another (deletion is answered, ADR-033; a merge's aliases are erased with the account)","Quest chains, leaderboards and per-community quests; a streamer's getting-started steps still live on OpenVibe.Live"],"capabilitiesRegistered": false,nnew file mode 100644@@@ -0,0 +1,13 @@++-- phase: expand++-- OpenVibe.Quest: the receipts of the ADR-033 account export and deletion deliveries this service applied++-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.++-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.++++CREATE TABLE IF NOT EXISTS account_data_events (++ id TEXT PRIMARY KEY,++ kind TEXT NOT NULL,++ subject TEXT NOT NULL,++ outcome JSONB,++ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),++ sent_at TIMESTAMPTZ++);@@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');const { createPageRoutes } = require('./http/pages');const { createCoins } = require('./quests/coins');const { createEventsConsumer } = require('./events-consumer');++const accountDataLib = require('./identity/account-data');++const { createNetworkSender } = require('openvibe-sdk/account-data');const { createServiceReadiness } = require('./observability');const { createCallerLimits } = require('./http/caller-limits');const { assetVersion, send } = require('./render/layout');@@@ -40,7 +42,7 @@ const VERSION = require('../package.json').version;/*** opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,-- * tests only), valkey++ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)*/async function createApp(opts = {}) {const config = opts.config || configLib.load();@@@ -52,8 +54,17 @@ async function createApp(opts = {}) {const sso = createSso({ config, keys, fetchImpl, now: s.now, log });const principal = createPrincipal({ config, keys });const coins = createCoins({ config, s, fetchImpl, log });-- const eventsConsumer = createEventsConsumer({ db: s.db, s, secrets: config.events.secrets, coins, now: s.now, log });-- const ctx = { config, s, keys, sso, principal, coins, eventsConsumer, log };++ // Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.++ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;++ // a test injects a stand-in through opts.accountSend. Without a client secret the service has no way to push a++ // part or a confirmation: the route still answers (bad signature, a forwarded request, no secret), and an event++ // that really arrives asks Events to retry.++ const accountData = accountDataLib.create({ db: s.db, log });++ const accountSend = opts.accountSend || (config.oauth.clientSecret++ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })++ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Quest cannot answer account events'); });++ const eventsConsumer = createEventsConsumer({ db: s.db, s, secrets: config.events.secrets, coins, accountData, accountSend, now: s.now, log });++ const ctx = { config, s, keys, sso, principal, coins, eventsConsumer, accountData, accountSend, log };const app = express();app.disable('x-powered-by');@@@ -106,7 +117,8 @@ async function createApp(opts = {}) {app.use('/auth', sso.routes());{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'quest', service: 'quest', host: 'openvibe.quest', name: 'OpenVibe.Quest', profile: 'ugc' })); }-- // ── OpenVibe.Events → the quest log (loopback only; the raw body is the signature's) ──++ // ── OpenVibe.Events → the quest log and account export/deletion (loopback only; the raw body is the++ // signature's). The two network.account.* topics (ADR-033) are answered inside the consumer. ──app.use('/internal/events', eventsConsumer.router);// ── Static assets (content-hashed ?v= → immutable) ──────@@@ -19,6 +19,11 @@** Which field carries the person, and what is ignored, is ./quests/apply.js.*++ * Two more topics are account export and deletion (ADR-033): network.account.export_requested and++ * network.account.deleted are handed straight to openvibe-sdk/account-data (server/identity/account-data.js — the++ * table map), which keeps its own idempotency in account_data_events. They never pass through quest_event_inbox:++ * they are a person's rows leaving, not a quest moving, and apply() answers 'exported' / 'erased' / 'confirmed'.++ ** Exactly once: the openvibe-sdk inbox claims (consumer, event_id) in the same transaction as the writes. A* replay of the same event is answered 200 with duplicate:true and changes nothing. Signature v2 only* (parseDelivery requireV2) under QUEST_EVENTS_SECRET (comma-separated for rotation, 32+ characters each); unset@@@ -29,12 +34,14 @@ const { http, serviceAuth } = require('openvibe-contracts');const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');const catalog = require('./quests/catalog');const { apply } = require('./quests/apply');++const accountDataLib = require('./identity/account-data');const CONSUMER = 'quest';--const TOPICS = catalog.EVENT_TYPES;++// The catalog's events plus the two account topics (ADR-033): one subscription list, so boot subscribes to both.++const TOPICS = Object.freeze([...catalog.EVENT_TYPES, ...accountDataLib.TOPICS]);const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;--function createEventsConsumer({ db, s, secrets = [], coins = null, now = () => Date.now(), log = console }) {++function createEventsConsumer({ db, s, secrets = [], coins = null, accountData = null, accountSend = null, now = () => Date.now(), log = console }) {const keys = (secrets || []).filter((x) => typeof x === 'string' && x.length >= 32);// Receipts (quest_event_inbox) are in migrations/0002_quest.sql.const inbox = createPgInbox(db, { table: 'quest_event_inbox', now });@@@ -54,6 +61,23 @@ function createEventsConsumer({ db, s, secrets = [], coins = null, now = () => Dif (!event || !EVENT_ID_RE.test(String(event.event_id || '')) || typeof event.event_type !== 'string') { stats.refused++; return problem(400, 'quest.bad_delivery', 'body must be { event: <envelope>, seq }'); }stats.received++; stats.last_at = new Date(now()).toISOString();++ // Account export and deletion (ADR-033) are not quest events: openvibe-sdk/account-data answers them with++ // its own idempotency (account_data_events), so they never touch quest_event_inbox. An ignored payload is++ // answered 200; a refusal worth retrying throws and asks Events to redeliver.++ if (accountData && accountData.TOPICS.includes(event.event_type)) {++ let accountOutcome;++ try {++ accountOutcome = await accountData.apply(event, { send: accountSend });++ } catch (err) {++ stats.failed++;++ log.error(`[Events consumer] ${event.event_id} (${event.event_type}) failed:`, err.message);++ return problem(503, 'quest.event_failed', 'processing failed; it will be retried');++ }++ if (String(accountOutcome).startsWith('ignored:')) stats.ignored++;++ else stats.applied++;++ return res.json({ event_id: event.event_id, outcome: accountOutcome });++ }++let outcome;try {const r = await inbox.once(CONSUMER, event.event_id, (t) => apply(t, event, { at: new Date(now()).toISOString(), newId: s.newId }));nnew file mode 100644@@@ -0,0 +1,39 @@++'use strict';++++/**++ * Account export and deletion → Quest (ADR-033; openvibe-sdk/account-data). Quest holds three things a person++ * made: how far they are through each quest (quest_progress), the quests they finished and the OpenCoins that go++ * with them (quest_completions), and the badges they earned (quest_badges). All three are keyed `subject = usr_…`++ * (server/quests/store.js writes the event's subject straight into the column), the person's own rows — nothing++ * anyone else hangs under them — so all three are deleted whole and nothing is kept.++ *++ * network.account.export_requested the person's progress, completions and badges, pushed to Network++ * (POST /internal/account-exports/:id/parts) with this service's token.++ * network.account.deleted the person's rows in all three tables go, and Quest confirms with counts.++ *++ * Coins: a completion is a loyalty reward; deleting the completion deletes its record here. The wallet credit at++ * OpenVibe.Network is Network's own data, not Quest's.++ *++ * quest_event_inbox holds this service's own delivery receipts, not a person's rows, so it is neither exported nor++ * erased (migrations/0002_quest.sql). Nothing here is a secret: no token, key or credential is stored, so every++ * column of all three tables may be exported.++ */++const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');++++/**++ * The tables that hold a person's rows, with the value the subject column really stores. All three store the raw++ * `usr_…` id (server/quests/store.js takes it from the event's schema field), so the value is the identity itself —++ * the SDK default, no prefix.++ */++const TABLES = [++ { table: 'quest_progress', subject: 'subject', file: 'progress.json' },++ { table: 'quest_completions', subject: 'subject', file: 'completions.json' },++ { table: 'quest_badges', subject: 'subject', file: 'badges.json' },++];++++/** The account-data handle for Quest's store (server/db.js createStore). */++function create({ db, note = null, log = console } = {}) {++ return createAccountData({ db, service: 'quest', tables: TABLES, note, log });++}++++module.exports = { create, TABLES, TOPICS };nnew file mode 100644@@@ -0,0 +1,145 @@++'use strict';++/**++ * ADR-033: Quest's part of an account export (progress, completions, badges) and of an account deletion, applied++ * through the service's own /internal/events route with a stand-in Network. Real rows are made through the++ * service's own store (server/quests/store.js), the signed delivery is answered inside server/events-consumer.js by++ * openvibe-sdk/account-data, and the part only ever carries person A's rows. A redelivery erases nothing twice, a++ * bare network.account.deleted delivery is never claimed by quest_event_inbox, and the route refuses a bad++ * signature and a forwarded request.++ */++const assert = require('assert');++const http = require('http');++const { boot, check, done } = require('./helpers/boot');++const { createNetworkSender } = require('openvibe-sdk/account-data');++const { signDeliveryHeaders } = require('openvibe-sdk/events');++const questStore = require('../server/quests/store');++++const A = 'usr_01JZ0000000000000000000AAA';++const B = 'usr_01JZ0000000000000000000BBB';++const OLD = 'usr_01JZ0000000000000000000MRG';++const EXP = 'exp_01JZ0000000000000000000EXP';++const DEL = 'del_01JZ0000000000000000000DEX';++// Fixture secrets, built so they never look like a real key to a scanner.++const SECRET = `whsec_${'fixture'.repeat(6)}`;++const WRONG = `whsec_${'mismatch'.repeat(5)}`;++++const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };++const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };++const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });++++/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */++async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {++ const calls = [];++ const statusOf = (v) => (typeof v === 'function' ? v() : v);++ const server = http.createServer((req, res) => {++ const chunks = [];++ req.on('data', (c) => chunks.push(c));++ req.on('end', () => {++ const raw = Buffer.concat(chunks);++ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };++ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_quest', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });++ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });++ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});++ });++ });++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };++}++++/** Make progress, a completion and a badge for one person, through the store the consumer writes with. */++async function makeRows(t, subject, { questId, badgeId }) {++ const s = t.ctx.s;++ const at = s.iso();++ await questStore.bumpProgress(s.db, { subject, questId, step: 0, target: 1, at });++ await questStore.complete(s.db, { id: s.newId('qcp'), subject, questId, at, coins: 10, coinsState: 'skipped' });++ await questStore.awardBadge(s.db, { subject, badgeId, questId, at });++}++++const rowsFor = async (t, table, subject) => Number(await t.ctx.s.db.value(`SELECT count(*)::int FROM ${table} WHERE subject = $1`, [subject]));++++(async () => {++ await check('an export part carries only the person\'s progress, completions and badges, and no secret', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });++ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ await makeRows(t, A, { questId: 'go-live', badgeId: 'first-broadcast' });++ await makeRows(t, B, { questId: 'follow-someone', badgeId: 'first-follow' });++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'exported');++++ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);++ assert.ok(part, 'the part was pushed to Network');++ assert.strictEqual(part.auth, 'Bearer tok_quest', 'with this service\'s own token');++ assert.strictEqual(part.body.subject, A);++ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['badges.json', 'completions.json', 'progress.json']);++ for (const file of part.body.files) {++ assert.strictEqual(file.content.length, 1, `one row for ${file.name}`);++ assert.strictEqual(file.content[0].subject, A, `only A's rows in ${file.name}`);++ }++ assert.strictEqual(part.body.files.find((f) => f.name === 'progress.json').content[0].quest_id, 'go-live');++ assert.strictEqual(part.body.files.find((f) => f.name === 'completions.json').content[0].quest_id, 'go-live');++ assert.strictEqual(part.body.files.find((f) => f.name === 'badges.json').content[0].badge_id, 'first-broadcast');++ assert.ok(!JSON.stringify(part.body).includes(B), 'nobody else\'s rows');++ assert.ok(!JSON.stringify(part.body).includes('follow-someone'), 'no B-only content');++ assert.ok(!JSON.stringify(part.body).includes('first-follow'), 'no B-only badge');++ assert.ok(!/token|secret|password/i.test(JSON.stringify(part.body)), 'no secret is exported');++ // The account topics never pass through the quest inbox: the SDK keeps its own idempotency.++ assert.strictEqual(Number(await t.ctx.s.db.value('SELECT count(*)::int FROM quest_event_inbox WHERE event_id = $1', [exportEvent.event_id])), 0, 'the export event is not claimed by quest_event_inbox');++ } finally { await t.close(); await stub.close(); }++ });++++ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });++ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ await makeRows(t, A, { questId: 'go-live', badgeId: 'first-broadcast' });++ await makeRows(t, B, { questId: 'follow-someone', badgeId: 'first-follow' });++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'erased');++ for (const table of ['quest_progress', 'quest_completions', 'quest_badges']) {++ assert.strictEqual(await rowsFor(t, table, A), 0, `the person's rows are gone from ${table}`);++ assert.strictEqual(await rowsFor(t, table, B), 1, `someone else's row stays in ${table}`);++ }++ // A bare network.account.deleted is never claimed by the quest inbox.++ assert.strictEqual(Number(await t.ctx.s.db.value('SELECT count(*)::int FROM quest_event_inbox WHERE event_id = $1', [deleteEvent.event_id])), 0, 'the deletion is not claimed by quest_event_inbox');++++ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);++ assert.ok(confirmation, 'the confirmation was sent');++ assert.deepStrictEqual(confirmation.body.erased, { quest_progress: 1, quest_completions: 1, quest_badges: 1 });++ assert.deepStrictEqual(confirmation.body.retained, {});++ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));++++ // A redelivery must not erase again: rows written after the deletion stay.++ await makeRows(t, A, { questId: 'use-a-tool', badgeId: 'toolsmith' });++ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(again.status, 200);++ assert.strictEqual(again.json().outcome, 'unchanged');++ for (const table of ['quest_progress', 'quest_completions', 'quest_badges']) {++ assert.strictEqual(await rowsFor(t, table, A), 1, `nothing was erased twice in ${table}`);++ }++ } finally { await t.close(); await stub.close(); }++ });++++ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });++ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });++ assert.strictEqual(bad.status, 401);++ assert.strictEqual(bad.json().code, 'quest.bad_signature');++++ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(forwarded.status, 403);++ assert.strictEqual(forwarded.json().code, 'quest.internal_only');++ } finally { await t.close(); await stub.close(); }++ });++++ done();++})();@@@ -38,7 +38,7 @@ async function boot(opts = {}) {const { createStore } = require('../../server/db');const testdb = await require('./db').testDb();const store = createStore(testdb.db, { now: opts.now });-- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });++ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });await built.ctx.keys.ensure();const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;
@@ -121,6 +121,31 @@ idempotent by key), so the OpenCoins tests need no real wallet.
@@ -121,6 +121,31 @@ idempotent by key), so the OpenCoins tests need no real wallet.
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: quest` in CI) and with
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: quest` in CI) and with
**OpenVibe.Services** before the first deploy.
**OpenVibe.Services** before the first deploy.
+## Account export and deletion
+
+A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its
+part (ADR-033). Quest receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`
+(loopback only) through the consumer it already runs — the two account topics are handed to
+[server/identity/account-data.js](server/identity/account-data.js), which maps the tables, and the boot-time
+subscriptions for them are created (alongside the catalog's events) by
+[server/events-consumer.js](server/events-consumer.js):
+
+- **Exported:** the quests a person is partway through (`progress.json`), the quests they completed and their OpenCoins
+ (`completions.json`) and the badges they earned (`badges.json`), pushed to `POST /internal/account-exports/:id/parts`
+ with this service's own token. Nothing here is a secret — Quest stores no token, key or credential.
+- **Erased:** all three tables hold the person's own rows and nothing anyone else's page hangs under them, so they are
+ deleted whole and nothing is kept. Deleting a completion deletes its OpenCoins record here; the wallet credit at
+ OpenVibe.Network is Network's own data. Quest then confirms with `POST /internal/account-deletions/:id/confirmations`
+ and the counts.
+- **Anonymized:** nothing. There is no row Quest keeps that was written by this person for another person to read.
+
+The delivery receipts (`quest_event_inbox`) are this service's own, not a person's rows, and are neither exported nor
+erased.
+
+Environment: `QUEST_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer
+503), `QUEST_EVENTS_URL` (or `EVENTS_URL`) is where the subscriptions are created at boot (off when unset), and
+`QUEST_EVENTS_ENDPOINT` overrides the loopback endpoint; `QUEST_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.
+
## Security (threat notes)
## Security (threat notes)
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
@@ -7,11 +7,11 @@
@@ -7,11 +7,11 @@
"port": "4980",
"port": "4980",
"domain": "openvibe.quest",
"domain": "openvibe.quest",
"contracts": "openvibe-contracts v0.115.0",
"contracts": "openvibe-contracts v0.115.0",
- "sdk": "openvibe-sdk v0.35.2",
+ "sdk": "openvibe-sdk v0.36.0",
"shared": "openvibe-shared v2.14.0",
"shared": "openvibe-shared v2.14.0",
"packages": [
"packages": [
"openvibe-contracts v0.115.0",
"openvibe-contracts v0.115.0",
- "openvibe-sdk v0.35.2",
+ "openvibe-sdk v0.36.0",
"openvibe-shared v2.14.0"
"openvibe-shared v2.14.0"
],
],
"summary": "OpenVibe.Quest: one quest log across every OpenVibe site. OpenVibe.Events delivers the domain events the sites already emit to POST /internal/events (signed v2, once-only through the openvibe-sdk inbox), the person taken from the field each event's own schema defines, and a progress count, a completion and a badge move because of it. 12 first-party quests in code, grouped by the site whose events each counts. Rewards are badges and OpenCoins — credited through OpenVibe.Network's wallet, off by default (QUEST_COINS), with per-person and network-wide daily caps, failures recorded and retried. Public API and server-rendered pages, complete without JavaScript. Not deployed; OpenCoins are off until Network grants the two capabilities.",
"summary": "OpenVibe.Quest: one quest log across every OpenVibe site. OpenVibe.Events delivers the domain events the sites already emit to POST /internal/events (signed v2, once-only through the openvibe-sdk inbox), the person taken from the field each event's own schema defines, and a progress count, a completion and a badge move because of it. 12 first-party quests in code, grouped by the site whose events each counts. Rewards are badges and OpenCoins — credited through OpenVibe.Network's wallet, off by default (QUEST_COINS), with per-person and network-wide daily caps, failures recorded and retried. Public API and server-rendered pages, complete without JavaScript. Not deployed; OpenCoins are off until Network grants the two capabilities.",
@@ -25,14 +25,15 @@
@@ -25,14 +25,15 @@
"QUEST_COINS off by default (coins_state=skipped, the pages say the rewards start soon); per-person (50) and network-wide (5000) daily caps; a failure recorded with its reason and retried by a timer",
"QUEST_COINS off by default (coins_state=skipped, the pages say the rewards start soon); per-person (50) and network-wide (5000) daily caps; a failure recorded with its reason and retried by a timer",
"GET /api/v1/quests, /quests/:id, /me/quests (a signed-in person's own log), /profiles/:subject/badges (id, name and date only)",
"GET /api/v1/quests, /quests/:id, /me/quests (a signed-in person's own log), /profiles/:subject/badges (id, name and date only)",
"Pages: home (the quests grouped by site, how progress works), a quest, your quest log (plain <progress> bars), the rules, the update log; robots, sitemap with every quest page, llms.txt, llms-full.txt",
"Pages: home (the quests grouped by site, how progress works), a quest, your quest log (plain <progress> bars), the rules, the update log; robots, sitemap with every quest page, llms.txt, llms-full.txt",
- "Tests on temp PGlite databases with a mock Network that also serves the wallet: signature refusal, idempotent replay, counting to completion, badge once, coins off, coins on with caps and a retry, events with no subject ignored, /me requires sign-in, hostile titles escaped"
+ "Tests on temp PGlite databases with a mock Network that also serves the wallet: signature refusal, idempotent replay, counting to completion, badge once, coins off, coins on with caps and a retry, events with no subject ignored, /me requires sign-in, hostile titles escaped",
+ "Account export and deletion (ADR-033): the quest progress, completions and badges a person has are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted, with the counts confirmed back; the two account topics join the catalog's events in the boot subscriptions (server/identity/account-data.js)"
],
],
"notYet": [
"notYet": [
"A deployment: the service is not registered with OpenVibe.Services nor deployed by OpenVibe.Host, and its capabilities are not in openvibe-contracts yet (no quest.* manifest exists)",
"A deployment: the service is not registered with OpenVibe.Services nor deployed by OpenVibe.Host, and its capabilities are not in openvibe-contracts yet (no quest.* manifest exists)",
"OpenCoins are off in production until OpenVibe.Network grants the quest client identity.subject.resolve and network.coins.credit",
"OpenCoins are off in production until OpenVibe.Network grants the quest client identity.subject.resolve and network.coins.credit",
"Community-made quests (streamers, communities, mods): they need moderation, which this release does not have",
"Community-made quests (streamers, communities, mods): they need moderation, which this release does not have",
"Repeatable quests and seasonal resets: every quest completes once per person",
"Repeatable quests and seasonal resets: every quest completes once per person",
- "Account deletion and merges (ADR-033, ADR-029): quest_progress, quest_completions and quest_badges rows are not removed or re-keyed when a Network account is deleted or merged",
+ "Account merges (ADR-029): quest_progress, quest_completions and quest_badges rows are not re-keyed when a Network account is merged into another (deletion is answered, ADR-033; a merge's aliases are erased with the account)",
"Quest chains, leaderboards and per-community quests; a streamer's getting-started steps still live on OpenVibe.Live"
"Quest chains, leaderboards and per-community quests; a streamer's getting-started steps still live on OpenVibe.Live"
],
],
"capabilitiesRegistered": false,
"capabilitiesRegistered": false,
new file mode 100644
new file mode 100644
@@ -0,0 +1,13 @@
@@ -0,0 +1,13 @@
+-- phase: expand
+-- OpenVibe.Quest: the receipts of the ADR-033 account export and deletion deliveries this service applied
+-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.
+-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.
+
+CREATE TABLE IF NOT EXISTS account_data_events (
+ id TEXT PRIMARY KEY,
+ kind TEXT NOT NULL,
+ subject TEXT NOT NULL,
+ outcome JSONB,
+ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+ sent_at TIMESTAMPTZ
+);
@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');
@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');
const { createPageRoutes } = require('./http/pages');
const { createPageRoutes } = require('./http/pages');
const { createCoins } = require('./quests/coins');
const { createCoins } = require('./quests/coins');
const { createEventsConsumer } = require('./events-consumer');
const { createEventsConsumer } = require('./events-consumer');
+const accountDataLib = require('./identity/account-data');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
const { createServiceReadiness } = require('./observability');
const { createServiceReadiness } = require('./observability');
const { createCallerLimits } = require('./http/caller-limits');
const { createCallerLimits } = require('./http/caller-limits');
const { assetVersion, send } = require('./render/layout');
const { assetVersion, send } = require('./render/layout');
@@ -40,7 +42,7 @@ const VERSION = require('../package.json').version;
@@ -40,7 +42,7 @@ const VERSION = require('../package.json').version;
/**
/**
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
- * tests only), valkey
+ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)
*/
*/
async function createApp(opts = {}) {
async function createApp(opts = {}) {
const config = opts.config || configLib.load();
const config = opts.config || configLib.load();
@@ -52,8 +54,17 @@ async function createApp(opts = {}) {
@@ -52,8 +54,17 @@ async function createApp(opts = {}) {
const sso = createSso({ config, keys, fetchImpl, now: s.now, log });
const sso = createSso({ config, keys, fetchImpl, now: s.now, log });
const principal = createPrincipal({ config, keys });
const principal = createPrincipal({ config, keys });
const coins = createCoins({ config, s, fetchImpl, log });
const coins = createCoins({ config, s, fetchImpl, log });
- const eventsConsumer = createEventsConsumer({ db: s.db, s, secrets: config.events.secrets, coins, now: s.now, log });
- const ctx = { config, s, keys, sso, principal, coins, eventsConsumer, log };
+ // Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.
+ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;
+ // a test injects a stand-in through opts.accountSend. Without a client secret the service has no way to push a
+ // part or a confirmation: the route still answers (bad signature, a forwarded request, no secret), and an event
+ // that really arrives asks Events to retry.
+ const accountData = accountDataLib.create({ db: s.db, log });
+ const accountSend = opts.accountSend || (config.oauth.clientSecret
+ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })
+ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Quest cannot answer account events'); });
+ const eventsConsumer = createEventsConsumer({ db: s.db, s, secrets: config.events.secrets, coins, accountData, accountSend, now: s.now, log });
+ const ctx = { config, s, keys, sso, principal, coins, eventsConsumer, accountData, accountSend, log };
const app = express();
const app = express();
app.disable('x-powered-by');
app.disable('x-powered-by');
@@ -106,7 +117,8 @@ async function createApp(opts = {}) {
@@ -106,7 +117,8 @@ async function createApp(opts = {}) {
app.use('/auth', sso.routes());
app.use('/auth', sso.routes());
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'quest', service: 'quest', host: 'openvibe.quest', name: 'OpenVibe.Quest', profile: 'ugc' })); }
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'quest', service: 'quest', host: 'openvibe.quest', name: 'OpenVibe.Quest', profile: 'ugc' })); }
- // ── OpenVibe.Events → the quest log (loopback only; the raw body is the signature's) ──
+ // ── OpenVibe.Events → the quest log and account export/deletion (loopback only; the raw body is the
+ // signature's). The two network.account.* topics (ADR-033) are answered inside the consumer. ──
app.use('/internal/events', eventsConsumer.router);
app.use('/internal/events', eventsConsumer.router);
// ── Static assets (content-hashed ?v= → immutable) ──────
// ── Static assets (content-hashed ?v= → immutable) ──────
@@ -19,6 +19,11 @@
@@ -19,6 +19,11 @@
*
*
* Which field carries the person, and what is ignored, is ./quests/apply.js.
* Which field carries the person, and what is ignored, is ./quests/apply.js.
*
*
+ * Two more topics are account export and deletion (ADR-033): network.account.export_requested and
+ * network.account.deleted are handed straight to openvibe-sdk/account-data (server/identity/account-data.js — the
+ * table map), which keeps its own idempotency in account_data_events. They never pass through quest_event_inbox:
+ * they are a person's rows leaving, not a quest moving, and apply() answers 'exported' / 'erased' / 'confirmed'.
+ *
* Exactly once: the openvibe-sdk inbox claims (consumer, event_id) in the same transaction as the writes. A
* Exactly once: the openvibe-sdk inbox claims (consumer, event_id) in the same transaction as the writes. A
* replay of the same event is answered 200 with duplicate:true and changes nothing. Signature v2 only
* replay of the same event is answered 200 with duplicate:true and changes nothing. Signature v2 only
* (parseDelivery requireV2) under QUEST_EVENTS_SECRET (comma-separated for rotation, 32+ characters each); unset
* (parseDelivery requireV2) under QUEST_EVENTS_SECRET (comma-separated for rotation, 32+ characters each); unset
@@ -29,12 +34,14 @@ const { http, serviceAuth } = require('openvibe-contracts');
@@ -29,12 +34,14 @@ const { http, serviceAuth } = require('openvibe-contracts');
const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');
const { parseDelivery, createPgInbox } = require('openvibe-sdk/events');
const catalog = require('./quests/catalog');
const catalog = require('./quests/catalog');
const { apply } = require('./quests/apply');
const { apply } = require('./quests/apply');
+const accountDataLib = require('./identity/account-data');
const CONSUMER = 'quest';
const CONSUMER = 'quest';
-const TOPICS = catalog.EVENT_TYPES;
+// The catalog's events plus the two account topics (ADR-033): one subscription list, so boot subscribes to both.
+const TOPICS = Object.freeze([...catalog.EVENT_TYPES, ...accountDataLib.TOPICS]);
const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;
const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;
-function createEventsConsumer({ db, s, secrets = [], coins = null, now = () => Date.now(), log = console }) {
+function createEventsConsumer({ db, s, secrets = [], coins = null, accountData = null, accountSend = null, now = () => Date.now(), log = console }) {
const keys = (secrets || []).filter((x) => typeof x === 'string' && x.length >= 32);
const keys = (secrets || []).filter((x) => typeof x === 'string' && x.length >= 32);
// Receipts (quest_event_inbox) are in migrations/0002_quest.sql.
// Receipts (quest_event_inbox) are in migrations/0002_quest.sql.
const inbox = createPgInbox(db, { table: 'quest_event_inbox', now });
const inbox = createPgInbox(db, { table: 'quest_event_inbox', now });
@@ -54,6 +61,23 @@ function createEventsConsumer({ db, s, secrets = [], coins = null, now = () => D
@@ -54,6 +61,23 @@ function createEventsConsumer({ db, s, secrets = [], coins = null, now = () => D
if (!event || !EVENT_ID_RE.test(String(event.event_id || '')) || typeof event.event_type !== 'string') { stats.refused++; return problem(400, 'quest.bad_delivery', 'body must be { event: <envelope>, seq }'); }
if (!event || !EVENT_ID_RE.test(String(event.event_id || '')) || typeof event.event_type !== 'string') { stats.refused++; return problem(400, 'quest.bad_delivery', 'body must be { event: <envelope>, seq }'); }
stats.received++; stats.last_at = new Date(now()).toISOString();
stats.received++; stats.last_at = new Date(now()).toISOString();
+ // Account export and deletion (ADR-033) are not quest events: openvibe-sdk/account-data answers them with
+ // its own idempotency (account_data_events), so they never touch quest_event_inbox. An ignored payload is
+ // answered 200; a refusal worth retrying throws and asks Events to redeliver.
+ if (accountData && accountData.TOPICS.includes(event.event_type)) {
+ let accountOutcome;
+ try {
+ accountOutcome = await accountData.apply(event, { send: accountSend });
+ } catch (err) {
+ stats.failed++;
+ log.error(`[Events consumer] ${event.event_id} (${event.event_type}) failed:`, err.message);
+ return problem(503, 'quest.event_failed', 'processing failed; it will be retried');
+ }
+ if (String(accountOutcome).startsWith('ignored:')) stats.ignored++;
+ else stats.applied++;
+ return res.json({ event_id: event.event_id, outcome: accountOutcome });
+ }
+
let outcome;
let outcome;
try {
try {
const r = await inbox.once(CONSUMER, event.event_id, (t) => apply(t, event, { at: new Date(now()).toISOString(), newId: s.newId }));
const r = await inbox.once(CONSUMER, event.event_id, (t) => apply(t, event, { at: new Date(now()).toISOString(), newId: s.newId }));
new file mode 100644
new file mode 100644
@@ -0,0 +1,39 @@
@@ -0,0 +1,39 @@
+'use strict';
+
+/**
+ * Account export and deletion → Quest (ADR-033; openvibe-sdk/account-data). Quest holds three things a person
+ * made: how far they are through each quest (quest_progress), the quests they finished and the OpenCoins that go
+ * with them (quest_completions), and the badges they earned (quest_badges). All three are keyed `subject = usr_…`
+ * (server/quests/store.js writes the event's subject straight into the column), the person's own rows — nothing
+ * anyone else hangs under them — so all three are deleted whole and nothing is kept.
+ *
+ * network.account.export_requested the person's progress, completions and badges, pushed to Network
+ * (POST /internal/account-exports/:id/parts) with this service's token.
+ * network.account.deleted the person's rows in all three tables go, and Quest confirms with counts.
+ *
+ * Coins: a completion is a loyalty reward; deleting the completion deletes its record here. The wallet credit at
+ * OpenVibe.Network is Network's own data, not Quest's.
+ *
+ * quest_event_inbox holds this service's own delivery receipts, not a person's rows, so it is neither exported nor
+ * erased (migrations/0002_quest.sql). Nothing here is a secret: no token, key or credential is stored, so every
+ * column of all three tables may be exported.
+ */
+const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');
+
+/**
+ * The tables that hold a person's rows, with the value the subject column really stores. All three store the raw
+ * `usr_…` id (server/quests/store.js takes it from the event's schema field), so the value is the identity itself —
+ * the SDK default, no prefix.
+ */
+const TABLES = [
+ { table: 'quest_progress', subject: 'subject', file: 'progress.json' },
+ { table: 'quest_completions', subject: 'subject', file: 'completions.json' },
+ { table: 'quest_badges', subject: 'subject', file: 'badges.json' },
+];
+
+/** The account-data handle for Quest's store (server/db.js createStore). */
+function create({ db, note = null, log = console } = {}) {
+ return createAccountData({ db, service: 'quest', tables: TABLES, note, log });
+}
+
+module.exports = { create, TABLES, TOPICS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,145 @@
@@ -0,0 +1,145 @@
+'use strict';
+/**
+ * ADR-033: Quest's part of an account export (progress, completions, badges) and of an account deletion, applied
+ * through the service's own /internal/events route with a stand-in Network. Real rows are made through the
+ * service's own store (server/quests/store.js), the signed delivery is answered inside server/events-consumer.js by
+ * openvibe-sdk/account-data, and the part only ever carries person A's rows. A redelivery erases nothing twice, a
+ * bare network.account.deleted delivery is never claimed by quest_event_inbox, and the route refuses a bad
+ * signature and a forwarded request.
+ */
+const assert = require('assert');
+const http = require('http');
+const { boot, check, done } = require('./helpers/boot');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
+const { signDeliveryHeaders } = require('openvibe-sdk/events');
+const questStore = require('../server/quests/store');
+
+const A = 'usr_01JZ0000000000000000000AAA';
+const B = 'usr_01JZ0000000000000000000BBB';
+const OLD = 'usr_01JZ0000000000000000000MRG';
+const EXP = 'exp_01JZ0000000000000000000EXP';
+const DEL = 'del_01JZ0000000000000000000DEX';
+// Fixture secrets, built so they never look like a real key to a scanner.
+const SECRET = `whsec_${'fixture'.repeat(6)}`;
+const WRONG = `whsec_${'mismatch'.repeat(5)}`;
+
+const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };
+const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };
+const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });
+
+/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */
+async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {
+ const calls = [];
+ const statusOf = (v) => (typeof v === 'function' ? v() : v);
+ const server = http.createServer((req, res) => {
+ const chunks = [];
+ req.on('data', (c) => chunks.push(c));
+ req.on('end', () => {
+ const raw = Buffer.concat(chunks);
+ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };
+ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_quest', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });
+ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });
+ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});
+ });
+ });
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };
+}
+
+/** Make progress, a completion and a badge for one person, through the store the consumer writes with. */
+async function makeRows(t, subject, { questId, badgeId }) {
+ const s = t.ctx.s;
+ const at = s.iso();
+ await questStore.bumpProgress(s.db, { subject, questId, step: 0, target: 1, at });
+ await questStore.complete(s.db, { id: s.newId('qcp'), subject, questId, at, coins: 10, coinsState: 'skipped' });
+ await questStore.awardBadge(s.db, { subject, badgeId, questId, at });
+}
+
+const rowsFor = async (t, table, subject) => Number(await t.ctx.s.db.value(`SELECT count(*)::int FROM ${table} WHERE subject = $1`, [subject]));
+
+(async () => {
+ await check('an export part carries only the person\'s progress, completions and badges, and no secret', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });
+ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ await makeRows(t, A, { questId: 'go-live', badgeId: 'first-broadcast' });
+ await makeRows(t, B, { questId: 'follow-someone', badgeId: 'first-follow' });
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'exported');
+
+ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);
+ assert.ok(part, 'the part was pushed to Network');
+ assert.strictEqual(part.auth, 'Bearer tok_quest', 'with this service\'s own token');
+ assert.strictEqual(part.body.subject, A);
+ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['badges.json', 'completions.json', 'progress.json']);
+ for (const file of part.body.files) {
+ assert.strictEqual(file.content.length, 1, `one row for ${file.name}`);
+ assert.strictEqual(file.content[0].subject, A, `only A's rows in ${file.name}`);
+ }
+ assert.strictEqual(part.body.files.find((f) => f.name === 'progress.json').content[0].quest_id, 'go-live');
+ assert.strictEqual(part.body.files.find((f) => f.name === 'completions.json').content[0].quest_id, 'go-live');
+ assert.strictEqual(part.body.files.find((f) => f.name === 'badges.json').content[0].badge_id, 'first-broadcast');
+ assert.ok(!JSON.stringify(part.body).includes(B), 'nobody else\'s rows');
+ assert.ok(!JSON.stringify(part.body).includes('follow-someone'), 'no B-only content');
+ assert.ok(!JSON.stringify(part.body).includes('first-follow'), 'no B-only badge');
+ assert.ok(!/token|secret|password/i.test(JSON.stringify(part.body)), 'no secret is exported');
+ // The account topics never pass through the quest inbox: the SDK keeps its own idempotency.
+ assert.strictEqual(Number(await t.ctx.s.db.value('SELECT count(*)::int FROM quest_event_inbox WHERE event_id = $1', [exportEvent.event_id])), 0, 'the export event is not claimed by quest_event_inbox');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });
+ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ await makeRows(t, A, { questId: 'go-live', badgeId: 'first-broadcast' });
+ await makeRows(t, B, { questId: 'follow-someone', badgeId: 'first-follow' });
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'erased');
+ for (const table of ['quest_progress', 'quest_completions', 'quest_badges']) {
+ assert.strictEqual(await rowsFor(t, table, A), 0, `the person's rows are gone from ${table}`);
+ assert.strictEqual(await rowsFor(t, table, B), 1, `someone else's row stays in ${table}`);
+ }
+ // A bare network.account.deleted is never claimed by the quest inbox.
+ assert.strictEqual(Number(await t.ctx.s.db.value('SELECT count(*)::int FROM quest_event_inbox WHERE event_id = $1', [deleteEvent.event_id])), 0, 'the deletion is not claimed by quest_event_inbox');
+
+ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);
+ assert.ok(confirmation, 'the confirmation was sent');
+ assert.deepStrictEqual(confirmation.body.erased, { quest_progress: 1, quest_completions: 1, quest_badges: 1 });
+ assert.deepStrictEqual(confirmation.body.retained, {});
+ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));
+
+ // A redelivery must not erase again: rows written after the deletion stay.
+ await makeRows(t, A, { questId: 'use-a-tool', badgeId: 'toolsmith' });
+ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(again.status, 200);
+ assert.strictEqual(again.json().outcome, 'unchanged');
+ for (const table of ['quest_progress', 'quest_completions', 'quest_badges']) {
+ assert.strictEqual(await rowsFor(t, table, A), 1, `nothing was erased twice in ${table}`);
+ }
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'quest', clientSecret: 'quest-secret' });
+ const t = await boot({ env: { QUEST_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });
+ assert.strictEqual(bad.status, 401);
+ assert.strictEqual(bad.json().code, 'quest.bad_signature');
+
+ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(forwarded.status, 403);
+ assert.strictEqual(forwarded.json().code, 'quest.internal_only');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ done();
+})();
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
const { createStore } = require('../../server/db');
const { createStore } = require('../../server/db');
const testdb = await require('./db').testDb();
const testdb = await require('./db').testDb();
const store = createStore(testdb.db, { now: opts.now });
const store = createStore(testdb.db, { now: opts.now });
- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });
+ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });
await built.ctx.keys.ensure();
await built.ctx.keys.ensure();
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;