openvibers/openvibe.rent Commits
Rent: account export and deletion (ADR-033) through openvibe-sdk/account-data
Changes
@@@ -30,5 +30,12 @@ OV_OAUTH_CLIENT_SECRET=# OV_OAUTH_REDIRECT_URI=https://openvibe.rent/auth/callback# RENT_AUDIENCE=openvibe.rent++# OpenVibe.Events → Rent (ADR-033 account export and deletion): the secret signs a delivery (comma-separated for++# rotation, 32+ characters each; unset makes POST /internal/events answer 503); the url is where the two subscriptions++# are created at boot (off when unset); RENT_EVENTS_SUBSCRIBE=0 turns that off.++RENT_EVENTS_SECRET=++RENT_EVENTS_URL=++#RENT_EVENTS_ENDPOINT=http://127.0.0.1:5010/internal/events++# optional: defaults to true in production# COOKIE_SECURE=true@@@ -84,8 +84,9 @@ Network), `BASE_URL`, `DATABASE_URL` and `DATABASE_DIRECT_URL`. The database isthe Network signing key, the OAuth client and Valkey are optional (the service says so, per check, on`/api/ready`).--The product adds no environment variables of its own: its numbers (10 active listings, 5 a day, 30 days, 3 reports,--30 a page, the per-caller budgets) are constants next to the rules they belong to, not deployment settings.++The product adds no environment variables of its own beyond the OpenVibe.Events ones below: its numbers (10 active++listings, 5 a day, 30 days, 3 reports, 30 a page, the per-caller budgets) are constants next to the rules they belong++to, not deployment settings.## Development and tests@@@ -110,6 +111,25 @@ through PostgreSQL and PgBouncer (see [.github/workflows/ci.yml](.github/workflo- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: rent` in CI) and with**OpenVibe.Services** before the first deploy.++## Account export and deletion++++A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its++part (ADR-033). Rent receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`++(loopback only) — the three tables are mapped in [server/identity/account-data.js](server/identity/account-data.js), and++the boot-time subscriptions are created by [server/events-consumer.js](server/events-consumer.js):++++- **Exported:** the listings a person posted (`listings.json`), the reports they filed (`reports.json`) and the searches++ they saved (`saved_searches.json`), pushed to `POST /internal/account-exports/:id/parts` with this service's own++ token. Nothing here is a secret — Rent stores no token, key or credential.++- **Erased:** all three tables hold the person's own rows, so they are deleted whole and nothing is kept. Rent then++ confirms with `POST /internal/account-deletions/:id/confirmations` and the counts.++- **Anonymized:** nothing. Rent keeps no row by this person that another person's page must still read — the reports++ other people left on a deleted listing cascade away with it.++++Environment: `RENT_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer++503), `RENT_EVENTS_URL` (or `EVENTS_URL`) is where the two subscriptions are created at boot (off when unset), and++`RENT_EVENTS_ENDPOINT` overrides the loopback endpoint; `RENT_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.++## Security (threat notes)Reporting a vulnerability: [SECURITY.md](SECURITY.md).@@@ -7,11 +7,11 @@"port": "5010","domain": "openvibe.rent","contracts": "openvibe-contracts v0.115.0",-- "sdk": "openvibe-sdk v0.35.2",++ "sdk": "openvibe-sdk v0.36.0","shared": "openvibe-shared v2.14.0","packages": ["openvibe-contracts v0.115.0",-- "openvibe-sdk v0.35.2",++ "openvibe-sdk v0.36.0","openvibe-shared v2.14.0"],"summary": "OpenVibe.Rent — Find a place, or rent yours out. One place to find and offer things to rent: apartments, rooms, houses, commercial space, parking and storage, and equipment. In this first version every listing is posted by a person on this site: no scraped listings and no partner feeds, no payments, no deposits and no messaging. Search with filters and facets, a listing page with a plain safety note, reports that hide a listing at 3 until staff look, saved searches that count what is new, and a 30-day expiry an owner can renew.",@@@ -29,7 +29,8 @@"Pages: /, /listings, /listings/:id, /listings/:id/edit, /post, /mine, /saved, /safety, /staff, /updates — server-rendered, complete without JavaScript, forms are plain POSTs","Same-origin writes for the session cookie, and per-caller budgets shared by the form and the API route","Discovery: robots.txt (disallows /auth/, /api/, /post, /mine, /saved, /staff), sitemap.xml with every active listing (≤1000), llms.txt, llms-full.txt",-- "Tests: posting rules, search and paging and facets, contact hidden from signed-out viewers, report → hide at 3 → staff restore, expiry, owner-only edits, same-origin writes, saved searches, hostile text escaped"++ "Tests: posting rules, search and paging and facets, contact hidden from signed-out viewers, report → hide at 3 → staff restore, expiry, owner-only edits, same-origin writes, saved searches, hostile text escaped",++ "Account export and deletion (ADR-033): the listings a person posted, the reports they filed and the searches they saved are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted (a report others left on a deleted listing cascades away with it), with the counts confirmed back; the two Events topics are subscribed at boot (server/identity/account-data.js)"],"notYet": ["Listing sources beyond people posting here: scraped and partner feeds arrive only when their terms are verified, and then each listing will carry its provenance (source name, link back, fetched_at)",@@@ -101,6 +101,10 @@ server {# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:5010/metrics) ──location = /metrics { return 404; }++ # ── OpenVibe.Events' delivery route: loopback only (POST /internal/events) ──++ # Events posts to http://127.0.0.1:5010/internal/events directly. Never proxied from outside (ADR-033).++ location /internal/ { return 404; }++# ── Pages, docs, robots.txt, sitemap, static assets ─────location / {proxy_pass http://127.0.0.1:5010;nnew file mode 100644@@@ -0,0 +1,13 @@++-- phase: expand++-- OpenVibe.Rent: the receipts of the ADR-033 account export and deletion deliveries this service applied++-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.++-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.++++CREATE TABLE IF NOT EXISTS account_data_events (++ id TEXT PRIMARY KEY,++ kind TEXT NOT NULL,++ subject TEXT NOT NULL,++ outcome JSONB,++ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),++ sent_at TIMESTAMPTZ++);@@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');const { createPageRoutes } = require('./http/pages');const { createServiceReadiness } = require('./observability');const { createCallerLimits } = require('./http/caller-limits');++const accountDataLib = require('./identity/account-data');++const { createNetworkSender } = require('openvibe-sdk/account-data');const { assetVersion, send } = require('./render/layout');const { html } = require('./render/html');@@@ -38,7 +40,7 @@ const VERSION = require('../package.json').version;/*** opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,-- * tests only), valkey++ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)*/async function createApp(opts = {}) {const config = opts.config || configLib.load();@@@ -51,6 +53,18 @@ async function createApp(opts = {}) {const principal = createPrincipal({ config, keys });const ctx = { config, s, keys, sso, principal, log };++ // Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.++ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;++ // a test injects a stand-in through opts.accountSend.++ const accountData = accountDataLib.create({ db: s.db, log });++ // Without a client secret the service has no way to push a part or a confirmation: the route still answers++ // (bad signature, a forwarded request, no secret), and an event that really arrives asks Events to retry.++ const accountSend = opts.accountSend || (config.oauth.clientSecret++ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })++ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Rent cannot answer account events'); });++ ctx.accountData = accountData;++ ctx.accountSend = accountSend;++const app = express();app.disable('x-powered-by');app.set('trust proxy', config.trustProxy);@@@ -102,6 +116,12 @@ async function createApp(opts = {}) {app.use('/auth', sso.routes());{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'rent', service: 'rent', host: 'openvibe.rent', name: 'OpenVibe.Rent', profile: 'ugc' })); }++ // ── OpenVibe.Events → this service (loopback only) ──────++ // network.account.export_requested and network.account.deleted (ADR-033), answered by the SDK's consumer. It++ // reads the raw body itself, so it is mounted before any body parser (there is none above it), and nginx answers++ // 404 for /internal/ so this is reachable only on 127.0.0.1.++ app.use('/internal/events', accountData.consumer({ secrets: config.events.secrets, send: accountSend, log }));++// ── Static assets (content-hashed ?v= → immutable) ──────app.use('/shared', require('openvibe-shared/serve').handler());app.use(express.static(PUBLIC_DIR, {@@@ -52,6 +52,16 @@ function load(env = process.env) {sessionAudience: env.OV_SESSION_AUDIENCE || 'openvibe.network',},cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },++++ // OpenVibe.Events → this service (server/events-consumer.js). The secret signs a delivery (comma-separated++ // for rotation, 32+ characters each); unset turns POST /internal/events off (503). The url is where the++ // network.account.export_requested and network.account.deleted subscriptions are created at boot, off when++ // unset. Both topics are ADR-033: account export and deletion (server/identity/account-data.js).++ events: {++ secrets: String(env.RENT_EVENTS_SECRET || '').split(',').map((x) => x.trim()).filter(Boolean),++ url: trim(env.RENT_EVENTS_URL || env.EVENTS_URL || ''),++ endpoint: env.RENT_EVENTS_ENDPOINT || '',++ },};}nnew file mode 100644@@@ -0,0 +1,52 @@++'use strict';++++/**++ * OpenVibe.Events → Rent. Rent has no consumer of its own: the two topics it takes are account export and deletion++ * (ADR-033), and POST /internal/events is answered by openvibe-sdk/account-data's own consumer, mounted in++ * server/app.js over server/identity/account-data.js (the table map). What is left here is the boot-time half:++ * startSubscriptions() creates the two subscriptions at Events if they are missing, idempotently, retried with++ * backoff, and off when the URL or the secret is unset (QUEST_EVENTS_SUBSCRIBE=0 style: RENT_EVENTS_SUBSCRIBE=0).++ *++ * The endpoint is this service's loopback /internal/events (Events posts to http://127.0.0.1:<port> directly; nginx++ * answers 404 there), and the token is this service's own for audience openvibe.events with++ * scope events.subscription.manage.++ */++const { serviceAuth } = require('openvibe-contracts');++const { TOPICS } = require('./identity/account-data');++++/** Create any missing subscription for TOPICS at Events (idempotent; retried in the background at boot). */++function startSubscriptions({ config, port, secret, eventsUrl = config.events.url, fetchImpl = globalThis.fetch, log = console }) {++ if (!eventsUrl || !secret || !config.oauth.clientSecret || process.env.RENT_EVENTS_SUBSCRIBE === '0') return null;++ const base = String(eventsUrl).replace(/\/+$/, '');++ const endpoint = config.events.endpoint || `http://127.0.0.1:${port}/internal/events`;++ const tokens = serviceAuth.createTokenClient({ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, audience: 'openvibe.events', scope: 'events.subscription.manage', fetchImpl });++ const call = async (method, path, body) => {++ const res = await fetchImpl(`${base}${path}`, { method, headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokens.authHeaders()) }, body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(15000) });++ const json = await res.json().catch(() => ({}));++ return { status: res.status, ok: res.ok, body: json };++ };++ const attempt = async () => {++ const listed = await call('GET', '/api/v1/subscriptions');++ if (!listed.ok) throw new Error(`listing subscriptions: ${listed.status}`);++ const mine = (listed.body.subscriptions || []).filter((s) => s.endpoint === endpoint);++ for (const topic of TOPICS) {++ if (mine.some((s) => s.topic_pattern === topic)) continue;++ const r = await call('POST', '/api/v1/subscriptions', { topic_pattern: topic, endpoint, secret });++ if (!r.ok && r.status !== 409) throw new Error(`subscribing to ${topic}: ${r.status} ${r.body.code || ''}`);++ if (r.ok) log.log(`[Events consumer] subscription created: ${r.body.id} (${topic} → ${endpoint})`);++ }++ };++ const delays = [0, 10_000, 60_000, 5 * 60_000, 15 * 60_000];++ let i = 0;++ let timer = null;++ let stopped = false;++ const run = () => { timer = null; if (stopped) return; attempt().catch((err) => {++ if (stopped) return;++ if (++i < delays.length) { timer = setTimeout(run, delays[i]); if (timer.unref) timer.unref(); } else log.warn('[Events consumer] subscriptions not created:', err.message);++ }); };++ timer = setTimeout(run, delays[0]); if (timer.unref) timer.unref();++ // Graceful stop: no further attempts (they run again at the next start).++ return { topics: TOPICS, endpoint, stop() { stopped = true; if (timer) clearTimeout(timer); timer = null; } };++}++++module.exports = { startSubscriptions, TOPICS };nnew file mode 100644@@@ -0,0 +1,39 @@++'use strict';++++/**++ * Account export and deletion → Rent (ADR-033; openvibe-sdk/account-data). Rent holds three things a person made:++ * a listing they posted, the reports they filed against listings, and the searches they saved. All three key the++ * person as `user:usr_…` — server/http/principal.js builds the principal's requester that way and server/http/api.js++ * writes it straight into the store (server/listings/store.js).++ *++ * network.account.export_requested the person's listings (listings.json), reports (reports.json) and saved++ * searches (saved_searches.json), newest first, pushed to Network++ * (POST /internal/account-exports/:id/parts) with this service's token.++ * network.account.deleted the person's listings, reports and saved searches go, and Rent confirms with++ * the counts.++ *++ * Nothing is anonymized: Rent keeps no row by this person that another person's page must still read. A listing is++ * the poster's own; the reports other people left on it cascade away with it (rent_reports.listing_id REFERENCES++ * rent_listings(id) ON DELETE CASCADE in migrations/0002_rent.sql), which is right — a report about a listing that no++ * longer exists has nothing left to say. Nothing here is a secret: no token, key or credential is stored, so every++ * column of the three tables may be exported.++ */++const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');++++/**++ * The tables that hold a person's rows, with the value the subject column really stores. Listings key on `owner`,++ * reports on `reporter` and saved searches on `subject`; all three store `user:usr_…` (server/listings/store.js takes++ * the value straight from the principal's requester).++ */++const TABLES = [++ { table: 'rent_listings', subject: 'owner', value: (usr) => `user:${usr}`, file: 'listings.json' },++ { table: 'rent_reports', subject: 'reporter', value: (usr) => `user:${usr}`, file: 'reports.json' },++ { table: 'rent_saved_searches', subject: 'subject', value: (usr) => `user:${usr}`, file: 'saved_searches.json' },++];++++/** The account-data handle for Rent's store (server/db.js createStore). */++function create({ db, note = 'A listing names a place by its city, never a street address; a saved search is a filter set.', log = console } = {}) {++ return createAccountData({ db, service: 'rent', tables: TABLES, note, log });++}++++module.exports = { create, TABLES, TOPICS };@@@ -12,13 +12,14 @@ const { gracefulStop } = require('openvibe-sdk/service');const { startExpiryTimer } = require('./listings/expiry');/**-- * The process stop (openvibe-sdk/service): the HTTP drain runs, then the JWKS refresher stops and the store closes.-- * Exported so a test can inject `exit` and `signals: false`.++ * The process stop (openvibe-sdk/service): the HTTP drain runs, then the timers clear, the Events subscriptions stop,++ * the JWKS refresher stops and the store closes. Exported so a test can inject `exit` and `signals: false`. `extra` is++ * what the product adds (the Events subscriptions); a test that passes none keeps the skeleton's order.*/--function createLifecycle({ server, ctx, exit, signals, timers = [] }) {++function createLifecycle({ server, ctx, exit, signals, timers = [], extra = [] }) {return gracefulStop({name: 'OpenVibe.Rent', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,-- close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.keys.client.stop(), () => ctx.s.close()],++ close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.keys.client.stop(), () => ctx.s.close(), ...extra],});}@@@ -36,7 +37,11 @@ async function start() {const expiry = startExpiryTimer(ctx.s);await expiry.tick();-- createLifecycle({ server, ctx, timers: [expiry.timer] });++ // Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without RENT_EVENTS_URL and++ // RENT_EVENTS_SECRET). The consumer itself is mounted in server/app.js.++ const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });++ const extra = [() => { if (subscriptions) subscriptions.stop(); }];++ createLifecycle({ server, ctx, timers: [expiry.timer], extra });return { server, ctx, expiry };}nnew file mode 100644@@@ -0,0 +1,172 @@++'use strict';++/**++ * ADR-033: Rent's part of an account export (listings, reports and saved searches) and of an account deletion,++ * applied through the service's own /internal/events route with a stand-in Network. Real rows are made through the++ * service's provider (server/listings/service.js) and store (server/listings/store.js), the signed delivery is++ * answered by openvibe-sdk/account-data, and the part only ever carries person A's rows. A redelivery erases nothing++ * twice, and the route refuses a bad signature and a forwarded request.++ */++const assert = require('assert');++const http = require('http');++const { boot, check, done } = require('./helpers/boot');++const { createNetworkSender } = require('openvibe-sdk/account-data');++const { signDeliveryHeaders } = require('openvibe-sdk/events');++const rentStore = require('../server/listings/store');++const rentService = require('../server/listings/service');++const listings = require('../server/listings/listings');++++const A = 'usr_01JZ0000000000000000000AAA';++const B = 'usr_01JZ0000000000000000000BBB';++const OLD = 'usr_01JZ0000000000000000000MRG';++const EXP = 'exp_01JZ0000000000000000000EXP';++const DEL = 'del_01JZ0000000000000000000DEX';++// Fixture secrets, built so they never look like a real key to a scanner.++const SECRET = `whsec_${'fixture'.repeat(6)}`;++const WRONG = `whsec_${'mismatch'.repeat(5)}`;++++const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };++const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };++const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });++++/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */++async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {++ const calls = [];++ const statusOf = (v) => (typeof v === 'function' ? v() : v);++ const server = http.createServer((req, res) => {++ const chunks = [];++ req.on('data', (c) => chunks.push(c));++ req.on('end', () => {++ const raw = Buffer.concat(chunks);++ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };++ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_rent', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });++ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });++ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});++ });++ });++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };++}++++/** A listing's fields, as the form and the API validate them (server/listings/listings.js validate). */++const fields = (title) => ({++ kind: 'apartment', title, description: `A quiet place: ${title}`,++ price: 1200, currency: 'USD', period: 'month',++ city: 'Shoreditch', region: null, country: 'GB', neighbourhood: null,++ bedrooms: 2, availableFrom: null, contactUrl: null,++});++++/** Post a listing for one person, through the service the API writes with (server/listings/service.js). */++async function makeListing(t, subject, title) {++ const r = await rentService.create(t.ctx.s, `user:${subject}`, fields(title));++ assert.ok(r.ok, `listing for ${subject}: ${r.error || r.code || ''}`);++ return r.row;++}++++/** Save one search for a person, through the store the API writes with (server/listings/store.js). */++function makeSearch(t, subject, city) {++ const s = t.ctx.s;++ const filters = listings.normalizeFilters({ city });++ return rentStore.saveSearch(s, { id: s.newId('ssv'), subject: `user:${subject}`, filters, key: listings.filtersKey(filters), createdAt: s.iso() });++}++++/** File one report, through the store the API writes with (server/listings/store.js). */++async function makeReport(t, listingId, subject, note) {++ const s = t.ctx.s;++ const r = await rentStore.addReport(s, listingId, { id: s.newId('rpt'), reporter: `user:${subject}`, reason: 'scam', note, createdAt: s.iso() });++ assert.ok(r.report, `report by ${subject}: ${r.error || ''}`);++ return r.report;++}++++/**++ * A listing, a saved search and a report each for A and B. The reports cross: A's is on B's listing and B's is on++ * A's — so A's part must carry A's report and not B's, and deleting A takes B's report on A's listing with it++ * (rent_reports.listing_id REFERENCES rent_listings(id) ON DELETE CASCADE), which is the documented cascade.++ */++async function seed(t) {++ const a = await makeListing(t, A, 'person-a');++ const b = await makeListing(t, B, 'person-b');++ await makeSearch(t, A, 'Aton');++ await makeSearch(t, B, 'Bville');++ await makeReport(t, b.id, A, 'a-report-note');++ await makeReport(t, a.id, B, 'b-report-note');++ return { a, b };++}++++(async () => {++ await check('an export part carries only the person\'s listings, reports and saved searches, and no secret', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });++ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ await seed(t);++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'exported');++++ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);++ assert.ok(part, 'the part was pushed to Network');++ assert.strictEqual(part.auth, 'Bearer tok_rent', 'with this service\'s own token');++ assert.strictEqual(part.body.subject, A);++ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['listings.json', 'reports.json', 'saved_searches.json']);++ const body = JSON.stringify(part.body);++ assert.ok(body.includes('person-a'), 'person A\'s listing');++ assert.ok(body.includes('a-report-note'), 'person A\'s report');++ assert.ok(body.includes('Aton'), 'person A\'s saved search');++ assert.ok(!body.includes('person-b'), 'nobody else\'s listing');++ assert.ok(!body.includes('b-report-note'), 'nobody else\'s report');++ assert.ok(!body.includes('Bville'), 'nobody else\'s saved search');++ assert.ok(!/token|secret|password/i.test(body), 'no secret is exported');++ } finally { await t.close(); await stub.close(); }++ });++++ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });++ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ const { a } = await seed(t);++ const count = async (sql, subject) => Number(await t.ctx.s.db.value(sql, [`user:${subject}`]));++ const listingsFor = (subject) => count('SELECT count(*)::int FROM rent_listings WHERE owner = $1', subject);++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'erased');++ assert.strictEqual(await listingsFor(A), 0, 'the person\'s listing is gone');++ assert.strictEqual(await count('SELECT count(*)::int FROM rent_reports WHERE reporter = $1', A), 0, 'the person\'s report is gone');++ assert.strictEqual(await count('SELECT count(*)::int FROM rent_saved_searches WHERE subject = $1', A), 0, 'the person\'s saved search is gone');++ assert.strictEqual(await listingsFor(B), 1, 'someone else\'s listing stays');++ assert.strictEqual(await count('SELECT count(*)::int FROM rent_saved_searches WHERE subject = $1', B), 1, 'someone else\'s saved search stays');++ // The report B left on A's listing cascades away with the listing it named.++ assert.strictEqual(await t.ctx.s.db.value("SELECT count(*)::int FROM rent_reports WHERE listing_id = $1", [a.id]), 0);++++ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);++ assert.ok(confirmation, 'the confirmation was sent');++ assert.deepStrictEqual(confirmation.body.erased, { rent_listings: 1, rent_reports: 1, rent_saved_searches: 1 });++ assert.deepStrictEqual(confirmation.body.retained, {});++ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));++++ // A redelivery must not erase again: rows written after the deletion stay.++ await makeListing(t, A, 'written-later');++ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(again.status, 200);++ assert.strictEqual(again.json().outcome, 'unchanged');++ assert.strictEqual(await listingsFor(A), 1, 'nothing was erased twice');++ } finally { await t.close(); await stub.close(); }++ });++++ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });++ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });++ assert.strictEqual(bad.status, 401);++++ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(forwarded.status, 403);++ } finally { await t.close(); await stub.close(); }++ });++++ done();++})();@@@ -38,7 +38,7 @@ async function boot(opts = {}) {const { createStore } = require('../../server/db');const testdb = await require('./db').testDb();const store = createStore(testdb.db, { now: opts.now });-- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });++ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });await built.ctx.keys.ensure();const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;
@@ -30,5 +30,12 @@ OV_OAUTH_CLIENT_SECRET=
@@ -30,5 +30,12 @@ OV_OAUTH_CLIENT_SECRET=
# OV_OAUTH_REDIRECT_URI=https://openvibe.rent/auth/callback
# OV_OAUTH_REDIRECT_URI=https://openvibe.rent/auth/callback
# RENT_AUDIENCE=openvibe.rent
# RENT_AUDIENCE=openvibe.rent
+# OpenVibe.Events → Rent (ADR-033 account export and deletion): the secret signs a delivery (comma-separated for
+# rotation, 32+ characters each; unset makes POST /internal/events answer 503); the url is where the two subscriptions
+# are created at boot (off when unset); RENT_EVENTS_SUBSCRIBE=0 turns that off.
+RENT_EVENTS_SECRET=
+RENT_EVENTS_URL=
+#RENT_EVENTS_ENDPOINT=http://127.0.0.1:5010/internal/events
+
# optional: defaults to true in production
# optional: defaults to true in production
# COOKIE_SECURE=true
# COOKIE_SECURE=true
@@ -84,8 +84,9 @@ Network), `BASE_URL`, `DATABASE_URL` and `DATABASE_DIRECT_URL`. The database is
@@ -84,8 +84,9 @@ Network), `BASE_URL`, `DATABASE_URL` and `DATABASE_DIRECT_URL`. The database is
the Network signing key, the OAuth client and Valkey are optional (the service says so, per check, on
the Network signing key, the OAuth client and Valkey are optional (the service says so, per check, on
`/api/ready`).
`/api/ready`).
-The product adds no environment variables of its own: its numbers (10 active listings, 5 a day, 30 days, 3 reports,
-30 a page, the per-caller budgets) are constants next to the rules they belong to, not deployment settings.
+The product adds no environment variables of its own beyond the OpenVibe.Events ones below: its numbers (10 active
+listings, 5 a day, 30 days, 3 reports, 30 a page, the per-caller budgets) are constants next to the rules they belong
+to, not deployment settings.
## Development and tests
## Development and tests
@@ -110,6 +111,25 @@ through PostgreSQL and PgBouncer (see [.github/workflows/ci.yml](.github/workflo
@@ -110,6 +111,25 @@ through PostgreSQL and PgBouncer (see [.github/workflows/ci.yml](.github/workflo
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: rent` in CI) and with
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: rent` in CI) and with
**OpenVibe.Services** before the first deploy.
**OpenVibe.Services** before the first deploy.
+## Account export and deletion
+
+A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its
+part (ADR-033). Rent receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`
+(loopback only) — the three tables are mapped in [server/identity/account-data.js](server/identity/account-data.js), and
+the boot-time subscriptions are created by [server/events-consumer.js](server/events-consumer.js):
+
+- **Exported:** the listings a person posted (`listings.json`), the reports they filed (`reports.json`) and the searches
+ they saved (`saved_searches.json`), pushed to `POST /internal/account-exports/:id/parts` with this service's own
+ token. Nothing here is a secret — Rent stores no token, key or credential.
+- **Erased:** all three tables hold the person's own rows, so they are deleted whole and nothing is kept. Rent then
+ confirms with `POST /internal/account-deletions/:id/confirmations` and the counts.
+- **Anonymized:** nothing. Rent keeps no row by this person that another person's page must still read — the reports
+ other people left on a deleted listing cascade away with it.
+
+Environment: `RENT_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer
+503), `RENT_EVENTS_URL` (or `EVENTS_URL`) is where the two subscriptions are created at boot (off when unset), and
+`RENT_EVENTS_ENDPOINT` overrides the loopback endpoint; `RENT_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.
+
## Security (threat notes)
## Security (threat notes)
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
@@ -7,11 +7,11 @@
@@ -7,11 +7,11 @@
"port": "5010",
"port": "5010",
"domain": "openvibe.rent",
"domain": "openvibe.rent",
"contracts": "openvibe-contracts v0.115.0",
"contracts": "openvibe-contracts v0.115.0",
- "sdk": "openvibe-sdk v0.35.2",
+ "sdk": "openvibe-sdk v0.36.0",
"shared": "openvibe-shared v2.14.0",
"shared": "openvibe-shared v2.14.0",
"packages": [
"packages": [
"openvibe-contracts v0.115.0",
"openvibe-contracts v0.115.0",
- "openvibe-sdk v0.35.2",
+ "openvibe-sdk v0.36.0",
"openvibe-shared v2.14.0"
"openvibe-shared v2.14.0"
],
],
"summary": "OpenVibe.Rent — Find a place, or rent yours out. One place to find and offer things to rent: apartments, rooms, houses, commercial space, parking and storage, and equipment. In this first version every listing is posted by a person on this site: no scraped listings and no partner feeds, no payments, no deposits and no messaging. Search with filters and facets, a listing page with a plain safety note, reports that hide a listing at 3 until staff look, saved searches that count what is new, and a 30-day expiry an owner can renew.",
"summary": "OpenVibe.Rent — Find a place, or rent yours out. One place to find and offer things to rent: apartments, rooms, houses, commercial space, parking and storage, and equipment. In this first version every listing is posted by a person on this site: no scraped listings and no partner feeds, no payments, no deposits and no messaging. Search with filters and facets, a listing page with a plain safety note, reports that hide a listing at 3 until staff look, saved searches that count what is new, and a 30-day expiry an owner can renew.",
@@ -29,7 +29,8 @@
@@ -29,7 +29,8 @@
"Pages: /, /listings, /listings/:id, /listings/:id/edit, /post, /mine, /saved, /safety, /staff, /updates — server-rendered, complete without JavaScript, forms are plain POSTs",
"Pages: /, /listings, /listings/:id, /listings/:id/edit, /post, /mine, /saved, /safety, /staff, /updates — server-rendered, complete without JavaScript, forms are plain POSTs",
"Same-origin writes for the session cookie, and per-caller budgets shared by the form and the API route",
"Same-origin writes for the session cookie, and per-caller budgets shared by the form and the API route",
"Discovery: robots.txt (disallows /auth/, /api/, /post, /mine, /saved, /staff), sitemap.xml with every active listing (≤1000), llms.txt, llms-full.txt",
"Discovery: robots.txt (disallows /auth/, /api/, /post, /mine, /saved, /staff), sitemap.xml with every active listing (≤1000), llms.txt, llms-full.txt",
- "Tests: posting rules, search and paging and facets, contact hidden from signed-out viewers, report → hide at 3 → staff restore, expiry, owner-only edits, same-origin writes, saved searches, hostile text escaped"
+ "Tests: posting rules, search and paging and facets, contact hidden from signed-out viewers, report → hide at 3 → staff restore, expiry, owner-only edits, same-origin writes, saved searches, hostile text escaped",
+ "Account export and deletion (ADR-033): the listings a person posted, the reports they filed and the searches they saved are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted (a report others left on a deleted listing cascades away with it), with the counts confirmed back; the two Events topics are subscribed at boot (server/identity/account-data.js)"
],
],
"notYet": [
"notYet": [
"Listing sources beyond people posting here: scraped and partner feeds arrive only when their terms are verified, and then each listing will carry its provenance (source name, link back, fetched_at)",
"Listing sources beyond people posting here: scraped and partner feeds arrive only when their terms are verified, and then each listing will carry its provenance (source name, link back, fetched_at)",
@@ -101,6 +101,10 @@ server {
@@ -101,6 +101,10 @@ server {
# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:5010/metrics) ──
# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:5010/metrics) ──
location = /metrics { return 404; }
location = /metrics { return 404; }
+ # ── OpenVibe.Events' delivery route: loopback only (POST /internal/events) ──
+ # Events posts to http://127.0.0.1:5010/internal/events directly. Never proxied from outside (ADR-033).
+ location /internal/ { return 404; }
+
# ── Pages, docs, robots.txt, sitemap, static assets ─────
# ── Pages, docs, robots.txt, sitemap, static assets ─────
location / {
location / {
proxy_pass http://127.0.0.1:5010;
proxy_pass http://127.0.0.1:5010;
new file mode 100644
new file mode 100644
@@ -0,0 +1,13 @@
@@ -0,0 +1,13 @@
+-- phase: expand
+-- OpenVibe.Rent: the receipts of the ADR-033 account export and deletion deliveries this service applied
+-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.
+-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.
+
+CREATE TABLE IF NOT EXISTS account_data_events (
+ id TEXT PRIMARY KEY,
+ kind TEXT NOT NULL,
+ subject TEXT NOT NULL,
+ outcome JSONB,
+ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+ sent_at TIMESTAMPTZ
+);
@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');
@@ -30,6 +30,8 @@ const { createApi } = require('./http/api');
const { createPageRoutes } = require('./http/pages');
const { createPageRoutes } = require('./http/pages');
const { createServiceReadiness } = require('./observability');
const { createServiceReadiness } = require('./observability');
const { createCallerLimits } = require('./http/caller-limits');
const { createCallerLimits } = require('./http/caller-limits');
+const accountDataLib = require('./identity/account-data');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
const { assetVersion, send } = require('./render/layout');
const { assetVersion, send } = require('./render/layout');
const { html } = require('./render/html');
const { html } = require('./render/html');
@@ -38,7 +40,7 @@ const VERSION = require('../package.json').version;
@@ -38,7 +40,7 @@ const VERSION = require('../package.json').version;
/**
/**
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
- * tests only), valkey
+ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)
*/
*/
async function createApp(opts = {}) {
async function createApp(opts = {}) {
const config = opts.config || configLib.load();
const config = opts.config || configLib.load();
@@ -51,6 +53,18 @@ async function createApp(opts = {}) {
@@ -51,6 +53,18 @@ async function createApp(opts = {}) {
const principal = createPrincipal({ config, keys });
const principal = createPrincipal({ config, keys });
const ctx = { config, s, keys, sso, principal, log };
const ctx = { config, s, keys, sso, principal, log };
+ // Account export and deletion (ADR-033, ./identity/account-data.js): the three tables that hold a person's rows.
+ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;
+ // a test injects a stand-in through opts.accountSend.
+ const accountData = accountDataLib.create({ db: s.db, log });
+ // Without a client secret the service has no way to push a part or a confirmation: the route still answers
+ // (bad signature, a forwarded request, no secret), and an event that really arrives asks Events to retry.
+ const accountSend = opts.accountSend || (config.oauth.clientSecret
+ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })
+ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Rent cannot answer account events'); });
+ ctx.accountData = accountData;
+ ctx.accountSend = accountSend;
+
const app = express();
const app = express();
app.disable('x-powered-by');
app.disable('x-powered-by');
app.set('trust proxy', config.trustProxy);
app.set('trust proxy', config.trustProxy);
@@ -102,6 +116,12 @@ async function createApp(opts = {}) {
@@ -102,6 +116,12 @@ async function createApp(opts = {}) {
app.use('/auth', sso.routes());
app.use('/auth', sso.routes());
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'rent', service: 'rent', host: 'openvibe.rent', name: 'OpenVibe.Rent', profile: 'ugc' })); }
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'rent', service: 'rent', host: 'openvibe.rent', name: 'OpenVibe.Rent', profile: 'ugc' })); }
+ // ── OpenVibe.Events → this service (loopback only) ──────
+ // network.account.export_requested and network.account.deleted (ADR-033), answered by the SDK's consumer. It
+ // reads the raw body itself, so it is mounted before any body parser (there is none above it), and nginx answers
+ // 404 for /internal/ so this is reachable only on 127.0.0.1.
+ app.use('/internal/events', accountData.consumer({ secrets: config.events.secrets, send: accountSend, log }));
+
// ── Static assets (content-hashed ?v= → immutable) ──────
// ── Static assets (content-hashed ?v= → immutable) ──────
app.use('/shared', require('openvibe-shared/serve').handler());
app.use('/shared', require('openvibe-shared/serve').handler());
app.use(express.static(PUBLIC_DIR, {
app.use(express.static(PUBLIC_DIR, {
@@ -52,6 +52,16 @@ function load(env = process.env) {
@@ -52,6 +52,16 @@ function load(env = process.env) {
sessionAudience: env.OV_SESSION_AUDIENCE || 'openvibe.network',
sessionAudience: env.OV_SESSION_AUDIENCE || 'openvibe.network',
},
},
cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },
cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },
+
+ // OpenVibe.Events → this service (server/events-consumer.js). The secret signs a delivery (comma-separated
+ // for rotation, 32+ characters each); unset turns POST /internal/events off (503). The url is where the
+ // network.account.export_requested and network.account.deleted subscriptions are created at boot, off when
+ // unset. Both topics are ADR-033: account export and deletion (server/identity/account-data.js).
+ events: {
+ secrets: String(env.RENT_EVENTS_SECRET || '').split(',').map((x) => x.trim()).filter(Boolean),
+ url: trim(env.RENT_EVENTS_URL || env.EVENTS_URL || ''),
+ endpoint: env.RENT_EVENTS_ENDPOINT || '',
+ },
};
};
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,52 @@
@@ -0,0 +1,52 @@
+'use strict';
+
+/**
+ * OpenVibe.Events → Rent. Rent has no consumer of its own: the two topics it takes are account export and deletion
+ * (ADR-033), and POST /internal/events is answered by openvibe-sdk/account-data's own consumer, mounted in
+ * server/app.js over server/identity/account-data.js (the table map). What is left here is the boot-time half:
+ * startSubscriptions() creates the two subscriptions at Events if they are missing, idempotently, retried with
+ * backoff, and off when the URL or the secret is unset (QUEST_EVENTS_SUBSCRIBE=0 style: RENT_EVENTS_SUBSCRIBE=0).
+ *
+ * The endpoint is this service's loopback /internal/events (Events posts to http://127.0.0.1:<port> directly; nginx
+ * answers 404 there), and the token is this service's own for audience openvibe.events with
+ * scope events.subscription.manage.
+ */
+const { serviceAuth } = require('openvibe-contracts');
+const { TOPICS } = require('./identity/account-data');
+
+/** Create any missing subscription for TOPICS at Events (idempotent; retried in the background at boot). */
+function startSubscriptions({ config, port, secret, eventsUrl = config.events.url, fetchImpl = globalThis.fetch, log = console }) {
+ if (!eventsUrl || !secret || !config.oauth.clientSecret || process.env.RENT_EVENTS_SUBSCRIBE === '0') return null;
+ const base = String(eventsUrl).replace(/\/+$/, '');
+ const endpoint = config.events.endpoint || `http://127.0.0.1:${port}/internal/events`;
+ const tokens = serviceAuth.createTokenClient({ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, audience: 'openvibe.events', scope: 'events.subscription.manage', fetchImpl });
+ const call = async (method, path, body) => {
+ const res = await fetchImpl(`${base}${path}`, { method, headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokens.authHeaders()) }, body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(15000) });
+ const json = await res.json().catch(() => ({}));
+ return { status: res.status, ok: res.ok, body: json };
+ };
+ const attempt = async () => {
+ const listed = await call('GET', '/api/v1/subscriptions');
+ if (!listed.ok) throw new Error(`listing subscriptions: ${listed.status}`);
+ const mine = (listed.body.subscriptions || []).filter((s) => s.endpoint === endpoint);
+ for (const topic of TOPICS) {
+ if (mine.some((s) => s.topic_pattern === topic)) continue;
+ const r = await call('POST', '/api/v1/subscriptions', { topic_pattern: topic, endpoint, secret });
+ if (!r.ok && r.status !== 409) throw new Error(`subscribing to ${topic}: ${r.status} ${r.body.code || ''}`);
+ if (r.ok) log.log(`[Events consumer] subscription created: ${r.body.id} (${topic} → ${endpoint})`);
+ }
+ };
+ const delays = [0, 10_000, 60_000, 5 * 60_000, 15 * 60_000];
+ let i = 0;
+ let timer = null;
+ let stopped = false;
+ const run = () => { timer = null; if (stopped) return; attempt().catch((err) => {
+ if (stopped) return;
+ if (++i < delays.length) { timer = setTimeout(run, delays[i]); if (timer.unref) timer.unref(); } else log.warn('[Events consumer] subscriptions not created:', err.message);
+ }); };
+ timer = setTimeout(run, delays[0]); if (timer.unref) timer.unref();
+ // Graceful stop: no further attempts (they run again at the next start).
+ return { topics: TOPICS, endpoint, stop() { stopped = true; if (timer) clearTimeout(timer); timer = null; } };
+}
+
+module.exports = { startSubscriptions, TOPICS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,39 @@
@@ -0,0 +1,39 @@
+'use strict';
+
+/**
+ * Account export and deletion → Rent (ADR-033; openvibe-sdk/account-data). Rent holds three things a person made:
+ * a listing they posted, the reports they filed against listings, and the searches they saved. All three key the
+ * person as `user:usr_…` — server/http/principal.js builds the principal's requester that way and server/http/api.js
+ * writes it straight into the store (server/listings/store.js).
+ *
+ * network.account.export_requested the person's listings (listings.json), reports (reports.json) and saved
+ * searches (saved_searches.json), newest first, pushed to Network
+ * (POST /internal/account-exports/:id/parts) with this service's token.
+ * network.account.deleted the person's listings, reports and saved searches go, and Rent confirms with
+ * the counts.
+ *
+ * Nothing is anonymized: Rent keeps no row by this person that another person's page must still read. A listing is
+ * the poster's own; the reports other people left on it cascade away with it (rent_reports.listing_id REFERENCES
+ * rent_listings(id) ON DELETE CASCADE in migrations/0002_rent.sql), which is right — a report about a listing that no
+ * longer exists has nothing left to say. Nothing here is a secret: no token, key or credential is stored, so every
+ * column of the three tables may be exported.
+ */
+const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');
+
+/**
+ * The tables that hold a person's rows, with the value the subject column really stores. Listings key on `owner`,
+ * reports on `reporter` and saved searches on `subject`; all three store `user:usr_…` (server/listings/store.js takes
+ * the value straight from the principal's requester).
+ */
+const TABLES = [
+ { table: 'rent_listings', subject: 'owner', value: (usr) => `user:${usr}`, file: 'listings.json' },
+ { table: 'rent_reports', subject: 'reporter', value: (usr) => `user:${usr}`, file: 'reports.json' },
+ { table: 'rent_saved_searches', subject: 'subject', value: (usr) => `user:${usr}`, file: 'saved_searches.json' },
+];
+
+/** The account-data handle for Rent's store (server/db.js createStore). */
+function create({ db, note = 'A listing names a place by its city, never a street address; a saved search is a filter set.', log = console } = {}) {
+ return createAccountData({ db, service: 'rent', tables: TABLES, note, log });
+}
+
+module.exports = { create, TABLES, TOPICS };
@@ -12,13 +12,14 @@ const { gracefulStop } = require('openvibe-sdk/service');
@@ -12,13 +12,14 @@ const { gracefulStop } = require('openvibe-sdk/service');
const { startExpiryTimer } = require('./listings/expiry');
const { startExpiryTimer } = require('./listings/expiry');
/**
/**
- * The process stop (openvibe-sdk/service): the HTTP drain runs, then the JWKS refresher stops and the store closes.
- * Exported so a test can inject `exit` and `signals: false`.
+ * The process stop (openvibe-sdk/service): the HTTP drain runs, then the timers clear, the Events subscriptions stop,
+ * the JWKS refresher stops and the store closes. Exported so a test can inject `exit` and `signals: false`. `extra` is
+ * what the product adds (the Events subscriptions); a test that passes none keeps the skeleton's order.
*/
*/
-function createLifecycle({ server, ctx, exit, signals, timers = [] }) {
+function createLifecycle({ server, ctx, exit, signals, timers = [], extra = [] }) {
return gracefulStop({
return gracefulStop({
name: 'OpenVibe.Rent', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,
name: 'OpenVibe.Rent', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,
- close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.keys.client.stop(), () => ctx.s.close()],
+ close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.keys.client.stop(), () => ctx.s.close(), ...extra],
});
});
}
}
@@ -36,7 +37,11 @@ async function start() {
@@ -36,7 +37,11 @@ async function start() {
const expiry = startExpiryTimer(ctx.s);
const expiry = startExpiryTimer(ctx.s);
await expiry.tick();
await expiry.tick();
- createLifecycle({ server, ctx, timers: [expiry.timer] });
+ // Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without RENT_EVENTS_URL and
+ // RENT_EVENTS_SECRET). The consumer itself is mounted in server/app.js.
+ const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });
+ const extra = [() => { if (subscriptions) subscriptions.stop(); }];
+ createLifecycle({ server, ctx, timers: [expiry.timer], extra });
return { server, ctx, expiry };
return { server, ctx, expiry };
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,172 @@
@@ -0,0 +1,172 @@
+'use strict';
+/**
+ * ADR-033: Rent's part of an account export (listings, reports and saved searches) and of an account deletion,
+ * applied through the service's own /internal/events route with a stand-in Network. Real rows are made through the
+ * service's provider (server/listings/service.js) and store (server/listings/store.js), the signed delivery is
+ * answered by openvibe-sdk/account-data, and the part only ever carries person A's rows. A redelivery erases nothing
+ * twice, and the route refuses a bad signature and a forwarded request.
+ */
+const assert = require('assert');
+const http = require('http');
+const { boot, check, done } = require('./helpers/boot');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
+const { signDeliveryHeaders } = require('openvibe-sdk/events');
+const rentStore = require('../server/listings/store');
+const rentService = require('../server/listings/service');
+const listings = require('../server/listings/listings');
+
+const A = 'usr_01JZ0000000000000000000AAA';
+const B = 'usr_01JZ0000000000000000000BBB';
+const OLD = 'usr_01JZ0000000000000000000MRG';
+const EXP = 'exp_01JZ0000000000000000000EXP';
+const DEL = 'del_01JZ0000000000000000000DEX';
+// Fixture secrets, built so they never look like a real key to a scanner.
+const SECRET = `whsec_${'fixture'.repeat(6)}`;
+const WRONG = `whsec_${'mismatch'.repeat(5)}`;
+
+const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };
+const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };
+const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });
+
+/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */
+async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {
+ const calls = [];
+ const statusOf = (v) => (typeof v === 'function' ? v() : v);
+ const server = http.createServer((req, res) => {
+ const chunks = [];
+ req.on('data', (c) => chunks.push(c));
+ req.on('end', () => {
+ const raw = Buffer.concat(chunks);
+ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };
+ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_rent', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });
+ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });
+ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});
+ });
+ });
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };
+}
+
+/** A listing's fields, as the form and the API validate them (server/listings/listings.js validate). */
+const fields = (title) => ({
+ kind: 'apartment', title, description: `A quiet place: ${title}`,
+ price: 1200, currency: 'USD', period: 'month',
+ city: 'Shoreditch', region: null, country: 'GB', neighbourhood: null,
+ bedrooms: 2, availableFrom: null, contactUrl: null,
+});
+
+/** Post a listing for one person, through the service the API writes with (server/listings/service.js). */
+async function makeListing(t, subject, title) {
+ const r = await rentService.create(t.ctx.s, `user:${subject}`, fields(title));
+ assert.ok(r.ok, `listing for ${subject}: ${r.error || r.code || ''}`);
+ return r.row;
+}
+
+/** Save one search for a person, through the store the API writes with (server/listings/store.js). */
+function makeSearch(t, subject, city) {
+ const s = t.ctx.s;
+ const filters = listings.normalizeFilters({ city });
+ return rentStore.saveSearch(s, { id: s.newId('ssv'), subject: `user:${subject}`, filters, key: listings.filtersKey(filters), createdAt: s.iso() });
+}
+
+/** File one report, through the store the API writes with (server/listings/store.js). */
+async function makeReport(t, listingId, subject, note) {
+ const s = t.ctx.s;
+ const r = await rentStore.addReport(s, listingId, { id: s.newId('rpt'), reporter: `user:${subject}`, reason: 'scam', note, createdAt: s.iso() });
+ assert.ok(r.report, `report by ${subject}: ${r.error || ''}`);
+ return r.report;
+}
+
+/**
+ * A listing, a saved search and a report each for A and B. The reports cross: A's is on B's listing and B's is on
+ * A's — so A's part must carry A's report and not B's, and deleting A takes B's report on A's listing with it
+ * (rent_reports.listing_id REFERENCES rent_listings(id) ON DELETE CASCADE), which is the documented cascade.
+ */
+async function seed(t) {
+ const a = await makeListing(t, A, 'person-a');
+ const b = await makeListing(t, B, 'person-b');
+ await makeSearch(t, A, 'Aton');
+ await makeSearch(t, B, 'Bville');
+ await makeReport(t, b.id, A, 'a-report-note');
+ await makeReport(t, a.id, B, 'b-report-note');
+ return { a, b };
+}
+
+(async () => {
+ await check('an export part carries only the person\'s listings, reports and saved searches, and no secret', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });
+ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ await seed(t);
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'exported');
+
+ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);
+ assert.ok(part, 'the part was pushed to Network');
+ assert.strictEqual(part.auth, 'Bearer tok_rent', 'with this service\'s own token');
+ assert.strictEqual(part.body.subject, A);
+ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['listings.json', 'reports.json', 'saved_searches.json']);
+ const body = JSON.stringify(part.body);
+ assert.ok(body.includes('person-a'), 'person A\'s listing');
+ assert.ok(body.includes('a-report-note'), 'person A\'s report');
+ assert.ok(body.includes('Aton'), 'person A\'s saved search');
+ assert.ok(!body.includes('person-b'), 'nobody else\'s listing');
+ assert.ok(!body.includes('b-report-note'), 'nobody else\'s report');
+ assert.ok(!body.includes('Bville'), 'nobody else\'s saved search');
+ assert.ok(!/token|secret|password/i.test(body), 'no secret is exported');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });
+ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ const { a } = await seed(t);
+ const count = async (sql, subject) => Number(await t.ctx.s.db.value(sql, [`user:${subject}`]));
+ const listingsFor = (subject) => count('SELECT count(*)::int FROM rent_listings WHERE owner = $1', subject);
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'erased');
+ assert.strictEqual(await listingsFor(A), 0, 'the person\'s listing is gone');
+ assert.strictEqual(await count('SELECT count(*)::int FROM rent_reports WHERE reporter = $1', A), 0, 'the person\'s report is gone');
+ assert.strictEqual(await count('SELECT count(*)::int FROM rent_saved_searches WHERE subject = $1', A), 0, 'the person\'s saved search is gone');
+ assert.strictEqual(await listingsFor(B), 1, 'someone else\'s listing stays');
+ assert.strictEqual(await count('SELECT count(*)::int FROM rent_saved_searches WHERE subject = $1', B), 1, 'someone else\'s saved search stays');
+ // The report B left on A's listing cascades away with the listing it named.
+ assert.strictEqual(await t.ctx.s.db.value("SELECT count(*)::int FROM rent_reports WHERE listing_id = $1", [a.id]), 0);
+
+ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);
+ assert.ok(confirmation, 'the confirmation was sent');
+ assert.deepStrictEqual(confirmation.body.erased, { rent_listings: 1, rent_reports: 1, rent_saved_searches: 1 });
+ assert.deepStrictEqual(confirmation.body.retained, {});
+ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));
+
+ // A redelivery must not erase again: rows written after the deletion stay.
+ await makeListing(t, A, 'written-later');
+ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(again.status, 200);
+ assert.strictEqual(again.json().outcome, 'unchanged');
+ assert.strictEqual(await listingsFor(A), 1, 'nothing was erased twice');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'rent', clientSecret: 'rent-secret' });
+ const t = await boot({ env: { RENT_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });
+ assert.strictEqual(bad.status, 401);
+
+ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(forwarded.status, 403);
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ done();
+})();
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
@@ -38,7 +38,7 @@ async function boot(opts = {}) {
const { createStore } = require('../../server/db');
const { createStore } = require('../../server/db');
const testdb = await require('./db').testDb();
const testdb = await require('./db').testDb();
const store = createStore(testdb.db, { now: opts.now });
const store = createStore(testdb.db, { now: opts.now });
- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });
+ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });
await built.ctx.keys.ensure();
await built.ctx.keys.ensure();
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;