openvibers/openvibe.work Commits
Work: account export and deletion — a person's saved searches are exported to Network and deleted on the two ADR-033 Events topics, subscribed at boot (openvibe-sdk v0.36.0)
Changes
@@@ -28,6 +28,13 @@ OV_OAUTH_CLIENT_SECRET=# The audience app, agent and service tokens for this service's API carry.# WORK_AUDIENCE=openvibe.work++# OpenVibe.Events → Work (ADR-033 account export and deletion): the secret signs a delivery (comma-separated for++# rotation, 32+ characters each; unset makes POST /internal/events answer 503); the url is where the two subscriptions++# are created at boot (off when unset); WORK_EVENTS_SUBSCRIBE=0 turns that off.++WORK_EVENTS_SECRET=++WORK_EVENTS_URL=++#WORK_EVENTS_ENDPOINT=http://127.0.0.1:4960/internal/events++# optional: defaults to true in production# COOKIE_SECURE=true@@@ -109,6 +109,27 @@ test:pg` runs the same suite through PostgreSQL and PgBouncer (see [.github/work- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: work` in CI) and with**OpenVibe.Services** before the first deploy.++## Account export and deletion++++A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its++part (ADR-033). Work receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`++(loopback only) — the one table is mapped in [server/identity/account-data.js](server/identity/account-data.js), and the++boot-time subscriptions are created by [server/events-consumer.js](server/events-consumer.js):++++- **Exported:** the searches a person saved (`saved_searches.json`), pushed to++ `POST /internal/account-exports/:id/parts` with this service's own token. Nothing here is a secret — Work stores no++ token, key or credential.++- **Erased:** a saved search is the person's own and nothing anyone else's page hangs under it, so it is deleted whole++ and nothing is kept. Work then confirms with `POST /internal/account-deletions/:id/confirmations` and the counts.++- **Anonymized:** nothing. There is no row Work keeps that was written by this person for another person to read.++++The listings (`work_listings`) and the per-source fetch record (`work_source_fetches`) hold no person's rows: neither++has a subject column, so neither is exported nor erased.++++Environment: `WORK_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer++503), `WORK_EVENTS_URL` (or `EVENTS_URL`) is where the two subscriptions are created at boot (off when unset), and++`WORK_EVENTS_ENDPOINT` overrides the loopback endpoint; `WORK_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.++## Security (threat notes)Reporting a vulnerability: [SECURITY.md](SECURITY.md).@@@ -134,6 +155,6 @@ Part of the [OpenVibe network](https://openvibe.network). Built in the open by [<!-- versions:start -->- openvibe-contracts: v0.116.0--- openvibe-sdk: v0.35.2++- openvibe-sdk: v0.36.0- openvibe-shared: v2.15.0<!-- versions:end -->@@@ -7,11 +7,11 @@"port": "4960","domain": "openvibe.work","contracts": "openvibe-contracts v0.115.0",-- "sdk": "openvibe-sdk v0.35.2",++ "sdk": "openvibe-sdk v0.36.0","shared": "openvibe-shared v2.14.0","packages": ["openvibe-contracts v0.115.0",-- "openvibe-sdk v0.35.2",++ "openvibe-sdk v0.36.0","openvibe-shared v2.14.0"],"summary": "OpenVibe.Work gathers job listings from three public job boards (Arbeitnow, Remotive, RemoteOK) under their terms — the source is named and the original listing is linked on every page, and only a short excerpt is kept. It searches them with facets, shows one listing with a schema.org JobPosting whose url is the original, and lets a signed-in person keep a search and see what is new since they last looked. Public reads need no token.",@@@ -25,7 +25,8 @@"The pages: home, /jobs, /jobs/:id, /saved, /sources, /docs and /updates — server-rendered, complete without JavaScript","Discovery: a sitemap carrying the newest listings, llms.txt, robots.txt (saved searches excluded) and JSON-LD","Per-caller limits on the search and on saved-search writes; problem+json errors with stable codes",-- "Tests: the ingest, the search, the pages and the saved searches against stand-in boards, on a temp PGlite database with a mock Network"++ "Tests: the ingest, the search, the pages and the saved searches against stand-in boards, on a temp PGlite database with a mock Network",++ "Account export and deletion (ADR-033): the searches a person saved are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted, with the counts confirmed back; the two Events topics are subscribed at boot (server/identity/account-data.js)"],"notYet": ["The agent pillar: drafting, tailoring and tracking an application with the person's approval at each step",@@@ -115,6 +115,10 @@ server {# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:4960/metrics) ──location = /metrics { return 404; }++ # ── OpenVibe.Events' delivery route: loopback only (POST /internal/events) ──++ # Events posts to http://127.0.0.1:4960/internal/events directly. Never proxied from outside (ADR-033).++ location /internal/ { return 404; }++# ── Pages, docs, robots.txt, sitemap, static assets ─────location / {proxy_pass http://127.0.0.1:4960;nnew file mode 100644@@@ -0,0 +1,13 @@++-- phase: expand++-- OpenVibe.Work: the receipts of the ADR-033 account export and deletion deliveries this service applied++-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.++-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.++++CREATE TABLE IF NOT EXISTS account_data_events (++ id TEXT PRIMARY KEY,++ kind TEXT NOT NULL,++ subject TEXT NOT NULL,++ outcome JSONB,++ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),++ sent_at TIMESTAMPTZ++);@@@ -16,7 +16,7 @@"helmet": "^8.1.0","iovalkey": "^0.4.0","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0","pg": "^8.23.0"},@@@ -887,9 +887,9 @@}},"node_modules/openvibe-sdk": {-- "version": "0.35.2",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",-- "integrity": "sha512-HudKZJK3lZ9i0+yRdMrz9pJmIrYNTN8AFqbHFCz5jDCF68CfZwUJQyhE61pHODzGRyTvUi4DtAzkd0WEzIWx/g==",++ "version": "0.36.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0",++ "integrity": "sha512-mb+VvCuFH3KnMw4LWvqMDEoF93rNEb+eQi8HfT6mj+RPN5eQFWMcc/DCqJ3L0Yzjr99tQ08+DLwaYdZDIlfD0A==","license": "MIT","engines": {"node": ">=20"@@@ -24,7 +24,7 @@"helmet": "^8.1.0","iovalkey": "^0.4.0","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0","pg": "^8.23.0"},@@@ -31,6 +31,8 @@ const { createPageRoutes } = require('./http/pages');const { createServiceReadiness } = require('./observability');const { createCallerLimits } = require('./http/caller-limits');const { createIngest } = require('./ingest');++const accountDataLib = require('./identity/account-data');++const { createNetworkSender } = require('openvibe-sdk/account-data');const { assetVersion, send } = require('./render/layout');const { html } = require('./render/html');@@@ -39,7 +41,7 @@ const VERSION = require('../package.json').version;/*** opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,-- * tests only), valkey++ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)*/async function createApp(opts = {}) {const config = opts.config || configLib.load();@@@ -67,6 +69,18 @@ async function createApp(opts = {}) {// server/index.js, and only when WORK_INGEST=on: nothing fetches a board in a test or a development run.ctx.ingest = createIngest({ config, s, fetchImpl, log });++ // Account export and deletion (ADR-033, ./identity/account-data.js): the one table that holds a person's rows.++ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;++ // a test injects a stand-in through opts.accountSend.++ const accountData = accountDataLib.create({ db: s.db, log });++ // Without a client secret the service has no way to push a part or a confirmation: the route still answers++ // (bad signature, a forwarded request, no secret), and an event that really arrives asks Events to retry.++ const accountSend = opts.accountSend || (config.oauth.clientSecret++ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })++ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Work cannot answer account events'); });++ ctx.accountData = accountData;++ ctx.accountSend = accountSend;++app.use(contracts.http.middleware());app.use(helmet({contentSecurityPolicy: {@@@ -106,6 +120,12 @@ async function createApp(opts = {}) {app.use('/auth', sso.routes());{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'work', service: 'work', host: 'openvibe.work', name: 'OpenVibe.Work', profile: 'ugc' })); }++ // ── OpenVibe.Events → this service (loopback only) ──────++ // network.account.export_requested and network.account.deleted (ADR-033), answered by the SDK's consumer. It++ // reads the raw body itself, so it is mounted before any body parser (there is none above it), and nginx answers++ // 404 for /internal/ so this is reachable only on 127.0.0.1.++ app.use('/internal/events', accountData.consumer({ secrets: config.events.secrets, send: accountSend, log }));++// ── Static assets (content-hashed ?v= → immutable) ──────app.use('/shared', require('openvibe-shared/serve').handler());app.use(express.static(PUBLIC_DIR, {@@@ -54,6 +54,16 @@ function load(env = process.env) {},cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },++ // OpenVibe.Events → this service (server/events-consumer.js). The secret signs a delivery (comma-separated++ // for rotation, 32+ characters each); unset turns POST /internal/events off (503). The url is where the++ // network.account.export_requested and network.account.deleted subscriptions are created at boot, off when++ // unset. Both topics are ADR-033: account export and deletion (server/identity/account-data.js).++ events: {++ secrets: String(env.WORK_EVENTS_SECRET || '').split(',').map((x) => x.trim()).filter(Boolean),++ url: trim(env.WORK_EVENTS_URL || env.EVENTS_URL || ''),++ endpoint: env.WORK_EVENTS_ENDPOINT || '',++ },++// The public job boards the ingest reads (server/jobs/sources.js). The base URLs are configurable so a// test points them at a stand-in server; nothing else in the service ever fetches a URL, and never one a// caller typed. Their terms are quoted in the manifest that describes each source, and applied on everynnew file mode 100644@@@ -0,0 +1,52 @@++'use strict';++++/**++ * OpenVibe.Events → Work. Work has no consumer of its own: the two topics it takes are account export and deletion++ * (ADR-033), and POST /internal/events is answered by openvibe-sdk/account-data's own consumer, mounted in++ * server/app.js over server/identity/account-data.js (the table map). What is left here is the boot-time half:++ * startSubscriptions() creates the two subscriptions at Events if they are missing, idempotently, retried with++ * backoff, and off when the URL or the secret is unset (QUEST_EVENTS_SUBSCRIBE=0 style: WORK_EVENTS_SUBSCRIBE=0).++ *++ * The endpoint is this service's loopback /internal/events (Events posts to http://127.0.0.1:<port> directly; nginx++ * answers 404 there), and the token is this service's own for audience openvibe.events with++ * scope events.subscription.manage.++ */++const { serviceAuth } = require('openvibe-contracts');++const { TOPICS } = require('./identity/account-data');++++/** Create any missing subscription for TOPICS at Events (idempotent; retried in the background at boot). */++function startSubscriptions({ config, port, secret, eventsUrl = config.events.url, fetchImpl = globalThis.fetch, log = console }) {++ if (!eventsUrl || !secret || !config.oauth.clientSecret || process.env.WORK_EVENTS_SUBSCRIBE === '0') return null;++ const base = String(eventsUrl).replace(/\/+$/, '');++ const endpoint = config.events.endpoint || `http://127.0.0.1:${port}/internal/events`;++ const tokens = serviceAuth.createTokenClient({ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, audience: 'openvibe.events', scope: 'events.subscription.manage', fetchImpl });++ const call = async (method, path, body) => {++ const res = await fetchImpl(`${base}${path}`, { method, headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokens.authHeaders()) }, body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(15000) });++ const json = await res.json().catch(() => ({}));++ return { status: res.status, ok: res.ok, body: json };++ };++ const attempt = async () => {++ const listed = await call('GET', '/api/v1/subscriptions');++ if (!listed.ok) throw new Error(`listing subscriptions: ${listed.status}`);++ const mine = (listed.body.subscriptions || []).filter((s) => s.endpoint === endpoint);++ for (const topic of TOPICS) {++ if (mine.some((s) => s.topic_pattern === topic)) continue;++ const r = await call('POST', '/api/v1/subscriptions', { topic_pattern: topic, endpoint, secret });++ if (!r.ok && r.status !== 409) throw new Error(`subscribing to ${topic}: ${r.status} ${r.body.code || ''}`);++ if (r.ok) log.log(`[Events consumer] subscription created: ${r.body.id} (${topic} → ${endpoint})`);++ }++ };++ const delays = [0, 10_000, 60_000, 5 * 60_000, 15 * 60_000];++ let i = 0;++ let timer = null;++ let stopped = false;++ const run = () => { timer = null; if (stopped) return; attempt().catch((err) => {++ if (stopped) return;++ if (++i < delays.length) { timer = setTimeout(run, delays[i]); if (timer.unref) timer.unref(); } else log.warn('[Events consumer] subscriptions not created:', err.message);++ }); };++ timer = setTimeout(run, delays[0]); if (timer.unref) timer.unref();++ // Graceful stop: no further attempts (they run again at the next start).++ return { topics: TOPICS, endpoint, stop() { stopped = true; if (timer) clearTimeout(timer); timer = null; } };++}++++module.exports = { startSubscriptions, TOPICS };nnew file mode 100644@@@ -0,0 +1,33 @@++'use strict';++++/**++ * Account export and deletion → Work (ADR-033; openvibe-sdk/account-data). Work holds one thing a person made: the++ * search they saved, keyed `subject = user:usr_…` (server/http/api.js builds the subject from req.principal.requester,++ * which server/http/principal.js sets to `user:${subject_id}`; server/jobs/store.js writes it). A saved search is the++ * person's own — nothing anyone else hangs under it — so it is deleted whole and nothing is kept.++ *++ * network.account.export_requested the person's saved searches (saved_searches.json), newest first, pushed to++ * Network (POST /internal/account-exports/:id/parts) with this service's token.++ * network.account.deleted the person's saved searches go, and Work confirms with counts.++ *++ * The listings (work_listings) and the per-source ingest bookkeeping (work_source_fetches) hold no person's rows:++ * neither has a subject column (a listing is scraped from a public job board and a fetch record is per source), so++ * neither is exported nor erased (see migrations/0002_listings.sql). Nothing here is a secret: no token, key or++ * credential is stored, so every column of the saved-search table may be exported.++ */++const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');++++/**++ * The tables that hold a person's rows, with the value the subject column really stores. A saved search stores++ * `user:usr_…` (server/jobs/store.js takes the subject straight from the principal's requester; server/http/api.js).++ */++const TABLES = [++ { table: 'work_saved_searches', subject: 'subject', value: (usr) => `user:${usr}`, file: 'saved_searches.json' },++];++++/** The account-data handle for Work's store (server/db.js createStore). */++function create({ db, log = console } = {}) {++ return createAccountData({ db, service: 'work', tables: TABLES, log });++}++++module.exports = { create, TABLES, TOPICS };@@@ -8,13 +8,14 @@ const { createApp } = require('./app');const { gracefulStop } = require('openvibe-sdk/service');/**-- * The process stop (openvibe-sdk/service): the HTTP drain runs, then the JWKS refresher stops and the store closes.-- * Exported so a test can inject `exit` and `signals: false`.++ * The process stop (openvibe-sdk/service): the HTTP drain runs, then the timers clear, the Events subscriptions stop,++ * the JWKS refresher stops and the store closes. Exported so a test can inject `exit` and `signals: false`. `extra` is++ * what the product adds (the Events subscriptions); a test that passes none keeps the skeleton's order.*/--function createLifecycle({ server, ctx, exit, signals, timers = [] }) {++function createLifecycle({ server, ctx, exit, signals, timers = [], extra = [] }) {return gracefulStop({name: 'OpenVibe.Work', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,-- close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.ingest.stop(), () => ctx.keys.client.stop(), () => ctx.s.close()],++ close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.ingest.stop(), () => ctx.keys.client.stop(), () => ctx.s.close(), ...extra],});}@@@ -29,7 +30,11 @@ async function start() {ctx.keys.client.start();ctx.ingest.start();-- createLifecycle({ server, ctx });++ // Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without WORK_EVENTS_URL and++ // WORK_EVENTS_SECRET). The consumer itself is mounted in server/app.js.++ const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });++ const extra = [() => { if (subscriptions) subscriptions.stop(); }];++ createLifecycle({ server, ctx, extra });return { server, ctx };}nnew file mode 100644@@@ -0,0 +1,125 @@++'use strict';++/**++ * ADR-033: Work's part of an account export (a person's saved searches) and of an account deletion, applied through++ * the service's own /internal/events route with a stand-in Network. Real rows are made through the service's store,++ * the signed delivery is answered by openvibe-sdk/account-data, and the part only ever carries person A's rows. A++ * redelivery erases nothing twice, and the route refuses a bad signature and a forwarded request.++ */++const assert = require('assert');++const http = require('http');++const { boot, check, done } = require('./helpers/boot');++const { createNetworkSender } = require('openvibe-sdk/account-data');++const { signDeliveryHeaders } = require('openvibe-sdk/events');++const jobsStore = require('../server/jobs/store');++++const A = 'usr_01JZ0000000000000000000AAA';++const B = 'usr_01JZ0000000000000000000BBB';++const OLD = 'usr_01JZ0000000000000000000MRG';++const EXP = 'exp_01JZ0000000000000000000EXP';++const DEL = 'del_01JZ0000000000000000000DEX';++// Fixture secrets, built so they never look like a real key to a scanner.++const SECRET = `whsec_${'fixture'.repeat(6)}`;++const WRONG = `whsec_${'mismatch'.repeat(5)}`;++++const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };++const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };++const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });++++/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */++async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {++ const calls = [];++ const statusOf = (v) => (typeof v === 'function' ? v() : v);++ const server = http.createServer((req, res) => {++ const chunks = [];++ req.on('data', (c) => chunks.push(c));++ req.on('end', () => {++ const raw = Buffer.concat(chunks);++ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };++ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_work', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });++ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });++ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});++ });++ });++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };++}++++/** Save a search for one person, through the store the API writes with (server/jobs/store.js). */++async function makeRows(t, subject, q) {++ const s = t.ctx.s;++ const out = await jobsStore.createSaved(s, `user:${subject}`, { q, remote: 'true', type: 'full-time', location: 'berlin' }, { id: s.newId('ssc'), now: s.iso() });++ assert.ok(out.row, 'the saved search was created');++ return out.row;++}++++(async () => {++ await check('an export part carries only the person\'s saved searches, and no secret', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });++ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ await makeRows(t, A, 'person-a');++ await makeRows(t, B, 'person-b');++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'exported');++++ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);++ assert.ok(part, 'the part was pushed to Network');++ assert.strictEqual(part.auth, 'Bearer tok_work', 'with this service\'s own token');++ assert.strictEqual(part.body.subject, A);++ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['saved_searches.json']);++ const saved = part.body.files.find((f) => f.name === 'saved_searches.json').content;++ assert.strictEqual(saved.length, 1);++ assert.strictEqual(saved[0].query, 'person-a');++ assert.ok(!JSON.stringify(part.body).includes('person-b'), 'nobody else\'s rows');++ assert.ok(!/token|secret|password/i.test(JSON.stringify(part.body)), 'no secret is exported');++ } finally { await t.close(); await stub.close(); }++ });++++ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });++ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ await makeRows(t, A, 'person-a');++ await makeRows(t, B, 'person-b');++ const gone = async (subject) => await t.ctx.s.db.value('SELECT count(*)::int FROM work_saved_searches WHERE subject = $1', [`user:${subject}`]);++++ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(res.status, 200);++ assert.strictEqual(res.json().outcome, 'erased');++ assert.strictEqual(await gone(A), 0, 'the person\'s saved searches are gone');++ assert.strictEqual(await gone(B), 1, 'someone else\'s saved search stays');++++ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);++ assert.ok(confirmation, 'the confirmation was sent');++ assert.deepStrictEqual(confirmation.body.erased, { work_saved_searches: 1 });++ assert.deepStrictEqual(confirmation.body.retained, {});++ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));++++ // A redelivery must not erase again: rows written after the deletion stay.++ await makeRows(t, A, 'written-later');++ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });++ assert.strictEqual(again.status, 200);++ assert.strictEqual(again.json().outcome, 'unchanged');++ assert.strictEqual(await gone(A), 1, 'nothing was erased twice');++ } finally { await t.close(); await stub.close(); }++ });++++ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {++ const stub = await startNetworkStub();++ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });++ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });++ try {++ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });++ assert.strictEqual(bad.status, 401);++++ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });++ assert.strictEqual(forwarded.status, 403);++ } finally { await t.close(); await stub.close(); }++ });++++ done();++})();@@@ -45,7 +45,7 @@ async function boot(opts = {}) {const { createStore } = require('../../server/db');const testdb = await require('./db').testDb();const store = createStore(testdb.db, { now: opts.now });-- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });++ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });await built.ctx.keys.ensure();const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;
@@ -28,6 +28,13 @@ OV_OAUTH_CLIENT_SECRET=
@@ -28,6 +28,13 @@ OV_OAUTH_CLIENT_SECRET=
# The audience app, agent and service tokens for this service's API carry.
# The audience app, agent and service tokens for this service's API carry.
# WORK_AUDIENCE=openvibe.work
# WORK_AUDIENCE=openvibe.work
+# OpenVibe.Events → Work (ADR-033 account export and deletion): the secret signs a delivery (comma-separated for
+# rotation, 32+ characters each; unset makes POST /internal/events answer 503); the url is where the two subscriptions
+# are created at boot (off when unset); WORK_EVENTS_SUBSCRIBE=0 turns that off.
+WORK_EVENTS_SECRET=
+WORK_EVENTS_URL=
+#WORK_EVENTS_ENDPOINT=http://127.0.0.1:4960/internal/events
+
# optional: defaults to true in production
# optional: defaults to true in production
# COOKIE_SECURE=true
# COOKIE_SECURE=true
@@ -109,6 +109,27 @@ test:pg` runs the same suite through PostgreSQL and PgBouncer (see [.github/work
@@ -109,6 +109,27 @@ test:pg` runs the same suite through PostgreSQL and PgBouncer (see [.github/work
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: work` in CI) and with
- Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: work` in CI) and with
**OpenVibe.Services** before the first deploy.
**OpenVibe.Services** before the first deploy.
+## Account export and deletion
+
+A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its
+part (ADR-033). Work receives `network.account.export_requested` and `network.account.deleted` at `POST /internal/events`
+(loopback only) — the one table is mapped in [server/identity/account-data.js](server/identity/account-data.js), and the
+boot-time subscriptions are created by [server/events-consumer.js](server/events-consumer.js):
+
+- **Exported:** the searches a person saved (`saved_searches.json`), pushed to
+ `POST /internal/account-exports/:id/parts` with this service's own token. Nothing here is a secret — Work stores no
+ token, key or credential.
+- **Erased:** a saved search is the person's own and nothing anyone else's page hangs under it, so it is deleted whole
+ and nothing is kept. Work then confirms with `POST /internal/account-deletions/:id/confirmations` and the counts.
+- **Anonymized:** nothing. There is no row Work keeps that was written by this person for another person to read.
+
+The listings (`work_listings`) and the per-source fetch record (`work_source_fetches`) hold no person's rows: neither
+has a subject column, so neither is exported nor erased.
+
+Environment: `WORK_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer
+503), `WORK_EVENTS_URL` (or `EVENTS_URL`) is where the two subscriptions are created at boot (off when unset), and
+`WORK_EVENTS_ENDPOINT` overrides the loopback endpoint; `WORK_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off.
+
## Security (threat notes)
## Security (threat notes)
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
Reporting a vulnerability: [SECURITY.md](SECURITY.md).
@@ -134,6 +155,6 @@ Part of the [OpenVibe network](https://openvibe.network). Built in the open by [
@@ -134,6 +155,6 @@ Part of the [OpenVibe network](https://openvibe.network). Built in the open by [
<!-- versions:start -->
<!-- versions:start -->
- openvibe-contracts: v0.116.0
- openvibe-contracts: v0.116.0
-- openvibe-sdk: v0.35.2
+- openvibe-sdk: v0.36.0
- openvibe-shared: v2.15.0
- openvibe-shared: v2.15.0
<!-- versions:end -->
<!-- versions:end -->
@@ -7,11 +7,11 @@
@@ -7,11 +7,11 @@
"port": "4960",
"port": "4960",
"domain": "openvibe.work",
"domain": "openvibe.work",
"contracts": "openvibe-contracts v0.115.0",
"contracts": "openvibe-contracts v0.115.0",
- "sdk": "openvibe-sdk v0.35.2",
+ "sdk": "openvibe-sdk v0.36.0",
"shared": "openvibe-shared v2.14.0",
"shared": "openvibe-shared v2.14.0",
"packages": [
"packages": [
"openvibe-contracts v0.115.0",
"openvibe-contracts v0.115.0",
- "openvibe-sdk v0.35.2",
+ "openvibe-sdk v0.36.0",
"openvibe-shared v2.14.0"
"openvibe-shared v2.14.0"
],
],
"summary": "OpenVibe.Work gathers job listings from three public job boards (Arbeitnow, Remotive, RemoteOK) under their terms — the source is named and the original listing is linked on every page, and only a short excerpt is kept. It searches them with facets, shows one listing with a schema.org JobPosting whose url is the original, and lets a signed-in person keep a search and see what is new since they last looked. Public reads need no token.",
"summary": "OpenVibe.Work gathers job listings from three public job boards (Arbeitnow, Remotive, RemoteOK) under their terms — the source is named and the original listing is linked on every page, and only a short excerpt is kept. It searches them with facets, shows one listing with a schema.org JobPosting whose url is the original, and lets a signed-in person keep a search and see what is new since they last looked. Public reads need no token.",
@@ -25,7 +25,8 @@
@@ -25,7 +25,8 @@
"The pages: home, /jobs, /jobs/:id, /saved, /sources, /docs and /updates — server-rendered, complete without JavaScript",
"The pages: home, /jobs, /jobs/:id, /saved, /sources, /docs and /updates — server-rendered, complete without JavaScript",
"Discovery: a sitemap carrying the newest listings, llms.txt, robots.txt (saved searches excluded) and JSON-LD",
"Discovery: a sitemap carrying the newest listings, llms.txt, robots.txt (saved searches excluded) and JSON-LD",
"Per-caller limits on the search and on saved-search writes; problem+json errors with stable codes",
"Per-caller limits on the search and on saved-search writes; problem+json errors with stable codes",
- "Tests: the ingest, the search, the pages and the saved searches against stand-in boards, on a temp PGlite database with a mock Network"
+ "Tests: the ingest, the search, the pages and the saved searches against stand-in boards, on a temp PGlite database with a mock Network",
+ "Account export and deletion (ADR-033): the searches a person saved are exported to OpenVibe.Network on network.account.export_requested and deleted whole on network.account.deleted, with the counts confirmed back; the two Events topics are subscribed at boot (server/identity/account-data.js)"
],
],
"notYet": [
"notYet": [
"The agent pillar: drafting, tailoring and tracking an application with the person's approval at each step",
"The agent pillar: drafting, tailoring and tracking an application with the person's approval at each step",
@@ -115,6 +115,10 @@ server {
@@ -115,6 +115,10 @@ server {
# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:4960/metrics) ──
# ── Prometheus metrics: host-local scrapers only (http://127.0.0.1:4960/metrics) ──
location = /metrics { return 404; }
location = /metrics { return 404; }
+ # ── OpenVibe.Events' delivery route: loopback only (POST /internal/events) ──
+ # Events posts to http://127.0.0.1:4960/internal/events directly. Never proxied from outside (ADR-033).
+ location /internal/ { return 404; }
+
# ── Pages, docs, robots.txt, sitemap, static assets ─────
# ── Pages, docs, robots.txt, sitemap, static assets ─────
location / {
location / {
proxy_pass http://127.0.0.1:4960;
proxy_pass http://127.0.0.1:4960;
new file mode 100644
new file mode 100644
@@ -0,0 +1,13 @@
@@ -0,0 +1,13 @@
+-- phase: expand
+-- OpenVibe.Work: the receipts of the ADR-033 account export and deletion deliveries this service applied
+-- (openvibe-sdk/account-data's ACCOUNT_DATA_SCHEMA), so a redelivered export or deletion changes nothing.
+-- The row holds an export_id or deletion_id, the subject and the counts sent to Network; it is not a secret.
+
+CREATE TABLE IF NOT EXISTS account_data_events (
+ id TEXT PRIMARY KEY,
+ kind TEXT NOT NULL,
+ subject TEXT NOT NULL,
+ outcome JSONB,
+ applied_at TIMESTAMPTZ NOT NULL DEFAULT now(),
+ sent_at TIMESTAMPTZ
+);
@@ -16,7 +16,7 @@
@@ -16,7 +16,7 @@
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0",
"pg": "^8.23.0"
"pg": "^8.23.0"
},
},
@@ -887,9 +887,9 @@
@@ -887,9 +887,9 @@
}
}
},
},
"node_modules/openvibe-sdk": {
"node_modules/openvibe-sdk": {
- "version": "0.35.2",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",
- "integrity": "sha512-HudKZJK3lZ9i0+yRdMrz9pJmIrYNTN8AFqbHFCz5jDCF68CfZwUJQyhE61pHODzGRyTvUi4DtAzkd0WEzIWx/g==",
+ "version": "0.36.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0",
+ "integrity": "sha512-mb+VvCuFH3KnMw4LWvqMDEoF93rNEb+eQi8HfT6mj+RPN5eQFWMcc/DCqJ3L0Yzjr99tQ08+DLwaYdZDIlfD0A==",
"license": "MIT",
"license": "MIT",
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
@@ -24,7 +24,7 @@
@@ -24,7 +24,7 @@
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.116.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.35.2",
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.36.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.15.0",
"pg": "^8.23.0"
"pg": "^8.23.0"
},
},
@@ -31,6 +31,8 @@ const { createPageRoutes } = require('./http/pages');
@@ -31,6 +31,8 @@ const { createPageRoutes } = require('./http/pages');
const { createServiceReadiness } = require('./observability');
const { createServiceReadiness } = require('./observability');
const { createCallerLimits } = require('./http/caller-limits');
const { createCallerLimits } = require('./http/caller-limits');
const { createIngest } = require('./ingest');
const { createIngest } = require('./ingest');
+const accountDataLib = require('./identity/account-data');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
const { assetVersion, send } = require('./render/layout');
const { assetVersion, send } = require('./render/layout');
const { html } = require('./render/html');
const { html } = require('./render/html');
@@ -39,7 +41,7 @@ const VERSION = require('../package.json').version;
@@ -39,7 +41,7 @@ const VERSION = require('../package.json').version;
/**
/**
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
* opts: config, store, now (clock), fetchImpl (Network), log, limitsNow, callerLimits (false: count nobody,
- * tests only), valkey
+ * tests only), valkey, accountSend (a stand-in for Network's internal routes; tests only)
*/
*/
async function createApp(opts = {}) {
async function createApp(opts = {}) {
const config = opts.config || configLib.load();
const config = opts.config || configLib.load();
@@ -67,6 +69,18 @@ async function createApp(opts = {}) {
@@ -67,6 +69,18 @@ async function createApp(opts = {}) {
// server/index.js, and only when WORK_INGEST=on: nothing fetches a board in a test or a development run.
// server/index.js, and only when WORK_INGEST=on: nothing fetches a board in a test or a development run.
ctx.ingest = createIngest({ config, s, fetchImpl, log });
ctx.ingest = createIngest({ config, s, fetchImpl, log });
+ // Account export and deletion (ADR-033, ./identity/account-data.js): the one table that holds a person's rows.
+ // The sender posts to Network's internal export/deletion routes with this service's own client-credentials token;
+ // a test injects a stand-in through opts.accountSend.
+ const accountData = accountDataLib.create({ db: s.db, log });
+ // Without a client secret the service has no way to push a part or a confirmation: the route still answers
+ // (bad signature, a forwarded request, no secret), and an event that really arrives asks Events to retry.
+ const accountSend = opts.accountSend || (config.oauth.clientSecret
+ ? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })
+ : async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Work cannot answer account events'); });
+ ctx.accountData = accountData;
+ ctx.accountSend = accountSend;
+
app.use(contracts.http.middleware());
app.use(contracts.http.middleware());
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
@@ -106,6 +120,12 @@ async function createApp(opts = {}) {
@@ -106,6 +120,12 @@ async function createApp(opts = {}) {
app.use('/auth', sso.routes());
app.use('/auth', sso.routes());
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'work', service: 'work', host: 'openvibe.work', name: 'OpenVibe.Work', profile: 'ugc' })); }
{ const legal = require('openvibe-shared/legal'); app.get(legal.PATHS, legal.handler({ id: 'work', service: 'work', host: 'openvibe.work', name: 'OpenVibe.Work', profile: 'ugc' })); }
+ // ── OpenVibe.Events → this service (loopback only) ──────
+ // network.account.export_requested and network.account.deleted (ADR-033), answered by the SDK's consumer. It
+ // reads the raw body itself, so it is mounted before any body parser (there is none above it), and nginx answers
+ // 404 for /internal/ so this is reachable only on 127.0.0.1.
+ app.use('/internal/events', accountData.consumer({ secrets: config.events.secrets, send: accountSend, log }));
+
// ── Static assets (content-hashed ?v= → immutable) ──────
// ── Static assets (content-hashed ?v= → immutable) ──────
app.use('/shared', require('openvibe-shared/serve').handler());
app.use('/shared', require('openvibe-shared/serve').handler());
app.use(express.static(PUBLIC_DIR, {
app.use(express.static(PUBLIC_DIR, {
@@ -54,6 +54,16 @@ function load(env = process.env) {
@@ -54,6 +54,16 @@ function load(env = process.env) {
},
},
cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },
cookies: { secure: env.COOKIE_SECURE ? env.COOKIE_SECURE === 'true' : isProduction },
+ // OpenVibe.Events → this service (server/events-consumer.js). The secret signs a delivery (comma-separated
+ // for rotation, 32+ characters each); unset turns POST /internal/events off (503). The url is where the
+ // network.account.export_requested and network.account.deleted subscriptions are created at boot, off when
+ // unset. Both topics are ADR-033: account export and deletion (server/identity/account-data.js).
+ events: {
+ secrets: String(env.WORK_EVENTS_SECRET || '').split(',').map((x) => x.trim()).filter(Boolean),
+ url: trim(env.WORK_EVENTS_URL || env.EVENTS_URL || ''),
+ endpoint: env.WORK_EVENTS_ENDPOINT || '',
+ },
+
// The public job boards the ingest reads (server/jobs/sources.js). The base URLs are configurable so a
// The public job boards the ingest reads (server/jobs/sources.js). The base URLs are configurable so a
// test points them at a stand-in server; nothing else in the service ever fetches a URL, and never one a
// test points them at a stand-in server; nothing else in the service ever fetches a URL, and never one a
// caller typed. Their terms are quoted in the manifest that describes each source, and applied on every
// caller typed. Their terms are quoted in the manifest that describes each source, and applied on every
new file mode 100644
new file mode 100644
@@ -0,0 +1,52 @@
@@ -0,0 +1,52 @@
+'use strict';
+
+/**
+ * OpenVibe.Events → Work. Work has no consumer of its own: the two topics it takes are account export and deletion
+ * (ADR-033), and POST /internal/events is answered by openvibe-sdk/account-data's own consumer, mounted in
+ * server/app.js over server/identity/account-data.js (the table map). What is left here is the boot-time half:
+ * startSubscriptions() creates the two subscriptions at Events if they are missing, idempotently, retried with
+ * backoff, and off when the URL or the secret is unset (QUEST_EVENTS_SUBSCRIBE=0 style: WORK_EVENTS_SUBSCRIBE=0).
+ *
+ * The endpoint is this service's loopback /internal/events (Events posts to http://127.0.0.1:<port> directly; nginx
+ * answers 404 there), and the token is this service's own for audience openvibe.events with
+ * scope events.subscription.manage.
+ */
+const { serviceAuth } = require('openvibe-contracts');
+const { TOPICS } = require('./identity/account-data');
+
+/** Create any missing subscription for TOPICS at Events (idempotent; retried in the background at boot). */
+function startSubscriptions({ config, port, secret, eventsUrl = config.events.url, fetchImpl = globalThis.fetch, log = console }) {
+ if (!eventsUrl || !secret || !config.oauth.clientSecret || process.env.WORK_EVENTS_SUBSCRIBE === '0') return null;
+ const base = String(eventsUrl).replace(/\/+$/, '');
+ const endpoint = config.events.endpoint || `http://127.0.0.1:${port}/internal/events`;
+ const tokens = serviceAuth.createTokenClient({ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, audience: 'openvibe.events', scope: 'events.subscription.manage', fetchImpl });
+ const call = async (method, path, body) => {
+ const res = await fetchImpl(`${base}${path}`, { method, headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokens.authHeaders()) }, body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(15000) });
+ const json = await res.json().catch(() => ({}));
+ return { status: res.status, ok: res.ok, body: json };
+ };
+ const attempt = async () => {
+ const listed = await call('GET', '/api/v1/subscriptions');
+ if (!listed.ok) throw new Error(`listing subscriptions: ${listed.status}`);
+ const mine = (listed.body.subscriptions || []).filter((s) => s.endpoint === endpoint);
+ for (const topic of TOPICS) {
+ if (mine.some((s) => s.topic_pattern === topic)) continue;
+ const r = await call('POST', '/api/v1/subscriptions', { topic_pattern: topic, endpoint, secret });
+ if (!r.ok && r.status !== 409) throw new Error(`subscribing to ${topic}: ${r.status} ${r.body.code || ''}`);
+ if (r.ok) log.log(`[Events consumer] subscription created: ${r.body.id} (${topic} → ${endpoint})`);
+ }
+ };
+ const delays = [0, 10_000, 60_000, 5 * 60_000, 15 * 60_000];
+ let i = 0;
+ let timer = null;
+ let stopped = false;
+ const run = () => { timer = null; if (stopped) return; attempt().catch((err) => {
+ if (stopped) return;
+ if (++i < delays.length) { timer = setTimeout(run, delays[i]); if (timer.unref) timer.unref(); } else log.warn('[Events consumer] subscriptions not created:', err.message);
+ }); };
+ timer = setTimeout(run, delays[0]); if (timer.unref) timer.unref();
+ // Graceful stop: no further attempts (they run again at the next start).
+ return { topics: TOPICS, endpoint, stop() { stopped = true; if (timer) clearTimeout(timer); timer = null; } };
+}
+
+module.exports = { startSubscriptions, TOPICS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,33 @@
@@ -0,0 +1,33 @@
+'use strict';
+
+/**
+ * Account export and deletion → Work (ADR-033; openvibe-sdk/account-data). Work holds one thing a person made: the
+ * search they saved, keyed `subject = user:usr_…` (server/http/api.js builds the subject from req.principal.requester,
+ * which server/http/principal.js sets to `user:${subject_id}`; server/jobs/store.js writes it). A saved search is the
+ * person's own — nothing anyone else hangs under it — so it is deleted whole and nothing is kept.
+ *
+ * network.account.export_requested the person's saved searches (saved_searches.json), newest first, pushed to
+ * Network (POST /internal/account-exports/:id/parts) with this service's token.
+ * network.account.deleted the person's saved searches go, and Work confirms with counts.
+ *
+ * The listings (work_listings) and the per-source ingest bookkeeping (work_source_fetches) hold no person's rows:
+ * neither has a subject column (a listing is scraped from a public job board and a fetch record is per source), so
+ * neither is exported nor erased (see migrations/0002_listings.sql). Nothing here is a secret: no token, key or
+ * credential is stored, so every column of the saved-search table may be exported.
+ */
+const { createAccountData, TOPICS } = require('openvibe-sdk/account-data');
+
+/**
+ * The tables that hold a person's rows, with the value the subject column really stores. A saved search stores
+ * `user:usr_…` (server/jobs/store.js takes the subject straight from the principal's requester; server/http/api.js).
+ */
+const TABLES = [
+ { table: 'work_saved_searches', subject: 'subject', value: (usr) => `user:${usr}`, file: 'saved_searches.json' },
+];
+
+/** The account-data handle for Work's store (server/db.js createStore). */
+function create({ db, log = console } = {}) {
+ return createAccountData({ db, service: 'work', tables: TABLES, log });
+}
+
+module.exports = { create, TABLES, TOPICS };
@@ -8,13 +8,14 @@ const { createApp } = require('./app');
@@ -8,13 +8,14 @@ const { createApp } = require('./app');
const { gracefulStop } = require('openvibe-sdk/service');
const { gracefulStop } = require('openvibe-sdk/service');
/**
/**
- * The process stop (openvibe-sdk/service): the HTTP drain runs, then the JWKS refresher stops and the store closes.
- * Exported so a test can inject `exit` and `signals: false`.
+ * The process stop (openvibe-sdk/service): the HTTP drain runs, then the timers clear, the Events subscriptions stop,
+ * the JWKS refresher stops and the store closes. Exported so a test can inject `exit` and `signals: false`. `extra` is
+ * what the product adds (the Events subscriptions); a test that passes none keeps the skeleton's order.
*/
*/
-function createLifecycle({ server, ctx, exit, signals, timers = [] }) {
+function createLifecycle({ server, ctx, exit, signals, timers = [], extra = [] }) {
return gracefulStop({
return gracefulStop({
name: 'OpenVibe.Work', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,
name: 'OpenVibe.Work', server, deadlineExitCode: 0, exit, signals, deadlineMs: 10_000,
- close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.ingest.stop(), () => ctx.keys.client.stop(), () => ctx.s.close()],
+ close: [() => { for (const t of timers) clearInterval(t); }, () => ctx.ingest.stop(), () => ctx.keys.client.stop(), () => ctx.s.close(), ...extra],
});
});
}
}
@@ -29,7 +30,11 @@ async function start() {
@@ -29,7 +30,11 @@ async function start() {
ctx.keys.client.start();
ctx.keys.client.start();
ctx.ingest.start();
ctx.ingest.start();
- createLifecycle({ server, ctx });
+ // Subscribe to the two ADR-033 topics at OpenVibe.Events (idempotent; off without WORK_EVENTS_URL and
+ // WORK_EVENTS_SECRET). The consumer itself is mounted in server/app.js.
+ const subscriptions = require('./events-consumer').startSubscriptions({ config, port: config.port, secret: config.events.secrets[0] || '' });
+ const extra = [() => { if (subscriptions) subscriptions.stop(); }];
+ createLifecycle({ server, ctx, extra });
return { server, ctx };
return { server, ctx };
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,125 @@
@@ -0,0 +1,125 @@
+'use strict';
+/**
+ * ADR-033: Work's part of an account export (a person's saved searches) and of an account deletion, applied through
+ * the service's own /internal/events route with a stand-in Network. Real rows are made through the service's store,
+ * the signed delivery is answered by openvibe-sdk/account-data, and the part only ever carries person A's rows. A
+ * redelivery erases nothing twice, and the route refuses a bad signature and a forwarded request.
+ */
+const assert = require('assert');
+const http = require('http');
+const { boot, check, done } = require('./helpers/boot');
+const { createNetworkSender } = require('openvibe-sdk/account-data');
+const { signDeliveryHeaders } = require('openvibe-sdk/events');
+const jobsStore = require('../server/jobs/store');
+
+const A = 'usr_01JZ0000000000000000000AAA';
+const B = 'usr_01JZ0000000000000000000BBB';
+const OLD = 'usr_01JZ0000000000000000000MRG';
+const EXP = 'exp_01JZ0000000000000000000EXP';
+const DEL = 'del_01JZ0000000000000000000DEX';
+// Fixture secrets, built so they never look like a real key to a scanner.
+const SECRET = `whsec_${'fixture'.repeat(6)}`;
+const WRONG = `whsec_${'mismatch'.repeat(5)}`;
+
+const exportEvent = { event_id: 'evt_01JZ0000000000000000000E01', event_type: 'network.account.export_requested', source: 'network', payload: { export_id: EXP, subject: A } };
+const deleteEvent = { event_id: 'evt_01JZ0000000000000000000D01', event_type: 'network.account.deleted', source: 'network', payload: { deletion_id: DEL, subject: A, aliases: [OLD] } };
+const bodyOf = (ev) => JSON.stringify({ event: ev, seq: 1 });
+
+/** A stand-in for Network's internal routes: the token endpoint, the export part and the deletion confirmation. */
+async function startNetworkStub({ partStatus = 201, confirmStatus = 201 } = {}) {
+ const calls = [];
+ const statusOf = (v) => (typeof v === 'function' ? v() : v);
+ const server = http.createServer((req, res) => {
+ const chunks = [];
+ req.on('data', (c) => chunks.push(c));
+ req.on('end', () => {
+ const raw = Buffer.concat(chunks);
+ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };
+ if (req.url === '/oauth/token') return json(200, { access_token: 'tok_work', token_type: 'Bearer', expires_in: 300, scope: 'openvibe.network' });
+ calls.push({ url: req.url, auth: req.headers.authorization, body: JSON.parse(raw.toString() || 'null') });
+ return json(req.url.includes('/parts') ? statusOf(partStatus) : statusOf(confirmStatus), {});
+ });
+ });
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ return { url: `http://127.0.0.1:${server.address().port}`, calls, close: () => new Promise((r) => server.close(r)) };
+}
+
+/** Save a search for one person, through the store the API writes with (server/jobs/store.js). */
+async function makeRows(t, subject, q) {
+ const s = t.ctx.s;
+ const out = await jobsStore.createSaved(s, `user:${subject}`, { q, remote: 'true', type: 'full-time', location: 'berlin' }, { id: s.newId('ssc'), now: s.iso() });
+ assert.ok(out.row, 'the saved search was created');
+ return out.row;
+}
+
+(async () => {
+ await check('an export part carries only the person\'s saved searches, and no secret', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });
+ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ await makeRows(t, A, 'person-a');
+ await makeRows(t, B, 'person-b');
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'exported');
+
+ const part = stub.calls.find((c) => c.url === `/internal/account-exports/${EXP}/parts`);
+ assert.ok(part, 'the part was pushed to Network');
+ assert.strictEqual(part.auth, 'Bearer tok_work', 'with this service\'s own token');
+ assert.strictEqual(part.body.subject, A);
+ assert.deepStrictEqual(part.body.files.map((f) => f.name).sort(), ['saved_searches.json']);
+ const saved = part.body.files.find((f) => f.name === 'saved_searches.json').content;
+ assert.strictEqual(saved.length, 1);
+ assert.strictEqual(saved[0].query, 'person-a');
+ assert.ok(!JSON.stringify(part.body).includes('person-b'), 'nobody else\'s rows');
+ assert.ok(!/token|secret|password/i.test(JSON.stringify(part.body)), 'no secret is exported');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('a deletion erases the person and their aliases once, keeps someone else, and confirms with counts', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });
+ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ await makeRows(t, A, 'person-a');
+ await makeRows(t, B, 'person-b');
+ const gone = async (subject) => await t.ctx.s.db.value('SELECT count(*)::int FROM work_saved_searches WHERE subject = $1', [`user:${subject}`]);
+
+ const res = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(res.status, 200);
+ assert.strictEqual(res.json().outcome, 'erased');
+ assert.strictEqual(await gone(A), 0, 'the person\'s saved searches are gone');
+ assert.strictEqual(await gone(B), 1, 'someone else\'s saved search stays');
+
+ const confirmation = stub.calls.find((c) => c.url === `/internal/account-deletions/${DEL}/confirmations`);
+ assert.ok(confirmation, 'the confirmation was sent');
+ assert.deepStrictEqual(confirmation.body.erased, { work_saved_searches: 1 });
+ assert.deepStrictEqual(confirmation.body.retained, {});
+ assert.ok(!Number.isNaN(Date.parse(confirmation.body.completed_at)));
+
+ // A redelivery must not erase again: rows written after the deletion stay.
+ await makeRows(t, A, 'written-later');
+ const again = await t.get('/internal/events', { method: 'POST', body: bodyOf(deleteEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(deleteEvent), SECRET) } });
+ assert.strictEqual(again.status, 200);
+ assert.strictEqual(again.json().outcome, 'unchanged');
+ assert.strictEqual(await gone(A), 1, 'nothing was erased twice');
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ await check('the internal route refuses a bad signature (401) and a forwarded request (403)', async () => {
+ const stub = await startNetworkStub();
+ const sender = createNetworkSender({ networkInternalUrl: stub.url, clientId: 'work', clientSecret: 'work-secret' });
+ const t = await boot({ env: { WORK_EVENTS_SECRET: SECRET }, accountSend: sender });
+ try {
+ const bad = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', ...signDeliveryHeaders(bodyOf(exportEvent), WRONG) } });
+ assert.strictEqual(bad.status, 401);
+
+ const forwarded = await t.get('/internal/events', { method: 'POST', body: bodyOf(exportEvent), headers: { 'Content-Type': 'application/json', 'X-Forwarded-For': '203.0.113.9', ...signDeliveryHeaders(bodyOf(exportEvent), SECRET) } });
+ assert.strictEqual(forwarded.status, 403);
+ } finally { await t.close(); await stub.close(); }
+ });
+
+ done();
+})();
@@ -45,7 +45,7 @@ async function boot(opts = {}) {
@@ -45,7 +45,7 @@ async function boot(opts = {}) {
const { createStore } = require('../../server/db');
const { createStore } = require('../../server/db');
const testdb = await require('./db').testDb();
const testdb = await require('./db').testDb();
const store = createStore(testdb.db, { now: opts.now });
const store = createStore(testdb.db, { now: opts.now });
- const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true });
+ const built = await createApp({ config, store, log, limitsNow: opts.limitsNow, callerLimits: opts.callerLimits === true, accountSend: opts.accountSend });
await built.ctx.keys.ensure();
await built.ctx.keys.ensure();
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;