openvibers/openvibe.tools Commits
Tools: the migration era is deleted (SQLite migration scripts, legacy deploy path, SQLite readiness and reporting) (#33)
Changes
@@@ -16,7 +16,7 @@ offline against the Network's public key (JWKS).- the tool registry (families, tools, descriptors `tools.tool@1`, canonical, short and alias hosts),the catalog and every tool page on `openvibe.tools` and `*.openvibe.tools`- the run API and the jobs runtime (img, audio, docs jobs with their files and SSE progress), the-- guard (callers, quotas, sniffing, egress throttles, abuse log) and each app's `data/guard.db`++ guard (callers, quotas, sniffing, egress throttles, PostgreSQL abuse log)- the `tools.*` events, Tools' Search documents and the `tools.usage` user module## Does not own@@@ -69,7 +69,7 @@ apps/_shared # openvibe-tools-shared, a versioned package (package.json, CHANG# the SSRF guard (egress.js) that every tool reaching a visitor-chosen host or URL goes# through (public addresses only, checked after DNS and dialled as checked, redirect hops# re-checked). Visit analytics come from openvibe-shared/analytics.--scripts/analytics-prune.js # operator CLI: raw analytics retention + one-time scrub (dry run by default)++scripts/analytics-prune.js # operator CLI: PostgreSQL raw analytics retention (dry run by default)(openvibe-shared is a pinned OpenVibe.Shared release in each app's package.json; no vendor/ copy)```@@@ -109,9 +109,7 @@ as the checkout owner, installs in each app whose dependencies changed (apps/_*checks every dependency resolves and that the jobs runtime and the guard load, restarts every`openvibe-tools*` unit, waits for the gateway's `/api/ready` and its `/release.json` to name the new shawith every unit active, rolls back if they do not, and announces the release (`host.release.published`--to OpenVibe.Events, so open tabs check `/release.json` within seconds). When ovhost is missing, too old or--does not deploy Tools with that strategy, the wrapper runs `deploy/scripts/deploy-legacy.sh`, the previous--script, unchanged (`OVHOST_LEGACY=1` forces it).++to OpenVibe.Events, so open tabs check `/release.json` within seconds). If ovhost is missing, the wrapper exits with an error.## Tool registry API (ADR-027)@@@ -281,8 +279,8 @@ reload reattaches.`tools.job.created` (submit or retry; actor the owner), `tools.job.started` (a worker claimed it; a jobre-queued after a restart is announced again when it starts), `tools.job.succeeded` and `tools.job.failed`(payloads `openvibe-contracts` `tools.job.*@1`, validated before they are queued; subject `job <id>`, visibility-- `internal`, source `tools`). Each is written to an `event_outbox` table in the satellite's `jobs.db` (openvibe-sdk-- `createOutbox`) in the same SQLite transaction as the state change, so an event exists exactly when its++ `internal`, source `tools`). Each is written to the shared PostgreSQL `event_outbox` table (openvibe-sdk++ `createPgOutbox`) in the same transaction as the state change, so an event exists exactly when itstransition committed, and relayed to `EVENTS_URL/api/v1/events` (at least once; Events dedupes on `event_id`).Payloads carry ids, type, owner, state, attempts, times, where result files are (index, mime, size, sha256,`local` or a Media `media_id`) and the error (`status`, `code`, `detail` with server paths and the job's file@@@ -344,8 +342,8 @@ One module used by the gateway and every satellite, driven by each tool's descriaddress anything reads. The gateway's proxy to a satellite passes its own `req.ip` as that one hop; food passes itsvisitor's to maps. A first-party service calling on loopback is identified by its service token (each has its ownbucket at the service tier); without one it is anonymous, counted by the address it forwarded or its own.--- **Quotas.** A token bucket per quota class × tier in memory (`perMinute`, `burst`) and a UTC-day allowance in the-- app's `data/guard.db` (survives restarts), both counting the tool's `cost`. Browser sessions from one address share++- **Quotas.** A token bucket per quota class × tier (`perMinute`, `burst`) and a UTC-day allowance in++ Valkey when configured (process-local otherwise), both counting the tool's `cost`. Browser sessions from one address share3 × one session's allowance, so dropping the cookie never resets anything. Counted answers carry`RateLimit-Limit`/`-Remaining`/`-Reset`; a refusal is `429` problem+json `tools.quota.exceeded` with `Retry-After`and `quota_class`, `scope`, `tier`.@@@ -367,7 +365,7 @@ One module used by the gateway and every satellite, driven by each tool's describelong to their maker (session, sign-in or token; anyone else gets 404), at most 5 per owner and 10 per address.YouTube downloads belong to whoever started them. Nominatim is paced to one request a second; Overpass waits arecapped.--- **Abuse log.** Every refusal, and every would-be refusal in report mode, goes to `guard.db`: time, `HMAC-SHA256`++- **Abuse log.** Every refusal, and every would-be refusal in report mode, goes to PostgreSQL `guard_abuse`: time, `HMAC-SHA256`of the address (or /64) with a random daily salt (only today's is kept, so older hashes cannot be tied toanything), principal or user id, tool, reason, enforced or not; repeats within a minute are one row with a count;kept 30 days. No raw address is stored anywhere (pseudonymous, ADR-021). Metrics: `tools_guard_refused_total{reason,tool}`,@@@ -450,7 +448,7 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `opealso answers `location = /metrics` with 404. HTTP golden signals are labelled by route template(`/api/v1/jobs/:id`), proxied satellite traffic on the gateway by satellite (`proxy:img`), SPA pages andstatic files by a fixed label — never the raw URL. `release_info{service="tools"|"tools-<app>"}`, process-- metrics, and on img/audio/docs `tools_jobs{app,state}` (a group-by on jobs.db) and++ metrics, and on img/audio/docs `tools_jobs{app,state}` (a group-by on the shared job store) and`tools_jobs_executing{app,kind="executing"|"limit"}`.- `GET /api/ready` — named checks with `status`, `required`, `latency_ms`, `checked_at`; HTTP 503 only when arequired check fails, otherwise 200 with failed optional checks listed in `degraded`. `/api/health` is@@@ -458,12 +456,12 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `ope| Server | Required | Optional (degraded when failing) ||---|---|---|--| gateway (4001) | `catalog` | `network_key`, `service_directory` (Network registry vs fallback list), `community` (`/api/ready`), `satellite_<app>` for all seven (`/api/ready`, cached 15 s), `tool_registry` (every catalogue tool has a descriptor that meets the contracts) |--| img, docs (4012, 4016) | `jobs_db` (query), `job_runtime` (worker started), `data_dir`, `uploads_dir`, `output_dir` (write test) | `analytics_db`, `network_key`, `media_results` (only with `TOOLS_JOB_RESULTS=media`); img: `heif_decoder`; docs: `qpdf`, `pdftoppm`, `pdfinfo` |--| audio (4014) | same as img | same as img, plus `ffmpeg` on PATH |--| yt (4013) | `downloads_dir` | `analytics_db`, `yt_dlp`, `ffmpeg`, `yt_cookies` (when `YT_COOKIES_FILE` is set), `network_key` |--| food (4011) | `maps` (every food API is proxied to it) | `analytics_db` |--| maps, text (4010, 4015) | — | `analytics_db` (maps' external data sources are not probed) |++| gateway (4001) | `catalog`, `tools_db` | `network_key`, `service_directory`, `community`, `satellite_<app>`, `tool_registry` |++| img, docs (4012, 4016) | `tools_db`, `job_runtime`, `data_dir`, `uploads_dir`, `output_dir` | `network_key`, `media_results` (when enabled); img: `heif_decoder`; docs: `qpdf`, `pdftoppm`, `pdfinfo` |++| audio (4014) | same as img | same as img, plus `ffmpeg` |++| yt (4013) | `tools_db`, `downloads_dir` | `yt_dlp`, `ffmpeg`, `yt_cookies` (when set), `network_key` |++| food (4011) | `tools_db`, `maps` | — |++| maps, text (4010, 4015) | `tools_db` | — |`TOOLS_SATELLITE_PORTS="img=5012,…"` overrides the gateway's satellite ports (tests, a moved unit); every app readsit (`apps/_shared/tools/satellites.js`), so set it for all units if a port moves.@@@ -498,31 +496,14 @@ are installed (or `QPDF_PATH` / `HEIF_DEC_PATH` point at them). Install first wi## Analytics (ADR-021)--Every satellite except the gateway keeps visit analytics in its own `data/analytics.db` through--`openvibe-shared/analytics` (since openvibe-shared v1.4.0; the one module Live, Tools and Network use).--Bound by ADR-021 (OpenVibe.Contracts `docs/adr/ADR-021-analytics.md`):----- **A raw row carries** event type, service, route template (matched Express route, else a normaliser: no-- query string, ids/hashes → `:id`, the segment after `watch`, `jobs`, `recipe`, `place`, … → `:param`,-- `/@x` → `/@:user`), method, status, response time, a rotating session id, country (CDN header),-- user-agent class (`chrome/windows/desktop`, `bot:googlebot`) + browser/os/device, referer origin, bot-- flags, a signed-in flag, timestamp. **Never** an IP, a user id, a city, the UA string or a full referer-- (`ip`/`user_id`/`city` stay as always-NULL columns for compatibility).--- **Opt-out:** a request with `Sec-GPC: 1` or `DNT: 1` is not recorded at all (no raw row, visitor hash,-- session id or rate counter), so it is also missing from the rollups.--- **Session id:** random, in memory against the visitor hash, new after 30 idle minutes and at UTC midnight.--- **Bot rate check:** per-IP counters in memory only (current + previous minute); `analytics_rate_tracking`-- is emptied at boot and no longer written.--- **Unique visitors:** `HMAC-SHA256(day salt, ip + "\n" + ua)` (16 hex chars), random salt per UTC day. Hashes-- and the salt are kept only until the first hourly aggregation after their day ends (right after the-- day's final rollup), never in raw rows. Rollups keep counts; sub-48 h raw summaries count sessions.--- **Retention:** each satellite prunes raw rows older than 30 days in batches of 5000, 5 minutes after boot-- and every 24 h after; rollups stay.--- **CLI:** `node scripts/analytics-prune.js`, a wrapper over `openvibe-shared/analytics/prune-cli` (dry run-- over every `apps/*/data/analytics.db`; `--app`, `--db` to narrow). `--apply` needs `--backup <file|dir>`-- (verified owner-only online backup; a directory when several databases are targeted) or `--no-backup`;-- `--scrub` also rewrites rows written before ADR-021 and the rollups' top lists (counts unchanged). Ends-- with VACUUM unless `--no-vacuum`.++All eight apps store visit analytics in the shared PostgreSQL database through++`openvibe-shared/analytics/pg` (ADR-021). Raw events include the service, route template,++method, status, response time, session identifier and coarse visitor metadata. GPC and DNT opt-outs++are excluded. Daily and hourly rollups remain after raw events expire.++++- **Retention:** the application prunes raw rows older than 30 days; rollups stay.++- **CLI:** `node scripts/analytics-prune.js` counts old rows by service. Pass `--days N` to choose++ retention and `--apply` to delete them. It requires `DATABASE_URL` and applies to every service.## Host packages some tools need@@@ -635,36 +616,15 @@ when the gateway's `/api/ready` or `/release.json` does not come up with every u`openvibe-tools-<name>.service` per satellite(`apps/<name>/deploy/systemd/`). Resource bounds: `MemoryMax=2G` (img, audio,docs), `1G` (yt), `768M` (gateway, text, maps, food); `TasksMax=256`; `Nice=5`-- for img, audio, docs and yt. Every app writes `data/guard.db` (the gateway-- too: `ReadWritePaths=/opt/openvibe.tools/apps/gateway/data`; the deploy-- (`ovhost deploy tools`, or `deploy-legacy.sh`) creates each `data/` before restarting).++ for img, audio, docs and yt. The gateway data directory remains writable for local runtime files.- Nginx: satellites have specific `server_name` blocks; the gateway'swildcard `*.openvibe.tools` block catches everything else. TLS via`/etc/letsencrypt/live/openvibe.tools/`.--## Migration to PostgreSQL + Valkey (plan T8)----All eight apps move from `better-sqlite3` to **one `tools` PostgreSQL database** and to **Valkey** for the--guard's throttle state. The schema is in `migrations/0001_tools.sql` (job store, `guard_abuse`) and--`migrations/0002_analytics.sql` (request analytics, the DDL of `openvibe-shared/analytics/pg.js`),--`migrations/0003_token_revocations.sql` and `migrations/0004_event_outbox.sql` (the tools.job.* outbox,--`openvibe-sdk` createPgOutbox's table). All of them run as the owner (`DATABASE_DIRECT_URL`); nothing creates a--table at runtime, so the runtime role needs no DDL rights. `scripts/migrate-to-postgres.js` imports--every `apps/*/data/{guard,jobs,analytics}.db` into that one database (each file's rows tagged with its app):--run it with `--pglite` for an in-memory rehearsal, `--dry-run --url <scratch>` for a scratch database, or--with `DATABASE_DIRECT_URL` set for the cutover; it prints a per-table count and checksum report.----After the move the apps read `DATABASE_URL` / `DATABASE_DIRECT_URL` (PostgreSQL, via PgBouncer) and--`VALKEY_URL` (Valkey); `guard_salt`, `guard_day` and the minute/burst buckets live in Valkey (a throttle,--never authoritative), while `guard_abuse`, the job store and the analytics tables stay in PostgreSQL.--`TOOLS_GUARD` (default `enforce`) and its meaning are unchanged.----State (plan T8 landed 2026-10-02; production serves the PostgreSQL-only release): every app opens the one `tools`--database (`apps/_shared/db.js`, `DATABASE_URL` else an embedded PGlite database in development;--`DATABASE_DIRECT_URL` runs the migrations as owner) and the shared Valkey (`VALKEY_URL`, else the guard's--counters and salt stay in this process); the job store, `guard_abuse`, `token_revocations` and the request--analytics are on PostgreSQL, the net and dev pages call the run API, and the `/api/process`, `/api/net/*` and--`/api/dev/*` routes are removed (above). No app depends on `better-sqlite3` any more; the operational scripts--that still read the old SQLite files (the importer, `scripts/analytics-prune.js`, `scripts/guard-abuse-report.js`)--have it in `scripts/package.json`: run `npm --prefix scripts install` on the host only to read the archived SQLite--files (the importer has run; production writes PostgreSQL). The cutover evidence is in `docs/cutover-evidence-t8.md`.++## Database and guard state++++All eight apps use one PostgreSQL database through `openvibe-sdk/db`. Production requires++`DATABASE_URL`; `DATABASE_DIRECT_URL` lets the owner apply the immutable SQL files in `migrations/`++at boot. Development and tests use embedded PGlite through the same API. Job records, abuse logs,++revocations, analytics, and the event outbox live in PostgreSQL. The guard's transient throttle++counters and salt use Valkey through `VALKEY_URL`, with process-local state when it is unset.@@@ -3,7 +3,7 @@"stage": "stable","code": true,"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",-- "runtime": "node 22, express; PostgreSQL (openvibe-sdk/db, PGlite in development) + Valkey for the guard and analytics; the job store too (plan T8; better-sqlite3 only in scripts/ for the pre-cutover SQLite files); eight apps (gateway, maps, food, img, yt, audio, text, docs)",++ "runtime": "node 22, express; PostgreSQL (openvibe-sdk/db, PGlite in development) + Valkey for the guard and analytics; the job store; eight apps (gateway, maps, food, img, yt, audio, text, docs)","port": 4001,"ports": {"gateway": 4001,@@@ -28,7 +28,7 @@],"capabilitiesRegistered": true,"deployed": true,-- "deployment": "Public at https://openvibe.tools: deployed with `sudo ovhost deploy tools` (OpenVibe.Host strategy multi-app since the 2026-09-27 cutover; deploy/scripts/deploy.sh is the wrapper), /opt/openvibe.tools, units openvibe-tools (gateway, 127.0.0.1:4001) and openvibe-tools-<app> per satellite, env /etc/openvibe/tools.env.",++ "deployment": "Public at https://openvibe.tools: deployed with `sudo ovhost deploy tools` (OpenVibe.Host strategy multi-app; deploy/scripts/deploy.sh is the wrapper), /opt/openvibe.tools, units openvibe-tools (gateway, 127.0.0.1:4001) and openvibe-tools-<app> per satellite, env /etc/openvibe/tools.env.","features": ["tool registry with descriptors, canonical hosts, aliases and owner-attached custom domains","server-rendered catalog and tool pages with full SEO",@@@ -55,7 +55,7 @@ function openToolsDb({ createDb, service = 'tools', log = console, registry, pglif (typeof createDb !== 'function') throw new TypeError('openToolsDb needs createDb (require(\'openvibe-sdk/db\').createDb)');const url = String(process.env.DATABASE_URL || '').trim();if (!url) {-- if (String(process.env.NODE_ENV || '') === 'production') throw new Error('DATABASE_URL is not set: Tools serves from PostgreSQL (plan T8)');++ if (String(process.env.NODE_ENV || '') === 'production') throw new Error('DATABASE_URL is required in production');// A directory keeps development data across restarts; without one (tests) the database is in memory,// so every run starts from the empty, migrated schema.const dir = pgliteDir || process.env.TOOLS_PGLITE_DIR || '';@@@ -35,7 +35,7 @@ const COOKIE = 'ov_tools_jobs';* @param {string} o.issuer* @param {string} [o.audience='openvibe.tools']* @param {object} [o.contracts] openvibe-contracts, when the app has it-- * @param {() => string} [o.salt] today's salt (guard.db); a process-local one otherwise++ * @param {() => string} [o.salt] today's salt (Valkey when configured); a process-local one otherwise* @param {string} [o.cookieName='ov_tools_jobs']* @param {boolean} [o.secureCookie]*/@@@ -15,13 +15,13 @@// callers one resolver (./caller.js): anonymous (address, IPv6 /64) < session < user < service;// sandbox tokens keep their small allowance// quotas token buckets per class × tier weighted by the tool's cost (./quota.js), day allowances--// in guard.db; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal++// in Valkey when configured; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal// 429 problem+json tools.quota.exceeded with Retry-After// heavy a semaphore for synchronous heavy calls (503 tools.busy + Retry-After when it is full)// uploads the bytes are checked against the descriptor's files.accept (415 tools.file.unsupported_type);// heavy tools (auth.anonymous false) need a session cookie, a sign-in or a token// egress a per-target throttle across all callers (limits.perTargetPerMinute) and the port-scan cap--// log every refusal (and every would-be refusal in report mode) in guard.db's abuse log:++// log every refusal (and every would-be refusal in report mode) in PostgreSQL guard_abuse:// HMAC(address, today's salt), principal or user id, tool, reason — 30 days, no raw// address anywhere; metric tools_guard_refused_total{reason,tool}// challenge a hook for a person-check (Turnstile later; ./challenge.js, a no-op now)@@@ -8,7 +8,7 @@// image conversion, 10 an audio effect, 50 a video download). Each quota class × caller tier has// perMinute a token bucket refilled at this many units a minute (kept in memory)// burst the bucket's size: what may be spent at once after a quiet spell--// perDay units per UTC day, persisted in the app's guard.db (0 = no day allowance)++// perDay units per UTC day, persisted in the shared guard store (0 = no day allowance)// Tiers: anonymous (keyed by IP, IPv6 by /64) < session (the ov_tools_jobs browser cookie) < user// (a Network sign-in) < service (a first-party service or a developer app's production token).// sandbox is a developer app's sandbox token: the smallest allowance, keyed by the app.@@@ -95,7 +95,7 @@ function bounds(env = process.env) {},// Webhook request bins (in memory, one hour): per owner, per address, in all.webhook: { perOwner: 5, perIp: 10, total: 500 },-- // The abuse log (guard.db) keeps rows this long.++ // The abuse log (PostgreSQL guard_abuse) keeps rows this long.abuseRetentionDays: 30,};}@@@ -6,7 +6,7 @@// (lifecycle), http.js (routes), media.js (results in Media), events.js (tools.job.* to OpenVibe.Events),// usage.js (tools.usage.recorded, a developer project's jobs per hour), client.js (browser helper).//--// The job store is part of every app's one `tools` database: no per-satellite jobs.db.++// The job store is part of every app's one `tools` database: shared across apps.//// Environment (one /etc/openvibe/tools.env for every unit):// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)@@@ -8,7 +8,7 @@// const obs = require('../../_shared/observe').observe({// app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),// service: 'tools-img', release: release.release,--// checks: [ checks.sqlite('analytics_db', analyticsDb, { required: false }), … ],++// checks: [ checks.postgres('tools_db', db), … ],// jobs: () => jobs, // the satellite's job system, once it exists (tools_jobs{app,state})// });//@@@ -77,19 +77,6 @@ function observe(o) {// ── Check builders ──────────────────────────────────────────────const checks = {-- /** A real query on a better-sqlite3 handle (or a function returning one). */-- sqlite(name, db, { required = true, sql = 'SELECT 1 AS ok', description } = {}) {-- return {-- name, required, description,-- check: () => {-- const h = typeof db === 'function' ? db() : db;-- if (!h) return 'database not open';-- const row = h.prepare(sql).get();-- return row ? true : 'query returned no row';-- },-- };-- },--/** A real round trip on the one `tools` database (an openvibe-sdk/db handle, or a function* returning one); the detail names the store it answered from (postgresql / pglite). */postgres(name, db, { required = true, description } = {}) {@@@ -3,7 +3,7 @@// stores no IP, user id, city, raw user agent, raw referer or query string anywhere in the analytics// tables; paths are route templates; uniques come from day-scoped hashes deleted after the day's rollup;// pruneRawEventsPg removes strictly-older raw rows and day hashes in bounded batches and never touches--// rollups. (The SQLite tracker's scrub and file CLI are retired with better-sqlite3.)++// rollups.const assert = require('assert');const fs = require('fs');const os = require('os');ddeleted file mode 100644@@@ -1,22 +0,0 @@--'use strict';--// The T8 closeout record must exist and keep its checks, and the README must no longer say the cutover is--// pending (plan T8; a silent re-draft would lose the evidence). This fails loudly if either regresses.--const assert = require('assert');--const fs = require('fs');--const path = require('path');----const ROOT = path.join(__dirname, '..', '..', '..');--const doc = path.join(ROOT, 'docs', 'cutover-evidence-t8.md');--const readme = path.join(ROOT, 'README.md');----assert.ok(fs.existsSync(doc), 'docs/cutover-evidence-t8.md is missing');--const md = fs.readFileSync(doc, 'utf8');--for (const key of ['health tools', 'origin/main HEAD', 'env', 'Valkey ACL', 'Row parity', 'Reproduce']) {-- assert.ok(md.includes(key), `docs/cutover-evidence-t8.md is missing "${key}"`);--}----assert.ok(fs.existsSync(readme), 'README.md is missing');--const readmeText = fs.readFileSync(readme, 'utf8');--assert.ok(!readmeText.includes('the cutover is pending'), 'README.md still contains "the cutover is pending"');----console.log('cutover evidence exists, records its checks, and the README no longer says the cutover is pending: passed');@@@ -1,60 +1,33 @@'use strict';--/**-- * deploy/scripts/deploy.sh is a thin wrapper around `ovhost deploy tools` (OpenVibe.Host, strategy-- * multi-app; roadmap WS-N task 11) with deploy-legacy.sh (the previous script, unchanged) as its-- * fallback. A fake ovhost records what the wrapper asks for; a fake legacy script records the fallback.-- */const assert = require('assert');const fs = require('fs');const os = require('os');const path = require('path');const { spawnSync } = require('child_process');--const WRAPPER = path.join(__dirname, '..', '..', '..', 'deploy', 'scripts', 'deploy.sh');const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tools-deploy-wrapper-'));const log = path.join(tmp, 'calls.log');const ovhost = path.join(tmp, 'ovhost');--fs.writeFileSync(ovhost, `#!/usr/bin/env bash--if [ "$1" = capabilities ]; then-- [ -n "$FAKE_OLD" ] && exit 1-- printf '%b\\n' "\${FAKE_CAPS:-ovhost=0.3.0\\ndeploy-api=1\\nservice=tools\\nstrategy=multi-app\\nmanaged=yes\\nlayout=git}"-- exit 0--fi--echo "ovhost $*" >> "${log}"--exit "\${FAKE_EXIT:-0}"--`, { mode: 0o755 });--const legacy = path.join(tmp, 'legacy.sh');--fs.writeFileSync(legacy, `#!/usr/bin/env bash\necho "legacy $*" >> "${log}"\n`, { mode: 0o755 });--++fs.writeFileSync(ovhost, `#!/usr/bin/env bash\necho "$*" >> "${log}"\nexit "\${FAKE_EXIT:-0}"\n`, { mode: 0o755 });function run(args = [], env = {}) {fs.rmSync(log, { force: true });-- const r = spawnSync('bash', [WRAPPER, ...args], { env: { PATH: process.env.PATH, OVHOST: ovhost, OVHOST_SUDO: '', DEPLOY_LEGACY: legacy, ...env }, encoding: 'utf8' });++ const r = spawnSync('bash', [WRAPPER, ...args], { env: { PATH: process.env.PATH, OVHOST: ovhost, OVHOST_SUDO: '', ...env }, encoding: 'utf8' });let calls = [];try { calls = fs.readFileSync(log, 'utf8').trim().split('\n').filter(Boolean); } catch { /* none */ }return { code: r.status, out: r.stdout + r.stderr, calls };}----assert.deepStrictEqual(run().calls, ['ovhost deploy tools']);--assert.deepStrictEqual(run(['--wait-idle']).calls, ['ovhost deploy tools --wait-idle']);--assert.deepStrictEqual(run(['--rollback']).calls, ['ovhost rollback tools']);--assert.deepStrictEqual(run([], { DRY_RUN: '1' }).calls, ['ovhost plan tools']);--assert.strictEqual(run([], { FAKE_EXIT: '3' }).code, 3, "ovhost's exit code is the wrapper's");++assert.deepStrictEqual(run().calls, ['deploy tools']);++assert.deepStrictEqual(run(['--wait-idle']).calls, ['deploy tools --wait-idle']);++assert.deepStrictEqual(run(['--restart']).calls, ['deploy tools --restart']);++assert.deepStrictEqual(run(['--force']).calls, ['deploy tools --force']);++assert.deepStrictEqual(run(['--rollback']).calls, ['rollback tools']);++assert.deepStrictEqual(run([], { DRY_RUN: '1' }).calls, ['plan tools']);++assert.strictEqual(run([], { FAKE_EXIT: '3' }).code, 3);assert.strictEqual(run(['--nope']).code, 1);----let r = run([], { FAKE_OLD: '1' });--assert.deepStrictEqual(r.calls, ['legacy'], 'too old: the legacy script');--assert.match(r.out, /too old/);--r = run([], { FAKE_CAPS: 'deploy-api=1\\nstrategy=git-checkout\\nmanaged=yes' });--assert.deepStrictEqual(r.calls, ['legacy'], 'another strategy: the legacy script');--assert.match(r.out, /does not deploy tools with strategy multi-app \(git-checkout\)/);--r = run([], { OVHOST: path.join(tmp, 'missing') });--assert.deepStrictEqual(r.calls, ['legacy']);--for (const [args, env] of [[['--rollback'], {}], [['--wait-idle'], {}], [[], { DRY_RUN: '1' }]]) {-- r = run(args, { OVHOST_LEGACY: '1', ...env });-- assert.strictEqual(r.code, 1, `${args.join(' ')} ${JSON.stringify(env)}: the legacy script cannot, so nothing runs`);-- assert.deepStrictEqual(r.calls, []);--}--assert.match(fs.readFileSync(WRAPPER, 'utf8'), /^set -euo pipefail$/m);--++const missing = run([], { OVHOST: path.join(tmp, 'missing') });++assert.strictEqual(missing.code, 1);++assert.deepStrictEqual(missing.calls, []);++assert.match(missing.out, /ovhost not found/);++assert.doesNotMatch(fs.readFileSync(WRAPPER, 'utf8'), /legacy|capabilities|DEPLOY_LEGACY|OVHOST_LEGACY/i);fs.rmSync(tmp, { recursive: true, force: true });console.log('deploy wrapper: all checks passed');@@@ -39,4 +39,19 @@ for (const m of manifests) {assert.ok(!(pkg.dependencies && pkg.dependencies['better-sqlite3']), `${path.relative(APPS, m)} still depends on better-sqlite3`);}++// Migration tooling and legacy deployment must stay removed.++const ROOT = path.resolve(APPS, '..');++for (const name of [++ 'scripts/migrate-to-postgres.js', 'scripts/sqlite.js', 'scripts/package.json',++ 'scripts/package-lock.json', 'deploy/scripts/deploy-legacy.sh',++ 'docs/cutover-evidence-t8.md',++]) assert.ok(!fs.existsSync(path.join(ROOT, name)), `${name} must be removed`);++for (const name of ['deploy/scripts/deploy.sh', 'scripts/analytics-prune.js', 'scripts/guard-abuse-report.js']) {++ const source = fs.readFileSync(path.join(ROOT, name), 'utf8');++ assert.doesNotMatch(source, /DEPLOY_LEGACY|OVHOST_LEGACY|\.\/sqlite|analytics\.db|guard\.db|deploy-legacy/);++}++for (const m of manifests) {++ const pkg = JSON.parse(fs.readFileSync(m, 'utf8'));++ for (const dep of ['better-sqlite3', 'sqlite3']) assert.ok(!pkg.dependencies?.[dep], `${m} depends on ${dep}`);++}console.log(`no sqlite (nothing under apps/ requires better-sqlite3): all checks passed (${files.length} files scanned)`);@@@ -41,7 +41,7 @@ function assertShape(body, service) {const obs = observe({app, metrics, ready, service: 'tools-test', release: 'abc1234567',checks: [-- { name: 'db', required: true, check: () => (dbOk ? true : 'SQLITE_IOERR') },++ { name: 'db', required: true, check: () => (dbOk ? true : 'database unavailable') },checks.upstream('satellite_dead', `http://127.0.0.1:${await freePort()}/api/ready`),checks.binary('nope', 'definitely-not-a-binary-ov'),],@@@ -58,7 +58,7 @@ function assertShape(body, service) {r = await json(`${base}/api/ready`);assert.strictEqual(r.status, 503);assert.deepStrictEqual(r.body.failed, ['db']);-- assert.strictEqual(r.body.checks.db.error, 'SQLITE_IOERR');++ assert.strictEqual(r.body.checks.db.error, 'database unavailable');obs.stop();server.close();assert.ok(which('node') || which(process.execPath), 'which() finds executables');@@@ -1,55 +1,32 @@'use strict';--// The operational scripts and the deploy fallback after better-sqlite3 left every app (plan T8):--// - deploy/scripts/*.sh never ask an app to load better-sqlite3 (the legacy deploy's load checks would--// abort every deploy once the dependency is gone);--// - scripts/ carries better-sqlite3 itself (scripts/package.json) for the cutover importer and the--// SQLite-era reports, and they load it through scripts/sqlite.js, not from an app;--// - analytics-prune on the one tools database refuses --app instead of silently pruning every service.const assert = require('assert');--const fs = require('fs');const path = require('path');----const ROOT = path.resolve(__dirname, '..', '..', '..');--const SCRIPTS = path.join(ROOT, 'scripts');--const SQLITE = ['better', 'sqlite3'].join('-');--++const prune = require(path.resolve(__dirname, '..', '..', '..', 'scripts', 'analytics-prune.js'));++const report = require(path.resolve(__dirname, '..', '..', '..', 'scripts', 'guard-abuse-report.js'));++const { openToolsDb } = require('../db');(async () => {-- // 1. The deploy scripts.-- const deployDir = path.join(ROOT, 'deploy', 'scripts');-- for (const f of fs.readdirSync(deployDir).filter((n) => n.endsWith('.sh'))) {-- const src = fs.readFileSync(path.join(deployDir, f), 'utf8');-- assert.ok(!src.includes(`require('${SQLITE}')`) && !src.includes(`require("${SQLITE}")`), `deploy/scripts/${f} still loads ${SQLITE}`);-- }-- const legacy = fs.readFileSync(path.join(deployDir, 'deploy-legacy.sh'), 'utf8');-- assert.match(legacy, /require\('openvibe-sdk\/db'\); require\('\.\.\/_shared\/jobs'\)/, 'the jobs runtime check loads openvibe-sdk/db');-- assert.match(legacy, /require\('openvibe-sdk\/db'\); require\('\.\.\/_shared\/guard'\)/, 'the guard check loads openvibe-sdk/db');---- // 2. scripts/ owns the dependency; the scripts load it through scripts/sqlite.js.-- const pkg = JSON.parse(fs.readFileSync(path.join(SCRIPTS, 'package.json'), 'utf8'));-- assert.ok(pkg.dependencies && pkg.dependencies[SQLITE], `scripts/package.json lists ${SQLITE}`);-- assert.ok(fs.existsSync(path.join(SCRIPTS, 'package-lock.json')), 'with a lockfile');-- for (const f of ['migrate-to-postgres.js', 'analytics-prune.js', 'guard-abuse-report.js']) {-- const src = fs.readFileSync(path.join(SCRIPTS, f), 'utf8');-- assert.match(src, /require\('\.\/sqlite'\)/, `${f} loads it through scripts/sqlite.js`);-- assert.ok(!src.includes(`'${SQLITE}'`), `${f} does not resolve it by name from an app`);-- }-- try {-- require(path.join(SCRIPTS, 'sqlite.js')).loadSqlite();-- } catch (err) {-- assert.match(err.message, /npm --prefix scripts install/, 'a missing dependency says how to install it');-- }---- // 3. analytics-prune: --app on the one tools database is refused, before anything is opened.-- const prune = require(path.join(SCRIPTS, 'analytics-prune.js'));-- await assert.rejects(prune.pgMain(['--app', 'yt'], () => {}), /--app does not apply to the one tools database/);++ await assert.rejects(prune.pgMain(['--app', 'yt'], () => {}), /--app does not apply/);const before = process.env.DATABASE_URL;process.env.DATABASE_URL = 'postgres://[email protected]:1/none';try {-- await assert.rejects(Promise.resolve().then(() => prune.main(['--app', 'yt', '--apply'], () => {})), /--app does not apply/,-- 'DATABASE_URL set, no --db: the PostgreSQL branch, and --app is refused rather than pruning every service');++ await assert.rejects(Promise.resolve().then(() => prune.main(['--app', 'yt', '--apply'], () => {})), /unknown option|--app does not apply/);} finally {if (before === undefined) delete process.env.DATABASE_URL; else process.env.DATABASE_URL = before;}---- console.log('ops scripts: deploy checks without the SQLite dependency, scripts/ owns it, analytics-prune refuses --app on PostgreSQL: all checks passed');++ assert.throws(() => report.parseArgs(['--db', 'old.db']), /unknown option/);++ assert.throws(() => report.parseArgs(['--pg']), /unknown option/);++ const priorEnv = process.env.NODE_ENV;++ const priorUrl = process.env.DATABASE_URL;++ process.env.NODE_ENV = 'production';++ delete process.env.DATABASE_URL;++ let created = false;++ try {++ assert.throws(() => openToolsDb({ createDb: () => { created = true; } }), /DATABASE_URL is required in production/);++ assert.strictEqual(created, false, 'production fails before PGlite can open');++ } finally {++ if (priorEnv === undefined) delete process.env.NODE_ENV; else process.env.NODE_ENV = priorEnv;++ if (priorUrl === undefined) delete process.env.DATABASE_URL; else process.env.DATABASE_URL = priorUrl;++ }++ assert.deepStrictEqual(report.parseArgs(['--url', 'postgres://test/db', '--json']), { url: 'postgres://test/db', json: true });++ console.log('ops scripts: PostgreSQL commands reject obsolete SQLite options');})().catch((err) => { console.error(err); process.exit(1); });@@@ -2,7 +2,7 @@// The one `tools` database for a test run (plan T8): one PGlite database per test directory, migrated// with the repo's migrations. It is kept across a satellite restart (the data must survive it), so a// test that restarts a satellite calls testDb() with the same directory again; closeAllTestDbs() at the--// end of the run. Nothing here needs better-sqlite3: PGlite is real PostgreSQL in-process.++// end of the run. PGlite runs PostgreSQL in process.const fs = require('fs');const path = require('path');const { dep } = require('./deps');@@@ -50,14 +50,7 @@ assert.throws(() => requireShared('next', { appsDir: dir, version: '1.0.0' }), /assert.throws(() => require('../release').toolsRelease('no-such-app', require), /declares no "openvibeToolsShared" range/, 'toolsRelease refuses an undeclared app before anything else');fs.rmSync(dir, { recursive: true, force: true });--// The deploy walks apps/*/ to install, resolve and preflight each app; _shared has a package.json now--// but is not an app (2026-09-26: the guard preflight tried to load better-sqlite3 from it and aborted).--// deploy.sh hands over to `ovhost deploy tools` (OpenVibe.Host strategy multi-app, whose default--// skipPackages is apps/_*) and falls back to deploy-legacy.sh, whose app loops skip it themselves.--const legacy = fs.readFileSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy-legacy.sh'), 'utf8');--const loops = legacy.split('\n').map((l, i, all) => [l, all[i + 1] || '']).filter(([l]) => /^for app in apps\/\*\/; do/.test(l));--assert.ok(loops.length >= 3, 'deploy-legacy.sh has its app loops');--for (const [, next] of loops) assert.match(next, /case "\$app" in apps\/_\*\) continue ;; esac/, 'each app loop in deploy-legacy.sh skips apps/_*');--assert.match(fs.readFileSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy.sh'), 'utf8'), /^STRATEGY=multi-app$/m, 'deploy.sh hands over only to the multi-app strategy (apps/_* skipped)');++// Deployment uses the host's multi-app strategy, which excludes apps/_* packages.++assert.ok(fs.existsSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy.sh')));console.log('_shared version: all checks passed');@@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the--// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would++// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would// refuse. Every audio tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).const SPECS = require('./descriptors').SPECS;// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).@@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the--// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would++// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would// refuse. Every PDF tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).const SPECS = require('./descriptors').SPECS;// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).@@@ -21,7 +21,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota and--// the abuse log (data/guard.db). The data lookups are counted by maps (tools-map), which gets the++// the abuse log (PostgreSQL guard_abuse). The data lookups are counted by maps (tools-map), which gets the// visitor's address from here. TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).@@@ -22,7 +22,7 @@ NoNewPrivileges=truePrivateTmp=trueProtectSystem=strictProtectHome=true--# guard.db (apps/_shared/guard): quotas' day counters and the abuse log++# Writable gateway data directory for local runtime filesReadWritePaths=/opt/openvibe.tools/apps/gateway/data[Install]@@@ -56,7 +56,7 @@ let auth = null; // created below; the key check reads it at request time// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else// the address), the tools-api quota, the net and dev tools' quotas by descriptor, the per-target--// throttle, the port-scan cap, webhook bin caps and the abuse log (data/guard.db). The Network key is++// throttle, the port-scan cap, webhook bin caps and the abuse log (PostgreSQL guard_abuse). The Network key is// the OAuth client's (loaded and retried by it). TOOLS_GUARD=enforce (default) refuses; report records// what it would refuse.// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).@@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the--// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.++// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.const SPECS = require('./descriptors').SPECS;// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4): guard_abuse in// PostgreSQL, guard_salt/guard_day and the quota buckets in Valkey. `ready` resolves when the schema is in.@@@ -29,7 +29,7 @@ const { createPacer, Busy } = require('../../_shared/guard/semaphore');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, else the address — the food app forwards its// visitor's), the tools-map quota on the data routes (descriptor maps: cost 2 a call) and the abuse log--// (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.++// (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-maps' });@@@ -19,7 +19,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota on--// /api/ and the abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what++// /api/ and the abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what// it would refuse.const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).@@@ -26,7 +26,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');// ── Guard (apps/_shared/guard) ───────────────────────────────// Who is asking (Network sign-in with aud openvibe.tools, the browser session, else the address),--// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (data/guard.db).++// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (PostgreSQL guard_abuse).// TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-yt' });ddeleted file mode 100755@@@ -1,71 +0,0 @@--#!/usr/bin/env bash--# Deploy OpenVibe.Tools on the host: pull, install dependencies in each app whose package.json--# changed (that includes a new openvibe-shared release tag), restart the units, check health, then--# announce the release to open tabs (`ovhost announce tools`; best effort, never fails the deploy).--set -euo pipefail--cd "$(dirname "$0")/../.."--BEFORE=$(git rev-parse HEAD)--# npm leaves untracked lockfiles in apps that did not track one; once the repo starts tracking it,--# `git pull` refuses to overwrite it. Those files are generated, so remove exactly those before pulling.--git fetch -q origin--for f in $(git diff --name-only HEAD "origin/$(git rev-parse --abbrev-ref HEAD)" -- 'apps/*/package-lock.json'); do-- if [ -f "$f" ] && ! git ls-files --error-unmatch "$f" >/dev/null 2>&1; then echo "removing untracked $f (the release tracks it)"; rm -f "$f"; fi--done--git pull -q --ff-only; AFTER=$(git rev-parse HEAD)--for app in apps/*/; do-- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app-- [ -f "$app/package.json" ] || continue-- if git diff --name-only "$BEFORE" "$AFTER" -- "$app/package.json" | grep -q . || [ ! -d "$app/node_modules" ]; then (cd "$app" && npm install --omit=dev --no-audit --no-fund --loglevel=error); fi--done--# Every app must resolve its dependencies before anything restarts. A dependency that used to be a--# file: link can leave an empty directory npm treats as installed (2026-09-23: vendor/openvibe-shared--# kept an untracked lockfile, every unit crash-looped on 'openvibe-shared/brand'); reinstall those.--for app in apps/*/; do-- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app-- [ -f "$app/package.json" ] || continue-- for dep in $(node -e 'console.log(Object.keys(require("./"+process.argv[1]+"package.json").dependencies||{}).join(" "))' "$app"); do-- if [ ! -f "$app/node_modules/$dep/package.json" ]; then-- echo "reinstalling $dep in $app (missing package.json)"; rm -rf "$app/node_modules/$dep"-- (cd "$app" && npm install --omit=dev --no-audit --no-fund --loglevel=error)-- [ -f "$app/node_modules/$dep/package.json" ] || { echo "ABORT: $app cannot resolve $dep; nothing restarted" >&2; exit 1; }-- fi-- done--done--# Apps that run jobs (apps/_shared/jobs, required by relative path) hand it their own openvibe-contracts--# and openvibe-sdk (openvibe-sdk/db for the job store in the one tools database, and the tools.job.*--# outbox relay): check they load under this Node and that the shared runtime itself loads. Nothing here--# needs better-sqlite3 any more (plan T8: no app depends on it).--for app in img audio docs; do-- (cd "apps/$app" && node -e "require('openvibe-contracts'); require('openvibe-sdk'); require('openvibe-sdk/db'); require('../_shared/jobs')") \-- || { echo "ABORT: apps/$app cannot load the jobs runtime; nothing restarted" >&2; exit 1; }--done--# Every app's guard (apps/_shared/guard) keeps its state in PostgreSQL (guard_abuse) and Valkey (salt, day--# counters, buckets); check it loads. The units' ReadWritePaths still name each data directory (uploads,--# job files), which must exist before systemd starts them.--for app in apps/*/; do-- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app-- [ -f "$app/package.json" ] || continue-- mkdir -p "$app/data"-- (cd "$app" && node -e "require('openvibe-sdk/db'); require('../_shared/guard')") \-- || { echo "ABORT: $app cannot load the guard; nothing restarted" >&2; exit 1; }--done--UNITS=$(systemctl list-unit-files 'openvibe-tools*' --no-legend | awk '{print $1}')--sudo systemctl restart $UNITS--sleep 5--for u in $UNITS; do printf '%-34s %s\n' "$u" "$(systemctl is-active "$u")"; done--curl -fsS -o /dev/null -H 'Host: openvibe.tools' http://127.0.0.1:4001/api/health && echo "gateway healthy ($BEFORE → $AFTER)" || exit $?--# Release notification (roadmap WS-P task 9): OpenVibe.Host publishes host.release.published for the release--# the apps' /release.json reports, once per release, so open tabs check it now instead of at their next--# poll. Best effort: skipped without an ovhost whose --help has `announce <service>`, 20 s at most, and it--# never changes the exit code. ovhost reads Host's credentials as root (OpenVibe.Host--# docs/release-notifications.md).--OVHOST_BIN=$(command -v "${OVHOST:-ovhost}" 2>/dev/null || true)--if [ -n "$OVHOST_BIN" ]; then-- case "$("$OVHOST_BIN" --help 2>/dev/null || true)" in-- *"announce <service>"*)-- SUDO=""; [ "$(id -u)" -eq 0 ] || SUDO="sudo -n"-- timeout 20 $SUDO "$OVHOST_BIN" announce tools 2>&1 || echo "release notification not sent (the deploy stands)" ;;-- *) echo "release notification skipped: this ovhost has no announce" ;;-- esac--fi--exit 0@@@ -1,71 +1,20 @@#!/usr/bin/env bash--# OpenVibe.Tools — deploy: a thin wrapper around `ovhost deploy tools` (OpenVibe.Host, strategy multi-app;--# roadmap WS-N task 11; OpenVibe.Host docs/deploy-strategies.md).--#--# ovhost now does what this script did, as the checkout owner: untracked apps/*/package-lock.json files the--# release tracks are removed before the merge; npm install in each app whose lockfile or dependencies changed--# (apps/_* are packages, not apps); every dependency resolves (reinstalled once, else ABORT with nothing--# restarted); each app's data/ exists; the jobs runtime loads in img/audio/docs and the guard in every app;--# every openvibe-tools* unit restarts; the gateway answers /api/ready with Host: openvibe.tools and its--# /release.json names the new sha, every unit is active; the release is announced. New: a release that does--# not come up is rolled back (the checkout, its dependencies, a second restart; exit 3), and every attempt--# is in `ovhost releases tools`.--#--# deploy/scripts/deploy.sh ovhost deploy tools--# deploy/scripts/deploy.sh --wait-idle ovhost deploy tools --wait-idle (hold until no tool job runs)--# deploy/scripts/deploy.sh --restart ovhost deploy tools --restart--# deploy/scripts/deploy.sh --rollback ovhost rollback tools--# DRY_RUN=1 deploy/scripts/deploy.sh ovhost plan tools--#--# ovhost runs as root (sudo is used when this runs as another user). Fallback: deploy-legacy.sh, the--# previous script unchanged, when ovhost is missing or too old (no `capabilities`, deploy-api < 1), or the--# host inventory does not deploy tools with strategy multi-app; OVHOST_LEGACY=1 forces it. The legacy--# script takes no flags: --rollback, --wait-idle and DRY_RUN=1 are refused rather than ignored there.++# OpenVibe.Tools deploy wrapper: ovhost deploy, rollback, or plan tools.set -euo pipefail----SERVICE=tools--STRATEGY=multi-app--HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)--LEGACY="${DEPLOY_LEGACY:-$HERE/deploy-legacy.sh}"OVHOST="${OVHOST:-/usr/local/bin/ovhost}"if [ "${OVHOST_SUDO-auto}" = auto ]; then if [ "$(id -u)" -eq 0 ]; then SUDO=(); else SUDO=(sudo); fi; elif [ -n "${OVHOST_SUDO}" ]; then SUDO=("$OVHOST_SUDO"); else SUDO=(); fi--CMD=deployFLAGS=()while [ "$#" -gt 0 ]; docase "$1" in--wait-idle|--restart|--force) FLAGS+=("$1"); shift ;;--rollback) CMD=rollback; shift ;;-- --) shift; break ;;-- *) echo "Usage: $0 [--wait-idle] [--restart] [--force] [--rollback] (DRY_RUN=1 for the plan)"; exit 1 ;;++ *) echo "Usage: $0 [--wait-idle] [--restart] [--force] [--rollback] (DRY_RUN=1 for the plan)" >&2; exit 1 ;;esacdone----legacy() {-- echo "[tools-deploy] $1 — running deploy-legacy.sh (the previous deploy script) instead"-- if [ "$CMD" = rollback ] || [ "${DRY_RUN:-0}" = 1 ] || [[ " ${FLAGS[*]:-} " == *" --wait-idle "* ]]; then-- echo "[tools-deploy] ✗ deploy-legacy.sh has no --rollback, --wait-idle or DRY_RUN; nothing was done" >&2-- exit 1-- fi-- exec bash "$LEGACY"--}----REASON=""--probe() {-- if [ "${OVHOST_LEGACY:-0}" = 1 ]; then REASON="OVHOST_LEGACY=1"; return 1; fi-- if ! command -v "$OVHOST" >/dev/null 2>&1; then REASON="ovhost not found ($OVHOST)"; return 1; fi-- local caps api-- if ! caps=$("${SUDO[@]}" "$OVHOST" capabilities "$SERVICE" 2>/dev/null); then REASON="this ovhost has no 'capabilities' (too old) or no inventory entry for $SERVICE"; return 1; fi-- api=$(printf '%s\n' "$caps" | sed -n 's/^deploy-api=//p')-- case "$api" in ''|*[!0-9]*) REASON="this ovhost reports no deploy-api (too old)"; return 1 ;; esac-- if [ "$api" -lt 1 ]; then REASON="this ovhost's deploy-api is $api, 1 is needed"; return 1; fi-- if ! printf '%s\n' "$caps" | grep -qx "strategy=$STRATEGY"; then REASON="the host inventory does not deploy $SERVICE with strategy $STRATEGY ($(printf '%s\n' "$caps" | sed -n 's/^strategy=//p'))"; return 1; fi-- if ! printf '%s\n' "$caps" | grep -qx "managed=yes"; then REASON="ovhost does not manage $SERVICE"; return 1; fi-- return 0--}----probe || legacy "$REASON"----if [ "${DRY_RUN:-0}" = 1 ]; then exec "${SUDO[@]}" "$OVHOST" plan "$SERVICE"; fi--echo "[tools-deploy] ovhost $CMD $SERVICE ${FLAGS[*]:-}"--exec "${SUDO[@]}" "$OVHOST" "$CMD" "$SERVICE" "${FLAGS[@]}"++if ! command -v "$OVHOST" >/dev/null 2>&1; then++ echo "[tools-deploy] ovhost not found ($OVHOST)" >&2++ exit 1++fi++if [ "${DRY_RUN:-0}" = 1 ]; then exec "${SUDO[@]}" "$OVHOST" plan tools; fi++exec "${SUDO[@]}" "$OVHOST" "$CMD" tools "${FLAGS[@]}"ddeleted file mode 100644@@@ -1,67 +0,0 @@--# T8 cutover: closeout evidence — OpenVibe.Tools----What the read-only interfaces can actually prove about the Tools PostgreSQL + Valkey cutover (plan T8). The--cutover landed **2026-10-02**; production serves the PostgreSQL-only release. Checked **2026-10-04 (UTC)** through--`ov access run openvibe-ovh <action>` — `health`, `service-status`, `plan`, `releases`, `validate`, `env-set`,--`show`, `journal`. Every result below is a literal output of one of those actions.----Nothing here was deployed, restarted, rolled back or imported; no credential, URL or env **value** was read or--printed. Checks that need an interface the broker does not expose are marked **pending** in their own section and--were not guessed.----SHAs are printed as `ovhost` reports them (12 hex characters).----## Release and readiness — verified----- `health tools` (exit 0): `tools a331ea36963a ready openvibe-tools=active openvibe-tools-maps=active-- openvibe-tools-food=active openvibe-tools-img=active openvibe-tools-yt=active openvibe-tools-audio=active-- openvibe-tools-text=active openvibe-tools-docs=active running tool jobs=0`. `ovhost` needs each service's-- `/api/ready` to answer before it calls the service `ready`.--- `service-status tools` (exit 0): the same line.--- `plan tools` (exit 0): `tools: a331ea36963a → a331ea36963a (up to date) on main`; strategy `multi-app`, `0`-- files changed, `restart no`.--- `validate tools` (exit 0): `ok checkout /opt/openvibe.tools owned by ubuntu`; `ok checkout clean at-- a331ea36963a`; all eight `openvibe-tools*` units `active/running`; `ok port 4001 held by node`; `ok deps every-- dependency resolves`; `tools: valid (0 error(s), 0 warning(s))`.--- `releases tools` (exit 0): `834bed39e5b8 → 98f8bebf73ee → a331ea36963a deployed` (2026-10-03T11:35Z,-- 2026-10-04T00:24Z, 2026-10-04T03:00Z). `a331ea36963a` is origin/main HEAD and the release running in-- production; `834bed39e5b8` was the PostgreSQL-only release deployed during the cutover.----**Why this is a PostgreSQL-only release.** Every app boots through `apps/_shared/db.js`, which in production--throws `DATABASE_URL is not set: Tools serves from PostgreSQL (plan T8)` and only falls back to embedded PGlite--(PGlite outside production). A release that is `active` and `ready` therefore booted with `DATABASE_URL` set and--serves from PostgreSQL. No app depends on `better-sqlite3` any more, guarded by--`apps/_shared/test/no-sqlite.test.js`; the migrations (`migrations/0001_tools.sql` … `0004_event_outbox.sql`) run--only as the owner via `DATABASE_DIRECT_URL`, never at runtime.----## Env presence and mode — verified----- `validate tools`: `ok env /etc/openvibe/tools.env mode 600`; `ok env 4 required name(s) present and non-empty`.--- `env-set tools` (names only, `empty: []`): **`DATABASE_URL`**, **`DATABASE_DIRECT_URL`**, **`VALKEY_URL`**,-- `VALKEY_PREFIX`. `fromUnits`: `NODE_ENV`, `PATH`, `PORT`.----## Valkey ACL scope — pending----Presence of `VALKEY_URL` and `VALKEY_PREFIX` is provable (above). The scope of the Valkey user — which commands--it holds and that it is confined to `~ov:tools:*` (`/etc/valkey/services.acl`) — needs a connection to Valkey--that no read-only action opens. **Not checked.**----## Row parity / rollback copy — pending----The importer has already run and production writes PostgreSQL, but the per-table parity report and the rollback--copy (`$COPY` = `/var/backups/openvibe/tools-pre-t8-<T0>.db`, `chmod 0400`) are not exposed by any read-only--action, which runs no query and lists no file. **Not checked.**----## Reproduce----```bash--ov access run openvibe-ovh health tools--ov access run openvibe-ovh service-status tools--ov access run openvibe-ovh plan tools--ov access run openvibe-ovh releases tools--ov access run openvibe-ovh validate tools--ov access run openvibe-ovh env-set tools--ov access run openvibe-ovh journal openvibe-tools.service 200--```----Every one of these is a low-risk read. `deploy`, `rollback`, `restart` and `db-backup` were not used.@@@ -27,5 +27,5 @@"namespacesOwned": ["tools.*"],-- "notes": "Wave 11: asynchronous jobs on the img, audio and docs satellites (POST/GET/DELETE /api/v1/jobs, SSE events). Job lifecycle events tools.job.created|started|succeeded|failed (openvibe-contracts v0.30.1 payloads) go to OpenVibe.Events through an outbox in each satellite's jobs.db when EVENTS_URL is set. Tools consumes media.object.upload and media.object.read (namespace tools) when TOOLS_JOB_RESULTS=media."++ "notes": "Wave 11: asynchronous jobs on the img, audio and docs satellites (POST/GET/DELETE /api/v1/jobs, SSE events). Job lifecycle events tools.job.created|started|succeeded|failed (openvibe-contracts v0.30.1 payloads) go to OpenVibe.Events through the shared PostgreSQL event outbox when EVENTS_URL is set. Tools consumes media.object.upload and media.object.read (namespace tools) when TOOLS_JOB_RESULTS=media."}@@@ -1,66 +1,26 @@#!/usr/bin/env node--/**-- * Raw analytics retention and one-time scrub for the satellites' analytics.db files (ADR-021). The-- * command itself is openvibe-shared/analytics/prune-cli; this wrapper passes a better-sqlite3 from an-- * installed app, the --app flag and the discovery of apps/<app>/data/analytics.db.-- *-- * node scripts/analytics-prune.js # dry run over every apps/<app>/data/analytics.db-- * node scripts/analytics-prune.js --app yt --scrub # dry run for one app, including the scrub-- * node scripts/analytics-prune.js --apply --scrub --backup <dir> # back up each db into <dir>, then prune + scrub-- * node scripts/analytics-prune.js --apply --no-backup # prune without a backup (explicit)-- * node scripts/analytics-prune.js --help # every option-- *-- * --app <name> only this app (repeatable); default: every app that has data/analytics.db-- * (an app's DATA_DIR other than data/ needs --db)-- * --db <file> an analytics database by path (repeatable; replaces the app discovery)++/** Raw analytics retention in PostgreSQL. Dry run by default.++ * Usage: node scripts/analytics-prune.js [--days N] [--apply]*/'use strict';--const fs = require('fs');const path = require('path');----const ROOT = path.join(__dirname, '..');--const APPS = path.join(ROOT, 'apps');----/** A package from the first app that has it installed (the repo root has no node_modules). */++const APPS = path.join(__dirname, '..', 'apps');function fromApps(name) {for (const a of ['yt', 'maps', 'food', 'img', 'docs', 'audio', 'text', 'gateway']) {try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }}throw new Error(`${name} is not installed in any app (npm run install:all)`);}----/** better-sqlite3: scripts/' own dependency (npm --prefix scripts install), else an app that still has it. */--const { loadSqlite } = require('./sqlite');----/** [{ name, file }] when no --db is given: the --app list, else every app with data/analytics.db. */--function targets(args) {-- const names = args.apps.length ? args.apps : fs.readdirSync(APPS).filter((a) => !a.startsWith('_')).sort();-- return names.map((name) => ({ name, file: path.join(APPS, name, 'data', 'analytics.db') }))-- .filter((t) => args.apps.length || fs.existsSync(t.file));--}----const USAGE_APPS = ` --app <name> only this app (repeatable); default: every app that has data/analytics.db-- (an app's DATA_DIR other than data/ needs --db)-- --pg prune the one tools database (DATABASE_URL) with openvibe-shared/analytics/pg instead-- of the apps' analytics.db files (the default when DATABASE_URL is set and no --db is-- given); --days <n> and --apply apply here too, --app is refused (every service at once)`;----/** The tracker's service names in the one tools database (plan T8). */--const SERVICES = ['openvibe-gateway', 'openvibe-maps', 'openvibe-food', 'openvibe-img', 'openvibe-yt', 'openvibe-audio', 'openvibe-text', 'openvibe-docs'];--/**-- * The PostgreSQL branch (plan T8, decision 5): after the cutover the raw analytics events are rows in the-- * one tools database, not SQLite files. Prunes them with openvibe-shared/analytics/pg (the rollups stay);-- * without --apply it counts only, as the SQLite dry run does. The backup/scrub/VACUUM options are for the-- * pre-cutover files and do not apply here, and neither does --app: the retention is one rule for the whole-- * database (pruneRawEventsPg has no per-service filter), so an --app here is refused rather than silently-- * pruning every service.++ * Prune raw analytics rows in the one tools database. Rollups stay.++ * Without --apply the command only counts. Retention covers every service.*/async function pgMain(argv, log) {if (argv.includes('--app')) {throw new Error('--app does not apply to the one tools database: the PostgreSQL prune covers every service at once. '-- + 'Drop --app to prune them all, or pass --db <file> to prune one app\'s pre-cutover analytics.db');++ + 'Drop --app to prune them all.');}++ if (!String(process.env.DATABASE_URL || '').trim()) throw new Error('DATABASE_URL is required to prune analytics');const i = argv.indexOf('--days');const raw = i >= 0 ? parseInt(argv[i + 1], 10) : 30;const days = Number.isFinite(raw) ? Math.min(Math.max(raw, 1), 3650) : 30;@@@ -84,18 +44,12 @@ async function pgMain(argv, log) {}}--function main(argv, log) {-- if (argv.includes('--pg') || (process.env.DATABASE_URL && !argv.includes('--db'))) return pgMain(argv, log);-- const cli = fromApps('openvibe-shared/analytics/prune-cli');-- return cli.main(argv, {-- Database: loadSqlite(),-- log,-- options: { '--app': 'apps' },-- targets,-- usage: cli.USAGE.replace('\nOptions:\n', `\nOptions:\n${USAGE_APPS}\n`),-- });++function main(argv, log = console.log) {++ if (argv.includes('--help') || argv.includes('-h')) { log('Usage: node scripts/analytics-prune.js [--days N] [--apply]'); return; }++ const allowed = new Set(['--days', '--apply']);++ for (const arg of argv) if (arg.startsWith('--') && !allowed.has(arg)) throw new Error(`unknown option ${arg}`);++ if (!String(process.env.DATABASE_URL || '').trim()) throw new Error('DATABASE_URL is required to prune analytics');++ return pgMain(argv, log);}----if (require.main === module) fromApps('openvibe-shared/analytics/prune-cli').run(main);----module.exports = { main, pgMain, targets, loadSqlite, SERVICES };++if (require.main === module) Promise.resolve().then(() => main(process.argv.slice(2))).catch((err) => { console.error(err.message); process.exit(1); });++module.exports = { main, pgMain };@@@ -1,71 +1,25 @@#!/usr/bin/env node--/**-- * Phase 0 of plan T8 — enforce-mode readiness for the tools guard.-- *-- * In report mode (`TOOLS_GUARD` unset) the guard records every refusal it *would* make into-- * `guard_abuse` with `enforced = 0`. Before flipping `TOOLS_GUARD=enforce`, read those rows and-- * confirm no reason fires on ordinary traffic: a false positive that report mode merely logged-- * becomes a 429 for a real visitor in enforce mode.-- *-- * This groups `guard_abuse WHERE enforced = 0` by (reason, tool) and prints, per reason/tool, how-- * many rows (and how many repeats the `count` column folds in), across how many hashed addresses,-- * and the window it happened in. The PowerShell with `enforced = 1` rows is printed too, for-- * context. It reads copies — never a live file it also writes.-- *-- * node scripts/guard-abuse-report.js # every apps/<app>/data/guard.db-- * node scripts/guard-abuse-report.js --app maps --app gateway-- * node scripts/guard-abuse-report.js --db /backups/maps/guard.db-- * node scripts/guard-abuse-report.js --json # machine-readable, one JSON document-- * node scripts/guard-abuse-report.js --pg # the migrated database (DATABASE_URL)-- * node scripts/guard-abuse-report.js --pg --url postgres://… # a specific PostgreSQL database-- *-- * --app <name> only this app (repeatable); default: every app that has data/guard.db-- * --db <file> a guard database by path (repeatable; replaces the app discovery)-- * --pg read guard_abuse from PostgreSQL (DATABASE_URL, or --url) instead of SQLite files-- * --url <url> the PostgreSQL URL (implies --pg)-- * --json emit JSON instead of text-- *-- * After the T8 migration the guard's abuse log lives in the one `tools` database, so --pg is the-- * form to use on production once the rows have moved. Before that, the SQLite files are the source.-- */++/** Report guard abuse rows from PostgreSQL. Usage: node scripts/guard-abuse-report.js [--url URL] [--json] */'use strict';--const fs = require('fs');const path = require('path');--const { loadSqlite } = require('./sqlite');----const ROOT = path.join(__dirname, '..');--const APPS = path.join(ROOT, 'apps');----/** A package from the first app that has it installed (the repo root has no node_modules). */++const APPS = path.join(__dirname, '..', 'apps');function fromApps(name) {for (const a of ['gateway', 'maps', 'img', 'docs', 'audio', 'text', 'yt', 'food']) {try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }}throw new Error(`${name} is not installed in any app (npm run install:all)`);}--function parseArgs(argv) {-- const out = { apps: [], dbs: [], pg: false, url: null, json: false };++ const out = { url: null, json: false };for (let i = 0; i < argv.length; i++) {const a = argv[i];-- if (a === '--app') out.apps.push(argv[++i]);-- else if (a === '--db') out.dbs.push(argv[++i]);-- else if (a === '--pg') out.pg = true;-- else if (a === '--url') { out.url = argv[++i]; out.pg = true; }++ if (a === '--url') out.url = argv[++i];else if (a === '--json') out.json = true;else if (a === '--help' || a === '-h') out.help = true;else throw new Error(`unknown option "${a}" (--help for usage)`);}return out;}----/** [{ name, file }] when no --db is given: the --app list, else every app with data/guard.db. */--function targets(args) {-- const names = args.apps.length ? args.apps : fs.readdirSync(APPS).filter((a) => !a.startsWith('_')).sort();-- return names.map((name) => ({ name, file: path.join(APPS, name, 'data', 'guard.db') }))-- .filter((t) => args.apps.length || fs.existsSync(t.file));--}--// reason, tool, and how many rows/repeats/minutes/addresses a group covers, plus its window.const SELECT = `SELECT reason,@@@ -88,28 +42,6 @@ SELECT reason, tool, COUNT(*) AS rows, COALESCE(SUM(count), 0) AS eventsGROUP BY reason, toolORDER BY events DESC, reason, tool`;--function tableExists(db, name) {-- return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name));--}----function reportSqlite(results, target, Database) {-- if (!fs.existsSync(target.file)) { results.push({ app: target.name, file: target.file, present: false }); return; }-- // Read-only: a report never writes to the file it reads (a live file may be in WAL).-- const db = new Database(target.file, { readonly: true, fileMustExist: true });-- try {-- if (!tableExists(db, 'guard_abuse')) { results.push({ app: target.name, file: target.file, present: true, hasTable: false }); return; }-- results.push({-- app: target.name,-- file: target.file,-- present: true,-- hasTable: true,-- mode: 'sqlite',-- reported: db.prepare(SELECT).all(),-- enforced: db.prepare(SELECT_ENFORCED).all(),-- });-- } finally { db.close(); }--}--async function reportPg(results, url) {const { Client } = fromApps('pg');const client = new Client({ connectionString: url });@@@ -131,8 +63,8 @@ function printReport(results, log) {if (!r.present) { log(`\n${r.app}: no guard database (${r.file}) — skipped`); continue; }if (r.hasTable === false) { log(`\n${r.app}: ${r.file} has no guard_abuse table — skipped`); continue; }any = true;-- log(`\n${r.app} (${r.file})$ ${r.reported.length} reason/tool group${r.reported.length === 1 ? '' : 's'} recorded but not enforced`);-- if (!r.reported.length) { log(' (nothing was reported: no would-be refusal on this copy)'); }++ log(`\n${r.app} (${r.file}) ${r.reported.length} reason/tool group${r.reported.length === 1 ? '' : 's'} recorded but not enforced`);++ if (!r.reported.length) { log(' (nothing was reported: no would-be refusal in the database)'); }for (const row of r.reported) {log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`+ ` rows=${row.rows} addresses=${row.addresses} ${fmtTime(row.first_at)} … ${fmtTime(row.last_at)}`);@@@ -142,7 +74,7 @@ function printReport(results, log) {for (const row of r.enforced) log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`);}}-- if (!any) log('\nNo guard database with a guard_abuse table was found.');++ if (!any) log('\nNo guard abuse rows were found.');log('\nReview each reason/tool pair above: any one that fires on ordinary traffic is a false');log('positive that TOOLS_GUARD=enforce would turn into a 429 for a real visitor.');}@@@ -150,20 +82,13 @@ function printReport(results, log) {async function main(argv, log = console.log) {const args = parseArgs(argv);if (args.help) {-- const usage = fs.readFileSync(__filename, 'utf8').split('\n').filter((l) => /^ \*/.test(l)).slice(0, 22).join('\n');-- log(usage.replace(/^ \* ?/gm, ''));++ log('Usage: node scripts/guard-abuse-report.js [--url URL] [--json]');return;}const results = [];-- if (args.pg) {-- const url = args.url || process.env.DATABASE_URL;-- if (!url) throw new Error('--pg needs DATABASE_URL or --url');-- await reportPg(results, url);-- } else {-- const Database = loadSqlite();-- const list = args.dbs.length ? args.dbs.map((f) => ({ name: path.basename(path.dirname(path.dirname(f))) || 'db', file: path.resolve(f) })) : targets(args);-- for (const t of list) reportSqlite(results, t, Database);-- }++ const url = args.url || process.env.DATABASE_URL;++ if (!url) throw new Error('DATABASE_URL or --url is required');++ await reportPg(results, url);if (args.json) log(JSON.stringify({ generated_at: new Date().toISOString(), results }, null, 2));else printReport(results, log);}@@@ -172,4 +97,4 @@ if (require.main === module) {main(process.argv.slice(2)).catch((err) => { console.error(err.message); process.exit(1); });}--module.exports = { main, targets, parseArgs, SELECT, SELECT_ENFORCED, fromApps, printReport };++module.exports = { main, parseArgs, SELECT, SELECT_ENFORCED, fromApps, printReport };ddeleted file mode 100644@@@ -1,159 +0,0 @@--'use strict';--// ═══════════════════════════════════════════════════════════════--// scripts/migrate-to-postgres.js — the one-time move of every Tools SQLite file into the ONE `tools`--// PostgreSQL database (plan T8, decision 3/6; ADR-035). The operator (Opus) runs it at cutover.--//--// node scripts/migrate-to-postgres.js # production: DATABASE_DIRECT_URL (owner role)--// node scripts/migrate-to-postgres.js --pglite # an in-memory rehearsal (report only)--// node scripts/migrate-to-postgres.js --dry-run --url postgres://…/tools_scratch # a scratch database--// node scripts/migrate-to-postgres.js --app maps --app gateway [--json]--//--// Every app's data/<guard|jobs|analytics>.db is imported into the shared tables, tagged with its app id--// (`app` = the app name; the analytics tables already carry `service`). guard_salt and guard_day are not--// copied (Valkey, decision 4); the event outbox's rows and token_revocations are left behind (the tables are--// migrations 0004 and 0003; an undelivered SQLite outbox row is not replayed, and a revocation cutoff is re-derivable). The SQLite files are opened read-only.--//--// It relies on the newer openvibe-sdk/db (0.25+) and better-sqlite3; OV_SDK_DIR points at an openvibe-sdk--// checkout, else the gateway's node_modules is used. better-sqlite3 is scripts/' own dependency (no app--// has it after plan T8): run `npm --prefix scripts install` once before the cutover. Not part of `npm test`.--// ═══════════════════════════════════════════════════════════════--const fs = require('fs');--const path = require('path');--const { createRequire } = require('module');--const { loadSqlite } = require('./sqlite');----const ROOT = path.join(__dirname, '..');--const APPS = path.join(ROOT, 'apps');--const MIGRATIONS = path.join(ROOT, 'migrations');----// One row per kind of file, in every app. `tables` = the target tables that file fills; `app` = the ones--// that carry the app id; `skip` = source tables deliberately not imported (importSqlite would otherwise--// report them as having no target); `omit` = identity columns the target must assign itself (each app's--// ids are per-app and would collide in the one shared table: the import reads a temp table of the same--// name without them, so PostgreSQL issues fresh, globally unique ids — nothing references these ids).--const KINDS = {-- 'guard.db': { tables: ['guard_abuse'], app: ['guard_abuse'], skip: ['guard_salt', 'guard_day'], omit: { guard_abuse: ['id'] } },-- 'jobs.db': { tables: ['tool_jobs', 'tool_job_events', 'tool_job_references', 'tool_job_usage'], app: ['tool_jobs', 'tool_job_usage'], skip: ['event_outbox'], omit: { tool_job_events: ['seq'] } },-- 'analytics.db': { tables: ['analytics_events', 'analytics_hourly', 'analytics_daily', 'analytics_visitor_days', 'analytics_day_salts'], app: [], skip: ['analytics_rate_tracking'], omit: { analytics_events: ['id'], analytics_hourly: ['id'], analytics_daily: ['id'] } },--};--const TARGETS = [...new Set(Object.values(KINDS).flatMap((k) => k.tables))];----const argv = process.argv.slice(2);--const has = (f) => argv.includes(`--${f}`);--const opt = (n) => { const i = argv.indexOf(`--${n}`); return i >= 0 ? argv[i + 1] || null : null; };--const repeat = (n) => argv.flatMap((a, i) => (a === `--${n}` && argv[i + 1] ? [argv[i + 1]] : []));--const quiet = { log() {}, warn: console.warn, error: console.error };----if (has('help')) {-- console.log('usage: node scripts/migrate-to-postgres.js [--pglite | --url <DATABASE_DIRECT_URL>] [--dry-run] [--app <name>]… [--json]');-- process.exit(0);--}----/** Resolve openvibe-sdk/db from OV_SDK_DIR or the gateway's node_modules. */--function sdkRequire() {-- const dir = process.env.OV_SDK_DIR || path.join(APPS, 'gateway', 'node_modules', 'openvibe-sdk');-- try {-- const req = createRequire(path.join(dir, 'package.json'));-- return req('openvibe-sdk/db');-- } catch (err) {-- throw new Error(`openvibe-sdk/db could not be loaded from ${dir} (${err.message}); it needs openvibe-sdk 0.25+ (OV_SDK_DIR overrides)`);-- }--}----/** Every app's data/<name>.db, in a deterministic order, optionally narrowed with --app. */--function discover(apps) {-- const out = [];-- for (const app of fs.readdirSync(APPS).sort()) {-- if (app.startsWith('_') || (apps.length && !apps.includes(app))) continue;-- const dir = path.join(APPS, app, 'data');-- if (!fs.existsSync(dir)) continue;-- for (const name of Object.keys(KINDS).sort()) {-- const file = path.join(dir, name);-- if (fs.existsSync(file) && fs.statSync(file).isFile() && fs.statSync(file).size > 0) out.push({ app, kind: name, file });-- }-- }-- return out;--}----const q = (x) => `"${String(x).replace(/"/g, '""')}"`;----/**-- * Import one SQLite file. Identity-keyed tables (guard_abuse.id, tool_job_events.seq, the analytics ids)-- * are read through a temp table of the same name without the identity column, so PostgreSQL assigns a-- * fresh, globally unique id (per-app ids would collide in the one shared database). Nothing references-- * those ids. The file is opened read-only.-- */--async function importFile({ file, spec, db, importSqlite, first, Database }) {-- const src = new Database(file, { readonly: true, fileMustExist: true });-- try {-- for (const [table, drop] of Object.entries(spec.omit || {})) {-- const have = src.prepare(`PRAGMA table_info(${q(table)})`).all().map((c) => c.name);-- if (!have.length) continue; // the source has no such table (a lazily created one): nothing to import-- const keep = have.filter((c) => !drop.includes(c));-- src.exec(`CREATE TEMP TABLE ${q(table)} AS SELECT ${keep.map(q).join(', ')} FROM main.${q(table)}`);-- }-- return await importSqlite({ sqlite: src, db, only: spec.tables, tables: {}, skipSource: spec.skip, truncate: false, verify: first, log: quiet });-- } finally { src.close(); }--}----async function main() {-- const apps = repeat('app');-- const sdk = sdkRequire();-- const { createDb, importSqlite } = sdk;-- const Database = loadSqlite(); // before any database is opened: a missing dependency fails fast---- let db;-- if (has('pglite') || (has('dry-run') && !opt('url'))) {-- db = createDb({ pglite: true, service: 'tools-import', log: quiet });-- } else {-- const url = opt('url') || process.env.DATABASE_DIRECT_URL;-- if (!url) throw new Error('set DATABASE_DIRECT_URL (the owner, direct connection), or pass --pglite / --dry-run --url <scratch>');-- if (has('dry-run') && !opt('url')) throw new Error('--dry-run needs an explicit scratch --url (or --pglite) so production is never touched');-- db = createDb({ url, service: 'tools-import', max: 2, log: quiet });-- }---- const files = discover(apps);-- const report = { store: db.store, files: [], tables: {}, problems: [] };-- const sourceCount = {}; // target table → rows seen in the SQLite files-- const doneKind = new Set();---- try {-- await db.migrate({ dir: MIGRATIONS, log: quiet });-- await db.query(`TRUNCATE ${TARGETS.map((t) => `"${t}"`).join(', ')} RESTART IDENTITY CASCADE`);---- for (const { app, kind, file } of files) {-- const spec = KINDS[kind];-- const first = !doneKind.has(kind);-- doneKind.add(kind);-- const r = await importFile({ file, spec, db, importSqlite, first, Database });-- const rows = {};-- for (const t of r.tables) { rows[t.table] = t.rows; sourceCount[t.table] = (sourceCount[t.table] || 0) + t.rows; }-- // Tag the app-scoped rows this file just inserted (they came in with the '' default).-- for (const t of spec.app) await db.query(`UPDATE "${t}" SET app = $1 WHERE app = ''`, [app]);-- if (!r.ok) for (const p of r.problems) report.problems.push({ app, kind, ...p });-- report.files.push({ app, kind, file: path.relative(ROOT, file), rows, verified: first, checksums: Object.fromEntries(r.tables.map((t) => [t.table, t.checksum]).filter(([, c]) => c)) });-- if (process.stdout.isTTY) process.stderr.write(`[import] ${app}/${kind}: ${JSON.stringify(rows)}\n`);-- }---- // Reconcile counts across apps (per-file verification only covers the first file of a kind).-- for (const t of TARGETS) {-- const pg = Number((await db.maybe(`SELECT count(*) AS n FROM "${t}"`)).n);-- const src = sourceCount[t] || 0;-- report.tables[t] = { source: src, postgres: pg, ok: src === pg };-- if (src !== pg) report.problems.push({ table: t, problem: `count mismatch: ${src} SQLite rows, ${pg} PostgreSQL rows` });-- }-- report.ok = report.problems.length === 0;-- } finally {-- await db.close();-- }---- if (has('json')) { console.log(JSON.stringify(report, null, 2)); }-- else {-- console.log(`import → ${report.store}: ${report.ok ? 'OK' : 'PROBLEMS'}`);-- for (const [t, c] of Object.entries(report.tables)) console.log(` ${t.padEnd(26)} ${String(c.source).padStart(7)} SQLite rows → ${String(c.postgres).padStart(7)} PostgreSQL rows${c.ok ? '' : ' MISMATCH'}`);-- for (const p of report.problems) console.log(` problem: ${p.app ? `${p.app}/${p.kind} ` : ''}${p.table}: ${p.problem}`);-- }-- process.exit(report.ok ? 0 : 1);--}----main().catch((err) => { console.error(`migrate-to-postgres: ${err.message}`); process.exit(1); });ddeleted file mode 100644@@@ -1,468 +0,0 @@--{-- "name": "openvibe-tools-scripts",-- "version": "1.0.0",-- "lockfileVersion": 3,-- "requires": true,-- "packages": {-- "": {-- "name": "openvibe-tools-scripts",-- "version": "1.0.0",-- "license": "MIT",-- "dependencies": {-- "better-sqlite3": "^11.10.0"-- },-- "engines": {-- "node": ">=22"-- }-- },-- "node_modules/base64-js": {-- "version": "1.5.1",-- "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",-- "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT"-- },-- "node_modules/better-sqlite3": {-- "version": "11.10.0",-- "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",-- "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",-- "hasInstallScript": true,-- "license": "MIT",-- "dependencies": {-- "bindings": "^1.5.0",-- "prebuild-install": "^7.1.1"-- }-- },-- "node_modules/bindings": {-- "version": "1.5.0",-- "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",-- "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",-- "license": "MIT",-- "dependencies": {-- "file-uri-to-path": "1.0.0"-- }-- },-- "node_modules/bl": {-- "version": "4.1.0",-- "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",-- "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",-- "license": "MIT",-- "dependencies": {-- "buffer": "^5.5.0",-- "inherits": "^2.0.4",-- "readable-stream": "^3.4.0"-- }-- },-- "node_modules/buffer": {-- "version": "5.7.1",-- "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",-- "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT",-- "dependencies": {-- "base64-js": "^1.3.1",-- "ieee754": "^1.1.13"-- }-- },-- "node_modules/chownr": {-- "version": "1.1.4",-- "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",-- "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",-- "license": "ISC"-- },-- "node_modules/decompress-response": {-- "version": "6.0.0",-- "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",-- "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",-- "license": "MIT",-- "dependencies": {-- "mimic-response": "^3.1.0"-- },-- "engines": {-- "node": ">=10"-- },-- "funding": {-- "url": "https://github.com/sponsors/sindresorhus"-- }-- },-- "node_modules/deep-extend": {-- "version": "0.6.0",-- "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",-- "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",-- "license": "MIT",-- "engines": {-- "node": ">=4.0.0"-- }-- },-- "node_modules/detect-libc": {-- "version": "2.1.2",-- "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",-- "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",-- "license": "Apache-2.0",-- "engines": {-- "node": ">=8"-- }-- },-- "node_modules/end-of-stream": {-- "version": "1.4.5",-- "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",-- "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",-- "license": "MIT",-- "dependencies": {-- "once": "^1.4.0"-- }-- },-- "node_modules/expand-template": {-- "version": "2.0.3",-- "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",-- "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",-- "license": "(MIT OR WTFPL)",-- "engines": {-- "node": ">=6"-- }-- },-- "node_modules/file-uri-to-path": {-- "version": "1.0.0",-- "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",-- "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",-- "license": "MIT"-- },-- "node_modules/fs-constants": {-- "version": "1.0.0",-- "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",-- "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",-- "license": "MIT"-- },-- "node_modules/github-from-package": {-- "version": "0.0.0",-- "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",-- "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",-- "license": "MIT"-- },-- "node_modules/ieee754": {-- "version": "1.2.1",-- "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",-- "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "BSD-3-Clause"-- },-- "node_modules/inherits": {-- "version": "2.0.4",-- "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",-- "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",-- "license": "ISC"-- },-- "node_modules/ini": {-- "version": "1.3.8",-- "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",-- "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",-- "license": "ISC"-- },-- "node_modules/mimic-response": {-- "version": "3.1.0",-- "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",-- "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",-- "license": "MIT",-- "engines": {-- "node": ">=10"-- },-- "funding": {-- "url": "https://github.com/sponsors/sindresorhus"-- }-- },-- "node_modules/minimist": {-- "version": "1.2.8",-- "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",-- "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",-- "license": "MIT",-- "funding": {-- "url": "https://github.com/sponsors/ljharb"-- }-- },-- "node_modules/mkdirp-classic": {-- "version": "0.5.3",-- "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",-- "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",-- "license": "MIT"-- },-- "node_modules/napi-build-utils": {-- "version": "2.0.0",-- "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",-- "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",-- "license": "MIT"-- },-- "node_modules/node-abi": {-- "version": "3.96.0",-- "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",-- "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",-- "license": "MIT",-- "dependencies": {-- "semver": "^7.3.5"-- },-- "engines": {-- "node": ">=10"-- }-- },-- "node_modules/once": {-- "version": "1.4.0",-- "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",-- "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",-- "license": "ISC",-- "dependencies": {-- "wrappy": "1"-- }-- },-- "node_modules/prebuild-install": {-- "version": "7.1.3",-- "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",-- "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",-- "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",-- "license": "MIT",-- "dependencies": {-- "detect-libc": "^2.0.0",-- "expand-template": "^2.0.3",-- "github-from-package": "0.0.0",-- "minimist": "^1.2.3",-- "mkdirp-classic": "^0.5.3",-- "napi-build-utils": "^2.0.0",-- "node-abi": "^3.3.0",-- "pump": "^3.0.0",-- "rc": "^1.2.7",-- "simple-get": "^4.0.0",-- "tar-fs": "^2.0.0",-- "tunnel-agent": "^0.6.0"-- },-- "bin": {-- "prebuild-install": "bin.js"-- },-- "engines": {-- "node": ">=10"-- }-- },-- "node_modules/pump": {-- "version": "3.0.4",-- "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",-- "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",-- "license": "MIT",-- "dependencies": {-- "end-of-stream": "^1.1.0",-- "once": "^1.3.1"-- }-- },-- "node_modules/rc": {-- "version": "1.2.8",-- "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",-- "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",-- "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",-- "dependencies": {-- "deep-extend": "^0.6.0",-- "ini": "~1.3.0",-- "minimist": "^1.2.0",-- "strip-json-comments": "~2.0.1"-- },-- "bin": {-- "rc": "cli.js"-- }-- },-- "node_modules/readable-stream": {-- "version": "3.6.2",-- "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",-- "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",-- "license": "MIT",-- "dependencies": {-- "inherits": "^2.0.3",-- "string_decoder": "^1.1.1",-- "util-deprecate": "^1.0.1"-- },-- "engines": {-- "node": ">= 6"-- }-- },-- "node_modules/safe-buffer": {-- "version": "5.2.1",-- "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",-- "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT"-- },-- "node_modules/semver": {-- "version": "7.8.5",-- "resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",-- "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",-- "license": "ISC",-- "bin": {-- "semver": "bin/semver.js"-- },-- "engines": {-- "node": ">=10"-- }-- },-- "node_modules/simple-concat": {-- "version": "1.0.1",-- "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",-- "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT"-- },-- "node_modules/simple-get": {-- "version": "4.0.1",-- "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",-- "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",-- "funding": [-- {-- "type": "github",-- "url": "https://github.com/sponsors/feross"-- },-- {-- "type": "patreon",-- "url": "https://www.patreon.com/feross"-- },-- {-- "type": "consulting",-- "url": "https://feross.org/support"-- }-- ],-- "license": "MIT",-- "dependencies": {-- "decompress-response": "^6.0.0",-- "once": "^1.3.1",-- "simple-concat": "^1.0.0"-- }-- },-- "node_modules/string_decoder": {-- "version": "1.3.0",-- "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",-- "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",-- "license": "MIT",-- "dependencies": {-- "safe-buffer": "~5.2.0"-- }-- },-- "node_modules/strip-json-comments": {-- "version": "2.0.1",-- "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",-- "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",-- "license": "MIT",-- "engines": {-- "node": ">=0.10.0"-- }-- },-- "node_modules/tar-fs": {-- "version": "2.1.5",-- "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",-- "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",-- "license": "MIT",-- "dependencies": {-- "chownr": "^1.1.1",-- "mkdirp-classic": "^0.5.2",-- "pump": "^3.0.0",-- "tar-stream": "^2.1.4"-- }-- },-- "node_modules/tar-stream": {-- "version": "2.2.0",-- "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",-- "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",-- "license": "MIT",-- "dependencies": {-- "bl": "^4.0.3",-- "end-of-stream": "^1.4.1",-- "fs-constants": "^1.0.0",-- "inherits": "^2.0.3",-- "readable-stream": "^3.1.1"-- },-- "engines": {-- "node": ">=6"-- }-- },-- "node_modules/tunnel-agent": {-- "version": "0.6.0",-- "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",-- "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",-- "license": "Apache-2.0",-- "dependencies": {-- "safe-buffer": "^5.0.1"-- },-- "engines": {-- "node": "*"-- }-- },-- "node_modules/util-deprecate": {-- "version": "1.0.2",-- "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",-- "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",-- "license": "MIT"-- },-- "node_modules/wrappy": {-- "version": "1.0.2",-- "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",-- "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",-- "license": "ISC"-- }-- }--}ddeleted file mode 100644@@@ -1,13 +0,0 @@--{-- "name": "openvibe-tools-scripts",-- "version": "1.0.0",-- "private": true,-- "description": "The operational scripts' own dependencies: better-sqlite3 reads the pre-cutover SQLite files (scripts/migrate-to-postgres.js, scripts/analytics-prune.js, scripts/guard-abuse-report.js). No app depends on it (plan T8). Install with npm --prefix scripts install.",-- "license": "MIT",-- "engines": {-- "node": ">=22"-- },-- "dependencies": {-- "better-sqlite3": "^11.10.0"-- }--}ddeleted file mode 100644@@@ -1,17 +0,0 @@--'use strict';--// better-sqlite3 for the operational scripts that read the pre-cutover SQLite files (the importer, the--// analytics prune's SQLite branch, the guard abuse report). No app depends on it any more (plan T8), so--// scripts/ has its own package.json: `npm --prefix scripts install`. An app that still has it installed--// (a host before the cutover deploy) is the fallback.--const path = require('path');----const APPS = path.join(__dirname, '..', 'apps');----function loadSqlite() {-- for (const dir of [__dirname, ...['gateway', 'maps', 'food', 'img', 'yt', 'audio', 'text', 'docs'].map((a) => path.join(APPS, a))]) {-- try { return require(require.resolve('better-sqlite3', { paths: [dir] })); } catch { /* next */ }-- }-- throw new Error('better-sqlite3 is not installed for scripts/ (run npm --prefix scripts install)');--}----module.exports = { loadSqlite };
@@ -16,7 +16,7 @@ offline against the Network's public key (JWKS).
@@ -16,7 +16,7 @@ offline against the Network's public key (JWKS).
- the tool registry (families, tools, descriptors `tools.tool@1`, canonical, short and alias hosts),
- the tool registry (families, tools, descriptors `tools.tool@1`, canonical, short and alias hosts),
the catalog and every tool page on `openvibe.tools` and `*.openvibe.tools`
the catalog and every tool page on `openvibe.tools` and `*.openvibe.tools`
- the run API and the jobs runtime (img, audio, docs jobs with their files and SSE progress), the
- the run API and the jobs runtime (img, audio, docs jobs with their files and SSE progress), the
- guard (callers, quotas, sniffing, egress throttles, abuse log) and each app's `data/guard.db`
+ guard (callers, quotas, sniffing, egress throttles, PostgreSQL abuse log)
- the `tools.*` events, Tools' Search documents and the `tools.usage` user module
- the `tools.*` events, Tools' Search documents and the `tools.usage` user module
## Does not own
## Does not own
@@ -69,7 +69,7 @@ apps/_shared # openvibe-tools-shared, a versioned package (package.json, CHANG
@@ -69,7 +69,7 @@ apps/_shared # openvibe-tools-shared, a versioned package (package.json, CHANG
# the SSRF guard (egress.js) that every tool reaching a visitor-chosen host or URL goes
# the SSRF guard (egress.js) that every tool reaching a visitor-chosen host or URL goes
# through (public addresses only, checked after DNS and dialled as checked, redirect hops
# through (public addresses only, checked after DNS and dialled as checked, redirect hops
# re-checked). Visit analytics come from openvibe-shared/analytics.
# re-checked). Visit analytics come from openvibe-shared/analytics.
-scripts/analytics-prune.js # operator CLI: raw analytics retention + one-time scrub (dry run by default)
+scripts/analytics-prune.js # operator CLI: PostgreSQL raw analytics retention (dry run by default)
(openvibe-shared is a pinned OpenVibe.Shared release in each app's package.json; no vendor/ copy)
(openvibe-shared is a pinned OpenVibe.Shared release in each app's package.json; no vendor/ copy)
```
```
@@ -109,9 +109,7 @@ as the checkout owner, installs in each app whose dependencies changed (apps/_*
@@ -109,9 +109,7 @@ as the checkout owner, installs in each app whose dependencies changed (apps/_*
checks every dependency resolves and that the jobs runtime and the guard load, restarts every
checks every dependency resolves and that the jobs runtime and the guard load, restarts every
`openvibe-tools*` unit, waits for the gateway's `/api/ready` and its `/release.json` to name the new sha
`openvibe-tools*` unit, waits for the gateway's `/api/ready` and its `/release.json` to name the new sha
with every unit active, rolls back if they do not, and announces the release (`host.release.published`
with every unit active, rolls back if they do not, and announces the release (`host.release.published`
-to OpenVibe.Events, so open tabs check `/release.json` within seconds). When ovhost is missing, too old or
-does not deploy Tools with that strategy, the wrapper runs `deploy/scripts/deploy-legacy.sh`, the previous
-script, unchanged (`OVHOST_LEGACY=1` forces it).
+to OpenVibe.Events, so open tabs check `/release.json` within seconds). If ovhost is missing, the wrapper exits with an error.
## Tool registry API (ADR-027)
## Tool registry API (ADR-027)
@@ -281,8 +279,8 @@ reload reattaches.
@@ -281,8 +279,8 @@ reload reattaches.
`tools.job.created` (submit or retry; actor the owner), `tools.job.started` (a worker claimed it; a job
`tools.job.created` (submit or retry; actor the owner), `tools.job.started` (a worker claimed it; a job
re-queued after a restart is announced again when it starts), `tools.job.succeeded` and `tools.job.failed`
re-queued after a restart is announced again when it starts), `tools.job.succeeded` and `tools.job.failed`
(payloads `openvibe-contracts` `tools.job.*@1`, validated before they are queued; subject `job <id>`, visibility
(payloads `openvibe-contracts` `tools.job.*@1`, validated before they are queued; subject `job <id>`, visibility
- `internal`, source `tools`). Each is written to an `event_outbox` table in the satellite's `jobs.db` (openvibe-sdk
- `createOutbox`) in the same SQLite transaction as the state change, so an event exists exactly when its
+ `internal`, source `tools`). Each is written to the shared PostgreSQL `event_outbox` table (openvibe-sdk
+ `createPgOutbox`) in the same transaction as the state change, so an event exists exactly when its
transition committed, and relayed to `EVENTS_URL/api/v1/events` (at least once; Events dedupes on `event_id`).
transition committed, and relayed to `EVENTS_URL/api/v1/events` (at least once; Events dedupes on `event_id`).
Payloads carry ids, type, owner, state, attempts, times, where result files are (index, mime, size, sha256,
Payloads carry ids, type, owner, state, attempts, times, where result files are (index, mime, size, sha256,
`local` or a Media `media_id`) and the error (`status`, `code`, `detail` with server paths and the job's file
`local` or a Media `media_id`) and the error (`status`, `code`, `detail` with server paths and the job's file
@@ -344,8 +342,8 @@ One module used by the gateway and every satellite, driven by each tool's descri
@@ -344,8 +342,8 @@ One module used by the gateway and every satellite, driven by each tool's descri
address anything reads. The gateway's proxy to a satellite passes its own `req.ip` as that one hop; food passes its
address anything reads. The gateway's proxy to a satellite passes its own `req.ip` as that one hop; food passes its
visitor's to maps. A first-party service calling on loopback is identified by its service token (each has its own
visitor's to maps. A first-party service calling on loopback is identified by its service token (each has its own
bucket at the service tier); without one it is anonymous, counted by the address it forwarded or its own.
bucket at the service tier); without one it is anonymous, counted by the address it forwarded or its own.
-- **Quotas.** A token bucket per quota class × tier in memory (`perMinute`, `burst`) and a UTC-day allowance in the
- app's `data/guard.db` (survives restarts), both counting the tool's `cost`. Browser sessions from one address share
+- **Quotas.** A token bucket per quota class × tier (`perMinute`, `burst`) and a UTC-day allowance in
+ Valkey when configured (process-local otherwise), both counting the tool's `cost`. Browser sessions from one address share
3 × one session's allowance, so dropping the cookie never resets anything. Counted answers carry
3 × one session's allowance, so dropping the cookie never resets anything. Counted answers carry
`RateLimit-Limit`/`-Remaining`/`-Reset`; a refusal is `429` problem+json `tools.quota.exceeded` with `Retry-After`
`RateLimit-Limit`/`-Remaining`/`-Reset`; a refusal is `429` problem+json `tools.quota.exceeded` with `Retry-After`
and `quota_class`, `scope`, `tier`.
and `quota_class`, `scope`, `tier`.
@@ -367,7 +365,7 @@ One module used by the gateway and every satellite, driven by each tool's descri
@@ -367,7 +365,7 @@ One module used by the gateway and every satellite, driven by each tool's descri
belong to their maker (session, sign-in or token; anyone else gets 404), at most 5 per owner and 10 per address.
belong to their maker (session, sign-in or token; anyone else gets 404), at most 5 per owner and 10 per address.
YouTube downloads belong to whoever started them. Nominatim is paced to one request a second; Overpass waits are
YouTube downloads belong to whoever started them. Nominatim is paced to one request a second; Overpass waits are
capped.
capped.
-- **Abuse log.** Every refusal, and every would-be refusal in report mode, goes to `guard.db`: time, `HMAC-SHA256`
+- **Abuse log.** Every refusal, and every would-be refusal in report mode, goes to PostgreSQL `guard_abuse`: time, `HMAC-SHA256`
of the address (or /64) with a random daily salt (only today's is kept, so older hashes cannot be tied to
of the address (or /64) with a random daily salt (only today's is kept, so older hashes cannot be tied to
anything), principal or user id, tool, reason, enforced or not; repeats within a minute are one row with a count;
anything), principal or user id, tool, reason, enforced or not; repeats within a minute are one row with a count;
kept 30 days. No raw address is stored anywhere (pseudonymous, ADR-021). Metrics: `tools_guard_refused_total{reason,tool}`,
kept 30 days. No raw address is stored anywhere (pseudonymous, ADR-021). Metrics: `tools_guard_refused_total{reason,tool}`,
@@ -450,7 +448,7 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `ope
@@ -450,7 +448,7 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `ope
also answers `location = /metrics` with 404. HTTP golden signals are labelled by route template
also answers `location = /metrics` with 404. HTTP golden signals are labelled by route template
(`/api/v1/jobs/:id`), proxied satellite traffic on the gateway by satellite (`proxy:img`), SPA pages and
(`/api/v1/jobs/:id`), proxied satellite traffic on the gateway by satellite (`proxy:img`), SPA pages and
static files by a fixed label — never the raw URL. `release_info{service="tools"|"tools-<app>"}`, process
static files by a fixed label — never the raw URL. `release_info{service="tools"|"tools-<app>"}`, process
- metrics, and on img/audio/docs `tools_jobs{app,state}` (a group-by on jobs.db) and
+ metrics, and on img/audio/docs `tools_jobs{app,state}` (a group-by on the shared job store) and
`tools_jobs_executing{app,kind="executing"|"limit"}`.
`tools_jobs_executing{app,kind="executing"|"limit"}`.
- `GET /api/ready` — named checks with `status`, `required`, `latency_ms`, `checked_at`; HTTP 503 only when a
- `GET /api/ready` — named checks with `status`, `required`, `latency_ms`, `checked_at`; HTTP 503 only when a
required check fails, otherwise 200 with failed optional checks listed in `degraded`. `/api/health` is
required check fails, otherwise 200 with failed optional checks listed in `degraded`. `/api/health` is
@@ -458,12 +456,12 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `ope
@@ -458,12 +456,12 @@ Every server (gateway and satellites) mounts `apps/_shared/observe.js` with `ope
| Server | Required | Optional (degraded when failing) |
| Server | Required | Optional (degraded when failing) |
|---|---|---|
|---|---|---|
-| gateway (4001) | `catalog` | `network_key`, `service_directory` (Network registry vs fallback list), `community` (`/api/ready`), `satellite_<app>` for all seven (`/api/ready`, cached 15 s), `tool_registry` (every catalogue tool has a descriptor that meets the contracts) |
-| img, docs (4012, 4016) | `jobs_db` (query), `job_runtime` (worker started), `data_dir`, `uploads_dir`, `output_dir` (write test) | `analytics_db`, `network_key`, `media_results` (only with `TOOLS_JOB_RESULTS=media`); img: `heif_decoder`; docs: `qpdf`, `pdftoppm`, `pdfinfo` |
-| audio (4014) | same as img | same as img, plus `ffmpeg` on PATH |
-| yt (4013) | `downloads_dir` | `analytics_db`, `yt_dlp`, `ffmpeg`, `yt_cookies` (when `YT_COOKIES_FILE` is set), `network_key` |
-| food (4011) | `maps` (every food API is proxied to it) | `analytics_db` |
-| maps, text (4010, 4015) | — | `analytics_db` (maps' external data sources are not probed) |
+| gateway (4001) | `catalog`, `tools_db` | `network_key`, `service_directory`, `community`, `satellite_<app>`, `tool_registry` |
+| img, docs (4012, 4016) | `tools_db`, `job_runtime`, `data_dir`, `uploads_dir`, `output_dir` | `network_key`, `media_results` (when enabled); img: `heif_decoder`; docs: `qpdf`, `pdftoppm`, `pdfinfo` |
+| audio (4014) | same as img | same as img, plus `ffmpeg` |
+| yt (4013) | `tools_db`, `downloads_dir` | `yt_dlp`, `ffmpeg`, `yt_cookies` (when set), `network_key` |
+| food (4011) | `tools_db`, `maps` | — |
+| maps, text (4010, 4015) | `tools_db` | — |
`TOOLS_SATELLITE_PORTS="img=5012,…"` overrides the gateway's satellite ports (tests, a moved unit); every app reads
`TOOLS_SATELLITE_PORTS="img=5012,…"` overrides the gateway's satellite ports (tests, a moved unit); every app reads
it (`apps/_shared/tools/satellites.js`), so set it for all units if a port moves.
it (`apps/_shared/tools/satellites.js`), so set it for all units if a port moves.
@@ -498,31 +496,14 @@ are installed (or `QPDF_PATH` / `HEIF_DEC_PATH` point at them). Install first wi
@@ -498,31 +496,14 @@ are installed (or `QPDF_PATH` / `HEIF_DEC_PATH` point at them). Install first wi
## Analytics (ADR-021)
## Analytics (ADR-021)
-Every satellite except the gateway keeps visit analytics in its own `data/analytics.db` through
-`openvibe-shared/analytics` (since openvibe-shared v1.4.0; the one module Live, Tools and Network use).
-Bound by ADR-021 (OpenVibe.Contracts `docs/adr/ADR-021-analytics.md`):
-
-- **A raw row carries** event type, service, route template (matched Express route, else a normaliser: no
- query string, ids/hashes → `:id`, the segment after `watch`, `jobs`, `recipe`, `place`, … → `:param`,
- `/@x` → `/@:user`), method, status, response time, a rotating session id, country (CDN header),
- user-agent class (`chrome/windows/desktop`, `bot:googlebot`) + browser/os/device, referer origin, bot
- flags, a signed-in flag, timestamp. **Never** an IP, a user id, a city, the UA string or a full referer
- (`ip`/`user_id`/`city` stay as always-NULL columns for compatibility).
-- **Opt-out:** a request with `Sec-GPC: 1` or `DNT: 1` is not recorded at all (no raw row, visitor hash,
- session id or rate counter), so it is also missing from the rollups.
-- **Session id:** random, in memory against the visitor hash, new after 30 idle minutes and at UTC midnight.
-- **Bot rate check:** per-IP counters in memory only (current + previous minute); `analytics_rate_tracking`
- is emptied at boot and no longer written.
-- **Unique visitors:** `HMAC-SHA256(day salt, ip + "\n" + ua)` (16 hex chars), random salt per UTC day. Hashes
- and the salt are kept only until the first hourly aggregation after their day ends (right after the
- day's final rollup), never in raw rows. Rollups keep counts; sub-48 h raw summaries count sessions.
-- **Retention:** each satellite prunes raw rows older than 30 days in batches of 5000, 5 minutes after boot
- and every 24 h after; rollups stay.
-- **CLI:** `node scripts/analytics-prune.js`, a wrapper over `openvibe-shared/analytics/prune-cli` (dry run
- over every `apps/*/data/analytics.db`; `--app`, `--db` to narrow). `--apply` needs `--backup <file|dir>`
- (verified owner-only online backup; a directory when several databases are targeted) or `--no-backup`;
- `--scrub` also rewrites rows written before ADR-021 and the rollups' top lists (counts unchanged). Ends
- with VACUUM unless `--no-vacuum`.
+All eight apps store visit analytics in the shared PostgreSQL database through
+`openvibe-shared/analytics/pg` (ADR-021). Raw events include the service, route template,
+method, status, response time, session identifier and coarse visitor metadata. GPC and DNT opt-outs
+are excluded. Daily and hourly rollups remain after raw events expire.
+
+- **Retention:** the application prunes raw rows older than 30 days; rollups stay.
+- **CLI:** `node scripts/analytics-prune.js` counts old rows by service. Pass `--days N` to choose
+ retention and `--apply` to delete them. It requires `DATABASE_URL` and applies to every service.
## Host packages some tools need
## Host packages some tools need
@@ -635,36 +616,15 @@ when the gateway's `/api/ready` or `/release.json` does not come up with every u
@@ -635,36 +616,15 @@ when the gateway's `/api/ready` or `/release.json` does not come up with every u
`openvibe-tools-<name>.service` per satellite
`openvibe-tools-<name>.service` per satellite
(`apps/<name>/deploy/systemd/`). Resource bounds: `MemoryMax=2G` (img, audio,
(`apps/<name>/deploy/systemd/`). Resource bounds: `MemoryMax=2G` (img, audio,
docs), `1G` (yt), `768M` (gateway, text, maps, food); `TasksMax=256`; `Nice=5`
docs), `1G` (yt), `768M` (gateway, text, maps, food); `TasksMax=256`; `Nice=5`
- for img, audio, docs and yt. Every app writes `data/guard.db` (the gateway
- too: `ReadWritePaths=/opt/openvibe.tools/apps/gateway/data`; the deploy
- (`ovhost deploy tools`, or `deploy-legacy.sh`) creates each `data/` before restarting).
+ for img, audio, docs and yt. The gateway data directory remains writable for local runtime files.
- Nginx: satellites have specific `server_name` blocks; the gateway's
- Nginx: satellites have specific `server_name` blocks; the gateway's
wildcard `*.openvibe.tools` block catches everything else. TLS via
wildcard `*.openvibe.tools` block catches everything else. TLS via
`/etc/letsencrypt/live/openvibe.tools/`.
`/etc/letsencrypt/live/openvibe.tools/`.
-## Migration to PostgreSQL + Valkey (plan T8)
-
-All eight apps move from `better-sqlite3` to **one `tools` PostgreSQL database** and to **Valkey** for the
-guard's throttle state. The schema is in `migrations/0001_tools.sql` (job store, `guard_abuse`) and
-`migrations/0002_analytics.sql` (request analytics, the DDL of `openvibe-shared/analytics/pg.js`),
-`migrations/0003_token_revocations.sql` and `migrations/0004_event_outbox.sql` (the tools.job.* outbox,
-`openvibe-sdk` createPgOutbox's table). All of them run as the owner (`DATABASE_DIRECT_URL`); nothing creates a
-table at runtime, so the runtime role needs no DDL rights. `scripts/migrate-to-postgres.js` imports
-every `apps/*/data/{guard,jobs,analytics}.db` into that one database (each file's rows tagged with its app):
-run it with `--pglite` for an in-memory rehearsal, `--dry-run --url <scratch>` for a scratch database, or
-with `DATABASE_DIRECT_URL` set for the cutover; it prints a per-table count and checksum report.
-
-After the move the apps read `DATABASE_URL` / `DATABASE_DIRECT_URL` (PostgreSQL, via PgBouncer) and
-`VALKEY_URL` (Valkey); `guard_salt`, `guard_day` and the minute/burst buckets live in Valkey (a throttle,
-never authoritative), while `guard_abuse`, the job store and the analytics tables stay in PostgreSQL.
-`TOOLS_GUARD` (default `enforce`) and its meaning are unchanged.
-
-State (plan T8 landed 2026-10-02; production serves the PostgreSQL-only release): every app opens the one `tools`
-database (`apps/_shared/db.js`, `DATABASE_URL` else an embedded PGlite database in development;
-`DATABASE_DIRECT_URL` runs the migrations as owner) and the shared Valkey (`VALKEY_URL`, else the guard's
-counters and salt stay in this process); the job store, `guard_abuse`, `token_revocations` and the request
-analytics are on PostgreSQL, the net and dev pages call the run API, and the `/api/process`, `/api/net/*` and
-`/api/dev/*` routes are removed (above). No app depends on `better-sqlite3` any more; the operational scripts
-that still read the old SQLite files (the importer, `scripts/analytics-prune.js`, `scripts/guard-abuse-report.js`)
-have it in `scripts/package.json`: run `npm --prefix scripts install` on the host only to read the archived SQLite
-files (the importer has run; production writes PostgreSQL). The cutover evidence is in `docs/cutover-evidence-t8.md`.
+## Database and guard state
+
+All eight apps use one PostgreSQL database through `openvibe-sdk/db`. Production requires
+`DATABASE_URL`; `DATABASE_DIRECT_URL` lets the owner apply the immutable SQL files in `migrations/`
+at boot. Development and tests use embedded PGlite through the same API. Job records, abuse logs,
+revocations, analytics, and the event outbox live in PostgreSQL. The guard's transient throttle
+counters and salt use Valkey through `VALKEY_URL`, with process-local state when it is unset.
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
"stage": "stable",
"stage": "stable",
"code": true,
"code": true,
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
- "runtime": "node 22, express; PostgreSQL (openvibe-sdk/db, PGlite in development) + Valkey for the guard and analytics; the job store too (plan T8; better-sqlite3 only in scripts/ for the pre-cutover SQLite files); eight apps (gateway, maps, food, img, yt, audio, text, docs)",
+ "runtime": "node 22, express; PostgreSQL (openvibe-sdk/db, PGlite in development) + Valkey for the guard and analytics; the job store; eight apps (gateway, maps, food, img, yt, audio, text, docs)",
"port": 4001,
"port": 4001,
"ports": {
"ports": {
"gateway": 4001,
"gateway": 4001,
@@ -28,7 +28,7 @@
@@ -28,7 +28,7 @@
],
],
"capabilitiesRegistered": true,
"capabilitiesRegistered": true,
"deployed": true,
"deployed": true,
- "deployment": "Public at https://openvibe.tools: deployed with `sudo ovhost deploy tools` (OpenVibe.Host strategy multi-app since the 2026-09-27 cutover; deploy/scripts/deploy.sh is the wrapper), /opt/openvibe.tools, units openvibe-tools (gateway, 127.0.0.1:4001) and openvibe-tools-<app> per satellite, env /etc/openvibe/tools.env.",
+ "deployment": "Public at https://openvibe.tools: deployed with `sudo ovhost deploy tools` (OpenVibe.Host strategy multi-app; deploy/scripts/deploy.sh is the wrapper), /opt/openvibe.tools, units openvibe-tools (gateway, 127.0.0.1:4001) and openvibe-tools-<app> per satellite, env /etc/openvibe/tools.env.",
"features": [
"features": [
"tool registry with descriptors, canonical hosts, aliases and owner-attached custom domains",
"tool registry with descriptors, canonical hosts, aliases and owner-attached custom domains",
"server-rendered catalog and tool pages with full SEO",
"server-rendered catalog and tool pages with full SEO",
@@ -55,7 +55,7 @@ function openToolsDb({ createDb, service = 'tools', log = console, registry, pgl
@@ -55,7 +55,7 @@ function openToolsDb({ createDb, service = 'tools', log = console, registry, pgl
if (typeof createDb !== 'function') throw new TypeError('openToolsDb needs createDb (require(\'openvibe-sdk/db\').createDb)');
if (typeof createDb !== 'function') throw new TypeError('openToolsDb needs createDb (require(\'openvibe-sdk/db\').createDb)');
const url = String(process.env.DATABASE_URL || '').trim();
const url = String(process.env.DATABASE_URL || '').trim();
if (!url) {
if (!url) {
- if (String(process.env.NODE_ENV || '') === 'production') throw new Error('DATABASE_URL is not set: Tools serves from PostgreSQL (plan T8)');
+ if (String(process.env.NODE_ENV || '') === 'production') throw new Error('DATABASE_URL is required in production');
// A directory keeps development data across restarts; without one (tests) the database is in memory,
// A directory keeps development data across restarts; without one (tests) the database is in memory,
// so every run starts from the empty, migrated schema.
// so every run starts from the empty, migrated schema.
const dir = pgliteDir || process.env.TOOLS_PGLITE_DIR || '';
const dir = pgliteDir || process.env.TOOLS_PGLITE_DIR || '';
@@ -35,7 +35,7 @@ const COOKIE = 'ov_tools_jobs';
@@ -35,7 +35,7 @@ const COOKIE = 'ov_tools_jobs';
* @param {string} o.issuer
* @param {string} o.issuer
* @param {string} [o.audience='openvibe.tools']
* @param {string} [o.audience='openvibe.tools']
* @param {object} [o.contracts] openvibe-contracts, when the app has it
* @param {object} [o.contracts] openvibe-contracts, when the app has it
- * @param {() => string} [o.salt] today's salt (guard.db); a process-local one otherwise
+ * @param {() => string} [o.salt] today's salt (Valkey when configured); a process-local one otherwise
* @param {string} [o.cookieName='ov_tools_jobs']
* @param {string} [o.cookieName='ov_tools_jobs']
* @param {boolean} [o.secureCookie]
* @param {boolean} [o.secureCookie]
*/
*/
@@ -15,13 +15,13 @@
@@ -15,13 +15,13 @@
// callers one resolver (./caller.js): anonymous (address, IPv6 /64) < session < user < service;
// callers one resolver (./caller.js): anonymous (address, IPv6 /64) < session < user < service;
// sandbox tokens keep their small allowance
// sandbox tokens keep their small allowance
// quotas token buckets per class × tier weighted by the tool's cost (./quota.js), day allowances
// quotas token buckets per class × tier weighted by the tool's cost (./quota.js), day allowances
-// in guard.db; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal
+// in Valkey when configured; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal
// 429 problem+json tools.quota.exceeded with Retry-After
// 429 problem+json tools.quota.exceeded with Retry-After
// heavy a semaphore for synchronous heavy calls (503 tools.busy + Retry-After when it is full)
// heavy a semaphore for synchronous heavy calls (503 tools.busy + Retry-After when it is full)
// uploads the bytes are checked against the descriptor's files.accept (415 tools.file.unsupported_type);
// uploads the bytes are checked against the descriptor's files.accept (415 tools.file.unsupported_type);
// heavy tools (auth.anonymous false) need a session cookie, a sign-in or a token
// heavy tools (auth.anonymous false) need a session cookie, a sign-in or a token
// egress a per-target throttle across all callers (limits.perTargetPerMinute) and the port-scan cap
// egress a per-target throttle across all callers (limits.perTargetPerMinute) and the port-scan cap
-// log every refusal (and every would-be refusal in report mode) in guard.db's abuse log:
+// log every refusal (and every would-be refusal in report mode) in PostgreSQL guard_abuse:
// HMAC(address, today's salt), principal or user id, tool, reason — 30 days, no raw
// HMAC(address, today's salt), principal or user id, tool, reason — 30 days, no raw
// address anywhere; metric tools_guard_refused_total{reason,tool}
// address anywhere; metric tools_guard_refused_total{reason,tool}
// challenge a hook for a person-check (Turnstile later; ./challenge.js, a no-op now)
// challenge a hook for a person-check (Turnstile later; ./challenge.js, a no-op now)
@@ -8,7 +8,7 @@
@@ -8,7 +8,7 @@
// image conversion, 10 an audio effect, 50 a video download). Each quota class × caller tier has
// image conversion, 10 an audio effect, 50 a video download). Each quota class × caller tier has
// perMinute a token bucket refilled at this many units a minute (kept in memory)
// perMinute a token bucket refilled at this many units a minute (kept in memory)
// burst the bucket's size: what may be spent at once after a quiet spell
// burst the bucket's size: what may be spent at once after a quiet spell
-// perDay units per UTC day, persisted in the app's guard.db (0 = no day allowance)
+// perDay units per UTC day, persisted in the shared guard store (0 = no day allowance)
// Tiers: anonymous (keyed by IP, IPv6 by /64) < session (the ov_tools_jobs browser cookie) < user
// Tiers: anonymous (keyed by IP, IPv6 by /64) < session (the ov_tools_jobs browser cookie) < user
// (a Network sign-in) < service (a first-party service or a developer app's production token).
// (a Network sign-in) < service (a first-party service or a developer app's production token).
// sandbox is a developer app's sandbox token: the smallest allowance, keyed by the app.
// sandbox is a developer app's sandbox token: the smallest allowance, keyed by the app.
@@ -95,7 +95,7 @@ function bounds(env = process.env) {
@@ -95,7 +95,7 @@ function bounds(env = process.env) {
},
},
// Webhook request bins (in memory, one hour): per owner, per address, in all.
// Webhook request bins (in memory, one hour): per owner, per address, in all.
webhook: { perOwner: 5, perIp: 10, total: 500 },
webhook: { perOwner: 5, perIp: 10, total: 500 },
- // The abuse log (guard.db) keeps rows this long.
+ // The abuse log (PostgreSQL guard_abuse) keeps rows this long.
abuseRetentionDays: 30,
abuseRetentionDays: 30,
};
};
}
}
@@ -6,7 +6,7 @@
@@ -6,7 +6,7 @@
// (lifecycle), http.js (routes), media.js (results in Media), events.js (tools.job.* to OpenVibe.Events),
// (lifecycle), http.js (routes), media.js (results in Media), events.js (tools.job.* to OpenVibe.Events),
// usage.js (tools.usage.recorded, a developer project's jobs per hour), client.js (browser helper).
// usage.js (tools.usage.recorded, a developer project's jobs per hour), client.js (browser helper).
//
//
-// The job store is part of every app's one `tools` database: no per-satellite jobs.db.
+// The job store is part of every app's one `tools` database: shared across apps.
//
//
// Environment (one /etc/openvibe/tools.env for every unit):
// Environment (one /etc/openvibe/tools.env for every unit):
// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)
// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)
@@ -8,7 +8,7 @@
@@ -8,7 +8,7 @@
// const obs = require('../../_shared/observe').observe({
// const obs = require('../../_shared/observe').observe({
// app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
// app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
// service: 'tools-img', release: release.release,
// service: 'tools-img', release: release.release,
-// checks: [ checks.sqlite('analytics_db', analyticsDb, { required: false }), … ],
+// checks: [ checks.postgres('tools_db', db), … ],
// jobs: () => jobs, // the satellite's job system, once it exists (tools_jobs{app,state})
// jobs: () => jobs, // the satellite's job system, once it exists (tools_jobs{app,state})
// });
// });
//
//
@@ -77,19 +77,6 @@ function observe(o) {
@@ -77,19 +77,6 @@ function observe(o) {
// ── Check builders ──────────────────────────────────────────────
// ── Check builders ──────────────────────────────────────────────
const checks = {
const checks = {
- /** A real query on a better-sqlite3 handle (or a function returning one). */
- sqlite(name, db, { required = true, sql = 'SELECT 1 AS ok', description } = {}) {
- return {
- name, required, description,
- check: () => {
- const h = typeof db === 'function' ? db() : db;
- if (!h) return 'database not open';
- const row = h.prepare(sql).get();
- return row ? true : 'query returned no row';
- },
- };
- },
-
/** A real round trip on the one `tools` database (an openvibe-sdk/db handle, or a function
/** A real round trip on the one `tools` database (an openvibe-sdk/db handle, or a function
* returning one); the detail names the store it answered from (postgresql / pglite). */
* returning one); the detail names the store it answered from (postgresql / pglite). */
postgres(name, db, { required = true, description } = {}) {
postgres(name, db, { required = true, description } = {}) {
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
// stores no IP, user id, city, raw user agent, raw referer or query string anywhere in the analytics
// stores no IP, user id, city, raw user agent, raw referer or query string anywhere in the analytics
// tables; paths are route templates; uniques come from day-scoped hashes deleted after the day's rollup;
// tables; paths are route templates; uniques come from day-scoped hashes deleted after the day's rollup;
// pruneRawEventsPg removes strictly-older raw rows and day hashes in bounded batches and never touches
// pruneRawEventsPg removes strictly-older raw rows and day hashes in bounded batches and never touches
-// rollups. (The SQLite tracker's scrub and file CLI are retired with better-sqlite3.)
+// rollups.
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const os = require('os');
const os = require('os');
deleted file mode 100644
deleted file mode 100644
@@ -1,22 +0,0 @@
@@ -1,22 +0,0 @@
-'use strict';
-// The T8 closeout record must exist and keep its checks, and the README must no longer say the cutover is
-// pending (plan T8; a silent re-draft would lose the evidence). This fails loudly if either regresses.
-const assert = require('assert');
-const fs = require('fs');
-const path = require('path');
-
-const ROOT = path.join(__dirname, '..', '..', '..');
-const doc = path.join(ROOT, 'docs', 'cutover-evidence-t8.md');
-const readme = path.join(ROOT, 'README.md');
-
-assert.ok(fs.existsSync(doc), 'docs/cutover-evidence-t8.md is missing');
-const md = fs.readFileSync(doc, 'utf8');
-for (const key of ['health tools', 'origin/main HEAD', 'env', 'Valkey ACL', 'Row parity', 'Reproduce']) {
- assert.ok(md.includes(key), `docs/cutover-evidence-t8.md is missing "${key}"`);
-}
-
-assert.ok(fs.existsSync(readme), 'README.md is missing');
-const readmeText = fs.readFileSync(readme, 'utf8');
-assert.ok(!readmeText.includes('the cutover is pending'), 'README.md still contains "the cutover is pending"');
-
-console.log('cutover evidence exists, records its checks, and the README no longer says the cutover is pending: passed');
@@ -1,60 +1,33 @@
@@ -1,60 +1,33 @@
'use strict';
'use strict';
-/**
- * deploy/scripts/deploy.sh is a thin wrapper around `ovhost deploy tools` (OpenVibe.Host, strategy
- * multi-app; roadmap WS-N task 11) with deploy-legacy.sh (the previous script, unchanged) as its
- * fallback. A fake ovhost records what the wrapper asks for; a fake legacy script records the fallback.
- */
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const os = require('os');
const os = require('os');
const path = require('path');
const path = require('path');
const { spawnSync } = require('child_process');
const { spawnSync } = require('child_process');
-
const WRAPPER = path.join(__dirname, '..', '..', '..', 'deploy', 'scripts', 'deploy.sh');
const WRAPPER = path.join(__dirname, '..', '..', '..', 'deploy', 'scripts', 'deploy.sh');
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tools-deploy-wrapper-'));
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'tools-deploy-wrapper-'));
const log = path.join(tmp, 'calls.log');
const log = path.join(tmp, 'calls.log');
const ovhost = path.join(tmp, 'ovhost');
const ovhost = path.join(tmp, 'ovhost');
-fs.writeFileSync(ovhost, `#!/usr/bin/env bash
-if [ "$1" = capabilities ]; then
- [ -n "$FAKE_OLD" ] && exit 1
- printf '%b\\n' "\${FAKE_CAPS:-ovhost=0.3.0\\ndeploy-api=1\\nservice=tools\\nstrategy=multi-app\\nmanaged=yes\\nlayout=git}"
- exit 0
-fi
-echo "ovhost $*" >> "${log}"
-exit "\${FAKE_EXIT:-0}"
-`, { mode: 0o755 });
-const legacy = path.join(tmp, 'legacy.sh');
-fs.writeFileSync(legacy, `#!/usr/bin/env bash\necho "legacy $*" >> "${log}"\n`, { mode: 0o755 });
-
+fs.writeFileSync(ovhost, `#!/usr/bin/env bash\necho "$*" >> "${log}"\nexit "\${FAKE_EXIT:-0}"\n`, { mode: 0o755 });
function run(args = [], env = {}) {
function run(args = [], env = {}) {
fs.rmSync(log, { force: true });
fs.rmSync(log, { force: true });
- const r = spawnSync('bash', [WRAPPER, ...args], { env: { PATH: process.env.PATH, OVHOST: ovhost, OVHOST_SUDO: '', DEPLOY_LEGACY: legacy, ...env }, encoding: 'utf8' });
+ const r = spawnSync('bash', [WRAPPER, ...args], { env: { PATH: process.env.PATH, OVHOST: ovhost, OVHOST_SUDO: '', ...env }, encoding: 'utf8' });
let calls = [];
let calls = [];
try { calls = fs.readFileSync(log, 'utf8').trim().split('\n').filter(Boolean); } catch { /* none */ }
try { calls = fs.readFileSync(log, 'utf8').trim().split('\n').filter(Boolean); } catch { /* none */ }
return { code: r.status, out: r.stdout + r.stderr, calls };
return { code: r.status, out: r.stdout + r.stderr, calls };
}
}
-
-assert.deepStrictEqual(run().calls, ['ovhost deploy tools']);
-assert.deepStrictEqual(run(['--wait-idle']).calls, ['ovhost deploy tools --wait-idle']);
-assert.deepStrictEqual(run(['--rollback']).calls, ['ovhost rollback tools']);
-assert.deepStrictEqual(run([], { DRY_RUN: '1' }).calls, ['ovhost plan tools']);
-assert.strictEqual(run([], { FAKE_EXIT: '3' }).code, 3, "ovhost's exit code is the wrapper's");
+assert.deepStrictEqual(run().calls, ['deploy tools']);
+assert.deepStrictEqual(run(['--wait-idle']).calls, ['deploy tools --wait-idle']);
+assert.deepStrictEqual(run(['--restart']).calls, ['deploy tools --restart']);
+assert.deepStrictEqual(run(['--force']).calls, ['deploy tools --force']);
+assert.deepStrictEqual(run(['--rollback']).calls, ['rollback tools']);
+assert.deepStrictEqual(run([], { DRY_RUN: '1' }).calls, ['plan tools']);
+assert.strictEqual(run([], { FAKE_EXIT: '3' }).code, 3);
assert.strictEqual(run(['--nope']).code, 1);
assert.strictEqual(run(['--nope']).code, 1);
-
-let r = run([], { FAKE_OLD: '1' });
-assert.deepStrictEqual(r.calls, ['legacy'], 'too old: the legacy script');
-assert.match(r.out, /too old/);
-r = run([], { FAKE_CAPS: 'deploy-api=1\\nstrategy=git-checkout\\nmanaged=yes' });
-assert.deepStrictEqual(r.calls, ['legacy'], 'another strategy: the legacy script');
-assert.match(r.out, /does not deploy tools with strategy multi-app \(git-checkout\)/);
-r = run([], { OVHOST: path.join(tmp, 'missing') });
-assert.deepStrictEqual(r.calls, ['legacy']);
-for (const [args, env] of [[['--rollback'], {}], [['--wait-idle'], {}], [[], { DRY_RUN: '1' }]]) {
- r = run(args, { OVHOST_LEGACY: '1', ...env });
- assert.strictEqual(r.code, 1, `${args.join(' ')} ${JSON.stringify(env)}: the legacy script cannot, so nothing runs`);
- assert.deepStrictEqual(r.calls, []);
-}
-assert.match(fs.readFileSync(WRAPPER, 'utf8'), /^set -euo pipefail$/m);
-
+const missing = run([], { OVHOST: path.join(tmp, 'missing') });
+assert.strictEqual(missing.code, 1);
+assert.deepStrictEqual(missing.calls, []);
+assert.match(missing.out, /ovhost not found/);
+assert.doesNotMatch(fs.readFileSync(WRAPPER, 'utf8'), /legacy|capabilities|DEPLOY_LEGACY|OVHOST_LEGACY/i);
fs.rmSync(tmp, { recursive: true, force: true });
fs.rmSync(tmp, { recursive: true, force: true });
console.log('deploy wrapper: all checks passed');
console.log('deploy wrapper: all checks passed');
@@ -39,4 +39,19 @@ for (const m of manifests) {
@@ -39,4 +39,19 @@ for (const m of manifests) {
assert.ok(!(pkg.dependencies && pkg.dependencies['better-sqlite3']), `${path.relative(APPS, m)} still depends on better-sqlite3`);
assert.ok(!(pkg.dependencies && pkg.dependencies['better-sqlite3']), `${path.relative(APPS, m)} still depends on better-sqlite3`);
}
}
+// Migration tooling and legacy deployment must stay removed.
+const ROOT = path.resolve(APPS, '..');
+for (const name of [
+ 'scripts/migrate-to-postgres.js', 'scripts/sqlite.js', 'scripts/package.json',
+ 'scripts/package-lock.json', 'deploy/scripts/deploy-legacy.sh',
+ 'docs/cutover-evidence-t8.md',
+]) assert.ok(!fs.existsSync(path.join(ROOT, name)), `${name} must be removed`);
+for (const name of ['deploy/scripts/deploy.sh', 'scripts/analytics-prune.js', 'scripts/guard-abuse-report.js']) {
+ const source = fs.readFileSync(path.join(ROOT, name), 'utf8');
+ assert.doesNotMatch(source, /DEPLOY_LEGACY|OVHOST_LEGACY|\.\/sqlite|analytics\.db|guard\.db|deploy-legacy/);
+}
+for (const m of manifests) {
+ const pkg = JSON.parse(fs.readFileSync(m, 'utf8'));
+ for (const dep of ['better-sqlite3', 'sqlite3']) assert.ok(!pkg.dependencies?.[dep], `${m} depends on ${dep}`);
+}
console.log(`no sqlite (nothing under apps/ requires better-sqlite3): all checks passed (${files.length} files scanned)`);
console.log(`no sqlite (nothing under apps/ requires better-sqlite3): all checks passed (${files.length} files scanned)`);
@@ -41,7 +41,7 @@ function assertShape(body, service) {
@@ -41,7 +41,7 @@ function assertShape(body, service) {
const obs = observe({
const obs = observe({
app, metrics, ready, service: 'tools-test', release: 'abc1234567',
app, metrics, ready, service: 'tools-test', release: 'abc1234567',
checks: [
checks: [
- { name: 'db', required: true, check: () => (dbOk ? true : 'SQLITE_IOERR') },
+ { name: 'db', required: true, check: () => (dbOk ? true : 'database unavailable') },
checks.upstream('satellite_dead', `http://127.0.0.1:${await freePort()}/api/ready`),
checks.upstream('satellite_dead', `http://127.0.0.1:${await freePort()}/api/ready`),
checks.binary('nope', 'definitely-not-a-binary-ov'),
checks.binary('nope', 'definitely-not-a-binary-ov'),
],
],
@@ -58,7 +58,7 @@ function assertShape(body, service) {
@@ -58,7 +58,7 @@ function assertShape(body, service) {
r = await json(`${base}/api/ready`);
r = await json(`${base}/api/ready`);
assert.strictEqual(r.status, 503);
assert.strictEqual(r.status, 503);
assert.deepStrictEqual(r.body.failed, ['db']);
assert.deepStrictEqual(r.body.failed, ['db']);
- assert.strictEqual(r.body.checks.db.error, 'SQLITE_IOERR');
+ assert.strictEqual(r.body.checks.db.error, 'database unavailable');
obs.stop();
obs.stop();
server.close();
server.close();
assert.ok(which('node') || which(process.execPath), 'which() finds executables');
assert.ok(which('node') || which(process.execPath), 'which() finds executables');
@@ -1,55 +1,32 @@
@@ -1,55 +1,32 @@
'use strict';
'use strict';
-// The operational scripts and the deploy fallback after better-sqlite3 left every app (plan T8):
-// - deploy/scripts/*.sh never ask an app to load better-sqlite3 (the legacy deploy's load checks would
-// abort every deploy once the dependency is gone);
-// - scripts/ carries better-sqlite3 itself (scripts/package.json) for the cutover importer and the
-// SQLite-era reports, and they load it through scripts/sqlite.js, not from an app;
-// - analytics-prune on the one tools database refuses --app instead of silently pruning every service.
const assert = require('assert');
const assert = require('assert');
-const fs = require('fs');
const path = require('path');
const path = require('path');
-
-const ROOT = path.resolve(__dirname, '..', '..', '..');
-const SCRIPTS = path.join(ROOT, 'scripts');
-const SQLITE = ['better', 'sqlite3'].join('-');
-
+const prune = require(path.resolve(__dirname, '..', '..', '..', 'scripts', 'analytics-prune.js'));
+const report = require(path.resolve(__dirname, '..', '..', '..', 'scripts', 'guard-abuse-report.js'));
+const { openToolsDb } = require('../db');
(async () => {
(async () => {
- // 1. The deploy scripts.
- const deployDir = path.join(ROOT, 'deploy', 'scripts');
- for (const f of fs.readdirSync(deployDir).filter((n) => n.endsWith('.sh'))) {
- const src = fs.readFileSync(path.join(deployDir, f), 'utf8');
- assert.ok(!src.includes(`require('${SQLITE}')`) && !src.includes(`require("${SQLITE}")`), `deploy/scripts/${f} still loads ${SQLITE}`);
- }
- const legacy = fs.readFileSync(path.join(deployDir, 'deploy-legacy.sh'), 'utf8');
- assert.match(legacy, /require\('openvibe-sdk\/db'\); require\('\.\.\/_shared\/jobs'\)/, 'the jobs runtime check loads openvibe-sdk/db');
- assert.match(legacy, /require\('openvibe-sdk\/db'\); require\('\.\.\/_shared\/guard'\)/, 'the guard check loads openvibe-sdk/db');
-
- // 2. scripts/ owns the dependency; the scripts load it through scripts/sqlite.js.
- const pkg = JSON.parse(fs.readFileSync(path.join(SCRIPTS, 'package.json'), 'utf8'));
- assert.ok(pkg.dependencies && pkg.dependencies[SQLITE], `scripts/package.json lists ${SQLITE}`);
- assert.ok(fs.existsSync(path.join(SCRIPTS, 'package-lock.json')), 'with a lockfile');
- for (const f of ['migrate-to-postgres.js', 'analytics-prune.js', 'guard-abuse-report.js']) {
- const src = fs.readFileSync(path.join(SCRIPTS, f), 'utf8');
- assert.match(src, /require\('\.\/sqlite'\)/, `${f} loads it through scripts/sqlite.js`);
- assert.ok(!src.includes(`'${SQLITE}'`), `${f} does not resolve it by name from an app`);
- }
- try {
- require(path.join(SCRIPTS, 'sqlite.js')).loadSqlite();
- } catch (err) {
- assert.match(err.message, /npm --prefix scripts install/, 'a missing dependency says how to install it');
- }
-
- // 3. analytics-prune: --app on the one tools database is refused, before anything is opened.
- const prune = require(path.join(SCRIPTS, 'analytics-prune.js'));
- await assert.rejects(prune.pgMain(['--app', 'yt'], () => {}), /--app does not apply to the one tools database/);
+ await assert.rejects(prune.pgMain(['--app', 'yt'], () => {}), /--app does not apply/);
const before = process.env.DATABASE_URL;
const before = process.env.DATABASE_URL;
process.env.DATABASE_URL = 'postgres://[email protected]:1/none';
process.env.DATABASE_URL = 'postgres://[email protected]:1/none';
try {
try {
- await assert.rejects(Promise.resolve().then(() => prune.main(['--app', 'yt', '--apply'], () => {})), /--app does not apply/,
- 'DATABASE_URL set, no --db: the PostgreSQL branch, and --app is refused rather than pruning every service');
+ await assert.rejects(Promise.resolve().then(() => prune.main(['--app', 'yt', '--apply'], () => {})), /unknown option|--app does not apply/);
} finally {
} finally {
if (before === undefined) delete process.env.DATABASE_URL; else process.env.DATABASE_URL = before;
if (before === undefined) delete process.env.DATABASE_URL; else process.env.DATABASE_URL = before;
}
}
-
- console.log('ops scripts: deploy checks without the SQLite dependency, scripts/ owns it, analytics-prune refuses --app on PostgreSQL: all checks passed');
+ assert.throws(() => report.parseArgs(['--db', 'old.db']), /unknown option/);
+ assert.throws(() => report.parseArgs(['--pg']), /unknown option/);
+ const priorEnv = process.env.NODE_ENV;
+ const priorUrl = process.env.DATABASE_URL;
+ process.env.NODE_ENV = 'production';
+ delete process.env.DATABASE_URL;
+ let created = false;
+ try {
+ assert.throws(() => openToolsDb({ createDb: () => { created = true; } }), /DATABASE_URL is required in production/);
+ assert.strictEqual(created, false, 'production fails before PGlite can open');
+ } finally {
+ if (priorEnv === undefined) delete process.env.NODE_ENV; else process.env.NODE_ENV = priorEnv;
+ if (priorUrl === undefined) delete process.env.DATABASE_URL; else process.env.DATABASE_URL = priorUrl;
+ }
+ assert.deepStrictEqual(report.parseArgs(['--url', 'postgres://test/db', '--json']), { url: 'postgres://test/db', json: true });
+ console.log('ops scripts: PostgreSQL commands reject obsolete SQLite options');
})().catch((err) => { console.error(err); process.exit(1); });
})().catch((err) => { console.error(err); process.exit(1); });
@@ -2,7 +2,7 @@
@@ -2,7 +2,7 @@
// The one `tools` database for a test run (plan T8): one PGlite database per test directory, migrated
// The one `tools` database for a test run (plan T8): one PGlite database per test directory, migrated
// with the repo's migrations. It is kept across a satellite restart (the data must survive it), so a
// with the repo's migrations. It is kept across a satellite restart (the data must survive it), so a
// test that restarts a satellite calls testDb() with the same directory again; closeAllTestDbs() at the
// test that restarts a satellite calls testDb() with the same directory again; closeAllTestDbs() at the
-// end of the run. Nothing here needs better-sqlite3: PGlite is real PostgreSQL in-process.
+// end of the run. PGlite runs PostgreSQL in process.
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
const { dep } = require('./deps');
const { dep } = require('./deps');
@@ -50,14 +50,7 @@ assert.throws(() => requireShared('next', { appsDir: dir, version: '1.0.0' }), /
@@ -50,14 +50,7 @@ assert.throws(() => requireShared('next', { appsDir: dir, version: '1.0.0' }), /
assert.throws(() => require('../release').toolsRelease('no-such-app', require), /declares no "openvibeToolsShared" range/, 'toolsRelease refuses an undeclared app before anything else');
assert.throws(() => require('../release').toolsRelease('no-such-app', require), /declares no "openvibeToolsShared" range/, 'toolsRelease refuses an undeclared app before anything else');
fs.rmSync(dir, { recursive: true, force: true });
fs.rmSync(dir, { recursive: true, force: true });
-// The deploy walks apps/*/ to install, resolve and preflight each app; _shared has a package.json now
-// but is not an app (2026-09-26: the guard preflight tried to load better-sqlite3 from it and aborted).
-// deploy.sh hands over to `ovhost deploy tools` (OpenVibe.Host strategy multi-app, whose default
-// skipPackages is apps/_*) and falls back to deploy-legacy.sh, whose app loops skip it themselves.
-const legacy = fs.readFileSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy-legacy.sh'), 'utf8');
-const loops = legacy.split('\n').map((l, i, all) => [l, all[i + 1] || '']).filter(([l]) => /^for app in apps\/\*\/; do/.test(l));
-assert.ok(loops.length >= 3, 'deploy-legacy.sh has its app loops');
-for (const [, next] of loops) assert.match(next, /case "\$app" in apps\/_\*\) continue ;; esac/, 'each app loop in deploy-legacy.sh skips apps/_*');
-assert.match(fs.readFileSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy.sh'), 'utf8'), /^STRATEGY=multi-app$/m, 'deploy.sh hands over only to the multi-app strategy (apps/_* skipped)');
+// Deployment uses the host's multi-app strategy, which excludes apps/_* packages.
+assert.ok(fs.existsSync(path.join(APPS, '..', 'deploy', 'scripts', 'deploy.sh')));
console.log('_shared version: all checks passed');
console.log('_shared version: all checks passed');
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
-// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would
+// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would
// refuse. Every audio tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
// refuse. Every audio tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
const SPECS = require('./descriptors').SPECS;
const SPECS = require('./descriptors').SPECS;
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
-// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would
+// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would
// refuse. Every PDF tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
// refuse. Every PDF tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
const SPECS = require('./descriptors').SPECS;
const SPECS = require('./descriptors').SPECS;
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
@@ -21,7 +21,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
@@ -21,7 +21,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota and
// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota and
-// the abuse log (data/guard.db). The data lookups are counted by maps (tools-map), which gets the
+// the abuse log (PostgreSQL guard_abuse). The data lookups are counted by maps (tools-map), which gets the
// visitor's address from here. TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
// visitor's address from here. TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
@@ -22,7 +22,7 @@ NoNewPrivileges=true
@@ -22,7 +22,7 @@ NoNewPrivileges=true
PrivateTmp=true
PrivateTmp=true
ProtectSystem=strict
ProtectSystem=strict
ProtectHome=true
ProtectHome=true
-# guard.db (apps/_shared/guard): quotas' day counters and the abuse log
+# Writable gateway data directory for local runtime files
ReadWritePaths=/opt/openvibe.tools/apps/gateway/data
ReadWritePaths=/opt/openvibe.tools/apps/gateway/data
[Install]
[Install]
@@ -56,7 +56,7 @@ let auth = null; // created below; the key check reads it at request time
@@ -56,7 +56,7 @@ let auth = null; // created below; the key check reads it at request time
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// the address), the tools-api quota, the net and dev tools' quotas by descriptor, the per-target
// the address), the tools-api quota, the net and dev tools' quotas by descriptor, the per-target
-// throttle, the port-scan cap, webhook bin caps and the abuse log (data/guard.db). The Network key is
+// throttle, the port-scan cap, webhook bin caps and the abuse log (PostgreSQL guard_abuse). The Network key is
// the OAuth client's (loaded and retried by it). TOOLS_GUARD=enforce (default) refuses; report records
// the OAuth client's (loaded and retried by it). TOOLS_GUARD=enforce (default) refuses; report records
// what it would refuse.
// what it would refuse.
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
@@ -39,7 +39,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
-// abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
+// abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
const SPECS = require('./descriptors').SPECS;
const SPECS = require('./descriptors').SPECS;
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4): guard_abuse in
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4): guard_abuse in
// PostgreSQL, guard_salt/guard_day and the quota buckets in Valkey. `ready` resolves when the schema is in.
// PostgreSQL, guard_salt/guard_day and the quota buckets in Valkey. `ready` resolves when the schema is in.
@@ -29,7 +29,7 @@ const { createPacer, Busy } = require('../../_shared/guard/semaphore');
@@ -29,7 +29,7 @@ const { createPacer, Busy } = require('../../_shared/guard/semaphore');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, else the address — the food app forwards its
// Who is asking (Network sign-in with aud openvibe.tools, else the address — the food app forwards its
// visitor's), the tools-map quota on the data routes (descriptor maps: cost 2 a call) and the abuse log
// visitor's), the tools-map quota on the data routes (descriptor maps: cost 2 a call) and the abuse log
-// (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
+// (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-maps' });
const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-maps' });
@@ -19,7 +19,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
@@ -19,7 +19,7 @@ const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota on
// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota on
-// /api/ and the abuse log (data/guard.db). TOOLS_GUARD=enforce (default) refuses; report records what
+// /api/ and the abuse log (PostgreSQL guard_abuse). TOOLS_GUARD=enforce (default) refuses; report records what
// it would refuse.
// it would refuse.
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
@@ -26,7 +26,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
@@ -26,7 +26,7 @@ const { requireInternalAccess } = require('../../_shared/internal-token');
// ── Guard (apps/_shared/guard) ───────────────────────────────
// ── Guard (apps/_shared/guard) ───────────────────────────────
// Who is asking (Network sign-in with aud openvibe.tools, the browser session, else the address),
// Who is asking (Network sign-in with aud openvibe.tools, the browser session, else the address),
-// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (data/guard.db).
+// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (PostgreSQL guard_abuse).
// TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
// TOOLS_GUARD=enforce (default) refuses; report records what it would refuse.
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
// The one `tools` database and the shared Valkey (plan T8, decisions 3 and 4).
const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-yt' });
const toolsDb = require('../../_shared/db').openToolsDb({ createDb: require('openvibe-sdk/db').createDb, service: 'tools-yt' });
deleted file mode 100755
deleted file mode 100755
@@ -1,71 +0,0 @@
@@ -1,71 +0,0 @@
-#!/usr/bin/env bash
-# Deploy OpenVibe.Tools on the host: pull, install dependencies in each app whose package.json
-# changed (that includes a new openvibe-shared release tag), restart the units, check health, then
-# announce the release to open tabs (`ovhost announce tools`; best effort, never fails the deploy).
-set -euo pipefail
-cd "$(dirname "$0")/../.."
-BEFORE=$(git rev-parse HEAD)
-# npm leaves untracked lockfiles in apps that did not track one; once the repo starts tracking it,
-# `git pull` refuses to overwrite it. Those files are generated, so remove exactly those before pulling.
-git fetch -q origin
-for f in $(git diff --name-only HEAD "origin/$(git rev-parse --abbrev-ref HEAD)" -- 'apps/*/package-lock.json'); do
- if [ -f "$f" ] && ! git ls-files --error-unmatch "$f" >/dev/null 2>&1; then echo "removing untracked $f (the release tracks it)"; rm -f "$f"; fi
-done
-git pull -q --ff-only; AFTER=$(git rev-parse HEAD)
-for app in apps/*/; do
- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app
- [ -f "$app/package.json" ] || continue
- if git diff --name-only "$BEFORE" "$AFTER" -- "$app/package.json" | grep -q . || [ ! -d "$app/node_modules" ]; then (cd "$app" && npm install --omit=dev --no-audit --no-fund --loglevel=error); fi
-done
-# Every app must resolve its dependencies before anything restarts. A dependency that used to be a
-# file: link can leave an empty directory npm treats as installed (2026-09-23: vendor/openvibe-shared
-# kept an untracked lockfile, every unit crash-looped on 'openvibe-shared/brand'); reinstall those.
-for app in apps/*/; do
- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app
- [ -f "$app/package.json" ] || continue
- for dep in $(node -e 'console.log(Object.keys(require("./"+process.argv[1]+"package.json").dependencies||{}).join(" "))' "$app"); do
- if [ ! -f "$app/node_modules/$dep/package.json" ]; then
- echo "reinstalling $dep in $app (missing package.json)"; rm -rf "$app/node_modules/$dep"
- (cd "$app" && npm install --omit=dev --no-audit --no-fund --loglevel=error)
- [ -f "$app/node_modules/$dep/package.json" ] || { echo "ABORT: $app cannot resolve $dep; nothing restarted" >&2; exit 1; }
- fi
- done
-done
-# Apps that run jobs (apps/_shared/jobs, required by relative path) hand it their own openvibe-contracts
-# and openvibe-sdk (openvibe-sdk/db for the job store in the one tools database, and the tools.job.*
-# outbox relay): check they load under this Node and that the shared runtime itself loads. Nothing here
-# needs better-sqlite3 any more (plan T8: no app depends on it).
-for app in img audio docs; do
- (cd "apps/$app" && node -e "require('openvibe-contracts'); require('openvibe-sdk'); require('openvibe-sdk/db'); require('../_shared/jobs')") \
- || { echo "ABORT: apps/$app cannot load the jobs runtime; nothing restarted" >&2; exit 1; }
-done
-# Every app's guard (apps/_shared/guard) keeps its state in PostgreSQL (guard_abuse) and Valkey (salt, day
-# counters, buckets); check it loads. The units' ReadWritePaths still name each data directory (uploads,
-# job files), which must exist before systemd starts them.
-for app in apps/*/; do
- case "$app" in apps/_*) continue ;; esac # apps/_shared is a package the apps require, not an app
- [ -f "$app/package.json" ] || continue
- mkdir -p "$app/data"
- (cd "$app" && node -e "require('openvibe-sdk/db'); require('../_shared/guard')") \
- || { echo "ABORT: $app cannot load the guard; nothing restarted" >&2; exit 1; }
-done
-UNITS=$(systemctl list-unit-files 'openvibe-tools*' --no-legend | awk '{print $1}')
-sudo systemctl restart $UNITS
-sleep 5
-for u in $UNITS; do printf '%-34s %s\n' "$u" "$(systemctl is-active "$u")"; done
-curl -fsS -o /dev/null -H 'Host: openvibe.tools' http://127.0.0.1:4001/api/health && echo "gateway healthy ($BEFORE → $AFTER)" || exit $?
-# Release notification (roadmap WS-P task 9): OpenVibe.Host publishes host.release.published for the release
-# the apps' /release.json reports, once per release, so open tabs check it now instead of at their next
-# poll. Best effort: skipped without an ovhost whose --help has `announce <service>`, 20 s at most, and it
-# never changes the exit code. ovhost reads Host's credentials as root (OpenVibe.Host
-# docs/release-notifications.md).
-OVHOST_BIN=$(command -v "${OVHOST:-ovhost}" 2>/dev/null || true)
-if [ -n "$OVHOST_BIN" ]; then
- case "$("$OVHOST_BIN" --help 2>/dev/null || true)" in
- *"announce <service>"*)
- SUDO=""; [ "$(id -u)" -eq 0 ] || SUDO="sudo -n"
- timeout 20 $SUDO "$OVHOST_BIN" announce tools 2>&1 || echo "release notification not sent (the deploy stands)" ;;
- *) echo "release notification skipped: this ovhost has no announce" ;;
- esac
-fi
-exit 0
@@ -1,71 +1,20 @@
@@ -1,71 +1,20 @@
#!/usr/bin/env bash
#!/usr/bin/env bash
-# OpenVibe.Tools — deploy: a thin wrapper around `ovhost deploy tools` (OpenVibe.Host, strategy multi-app;
-# roadmap WS-N task 11; OpenVibe.Host docs/deploy-strategies.md).
-#
-# ovhost now does what this script did, as the checkout owner: untracked apps/*/package-lock.json files the
-# release tracks are removed before the merge; npm install in each app whose lockfile or dependencies changed
-# (apps/_* are packages, not apps); every dependency resolves (reinstalled once, else ABORT with nothing
-# restarted); each app's data/ exists; the jobs runtime loads in img/audio/docs and the guard in every app;
-# every openvibe-tools* unit restarts; the gateway answers /api/ready with Host: openvibe.tools and its
-# /release.json names the new sha, every unit is active; the release is announced. New: a release that does
-# not come up is rolled back (the checkout, its dependencies, a second restart; exit 3), and every attempt
-# is in `ovhost releases tools`.
-#
-# deploy/scripts/deploy.sh ovhost deploy tools
-# deploy/scripts/deploy.sh --wait-idle ovhost deploy tools --wait-idle (hold until no tool job runs)
-# deploy/scripts/deploy.sh --restart ovhost deploy tools --restart
-# deploy/scripts/deploy.sh --rollback ovhost rollback tools
-# DRY_RUN=1 deploy/scripts/deploy.sh ovhost plan tools
-#
-# ovhost runs as root (sudo is used when this runs as another user). Fallback: deploy-legacy.sh, the
-# previous script unchanged, when ovhost is missing or too old (no `capabilities`, deploy-api < 1), or the
-# host inventory does not deploy tools with strategy multi-app; OVHOST_LEGACY=1 forces it. The legacy
-# script takes no flags: --rollback, --wait-idle and DRY_RUN=1 are refused rather than ignored there.
+# OpenVibe.Tools deploy wrapper: ovhost deploy, rollback, or plan tools.
set -euo pipefail
set -euo pipefail
-
-SERVICE=tools
-STRATEGY=multi-app
-HERE=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
-LEGACY="${DEPLOY_LEGACY:-$HERE/deploy-legacy.sh}"
OVHOST="${OVHOST:-/usr/local/bin/ovhost}"
OVHOST="${OVHOST:-/usr/local/bin/ovhost}"
if [ "${OVHOST_SUDO-auto}" = auto ]; then if [ "$(id -u)" -eq 0 ]; then SUDO=(); else SUDO=(sudo); fi; elif [ -n "${OVHOST_SUDO}" ]; then SUDO=("$OVHOST_SUDO"); else SUDO=(); fi
if [ "${OVHOST_SUDO-auto}" = auto ]; then if [ "$(id -u)" -eq 0 ]; then SUDO=(); else SUDO=(sudo); fi; elif [ -n "${OVHOST_SUDO}" ]; then SUDO=("$OVHOST_SUDO"); else SUDO=(); fi
-
CMD=deploy
CMD=deploy
FLAGS=()
FLAGS=()
while [ "$#" -gt 0 ]; do
while [ "$#" -gt 0 ]; do
case "$1" in
case "$1" in
--wait-idle|--restart|--force) FLAGS+=("$1"); shift ;;
--wait-idle|--restart|--force) FLAGS+=("$1"); shift ;;
--rollback) CMD=rollback; shift ;;
--rollback) CMD=rollback; shift ;;
- --) shift; break ;;
- *) echo "Usage: $0 [--wait-idle] [--restart] [--force] [--rollback] (DRY_RUN=1 for the plan)"; exit 1 ;;
+ *) echo "Usage: $0 [--wait-idle] [--restart] [--force] [--rollback] (DRY_RUN=1 for the plan)" >&2; exit 1 ;;
esac
esac
done
done
-
-legacy() {
- echo "[tools-deploy] $1 — running deploy-legacy.sh (the previous deploy script) instead"
- if [ "$CMD" = rollback ] || [ "${DRY_RUN:-0}" = 1 ] || [[ " ${FLAGS[*]:-} " == *" --wait-idle "* ]]; then
- echo "[tools-deploy] ✗ deploy-legacy.sh has no --rollback, --wait-idle or DRY_RUN; nothing was done" >&2
- exit 1
- fi
- exec bash "$LEGACY"
-}
-
-REASON=""
-probe() {
- if [ "${OVHOST_LEGACY:-0}" = 1 ]; then REASON="OVHOST_LEGACY=1"; return 1; fi
- if ! command -v "$OVHOST" >/dev/null 2>&1; then REASON="ovhost not found ($OVHOST)"; return 1; fi
- local caps api
- if ! caps=$("${SUDO[@]}" "$OVHOST" capabilities "$SERVICE" 2>/dev/null); then REASON="this ovhost has no 'capabilities' (too old) or no inventory entry for $SERVICE"; return 1; fi
- api=$(printf '%s\n' "$caps" | sed -n 's/^deploy-api=//p')
- case "$api" in ''|*[!0-9]*) REASON="this ovhost reports no deploy-api (too old)"; return 1 ;; esac
- if [ "$api" -lt 1 ]; then REASON="this ovhost's deploy-api is $api, 1 is needed"; return 1; fi
- if ! printf '%s\n' "$caps" | grep -qx "strategy=$STRATEGY"; then REASON="the host inventory does not deploy $SERVICE with strategy $STRATEGY ($(printf '%s\n' "$caps" | sed -n 's/^strategy=//p'))"; return 1; fi
- if ! printf '%s\n' "$caps" | grep -qx "managed=yes"; then REASON="ovhost does not manage $SERVICE"; return 1; fi
- return 0
-}
-
-probe || legacy "$REASON"
-
-if [ "${DRY_RUN:-0}" = 1 ]; then exec "${SUDO[@]}" "$OVHOST" plan "$SERVICE"; fi
-echo "[tools-deploy] ovhost $CMD $SERVICE ${FLAGS[*]:-}"
-exec "${SUDO[@]}" "$OVHOST" "$CMD" "$SERVICE" "${FLAGS[@]}"
+if ! command -v "$OVHOST" >/dev/null 2>&1; then
+ echo "[tools-deploy] ovhost not found ($OVHOST)" >&2
+ exit 1
+fi
+if [ "${DRY_RUN:-0}" = 1 ]; then exec "${SUDO[@]}" "$OVHOST" plan tools; fi
+exec "${SUDO[@]}" "$OVHOST" "$CMD" tools "${FLAGS[@]}"
deleted file mode 100644
deleted file mode 100644
@@ -1,67 +0,0 @@
@@ -1,67 +0,0 @@
-# T8 cutover: closeout evidence — OpenVibe.Tools
-
-What the read-only interfaces can actually prove about the Tools PostgreSQL + Valkey cutover (plan T8). The
-cutover landed **2026-10-02**; production serves the PostgreSQL-only release. Checked **2026-10-04 (UTC)** through
-`ov access run openvibe-ovh <action>` — `health`, `service-status`, `plan`, `releases`, `validate`, `env-set`,
-`show`, `journal`. Every result below is a literal output of one of those actions.
-
-Nothing here was deployed, restarted, rolled back or imported; no credential, URL or env **value** was read or
-printed. Checks that need an interface the broker does not expose are marked **pending** in their own section and
-were not guessed.
-
-SHAs are printed as `ovhost` reports them (12 hex characters).
-
-## Release and readiness — verified
-
-- `health tools` (exit 0): `tools a331ea36963a ready openvibe-tools=active openvibe-tools-maps=active
- openvibe-tools-food=active openvibe-tools-img=active openvibe-tools-yt=active openvibe-tools-audio=active
- openvibe-tools-text=active openvibe-tools-docs=active running tool jobs=0`. `ovhost` needs each service's
- `/api/ready` to answer before it calls the service `ready`.
-- `service-status tools` (exit 0): the same line.
-- `plan tools` (exit 0): `tools: a331ea36963a → a331ea36963a (up to date) on main`; strategy `multi-app`, `0`
- files changed, `restart no`.
-- `validate tools` (exit 0): `ok checkout /opt/openvibe.tools owned by ubuntu`; `ok checkout clean at
- a331ea36963a`; all eight `openvibe-tools*` units `active/running`; `ok port 4001 held by node`; `ok deps every
- dependency resolves`; `tools: valid (0 error(s), 0 warning(s))`.
-- `releases tools` (exit 0): `834bed39e5b8 → 98f8bebf73ee → a331ea36963a deployed` (2026-10-03T11:35Z,
- 2026-10-04T00:24Z, 2026-10-04T03:00Z). `a331ea36963a` is origin/main HEAD and the release running in
- production; `834bed39e5b8` was the PostgreSQL-only release deployed during the cutover.
-
-**Why this is a PostgreSQL-only release.** Every app boots through `apps/_shared/db.js`, which in production
-throws `DATABASE_URL is not set: Tools serves from PostgreSQL (plan T8)` and only falls back to embedded PGlite
-(PGlite outside production). A release that is `active` and `ready` therefore booted with `DATABASE_URL` set and
-serves from PostgreSQL. No app depends on `better-sqlite3` any more, guarded by
-`apps/_shared/test/no-sqlite.test.js`; the migrations (`migrations/0001_tools.sql` … `0004_event_outbox.sql`) run
-only as the owner via `DATABASE_DIRECT_URL`, never at runtime.
-
-## Env presence and mode — verified
-
-- `validate tools`: `ok env /etc/openvibe/tools.env mode 600`; `ok env 4 required name(s) present and non-empty`.
-- `env-set tools` (names only, `empty: []`): **`DATABASE_URL`**, **`DATABASE_DIRECT_URL`**, **`VALKEY_URL`**,
- `VALKEY_PREFIX`. `fromUnits`: `NODE_ENV`, `PATH`, `PORT`.
-
-## Valkey ACL scope — pending
-
-Presence of `VALKEY_URL` and `VALKEY_PREFIX` is provable (above). The scope of the Valkey user — which commands
-it holds and that it is confined to `~ov:tools:*` (`/etc/valkey/services.acl`) — needs a connection to Valkey
-that no read-only action opens. **Not checked.**
-
-## Row parity / rollback copy — pending
-
-The importer has already run and production writes PostgreSQL, but the per-table parity report and the rollback
-copy (`$COPY` = `/var/backups/openvibe/tools-pre-t8-<T0>.db`, `chmod 0400`) are not exposed by any read-only
-action, which runs no query and lists no file. **Not checked.**
-
-## Reproduce
-
-```bash
-ov access run openvibe-ovh health tools
-ov access run openvibe-ovh service-status tools
-ov access run openvibe-ovh plan tools
-ov access run openvibe-ovh releases tools
-ov access run openvibe-ovh validate tools
-ov access run openvibe-ovh env-set tools
-ov access run openvibe-ovh journal openvibe-tools.service 200
-```
-
-Every one of these is a low-risk read. `deploy`, `rollback`, `restart` and `db-backup` were not used.
@@ -27,5 +27,5 @@
@@ -27,5 +27,5 @@
"namespacesOwned": [
"namespacesOwned": [
"tools.*"
"tools.*"
],
],
- "notes": "Wave 11: asynchronous jobs on the img, audio and docs satellites (POST/GET/DELETE /api/v1/jobs, SSE events). Job lifecycle events tools.job.created|started|succeeded|failed (openvibe-contracts v0.30.1 payloads) go to OpenVibe.Events through an outbox in each satellite's jobs.db when EVENTS_URL is set. Tools consumes media.object.upload and media.object.read (namespace tools) when TOOLS_JOB_RESULTS=media."
+ "notes": "Wave 11: asynchronous jobs on the img, audio and docs satellites (POST/GET/DELETE /api/v1/jobs, SSE events). Job lifecycle events tools.job.created|started|succeeded|failed (openvibe-contracts v0.30.1 payloads) go to OpenVibe.Events through the shared PostgreSQL event outbox when EVENTS_URL is set. Tools consumes media.object.upload and media.object.read (namespace tools) when TOOLS_JOB_RESULTS=media."
}
}
@@ -1,66 +1,26 @@
@@ -1,66 +1,26 @@
#!/usr/bin/env node
#!/usr/bin/env node
-/**
- * Raw analytics retention and one-time scrub for the satellites' analytics.db files (ADR-021). The
- * command itself is openvibe-shared/analytics/prune-cli; this wrapper passes a better-sqlite3 from an
- * installed app, the --app flag and the discovery of apps/<app>/data/analytics.db.
- *
- * node scripts/analytics-prune.js # dry run over every apps/<app>/data/analytics.db
- * node scripts/analytics-prune.js --app yt --scrub # dry run for one app, including the scrub
- * node scripts/analytics-prune.js --apply --scrub --backup <dir> # back up each db into <dir>, then prune + scrub
- * node scripts/analytics-prune.js --apply --no-backup # prune without a backup (explicit)
- * node scripts/analytics-prune.js --help # every option
- *
- * --app <name> only this app (repeatable); default: every app that has data/analytics.db
- * (an app's DATA_DIR other than data/ needs --db)
- * --db <file> an analytics database by path (repeatable; replaces the app discovery)
+/** Raw analytics retention in PostgreSQL. Dry run by default.
+ * Usage: node scripts/analytics-prune.js [--days N] [--apply]
*/
*/
'use strict';
'use strict';
-const fs = require('fs');
const path = require('path');
const path = require('path');
-
-const ROOT = path.join(__dirname, '..');
-const APPS = path.join(ROOT, 'apps');
-
-/** A package from the first app that has it installed (the repo root has no node_modules). */
+const APPS = path.join(__dirname, '..', 'apps');
function fromApps(name) {
function fromApps(name) {
for (const a of ['yt', 'maps', 'food', 'img', 'docs', 'audio', 'text', 'gateway']) {
for (const a of ['yt', 'maps', 'food', 'img', 'docs', 'audio', 'text', 'gateway']) {
try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }
try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }
}
}
throw new Error(`${name} is not installed in any app (npm run install:all)`);
throw new Error(`${name} is not installed in any app (npm run install:all)`);
}
}
-
-/** better-sqlite3: scripts/' own dependency (npm --prefix scripts install), else an app that still has it. */
-const { loadSqlite } = require('./sqlite');
-
-/** [{ name, file }] when no --db is given: the --app list, else every app with data/analytics.db. */
-function targets(args) {
- const names = args.apps.length ? args.apps : fs.readdirSync(APPS).filter((a) => !a.startsWith('_')).sort();
- return names.map((name) => ({ name, file: path.join(APPS, name, 'data', 'analytics.db') }))
- .filter((t) => args.apps.length || fs.existsSync(t.file));
-}
-
-const USAGE_APPS = ` --app <name> only this app (repeatable); default: every app that has data/analytics.db
- (an app's DATA_DIR other than data/ needs --db)
- --pg prune the one tools database (DATABASE_URL) with openvibe-shared/analytics/pg instead
- of the apps' analytics.db files (the default when DATABASE_URL is set and no --db is
- given); --days <n> and --apply apply here too, --app is refused (every service at once)`;
-
-/** The tracker's service names in the one tools database (plan T8). */
-const SERVICES = ['openvibe-gateway', 'openvibe-maps', 'openvibe-food', 'openvibe-img', 'openvibe-yt', 'openvibe-audio', 'openvibe-text', 'openvibe-docs'];
-
/**
/**
- * The PostgreSQL branch (plan T8, decision 5): after the cutover the raw analytics events are rows in the
- * one tools database, not SQLite files. Prunes them with openvibe-shared/analytics/pg (the rollups stay);
- * without --apply it counts only, as the SQLite dry run does. The backup/scrub/VACUUM options are for the
- * pre-cutover files and do not apply here, and neither does --app: the retention is one rule for the whole
- * database (pruneRawEventsPg has no per-service filter), so an --app here is refused rather than silently
- * pruning every service.
+ * Prune raw analytics rows in the one tools database. Rollups stay.
+ * Without --apply the command only counts. Retention covers every service.
*/
*/
async function pgMain(argv, log) {
async function pgMain(argv, log) {
if (argv.includes('--app')) {
if (argv.includes('--app')) {
throw new Error('--app does not apply to the one tools database: the PostgreSQL prune covers every service at once. '
throw new Error('--app does not apply to the one tools database: the PostgreSQL prune covers every service at once. '
- + 'Drop --app to prune them all, or pass --db <file> to prune one app\'s pre-cutover analytics.db');
+ + 'Drop --app to prune them all.');
}
}
+ if (!String(process.env.DATABASE_URL || '').trim()) throw new Error('DATABASE_URL is required to prune analytics');
const i = argv.indexOf('--days');
const i = argv.indexOf('--days');
const raw = i >= 0 ? parseInt(argv[i + 1], 10) : 30;
const raw = i >= 0 ? parseInt(argv[i + 1], 10) : 30;
const days = Number.isFinite(raw) ? Math.min(Math.max(raw, 1), 3650) : 30;
const days = Number.isFinite(raw) ? Math.min(Math.max(raw, 1), 3650) : 30;
@@ -84,18 +44,12 @@ async function pgMain(argv, log) {
@@ -84,18 +44,12 @@ async function pgMain(argv, log) {
}
}
}
}
-function main(argv, log) {
- if (argv.includes('--pg') || (process.env.DATABASE_URL && !argv.includes('--db'))) return pgMain(argv, log);
- const cli = fromApps('openvibe-shared/analytics/prune-cli');
- return cli.main(argv, {
- Database: loadSqlite(),
- log,
- options: { '--app': 'apps' },
- targets,
- usage: cli.USAGE.replace('\nOptions:\n', `\nOptions:\n${USAGE_APPS}\n`),
- });
+function main(argv, log = console.log) {
+ if (argv.includes('--help') || argv.includes('-h')) { log('Usage: node scripts/analytics-prune.js [--days N] [--apply]'); return; }
+ const allowed = new Set(['--days', '--apply']);
+ for (const arg of argv) if (arg.startsWith('--') && !allowed.has(arg)) throw new Error(`unknown option ${arg}`);
+ if (!String(process.env.DATABASE_URL || '').trim()) throw new Error('DATABASE_URL is required to prune analytics');
+ return pgMain(argv, log);
}
}
-
-if (require.main === module) fromApps('openvibe-shared/analytics/prune-cli').run(main);
-
-module.exports = { main, pgMain, targets, loadSqlite, SERVICES };
+if (require.main === module) Promise.resolve().then(() => main(process.argv.slice(2))).catch((err) => { console.error(err.message); process.exit(1); });
+module.exports = { main, pgMain };
@@ -1,71 +1,25 @@
@@ -1,71 +1,25 @@
#!/usr/bin/env node
#!/usr/bin/env node
-/**
- * Phase 0 of plan T8 — enforce-mode readiness for the tools guard.
- *
- * In report mode (`TOOLS_GUARD` unset) the guard records every refusal it *would* make into
- * `guard_abuse` with `enforced = 0`. Before flipping `TOOLS_GUARD=enforce`, read those rows and
- * confirm no reason fires on ordinary traffic: a false positive that report mode merely logged
- * becomes a 429 for a real visitor in enforce mode.
- *
- * This groups `guard_abuse WHERE enforced = 0` by (reason, tool) and prints, per reason/tool, how
- * many rows (and how many repeats the `count` column folds in), across how many hashed addresses,
- * and the window it happened in. The PowerShell with `enforced = 1` rows is printed too, for
- * context. It reads copies — never a live file it also writes.
- *
- * node scripts/guard-abuse-report.js # every apps/<app>/data/guard.db
- * node scripts/guard-abuse-report.js --app maps --app gateway
- * node scripts/guard-abuse-report.js --db /backups/maps/guard.db
- * node scripts/guard-abuse-report.js --json # machine-readable, one JSON document
- * node scripts/guard-abuse-report.js --pg # the migrated database (DATABASE_URL)
- * node scripts/guard-abuse-report.js --pg --url postgres://… # a specific PostgreSQL database
- *
- * --app <name> only this app (repeatable); default: every app that has data/guard.db
- * --db <file> a guard database by path (repeatable; replaces the app discovery)
- * --pg read guard_abuse from PostgreSQL (DATABASE_URL, or --url) instead of SQLite files
- * --url <url> the PostgreSQL URL (implies --pg)
- * --json emit JSON instead of text
- *
- * After the T8 migration the guard's abuse log lives in the one `tools` database, so --pg is the
- * form to use on production once the rows have moved. Before that, the SQLite files are the source.
- */
+/** Report guard abuse rows from PostgreSQL. Usage: node scripts/guard-abuse-report.js [--url URL] [--json] */
'use strict';
'use strict';
-const fs = require('fs');
const path = require('path');
const path = require('path');
-const { loadSqlite } = require('./sqlite');
-
-const ROOT = path.join(__dirname, '..');
-const APPS = path.join(ROOT, 'apps');
-
-/** A package from the first app that has it installed (the repo root has no node_modules). */
+const APPS = path.join(__dirname, '..', 'apps');
function fromApps(name) {
function fromApps(name) {
for (const a of ['gateway', 'maps', 'img', 'docs', 'audio', 'text', 'yt', 'food']) {
for (const a of ['gateway', 'maps', 'img', 'docs', 'audio', 'text', 'yt', 'food']) {
try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }
try { return require(require.resolve(name, { paths: [path.join(APPS, a)] })); } catch { /* next */ }
}
}
throw new Error(`${name} is not installed in any app (npm run install:all)`);
throw new Error(`${name} is not installed in any app (npm run install:all)`);
}
}
-
function parseArgs(argv) {
function parseArgs(argv) {
- const out = { apps: [], dbs: [], pg: false, url: null, json: false };
+ const out = { url: null, json: false };
for (let i = 0; i < argv.length; i++) {
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
const a = argv[i];
- if (a === '--app') out.apps.push(argv[++i]);
- else if (a === '--db') out.dbs.push(argv[++i]);
- else if (a === '--pg') out.pg = true;
- else if (a === '--url') { out.url = argv[++i]; out.pg = true; }
+ if (a === '--url') out.url = argv[++i];
else if (a === '--json') out.json = true;
else if (a === '--json') out.json = true;
else if (a === '--help' || a === '-h') out.help = true;
else if (a === '--help' || a === '-h') out.help = true;
else throw new Error(`unknown option "${a}" (--help for usage)`);
else throw new Error(`unknown option "${a}" (--help for usage)`);
}
}
return out;
return out;
}
}
-
-/** [{ name, file }] when no --db is given: the --app list, else every app with data/guard.db. */
-function targets(args) {
- const names = args.apps.length ? args.apps : fs.readdirSync(APPS).filter((a) => !a.startsWith('_')).sort();
- return names.map((name) => ({ name, file: path.join(APPS, name, 'data', 'guard.db') }))
- .filter((t) => args.apps.length || fs.existsSync(t.file));
-}
-
// reason, tool, and how many rows/repeats/minutes/addresses a group covers, plus its window.
// reason, tool, and how many rows/repeats/minutes/addresses a group covers, plus its window.
const SELECT = `
const SELECT = `
SELECT reason,
SELECT reason,
@@ -88,28 +42,6 @@ SELECT reason, tool, COUNT(*) AS rows, COALESCE(SUM(count), 0) AS events
@@ -88,28 +42,6 @@ SELECT reason, tool, COUNT(*) AS rows, COALESCE(SUM(count), 0) AS events
GROUP BY reason, tool
GROUP BY reason, tool
ORDER BY events DESC, reason, tool`;
ORDER BY events DESC, reason, tool`;
-function tableExists(db, name) {
- return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name));
-}
-
-function reportSqlite(results, target, Database) {
- if (!fs.existsSync(target.file)) { results.push({ app: target.name, file: target.file, present: false }); return; }
- // Read-only: a report never writes to the file it reads (a live file may be in WAL).
- const db = new Database(target.file, { readonly: true, fileMustExist: true });
- try {
- if (!tableExists(db, 'guard_abuse')) { results.push({ app: target.name, file: target.file, present: true, hasTable: false }); return; }
- results.push({
- app: target.name,
- file: target.file,
- present: true,
- hasTable: true,
- mode: 'sqlite',
- reported: db.prepare(SELECT).all(),
- enforced: db.prepare(SELECT_ENFORCED).all(),
- });
- } finally { db.close(); }
-}
-
async function reportPg(results, url) {
async function reportPg(results, url) {
const { Client } = fromApps('pg');
const { Client } = fromApps('pg');
const client = new Client({ connectionString: url });
const client = new Client({ connectionString: url });
@@ -131,8 +63,8 @@ function printReport(results, log) {
@@ -131,8 +63,8 @@ function printReport(results, log) {
if (!r.present) { log(`\n${r.app}: no guard database (${r.file}) — skipped`); continue; }
if (!r.present) { log(`\n${r.app}: no guard database (${r.file}) — skipped`); continue; }
if (r.hasTable === false) { log(`\n${r.app}: ${r.file} has no guard_abuse table — skipped`); continue; }
if (r.hasTable === false) { log(`\n${r.app}: ${r.file} has no guard_abuse table — skipped`); continue; }
any = true;
any = true;
- log(`\n${r.app} (${r.file})$ ${r.reported.length} reason/tool group${r.reported.length === 1 ? '' : 's'} recorded but not enforced`);
- if (!r.reported.length) { log(' (nothing was reported: no would-be refusal on this copy)'); }
+ log(`\n${r.app} (${r.file}) ${r.reported.length} reason/tool group${r.reported.length === 1 ? '' : 's'} recorded but not enforced`);
+ if (!r.reported.length) { log(' (nothing was reported: no would-be refusal in the database)'); }
for (const row of r.reported) {
for (const row of r.reported) {
log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`
log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`
+ ` rows=${row.rows} addresses=${row.addresses} ${fmtTime(row.first_at)} … ${fmtTime(row.last_at)}`);
+ ` rows=${row.rows} addresses=${row.addresses} ${fmtTime(row.first_at)} … ${fmtTime(row.last_at)}`);
@@ -142,7 +74,7 @@ function printReport(results, log) {
@@ -142,7 +74,7 @@ function printReport(results, log) {
for (const row of r.enforced) log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`);
for (const row of r.enforced) log(` ${String(row.events).padStart(6)} event${Number(row.events) === 1 ? ' ' : 's'} reason=${row.reason} tool=${row.tool || '(none)'}`);
}
}
}
}
- if (!any) log('\nNo guard database with a guard_abuse table was found.');
+ if (!any) log('\nNo guard abuse rows were found.');
log('\nReview each reason/tool pair above: any one that fires on ordinary traffic is a false');
log('\nReview each reason/tool pair above: any one that fires on ordinary traffic is a false');
log('positive that TOOLS_GUARD=enforce would turn into a 429 for a real visitor.');
log('positive that TOOLS_GUARD=enforce would turn into a 429 for a real visitor.');
}
}
@@ -150,20 +82,13 @@ function printReport(results, log) {
@@ -150,20 +82,13 @@ function printReport(results, log) {
async function main(argv, log = console.log) {
async function main(argv, log = console.log) {
const args = parseArgs(argv);
const args = parseArgs(argv);
if (args.help) {
if (args.help) {
- const usage = fs.readFileSync(__filename, 'utf8').split('\n').filter((l) => /^ \*/.test(l)).slice(0, 22).join('\n');
- log(usage.replace(/^ \* ?/gm, ''));
+ log('Usage: node scripts/guard-abuse-report.js [--url URL] [--json]');
return;
return;
}
}
const results = [];
const results = [];
- if (args.pg) {
- const url = args.url || process.env.DATABASE_URL;
- if (!url) throw new Error('--pg needs DATABASE_URL or --url');
- await reportPg(results, url);
- } else {
- const Database = loadSqlite();
- const list = args.dbs.length ? args.dbs.map((f) => ({ name: path.basename(path.dirname(path.dirname(f))) || 'db', file: path.resolve(f) })) : targets(args);
- for (const t of list) reportSqlite(results, t, Database);
- }
+ const url = args.url || process.env.DATABASE_URL;
+ if (!url) throw new Error('DATABASE_URL or --url is required');
+ await reportPg(results, url);
if (args.json) log(JSON.stringify({ generated_at: new Date().toISOString(), results }, null, 2));
if (args.json) log(JSON.stringify({ generated_at: new Date().toISOString(), results }, null, 2));
else printReport(results, log);
else printReport(results, log);
}
}
@@ -172,4 +97,4 @@ if (require.main === module) {
@@ -172,4 +97,4 @@ if (require.main === module) {
main(process.argv.slice(2)).catch((err) => { console.error(err.message); process.exit(1); });
main(process.argv.slice(2)).catch((err) => { console.error(err.message); process.exit(1); });
}
}
-module.exports = { main, targets, parseArgs, SELECT, SELECT_ENFORCED, fromApps, printReport };
+module.exports = { main, parseArgs, SELECT, SELECT_ENFORCED, fromApps, printReport };
deleted file mode 100644
deleted file mode 100644
@@ -1,159 +0,0 @@
@@ -1,159 +0,0 @@
-'use strict';
-// ═══════════════════════════════════════════════════════════════
-// scripts/migrate-to-postgres.js — the one-time move of every Tools SQLite file into the ONE `tools`
-// PostgreSQL database (plan T8, decision 3/6; ADR-035). The operator (Opus) runs it at cutover.
-//
-// node scripts/migrate-to-postgres.js # production: DATABASE_DIRECT_URL (owner role)
-// node scripts/migrate-to-postgres.js --pglite # an in-memory rehearsal (report only)
-// node scripts/migrate-to-postgres.js --dry-run --url postgres://…/tools_scratch # a scratch database
-// node scripts/migrate-to-postgres.js --app maps --app gateway [--json]
-//
-// Every app's data/<guard|jobs|analytics>.db is imported into the shared tables, tagged with its app id
-// (`app` = the app name; the analytics tables already carry `service`). guard_salt and guard_day are not
-// copied (Valkey, decision 4); the event outbox's rows and token_revocations are left behind (the tables are
-// migrations 0004 and 0003; an undelivered SQLite outbox row is not replayed, and a revocation cutoff is re-derivable). The SQLite files are opened read-only.
-//
-// It relies on the newer openvibe-sdk/db (0.25+) and better-sqlite3; OV_SDK_DIR points at an openvibe-sdk
-// checkout, else the gateway's node_modules is used. better-sqlite3 is scripts/' own dependency (no app
-// has it after plan T8): run `npm --prefix scripts install` once before the cutover. Not part of `npm test`.
-// ═══════════════════════════════════════════════════════════════
-const fs = require('fs');
-const path = require('path');
-const { createRequire } = require('module');
-const { loadSqlite } = require('./sqlite');
-
-const ROOT = path.join(__dirname, '..');
-const APPS = path.join(ROOT, 'apps');
-const MIGRATIONS = path.join(ROOT, 'migrations');
-
-// One row per kind of file, in every app. `tables` = the target tables that file fills; `app` = the ones
-// that carry the app id; `skip` = source tables deliberately not imported (importSqlite would otherwise
-// report them as having no target); `omit` = identity columns the target must assign itself (each app's
-// ids are per-app and would collide in the one shared table: the import reads a temp table of the same
-// name without them, so PostgreSQL issues fresh, globally unique ids — nothing references these ids).
-const KINDS = {
- 'guard.db': { tables: ['guard_abuse'], app: ['guard_abuse'], skip: ['guard_salt', 'guard_day'], omit: { guard_abuse: ['id'] } },
- 'jobs.db': { tables: ['tool_jobs', 'tool_job_events', 'tool_job_references', 'tool_job_usage'], app: ['tool_jobs', 'tool_job_usage'], skip: ['event_outbox'], omit: { tool_job_events: ['seq'] } },
- 'analytics.db': { tables: ['analytics_events', 'analytics_hourly', 'analytics_daily', 'analytics_visitor_days', 'analytics_day_salts'], app: [], skip: ['analytics_rate_tracking'], omit: { analytics_events: ['id'], analytics_hourly: ['id'], analytics_daily: ['id'] } },
-};
-const TARGETS = [...new Set(Object.values(KINDS).flatMap((k) => k.tables))];
-
-const argv = process.argv.slice(2);
-const has = (f) => argv.includes(`--${f}`);
-const opt = (n) => { const i = argv.indexOf(`--${n}`); return i >= 0 ? argv[i + 1] || null : null; };
-const repeat = (n) => argv.flatMap((a, i) => (a === `--${n}` && argv[i + 1] ? [argv[i + 1]] : []));
-const quiet = { log() {}, warn: console.warn, error: console.error };
-
-if (has('help')) {
- console.log('usage: node scripts/migrate-to-postgres.js [--pglite | --url <DATABASE_DIRECT_URL>] [--dry-run] [--app <name>]… [--json]');
- process.exit(0);
-}
-
-/** Resolve openvibe-sdk/db from OV_SDK_DIR or the gateway's node_modules. */
-function sdkRequire() {
- const dir = process.env.OV_SDK_DIR || path.join(APPS, 'gateway', 'node_modules', 'openvibe-sdk');
- try {
- const req = createRequire(path.join(dir, 'package.json'));
- return req('openvibe-sdk/db');
- } catch (err) {
- throw new Error(`openvibe-sdk/db could not be loaded from ${dir} (${err.message}); it needs openvibe-sdk 0.25+ (OV_SDK_DIR overrides)`);
- }
-}
-
-/** Every app's data/<name>.db, in a deterministic order, optionally narrowed with --app. */
-function discover(apps) {
- const out = [];
- for (const app of fs.readdirSync(APPS).sort()) {
- if (app.startsWith('_') || (apps.length && !apps.includes(app))) continue;
- const dir = path.join(APPS, app, 'data');
- if (!fs.existsSync(dir)) continue;
- for (const name of Object.keys(KINDS).sort()) {
- const file = path.join(dir, name);
- if (fs.existsSync(file) && fs.statSync(file).isFile() && fs.statSync(file).size > 0) out.push({ app, kind: name, file });
- }
- }
- return out;
-}
-
-const q = (x) => `"${String(x).replace(/"/g, '""')}"`;
-
-/**
- * Import one SQLite file. Identity-keyed tables (guard_abuse.id, tool_job_events.seq, the analytics ids)
- * are read through a temp table of the same name without the identity column, so PostgreSQL assigns a
- * fresh, globally unique id (per-app ids would collide in the one shared database). Nothing references
- * those ids. The file is opened read-only.
- */
-async function importFile({ file, spec, db, importSqlite, first, Database }) {
- const src = new Database(file, { readonly: true, fileMustExist: true });
- try {
- for (const [table, drop] of Object.entries(spec.omit || {})) {
- const have = src.prepare(`PRAGMA table_info(${q(table)})`).all().map((c) => c.name);
- if (!have.length) continue; // the source has no such table (a lazily created one): nothing to import
- const keep = have.filter((c) => !drop.includes(c));
- src.exec(`CREATE TEMP TABLE ${q(table)} AS SELECT ${keep.map(q).join(', ')} FROM main.${q(table)}`);
- }
- return await importSqlite({ sqlite: src, db, only: spec.tables, tables: {}, skipSource: spec.skip, truncate: false, verify: first, log: quiet });
- } finally { src.close(); }
-}
-
-async function main() {
- const apps = repeat('app');
- const sdk = sdkRequire();
- const { createDb, importSqlite } = sdk;
- const Database = loadSqlite(); // before any database is opened: a missing dependency fails fast
-
- let db;
- if (has('pglite') || (has('dry-run') && !opt('url'))) {
- db = createDb({ pglite: true, service: 'tools-import', log: quiet });
- } else {
- const url = opt('url') || process.env.DATABASE_DIRECT_URL;
- if (!url) throw new Error('set DATABASE_DIRECT_URL (the owner, direct connection), or pass --pglite / --dry-run --url <scratch>');
- if (has('dry-run') && !opt('url')) throw new Error('--dry-run needs an explicit scratch --url (or --pglite) so production is never touched');
- db = createDb({ url, service: 'tools-import', max: 2, log: quiet });
- }
-
- const files = discover(apps);
- const report = { store: db.store, files: [], tables: {}, problems: [] };
- const sourceCount = {}; // target table → rows seen in the SQLite files
- const doneKind = new Set();
-
- try {
- await db.migrate({ dir: MIGRATIONS, log: quiet });
- await db.query(`TRUNCATE ${TARGETS.map((t) => `"${t}"`).join(', ')} RESTART IDENTITY CASCADE`);
-
- for (const { app, kind, file } of files) {
- const spec = KINDS[kind];
- const first = !doneKind.has(kind);
- doneKind.add(kind);
- const r = await importFile({ file, spec, db, importSqlite, first, Database });
- const rows = {};
- for (const t of r.tables) { rows[t.table] = t.rows; sourceCount[t.table] = (sourceCount[t.table] || 0) + t.rows; }
- // Tag the app-scoped rows this file just inserted (they came in with the '' default).
- for (const t of spec.app) await db.query(`UPDATE "${t}" SET app = $1 WHERE app = ''`, [app]);
- if (!r.ok) for (const p of r.problems) report.problems.push({ app, kind, ...p });
- report.files.push({ app, kind, file: path.relative(ROOT, file), rows, verified: first, checksums: Object.fromEntries(r.tables.map((t) => [t.table, t.checksum]).filter(([, c]) => c)) });
- if (process.stdout.isTTY) process.stderr.write(`[import] ${app}/${kind}: ${JSON.stringify(rows)}\n`);
- }
-
- // Reconcile counts across apps (per-file verification only covers the first file of a kind).
- for (const t of TARGETS) {
- const pg = Number((await db.maybe(`SELECT count(*) AS n FROM "${t}"`)).n);
- const src = sourceCount[t] || 0;
- report.tables[t] = { source: src, postgres: pg, ok: src === pg };
- if (src !== pg) report.problems.push({ table: t, problem: `count mismatch: ${src} SQLite rows, ${pg} PostgreSQL rows` });
- }
- report.ok = report.problems.length === 0;
- } finally {
- await db.close();
- }
-
- if (has('json')) { console.log(JSON.stringify(report, null, 2)); }
- else {
- console.log(`import → ${report.store}: ${report.ok ? 'OK' : 'PROBLEMS'}`);
- for (const [t, c] of Object.entries(report.tables)) console.log(` ${t.padEnd(26)} ${String(c.source).padStart(7)} SQLite rows → ${String(c.postgres).padStart(7)} PostgreSQL rows${c.ok ? '' : ' MISMATCH'}`);
- for (const p of report.problems) console.log(` problem: ${p.app ? `${p.app}/${p.kind} ` : ''}${p.table}: ${p.problem}`);
- }
- process.exit(report.ok ? 0 : 1);
-}
-
-main().catch((err) => { console.error(`migrate-to-postgres: ${err.message}`); process.exit(1); });
deleted file mode 100644
deleted file mode 100644
@@ -1,468 +0,0 @@
@@ -1,468 +0,0 @@
-{
- "name": "openvibe-tools-scripts",
- "version": "1.0.0",
- "lockfileVersion": 3,
- "requires": true,
- "packages": {
- "": {
- "name": "openvibe-tools-scripts",
- "version": "1.0.0",
- "license": "MIT",
- "dependencies": {
- "better-sqlite3": "^11.10.0"
- },
- "engines": {
- "node": ">=22"
- }
- },
- "node_modules/base64-js": {
- "version": "1.5.1",
- "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
- "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/better-sqlite3": {
- "version": "11.10.0",
- "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",
- "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",
- "hasInstallScript": true,
- "license": "MIT",
- "dependencies": {
- "bindings": "^1.5.0",
- "prebuild-install": "^7.1.1"
- }
- },
- "node_modules/bindings": {
- "version": "1.5.0",
- "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
- "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
- "license": "MIT",
- "dependencies": {
- "file-uri-to-path": "1.0.0"
- }
- },
- "node_modules/bl": {
- "version": "4.1.0",
- "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
- "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
- "license": "MIT",
- "dependencies": {
- "buffer": "^5.5.0",
- "inherits": "^2.0.4",
- "readable-stream": "^3.4.0"
- }
- },
- "node_modules/buffer": {
- "version": "5.7.1",
- "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
- "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT",
- "dependencies": {
- "base64-js": "^1.3.1",
- "ieee754": "^1.1.13"
- }
- },
- "node_modules/chownr": {
- "version": "1.1.4",
- "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
- "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
- "license": "ISC"
- },
- "node_modules/decompress-response": {
- "version": "6.0.0",
- "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
- "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
- "license": "MIT",
- "dependencies": {
- "mimic-response": "^3.1.0"
- },
- "engines": {
- "node": ">=10"
- },
- "funding": {
- "url": "https://github.com/sponsors/sindresorhus"
- }
- },
- "node_modules/deep-extend": {
- "version": "0.6.0",
- "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
- "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
- "license": "MIT",
- "engines": {
- "node": ">=4.0.0"
- }
- },
- "node_modules/detect-libc": {
- "version": "2.1.2",
- "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
- "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
- "license": "Apache-2.0",
- "engines": {
- "node": ">=8"
- }
- },
- "node_modules/end-of-stream": {
- "version": "1.4.5",
- "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
- "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
- "license": "MIT",
- "dependencies": {
- "once": "^1.4.0"
- }
- },
- "node_modules/expand-template": {
- "version": "2.0.3",
- "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
- "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
- "license": "(MIT OR WTFPL)",
- "engines": {
- "node": ">=6"
- }
- },
- "node_modules/file-uri-to-path": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
- "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
- "license": "MIT"
- },
- "node_modules/fs-constants": {
- "version": "1.0.0",
- "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
- "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
- "license": "MIT"
- },
- "node_modules/github-from-package": {
- "version": "0.0.0",
- "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
- "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
- "license": "MIT"
- },
- "node_modules/ieee754": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
- "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "BSD-3-Clause"
- },
- "node_modules/inherits": {
- "version": "2.0.4",
- "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
- "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
- "license": "ISC"
- },
- "node_modules/ini": {
- "version": "1.3.8",
- "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
- "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
- "license": "ISC"
- },
- "node_modules/mimic-response": {
- "version": "3.1.0",
- "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
- "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
- "license": "MIT",
- "engines": {
- "node": ">=10"
- },
- "funding": {
- "url": "https://github.com/sponsors/sindresorhus"
- }
- },
- "node_modules/minimist": {
- "version": "1.2.8",
- "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
- "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
- "license": "MIT",
- "funding": {
- "url": "https://github.com/sponsors/ljharb"
- }
- },
- "node_modules/mkdirp-classic": {
- "version": "0.5.3",
- "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
- "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
- "license": "MIT"
- },
- "node_modules/napi-build-utils": {
- "version": "2.0.0",
- "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
- "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
- "license": "MIT"
- },
- "node_modules/node-abi": {
- "version": "3.96.0",
- "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",
- "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",
- "license": "MIT",
- "dependencies": {
- "semver": "^7.3.5"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/once": {
- "version": "1.4.0",
- "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
- "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
- "license": "ISC",
- "dependencies": {
- "wrappy": "1"
- }
- },
- "node_modules/prebuild-install": {
- "version": "7.1.3",
- "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
- "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
- "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
- "license": "MIT",
- "dependencies": {
- "detect-libc": "^2.0.0",
- "expand-template": "^2.0.3",
- "github-from-package": "0.0.0",
- "minimist": "^1.2.3",
- "mkdirp-classic": "^0.5.3",
- "napi-build-utils": "^2.0.0",
- "node-abi": "^3.3.0",
- "pump": "^3.0.0",
- "rc": "^1.2.7",
- "simple-get": "^4.0.0",
- "tar-fs": "^2.0.0",
- "tunnel-agent": "^0.6.0"
- },
- "bin": {
- "prebuild-install": "bin.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/pump": {
- "version": "3.0.4",
- "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
- "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
- "license": "MIT",
- "dependencies": {
- "end-of-stream": "^1.1.0",
- "once": "^1.3.1"
- }
- },
- "node_modules/rc": {
- "version": "1.2.8",
- "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
- "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
- "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
- "dependencies": {
- "deep-extend": "^0.6.0",
- "ini": "~1.3.0",
- "minimist": "^1.2.0",
- "strip-json-comments": "~2.0.1"
- },
- "bin": {
- "rc": "cli.js"
- }
- },
- "node_modules/readable-stream": {
- "version": "3.6.2",
- "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
- "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
- "license": "MIT",
- "dependencies": {
- "inherits": "^2.0.3",
- "string_decoder": "^1.1.1",
- "util-deprecate": "^1.0.1"
- },
- "engines": {
- "node": ">= 6"
- }
- },
- "node_modules/safe-buffer": {
- "version": "5.2.1",
- "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
- "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/semver": {
- "version": "7.8.5",
- "resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
- "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
- "license": "ISC",
- "bin": {
- "semver": "bin/semver.js"
- },
- "engines": {
- "node": ">=10"
- }
- },
- "node_modules/simple-concat": {
- "version": "1.0.1",
- "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
- "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT"
- },
- "node_modules/simple-get": {
- "version": "4.0.1",
- "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
- "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
- "funding": [
- {
- "type": "github",
- "url": "https://github.com/sponsors/feross"
- },
- {
- "type": "patreon",
- "url": "https://www.patreon.com/feross"
- },
- {
- "type": "consulting",
- "url": "https://feross.org/support"
- }
- ],
- "license": "MIT",
- "dependencies": {
- "decompress-response": "^6.0.0",
- "once": "^1.3.1",
- "simple-concat": "^1.0.0"
- }
- },
- "node_modules/string_decoder": {
- "version": "1.3.0",
- "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
- "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
- "license": "MIT",
- "dependencies": {
- "safe-buffer": "~5.2.0"
- }
- },
- "node_modules/strip-json-comments": {
- "version": "2.0.1",
- "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
- "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
- "license": "MIT",
- "engines": {
- "node": ">=0.10.0"
- }
- },
- "node_modules/tar-fs": {
- "version": "2.1.5",
- "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",
- "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",
- "license": "MIT",
- "dependencies": {
- "chownr": "^1.1.1",
- "mkdirp-classic": "^0.5.2",
- "pump": "^3.0.0",
- "tar-stream": "^2.1.4"
- }
- },
- "node_modules/tar-stream": {
- "version": "2.2.0",
- "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
- "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
- "license": "MIT",
- "dependencies": {
- "bl": "^4.0.3",
- "end-of-stream": "^1.4.1",
- "fs-constants": "^1.0.0",
- "inherits": "^2.0.3",
- "readable-stream": "^3.1.1"
- },
- "engines": {
- "node": ">=6"
- }
- },
- "node_modules/tunnel-agent": {
- "version": "0.6.0",
- "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
- "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
- "license": "Apache-2.0",
- "dependencies": {
- "safe-buffer": "^5.0.1"
- },
- "engines": {
- "node": "*"
- }
- },
- "node_modules/util-deprecate": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
- "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
- "license": "MIT"
- },
- "node_modules/wrappy": {
- "version": "1.0.2",
- "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
- "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
- "license": "ISC"
- }
- }
-}
deleted file mode 100644
deleted file mode 100644
@@ -1,13 +0,0 @@
@@ -1,13 +0,0 @@
-{
- "name": "openvibe-tools-scripts",
- "version": "1.0.0",
- "private": true,
- "description": "The operational scripts' own dependencies: better-sqlite3 reads the pre-cutover SQLite files (scripts/migrate-to-postgres.js, scripts/analytics-prune.js, scripts/guard-abuse-report.js). No app depends on it (plan T8). Install with npm --prefix scripts install.",
- "license": "MIT",
- "engines": {
- "node": ">=22"
- },
- "dependencies": {
- "better-sqlite3": "^11.10.0"
- }
-}
deleted file mode 100644
deleted file mode 100644
@@ -1,17 +0,0 @@
@@ -1,17 +0,0 @@
-'use strict';
-// better-sqlite3 for the operational scripts that read the pre-cutover SQLite files (the importer, the
-// analytics prune's SQLite branch, the guard abuse report). No app depends on it any more (plan T8), so
-// scripts/ has its own package.json: `npm --prefix scripts install`. An app that still has it installed
-// (a host before the cutover deploy) is the fallback.
-const path = require('path');
-
-const APPS = path.join(__dirname, '..', 'apps');
-
-function loadSqlite() {
- for (const dir of [__dirname, ...['gateway', 'maps', 'food', 'img', 'yt', 'audio', 'text', 'docs'].map((a) => path.join(APPS, a))]) {
- try { return require(require.resolve('better-sqlite3', { paths: [dir] })); } catch { /* next */ }
- }
- throw new Error('better-sqlite3 is not installed for scripts/ (run npm --prefix scripts install)');
-}
-
-module.exports = { loadSqlite };