Commits

main

README: the removed routes are documented as removed (plan T8 phase 6: /api/process*, /api/net/*, /api/dev/* and the /api/net/tools and /api/dev/tools lists, with their successors), the job store is in the tools database, and the T8 state is current (better-sqlite3 only in scripts/).OpenVibers · 29 Sept 2026 cf706b6
Net page: each run's input is built from the tool's descriptor (plan T8 review): the page fetches GET /api/v1/tools/<id>/schema and sends only the fields its input schema declares (mx., txt., ns., spf., dkim. and dmarc. got a types field their schemas refuse, latency. a count), and the plain domain to dkim. (its selector as its own field) and dmarc. (the descriptor's route adds _dmarc. and <selector>._domainkey.; the page prefixed too). net-page-input.test.js checks every net tool's page input against its real schema.OpenVibers · 29 Sept 2026 78b24f8
Tools jobs: a failed boot recovery no longer strands queued jobs (plan T8 review): start() rejects and leaves the system unstarted (started is set only after recovery, so the worker never claims during it) and can be called again; setupJobs retries it with a backoff (1 s doubling to 60 s) until it runs or the app closes. jobs-start-retry.test.js.OpenVibers · 29 Sept 2026 ecb37ee
Tools job store: events and references go through the owning app (plan T8 review): tool_job_events and tool_job_references carry no app, so their reads, inserts and deletes now join or check this app's tool_jobs row, and every write by job id filters on app too; another app that knows an id reads and changes nothing. jobs-store-scope.test.js.OpenVibers · 29 Sept 2026 b2ade2c
Tools ops after better-sqlite3 left the apps (plan T8): the legacy deploy's load checks use openvibe-sdk/db instead of better-sqlite3 (they would abort every deploy); scripts/ has its own package.json with better-sqlite3 for the cutover importer and the SQLite-era reports (npm --prefix scripts install), loaded through scripts/sqlite.js; analytics-prune on the tools database refuses --app instead of pruning every service; the apps' lockfiles regenerated without it. ops-scripts.test.js.OpenVibers · 29 Sept 2026 83f844c
Tools guard review fixes (plan T8): one Lua call decides and charges a run's buckets and day allowances all or nothing (a run refused on the day leg spent bucket tokens, so refused requests could drain a bucket); the store's day counters are one atomic check-and-increment (HGET then HINCRBYFLOAT let concurrent runs overshoot the day); the salt is asked of the store every time (one cached before Valkey answered stayed process-local all day); each app prunes only its own guard_abuse rows. guard.test.js covers each; guard-valkey.test.js runs the Lua on a real Valkey under test:pg.OpenVibers · 29 Sept 2026 f2bfb53
Tools job store on PostgreSQL (plan T8 phases 2-3; not deployed yet): apps/_shared/jobs on openvibe-sdk/db with async ambient transactions (finish, the usage rollup and the outbox in one), every app sharing the one tools database, ready.postgres, better-sqlite3 gone with a guard test; job reads scoped by app (a shared database let one app read another's job by id); a cancel that lands while a job prepares ends it. npm test and test:pg 58/58.OpenVibers · 29 Sept 2026 d276e3b
Tools: the legacy routes are gone (plan T8, phase 6; not deployed yet): /api/process, /api/process/direct and /api/process/multi on img, docs and audio, and the /api/net and /api/dev mounts on the gateway, with their deprecation shims, successors and descriptor legacy lists; the net and dev handlers stay behind the run API, and /api/dev/webhook/* keeps its URL on its own router. The SSRF test runs through the run route with every case kept; the other legacy-route tests are converted or removed. npm test 57/57 and test:pg 57/57.OpenVibers · 29 Sept 2026 d2124f4
Tools on PostgreSQL, part 2 (plan T8, phases 1 and 4; not deployed yet): the network and dev pages call the tools run API (/api/v1/tools/<id>/run) instead of /api/net and /api/dev, a signed-in person's probe is a normal quota-metered run and a signed-out visitor gets a sign-in card for probes (port, ping, latency, traceroute, mtr) while lookups stay open; the guard's day counters, salt and buckets live in Valkey (two processes share one budget; an outage test). npm test 57/57 and test:pg 57/57 (verified). The job store on openvibe-sdk/db and the legacy route deletion follow before the deploy.OpenVibers · 29 Sept 2026 c740ca9
Every Tools app on openvibe-sdk v0.25.1 and openvibe-shared v2.3.1 (from 0.12.0 and 1.29.2; no removed Shared export was in use): the PostgreSQL port needs openvibe-sdk/db, valkey, the limit store, createPgOutbox and the revocation store. npm test 57/57.OpenVibers · 29 Sept 2026 4a5479b
Tools on PostgreSQL, the groundwork (plan T8): migrations 0001_tools (one tools database for the eight apps: the job store and guard_abuse with app in every key that could collide; guard counters and salt go to Valkey) and 0002_analytics; scripts/migrate-to-postgres.js imports every app's SQLite files into it with per-table counts and checksums (rehearsed on PGlite: every table reconciled); scripts/guard-abuse-report.js (Phase 0: only reason=quota was ever recorded in report mode); the img, docs and audio pages lose their dead /api/process fallback (they submit through OVJobs, as production already did). npm test 57/57 (verified).OpenVibers · 29 Sept 2026 58f8b51
X-Internal-Key is gone from Tools (plan T2): GET /api/internal/analytics[/bots] on the gateway and the seven tool sites take only a Network service token with tools.analytics.read (401 token.missing without one), loopback only; the gateway forwards only the caller's Bearer. apps/_shared/internal-auth.js is deleted (loopbackOnly moved into internal-token.js); INTERNAL_API_KEY/OV_INTERNAL_KEY leave .env.example and the README. Tests: the key alone and a bad Bearer beside the key are 401, the token passes and is forwarded, and no server file reads the key. npm test green (verified).OpenVibers · 29 Sept 2026 139ece3
Internal analytics take Network service tokens (plan T2, X-Internal-Key retirement): GET /api/internal/analytics (and /bots) on the gateway and all seven tool sites accept a token holding tools.analytics.read (audience openvibe.tools, never sandbox) through one shared helper (apps/_shared/internal-token.js), loopback only. A Bearer is judged on the token alone, never downgraded to the key; without one the key still passes until Network stops sending it. The gateway forwards the caller's proof to the satellites: the Bearer with a token, the key otherwise. openvibe-contracts v0.79.0 in the four apps that pin it. Tests: capability, audience, sandbox, a bad Bearer plus the right key refused, the public edge refused, the key alone, the Bearer forwarded. npm test 57/57.OpenVibers · 29 Sept 2026 55e4b84
openvibe-shared v1.29.2 in every app (from v1.29.1): navbar dropdowns are no longer clipped by the links row. npm test 55/55.OpenVibers · 29 Sept 2026 d155452
openvibe-shared v1.29.1 in every app (from v1.28.1): changing the theme repaints the page background, bright accents get dark button text (WCAG), the navbar brand is more compact. npm test 55/55.OpenVibers · 29 Sept 2026 d92c726
Whois and RDAP lookups work again: rdap.org answered every request with 403 because Node's fetch sends no real User-Agent. Domains now go straight to their TLD's own RDAP server from IANA's bootstrap file (kept a day), with a descriptive User-Agent; rdap.org, with the same header, is the fallback and serves IPs. Input that is neither a domain nor an IP gets 'Enter a full domain name (like example.com) or an IP address', and a registry's 404 reads 'No registration record for <name>' instead of a bare status. openvibe-shared v1.28.1 in all eight apps (from v1.23.0): the account panel on phones, no empty space under the footer on phones. npm test 55/55.OpenVibers · 28 Sept 2026 7c8af01
npm test no longer counts a skipped test as passed (roadmap WS-Q task 7, the rule of openvibe-shared/test-runner). test/run.js counted every app test that exited 0 as passed, so the three audio tests without ffmpeg and sdk-client without an OpenVibe.SDK checkout, which check nothing and exit 0, read as ok. A test now prints "<label>: skipped (<why>)" for what it could not run, and the runner lists it as skip with its reasons and does not count it as passed: the summary reads "52/55 test files passed, 3 skipped (…)", and only a run with nothing skipped says N/N; --strict (or OV_TEST_STRICT=1) makes a skip fail the run. The per-app runner stays (the syntax check, each test from its app directory, 180 s timeout); SKIP_RE and skipsIn are copied from the module because the root has no dependencies (each app pins openvibe-shared on its own, unchanged here). audio merge, audio jobs and audio hardening printed "SKIPPED (no ffmpeg on this machine)" and now print "skipped (…)"; sdk-client already printed the line. Without ffmpeg the three audio files are listed as skipped. npm test (Node 22, ffmpeg and the SDK checkout present): syntax 199/199, 55/55 test files passed.OpenVibers · 28 Sept 2026 9339e15
README and STATUS.json are current (roadmap WS-U task 3): the README gains Purpose (the opening overview), Owns, Does not own, Depends on (the per-app pins: contracts v0.63.0, sdk v0.12.0, shared v1.23.0), Capabilities (the six tools.* capabilities and the grants Tools uses at Media, Events and Search) and Security (SECURITY.md, the guard, egress, auth, secrets by name) sections; Deploy opens with sudo ovhost deploy tools and the rollback. Stale facts corrected: pages load the shared files from their own app's pinned copy (since 390d3df), not openvibe.network, and the SDK test uses the pinned SDK, not v0.6.0. STATUS.json is new, in the shape of Search's, updated 2026-09-28.OpenVibers · 28 Sept 2026 4f4ed6e
Per-actor rate limits in every Tools app (roadmap WS-R task 4, openvibe-sdk v0.12.0 limits; openvibe-tools-shared 1.2.0), only where the guard has no per-caller limit of its own. apps/_shared/actor-limits.js makes one limiter per app from the SDK the app passes in (apps/_shared takes no dependencies). It reads the guard's resolved caller: a person as user:usr_…, a service or app by its principal, a browser session or nobody by the guard's hashed address key, so no raw address is ever logged. The guard, its tiered quotas, the older express-rate-limit limiters and TOOLS_GUARD are unchanged. Past a limit a route answers 429 problem+json rate_limited with Retry-After, logs one [Limits] line and counts tools_rate_limited_total{limit,window}.OpenVibers · 28 Sept 2026 e55469f
Tools pages run their own pinned shared files (roadmap WS-P task 4, D42: the Tools satellites' static pages). apps/_shared/release.js mounts /shared (openvibe-shared/serve, content-addressed, only the browser files) in every app with the release routes, before any page fallback: until now only the gateway host served it, and the satellites answered /shared/navbar.js with an HTML page. 45 pages in audio, docs, food, gateway, img, maps, text and yt load navbar, footer, theme loader, bell, account switcher, shipped, icons, history, ui and island from /shared/ instead of https://openvibe.network/shared/ (217 references). So each page runs what its app's pin says (openvibe-shared 1.23.0), not what Network happens to pin, and keeps its frame while Network is down. The gateway's own /shared line is folded into the same mount. Tests: release.test serves the hashed navbar (immutable) and refuses a non-browser file; shared-pins.test fails on any page loading a shared file from openvibe.network or naming one its pin does not serve. npm test 53/53.OpenVibers · 27 Sept 2026 390d3df
deploy/scripts/deploy.sh hands over to ovhost deploy tools (OpenVibe.Host strategy multi-app; roadmap WS-N task 11). The old script is kept, unchanged, as deploy/scripts/deploy-legacy.sh. ovhost does what it did (untracked lockfiles the release tracks removed first, per-app installs with apps/_* skipped, every dependency resolves, the jobs runtime in img/audio/docs and the guard in every app load, data/ made, every openvibe-tools* unit restarted) and adds an automatic rollback, /api/ready with the new sha on the gateway's /release.json and every unit active, freezes and the release log. The wrapper passes --wait-idle, --restart, --force, --rollback and DRY_RUN=1 (ovhost plan tools); it asks ovhost capabilities tools first and runs deploy-legacy.sh, saying why, when ovhost is missing, too old or the host inventory does not deploy tools with multi-app (OVHOST_LEGACY=1 forces it), refusing --rollback, --wait-idle and DRY_RUN there rather than ignoring them. apps/_shared/test/deploy-wrapper.test.js; apps/_shared/test/version.test.js checks the apps/_* skip in deploy-legacy.sh and that the wrapper only hands over to multi-app. README.OpenVibers · 27 Sept 2026 5768232
Dev tools run on the network's web runtime; the webhook viewer shows a sender's method and headers as text (roadmap WS-P task 6, the second site; openvibe-shared 1.22.0 -> 1.23.0 in every app). dev.html loads /shared/web-runtime.js (the gateway serves it from its pin) and makes each Run a route generation: a new Run or Clear ends the previous one's work. Before, every Run of the webhook tool left a 2 s poller running for an hour (the pollers piled up), a slow earlier result could overwrite a newer one, and two quick minifier runs added Terser's script twice. Now the bin's poller is a child of the run's scope (stops on the next Run or Clear, an hour at most) and polls with the run's AbortSignal, a stale run never writes the output, and Terser loads once through the runtime. Security (low/medium, stored XSS): the webhook viewer inserted each captured request's method and header names and values into innerHTML unescaped (only the body was escaped), so whoever sends to a bin URL could run script on *.openvibe.tools, where ov_token sits in localStorage; they are escaped now. apps/gateway/test/dev-runtime.test.js runs the page's script in a vm: one poller after repeated runs, the aborted request, Clear, a hostile method and header shown as text, one Terser tag. Checked in headless Chrome against a local gateway: two runs at once, one Terser tag, the right output, RT.leaks() empty. npm test 51/51.OpenVibers · 27 Sept 2026 68f1330
maps: no RIDB API key in the code. RIDB_API_KEY comes from the environment only (set in production's /etc/openvibe/tools.env on 2026-09-27, before this deploy); without it RIDB is skipped with one warning. The two keys that were defaults here are in the repository's history and should be rotated at ridb.recreation.gov (the unused config one can simply be dropped).OpenVibers · 27 Sept 2026 b635f97
Secrets never reach a visitor of the gateway or the maps satellite, by test (roadmap WS-R task 5: internal-secret leak). apps/_shared/test/no-network.js is a preload for a started satellite: every connection or lookup that is not loopback fails at once, so third-party calls take their error paths without the internet.OpenVibers · 26 Sept 2026 427b354
The paste proxy reaches only Community's paste API, and yt-dlp only YouTube links (roadmap WS-R task 5: SSRF; low). The gateway forwarded /api/pastes/* to `${communityUrl}/api/pastes${req.url}` with the visitor's token: "/api/pastes/../../internal/x" (or its %2e%2e form) resolved to Community's /internal/x, any other Community route likewise, and encoded slashes or backslashes became path separators on the other side (the hole Community's own Live proxy had). A path that does not resolve inside /api/pastes, or carries %2f/%5c, is now a 404 and never sent. yt's link check accepted any scheme and port on a YouTube host (file:, ftp:, gopher:, :6379); it now takes http(s) on the default ports without credentials.OpenVibers · 26 Sept 2026 3680e83
Project usage rollups (roadmap WS-N task 4; Contracts v0.63.0 tools.usage.recorded, common.usage-recorded@1): a developer app's job (it carries a project) that ends is counted in the transaction that records its end (apps/_shared/jobs/usage.js, table tool_job_usage in the satellite's jobs.db), per project, environment, capability (tools.tool.run for a tool run, tools.job.create for POST /api/v1/jobs), job type or tool and UTC hour: jobs that ended (succeeded or failed; cancelled ones are not counted) and the failed ones by problem code, with the last ten failures' job id, status and the trace id of the request that submitted the job (new tool_jobs.trace_id, kept by submit({ traceId }) from jobs/http.js and tools/run.js). A minute after an hour closes, the pruner's timer writes each rollup once to the satellite's outbox as tools.usage.recorded (subject the project, visibility internal, priority low) in the transaction that marks it sent; a job ending in an hour already sent re-sends it as revision 2. Sandbox jobs count under env sandbox. No owner, session, address, input, file name or output leaves. Inert without EVENTS_URL, like the job events; GET /api/health shows jobs.usage. openvibe-tools-shared 1.0.0 -> 1.1.0 (additive). Pins: openvibe-contracts 0.49.0 -> 0.63.0 (audio, docs, gateway, img). test: apps/_shared/test/job-usage.test.jsOpenVibers · 26 Sept 2026 f15b263
deploy.sh: apps/_shared is not an app. Now that it has a package.json (37e6cf3), the install, dependency and guard-preflight loops over apps/*/ took it for one, and the guard preflight aborted the deploy (nothing restarted) trying to load better-sqlite3 from it. Each loop skips apps/_*; version.test.js checks they keep doing so.OpenVibers · 26 Sept 2026 9102d60
apps/_shared is a versioned package, openvibe-tools-shared 1.0.0 (roadmap WS-L task 3, the last part): package.json and CHANGELOG.md carry its version, and every app declares the range it was written against ("openvibeToolsShared": "^1.0.0" in its package.json). The apps still require it by relative path from the same checkout, so the declaration is checked at boot: toolsRelease() calls requireShared(app), and an app whose range this checkout does not satisfy refuses to start. A breaking change bumps the major and every app's range in the same commit. test: apps/_shared/test/version.test.jsOpenVibers · 26 Sept 2026 37e6cf3
Pins: openvibe-shared 1.20.0 -> 1.22.0 (realtime notification badge, off until a site turns it on; the unreachable-host browser check)OpenVibers · 26 Sept 2026 168e511
One upload runtime (roadmap WS-L task 3): apps/_shared/upload.js createUploads({ multer, config, storage, maxFiles, rejectHint }) replaces img's, audio's and docs' own multer middleware (memory for img, disk for audio and docs, 5 and 50 files); disk names are random with a sanitised extension everywhere (audio kept any extension before); the per-app copies are deleted and a test keeps them from coming backOpenVibers · 26 Sept 2026 20bb30c

More commits