openvibers/openvibe.tools Commits
Tools guard (apps/_shared/guard), used by the gateway and every satellite and driven by the tool descriptors: one caller resolver replaces the four copied auth.js files and the job routes' owner logic (service/app principals by token, aud openvibe.tools, sandbox apps at their own small tier; users by the Network token with the audience check the copies lacked; the ov_tools_jobs browser session; else anonymous by address, IPv6 by /64, hashed). trust proxy is TRUST_PROXY everywhere: exactly one hop, on loopback only (satellites used 2, maps/food 'loopback'); the gateway's proxy to a satellite now passes its own req.ip as that hop instead of appending nginx's socket address, and loopback services are told apart by their tokens. Tiered token-bucket quotas by quotaClass x tier weighted by each descriptor's cost (limits.js holds every number; tools-api for every /api/ call), UTC-day allowances in data/guard.db that survive restarts, browser sessions from one address sharing 3x one session's allowance, RateLimit-* headers, 429 problem+json tools.quota.exceeded with Retry-After. Heavy work: a semaphore for sync /api/process in img, audio and docs (503 tools.busy), the job store's queue-depth and disk bounds (system.busy(): TOOLS_JOBS_MAX_QUEUED, TOOLS_DISK_BUDGET_MB) and unfinished session jobs bounded per address as well as per owner (jobs keep only the hashed address key). Uploads: magic-byte sniffing against files.accept (415 tools.file.unsupported_type; a misleading extension is corrected; audio no longer takes whatever says octet-stream), docs uploads on disk instead of 100 MB x 50 in memory, sharp and the BMP/ICO/HEIC readers refuse inputs over 40 MP from the header (413), audio and PDF tools need a session, sign-in or token (auth.anonymous false; /api/context starts the page's session). ffmpeg: every input and probe gets -protocol_whitelist file,pipe, the operation's demuxer whitelist and -t = the descriptor's maxDurationSec (now 3 h on every audio tool), inputs are probed and a longer one refused first, and the sync path runs in the job context with the descriptor's timeout and is killed at it (504 tools.run.timeout) or when the client goes away. Egress: a per-target throttle across all callers and tools from limits.perTargetPerMinute (net tools, Open Graph), the port checker capped per request (20, refused not cut) and per caller (100 ports, 10 hosts per 10 minutes; sessions by address), probes through a token need tools.net.probe, webhook bins owned by their maker (session cookie, sent by the dev page with credentials, or token) with per-owner and per-address caps, yt downloads owned by whoever started them, Nominatim paced to 1/s and the Overpass wait list capped. Abuse log in guard.db: HMAC-SHA256(address, a random daily salt, only today's kept), principal or user id, tool, reason, enforced or not, repeats folded per minute, pruned after 30 days, no raw address anywhere; metrics tools_guard_refused_total{reason,tool}, tools_guard_enforcing, tools_guard_sync. A no-op challenge hook for Turnstile later. TOOLS_GUARD=report (default) records and counts what it would refuse and keeps the apps' older limiters in force, now keyed by the resolved caller after sign-in is read (the limiter-before-optionalAuth order bug in img, audio and docs); enforce refuses and the older limiters step aside. Hard limits apply in both modes: pixels, ffmpeg whitelists and duration, sniffing, the port-scan cap and the per-target throttle. The gateway gets better-sqlite3 for its guard.db. Tests: _shared guard.test.js (keys, tiers, bucket math and cost, session sharing, report vs enforce, headers and problems, day counters across a restart, sniffing, target throttle, port cap, abuse log without raw IPs and its pruning, the semaphore, job bounds, the challenge hook) and per-app guard/hardening tests (img pixel limit and sniffing, audio playlist/concat refusal at every layer with the duration cap and sync kill, docs disk uploads and session rule, gateway quotas, throttle, port cap, webhook ownership and proxy address)
Changes
nnew file mode 100644@@@ -0,0 +1,152 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// Who is asking — the one resolver for the gateway and every satellite (it replaces the four copied++// auth.js files and the job routes' own owner logic).++//++// principal Authorization: Bearer <service or app token>, aud openvibe.tools (tier service; a++// developer app's sandbox token: tier sandbox, job routes only take it from apps)++// user a Network sign-in (the ov_token cookie or a Bearer user token, aud openvibe.tools)++// session this browser's ov_tools_jobs cookie (only its hash is ever kept)++// anonymous nobody: counted by address, an IPv6 address by its /64++//++// Addresses are req.ip only (`trust proxy` is one loopback hop, ./ip.js). A first-party service on++// loopback is not special: it is identified by its service token, and without one it is anonymous,++// counted by the address it forwarded (as the one trusted hop) or else by its own.++//++// → { kind, tier, key, ipKey, ipHash, owner, id, claims, env, error? }++// key what quotas count (session hash, user:usr_…, the principal's sub, ip:<hash>)++// ipKey ip:<HMAC-SHA256(today's salt, address or /64)>, never the address itself++// owner the job owner (user:usr_…, the principal's sub, session:<hash>) or null++// error [status, code, detail] for a bad principal token on the job routes++// ═══════════════════════════════════════════════════════════════++++const crypto = require('crypto');++const { ipBucket } = require('./ip');++const tokens = require('./tokens');++++const SESSION_RE = /^[A-Za-z0-9_-]{32,64}$/;++const USER_ID_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;++const JOB_CAPS = { create: 'tools.job.create', read: 'tools.job.read', cancel: 'tools.job.cancel' };++const COOKIE = 'ov_tools_jobs';++++/**++ * @param {object} o++ * @param {() => string|null} o.getPublicKey++ * @param {string} o.issuer++ * @param {string} [o.audience='openvibe.tools']++ * @param {object} [o.contracts] openvibe-contracts, when the app has it++ * @param {() => string} [o.salt] today's salt (guard.db); a process-local one otherwise++ * @param {string} [o.cookieName='ov_tools_jobs']++ * @param {boolean} [o.secureCookie]++ */++function createCallerResolver(o) {++ const audience = o.audience || tokens.AUDIENCE;++ const cookieName = o.cookieName || COOKIE;++ const contracts = o.contracts || null;++ let localSalt = null;++ const salt = o.salt || (() => {++ const day = new Date().toISOString().slice(0, 10);++ if (!localSalt || localSalt.day !== day) localSalt = { day, salt: crypto.randomBytes(32).toString('hex') };++ return localSalt.salt;++ });++ const isUserId = (sid) => (contracts ? contracts.ids.isSubjectId('user', sid) : USER_ID_RE.test(String(sid || '')));++++ function hashIp(ip) {++ return crypto.createHmac('sha256', salt()).update(ipBucket(ip) || 'unknown').digest('hex').slice(0, 32);++ }++++ function sessionOwner(value) {++ return `session:${crypto.createHash('sha256').update(value).digest('hex').slice(0, 40)}`;++ }++++ /** Mint the browser-session cookie now (pages call this so their first job already has one). */++ function ensureSession(req, res) {++ let v = req.cookies && req.cookies[cookieName];++ if (SESSION_RE.test(String(v || ''))) return v;++ v = crypto.randomBytes(24).toString('base64url');++ res.cookie(cookieName, v, {++ httpOnly: true, sameSite: 'lax', path: '/', maxAge: 7 * 24 * 60 * 60 * 1000,++ secure: o.secureCookie != null ? o.secureCookie : process.env.NODE_ENV === 'production',++ });++ if (req.cookies) req.cookies[cookieName] = v; else req.cookies = { [cookieName]: v };++ return v;++ }++++ /**++ * @param {object} req++ * @param {object} [res] needed with create++ * @param {object} [opt]++ * @param {boolean} [opt.create] mint a session cookie for an anonymous caller++ * @param {string} [opt.action] job routes: create | read | cancel (a principal needs tools.job.<action>)++ * @param {boolean} [opt.jobs] job routes: a bad principal token is an error, not anonymity++ */++ function resolve(req, res, opt = {}) {++ const ipHash = hashIp(req.ip || (req.socket && req.socket.remoteAddress));++ const base = { ipKey: `ip:${ipHash}`, ipHash, claims: null, env: 'production' };++++ // 1. A principal's token (service or developer app).++ const h = String((req.headers && req.headers.authorization) || '');++ if (h.startsWith('Bearer ')) {++ const token = h.slice(7).trim();++ if (tokens.looksLikePrincipal(tokens.peek(token))) {++ const r = tokens.verifyServiceToken(token, { publicKey: o.getPublicKey(), issuer: o.issuer, audience, acceptSandbox: true, contracts });++ let error = null;++ if (!r.ok) error = [401, r.code, r.reason];++ else if (r.claims.env === 'sandbox' && !(r.claims.actor_type === 'app' && /^app:/.test(String(r.claims.sub || '')))) error = [401, 'token.sandbox_refused', 'sandbox tokens are accepted only from developer apps'];++ if (!error && opt.action) {++ const cap = JOB_CAPS[opt.action] || JOB_CAPS.read;++ const denied = tokens.capabilityDenied(r.claims, cap, contracts);++ if (denied) error = [403, denied, `${cap} not granted`];++ }++ if (error && opt.jobs) return { ...base, kind: 'anonymous', tier: 'anonymous', key: base.ipKey, owner: null, id: null, error };++ if (!error) {++ const sandbox = r.claims.env === 'sandbox';++ return { ...base, kind: 'principal', tier: sandbox ? 'sandbox' : 'service', key: r.claims.sub, owner: r.claims.sub, id: r.claims.sub, claims: r.claims, env: sandbox ? 'sandbox' : 'production' };++ }++ // Elsewhere a token that does not verify is simply not believed: the caller is anonymous.++ }++ }++++ // 2. A signed-in person (req.user comes from the guard's identify(): signature, issuer, audience).++ const user = req.user || null;++ const sid = user && user.subject_id;++ let session = req.cookies && req.cookies[cookieName];++ if (!SESSION_RE.test(String(session || ''))) session = null;++ if (user) {++ const id = isUserId(sid) ? `user:${sid}` : `user:${String(user.sub || user.id || 'unknown')}`;++ // A job belongs to the person's subject; a token without one (older sign-ins) keeps its jobs in the browser session.++ let owner = isUserId(sid) ? `user:${sid}` : null;++ if (!owner) {++ if (!session && opt.create && res) session = ensureSession(req, res);++ owner = session ? sessionOwner(session) : null;++ }++ return { ...base, kind: 'user', tier: 'user', key: id, owner, id, claims: user };++ }++++ // 3. This browser's session, 4. nobody.++ if (!session && opt.create && res) session = ensureSession(req, res);++ if (session) {++ const owner = sessionOwner(session);++ return { ...base, kind: 'session', tier: 'session', key: owner, owner, id: null };++ }++ return { ...base, kind: 'anonymous', tier: 'anonymous', key: base.ipKey, owner: null, id: null };++ }++++ return { resolve, ensureSession, hashIp, sessionOwner, cookieName };++}++++/**++ * The job routes' owner resolver (apps/_shared/jobs/http.js), on top of the one caller resolver:++ * → { owner, kind: 'principal'|'user'|'session', claims?, env } | { error } | null (anonymous, not creating).++ */++function jobOwnerResolver(callers) {++ return function resolveOwner(req, res, { create = false, action = 'read' } = {}) {++ const c = callers.resolve(req, res, { create, action, jobs: true });++ if (c.error) return { error: c.error };++ if (!c.owner) return null;++ const kind = c.kind === 'principal' ? 'principal' : c.owner.startsWith('user:') ? 'user' : 'session';++ return { owner: c.owner, kind, claims: c.claims || undefined, env: c.env, caller: c };++ };++}++++module.exports = { createCallerResolver, jobOwnerResolver, JOB_CAPS, SESSION_RE, COOKIE };nnew file mode 100644@@@ -0,0 +1,30 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// The challenge hook (decision 4: Turnstile later, not now).++//++// A challenge provider decides whether a caller about to be refused for a quota may instead prove it++// is a person, and checks that proof. The guard asks it before a quota refusal of an anonymous or++// session caller; with the default (none) nobody is ever challenged and nothing changes.++//++// {++// name: 'turnstile',++// required(req, caller, { reason, tool }) → boolean challenge instead of refusing?++// verify(req, caller) → Promise<boolean> the proof the request carries is good++// }++//++// A request that passes verify() is let through once (its quota is still counted); one that needs a++// challenge and carries no good proof gets 403 problem tools.challenge.required, with the provider's++// name in `challenge` so the page knows which widget to show.++// ═══════════════════════════════════════════════════════════════++++const NO_CHALLENGE = Object.freeze({++ name: 'none',++ required() { return false; },++ async verify() { return true; },++});++++function validChallenge(c) {++ return !!c && typeof c.required === 'function' && typeof c.verify === 'function';++}++++module.exports = { NO_CHALLENGE, validChallenge };nnew file mode 100644@@@ -0,0 +1,103 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// ffmpeg hardening (hard limits: they apply in report mode too).++//++// An upload is a file on disk, but ffmpeg decides by the bytes what to do with it: an HLS playlist or an++// ffconcat list makes it open other files (anything the service user can read) or URLs. So every++// input of every command and every probe gets++// -protocol_whitelist file,pipe nothing but local files and pipes++// -format_whitelist <the operation's demuxers> no hls, concat, image2, lavfi, sdp, tty, …++// -t <limits.maxDurationSec> never more than the descriptor's longest input is read++// and probeInputs() refuses an input longer than maxDurationSec before any work starts (the -t is the++// backstop for a file whose header lies). A synchronous run gets a deadline (limits.timeoutMs) and is++// killed like a cancelled job when it passes (the audio app runs sync calls in its job context).++//++// No dependencies: the app passes fluent-ffmpeg.++// ═══════════════════════════════════════════════════════════════++++// Demuxer names (ffmpeg -formats): audio containers, and the video containers only the extractor takes.++const AUDIO_DEMUXERS = ['mp3', 'wav', 'flac', 'ogg', 'mov', 'aac', 'asf', 'aiff', 'ac3', 'eac3', 'matroska', 'amr'];++const VIDEO_DEMUXERS = ['avi', 'flv'];++const PROTOCOLS = 'file,pipe';++++/** The demuxers an operation may open (extract takes video files too). */++function formatsFor(operation) {++ return operation === 'extract' ? [...AUDIO_DEMUXERS, ...VIDEO_DEMUXERS] : AUDIO_DEMUXERS;++}++++/** Input options for one input. */++function inputArgs({ formats = AUDIO_DEMUXERS, maxDurationSec = null } = {}) {++ return [++ '-protocol_whitelist', PROTOCOLS,++ '-format_whitelist', formats.join(','),++ ...(maxDurationSec > 0 ? ['-t', String(maxDurationSec)] : []),++ ];++}++++/** Options for ffprobe (no -t: a probe reads headers). */++function probeArgs({ formats = AUDIO_DEMUXERS } = {}) {++ return ['-protocol_whitelist', PROTOCOLS, '-format_whitelist', formats.join(',')];++}++++/** Put the hardening options on every input of a fluent-ffmpeg command (once). */++function applyToCommand(cmd, opts) {++ for (const input of cmd._inputs || []) {++ if (input.__ovHardened) continue;++ input.options(...inputArgs(opts));++ input.__ovHardened = true;++ }++}++++class InputRefused extends Error {++ constructor(message, reason) { super(message); this.status = reason === 'duration' ? 413 : 415; this.code = reason === 'duration' ? 'tools.file.too_large' : 'tools.file.unsupported_type'; this.guardReason = `ffmpeg.${reason}`; this.expose = true; }++}++++/**++ * Wrap fluent-ffmpeg once: every command's inputs and every ffprobe get the options above.++ * @param {Function} ffmpeg require('fluent-ffmpeg')++ * @param {() => object} optionsNow { formats, maxDurationSec } for the command being started++ * (the app reads its AsyncLocalStorage context)++ */++function harden(ffmpeg, optionsNow) {++ const proto = ffmpeg.prototype;++ if (proto.__ovHardened) return;++ const run = proto.run;++ proto.run = function hardenedRun(...args) {++ applyToCommand(this, optionsNow() || {});++ return run.apply(this, args);++ };++ const probe = proto.ffprobe;++ proto.ffprobe = function hardenedProbe(...args) {++ // ffprobe([index], [options], callback): the options go first; ours are added to them.++ const cb = args[args.length - 1];++ let index = null, options = [];++ if (args.length === 3) { index = args[0]; options = args[1] || []; } else if (args.length === 2) { if (typeof args[0] === 'number') index = args[0]; else if (Array.isArray(args[0])) options = args[0]; }++ const all = [...probeArgs(optionsNow() || {}), ...options];++ return index === null ? probe.call(this, all, cb) : probe.call(this, index, all, cb);++ };++ proto.__ovHardened = true;++}++++/**++ * Probe every input (hardened) before work starts: each must open with the allowed demuxers, and++ * together they must fit maxDurationSec. → the total duration in seconds (0 when unknown).++ * @param {Function} probe (path) → Promise<ffprobe data> (the app's helper; hardened by harden())++ */++async function probeInputs(probe, paths, { maxDurationSec = null } = {}) {++ let total = 0;++ for (let i = 0; i < paths.length; i++) {++ let info;++ try { info = await probe(paths[i]); } catch {++ throw new InputRefused(paths.length > 1 ? `File ${i + 1} is not an audio or video file this tool can read.` : 'This is not an audio or video file this tool can read.', 'format');++ }++ const d = parseFloat(info && info.format && info.format.duration);++ if (Number.isFinite(d) && d > 0) total += d;++ }++ if (maxDurationSec > 0 && total > maxDurationSec) {++ const len = (s) => (s >= 120 ? `${Math.round(s / 60)} minutes` : `${Math.round(s)} seconds`);++ throw new InputRefused(`The audio is ${len(total)} long; this tool takes at most ${len(maxDurationSec)}.`, 'duration');++ }++ return total;++}++++module.exports = { AUDIO_DEMUXERS, VIDEO_DEMUXERS, PROTOCOLS, formatsFor, inputArgs, probeArgs, applyToCommand, harden, probeInputs, InputRefused };nnew file mode 100644@@@ -0,0 +1,449 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// The Tools guard (apps/_shared/guard): anti-abuse for the gateway and every satellite, driven by++// the tools' descriptors (tools.tool@1: quotaClass, cost, auth, files.accept, limits) and the host-wide++// numbers in ./limits.js. Like the rest of apps/_shared it has no dependencies: the app passes its++// better-sqlite3 class, openvibe-contracts (when it has it) and its descriptor specs.++//++// const guard = createGuard({ app: 'img', dataDir, Database, contracts, specs, networkUrl, … });++// app.set('trust proxy', TRUST_PROXY); // ./ip.js: one loopback hop++// app.use(guard.identify); // req.user (Network sign-in, aud checked)++// app.use('/api/', legacyLimiter, guard.apiQuota); // every /api/ request (tools-api)++// app.post('/api/process', guard.toolQuota(hostTool), upload, guard.admitUpload(toolOf), guard.heavy(), …)++//++// What it does:++// callers one resolver (./caller.js): anonymous (address, IPv6 /64) < session < user < service;++// sandbox tokens keep their small allowance++// quotas token buckets per class × tier weighted by the tool's cost (./quota.js), day allowances++// in guard.db; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal++// 429 problem+json tools.quota.exceeded with Retry-After++// heavy a semaphore for synchronous heavy calls (503 tools.busy + Retry-After when it is full)++// uploads the bytes are checked against the descriptor's files.accept (415 tools.file.unsupported_type);++// heavy tools (auth.anonymous false) need a session cookie, a sign-in or a token++// egress a per-target throttle across all callers (limits.perTargetPerMinute) and the port-scan cap++// log every refusal (and every would-be refusal in report mode) in guard.db's abuse log:++// HMAC(address, today's salt), principal or user id, tool, reason — 30 days, no raw++// address anywhere; metric tools_guard_refused_total{reason,tool}++// challenge a hook for a person-check (Turnstile later; ./challenge.js, a no-op now)++//++// Mode: TOOLS_GUARD=report (default) logs and counts what it would refuse and refuses nothing, except++// the hard limits that apply in both modes: the image pixel limit, ffmpeg's protocol and format++// whitelists and duration cap, upload sniffing, the port-scan cap and the per-target throttle.++// TOOLS_GUARD=enforce refuses. The apps' older per-route limiters keep working in report mode and step++// aside in enforce mode (guard.enforcing), where the tiered quotas replace them.++// ═══════════════════════════════════════════════════════════════++++const fs = require('fs');++const path = require('path');++const limits = require('./limits');++const ip = require('./ip');++const tokens = require('./tokens');++const sniff = require('./sniff');++const ffmpeg = require('./ffmpeg');++const { createGuardStore, dayOf, DAY_MS } = require('./store');++const { createQuotas, setHeaders } = require('./quota');++const { createSemaphore, Busy } = require('./semaphore');++const { createTargetThrottle, createPortScanCap, normalizeTarget } = require('./targets');++const { createCallerResolver, jobOwnerResolver } = require('./caller');++const { NO_CHALLENGE, validChallenge } = require('./challenge');++++const TITLES = { 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found', 413: 'Payload Too Large', 415: 'Unsupported Media Type', 422: 'Unprocessable Content', 429: 'Too Many Requests', 503: 'Service Unavailable', 504: 'Gateway Timeout' };++++/** RFC 9457 problem+json in the contracts' shape (errors.problem@1), with the legacy { error } field. */++function sendProblem(req, res, status, code, detail, extra, contracts) {++ if (contracts && contracts.http) return contracts.http.sendProblem(res, status, code, { detail, extra, ctx: req.ov });++ const body = { type: `https://openvibe.network/problems/${code}`, title: TITLES[status] || 'Error', status, code };++ if (detail) body.detail = detail;++ if (req.ov) { body.request_id = req.ov.requestId; body.trace_id = req.ov.traceId; }++ body.error = detail || body.title;++ Object.assign(body, extra);++ res.statusCode = status;++ res.setHeader('Content-Type', 'application/problem+json');++ res.end(JSON.stringify(body));++ return body;++}++++/** Every uploaded file on a request (multer single, array or fields). */++function uploadsOf(req) {++ if (Array.isArray(req.files)) return req.files;++ if (req.files && typeof req.files === 'object') return Object.values(req.files).flat();++ return req.file ? [req.file] : [];++}++++/** Delete the files multer wrote to disk for a request that is refused. */++function discardUploads(req) {++ for (const f of uploadsOf(req)) if (f.path) fs.unlink(f.path, () => {});++}++++/**++ * @param {object} o++ * @param {string} o.app 'gateway' | 'img' | … (logs, metrics)++ * @param {string} [o.dataDir] guard.db goes here (with o.Database)++ * @param {Function} [o.Database] require('better-sqlite3'); without it everything stays in memory++ * @param {object} [o.contracts] openvibe-contracts, when the app has it++ * @param {object[]} [o.specs] the app's descriptor specs (server/descriptors.js SPECS)++ * @param {object} [o.keys] { get(), ensure() } — or networkUrl/networkInternalUrl/publicKeyFiles++ * @param {string} o.issuer the Network issuer URL++ * @param {object} [o.challenge] a challenge provider (./challenge.js); default: none++ * @param {object} [o.env] [o.log] [o.now] [o.secureCookie]++ */++function createGuard(o = {}) {++ const env = o.env || process.env;++ const log = o.log || console;++ const now = o.now || Date.now;++ const appName = o.app || 'tools';++ const mode = String(env.TOOLS_GUARD || 'report').trim().toLowerCase() === 'enforce' ? 'enforce' : 'report';++ const enforcing = mode === 'enforce';++ const bounds = limits.bounds(env);++ const table = limits.quotas(env, log);++ const store = createGuardStore({ Database: o.Database, dataDir: o.dataDir, now, log });++++ let saltDay = null, saltValue = null;++ const salt = () => {++ const d = dayOf(now());++ if (d !== saltDay) { saltValue = store.salt(d); saltDay = d; }++ return saltValue;++ };++++ const keys = o.keys || tokens.createKeySource({ networkUrl: o.networkUrl, networkInternalUrl: o.networkInternalUrl, files: o.publicKeyFiles || [], log });++ const issuer = o.issuer;++ const audience = env.OV_TOOLS_AUDIENCE || tokens.AUDIENCE;++ const callers = createCallerResolver({ getPublicKey: () => keys.get(), issuer, audience, contracts: o.contracts, salt, secureCookie: o.secureCookie });++ const quotas = createQuotas({ store, quotas: () => table, now });++ const targets = createTargetThrottle({ now });++ const ports = createPortScanCap({ ...bounds.ports, now });++ const sync = createSemaphore({ max: bounds.sync.concurrency, queue: bounds.sync.queue, waitMs: bounds.sync.waitMs });++ const challenge = validChallenge(o.challenge) ? o.challenge : NO_CHALLENGE;++++ // ── Descriptors ──────────────────────────────────────────++ const specs = new Map((o.specs || []).map(s => [s.id, s]));++ const tool = (id) => (id && specs.get(id)) || null;++ /** The descriptor of a job run: the host's own tool when it runs this operation, else the tool named after it, else the first. */++ function toolForJob(type, operation, preferId) {++ const pref = tool(preferId);++ if (pref && pref.job && pref.job.type === type && (!operation || pref.job.operation === operation)) return pref;++ let first = null;++ for (const s of specs.values()) {++ if (!s.job || s.job.type !== type || (operation && s.job.operation !== operation)) continue;++ if (s.id === operation) return s;++ if (!first) first = s;++ }++ return first;++ }++ const label = (id) => (id && specs.has(id) ? id : id ? 'other' : '-');++++ // ── Metrics and the log ──────────────────────────────────++ let refused = null;++ function attachMetrics(registry) {++ if (!registry || refused) return;++ refused = registry.counter({ name: 'tools_guard_refused_total', help: 'Requests the guard refused, or in report mode would have refused, by reason and tool', labelNames: ['reason', 'tool'], maxSeries: 2000 });++ registry.gauge({ name: 'tools_guard_enforcing', help: '1 when TOOLS_GUARD=enforce, 0 in report mode (hard limits refuse in both)', collect: () => [{ labels: {}, value: enforcing ? 1 : 0 }] });++ registry.gauge({++ name: 'tools_guard_sync', help: 'Synchronous heavy calls running and waiting, and the limit', labelNames: ['kind'],++ collect: () => { const s = sync.stats(); return [{ labels: { kind: 'running' }, value: s.active }, { labels: { kind: 'waiting' }, value: s.waiting }, { labels: { kind: 'limit' }, value: s.max }]; },++ });++ }++ const said = new Map();++ function record(req, c, { reason, tool: toolId, enforced }) {++ try {++ store.logAbuse({ at: now(), ipHash: c ? c.ipHash : callers.hashIp(req.ip), principal: (c && c.id) || '', tool: toolId || '', reason, enforced });++ } catch (err) { log.error(`[Guard] ${appName}: abuse log write failed: ${err.message}`); }++ if (refused) refused.inc({ reason, tool: label(toolId) });++ const k = `${reason}|${toolId || ''}|${enforced}`;++ const t = now();++ if (!said.has(k) || t - said.get(k) > 60_000) {++ said.set(k, t);++ if (said.size > 1000) said.clear();++ log.warn(`[Guard] ${appName}: ${enforced ? 'refused' : 'would refuse (report mode)'} ${reason}${toolId ? ` on ${toolId}` : ''} (${c ? c.tier : 'anonymous'})`);++ }++ }++++ // ── Callers ──────────────────────────────────────────────++ /** The request's caller, resolved once (before any cookie is minted during the request). */++ function caller(req, res) {++ if (!req._ovCaller) req._ovCaller = callers.resolve(req, res);++ return req._ovCaller;++ }++++ /** Attach req.user for a valid Network sign-in (aud openvibe.tools); a principal's token is left to the resolver. */++ function identify(req, _res, next) {++ const { token } = tokens.extractToken(req);++ if (!token || tokens.looksLikePrincipal(tokens.peek(token))) return next();++ const verify = () => {++ const claims = tokens.verifyUserToken(token, { publicKey: keys.get(), issuer, audience, now: now() });++ if (claims) { req.user = claims; req.token = token; }++ };++ if (keys.get() || typeof keys.ensure !== 'function') { verify(); return next(); }++ Promise.resolve(keys.ensure()).then(verify, () => {}).finally(() => next());++ }++++ // ── Refusals ─────────────────────────────────────────────++ /**++ * Refuse (enforce mode, or a hard limit) or only record it (report mode).++ * → true when the answer was sent, false when the request may go on.++ */++ function refuse(req, res, r) {++ const c = caller(req, res);++ const enforced = !!r.hard || enforcing;++ record(req, c, { reason: r.reason, tool: r.tool, enforced });++ if (!enforced) return false;++ if (r.discard !== false) discardUploads(req);++ if (r.retryAfter) res.setHeader('Retry-After', String(Math.max(1, Math.ceil(r.retryAfter))));++ sendProblem(req, res, r.status, r.code, r.detail, r.extra, o.contracts);++ return true;++ }++++ // ── Quotas ───────────────────────────────────────────────++ async function charge(req, res, { quotaClass, cost, tool: toolId }) {++ const c = caller(req, res);++ const r = quotas.check(c, { quotaClass, cost }, { report: !enforcing });++ // In report mode an older limiter that ran on this request (req.rateLimit) is the one in force:++ // its RateLimit-* headers stay. Otherwise the guard's describe the allowance closest to running out.++ if (enforcing || !req.rateLimit) setHeaders(res, r);++ if (r.ok) return true;++ if (enforcing && (c.tier === 'anonymous' || c.tier === 'session') && challenge.required(req, c, { reason: 'quota', tool: toolId })) {++ if (await challenge.verify(req, c)) return true;++ record(req, c, { reason: 'challenge', tool: toolId, enforced: true });++ discardUploads(req);++ sendProblem(req, res, 403, 'tools.challenge.required', 'Show that you are a person to go on.', { challenge: challenge.name }, o.contracts);++ return false;++ }++ const detail = r.binding === 'day'++ ? `You have used today's allowance for ${quotaClass.replace(/^tools-/, '')} requests; it renews at midnight UTC.`++ : 'Too many requests; slow down a little.';++ return !refuse(req, res, { status: 429, code: 'tools.quota.exceeded', detail, reason: 'quota', tool: toolId, retryAfter: r.retryAfter, extra: { quota_class: quotaClass, scope: r.scope, tier: c.tier, retry_after: r.retryAfter } });++ }++++ /** Middleware: charge a fixed class and cost (tools-api for every /api/ request). */++ function quota({ quotaClass, cost = 1, skip = null }) {++ return (req, res, next) => {++ if (skip && skip(req)) return next();++ charge(req, res, { quotaClass, cost }).then(ok => { if (ok) next(); }, next);++ };++ }++ // Liveness and readiness probes (the gateway polls every satellite's) are never counted.++ const PROBES = new Set(['/api/health', '/api/ready']);++ const isProbe = (req) => req.method === 'GET' && PROBES.has(`${req.baseUrl || ''}${req.path}`);++++ /**++ * Middleware: charge the tool's own class and cost (its descriptor). toolOf(req) → a tool id. Records++ * what was taken on req._ovCharged, so admit()/admitJob() can settle the difference once the real++ * tool is known (a multipart body names it only after the upload).++ */++ function toolQuota(toolOf) {++ return (req, res, next) => {++ const d = tool(typeof toolOf === 'function' ? toolOf(req) : toolOf);++ if (!d) return next();++ req._ovCharged = { quotaClass: d.quotaClass, cost: d.cost || 1, tool: d.id };++ charge(req, res, { quotaClass: d.quotaClass, cost: d.cost || 1, tool: d.id }).then(ok => { if (ok) next(); }, next);++ };++ }++++ /** Settle a pre-charge against the real tool: charge the difference or give it back. → ok */++ async function settle(req, res, d) {++ const was = req._ovCharged;++ const cost = d.cost || 1;++ if (!was || was.quotaClass !== d.quotaClass) {++ req._ovCharged = { quotaClass: d.quotaClass, cost, tool: d.id };++ return charge(req, res, { quotaClass: d.quotaClass, cost, tool: d.id });++ }++ if (cost > was.cost) { req._ovCharged = { ...was, cost }; return charge(req, res, { quotaClass: d.quotaClass, cost: cost - was.cost, tool: d.id }); }++ if (cost < was.cost) { quotas.refund(caller(req, res), { quotaClass: d.quotaClass, cost: was.cost - cost }); req._ovCharged = { ...was, cost }; }++ return true;++ }++++ // ── Who may run it ───────────────────────────────────────++ /** Heavy tools (descriptor auth.anonymous false) need a session that existed before this request, a sign-in or a token. → ok */++ function sessionRule(req, res, d) {++ if (!d || !d.auth || d.auth.anonymous !== false) return true;++ const c = caller(req, res);++ if (c.kind !== 'anonymous') return true;++ return !refuse(req, res, { status: 401, code: 'tools.session_required', detail: 'This tool needs a browser session (open its page first), a sign-in or an API token.', reason: 'session', tool: d.id });++ }++++ /** A principal must hold the tool's capability (tools.net.probe for probes). People use the pages. → ok */++ function capabilityRule(req, res, d) {++ if (!d || !d.auth) return true;++ const c = caller(req, res);++ if (c.kind !== 'principal') return true;++ if (tokens.hasCapability(c.claims, d.auth.capability, o.contracts)) return true;++ return !refuse(req, res, { status: 403, code: 'capability.denied', detail: `${d.auth.capability} not granted`, reason: 'capability', tool: d.id });++ }++++ // ── Uploads ──────────────────────────────────────────────++ /**++ * The bytes of every file against `accept` (hard limit). Fixes each file's type (and a misleading++ * extension) to what the bytes are. files: multer files ({ buffer|path, originalname, mimetype })++ * or job files ({ buffer|path, name, mime }). → ok++ */++ function checkFiles(req, res, files, accept, toolId) {++ if (!Array.isArray(accept) || !accept.length) return true;++ for (let i = 0; i < files.length; i++) {++ const f = files[i];++ let found = null;++ try { found = sniff.detect(sniff.head(f)); } catch { found = null; }++ const name = f.originalname != null ? f.originalname : f.name;++ if (!sniff.accepts(accept, found)) {++ const what = found ? `a ${found.ext.toUpperCase()} file` : 'not a file type this tool knows';++ refuse(req, res, { status: 415, code: 'tools.file.unsupported_type', hard: true, reason: 'sniff', tool: toolId, detail: `${files.length > 1 ? `File ${i + 1}` : 'The file'} is ${what}; this tool takes ${sniff.describe(accept)}.`, extra: { detected: found ? found.mime : null } });++ return false;++ }++ if ('mimetype' in f) f.mimetype = found.mime; else f.mime = found.mime;++ if (!sniff.extFits(name, found)) {++ const base = String(name || 'file').replace(/\.[A-Za-z0-9]{1,8}$/, '') || 'file';++ if (f.originalname != null) f.originalname = `${base}.${found.ext}`; else f.name = `${base}.${found.ext}`;++ if (f.path && !sniff.extFits(f.path, found)) {++ const to = `${f.path.replace(/\.[A-Za-z0-9]{1,8}$/, '')}.${found.ext}`;++ try { fs.renameSync(f.path, to); f.path = to; if (f.filename) f.filename = path.basename(to); } catch { /* keep the old name */ }++ }++ }++ }++ return true;++ }++++ /**++ * After the upload middleware: the real tool (toolOf(req) once the body is parsed) — its session rule,++ * the bytes against its files.accept, and the quota difference. → middleware++ */++ function admitUpload(toolOf) {++ return (req, res, next) => {++ const d = tool(toolOf(req));++ if (!d) return next();++ if (!sessionRule(req, res, d)) return;++ if (!checkFiles(req, res, uploadsOf(req), d.files && d.files.accept, d.id)) return;++ settle(req, res, d).then(ok => { if (ok) next(); }, next);++ };++ }++++ /** The job routes' admission hook (apps/_shared/jobs/http.js o.admit): the same checks for a job submit. → ok */++ async function admitJob(req, res, { type, input, files, toolId }) {++ const d = tool(toolId) || toolForJob(type, input && input.tool, null);++ if (!d) return true;++ if (!sessionRule(req, res, d)) return false;++ if (!checkFiles(req, res, files, d.files && d.files.accept, d.id)) return false;++ return settle(req, res, d);++ }++++ // ── Heavy work ───────────────────────────────────────────++ /** Middleware: hold one of the synchronous heavy slots until the answer is sent. */++ function heavy(toolOf) {++ return (req, res, next) => {++ sync.acquire().then((release) => {++ let done = false;++ const free = () => { if (!done) { done = true; release(); } };++ res.on('finish', free);++ res.on('close', free);++ next();++ }, (err) => {++ if (!(err instanceof Busy)) return next(err);++ const toolId = typeof toolOf === 'function' ? toolOf(req) : toolOf;++ if (refuse(req, res, { status: 503, code: 'tools.busy', detail: 'The server is busy with other files right now; try again in a moment.', reason: `busy.${err.reason === 'queue' ? 'sync' : 'wait'}`, tool: toolId, retryAfter: 10 })) return;++ next(); // report mode: runs without a slot++ });++ };++ }++++ /** The job store is over its queue or disk bound (system.busy()): refuse or record. → ok */++ function jobsBusy(req, res, busy, toolId) {++ if (!busy) return true;++ return !refuse(req, res, { status: 503, code: 'tools.busy', detail: busy.detail, reason: `busy.${busy.reason}`, tool: toolId, retryAfter: busy.retryAfter || 30 });++ }++++ // ── Egress ───────────────────────────────────────────────++ /** Per-target throttle (hard). → ok */++ function target(req, res, { tool: toolId, target: t, perMinute }) {++ const d = tool(toolId);++ const per = perMinute || (d && d.limits && d.limits.perTargetPerMinute);++ const r = targets.take(t, per);++ if (r.ok) return true;++ refuse(req, res, { status: 429, code: 'tools.quota.exceeded', hard: true, reason: 'target', tool: toolId, retryAfter: r.retryAfter, detail: `${r.target} was checked too often in the last minute; try again in ${r.retryAfter} s.`, extra: { scope: 'target', retry_after: r.retryAfter } });++ return false;++ }++++ /** The port checker's caps (hard). Browser sessions count by address, so a new cookie is no new allowance. → ok */++ function portScan(req, res, { tool: toolId, target: t, ports: list }) {++ const c = caller(req, res);++ const key = c.tier === 'anonymous' || c.tier === 'session' ? c.ipKey : c.key;++ const r = ports.take(key, t, list);++ if (r.ok) return true;++ if (r.reason === 'ports.per_request') {++ refuse(req, res, { status: 400, code: 'tools.run.invalid', hard: true, reason: 'ports', tool: toolId, detail: r.detail });++ } else {++ refuse(req, res, { status: 429, code: 'tools.quota.exceeded', hard: true, reason: 'ports', tool: toolId, retryAfter: r.retryAfter, detail: `${r.detail}: the port checker is for your own servers, not for scanning.`, extra: { scope: 'ports', retry_after: r.retryAfter } });++ }++ return false;++ }++++ // ── Older limiters, and tool errors ──────────────────────++ /**++ * One of the apps' older express-rate-limit limiters, keyed by the resolved caller: a signed-in++ * person or a principal by who they are (their higher number applies — these limiters used to run++ * before sign-in was read), everyone else by address (IPv6 /64, hashed). Skipped in enforce mode.++ * @param {Function} rateLimit require('express-rate-limit')++ */++ function legacyLimiter(rateLimit, { windowMs, anonymous, signedIn = anonymous, message, headers = true }) {++ const signed = (c) => c.tier === 'user' || c.tier === 'service';++ return rateLimit({++ windowMs,++ max: (req) => (signed(caller(req)) ? signedIn : anonymous),++ standardHeaders: headers,++ legacyHeaders: false,++ keyGenerator: (req) => { const c = caller(req); return signed(c) || c.tier === 'sandbox' ? c.key : c.ipKey; },++ skip: () => enforcing,++ message: { error: message },++ });++ }++ /** The three limiters an upload app had (LEGACY numbers in ./limits.js). */++ function legacyLimiters(rateLimit, numbers) {++ const n = numbers || limits.LEGACY[appName];++ return {++ apiLimiter: legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: n.api[0], signedIn: n.api[1], message: 'Too many requests. Please try again later.' }),++ processLimiter: legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: n.process[0], signedIn: n.process[1], message: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' }),++ burstLimiter: legacyLimiter(rateLimit, { windowMs: 5_000, anonymous: n.burst, signedIn: n.burst, headers: false, message: 'Too many requests in quick succession. Please slow down.' }),++ };++ }++++ /**++ * A tool refused its input on a hard limit (err.guardReason: 'pixels', 'ffmpeg.format',++ * 'ffmpeg.duration', 'timeout'): record it and answer its problem. → true when answered.++ */++ function toolRefused(req, res, err, toolId) {++ if (!err || !err.guardReason || res.headersSent) return false;++ refuse(req, res, { status: err.status || 422, code: err.code || 'tools.input.invalid', hard: true, reason: err.guardReason, tool: toolId, detail: err.message });++ return true;++ }++++ // ── Upkeep ───────────────────────────────────────────────++ function prune() {++ try {++ const cutoff = now() - bounds.abuseRetentionDays * DAY_MS;++ store.pruneAbuse(cutoff);++ store.pruneDays(dayOf(now() - DAY_MS));++ } catch (err) { log.error(`[Guard] ${appName}: prune failed: ${err.message}`); }++ }++ prune();++ const pruneTimer = o.pruneIntervalMs === 0 ? null : setInterval(prune, o.pruneIntervalMs || 6 * 60 * 60 * 1000);++ if (pruneTimer && pruneTimer.unref) pruneTimer.unref();++++ const api = {++ app: appName, mode, enforcing, bounds, table, store, keys, challenge,++ issuer, audience,++ identify, caller, resolveCaller: (req, res, opt) => callers.resolve(req, res, opt),++ ensureSession: (req, res) => callers.ensureSession(req, res),++ ownerResolver: () => jobOwnerResolver(callers),++ hashIp: (addr) => callers.hashIp(addr),++ tool, toolForJob,++ refuse, record, sendProblem: (req, res, status, code, detail, extra) => sendProblem(req, res, status, code, detail, extra, o.contracts),++ quota, apiQuota: quota({ quotaClass: 'tools-api', cost: 1, skip: isProbe }), toolQuota, charge, settle, quotas,++ sessionRule, capabilityRule,++ checkFiles, admitUpload, admitJob, uploadsOf, discardUploads,++ heavy, sync, jobsBusy,++ target, portScan, targets, ports, normalizeTarget,++ legacyLimiter, legacyLimiters, toolRefused,++ attachMetrics, prune,++ close() { if (pruneTimer) clearInterval(pruneTimer); if (keys.stop) keys.stop(); store.close(); },++ };++ return api;++}++++module.exports = { createGuard, sendProblem, uploadsOf, discardUploads, TRUST_PROXY: ip.TRUST_PROXY, limits, ip, tokens, sniff, ffmpeg, NO_CHALLENGE };nnew file mode 100644@@@ -0,0 +1,63 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// Client addresses (decision 5).++//++// The host's nginx applies Cloudflare's real IP (real_ip_header CF-Connecting-IP, Cloudflare ranges++// only) and every proxied location sets X-Forwarded-For to $remote_addr. So there is exactly one++// proxy hop, and it is on loopback: `trust proxy` is TRUST_PROXY below in every app, req.ip is the++// only address anyone reads, and a client's own X-Forwarded-For is never believed. A first-party++// process that calls a Tools app on loopback on behalf of a visitor (the gateway's proxy to a++// satellite, food → maps) is that one hop too, and forwards the visitor's address the same way.++// An app listening beyond loopback (maps binds 0.0.0.0 by default) believes nobody else's header.++// ═══════════════════════════════════════════════════════════════++++const net = require('net');++++/** '::ffff:1.2.3.4' → '1.2.3.4'; zone ids dropped; lower case. */++function normalizeIp(ip) {++ let s = String(ip || '').trim().toLowerCase();++ const pct = s.indexOf('%');++ if (pct >= 0) s = s.slice(0, pct);++ const mapped = /^::ffff:(\d+\.\d+\.\d+\.\d+)$/.exec(s);++ if (mapped) s = mapped[1];++ return s;++}++++function isLoopback(ip) {++ const s = normalizeIp(ip);++ return s === '::1' || /^127\./.test(s);++}++++/** An IPv6 address → its eight groups, expanded. */++function expand6(ip) {++ let s = ip;++ const v4 = /(\d+\.\d+\.\d+\.\d+)$/.exec(s);++ if (v4) {++ const n = v4[1].split('.').reduce((a, o) => a * 256 + Number(o), 0) >>> 0;++ s = s.slice(0, -v4[1].length) + `${(n >>> 16).toString(16)}:${(n & 0xffff).toString(16)}`;++ }++ const [head, tail] = s.split('::');++ const left = head ? head.split(':') : [];++ const right = tail !== undefined && tail ? tail.split(':') : [];++ const groups = tail !== undefined ? [...left, ...Array(Math.max(0, 8 - left.length - right.length)).fill('0'), ...right] : left;++ return groups.map(g => g.padStart(4, '0'));++}++++/**++ * The unit an address is counted as: an IPv4 address itself, an IPv6 address by its /64 (one++ * customer's network: a host can pick any of its 2^64 addresses). Not an IP → '' (never throws).++ */++function ipBucket(ip) {++ const s = normalizeIp(ip);++ const v = net.isIP(s);++ if (v === 4) return s;++ if (v === 6) return `${expand6(s).slice(0, 4).join(':')}::/64`;++ return '';++}++++/** Express `trust proxy`: exactly one hop, and only when that hop is on loopback. */++function TRUST_PROXY(addr, i) {++ return i === 0 && isLoopback(addr);++}++++module.exports = { normalizeIp, isLoopback, ipBucket, expand6, TRUST_PROXY };nnew file mode 100644@@@ -0,0 +1,131 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// The guard's numbers, in one place. Everything per tool comes from its descriptor (tools.tool@1:++// cost, quotaClass, limits.perTargetPerMinute, limits.maxDurationSec, limits.timeoutMs, files.accept);++// everything per caller tier and every host-wide bound is here. Pure data, no dependencies.++//++// Quotas count weight, not requests: a run costs its descriptor's `cost` (1 = a cheap lookup, 5 an++// image conversion, 10 an audio effect, 50 a video download). Each quota class × caller tier has++// perMinute a token bucket refilled at this many units a minute (kept in memory)++// burst the bucket's size: what may be spent at once after a quiet spell++// perDay units per UTC day, persisted in the app's guard.db (0 = no day allowance)++// Tiers: anonymous (keyed by IP, IPv6 by /64) < session (the ov_tools_jobs browser cookie) < user++// (a Network sign-in) < service (a first-party service or a developer app's production token).++// sandbox is a developer app's sandbox token: the smallest allowance, keyed by the app.++//++// The anonymous numbers start from the per-app limits the satellites had (img 10 conversions a++// minute, audio 6, docs 10, net/dev 60 lookups, maps 30 calls, yt 5 downloads an hour); signed-in++// and service tiers get multiples. Normal use of a page stays well inside them.++//++// Class names describe the work, never a tier or a price (decision 6): a paid tier later adds a++// column, not a class. tools-api is the guard's own class for every /api/ request (cost 1).++//++// Overrides (one /etc/openvibe/tools.env for every unit):++// TOOLS_GUARD_LIMITS='{"tools-job":{"anonymous":{"perDay":800}}}' merged into QUOTAS++// and the TOOLS_* variables named below for the host-wide bounds.++// ═══════════════════════════════════════════════════════════════++++const TIERS = ['anonymous', 'session', 'user', 'service', 'sandbox'];++++const q = (perMinute, burst, perDay) => ({ perMinute, burst, perDay });++++const QUOTAS = {++ // Every /api/ request of an app (registry reads, job polls, page context…).++ 'tools-api': {++ anonymous: q(120, 60, 0), session: q(180, 90, 0), user: q(480, 240, 0), service: q(2400, 1200, 0), sandbox: q(60, 30, 0),++ },++ // Pure transforms (text and dev engines): cheap, run inline.++ 'tools-run': {++ anonymous: q(120, 60, 5000), session: q(180, 90, 10000), user: q(480, 240, 50000), service: q(2400, 1200, 500000), sandbox: q(60, 30, 1000),++ },++ // Lookups that reach a host the caller chose (DNS, WHOIS, headers, Open Graph…); also throttled per target.++ 'tools-fetch': {++ anonymous: q(60, 30, 2000), session: q(90, 45, 4000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 15, 500),++ },++ // Network probes (port, ping, latency): traffic to the target itself; also throttled per target.++ 'tools-probe': {++ anonymous: q(30, 15, 600), session: q(45, 25, 1000), user: q(120, 60, 5000), service: q(600, 300, 50000), sandbox: q(15, 10, 200),++ },++ // Uploads processed by sharp, ffmpeg, pdf-lib, qpdf, poppler (sync /api/process and job submits).++ 'tools-job': {++ anonymous: q(60, 30, 1500), session: q(90, 45, 3000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 15, 300),++ },++ // The map and food finder pages' own lookups (OpenStreetMap, weather), cost 2 a call.++ 'tools-map': {++ anonymous: q(60, 40, 6000), session: q(90, 60, 8000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 20, 500),++ },++ // Server downloads (yt: cost 50 each, so anonymous = two at once, one a minute, 30 a day).++ 'tools-download': {++ anonymous: q(50, 100, 1500), session: q(75, 150, 2500), user: q(150, 300, 5000), service: q(300, 600, 10000), sandbox: q(25, 50, 100),++ },++};++++// All browser sessions from one address together may use this many times one session's allowance:++// dropping the cookie starts a new session, never a new allowance.++const SESSION_IP_SHARE = 3;++++// The apps' older per-route limiters (express-rate-limit, requests per window per address, or per++// person when signed in). They stay in force in report mode and step aside with TOOLS_GUARD=enforce,++// where the tiered quotas above take over. [anonymous or session, signed in or service].++const LEGACY = {++ img: { api: [60, 120], process: [10, 30], burst: 4 },++ audio: { api: [60, 120], process: [6, 20], burst: 3 },++ docs: { api: [60, 120], process: [10, 30], burst: 4 },++};++++function envInt(name, fallback, env = process.env) {++ const v = parseInt(env[name], 10);++ return Number.isFinite(v) && v >= 0 ? v : fallback;++}++++/** The host-wide bounds, read from the environment at call time (tests set it per process). */++function bounds(env = process.env) {++ return {++ // Synchronous heavy calls (img, audio, docs /api/process…) running at once per satellite, and++ // how many may wait for a slot, and for how long, before 503 tools.busy.++ sync: {++ concurrency: Math.max(1, envInt('TOOLS_SYNC_CONCURRENCY', 2, env)),++ queue: envInt('TOOLS_SYNC_QUEUE', 8, env),++ waitMs: envInt('TOOLS_SYNC_WAIT_MS', 30_000, env),++ },++ // Every satellite's job store: queued jobs (all owners) and bytes under <dataDir>/jobs.++ jobs: {++ maxQueued: envInt('TOOLS_JOBS_MAX_QUEUED', 200, env),++ diskBudgetBytes: envInt('TOOLS_DISK_BUDGET_MB', 8192, env) * 1024 * 1024,++ },++ // Largest image input decoded by sharp (width × height); OpenVibe.Media uses the same value.++ maxInputPixels: envInt('TOOLS_MAX_INPUT_PIXELS', 40_000_000, env),++ // The port checker: ports per request, and per caller in a rolling window.++ ports: {++ perRequest: 20,++ windowMs: 10 * 60_000,++ perCaller: envInt('TOOLS_PORTS_PER_CALLER', 100, env),++ targetsPerCaller: envInt('TOOLS_PORT_TARGETS_PER_CALLER', 10, env),++ },++ // Webhook request bins (in memory, one hour): per owner, per address, in all.++ webhook: { perOwner: 5, perIp: 10, total: 500 },++ // The abuse log (guard.db) keeps rows this long.++ abuseRetentionDays: 30,++ };++}++++function merge(base, over) {++ const out = JSON.parse(JSON.stringify(base));++ for (const [cls, tiers] of Object.entries(over || {})) {++ if (!out[cls]) out[cls] = {};++ for (const [tier, v] of Object.entries(tiers || {})) out[cls][tier] = { ...(out[cls][tier] || q(0, 0, 0)), ...v };++ }++ return out;++}++++/** QUOTAS with TOOLS_GUARD_LIMITS merged in (a bad value is reported and ignored). */++function quotas(env = process.env, log = console) {++ const raw = env.TOOLS_GUARD_LIMITS;++ if (!raw) return QUOTAS;++ try { return merge(QUOTAS, JSON.parse(raw)); } catch (err) {++ log.warn(`[Guard] TOOLS_GUARD_LIMITS is not valid JSON (${err.message}); using the defaults`);++ return QUOTAS;++ }++}++++module.exports = { TIERS, QUOTAS, SESSION_IP_SHARE, LEGACY, bounds, quotas, merge };nnew file mode 100644@@@ -0,0 +1,143 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// Tiered quotas: a token bucket per (key, class) in memory, and a UTC-day allowance per (key, class)++// in guard.db. A run takes its descriptor's `cost` from both, all or nothing.++//++// Who is charged (keys come from the caller resolver, addresses already hashed):++// anonymous its address (IPv6: its /64) at the anonymous tier++// session the session at the session tier, AND every session from that address together at++// SESSION_IP_SHARE × the session tier — so dropping the cookie never resets anything++// user the person; service / sandbox: the principal++//++// check() → { ok, limit, remaining, reset, retryAfter, scope, binding }: `limit`/`remaining`/`reset`++// describe whichever allowance is closest to running out (the RateLimit-* headers), `retryAfter` the++// seconds until this run would fit. With { report: true } a refused run is still counted (the work++// happens), which keeps report mode's numbers honest.++// ═══════════════════════════════════════════════════════════════++++const { SESSION_IP_SHARE } = require('./limits');++const { dayOf, DAY_MS } = require('./store');++++/**++ * @param {object} o++ * @param {object} o.store guard store (dayUsed, dayAdd)++ * @param {() => object} o.quotas the QUOTAS table (limits.quotas())++ * @param {() => number} [o.now]++ * @param {number} [o.maxBuckets]++ */++function createQuotas(o) {++ const now = o.now || Date.now;++ const buckets = new Map(); // `${cls}|${key}` → { tokens, at }++ const maxBuckets = o.maxBuckets || 200_000;++ let lastSweep = now();++++ function sweep(t) {++ // Buckets that are full again carry no information: drop them.++ if (t - lastSweep < 60_000 && buckets.size < maxBuckets) return;++ lastSweep = t;++ for (const [k, b] of buckets) if (t - b.at > 10 * 60_000) buckets.delete(k);++ }++++ /** A bucket's tokens right now (refilled since it was last touched). */++ function level(id, lim, t) {++ const b = buckets.get(id);++ if (!b) return lim.burst;++ const rate = lim.perMinute / 60_000;++ return Math.min(lim.burst, b.tokens + (t - b.at) * rate);++ }++++ function chargesFor(caller, cls, table) {++ const tiers = table[cls];++ if (!tiers) return [];++ const tier = tiers[caller.tier] || tiers.anonymous;++ if (caller.tier === 'anonymous') return [{ key: caller.ipKey, lim: tier, scope: 'address' }];++ if (caller.tier === 'session') {++ const share = { perMinute: tier.perMinute * SESSION_IP_SHARE, burst: tier.burst * SESSION_IP_SHARE, perDay: tier.perDay * SESSION_IP_SHARE };++ return [{ key: caller.key, lim: tier, scope: 'session' }, { key: `sessions@${caller.ipKey}`, lim: share, scope: 'address' }];++ }++ return [{ key: caller.key, lim: tier, scope: caller.tier }];++ }++++ /**++ * @param {object} caller { tier, key, ipKey }++ * @param {object} q { quotaClass, cost }++ * @param {object} [opt] { report: charge even when refused }++ */++ function check(caller, { quotaClass, cost = 1 }, opt = {}) {++ const t = now();++ sweep(t);++ const table = o.quotas();++ const charges = chargesFor(caller, quotaClass, table);++ if (!charges.length) return { ok: true, limit: null, remaining: null, reset: null, retryAfter: 0 };++ const day = dayOf(t);++ const nextDay = Math.ceil((Math.floor(t / DAY_MS) + 1) * DAY_MS / 1000) - Math.floor(t / 1000);++ let ok = true, retryAfter = 0, binding = null, scope = null;++ const views = [];++ for (const c of charges) {++ const { lim } = c;++ if (!(lim.burst > 0) || !(lim.perMinute > 0)) continue;++ const need = Math.min(cost, lim.burst); // one run never costs more than a full bucket++ const tokens = level(`${quotaClass}|${c.key}`, lim, t);++ const rate = lim.perMinute / 60_000;++ const minuteOk = tokens >= need;++ const used = lim.perDay > 0 ? o.store.dayUsed(day, c.key, quotaClass) : 0;++ const dayOk = !(lim.perDay > 0) || used + cost <= lim.perDay;++ const after = Math.max(0, tokens - need);++ views.push({++ c, need, tokens, rate, dayUsed: used,++ minute: { limit: lim.burst, remaining: Math.floor(after), reset: Math.ceil((lim.burst - after) / rate / 1000) },++ day: lim.perDay > 0 ? { limit: lim.perDay, remaining: Math.max(0, Math.floor(lim.perDay - used - cost)), reset: nextDay } : null,++ });++ if (!minuteOk) {++ ok = false;++ const wait = Math.ceil((need - tokens) / rate / 1000);++ if (wait > retryAfter) { retryAfter = wait; binding = 'minute'; scope = c.scope; }++ }++ if (!dayOk) {++ ok = false;++ if (nextDay > retryAfter) { retryAfter = nextDay; binding = 'day'; scope = c.scope; }++ }++ }++ if (ok || opt.report) {++ for (const v of views) {++ buckets.set(`${quotaClass}|${v.c.key}`, { tokens: Math.max(0, v.tokens - v.need), at: t });++ if (v.c.lim.perDay > 0) o.store.dayAdd(day, v.c.key, quotaClass, cost);++ }++ }++ // The headers describe the allowance closest to running out, in units of this run's cost.++ let head = null;++ for (const v of views) {++ for (const w of [v.minute, v.day]) {++ if (!w) continue;++ const runsLeft = w.remaining / Math.max(1, cost);++ if (!head || runsLeft < head.runsLeft) head = { ...w, runsLeft };++ }++ }++ return {++ ok, retryAfter: ok ? 0 : Math.max(1, retryAfter), binding, scope,++ limit: head ? head.limit : null, remaining: head ? (ok ? head.remaining : 0) : null, reset: head ? head.reset : null,++ };++ }++++ /** Give tokens back (a run that turned out to cost less than was taken up front). Day units stay counted. */++ function refund(caller, { quotaClass, cost }) {++ if (!(cost > 0)) return;++ const t = now();++ for (const c of chargesFor(caller, quotaClass, o.quotas())) {++ const id = `${quotaClass}|${c.key}`;++ if (buckets.has(id)) buckets.set(id, { tokens: Math.min(c.lim.burst, level(id, c.lim, t) + cost), at: t });++ }++ }++++ return { check, refund, size: () => buckets.size };++}++++/** RateLimit-* headers (IETF draft) from a check() answer. */++function setHeaders(res, r) {++ if (!r || r.limit == null || res.headersSent) return;++ res.setHeader('RateLimit-Limit', String(r.limit));++ res.setHeader('RateLimit-Remaining', String(Math.max(0, r.remaining)));++ res.setHeader('RateLimit-Reset', String(Math.max(0, r.reset)));++}++++module.exports = { createQuotas, setHeaders };nnew file mode 100644@@@ -0,0 +1,61 @@++'use strict';++// A counting semaphore with a bounded wait list: at most `max` holders, at most `queue` waiting, each++// for at most `waitMs`. acquire() → a release function, or a Busy error ('queue' | 'timeout').++// createPacer spaces calls to an upstream that asks for it (below).++++class Busy extends Error {++ constructor(reason) { super(reason === 'queue' ? 'Too many requests are waiting' : 'Waited too long for a free slot'); this.reason = reason; }++}++++function createSemaphore({ max = 2, queue = 8, waitMs = 30_000 } = {}) {++ let active = 0;++ const waiting = [];++++ function release() {++ const next = waiting.shift();++ if (next) { clearTimeout(next.timer); next.resolve(once()); } else active = Math.max(0, active - 1);++ }++ function once() {++ let done = false;++ return () => { if (!done) { done = true; release(); } };++ }++++ function acquire() {++ if (active < max) { active++; return Promise.resolve(once()); }++ if (waiting.length >= queue) return Promise.reject(new Busy('queue'));++ return new Promise((resolve, reject) => {++ const w = { resolve, timer: null };++ w.timer = setTimeout(() => {++ const i = waiting.indexOf(w);++ if (i >= 0) waiting.splice(i, 1);++ reject(new Busy('timeout'));++ }, waitMs);++ waiting.push(w);++ });++ }++++ return { acquire, stats: () => ({ active, waiting: waiting.length, max, queue }) };++}++++/**++ * A pacer for an upstream with a usage policy (Nominatim: one request a second): run() starts each++ * call at least `minIntervalMs` after the previous one, with at most `queue` calls waiting; a call++ * that would wait longer than `maxWaitMs` is refused with Busy('queue') rather than piling up.++ */++function createPacer({ minIntervalMs = 1000, queue = 20, maxWaitMs = 15_000 } = {}) {++ let next = 0, waiting = 0;++ async function run(fn) {++ const t = Date.now();++ const at = Math.max(t, next);++ if (waiting >= queue || at - t > maxWaitMs) throw new Busy('queue');++ next = at + minIntervalMs;++ waiting++;++ try {++ if (at > t) await new Promise(r => setTimeout(r, at - t));++ } finally { waiting--; }++ return fn();++ }++ return { run, stats: () => ({ waiting }) };++}++++module.exports = { createSemaphore, createPacer, Busy };nnew file mode 100644@@@ -0,0 +1,169 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// What an upload really is, from its first bytes (never the client's Content-Type or file name), and++// whether a tool's descriptor accepts it (files.accept: media types, `type/*` allowed).++//++// detect(buf) → { mime, ext, also: [equivalent media types] } | null. Text formats a demuxer would++// follow (HLS playlists, ffconcat lists, SDP) are never detected as media, so they never pass.++// ═══════════════════════════════════════════════════════════════++++const fs = require('fs');++++const HEAD_BYTES = 4096;++++const EQUIV = {++ 'image/png': [], 'image/gif': [], 'image/webp': [], 'image/avif': [], 'image/tiff': [], 'image/svg+xml': [],++ 'image/jpeg': ['image/jpg', 'image/pjpeg'],++ 'image/bmp': ['image/x-bmp', 'image/x-ms-bmp'],++ 'image/x-icon': ['image/vnd.microsoft.icon', 'image/ico'],++ 'image/heic': ['image/heif', 'image/heic-sequence', 'image/heif-sequence'],++ 'image/heif': ['image/heic'],++ 'application/pdf': ['application/x-pdf'],++ 'audio/mpeg': ['audio/mp3', 'audio/mpeg3', 'audio/x-mpeg'],++ 'audio/wav': ['audio/x-wav', 'audio/wave', 'audio/vnd.wave'],++ 'audio/flac': ['audio/x-flac'],++ 'audio/ogg': ['application/ogg', 'audio/vorbis', 'video/ogg'],++ 'audio/opus': ['audio/ogg', 'application/ogg'],++ 'audio/aac': ['audio/x-aac', 'audio/aacp'],++ 'audio/mp4': ['audio/x-m4a', 'audio/m4a', 'video/mp4'],++ 'video/mp4': ['audio/mp4', 'audio/x-m4a'],++ 'video/quicktime': ['video/mp4'],++ 'video/3gpp': ['audio/3gpp', 'video/mp4'],++ 'audio/x-ms-wma': ['video/x-ms-asf', 'audio/x-ms-asf', 'video/x-ms-wmv'],++ 'audio/aiff': ['audio/x-aiff'],++ 'audio/ac3': ['audio/x-ac3'],++ 'audio/amr': [],++ 'video/webm': ['audio/webm', 'video/x-matroska'],++ 'video/x-matroska': ['video/webm', 'audio/webm', 'audio/x-matroska'],++ 'video/x-msvideo': ['video/avi', 'video/msvideo'],++ 'video/x-flv': [],++};++++const EXT = {++ 'image/png': 'png', 'image/jpeg': 'jpg', 'image/gif': 'gif', 'image/webp': 'webp', 'image/avif': 'avif', 'image/tiff': 'tiff',++ 'image/bmp': 'bmp', 'image/x-icon': 'ico', 'image/heic': 'heic', 'image/heif': 'heif', 'image/svg+xml': 'svg',++ 'application/pdf': 'pdf',++ 'audio/mpeg': 'mp3', 'audio/wav': 'wav', 'audio/flac': 'flac', 'audio/ogg': 'ogg', 'audio/opus': 'opus', 'audio/aac': 'aac',++ 'audio/mp4': 'm4a', 'video/mp4': 'mp4', 'video/quicktime': 'mov', 'video/3gpp': '3gp', 'audio/x-ms-wma': 'wma',++ 'audio/aiff': 'aiff', 'audio/ac3': 'ac3', 'audio/amr': 'amr', 'video/webm': 'webm', 'video/x-matroska': 'mkv',++ 'video/x-msvideo': 'avi', 'video/x-flv': 'flv',++};++++// File name extensions that plausibly go with each type. An upload whose name says something else++// (song.m3u8 holding MP3 bytes) is renamed to the canonical extension, because some tools derive the++// output format from the input's extension and ffmpeg takes extensions as hints.++const FITTING = {++ 'image/jpeg': ['jpg', 'jpeg', 'jpe', 'jfif'], 'image/tiff': ['tif', 'tiff'], 'image/bmp': ['bmp', 'dib'], 'image/x-icon': ['ico'],++ 'image/heic': ['heic', 'heif', 'hif'], 'image/heif': ['heif', 'heic', 'hif'],++ 'audio/mpeg': ['mp3', 'mpga', 'mp2'], 'audio/wav': ['wav', 'wave'], 'audio/ogg': ['ogg', 'oga', 'ogv', 'opus', 'spx'],++ 'audio/opus': ['opus', 'ogg', 'oga'], 'audio/aac': ['aac', 'adts'], 'audio/mp4': ['m4a', 'm4b', 'm4p', 'm4r', 'mp4'],++ 'video/mp4': ['mp4', 'm4a', 'm4v', 'm4b', 'm4r', 'mov'], 'video/quicktime': ['mov', 'qt', 'mp4', 'm4a'], 'video/3gpp': ['3gp', '3g2', '3gpp'],++ 'audio/x-ms-wma': ['wma', 'wmv', 'asf'], 'audio/aiff': ['aiff', 'aif', 'aifc'], 'video/webm': ['webm', 'weba', 'mkv'],++ 'video/x-matroska': ['mkv', 'mka', 'webm'],++};++++/** Is `name`'s extension one that fits the detected type? */++function extFits(name, detected) {++ const ext = String(name || '').toLowerCase().match(/\.([a-z0-9]{1,8})$/);++ if (!ext || !detected) return false;++ return (FITTING[detected.mime] || [detected.ext]).includes(ext[1]);++}++++const HEIC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);++++const at = (buf, off, str) => buf.length >= off + str.length && buf.toString('latin1', off, off + str.length) === str;++const found = (mime) => ({ mime, ext: EXT[mime], also: EQUIV[mime] || [] });++++function ftyp(buf) {++ const size = Math.min(buf.readUInt32BE(0) || 0, buf.length);++ const brands = [buf.toString('latin1', 8, 12)];++ for (let o = 16; o + 4 <= size; o += 4) brands.push(buf.toString('latin1', o, o + 4));++ if (brands.includes('avif') || brands.includes('avis')) return 'image/avif';++ if (brands.some(b => HEIC_BRANDS.has(b))) return 'image/heic';++ if (brands.includes('mif1') || brands.includes('msf1')) return 'image/heif';++ const major = brands[0];++ if (/^M4[ABP] $/.test(major)) return 'audio/mp4';++ if (major === 'qt ') return 'video/quicktime';++ if (/^3g[p2]/.test(major)) return 'video/3gpp';++ return 'video/mp4';++}++++/** Plausible MPEG audio frame header at `o` (layer I–III, a real bitrate and sample rate). */++function mpegFrame(buf, o = 0) {++ if (buf.length < o + 4 || buf[o] !== 0xff || (buf[o + 1] & 0xe0) !== 0xe0) return false;++ const version = (buf[o + 1] >> 3) & 3, layer = (buf[o + 1] >> 1) & 3, bitrate = buf[o + 2] >> 4, rate = (buf[o + 2] >> 2) & 3;++ return version !== 1 && layer !== 0 && bitrate !== 0 && bitrate !== 15 && rate !== 3;++}++++function isSvg(buf) {++ let s = buf.toString('utf8', 0, Math.min(buf.length, HEAD_BYTES)).replace(/^/, '').trimStart();++ // An XML declaration, doctype and comments may come before the root element.++ for (let i = 0; i < 20; i++) {++ const m = /^(<\?xml[\s\S]*?\?>|<!--[\s\S]*?-->|<!DOCTYPE[^>]*>)\s*/i.exec(s);++ if (!m) break;++ s = s.slice(m[0].length);++ }++ return /^<svg[\s>]/i.test(s);++}++++/** The media type of the bytes, or null. */++function detect(buf) {++ if (!Buffer.isBuffer(buf) || buf.length < 4) return null;++ if (at(buf, 0, '\x89PNG\r\n\x1a\n')) return found('image/png');++ if (buf[0] === 0xff && buf[1] === 0xd8 && buf[2] === 0xff) return found('image/jpeg');++ if (at(buf, 0, 'GIF87a') || at(buf, 0, 'GIF89a')) return found('image/gif');++ if (at(buf, 0, 'RIFF') && at(buf, 8, 'WEBP')) return found('image/webp');++ if (at(buf, 0, 'RIFF') && at(buf, 8, 'WAVE')) return found('audio/wav');++ if (at(buf, 0, 'RIFF') && at(buf, 8, 'AVI ')) return found('video/x-msvideo');++ if (at(buf, 0, 'BM') && buf.length >= 18 && [12, 40, 52, 56, 64, 108, 124].includes(buf.readUInt32LE(14))) return found('image/bmp');++ if (at(buf, 0, 'II*\0') || at(buf, 0, 'MM\0*') || at(buf, 0, 'II+\0') || at(buf, 0, 'MM\0+')) return found('image/tiff');++ if (buf.length >= 22 && buf.readUInt16LE(0) === 0 && buf.readUInt16LE(2) === 1 && buf.readUInt16LE(4) > 0 && buf[9] === 0) return found('image/x-icon');++ if (buf.length >= 12 && at(buf, 4, 'ftyp')) return found(ftyp(buf));++ if (buf.indexOf('%PDF-', 0, 'latin1') >= 0 && buf.indexOf('%PDF-', 0, 'latin1') < 1024) return found('application/pdf');++ if (at(buf, 0, 'fLaC')) return found('audio/flac');++ if (at(buf, 0, 'OggS')) return found(buf.indexOf('OpusHead', 0, 'latin1') > 0 && buf.indexOf('OpusHead', 0, 'latin1') < 100 ? 'audio/opus' : 'audio/ogg');++ if (at(buf, 0, 'ID3')) return found('audio/mpeg');++ if (buf[0] === 0xff && (buf[1] & 0xf6) === 0xf0) return found('audio/aac'); // ADTS: sync + layer 00++ if (mpegFrame(buf, 0)) return found('audio/mpeg');++ if (at(buf, 0, '\x30\x26\xb2\x75\x8e\x66\xcf\x11')) return found('audio/x-ms-wma');++ if (at(buf, 0, 'FORM') && (at(buf, 8, 'AIFF') || at(buf, 8, 'AIFC'))) return found('audio/aiff');++ if (buf[0] === 0x0b && buf[1] === 0x77) return found('audio/ac3');++ if (at(buf, 0, '#!AMR')) return found('audio/amr');++ if (at(buf, 0, '\x1a\x45\xdf\xa3')) return found(buf.indexOf('webm', 0, 'latin1') > 0 && buf.indexOf('webm', 0, 'latin1') < 64 ? 'video/webm' : 'video/x-matroska');++ if (at(buf, 0, 'FLV\x01')) return found('video/x-flv');++ if (isSvg(buf)) return found('image/svg+xml');++ return null;++}++++/** Does `accept` (media types, type/* allowed) cover what was detected? */++function accepts(accept, detected) {++ if (!detected) return false;++ const list = Array.isArray(accept) ? accept : [];++ return [detected.mime, ...detected.also].some(m => list.includes(m) || list.includes(`${m.split('/')[0]}/*`));++}++++/** An accept list for people: 'PNG, JPG, WEBP…' (aliases folded into the type they stand for). */++function describe(accept) {++ const out = [];++ for (const m of Array.isArray(accept) ? accept : []) {++ if (/octet-stream/.test(m)) continue;++ const canon = EXT[m] ? m : Object.keys(EQUIV).find(c => EQUIV[c].includes(m) && EXT[c]);++ const word = (canon ? EXT[canon] : m.split('/')[1].replace(/^(x-|vnd\.)/, '').replace(/\+xml$/, '')).toUpperCase();++ if (!out.includes(word)) out.push(word);++ }++ return out.join(', ');++}++++/** The first HEAD_BYTES of an upload (multer memory `buffer` or disk `path`). */++function head(file) {++ if (file.buffer) return file.buffer.subarray(0, HEAD_BYTES);++ if (!file.path) return Buffer.alloc(0);++ const fd = fs.openSync(file.path, 'r');++ try {++ const b = Buffer.alloc(HEAD_BYTES);++ const n = fs.readSync(fd, b, 0, HEAD_BYTES, 0);++ return b.subarray(0, n);++ } finally { fs.closeSync(fd); }++}++++module.exports = { detect, accepts, head, extFits, describe, EQUIV, EXT, HEAD_BYTES };nnew file mode 100644@@@ -0,0 +1,152 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// guard.db — the guard's own small SQLite file in each app's data directory (better-sqlite3 handle++// class passed in, like the job runtime). Three tables:++//++// guard_salt today's random salt (one row; the previous day's is deleted when the day turns). Every++// address the guard counts or logs is HMAC-SHA256(salt, address) — never the address.++// Kept on disk so a restart keeps today's counts; gone the next day, so older hashes++// can no longer be tied to anything (pseudonymous, ADR-021).++// guard_day quota units used per UTC day, key and quota class (the day allowances survive restarts;++// rows older than yesterday are pruned).++// guard_abuse the abuse log: time, hashed address, principal or user id, tool, reason, whether it was++// refused (enforce) or only reported; repeats of the same row within a minute add to++// `count`. Kept 30 days.++//++// Without a Database class (an app that has none) the same API is kept in memory.++// ═══════════════════════════════════════════════════════════════++++const crypto = require('crypto');++const fs = require('fs');++const path = require('path');++++const DAY_MS = 24 * 60 * 60 * 1000;++const dayOf = (ms) => new Date(ms).toISOString().slice(0, 10);++++const SCHEMA = `++CREATE TABLE IF NOT EXISTS guard_salt (++ day TEXT PRIMARY KEY,++ salt TEXT NOT NULL++);++CREATE TABLE IF NOT EXISTS guard_day (++ day TEXT NOT NULL,++ key TEXT NOT NULL,++ class TEXT NOT NULL,++ used REAL NOT NULL DEFAULT 0,++ PRIMARY KEY (day, key, class)++);++CREATE TABLE IF NOT EXISTS guard_abuse (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ at INTEGER NOT NULL,++ minute INTEGER NOT NULL,++ ip_hash TEXT NOT NULL DEFAULT '',++ principal TEXT NOT NULL DEFAULT '',++ tool TEXT NOT NULL DEFAULT '',++ reason TEXT NOT NULL,++ enforced INTEGER NOT NULL,++ count INTEGER NOT NULL DEFAULT 1++);++CREATE UNIQUE INDEX IF NOT EXISTS guard_abuse_once ON guard_abuse(minute, ip_hash, principal, tool, reason, enforced);++CREATE INDEX IF NOT EXISTS guard_abuse_at ON guard_abuse(at);++`;++++/**++ * @param {object} o++ * @param {Function} [o.Database] require('better-sqlite3'); omitted → in memory++ * @param {string} [o.dataDir]++ * @param {string} [o.file='guard.db']++ * @param {() => number} [o.now]++ */++function createGuardStore(o = {}) {++ const now = o.now || Date.now;++ if (!o.Database || !o.dataDir) return memoryStore(now);++ try { return sqliteStore(o, now); } catch (err) {++ // An unwritable data directory must not take the app down: quotas and the log stay in memory.++ (o.log || console).error(`[Guard] ${path.join(o.dataDir, o.file || 'guard.db')} cannot be opened (${err.message}); keeping guard state in memory`);++ return memoryStore(now);++ }++}++++function sqliteStore(o, now) {++ fs.mkdirSync(o.dataDir, { recursive: true });++ const db = new o.Database(path.join(o.dataDir, o.file || 'guard.db'));++ db.pragma('journal_mode = WAL');++ db.pragma('synchronous = NORMAL');++ db.pragma('busy_timeout = 5000');++ db.exec(SCHEMA);++ const st = {++ saltGet: db.prepare('SELECT salt FROM guard_salt WHERE day = ?'),++ saltPut: db.prepare('INSERT OR IGNORE INTO guard_salt (day, salt) VALUES (?, ?)'),++ saltDrop: db.prepare('DELETE FROM guard_salt WHERE day <> ?'),++ dayGet: db.prepare('SELECT used FROM guard_day WHERE day = ? AND key = ? AND class = ?'),++ dayAdd: db.prepare('INSERT INTO guard_day (day, key, class, used) VALUES (?, ?, ?, ?) ON CONFLICT(day, key, class) DO UPDATE SET used = used + excluded.used'),++ dayPrune: db.prepare('DELETE FROM guard_day WHERE day < ?'),++ abuse: db.prepare(`INSERT INTO guard_abuse (at, minute, ip_hash, principal, tool, reason, enforced) VALUES (?, ?, ?, ?, ?, ?, ?)++ ON CONFLICT(minute, ip_hash, principal, tool, reason, enforced) DO UPDATE SET count = count + 1, at = excluded.at`),++ abusePrune: db.prepare('DELETE FROM guard_abuse WHERE at < ?'),++ abuseAll: db.prepare('SELECT at, ip_hash, principal, tool, reason, enforced, count FROM guard_abuse ORDER BY id'),++ };++ const cache = new Map(); // `${day}|${key}|${cls}` → used (today only)++ let cacheDay = null;++ const saltTx = db.transaction((day) => {++ st.saltPut.run(day, crypto.randomBytes(32).toString('hex'));++ st.saltDrop.run(day);++ return st.saltGet.get(day).salt;++ });++ return {++ kind: 'sqlite',++ db,++ salt(day = dayOf(now())) {++ const row = st.saltGet.get(day);++ return row ? row.salt : saltTx(day);++ },++ dayUsed(day, key, cls) {++ if (cacheDay !== day) { cache.clear(); cacheDay = day; }++ const k = `${day}|${key}|${cls}`;++ if (!cache.has(k)) { const r = st.dayGet.get(day, key, cls); cache.set(k, r ? r.used : 0); }++ return cache.get(k);++ },++ dayAdd(day, key, cls, n) {++ if (!(n > 0)) return;++ const used = this.dayUsed(day, key, cls) + n;++ cache.set(`${day}|${key}|${cls}`, used);++ st.dayAdd.run(day, key, cls, n);++ },++ pruneDays(keepFromDay) { return st.dayPrune.run(keepFromDay).changes; },++ logAbuse(r) {++ const at = r.at || now();++ st.abuse.run(at, Math.floor(at / 60_000), r.ipHash || '', r.principal || '', r.tool || '', String(r.reason), r.enforced ? 1 : 0);++ },++ pruneAbuse(beforeMs) { return st.abusePrune.run(beforeMs).changes; },++ abuseRows() { return st.abuseAll.all(); },++ close() { try { db.close(); } catch { /* already closed */ } },++ };++}++++function memoryStore(now) {++ let salt = null;++ const days = new Map();++ const abuse = [];++ return {++ kind: 'memory',++ db: null,++ salt(day = dayOf(now())) {++ if (!salt || salt.day !== day) salt = { day, salt: crypto.randomBytes(32).toString('hex') };++ return salt.salt;++ },++ dayUsed(day, key, cls) { return days.get(`${day}|${key}|${cls}`) || 0; },++ dayAdd(day, key, cls, n) { if (n > 0) days.set(`${day}|${key}|${cls}`, this.dayUsed(day, key, cls) + n); },++ pruneDays(keepFromDay) { let n = 0; for (const k of days.keys()) if (k.slice(0, 10) < keepFromDay) { days.delete(k); n++; } return n; },++ logAbuse(r) {++ const at = r.at || now();++ const row = { at, minute: Math.floor(at / 60_000), ip_hash: r.ipHash || '', principal: r.principal || '', tool: r.tool || '', reason: String(r.reason), enforced: r.enforced ? 1 : 0, count: 1 };++ const same = abuse.find(x => x.minute === row.minute && x.ip_hash === row.ip_hash && x.principal === row.principal && x.tool === row.tool && x.reason === row.reason && x.enforced === row.enforced);++ if (same) { same.count++; same.at = at; } else { abuse.push(row); if (abuse.length > 10000) abuse.shift(); }++ },++ pruneAbuse(beforeMs) { const n = abuse.length; for (let i = abuse.length - 1; i >= 0; i--) if (abuse[i].at < beforeMs) abuse.splice(i, 1); return n - abuse.length; },++ abuseRows() { return abuse.map(({ minute, ...r }) => r); },++ close() {},++ };++}++++module.exports = { createGuardStore, dayOf, DAY_MS };nnew file mode 100644@@@ -0,0 +1,83 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// Egress bounds (hard limits: they apply in report mode too).++//++// Per-target throttle: one bucket per target host or address, across every caller and every tool.++// It holds one minute of work; a run of a tool costs 1 / its descriptor's limits.perTargetPerMinute of++// it, so headers (10 a minute) and SMTP (3 a minute) against one host share one budget, and nobody++// can point the network tools at a host faster than the strictest of them allows.++//++// Port-scan cap: the port checker takes at most `perRequest` ports a request, and one caller at most++// `perCaller` ports and `targetsPerCaller` distinct targets in a rolling window, so it can check a++// server's own ports but cannot sweep a range.++// ═══════════════════════════════════════════════════════════════++++const { ipBucket } = require('./ip');++++/** 'https://Example.COM:8443/x' → 'example.com'; an IPv6 address → its /64. '' when there is nothing. */++function normalizeTarget(raw) {++ let t = String(raw || '').trim().toLowerCase();++ if (!t) return '';++ t = t.replace(/^[a-z][a-z0-9+.-]*:\/\//, '');++ t = t.replace(/^[^@/]*@/, ''); // user:pass@++ const bracket = /^\[([0-9a-f:.]+)\]/.exec(t);++ if (bracket) t = bracket[1];++ else {++ t = t.replace(/[/?#].*$/, '');++ if ((t.match(/:/g) || []).length === 1) t = t.replace(/:\d*$/, '');++ }++ t = t.replace(/\.+$/, '');++ return ipBucket(t) || t.slice(0, 253);++}++++function createTargetThrottle({ now = Date.now } = {}) {++ const buckets = new Map(); // target → { units, at }++ const FULL = 1; // one minute of work++++ function take(target, perMinute) {++ const key = normalizeTarget(target);++ if (!key || !(perMinute > 0)) return { ok: true, retryAfter: 0, target: key };++ const t = now();++ if (buckets.size > 50_000) for (const [k, b] of buckets) if (t - b.at > 60_000) buckets.delete(k);++ const b = buckets.get(key);++ const units = b ? Math.min(FULL, b.units + (t - b.at) / 60_000) : FULL;++ const cost = 1 / perMinute;++ if (units + 1e-9 < cost) return { ok: false, retryAfter: Math.max(1, Math.ceil((cost - units) * 60)), target: key };++ buckets.set(key, { units: units - cost, at: t });++ return { ok: true, retryAfter: 0, target: key };++ }++++ return { take, size: () => buckets.size };++}++++function createPortScanCap({ perRequest = 20, windowMs = 10 * 60_000, perCaller = 100, targetsPerCaller = 10, now = Date.now } = {}) {++ const callers = new Map(); // caller key → [{ at, target, ports }]++++ /** → { ok, reason?, detail?, retryAfter? }; records the probe when it is allowed. */++ function take(callerKey, target, ports) {++ const n = Array.isArray(ports) ? ports.length : Number(ports) || 0;++ if (n > perRequest) return { ok: false, reason: 'ports.per_request', detail: `At most ${perRequest} ports per check` };++ const t = now();++ const key = normalizeTarget(target);++ const list = (callers.get(callerKey) || []).filter(e => t - e.at < windowMs);++ const used = list.reduce((s, e) => s + e.ports, 0);++ const targets = new Set(list.map(e => e.target));++ const retryAfter = list.length ? Math.max(1, Math.ceil((list[0].at + windowMs - t) / 1000)) : 1;++ if (used + n > perCaller) {++ callers.set(callerKey, list);++ return { ok: false, reason: 'ports.per_caller', detail: `At most ${perCaller} ports every ${Math.round(windowMs / 60_000)} minutes`, retryAfter };++ }++ if (!targets.has(key) && targets.size >= targetsPerCaller) {++ callers.set(callerKey, list);++ return { ok: false, reason: 'ports.targets', detail: `At most ${targetsPerCaller} different hosts every ${Math.round(windowMs / 60_000)} minutes`, retryAfter };++ }++ list.push({ at: t, target: key, ports: n });++ callers.set(callerKey, list);++ if (callers.size > 50_000) for (const [k, l] of callers) if (!l.length || t - l[l.length - 1].at > windowMs) callers.delete(k);++ return { ok: true };++ }++++ return { take, perRequest };++}++++module.exports = { normalizeTarget, createTargetThrottle, createPortScanCap };nnew file mode 100644@@@ -0,0 +1,171 @@++'use strict';++// ═══════════════════════════════════════════════════════════════++// Network tokens, verified offline against the Network's RS256 key (no dependencies: node:crypto).++//++// What the Network issues (OpenVibe.Network server/auth, checked 2026-09-23):++// user (browser sign-in, OAuth code or FedCM exchange, and the Network's own session tokens)++// iss = the Network, aud = [openvibe.live, openvibe.tools, openvibe.games, openvibe.media,++// openvibe.network] (+ registry hosts on white-label installs), 24 h; claims sub/id (the++// numeric account id), subject_id (usr_…), username, display_name, role, avatar_url,++// profile_color. No cap.++// service / app principal (client_credentials, ADR-014 developer apps)++// aud = [the one audience asked for], sub svc:<client> | app:<id>, actor_type, cap[], ns[],++// env (apps: production | sandbox), 15 min.++// /internal/issue-token (Network-side cross-service features) carries no aud and no subject_id:++// not for Tools, and refused here.++// A user token is accepted only with aud openvibe.tools (the copies this replaces checked the++// issuer alone). A token shaped like a principal (cap[] or an svc:/app:/mod: subject) is never a user.++// ═══════════════════════════════════════════════════════════════++++const crypto = require('crypto');++const fs = require('fs');++const path = require('path');++++const AUDIENCE = 'openvibe.tools';++const JWKS_PATH = '/api/.well-known/jwks';++++const fromB64url = (s) => Buffer.from(String(s).replace(/-/g, '+').replace(/_/g, '/'), 'base64');++const json = (part) => { try { const v = JSON.parse(fromB64url(part).toString('utf8')); return v && typeof v === 'object' ? v : null; } catch { return null; } };++++/** A JWT's payload, unverified (to route it to the right check). */++function peek(token) {++ const parts = typeof token === 'string' ? token.split('.') : [];++ return parts.length === 3 ? json(parts[1]) : null;++}++++function looksLikePrincipal(claims) {++ return !!claims && (Array.isArray(claims.cap) || /^(svc|app|mod):/.test(String(claims.sub || '')));++}++++/** RS256 signature check → { header, claims } or null. */++function verifySignature(token, publicKey) {++ if (!publicKey) return null;++ const parts = typeof token === 'string' ? token.split('.') : [];++ if (parts.length !== 3) return null;++ const header = json(parts[0]);++ const claims = json(parts[1]);++ if (!header || !claims || header.alg !== 'RS256') return null;++ let good = false;++ try { good = crypto.verify('RSA-SHA256', Buffer.from(`${parts[0]}.${parts[1]}`), publicKey, fromB64url(parts[2])); } catch { good = false; }++ return good ? { header, claims } : null;++}++++function audiences(claims) { return Array.isArray(claims.aud) ? claims.aud : claims.aud == null ? [] : [claims.aud]; }++++function timely(claims, now, skewSec) {++ const t = Math.floor(now / 1000);++ if (typeof claims.exp !== 'number' || claims.exp + skewSec < t) return false;++ if (typeof claims.nbf === 'number' && claims.nbf - skewSec > t) return false;++ if (typeof claims.iat === 'number' && claims.iat - skewSec > t) return false;++ return true;++}++++/**++ * A signed-in person's token → its claims, or null (bad signature, expired, wrong issuer, not for++ * openvibe.tools, or a principal's token).++ */++function verifyUserToken(token, { publicKey, issuer, audience = AUDIENCE, now = Date.now(), skewSec = 30 } = {}) {++ const v = verifySignature(token, publicKey);++ if (!v) return null;++ const c = v.claims;++ if (looksLikePrincipal(c)) return null;++ if (issuer && c.iss !== issuer) return null;++ if (!audiences(c).includes(audience)) return null;++ if (!timely(c, now, skewSec)) return null;++ return c;++}++++/**++ * A service or app principal's token → { ok: true, claims } | { ok: false, code, reason }.++ * With openvibe-contracts it is contracts.serviceAuth.verifyServiceToken (claims checked against++ * identity.service-token-claims@1); apps without contracts get the same checks minus the schema.++ */++function verifyServiceToken(token, { publicKey, issuer, audience = AUDIENCE, acceptSandbox = false, contracts = null, now = Date.now() } = {}) {++ if (contracts && contracts.serviceAuth) return contracts.serviceAuth.verifyServiceToken(token, { publicKey, issuer, audience, acceptSandbox, now });++ const fail = (code, reason) => ({ ok: false, code, reason });++ if (typeof token !== 'string' || token.split('.').length !== 3) return fail('token.malformed', 'not a JWT');++ const v = verifySignature(token, publicKey);++ if (!v) return fail('token.bad_signature', 'signature does not verify');++ const c = v.claims;++ if (!timely(c, now, 30)) return fail('token.expired', 'expired');++ if (issuer && c.iss !== issuer) return fail('token.wrong_issuer', `issuer ${c.iss}`);++ if (!audiences(c).includes(audience)) return fail('token.wrong_audience', `not for ${audience}`);++ if (!/^(svc|app|mod):[A-Za-z0-9_.-]+$/.test(String(c.sub || '')) || !Array.isArray(c.cap) || !c.cap.every(x => typeof x === 'string')) return fail('token.invalid_claims', 'sub or cap');++ if (c.env === 'sandbox' && !acceptSandbox) return fail('token.sandbox_refused', 'sandbox tokens are not accepted here');++ return { ok: true, claims: c };++}++++/**++ * Why a principal may not use `cap` → a problem code (capability.denied, or contracts' own, e.g. an++ * audience or namespace mismatch), or null when it may. contracts' manifest-aware check when available.++ */++function capabilityDenied(claims, cap, contracts = null) {++ if (contracts && contracts.capabilities) {++ const c = contracts.capabilities.check(claims, cap);++ if (c.allowed) return null;++ // A capability the installed contracts do not know yet: an explicit grant (only Network signs) is enough.++ if (c.code === 'capability.unknown' && contracts.capabilities.grants(claims.cap, cap)) return null;++ return c.code === 'capability.unknown' ? 'capability.denied' : c.code;++ }++ return Array.isArray(claims.cap) && claims.cap.includes(cap) ? null : 'capability.denied';++}++const hasCapability = (claims, cap, contracts = null) => !capabilityDenied(claims, cap, contracts);++++/** The token a request carries: Authorization: Bearer, else the shared ov_token cookie (or the older token cookie). */++function extractToken(req) {++ const h = String((req.headers && req.headers.authorization) || '');++ if (h.startsWith('Bearer ')) return { token: h.slice(7).trim(), from: 'header' };++ const c = req.cookies || {};++ if (c.ov_token) return { token: c.ov_token, from: 'cookie' };++ if (c.token) return { token: c.token, from: 'cookie' };++ return { token: null, from: null };++}++++/**++ * The Network's public key: a PEM file (OV_NETWORK_PUBLIC_KEY, air-gapped setups) or the Network's++ * JWKS endpoint (internal URL first), fetched at start and every `retryMs` until it loads. Until++ * then every caller is anonymous (nothing fails because the Network is down).++ *++ * @returns {{ get(): string|null, ensure(): Promise<string|null>, stop(): void }}++ */++function createKeySource({ networkUrl, networkInternalUrl, files = [], fetchImpl = (...a) => globalThis.fetch(...a), retryMs = 60_000, timeoutMs = 5000, log = console, start = true } = {}) {++ let key = null, timer = null, inflight = null, lastTry = 0;++ for (const p of files.filter(Boolean)) {++ try {++ const resolved = path.resolve(p);++ if (fs.existsSync(resolved)) { key = fs.readFileSync(resolved, 'utf8'); log.log(`[Auth] Network public key from ${resolved}`); break; }++ } catch { /* next */ }++ }++ async function fetchKey() {++ for (const base of [networkInternalUrl, networkUrl].filter(Boolean)) {++ try {++ const res = await fetchImpl(`${String(base).replace(/\/+$/, '')}${JWKS_PATH}`, { signal: AbortSignal.timeout(timeoutMs) });++ if (!res.ok) continue;++ const body = await res.json();++ if (body && body.public_key) { key = body.public_key; log.log(`[Auth] Network public key from ${base}${JWKS_PATH}`); return key; }++ } catch { /* next base */ }++ }++ return null;++ }++ // Requests carrying a token may ask for the key while it is missing; the Network is asked at most++ // every 30 s for them (the retry timer below keeps trying on its own).++ function ensure() {++ if (key) return Promise.resolve(key);++ if (inflight) return inflight;++ if (Date.now() - lastTry < 30_000) return Promise.resolve(null);++ lastTry = Date.now();++ inflight = fetchKey().finally(() => { inflight = null; });++ return inflight;++ }++ if (!key && start) {++ ensure().then((k) => {++ if (k) return;++ log.warn(`[Auth] OpenVibe Network unreachable — retrying every ${retryMs / 1000}s (everyone is anonymous until the key loads)`);++ timer = setInterval(async () => { if (await ensure()) { clearInterval(timer); timer = null; } }, retryMs);++ if (timer.unref) timer.unref();++ });++ }++ return { get: () => key, ensure, stop() { if (timer) clearInterval(timer); timer = null; } };++}++++module.exports = { AUDIENCE, peek, looksLikePrincipal, verifySignature, verifyUserToken, verifyServiceToken, hasCapability, capabilityDenied, extractToken, createKeySource };@@@ -28,16 +28,13 @@const fs = require('fs');const path = require('path');--const crypto = require('crypto');const { Readable } = require('stream');++const { createCallerResolver, jobOwnerResolver, JOB_CAPS: CAPS } = require('../guard/caller');++const ID_RE = /^job_[0-9A-HJKMNP-TV-Z]{26}$/;--const SESSION_RE = /^[A-Za-z0-9_-]{32,64}$/;const TERMINAL = new Set(['succeeded', 'failed', 'cancelled']);const CLIENT_JS = path.join(__dirname, 'client.js');--const CAPS = { create: 'tools.job.create', read: 'tools.job.read', cancel: 'tools.job.cancel' };----const b64json = (part) => { try { return JSON.parse(Buffer.from(String(part).replace(/-/g, '+').replace(/_/g, '/'), 'base64').toString('utf8')); } catch { return null; } };function contentDisposition(kind, name) {const ascii = String(name || 'result').replace(/[^\x20-\x7e]/g, '_').replace(/["\\]/g, '_');@@@ -45,7 +42,9 @@ function contentDisposition(kind, name) {}/**-- * Who is asking. → { owner, kind: 'principal'|'user'|'session', claims? } | { error: [status, code, detail] } | null++ * Who is asking. → { owner, kind: 'principal'|'user'|'session', claims?, env } | { error: [status, code, detail] } | null++ * The guard's one caller resolver (apps/_shared/guard/caller.js); an app with a guard passes++ * guard.ownerResolver() instead, which is the same resolver with the guard's daily address salt.** @param {object} o* @param {object} o.contracts@@@ -56,54 +55,7 @@ function contentDisposition(kind, name) {* @param {boolean} [o.secureCookie]*/function createOwnerResolver(o) {-- const audience = o.audience || 'openvibe.tools';-- const cookieName = o.cookieName || 'ov_tools_jobs';-- const { contracts } = o;---- function principalCapability(claims, cap) {-- const c = contracts.capabilities.check(claims, cap);-- if (c.allowed) return null;-- // The tools.job.* manifests are proposed for the next contracts release; until the installed-- // version knows them, an explicit grant in the token (which only Network can sign) is enough.-- if (c.code === 'capability.unknown' && contracts.capabilities.grants(claims.cap, cap)) return null;-- return [403, c.code === 'capability.unknown' ? 'capability.denied' : c.code, `${cap} not granted`];-- }---- return function resolveOwner(req, res, { create = false, action = 'read' } = {}) {-- const auth = String(req.headers.authorization || '');-- if (auth.startsWith('Bearer ')) {-- const token = auth.slice(7).trim();-- const parts = token.split('.');-- const claims = parts.length === 3 ? b64json(parts[1]) : null;-- const looksLikePrincipal = claims && (Array.isArray(claims.cap) || /^(svc|app|mod):/.test(String(claims.sub || '')));-- if (looksLikePrincipal) {-- // Job routes are the only Tools routes that take sandbox tokens (developer apps, ADR-014), and-- // only from apps: a sandbox token that is not an app's is refused.-- const r = contracts.serviceAuth.verifyServiceToken(token, { publicKey: o.getPublicKey(), issuer: o.issuer, audience, acceptSandbox: true });-- if (r.ok && r.claims.env === 'sandbox' && !(r.claims.actor_type === 'app' && /^app:/.test(String(r.claims.sub || '')))) {-- return { error: [401, 'token.sandbox_refused', 'sandbox tokens are accepted only from developer apps'] };-- }-- if (!r.ok) return { error: [401, r.code, r.reason] };-- const denied = principalCapability(r.claims, CAPS[action] || CAPS.read);-- if (denied) return { error: denied };-- return { owner: r.claims.sub, kind: 'principal', claims: r.claims, env: r.claims.env === 'sandbox' ? 'sandbox' : 'production' };-- }-- }-- const sid = req.user && req.user.subject_id;-- if (sid && contracts.ids.isSubjectId('user', sid)) return { owner: `user:${sid}`, kind: 'user' };---- let session = req.cookies && req.cookies[cookieName];-- if (!SESSION_RE.test(String(session || ''))) {-- if (!create) return null;-- session = crypto.randomBytes(24).toString('base64url');-- res.cookie(cookieName, session, {-- httpOnly: true, sameSite: 'lax', path: '/', maxAge: 7 * 24 * 60 * 60 * 1000,-- secure: o.secureCookie != null ? o.secureCookie : process.env.NODE_ENV === 'production',-- });-- }-- // Only a hash of the cookie is stored, so the database cannot be used to take over a session.-- return { owner: `session:${crypto.createHash('sha256').update(session).digest('hex').slice(0, 40)}`, kind: 'session' };-- };++ return jobOwnerResolver(createCallerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer, audience: o.audience, cookieName: o.cookieName, secureCookie: o.secureCookie }));}/**@@@ -118,6 +70,13 @@ function createOwnerResolver(o) {* @param {function[]} [o.limiters] middleware run before accepting a submit (rate limits)* @param {(req, who) => number} [o.ttlMs] how long a finished job is kept* @param {(req, input) => object} [o.defaults] fills input from the host (e.g. the format a png.* host implies)++ * @param {(req, res, { type, input, files }) => Promise<boolean>} [o.admit] the guard's admission of a++ * submit once its body is parsed (session rule, the bytes against the tool's accept, the quota++ * difference); false = it answered++ * @param {(req, res, busy) => boolean} [o.onBusy] system.busy() said so: true = go on (report mode);++ * without it a busy store answers 503 tools.busy++ * @param {(req, res, full) => boolean} [o.onAddressFull] a browser session's address already has its++ * unfinished jobs (system.addressFull): true = go on (report mode); without it 429*/function mountJobRoutes(app, o) {const { system, contracts } = o;@@@ -158,7 +117,16 @@ function mountJobRoutes(app, o) {receive(req, res, (err) => { res.json = json; next(err); });};-- app.post('/api/v1/jobs', ...(o.limiters || []), multipartOnly, async (req, res) => {++ // Over the store's bounds (queued jobs, disk): answer before an upload is accepted.++ const notBusy = (req, res, next) => {++ const busy = system.busy ? system.busy() : null;++ if (!busy) return next();++ if (o.onBusy) return o.onBusy(req, res, busy) ? next() : undefined;++ res.set('Retry-After', String(busy.retryAfter || 30));++ return send(res, 503, 'tools.busy', busy.detail);++ };++++ app.post('/api/v1/jobs', ...(o.limiters || []), notBusy, multipartOnly, async (req, res) => {noStore(res);const w = who(req, res, { create: true, action: 'create' });if (!w) return;@@@ -173,8 +141,22 @@ function mountJobRoutes(app, o) {const list = Array.isArray(req.files) ? req.files : req.files && typeof req.files === 'object' ? Object.values(req.files).flat() : req.file ? [req.file] : [];const files = list.map(f => ({ path: f.path, buffer: f.path ? undefined : f.buffer, name: f.originalname, mime: f.mimetype, size: f.size }));const key = req.headers['idempotency-key'] != null ? String(req.headers['idempotency-key']) : (b.idempotency_key != null ? String(b.idempotency_key) : null);++ // Sessions from one address share a bound on unfinished jobs: a new cookie is no new allowance.++ const full = w.caller && system.addressFull ? system.addressFull(w.owner, w.caller.ipKey) : null;++ if (full) {++ const goOn = o.onAddressFull ? o.onAddressFull(req, res, full) : (send(res, 429, 'tools.job.too_many_active', `At most ${full.limit} unfinished jobs from one address at a time; wait for one to finish`), false);++ if (!goOn) { for (const f of files) if (f.path) fs.unlink(f.path, () => {}); return; }++ }++ if (o.admit) {++ let ok = false;++ try { ok = await o.admit(req, res, { type: String(b.type || ''), input, files }); } catch (err) {++ console.error('[Jobs] admission failed:', err.message);++ if (!res.headersSent) send(res, 500, 'tools.job.submit_failed', 'The job could not be accepted');++ }++ if (!ok) { for (const f of files) if (f.path) fs.unlink(f.path, () => {}); return; }++ }try {-- const { job, replayed } = await system.submit({ owner: w.owner, type: String(b.type || ''), input, files, idempotencyKey: key, ttlMs: ttlMs(req, w), env: w.env || 'production' });++ const { job, replayed } = await system.submit({ owner: w.owner, type: String(b.type || ''), input, files, idempotencyKey: key, ttlMs: ttlMs(req, w), env: w.env || 'production', ipKey: w.caller ? w.caller.ipKey : null });res.set('Location', `/api/v1/jobs/${job.id}`);if (replayed) res.set('Idempotent-Replayed', 'true');return res.status(replayed ? 200 : 202).json(system.view(job));@@@ -201,7 +183,7 @@ function mountJobRoutes(app, o) {});// Retry and references are writes: a principal needs tools.job.create, like a submit.-- app.post('/api/v1/jobs/:id/retry', ...(o.limiters || []), (req, res) => {++ app.post('/api/v1/jobs/:id/retry', ...(o.limiters || []), notBusy, (req, res) => {noStore(res);const row = load(req, res, 'create');if (!row) return;@@@ -9,6 +9,10 @@// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)// TOOLS_JOBS_CONCURRENCY_<APP> per-satellite override, e.g. TOOLS_JOBS_CONCURRENCY_AUDIO=1// TOOLS_JOBS_MAX_ACTIVE unfinished jobs per owner (default 10)++// TOOLS_JOBS_MAX_ACTIVE_PER_ADDRESS unfinished browser-session jobs per address, all sessions (default 3 × the above)++// TOOLS_JOBS_MAX_QUEUED queued jobs (all owners) per satellite before 503 tools.busy (default 200)++// TOOLS_DISK_BUDGET_MB MB under <dataDir>/jobs per satellite before 503 tools.busy (default 8192)++// (both through the guard: refused with TOOLS_GUARD=enforce, recorded in report mode)// TOOLS_JOB_RESULTS local (default) | media — where result files go// TOOLS_MEDIA_NAMESPACE Media namespace for results (default tools)// OV_MEDIA_INTERNAL_URL Media on this host (default http://127.0.0.1:4100; MEDIA_URL is read as a fallback)@@@ -76,6 +80,11 @@ function mediaOrigin(env = process.env) {* @param {Function} [o.receive] multipart middleware (the app's multer)* @param {Function[]} [o.limiters]* @param {Function} [o.defaults] (req, input) → input with host defaults++ * @param {object} [o.guard] apps/_shared/guard: its caller resolver, admission (session rule,++ * sniffing, quota) and busy handling; without it the job routes++ * resolve owners themselves and a busy store is always refused++ * @param {(req, type, input) => string|null} [o.jobTool] the descriptor id a submit runs (the host's++ * tool, or the one its input.tool names)*/function setupJobs(o) {fs.mkdirSync(o.dataDir, { recursive: true });@@@ -89,12 +98,23 @@ function setupJobs(o) {db, contracts: o.contracts, service: o.service, dataDir: o.dataDir,concurrency: envInt(`TOOLS_JOBS_CONCURRENCY_${o.service.toUpperCase()}`, envInt('TOOLS_JOBS_CONCURRENCY', 2)),maxActivePerOwner: envInt('TOOLS_JOBS_MAX_ACTIVE', 10) || 10,++ maxActivePerAddress: envInt('TOOLS_JOBS_MAX_ACTIVE_PER_ADDRESS', 0),++ maxQueued: envInt('TOOLS_JOBS_MAX_QUEUED', 200),++ diskBudgetBytes: envInt('TOOLS_DISK_BUDGET_MB', 8192) * 1024 * 1024,media: results.media,outbox: events.outbox,});o.define(system);-- const resolveOwner = createOwnerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer });-- mountJobRoutes(o.app, { system, contracts: o.contracts, resolveOwner, receive: o.receive, limiters: o.limiters, defaults: o.defaults });++ const g = o.guard || null;++ const resolveOwner = g ? g.ownerResolver() : createOwnerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer });++ mountJobRoutes(o.app, {++ system, contracts: o.contracts, resolveOwner, receive: o.receive, limiters: o.limiters, defaults: o.defaults,++ ...(g && {++ admit: (req, res, job) => g.admitJob(req, res, { ...job, toolId: o.jobTool ? o.jobTool(req, job.type, job.input) : null }),++ onBusy: (req, res, busy) => g.jobsBusy(req, res, busy, o.jobTool ? o.jobTool(req, null, {}) : null),++ onAddressFull: (req, res, full) => !g.refuse(req, res, { status: 429, code: 'tools.job.too_many_active', reason: 'jobs.address', tool: o.jobTool ? o.jobTool(req, null, {}) : null, retryAfter: 30, detail: `At most ${full.limit} unfinished jobs from one address at a time; wait for one to finish.`, extra: { scope: 'address' } }),++ }),++ });system.start();let pruneOutbox = null;if (events.outbox) {@@@ -65,6 +65,8 @@ const ADDED_COLUMNS = [['env', "TEXT NOT NULL DEFAULT 'production' CHECK (env IN ('production','sandbox'))"], // developer-app sandboxes['retry_of', 'TEXT'], // the failed job this one retries['retried_by', 'TEXT'], // set on a failed job once it has been retried: the retry's id++ ['ip_key', 'TEXT'], // a browser session's job: the guard's hashed address key (never the address), so++ // unfinished jobs are also bounded per address (a new cookie is no new allowance)];const parse = (s, fallback) => { if (s == null) return fallback; try { return JSON.parse(s); } catch { return fallback; } };@@@ -75,10 +77,11 @@ function createStore(db) {db.exec(SCHEMA);const have = new Set(db.prepare('PRAGMA table_info(tool_jobs)').all().map((c) => c.name));for (const [name, decl] of ADDED_COLUMNS) if (!have.has(name)) db.exec(`ALTER TABLE tool_jobs ADD COLUMN ${name} ${decl}`);++ db.exec("CREATE INDEX IF NOT EXISTS tool_jobs_ip ON tool_jobs(ip_key, state) WHERE ip_key IS NOT NULL");const q = {-- insert: db.prepare(`INSERT INTO tool_jobs (id, type, type_version, owner, state, input_json, files_json, idempotency_key, request_hash, max_attempts, ttl_ms, created_at, updated_at, env, retry_of)-- VALUES (@id, @type, @type_version, @owner, 'queued', @input_json, @files_json, @idempotency_key, @request_hash, @max_attempts, @ttl_ms, @now, @now, @env, @retry_of)`),++ insert: db.prepare(`INSERT INTO tool_jobs (id, type, type_version, owner, state, input_json, files_json, idempotency_key, request_hash, max_attempts, ttl_ms, created_at, updated_at, env, retry_of, ip_key)++ VALUES (@id, @type, @type_version, @owner, 'queued', @input_json, @files_json, @idempotency_key, @request_hash, @max_attempts, @ttl_ms, @now, @now, @env, @retry_of, @ip_key)`),markRetried: db.prepare("UPDATE tool_jobs SET retried_by = @next, updated_at = @now WHERE id = @id AND state = 'failed' AND retried_by IS NULL"),get: db.prepare('SELECT * FROM tool_jobs WHERE id = ?'),byIdem: db.prepare('SELECT * FROM tool_jobs WHERE owner = ? AND idempotency_key = ?'),@@@ -86,6 +89,7 @@ function createStore(db) {claim: db.prepare("UPDATE tool_jobs SET state = 'running', attempts = attempts + 1, started_at = @now, updated_at = @now, progress = NULL, progress_message = NULL WHERE id = @id AND state = 'queued'"),running: db.prepare("SELECT * FROM tool_jobs WHERE state = 'running'"),activeForOwner: db.prepare("SELECT COUNT(*) AS n FROM tool_jobs WHERE owner = ? AND state IN ('queued','running')"),++ activeForIp: db.prepare("SELECT COUNT(*) AS n FROM tool_jobs WHERE ip_key = ? AND state IN ('queued','running')"),progress: db.prepare('UPDATE tool_jobs SET progress = @progress, progress_message = @message, updated_at = @now WHERE id = @id AND state = \'running\''),requestCancel: db.prepare("UPDATE tool_jobs SET cancel_requested = 1, updated_at = @now WHERE id = @id AND state IN ('queued','running')"),requeue: db.prepare("UPDATE tool_jobs SET state = 'queued', started_at = NULL, progress = NULL, progress_message = NULL, updated_at = @now WHERE id = @id AND state = 'running'"),@@@ -120,7 +124,7 @@ function createStore(db) {return {db,TERMINAL, STATES,-- insert(row) { q.insert.run({ retry_of: null, ...row }); },++ insert(row) { q.insert.run({ retry_of: null, ip_key: null, ...row }); },markRetried: (id, next, now = Date.now()) => q.markRetried.run({ id, next, now }).changes === 1,get: (id) => q.get.get(id) || null,byIdempotencyKey: (owner, key) => q.byIdem.get(owner, key) || null,@@@ -128,6 +132,7 @@ function createStore(db) {claim: (id, now = Date.now()) => q.claim.run({ id, now }).changes === 1,running: () => q.running.all(),activeForOwner: (owner) => q.activeForOwner.get(owner).n,++ activeForIp: (ipKey) => (ipKey ? q.activeForIp.get(ipKey).n : 0),setProgress: (id, progress, message, now = Date.now()) => q.progress.run({ id, progress, message, now }).changes === 1,requestCancel: (id, now = Date.now()) => q.requestCancel.run({ id, now }).changes === 1,requeue: (id, now = Date.now()) => q.requeue.run({ id, now }).changes === 1,@@@ -10,7 +10,10 @@// (onRestart: 'fail'), per job type. Queued rows simply start again.//// Bounded: at most `concurrency` jobs run at once in this process, and each owner may have at--// most `maxActivePerOwner` queued + running jobs. Finished jobs expire (ttl chosen at submit),++// most `maxActivePerOwner` queued + running jobs. busy() says when the whole store is over its++// bounds — `maxQueued` jobs waiting (all owners), or `diskBudgetBytes` under <dataDir>/jobs — so the++// HTTP layer can answer 503 tools.busy with Retry-After before it accepts an upload (through the++// guard: refused in enforce mode, recorded in report mode). Finished jobs expire (ttl chosen at submit),// and the pruner deletes the row, its events, its files and any Media objects it made — except// while something references the result (reference(), e.g. a paste or a project that points at the// Media object), and never while Media keeps an object (a retention hold, or Media unreachable):@@@ -75,6 +78,10 @@ const scrub = (msg) => String(msg || '').replace(/(?:\/[\w.-]+){2,}/g, '[file]')* @param {string} o.dataDir where job files live (<dataDir>/jobs/<id>/…)* @param {number} [o.concurrency=2]* @param {number} [o.maxActivePerOwner=10]++ * @param {number} [o.maxActivePerAddress] unfinished browser-session jobs per address (ipKey), all++ * sessions together (default 3 × maxActivePerOwner); see addressFull()++ * @param {number} [o.maxQueued=0] queued jobs (all owners) before busy() says so; 0 = no bound++ * @param {number} [o.diskBudgetBytes=0] bytes under <dataDir>/jobs before busy() says so; 0 = no bound* @param {object} [o.media] result store from ./media (null = keep results on local disk)* @param {object} [o.outbox] openvibe-sdk outbox on `db` (./events outboxFromEnv); null = no platform events* @param {number} [o.pruneIntervalMs=300000]@@@ -91,6 +98,10 @@ function createJobSystem(o) {const throttleMs = o.progressThrottleMs == null ? 250 : o.progressThrottleMs;const root = path.resolve(o.dataDir, 'jobs');fs.mkdirSync(root, { recursive: true });++ const maxQueued = o.maxQueued > 0 ? o.maxQueued : 0;++ const maxActivePerAddress = o.maxActivePerAddress > 0 ? o.maxActivePerAddress : 3 * maxActivePerOwner;++ const diskBudget = o.diskBudgetBytes > 0 ? o.diskBudgetBytes : 0;++ const disk = { bytes: 0, at: 0, scanning: null };const announce = createJobEvents({ contracts, service: o.service, outbox: o.outbox || null, referenceCount: (id) => store.referenceCount(id), log });const types = new Map();@@@ -191,7 +202,7 @@ function createJobSystem(o) {// Developer-app sandbox jobs: fewer at a time, kept briefly, results never leave this server.const SANDBOX_MAX_ACTIVE = Math.max(1, o.sandboxMaxActivePerOwner || 2);const SANDBOX_TTL_MS = 30 * 60 * 1000;-- async function submit({ owner, type, input = {}, files = [], idempotencyKey = null, ttlMs = HOUR, env = 'production' }) {++ async function submit({ owner, type, input = {}, files = [], idempotencyKey = null, ttlMs = HOUR, env = 'production', ipKey = null }) {if (env !== 'sandbox') env = 'production';if (env === 'sandbox') ttlMs = Math.min(ttlMs, SANDBOX_TTL_MS);if (stopped) throw new JobError(503, 'tools.job.unavailable', 'The job system is shutting down');@@@ -240,6 +251,7 @@ function createJobSystem(o) {id, type, type_version: def.version, owner, input_json: JSON.stringify(input), files_json: JSON.stringify(stored),idempotency_key: idempotencyKey == null ? null : String(idempotencyKey), request_hash: requestHash,max_attempts: def.maxAttempts, ttl_ms: ttlMs, now: Date.now(), env,++ ip_key: String(owner).startsWith('session:') && ipKey ? String(ipKey) : null,});announce.created(store.get(id));})();@@@ -253,6 +265,7 @@ function createJobSystem(o) {}throw err;}++ disk.bytes += stored.reduce((n, f) => n + (f.size || 0), 0);emit(id, 'job.queued');kick();return { job: store.get(id), replayed: false };@@@ -461,7 +474,7 @@ function createJobSystem(o) {store.insert({id: next, type: row.type, type_version: def.version, owner: row.owner, input_json: row.input_json, files_json: JSON.stringify(stored),idempotency_key: null, request_hash: row.request_hash, max_attempts: def.maxAttempts, ttl_ms: row.ttl_ms, now: Date.now(),-- env: row.env, retry_of: row.id,++ env: row.env, retry_of: row.id, ip_key: row.ip_key || null,});announce.created(store.get(next));})();@@@ -573,9 +586,60 @@ function createJobSystem(o) {return n;}++ // ── Host-wide bounds ───────────────────────────────────────++ /** Bytes under <dataDir>/jobs (inputs and results), walked in the background at most every 30 s. */++ async function scanDisk() {++ let total = 0;++ const walk = async (dir) => {++ let entries = [];++ try { entries = await fsp.readdir(dir, { withFileTypes: true }); } catch { return; }++ for (const e of entries) {++ const p = path.join(dir, e.name);++ if (e.isDirectory()) await walk(p);++ else if (e.isFile()) { try { total += (await fsp.stat(p)).size; } catch { /* gone meanwhile */ } }++ }++ };++ await walk(root);++ disk.bytes = total;++ disk.at = Date.now();++ return total;++ }++ function refreshDisk() {++ if (!disk.scanning) disk.scanning = scanDisk().catch(() => disk.bytes).finally(() => { disk.scanning = null; });++ return disk.scanning;++ }++++ /**++ * Is the store over its bounds? → null, or { reason: 'queue'|'disk', detail, retryAfter } for a++ * 503 tools.busy. Cheap: one COUNT and the last disk walk (a stale walk is refreshed in the background).++ */++ function busy() {++ if (maxQueued) {++ const queued = store.counts().queued || 0;++ if (queued >= maxQueued) return { reason: 'queue', detail: `${queued} jobs are already waiting on this server; try again in a minute.`, retryAfter: 60, queued, limit: maxQueued };++ }++ if (diskBudget) {++ if (Date.now() - disk.at > 30_000) refreshDisk();++ if (disk.bytes >= diskBudget) return { reason: 'disk', detail: 'This server is holding as many files as it can right now; try again in a few minutes.', retryAfter: 300, bytes: disk.bytes, limit: diskBudget };++ }++ return null;++ }++++ /**++ * Browser sessions from one address together have maxActivePerAddress unfinished jobs: → null, or++ * { active, limit } when a new session job from `ipKey` would pass it (the HTTP layer answers 429++ * tools.job.too_many_active through the guard). Principals and people are bounded per owner only.++ */++ function addressFull(owner, ipKey) {++ if (!ipKey || !String(owner || '').startsWith('session:')) return null;++ const active = store.activeForIp(ipKey);++ return active >= maxActivePerAddress ? { active, limit: maxActivePerAddress } : null;++ }++function start() {if (started) return api;started = true;++ if (diskBudget) refreshDisk();const r = recover();if (r.requeued || r.failed) log.log(`[Jobs] ${o.service}: ${r.requeued} job(s) re-queued, ${r.failed} failed after restart`);prune().catch(() => {});@@@ -610,6 +674,7 @@ function createJobSystem(o) {const api = {define, submit, cancel, retry, reference, unreference, subscribe, start, stop, onStop, prune, recover, view, resultFile,++ busy, refreshDisk, addressFull, bounds: { maxQueued, diskBudgetBytes: diskBudget, maxActivePerAddress },get: (id) => store.get(id),eventsAfter: (id, seq) => store.eventsAfter(id, seq),lastSeq: (id) => store.lastSeq(id),nnew file mode 100644@@@ -0,0 +1,470 @@++'use strict';++// The shared guard (apps/_shared/guard), end to end where it matters:++// callers and keys (IPv6 /64, one loopback hop, loopback services by their token, sessions, users++// with the audience check, sandbox apps), token-bucket math and cost weighting from descriptors,++// session-per-address sharing, report vs enforce, day allowances persisted across a restart,++// RateLimit-* headers and problem+json, upload sniffing (a renamed file), the per-target throttle,++// the port-scan cap, the abuse log (no raw address, pruned after 30 days) and the metric, the sync++// semaphore, the jobs' queue and disk bounds, and the challenge hook.++const assert = require('assert');++const crypto = require('crypto');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { dep } = require('./deps');++++const express = dep('express');++const cookieParser = dep('cookie-parser');++const multer = dep('multer');++const Database = dep('better-sqlite3');++const contracts = dep('openvibe-contracts');++const metrics = dep('openvibe-shared/metrics');++const guardLib = require('../guard');++const { createGuard, TRUST_PROXY } = guardLib;++const { createSemaphore, Busy } = require('../guard/semaphore');++const { createQuotas } = require('../guard/quota');++const { createGuardStore } = require('../guard/store');++const { ipBucket } = require('../guard/ip');++const sniff = require('../guard/sniff');++const jobs = require('../jobs');++++const ISSUER = 'https://openvibe.network';++const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });++const b64 = (o) => Buffer.from(JSON.stringify(o)).toString('base64url');++function jwt(claims) {++ const input = `${b64({ alg: 'RS256', typ: 'JWT' })}.${b64(claims)}`;++ return `${input}.${crypto.sign('RSA-SHA256', Buffer.from(input), privateKey).toString('base64url')}`;++}++const now0 = Date.UTC(2026, 8, 23, 12, 0, 0);++const t = () => Math.floor(Date.now() / 1000);++const USER = 'usr_01JAAAAAAAAAAAAAAAAAAAAAAA';++const userToken = (extra = {}) => jwt({ iss: ISSUER, sub: '42', id: 42, subject_id: USER, username: 'alice', aud: ['openvibe.live', 'openvibe.tools', 'openvibe.network'], iat: t(), exp: t() + 3600, ...extra });++const svcToken = (name, extra = {}) => contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: `svc:${name}`, actor_type: 'service', aud: ['openvibe.tools'], cap: ['tools.tool.run'], ns: [], iat: t(), exp: t() + 900, jti: `tok_${crypto.randomBytes(12).toString('hex')}`, ...extra }, privateKey);++const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'ov-guard-'));++const quiet = { log() {}, warn() {}, error() {} };++++// Pure-data descriptors of the real apps (nothing loads sharp or ffmpeg).++const IMG = require('../../img/server/descriptors').SPECS;++const AUDIO = require('../../audio/server/descriptors').SPECS;++const NET = require('../../gateway/server/net/descriptors').SPECS;++++function makeGuard(o = {}) {++ return createGuard({++ app: o.app || 'test', dataDir: o.dataDir, Database: o.dataDir ? Database : undefined, contracts,++ specs: o.specs || [...IMG, ...AUDIO, ...NET], issuer: ISSUER,++ keys: { get: () => publicKey, ensure: async () => publicKey },++ env: { TOOLS_GUARD: o.mode || 'report', ...(o.env || {}) }, log: quiet, now: o.now, pruneIntervalMs: 0,++ challenge: o.challenge,++ });++}++++async function serve(app) {++ const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });++ return { base: `http://127.0.0.1:${srv.address().port}`, close: () => new Promise(r => srv.close(r)) };++}++++(async () => {++ // ── IPv6 by /64, IPv4-mapped as IPv4 ──++ assert.strictEqual(ipBucket('2001:db8:1:2:aaaa::1'), '2001:0db8:0001:0002::/64');++ assert.strictEqual(ipBucket('2001:db8:1:2:bbbb:cccc:dddd:9'), ipBucket('2001:db8:1:2::77'));++ assert.notStrictEqual(ipBucket('2001:db8:1:3::1'), ipBucket('2001:db8:1:2::1'));++ assert.strictEqual(ipBucket('::ffff:203.0.113.5'), '203.0.113.5');++ assert.strictEqual(TRUST_PROXY('127.0.0.1', 0), true);++ assert.strictEqual(TRUST_PROXY('::ffff:127.0.0.1', 0), true);++ assert.strictEqual(TRUST_PROXY('127.0.0.1', 1), false, 'exactly one hop');++ assert.strictEqual(TRUST_PROXY('203.0.113.9', 0), false, 'a non-loopback peer is never a proxy');++++ // ── Callers over HTTP: one loopback hop, tokens, sessions ──++ {++ const g = makeGuard();++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ app.use(cookieParser());++ app.use(g.identify);++ app.get('/who', (req, res) => { const c = g.caller(req, res); res.json({ kind: c.kind, tier: c.tier, key: c.key, ipKey: c.ipKey, user: req.user ? req.user.subject_id : null, ip: req.ip }); });++ app.get('/job-who', (req, res) => res.json(g.ownerResolver()(req, res, { create: true, action: 'create' }) || {}));++ const s = await serve(app);++ const who = async (headers = {}, p = '/who') => (await fetch(s.base + p, { headers })).json();++ try {++ const direct = await who();++ assert.strictEqual(direct.ip, '127.0.0.1');++ assert.strictEqual(direct.kind, 'anonymous', 'a loopback caller without a token is anonymous');++ assert.ok(direct.ipKey.startsWith('ip:') && !direct.ipKey.includes('127.0.0.1'), 'addresses are hashed');++ const viaNginx = await who({ 'X-Forwarded-For': '6.6.6.6, 198.51.100.7' });++ assert.strictEqual(viaNginx.ip, '198.51.100.7', 'only the one hop nginx adds is believed');++ assert.notStrictEqual(viaNginx.ipKey, direct.ipKey, 'a visitor forwarded by the proxy is not the loopback bucket');++ const v6a = await who({ 'X-Forwarded-For': '2001:db8:1:2::1' });++ const v6b = await who({ 'X-Forwarded-For': '2001:db8:1:2:ffff::2' });++ const v6c = await who({ 'X-Forwarded-For': '2001:db8:9:9::1' });++ assert.strictEqual(v6a.ipKey, v6b.ipKey, 'one /64 is one anonymous caller');++ assert.notStrictEqual(v6a.ipKey, v6c.ipKey);++++ // Two first-party services on loopback are two callers, by their tokens.++ const live = await who({ Authorization: `Bearer ${svcToken('live')}` });++ const chat = await who({ Authorization: `Bearer ${svcToken('chat')}` });++ assert.deepStrictEqual([live.kind, live.tier, live.key], ['principal', 'service', 'svc:live']);++ assert.strictEqual(chat.key, 'svc:chat');++ assert.strictEqual(live.ipKey, chat.ipKey, 'same loopback address, different callers');++ const wrongAud = await who({ Authorization: `Bearer ${svcToken('live', { aud: ['openvibe.media'] })}` });++ assert.strictEqual(wrongAud.kind, 'anonymous', 'a token for another audience is not believed');++ const jobWrongAud = await who({ Authorization: `Bearer ${svcToken('live', { aud: ['openvibe.media'] })}` }, '/job-who');++ assert.deepStrictEqual(jobWrongAud.error, [401, 'token.wrong_audience', 'not for openvibe.tools'], 'on the job routes it is an error');++++ // Users: the audience is checked now.++ const alice = await who({ Cookie: `ov_token=${userToken()}` });++ assert.deepStrictEqual([alice.kind, alice.tier, alice.key, alice.user], ['user', 'user', `user:${USER}`, USER]);++ const noAud = await who({ Authorization: `Bearer ${jwt({ iss: ISSUER, sub: '42', username: 'alice', role: 'user', iat: t(), exp: t() + 3600 })}` });++ assert.strictEqual(noAud.kind, 'anonymous', 'a Network token without aud openvibe.tools (/internal/issue-token) is not a sign-in here');++ const otherIssuer = await who({ Cookie: `ov_token=${userToken({ iss: 'https://evil.example' })}` });++ assert.strictEqual(otherIssuer.kind, 'anonymous');++ const expired = await who({ Cookie: `ov_token=${userToken({ exp: t() - 120 })}` });++ assert.strictEqual(expired.kind, 'anonymous');++++ // Sessions; a sandbox app keeps its small tier.++ const sid = crypto.randomBytes(24).toString('base64url');++ const session = await who({ Cookie: `ov_tools_jobs=${sid}` });++ assert.strictEqual(session.tier, 'session');++ assert.match(session.key, /^session:[0-9a-f]{40}$/);++ const sandbox = await who({ Authorization: `Bearer ${contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: 'app:app_01JCCCCCCCCCCCCCCCCCCCCCCC', actor_type: 'app', aud: ['openvibe.tools'], cap: ['tools.job.create'], ns: ['prj_01JCCCCCCCCCCCCCCCCCCCCCCC'], project_id: 'prj_01JCCCCCCCCCCCCCCCCCCCCCCC', env: 'sandbox', iat: t(), exp: t() + 900, jti: 'tok_' + 'a'.repeat(24) }, privateKey)}` });++ assert.deepStrictEqual([sandbox.kind, sandbox.tier], ['principal', 'sandbox']);++ } finally { await s.close(); g.close(); }++ }++++ // ── Bucket math and cost weighting from the descriptors ──++ {++ let clock = now0;++ const store = createGuardStore({ now: () => clock });++ const q = createQuotas({ store, quotas: () => guardLib.limits.QUOTAS, now: () => clock });++ const anon = { tier: 'anonymous', key: 'ip:a', ipKey: 'ip:a' };++ const cost = (id) => [...IMG, ...AUDIO].find(s => s.id === id).cost;++ assert.strictEqual(cost('png'), 5);++ assert.strictEqual(cost('avif'), 8, 'AVIF encoding weighs more');++ assert.strictEqual(cost('merge'), 20);++ // tools-job anonymous: a bucket of 30 refilled at 60 a minute.++ for (let i = 0; i < 3; i++) assert.ok(q.check(anon, { quotaClass: 'tools-job', cost: cost('avif') }).ok, `avif ${i + 1}`);++ let r = q.check(anon, { quotaClass: 'tools-job', cost: cost('png') });++ assert.ok(r.ok, 'png: 24 + 5 = 29 of 30');++ assert.strictEqual(r.limit, 30);++ assert.strictEqual(r.remaining, 1);++ r = q.check(anon, { quotaClass: 'tools-job', cost: cost('png') });++ assert.strictEqual(r.ok, false, 'the next png does not fit');++ assert.strictEqual(r.binding, 'minute');++ assert.strictEqual(r.retryAfter, 4, 'four more units at one a second');++ clock += 4000;++ assert.ok(q.check(anon, { quotaClass: 'tools-job', cost: cost('png') }).ok, 'refilled');++ // Tiers: a signed-in person has four times the bucket, a service forty.++ const user = { tier: 'user', key: 'user:x', ipKey: 'ip:a' };++ let n = 0; while (q.check(user, { quotaClass: 'tools-job', cost: 5 }).ok) n++;++ assert.strictEqual(n, 24, 'user burst 120 / 5');++ // Sessions from one address share SESSION_IP_SHARE × one session's allowance: dropping the cookie is no reset.++ const sessions = [1, 2, 3, 4].map(i => ({ tier: 'session', key: `session:${i}`, ipKey: 'ip:nat' }));++ const spent = sessions.map(sess => { let k = 0; while (q.check(sess, { quotaClass: 'tools-job', cost: 5 }).ok) k++; return k; });++ assert.deepStrictEqual(spent, [9, 9, 9, 0], 'three sessions use the address allowance, a fourth cookie gets nothing more');++ // A run costing more than the whole bucket still runs when the bucket is full.++ assert.ok(q.check({ tier: 'anonymous', key: 'ip:big', ipKey: 'ip:big' }, { quotaClass: 'tools-download', cost: 500 }).ok);++ }++++ // ── Report vs enforce; headers and problem+json; the metric; the challenge hook ──++ for (const mode of ['report', 'enforce']) {++ const dataDir = tmp();++ const g = makeGuard({ mode, dataDir, env: { TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-api': { anonymous: { perMinute: 2, burst: 2 } } }) } });++ const registry = metrics.createRegistry();++ g.attachMetrics(registry);++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ app.use(cookieParser());++ app.use(g.identify);++ app.use('/api/', g.apiQuota);++ app.get('/api/x', (_req, res) => res.json({ ok: true }));++ app.get('/api/probe', (req, res) => (g.target(req, res, { tool: 'ping', target: req.query.t }) ? res.json({ ok: true }) : undefined));++ const s = await serve(app);++ const ip = { 'X-Forwarded-For': '203.0.113.77' };++ try {++ const r1 = await fetch(`${s.base}/api/x`, { headers: ip });++ assert.strictEqual(r1.status, 200);++ assert.strictEqual(r1.headers.get('ratelimit-limit'), '2');++ assert.strictEqual(r1.headers.get('ratelimit-remaining'), '1');++ await fetch(`${s.base}/api/x`, { headers: ip });++ const r3 = await fetch(`${s.base}/api/x`, { headers: ip });++ if (mode === 'report') {++ assert.strictEqual(r3.status, 200, 'report mode refuses nothing');++ } else {++ assert.strictEqual(r3.status, 429);++ assert.strictEqual(r3.headers.get('content-type'), 'application/problem+json');++ assert.ok(Number(r3.headers.get('retry-after')) >= 1);++ assert.strictEqual(r3.headers.get('ratelimit-remaining'), '0');++ const body = await r3.json();++ assert.strictEqual(body.code, 'tools.quota.exceeded');++ assert.strictEqual(body.status, 429);++ assert.strictEqual(body.type, 'https://openvibe.network/problems/tools.quota.exceeded');++ assert.strictEqual(body.quota_class, 'tools-api');++ assert.strictEqual(body.tier, 'anonymous');++ assert.ok(body.error, 'the legacy { error } field is there');++ }++ // Another address is not affected.++ assert.strictEqual((await fetch(`${s.base}/api/x`, { headers: { 'X-Forwarded-For': '203.0.113.78' } })).status, 200);++ // A hard limit (the per-target throttle: ping 6 a minute) refuses in both modes.++ const codes = [];++ for (let i = 0; i < 7; i++) codes.push((await fetch(`${s.base}/api/probe?t=${i % 2 ? 'Example.COM' : 'https://example.com:443/x'}`, { headers: { 'X-Forwarded-For': `198.51.100.${i + 1}` } })).status);++ assert.deepStrictEqual(codes, [200, 200, 200, 200, 200, 200, 429], `${mode}: one target, all callers, however it is written`);++ assert.strictEqual((await fetch(`${s.base}/api/probe?t=example.org`, { headers: { 'X-Forwarded-For': '198.51.100.99' } })).status, 200, 'another target is fine');++ const rows = g.store.abuseRows();++ const quota = rows.find(r => r.reason === 'quota');++ assert.ok(quota, 'the quota refusal is in the abuse log');++ assert.strictEqual(quota.enforced, mode === 'enforce' ? 1 : 0);++ assert.ok(rows.find(r => r.reason === 'target' && r.enforced === 1 && r.tool === 'ping'));++ const text = registry.metrics();++ assert.match(text, /tools_guard_refused_total\{reason="quota",tool="-"\} [1-9]/);++ assert.match(text, /tools_guard_refused_total\{reason="target",tool="ping"\} 1/);++ assert.match(text, new RegExp(`tools_guard_enforcing ${mode === 'enforce' ? 1 : 0}`));++ } finally { await s.close(); g.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }++ }++ {++ // The challenge hook: a provider may let a person through instead of a quota refusal.++ let asked = 0;++ const g = makeGuard({ mode: 'enforce', env: { TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-api': { anonymous: { perMinute: 1, burst: 1 } } }) }, challenge: { name: 'test', required: () => { asked++; return true; }, verify: async (req) => req.headers['x-proof'] === 'ok' } });++ assert.strictEqual(makeGuard().challenge.name, 'none', 'no challenge by default');++ const app = express();++ app.use('/api/', g.apiQuota);++ app.get('/api/x', (_req, res) => res.json({ ok: true }));++ const s = await serve(app);++ try {++ assert.strictEqual((await fetch(`${s.base}/api/x`)).status, 200);++ const denied = await fetch(`${s.base}/api/x`);++ assert.strictEqual(denied.status, 403);++ assert.deepStrictEqual([(await denied.json()).code, asked], ['tools.challenge.required', 1]);++ assert.strictEqual((await fetch(`${s.base}/api/x`, { headers: { 'x-proof': 'ok' } })).status, 200, 'a passed challenge goes on');++ } finally { await s.close(); g.close(); }++ }++++ // ── Day allowances survive a restart (guard.db) ──++ {++ const dataDir = tmp();++ const env = { TOOLS_GUARD: 'enforce', TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-fetch': { anonymous: { perMinute: 600, burst: 600, perDay: 10 } } }) };++ const req = { ip: '198.51.100.20', headers: {}, cookies: {} };++ let g = makeGuard({ dataDir, env });++ const caller1 = g.resolveCaller(req);++ assert.ok(g.quotas.check(caller1, { quotaClass: 'tools-fetch', cost: 4 }).ok);++ assert.ok(g.quotas.check(caller1, { quotaClass: 'tools-fetch', cost: 4 }).ok);++ g.close();++ g = makeGuard({ dataDir, env });++ const caller2 = g.resolveCaller(req);++ assert.strictEqual(caller2.ipKey, caller1.ipKey, "today's salt is kept, so the same address is the same key after a restart");++ const r = g.quotas.check(caller2, { quotaClass: 'tools-fetch', cost: 4 });++ assert.strictEqual(r.ok, false, '8 of 10 used before the restart: 4 more do not fit');++ assert.strictEqual(r.binding, 'day');++ assert.ok(g.quotas.check(caller2, { quotaClass: 'tools-fetch', cost: 2 }).ok, 'the last 2 still do');++ g.close();++ fs.rmSync(dataDir, { recursive: true, force: true });++ }++++ // ── The abuse log: no raw address anywhere, 30 days ──++ {++ const dataDir = tmp();++ let clock = now0;++ const g = makeGuard({ dataDir, now: () => clock, mode: 'enforce' });++ const RAW = '203.0.113.201';++ const req = { ip: RAW, headers: {}, cookies: {} };++ const res = { headersSent: false, statusCode: 200, setHeader() {}, end() {} };++ for (let i = 0; i < 3; i++) g.refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'quota', tool: 'png', detail: 'x' });++ clock += 25 * 24 * 3600 * 1000;++ g.refuse({ ip: RAW, headers: {}, cookies: {} }, res, { status: 415, code: 'tools.file.unsupported_type', reason: 'sniff', tool: 'png', detail: 'x', hard: true });++ let rows = g.store.abuseRows();++ assert.strictEqual(rows.length, 2);++ assert.strictEqual(rows[0].count, 3, 'repeats within a minute are one row with a count');++ assert.match(rows[0].ip_hash, /^[0-9a-f]{32}$/);++ assert.notStrictEqual(rows[0].ip_hash, rows[1].ip_hash, 'the salt rotates daily: yesterday\'s hash no longer matches');++ g.store.db.pragma('wal_checkpoint(TRUNCATE)');++ for (const f of fs.readdirSync(dataDir)) assert.ok(!fs.readFileSync(path.join(dataDir, f)).includes(RAW), `${f} holds no raw address`);++ assert.strictEqual(g.store.db.prepare('SELECT COUNT(*) AS n FROM guard_salt').get().n, 1, 'only today\'s salt is kept');++ clock += 6 * 24 * 3600 * 1000; // the first row is now 31 days old, the second 6++ g.prune();++ rows = g.store.abuseRows();++ assert.deepStrictEqual(rows.map(r => r.reason), ['sniff'], 'rows older than 30 days are pruned');++ g.close();++ fs.rmSync(dataDir, { recursive: true, force: true });++ }++++ // ── Upload sniffing: the bytes, not the name or the declared type ──++ {++ const png = Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d4944415478da63f8ffff3f0005fe02fea7d6a4c50000000049454e44ae426082', 'hex');++ const pdf = Buffer.from('%PDF-1.7\n1 0 obj << >> endobj\ntrailer << >>\n%%EOF\n');++ const hls = Buffer.from('#EXTM3U\n#EXT-X-TARGETDURATION:10\n#EXTINF:2,\nfile:///etc/passwd\n#EXT-X-ENDLIST\n');++ const concat = Buffer.from("ffconcat version 1.0\nfile '/etc/passwd'\n");++ const wav = Buffer.concat([Buffer.from('RIFF'), Buffer.alloc(4), Buffer.from('WAVEfmt '), Buffer.alloc(32)]);++ assert.strictEqual(sniff.detect(png).mime, 'image/png');++ assert.strictEqual(sniff.detect(pdf).mime, 'application/pdf');++ assert.strictEqual(sniff.detect(wav).mime, 'audio/wav');++ assert.strictEqual(sniff.detect(hls), null, 'a playlist is never media');++ assert.strictEqual(sniff.detect(concat), null);++ assert.strictEqual(sniff.detect(Buffer.from('<?xml version="1.0"?>\n<!-- x -->\n<svg xmlns="http://www.w3.org/2000/svg"/>')).mime, 'image/svg+xml');++++ const g = makeGuard(); // report mode: sniffing refuses anyway++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ const upload = multer({ storage: multer.memoryStorage() }).single('file');++ const dir = tmp();++ const disk = multer({ storage: multer.diskStorage({ destination: dir, filename: (_q, f, cb) => cb(null, `${crypto.randomBytes(8).toString('hex')}${path.extname(f.originalname)}`) }) }).single('file');++ app.post('/img', upload, g.admitUpload(() => 'png'), (req, res) => res.json({ mime: req.file.mimetype, name: req.file.originalname }));++ app.post('/audio', disk, g.admitUpload(() => 'reverb'), (req, res) => res.json({ mime: req.file.mimetype, path: path.basename(req.file.path), name: req.file.originalname }));++ const s = await serve(app);++ const send = (p, buf, name, type) => { const f = new FormData(); f.append('file', new Blob([buf], { type }), name); return fetch(s.base + p, { method: 'POST', body: f }); };++ try {++ let r = await send('/img', pdf, 'holiday.png', 'image/png');++ assert.strictEqual(r.status, 415, 'a PDF renamed to .png is refused');++ let body = await r.json();++ assert.strictEqual(body.code, 'tools.file.unsupported_type');++ assert.strictEqual(body.detected, 'application/pdf');++ r = await send('/img', png, 'scan.pdf', 'application/pdf');++ assert.strictEqual(r.status, 200, 'a PNG with the wrong name and type is what its bytes say');++ assert.deepStrictEqual(await r.json(), { mime: 'image/png', name: 'scan.png' });++ r = await send('/audio', hls, 'song.mp3', 'audio/mpeg');++ assert.strictEqual(r.status, 415, 'an HLS playlist named .mp3 never reaches ffmpeg');++ assert.strictEqual(fs.readdirSync(dir).length, 0, 'and the refused upload is deleted');++ r = await send('/audio', wav, 'take.m3u8', 'application/octet-stream');++ body = await r.json();++ assert.strictEqual(r.status, 200);++ assert.strictEqual(body.mime, 'audio/wav');++ assert.match(body.path, /\.wav$/, 'the stored upload gets the extension of what it is');++ assert.strictEqual(body.name, 'take.wav');++ } finally { await s.close(); g.close(); fs.rmSync(dir, { recursive: true, force: true }); }++ }++++ // ── The port-scan cap (hard: report mode too) ──++ {++ const g = makeGuard();++ const res = () => { const r = { statusCode: 200, headers: {}, body: null, headersSent: false, setHeader(k, v) { this.headers[k.toLowerCase()] = v; }, end(b) { this.body = JSON.parse(b); } }; return r; };++ const reqFrom = (ip, cookie) => ({ ip, headers: {}, cookies: cookie ? { ov_tools_jobs: cookie } : {} });++ const twenty = Array.from({ length: 20 }, (_, i) => 1000 + i);++ let out = res();++ assert.strictEqual(g.portScan(reqFrom('198.51.100.40'), out, { tool: 'port', target: 'a.example', ports: [...twenty, 9999] }), false);++ assert.deepStrictEqual([out.statusCode, out.body.code], [400, 'tools.run.invalid'], 'at most 20 ports a request');++ for (let i = 0; i < 5; i++) assert.ok(g.portScan(reqFrom('198.51.100.41'), res(), { tool: 'port', target: 'a.example', ports: twenty }), `check ${i + 1}`);++ out = res();++ assert.strictEqual(g.portScan(reqFrom('198.51.100.41'), out, { tool: 'port', target: 'a.example', ports: [22] }), false, '100 ports in ten minutes, then no more');++ assert.deepStrictEqual([out.statusCode, out.body.code, out.body.scope], [429, 'tools.quota.exceeded', 'ports']);++ assert.ok(Number(out.headers['retry-after']) > 0);++ // Distinct hosts: ten, whatever cookie the browser shows (sessions count by address).++ for (let i = 0; i < 10; i++) assert.ok(g.portScan(reqFrom('198.51.100.42', crypto.randomBytes(24).toString('base64url')), res(), { tool: 'port', target: `h${i}.example`, ports: [443] }));++ out = res();++ assert.strictEqual(g.portScan(reqFrom('198.51.100.42', crypto.randomBytes(24).toString('base64url')), out, { tool: 'port', target: 'h10.example', ports: [443] }), false, 'an eleventh host is a sweep');++ assert.ok(g.portScan(reqFrom('198.51.100.42'), res(), { tool: 'port', target: 'h3.example', ports: [80] }), 'the same hosts again are fine');++ g.close();++ }++++ // ── The sync semaphore holds heavy calls to the cap ──++ {++ const sem = createSemaphore({ max: 2, queue: 3, waitMs: 200 });++ let running = 0, peak = 0;++ const work = async () => { const release = await sem.acquire(); running++; peak = Math.max(peak, running); await new Promise(r => setTimeout(r, 30)); running--; release(); };++ await Promise.all([work(), work(), work(), work(), work()]);++ assert.strictEqual(peak, 2, 'never more than two at once');++ const held = [await sem.acquire(), await sem.acquire()];++ const waiters = [sem.acquire(), sem.acquire(), sem.acquire()];++ await assert.rejects(sem.acquire(), (e) => e instanceof Busy && e.reason === 'queue', 'a full wait list refuses');++ held.forEach(r => r());++ (await Promise.all(waiters.slice(0, 2))).forEach(r => r());++ (await waiters[2])();++ const again = [await sem.acquire(), await sem.acquire()];++ await assert.rejects(sem.acquire(), (e) => e instanceof Busy && e.reason === 'timeout', 'waiting too long refuses');++ again.forEach(r => r());++++ // Over HTTP: TOOLS_SYNC_CONCURRENCY=1 runs one request at a time; with no room to wait, 503 tools.busy.++ for (const mode of ['enforce', 'report']) {++ const g = makeGuard({ mode, env: { TOOLS_SYNC_CONCURRENCY: '1', TOOLS_SYNC_QUEUE: '1' } });++ let now = 0, max = 0;++ const app = express();++ app.post('/p', g.heavy('png'), async (_req, res) => { now++; max = Math.max(max, now); await new Promise(r => setTimeout(r, 80)); now--; res.json({ ok: true }); });++ const s = await serve(app);++ try {++ const codes = await Promise.all([1, 2, 3].map(() => fetch(`${s.base}/p`, { method: 'POST' }).then(r => r.status)));++ if (mode === 'enforce') {++ assert.deepStrictEqual(codes.sort(), [200, 200, 503], 'one runs, one waits, the third is told to come back');++ assert.strictEqual(max, 1);++ } else {++ assert.deepStrictEqual(codes, [200, 200, 200], 'report mode lets the overflow run');++ assert.ok(g.store.abuseRows().some(r => r.reason === 'busy.sync' && r.enforced === 0));++ }++ } finally { await s.close(); g.close(); }++ }++ }++++ // ── The job store's bounds: queued jobs and disk (system.busy) through the guard ──++ for (const mode of ['enforce', 'report']) {++ const dataDir = tmp();++ const db = new Database(':memory:');++ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, maxQueued: 2, diskBudgetBytes: 1_000_000, log: quiet });++ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });++ system.start();++ const g = makeGuard({ mode });++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ app.use(express.json());++ app.use(cookieParser());++ jobs.mountJobRoutes(app, { system, contracts, resolveOwner: g.ownerResolver(), onBusy: (req, res, busy) => g.jobsBusy(req, res, busy) });++ const s = await serve(app);++ const submit = () => fetch(`${s.base}/api/v1/jobs`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ type: 'test.noop', input: {} }) });++ try {++ assert.strictEqual((await submit()).status, 202);++ assert.strictEqual((await submit()).status, 202);++ assert.strictEqual(system.busy().reason, 'queue');++ const r = await submit();++ if (mode === 'enforce') {++ assert.strictEqual(r.status, 503, 'a third queued job is refused');++ assert.strictEqual((await r.json()).code, 'tools.busy');++ assert.strictEqual(r.headers.get('retry-after'), '60');++ } else {++ assert.strictEqual(r.status, 202, 'report mode accepts it');++ assert.ok(g.store.abuseRows().some(x => x.reason === 'busy.queue' && x.enforced === 0));++ }++ } finally { await s.close(); g.close(); system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }++ }++ // Unfinished browser-session jobs are bounded per address too: a new cookie is no new allowance.++ for (const mode of ['enforce', 'report']) {++ const dataDir = tmp();++ const db = new Database(':memory:');++ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, maxActivePerOwner: 1, log: quiet });++ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });++ system.start();++ assert.strictEqual(system.bounds.maxActivePerAddress, 3, '3 × the per-owner bound by default');++ const g = makeGuard({ mode });++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ app.use(express.json());++ app.use(cookieParser());++ jobs.mountJobRoutes(app, {++ system, contracts, resolveOwner: g.ownerResolver(),++ onAddressFull: (req, res, full) => !g.refuse(req, res, { status: 429, code: 'tools.job.too_many_active', reason: 'jobs.address', retryAfter: 30, detail: `at most ${full.limit}` }),++ });++ const s = await serve(app);++ const submit = (ip, cookie) => fetch(`${s.base}/api/v1/jobs`, { method: 'POST', headers: { 'content-type': 'application/json', 'X-Forwarded-For': ip, ...(cookie ? { cookie } : {}) }, body: JSON.stringify({ type: 'test.noop', input: {} }) });++ try {++ const first = await submit('198.51.100.60');++ assert.strictEqual(first.status, 202);++ const cookie = String(first.headers.get('set-cookie')).split(';')[0];++ assert.strictEqual((await submit('198.51.100.60', cookie)).status, 429, 'the per-owner bound (1) as before');++ const codes = [];++ for (let i = 0; i < 3; i++) codes.push((await submit('198.51.100.60')).status); // a fresh cookie each time++ assert.deepStrictEqual(codes, mode === 'enforce' ? [202, 202, 429] : [202, 202, 202], `${mode}: three per address`);++ assert.strictEqual((await submit('198.51.100.61')).status, 202, 'another address is fine');++ const row = db.prepare('SELECT ip_key FROM tool_jobs LIMIT 1').get();++ assert.match(row.ip_key, /^ip:[0-9a-f]{32}$/, 'the job keeps the hashed key, never the address');++ if (mode === 'report') assert.ok(g.store.abuseRows().some(x => x.reason === 'jobs.address' && x.enforced === 0));++ } finally { await s.close(); g.close(); system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }++ }++ {++ const dataDir = tmp();++ const db = new Database(':memory:');++ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, diskBudgetBytes: 1000, log: quiet });++ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });++ system.start();++ await system.refreshDisk();++ assert.strictEqual(system.busy(), null);++ await system.submit({ owner: 'session:x', type: 'test.noop', input: {}, files: [{ buffer: Buffer.alloc(1500), name: 'a.bin', mime: 'application/octet-stream', size: 1500 }] });++ assert.strictEqual(system.busy().reason, 'disk', 'accepted inputs count against the disk budget at once');++ assert.strictEqual((await system.refreshDisk()) >= 1500, true, 'and the walk finds them');++ system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true });++ }++++ console.log('guard: callers (/64, one loopback hop, services by token, aud-checked users, sessions, sandbox), bucket math + cost, session sharing, report vs enforce, headers + problem+json, day counters across restarts, sniffing, target throttle, port cap, abuse log + prune + metric, semaphore, job bounds (queue, disk, per address), challenge hook: all checks passed');++})().catch((err) => { console.error(err); process.exit(1); });@@@ -71,7 +71,9 @@ function assertShape(body, service) {const fd = new FormData();fd.append('type', 'img.process');fd.append('input', JSON.stringify({ tool: 'convert', format: 'webp' }));-- fd.append('file', new Blob([Buffer.from('not really a png')], { type: 'image/png' }), 'a.png');++ // A real 1×1 PNG: uploads are sniffed (apps/_shared/guard), so bytes must be what they claim.++ const onePx = Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d4944415478da63f8ffff3f0005fe02fea7d6a4c50000000049454e44ae426082', 'hex');++ fd.append('file', new Blob([onePx], { type: 'image/png' }), 'a.png');const sub = await fetch(`${img.base}/api/v1/jobs`, { method: 'POST', body: fd });assert.strictEqual(sub.status, 202, await sub.clone().text());const job = await sub.json();ddeleted file mode 100644@@@ -1,120 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Audio.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)--// Lightweight auth — no DB, no linked_accounts.--// Verifies the shared ov_token cookie offline against the--// Network's public key and attaches decoded claims to req.user.--// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise--// fetched from the Network's /api/.well-known/jwks endpoint.--// ═══════════════════════════════════════════════════════════════----const jwt = require('jsonwebtoken');--const fs = require('fs');--const path = require('path');--const config = require('./config');----let publicKey = null;--let retryTimer = null;----const ISSUER = config.networkUrl;--const JWKS_PATH = '/api/.well-known/jwks';--const FETCH_TIMEOUT_MS = 5000;--const RETRY_INTERVAL_MS = 60 * 1000;----function loadKeyFromFile() {-- for (const p of config.publicKeyPaths) {-- try {-- const resolved = path.resolve(p);-- if (fs.existsSync(resolved)) {-- publicKey = fs.readFileSync(resolved, 'utf8');-- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);-- return true;-- }-- } catch { /* try next */ }-- }-- return false;--}----async function fetchKeyFromNetwork() {-- for (const base of [config.networkInternalUrl, config.networkUrl]) {-- try {-- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });-- if (!res.ok) continue;-- const body = await res.json();-- if (body && body.public_key) {-- publicKey = body.public_key;-- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);-- return true;-- }-- } catch { /* try next base */ }-- }-- return false;--}----function loadPublicKey() {-- if (loadKeyFromFile()) return;-- fetchKeyFromNetwork().then((loaded) => {-- if (loaded || retryTimer) return;-- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);-- retryTimer = setInterval(async () => {-- if (await fetchKeyFromNetwork()) {-- clearInterval(retryTimer);-- retryTimer = null;-- }-- }, RETRY_INTERVAL_MS);-- if (retryTimer.unref) retryTimer.unref();-- });--}--loadPublicKey();----function extractToken(req) {-- const authHeader = req.headers.authorization;-- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);-- if (req.cookies?.ov_token) return req.cookies.ov_token;-- if (req.cookies?.token) return req.cookies.token;-- return null;--}----function verifyToken(token) {-- if (!publicKey || !token) return null;-- try {-- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });-- } catch {-- return null;-- }--}----/**-- * Optional auth — attaches req.user if valid token present.-- * Never blocks the request; anonymous users get req.user = null.-- */--function optionalAuth(req, _res, next) {-- const token = extractToken(req);-- if (token) {-- const decoded = verifyToken(token);-- if (decoded) {-- req.user = decoded;-- req.token = token;-- }-- }-- next();--}----/**-- * Required auth — 401 if no valid token.-- */--function requireAuth(req, res, next) {-- const token = extractToken(req);-- if (!token) return res.status(401).json({ error: 'Authentication required' });-- const decoded = verifyToken(token);-- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });-- req.user = decoded;-- req.token = token;-- next();--}----/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */--function getPublicKey() { return publicKey; }----module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };@@@ -14,11 +14,17 @@ const { DOMAIN_MAP } = require('./domain-map');// tools/convert.js FORMAT_CONFIG and tools/ffmpeg-helper.js FORMAT_MIME.const FORMAT_MIME = { mp3: 'audio/mpeg', wav: 'audio/wav', flac: 'audio/flac', ogg: 'audio/ogg', m4a: 'audio/mp4', aac: 'audio/aac', opus: 'audio/opus', wma: 'audio/x-ms-wma', aiff: 'audio/aiff', ac3: 'audio/ac3', webm: 'audio/webm' };const FORMATS = Object.keys(FORMAT_MIME);--// config.js upload.allowedMimes (video types are there for the extractor; octet-stream for mis-typed uploads).--const ACCEPT = ['audio/mpeg', 'audio/mp3', 'audio/wav', 'audio/x-wav', 'audio/wave', 'audio/flac', 'audio/x-flac', 'audio/ogg', 'audio/vorbis', 'audio/mp4', 'audio/x-m4a', 'audio/aac', 'audio/opus', 'audio/x-ms-wma', 'audio/aiff', 'audio/x-aiff', 'audio/ac3', 'audio/webm', 'audio/amr', 'video/mp4', 'video/webm', 'video/x-matroska', 'video/avi', 'video/x-msvideo', 'video/quicktime', 'video/x-flv', 'video/ogg', 'application/ogg', 'application/octet-stream'];++// What the bytes may be (checked against the bytes by the guard, apps/_shared/guard/sniff.js; the upload++// itself also lets application/octet-stream through, since browsers send it for audio they do not know).++// Every operation reads audio containers, including the video containers that carry audio (MP4, WebM,++// Matroska, QuickTime, Ogg); only the extractor also reads AVI and FLV (guard/ffmpeg.js formatsFor).++const AUDIO_ACCEPT = ['audio/mpeg', 'audio/mp3', 'audio/wav', 'audio/x-wav', 'audio/wave', 'audio/flac', 'audio/x-flac', 'audio/ogg', 'audio/vorbis', 'audio/mp4', 'audio/x-m4a', 'audio/aac', 'audio/opus', 'audio/x-ms-wma', 'audio/aiff', 'audio/x-aiff', 'audio/ac3', 'audio/webm', 'audio/amr', 'video/mp4', 'video/webm', 'video/x-matroska', 'video/quicktime', 'video/ogg', 'application/ogg'];++const ACCEPT = [...AUDIO_ACCEPT, 'video/avi', 'video/x-msvideo', 'video/x-flv'];const MAX_BYTES = 100 * 1024 * 1024; // config.js upload.maxFileSizeconst TIMEOUT_MS = 15 * 60 * 1000; // process.js audio.process timeoutMsconst MERGE_MAX_SECONDS = 3 * 60 * 60; // tools/merge.js MAX_TOTAL_SECONDS++// Longest input any operation reads (process.js probes it first; ffmpeg reads no more than this, -t).++const MAX_SECONDS = Math.max(1, parseInt(process.env.AUDIO_MAX_DURATION, 10) || 3 * 60 * 60);// An effect keeps the input's own format unless `format` names one (ffmpeg-helper getMime: amr too, anything else octet-stream).const ALL_OUT = [...new Set([...Object.values(FORMAT_MIME), 'audio/amr', 'application/octet-stream'])];@@@ -85,9 +91,9 @@ function tool(id, extra = {}) {job: { type: 'audio.process', operation: op, ...(fixed && { preset: { format: ctx.defaultFormat } }) },legacy: [op === 'merge' ? 'POST /api/process/multi' : 'POST /api/process'],input: INPUT[op](),-- files: op === 'merge' ? { min: 2, max: 5, accept: ACCEPT, maxBytes: MAX_BYTES } : { min: 1, max: 1, accept: ACCEPT, maxBytes: MAX_BYTES },++ files: { min: op === 'merge' ? 2 : 1, max: op === 'merge' ? 5 : 1, accept: op === 'extract' ? ACCEPT : AUDIO_ACCEPT, maxBytes: MAX_BYTES },output: { kind: 'file', mime, schema: RESULT },-- limits: { timeoutMs: TIMEOUT_MS, ...(op === 'merge' && { maxDurationSec: MERGE_MAX_SECONDS }) },++ limits: { timeoutMs: TIMEOUT_MS, maxDurationSec: op === 'merge' ? Math.min(MERGE_MAX_SECONDS, MAX_SECONDS) : MAX_SECONDS },// ffmpeg over an upload is heavy work: a browser session (the page has one), a person or a token.auth: { anonymous: false, capability: 'tools.tool.run' },quotaClass: 'tools-job', cost: op === 'merge' ? 20 : op === 'metadata' || op === 'waveform' ? 5 : 10, egress: false,@@@ -100,4 +106,4 @@ function tool(id, extra = {}) {const IDS = ['mp3', 'wav', 'flac', 'ogg', 'm4a', 'opus', 'aac', 'wma', 'aiff', 'ac3', 'extract', 'trim', 'merge', 'ringtone', 'speed', 'pitch', 'reverse', 'normalize', 'noise', 'silence', 'vocal', 'fade', 'loop', 'bass', 'equalizer', 'compressor', 'podcast', 'voice', 'echo', 'reverb', 'chorus', 'distortion', 'bitcrusher', 'stereo', 'waveform', 'metadata'];const SPECS = IDS.map(id => tool(id));--module.exports = { SPECS, FORMAT_MIME, ACCEPT, MAX_BYTES, TIMEOUT_MS };++module.exports = { SPECS, FORMAT_MIME, ACCEPT, AUDIO_ACCEPT, MAX_BYTES, TIMEOUT_MS, MAX_SECONDS };@@@ -13,21 +13,20 @@ const cookieParser = require('cookie-parser');const path = require('path');const fs = require('fs');++const rateLimit = require('express-rate-limit');const config = require('./config');--const auth = require('./auth');--const { optionalAuth } = auth;const { resolveContext, DOMAIN_MAP } = require('./domain-map');const { getTool, listTools } = require('./tools');const { readMetadata } = require('./tools/metadata');const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');--const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');const retention = require('./retention/manager');const { probe, getDuration, cleanTmp } = require('./tools/ffmpeg-helper');--const { buildOptions, describe, defineJobs } = require('./process');++const { buildOptions, describe, defineJobs, runSync, jobContext, limitsFor } = require('./process');const { hostGuard, ownHost } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');const jobsRuntime = require('../../_shared/jobs');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const contracts = require('openvibe-contracts');const sdk = require('openvibe-sdk');@@@ -41,13 +40,29 @@ const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-audio', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else++// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the++// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse. Every audio++// tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).++const SPECS = require('./descriptors').SPECS;++const guard = createGuard({++ app: 'audio', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,++ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,++});++const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);++/** The tool a request is for before its body is read: the host's own, else the operation it defaults to. */++const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('audio.process', req.ctx.defaultOp || 'convert', null) || { id: null }).id);++/** The tool a parsed request runs: its `tool` (operation) on this host. */++const toolOf = (req) => { const d = guard.toolForJob('audio.process', String((req.body && req.body.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : hostTool(req); };++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready, which } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-audio', release: release.release,checks: [@@@ -57,19 +72,20 @@ observe({ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),-- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),++ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),...jobsRuntime.readyChecks(() => jobs),ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'every audio tool runs ffmpeg; without it jobs and conversions fail' }),],jobs: () => jobs,});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));// ── Security ─────────────────────────────────────────────────--app.set('trust proxy', 2); // Cloudflare → Nginx → Node++app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only addressapp.use(helmet({contentSecurityPolicy: {directives: {@@@ -101,8 +117,9 @@ app.use(exceptRegistry(cors({credentials: true,})));--// ── Rate Limiting ────────────────────────────────────────────--app.use('/api/', apiLimiter);++// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──++app.use(guard.identify);++app.use('/api/', apiLimiter, guard.apiQuota);// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's audio tools ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -113,9 +130,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));// ── Analytics Middleware ─────────────────────────────────────app.use(analytics.middleware());--// ── Auth (optional on all routes) ────────────────────────────--app.use(optionalAuth);--// ── Hosts ────────────────────────────────────────────────────// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the@@@ -137,8 +151,11 @@ app.get('/api/health', (_req, res) => {res.json({ status: 'ok', service: 'openvibe-audio', version: '1.0.0', files: stats, jobs: jobs.stats() });});--// Domain context (frontend calls this on load to get branding)++// Domain context (frontend calls this on load to get branding). It also starts this browser's session++// (ov_tools_jobs): audio tools need one (or a sign-in, or a token) before the first upload.app.get('/api/context', (req, res) => {++ guard.ensureSession(req, res);++ res.set('Cache-Control', 'private, no-store');const ctx = req.ctx;const tools = listTools();res.json({@@@ -160,10 +177,10 @@ app.get('/api/tools', (_req, res) => {});// ── Probe / Metadata Endpoint ────────────────────────────────--app.post('/api/probe', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/probe', burstLimiter, processLimiter, guard.toolQuota('metadata'), uploadSingle, guard.admitUpload(() => 'metadata'), async (req, res) => {try {-- const info = await probe(req.file.path);-- const meta = await readMetadata(req.file.path);++ // Hardened like every run: local files only, audio demuxers only.++ const { info, meta } = await jobContext.run(limitsFor('metadata'), async () => ({ info: await probe(req.file.path), meta: await readMetadata(req.file.path) }));cleanTmp(req.file.path);res.json({ success: true, ...meta, probe: info.format });} catch (err) {@@@ -173,7 +190,7 @@ app.post('/api/probe', burstLimiter, processLimiter, uploadSingle, async (req, r});// ── Main Processing Endpoint ─────────────────────────────────--app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {try {const toolId = req.body.tool || req.ctx.defaultOp || 'convert';const tool = getTool(toolId);@@@ -186,8 +203,8 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,return res.status(400).json({ error: `${tool.label} takes several files: send them in "files" to /api/process/multi` });}-- // Execute the tool (same options and code path as the audio.process job)-- const result = await tool.handler(req.file.path, buildOptions(req.body, toolId, req.ctx));++ // Execute the tool (same options, hardening and code path as the audio.process job; killed at its timeout)++ const result = await runSync(res, toolId, req.file.path, buildOptions(req.body, toolId, req.ctx));// Clean up the uploaded input filecleanTmp(req.file.path);@@@ -207,13 +224,14 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize) });} catch (err) {cleanTmp(req.file?.path);++ if (guard.toolRefused(req, res, err, toolOf(req))) return;console.error('[Process] Error:', err.message);res.status(422).json({ error: err.message || 'Audio processing failed' });}});// ── Multi-File Processing Endpoint (merge) ───────────────────--app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, async (req, res) => {++app.post('/api/process/multi', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadMultiple, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {const paths = req.files.map(f => f.path);try {const toolId = req.body.tool || req.ctx.defaultOp;@@@ -222,7 +240,7 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asycleanTmp(...paths);return res.status(400).json({ error: tool ? `Tool "${toolId}" takes one file: use /api/process` : `Unknown tool: ${toolId}` });}-- const result = await tool.handler(paths, buildOptions(req.body, toolId, req.ctx));++ const result = await runSync(res, toolId, req.files.map(f => f.path), buildOptions(req.body, toolId, req.ctx));cleanTmp(...paths);const first = req.files[0].originalname || 'audio';const saved = retention.saveOutputFromFile(result.outputPath, result.ext, result.mime, !!req.user,@@@ -230,7 +248,8 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asyconst inputSize = req.files.reduce((n, f) => n + f.size, 0);res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize), fileCount: req.files.length });} catch (err) {-- cleanTmp(...paths);++ cleanTmp(...req.files.map(f => f.path));++ if (guard.toolRefused(req, res, err, toolOf(req))) return;console.error('[Process/Multi] Error:', err.message);res.status(422).json({ error: err.message || 'Audio processing failed' });}@@@ -242,10 +261,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy// after a reload or a restart (apps/_shared/jobs).const jobs = jobsRuntime.setupJobs({app, service: 'audio', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,-- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,++ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,define: defineJobs,receive: uploadAny,-- limiters: [burstLimiter, processLimiter],++ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],++ jobTool: (req, type, input) => { const d = guard.toolForJob('audio.process', String((input && input.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : null; },defaults(req, input) {const out = { ...input };if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';@@@ -355,6 +375,7 @@ function shutdown() {analytics.destroy();analyticsDb.close();jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}ddeleted file mode 100644@@@ -1,51 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Audio.OpenVibe — Rate Limiting Middleware--// Tiered rate limits: anonymous vs authenticated users.--// Includes anti-abuse burst protection and stricter anon limits.--// ═══════════════════════════════════════════════════════════════----const rateLimit = require('express-rate-limit');----/** General API rate limit */--const apiLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 120 : 60,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Too many requests. Please try again later.' },--});----/**-- * Processing rate limit — stricter, tiered by auth.-- * Audio processing is more CPU-intensive than image processing,-- * so limits are a bit tighter.-- * Anonymous: 6 conversions/minute-- * Authenticated: 20 conversions/minute-- */--const processLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 20 : 6,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },--});----/**-- * Anti-burst limiter — prevents rapid-fire abuse (bot attacks).-- * Very short window: max 3 requests per 5 seconds per IP.-- * Applies to processing endpoints only.-- */--const burstLimiter = rateLimit({-- windowMs: 5 * 1000,-- max: 3,-- standardHeaders: false,-- legacyHeaders: false,-- keyGenerator: (req) => req.ip,-- message: { error: 'Too many requests in quick succession. Please slow down.' },--});----module.exports = { apiLimiter, processLimiter, burstLimiter };@@@ -6,8 +6,14 @@//// Jobs can be cancelled and report progress although the tools build their own ffmpeg commands:// FfmpegCommand#run is wrapped once, and a command started inside a job's AsyncLocalStorage--// context reports ffmpeg's percent to the job and is killed when the job's signal aborts.--// Commands started outside a job (the synchronous endpoint) are untouched.++// context reports ffmpeg's percent to the job and is killed when the job's signal aborts. The++// synchronous endpoint runs in the same kind of context (runSync): a deadline (the descriptor's++// limits.timeoutMs) and a client that goes away abort it, and ffmpeg is killed.++//++// Hardening (apps/_shared/guard/ffmpeg.js, in both paths): every input of every command and every++// probe reads only local files and pipes, only through the operation's demuxers (never hls, concat,++// image2, lavfi…), and at most limits.maxDurationSec of it; inputs are probed first and a longer one++// is refused before any work starts.// ═══════════════════════════════════════════════════════════════const fs = require('fs');@@@ -15,21 +21,36 @@ const path = require('path');const { AsyncLocalStorage } = require('async_hooks');const ffmpeg = require('fluent-ffmpeg');const { getTool, listTools } = require('./tools');++const { probe } = require('./tools/ffmpeg-helper');++const hardening = require('../../_shared/guard/ffmpeg');++const { SPECS } = require('./descriptors');const jobContext = new AsyncLocalStorage();const RESERVED = new Set(['tool', 'file', 'files']);++/** An operation's limits from its descriptor (every audio tool on one operation has the same). */++const LIMITS = new Map();++for (const s of SPECS) if (!LIMITS.has(s.job.operation)) LIMITS.set(s.job.operation, s.limits);++function limitsFor(op) {++ const l = LIMITS.get(op) || LIMITS.get('convert') || {};++ return { formats: hardening.formatsFor(op), maxDurationSec: l.maxDurationSec || null, timeoutMs: l.timeoutMs || 15 * 60 * 1000 };++}++++// Every command and probe: the whitelists and the duration cap of the operation running (a command++// outside any context gets the widest audio list).++hardening.harden(ffmpeg, () => jobContext.getStore() || { formats: hardening.formatsFor('extract') });++const proto = ffmpeg.prototype;if (!proto.__ovJobs) {const run = proto.run;proto.run = function runInJob(...args) {const job = jobContext.getStore();-- if (job) {++ if (job && job.signal) {const kill = () => {if (this.ffmpegProc) { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* already gone */ } } else this.once('start', () => { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* gone */ } });};if (job.signal.aborted) kill(); else job.signal.addEventListener('abort', kill, { once: true });-- this.on('progress', (p) => { if (p && Number.isFinite(p.percent)) job.progress(10 + Math.min(100, Math.max(0, p.percent)) * 0.8, 'Encoding'); });++ if (job.progress) this.on('progress', (p) => { if (p && Number.isFinite(p.percent)) job.progress(10 + Math.min(100, Math.max(0, p.percent)) * 0.8, 'Encoding'); });}return run.apply(this, args);};@@@ -55,6 +76,38 @@ function describe(toolId, result, size, inputSize) {};}++/** Probe the inputs (hardened) and run the tool inside the current context. */++async function runTool(toolId, arg, options) {++ const tool = getTool(toolId);++ const lim = jobContext.getStore() || limitsFor(toolId);++ await hardening.probeInputs(probe, Array.isArray(arg) ? arg : [arg], lim);++ return tool.handler(arg, options);++}++++/**++ * The synchronous endpoint's run: the job context without a job — hardened, killed at the++ * descriptor's timeout (504 tools.run.timeout) or when the client goes away.++ */++async function runSync(res, toolId, arg, options, override = {}) {++ const lim = { ...limitsFor(toolId), ...override };++ const ctrl = new AbortController();++ let why = null;++ const timer = setTimeout(() => { why = 'timeout'; ctrl.abort(); }, lim.timeoutMs);++ const gone = () => { if (!res.writableFinished && !why) { why = 'client'; ctrl.abort(); } };++ res.on('close', gone);++ try {++ return await jobContext.run({ signal: ctrl.signal, progress: () => {}, ...lim }, () => runTool(toolId, arg, options));++ } catch (err) {++ if (why === 'timeout') {++ throw Object.assign(new Error(`This took longer than ${Math.round(lim.timeoutMs / 60000)} minutes and was stopped.`), { status: 504, code: 'tools.run.timeout', guardReason: 'timeout', expose: true });++ }++ throw err;++ } finally {++ clearTimeout(timer);++ res.removeListener('close', gone);++ }++}++/** Job type audio.process: input { tool, …options } + one audio or video file (2–5 for merge). */function defineJobs(system) {const known = new Set(listTools().map(t => t.id));@@@ -84,7 +137,7 @@ function defineJobs(system) {const src = files[0];progress(5, files.length > 1 ? `Reading ${files.length} files` : 'Reading the audio');const arg = tool.multiFile ? files.map(f => f.path) : src.path;-- const result = await jobContext.run({ signal, progress }, () => tool.handler(arg, buildOptions(input, toolId, null)));++ const result = await jobContext.run({ signal, progress, ...limitsFor(toolId) }, () => runTool(toolId, arg, buildOptions(input, toolId, null)));if (signal.aborted) { fs.rm(result.outputPath, { force: true }, () => {}); throw new Error('cancelled'); }const size = fs.statSync(result.outputPath).size;const base = path.basename(src.name || 'audio', path.extname(src.name || '')) || 'openvibeaudio-output';@@@ -97,4 +150,4 @@ function defineJobs(system) {});}--module.exports = { buildOptions, describe, defineJobs, jobContext };++module.exports = { buildOptions, describe, defineJobs, jobContext, runSync, runTool, limitsFor };nnew file mode 100644@@@ -0,0 +1,111 @@++'use strict';++// ffmpeg hardening (apps/_shared/guard/ffmpeg.js through server/process.js): an HLS playlist or an++// ffconcat list that points at a local file is refused — by the upload sniffing at the door, and by++// ffmpeg's own protocol and format whitelists if one ever got further; inputs longer than the++// descriptor's maxDurationSec are refused before work starts (and -t caps what is read); a++// synchronous run is killed at its deadline.++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { EventEmitter } = require('events');++const { execFileSync } = require('child_process');++const { startApp } = require('../../_shared/test/spawn');++++try { execFileSync('ffmpeg', ['-version'], { stdio: 'ignore' }); } catch {++ console.log('audio hardening: SKIPPED (no ffmpeg on this machine)');++ process.exit(0);++}++++(async () => {++ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-audio-guard-'));++ process.env.DATA_DIR = data; // the tools' temp files go here++ const ff = (...args) => execFileSync('ffmpeg', ['-loglevel', 'error', '-y', ...args]);++ const secret = path.join(data, 'secret.wav');++ ff('-f', 'lavfi', '-i', 'sine=frequency=440:duration=2', secret);++ const hls = path.join(data, 'evil.m3u8');++ fs.writeFileSync(hls, `#EXTM3U\n#EXT-X-TARGETDURATION:10\n#EXTINF:2,\nfile://${secret}\n#EXT-X-ENDLIST\n`);++ const concat = path.join(data, 'evil.ffconcat');++ fs.writeFileSync(concat, "ffconcat version 1.0\nfile 'secret.wav'\n");++ // The attack is real on this ffmpeg: unhardened, the playlist reads the local file.++ const plain = execFileSync('ffprobe', ['-v', 'error', '-show_entries', 'format=format_name,duration', '-of', 'csv=p=0', hls]).toString();++ assert.match(plain, /hls/, 'without the whitelists ffprobe follows the playlist');++++ const { jobContext, runTool, runSync, limitsFor } = require('../server/process');++ const { getTool } = require('../server/tools');++ const hardening = require('../../_shared/guard/ffmpeg');++ const ffmpeg = require('fluent-ffmpeg');++++ // ── Every command's inputs carry the whitelists and the duration cap ──++ const lim = limitsFor('convert');++ assert.strictEqual(lim.maxDurationSec, 3 * 60 * 60, 'the descriptor\'s limits.maxDurationSec');++ assert.ok(!lim.formats.includes('hls') && !lim.formats.includes('concat'));++ assert.ok(limitsFor('extract').formats.includes('avi') && !lim.formats.includes('avi'), 'only the extractor opens video-only containers');++ const cmd = ffmpeg(secret).noVideo().output(path.join(data, 'x.mp3'));++ hardening.applyToCommand(cmd, lim);++ const args = cmd._getArguments();++ const at = args.indexOf('-i');++ assert.deepStrictEqual(args.slice(at - 6, at), ['-protocol_whitelist', 'file,pipe', '-format_whitelist', lim.formats.join(','), '-t', '10800'], 'input options before -i');++++ // ── The playlist and the concat list are refused, at the probe and by ffmpeg itself ──++ for (const evil of [hls, concat]) {++ await assert.rejects(jobContext.run(lim, () => runTool('convert', evil, { format: 'mp3' })),++ (e) => e.status === 415 && e.code === 'tools.file.unsupported_type' && e.guardReason === 'ffmpeg.format', `${path.basename(evil)}: refused before work starts`);++ // Even the tool's own ffmpeg command (no probe first) cannot open it.++ await assert.rejects(jobContext.run(lim, () => getTool('convert').handler(evil, { format: 'mp3' })), /whitelist|Invalid argument|Invalid data/i, `${path.basename(evil)}: ffmpeg refuses it`);++ }++ const ok = await jobContext.run(lim, () => runTool('convert', secret, { format: 'mp3' }));++ assert.strictEqual(ok.ext, 'mp3', 'a real WAV converts as before');++ fs.rmSync(ok.outputPath, { force: true });++++ // ── Longer than the descriptor allows: refused before work starts ──++ await assert.rejects(jobContext.run({ ...lim, maxDurationSec: 1 }, () => runTool('convert', secret, { format: 'mp3' })),++ (e) => e.status === 413 && e.guardReason === 'ffmpeg.duration' && /2 seconds long; this tool takes at most 1 seconds/.test(e.message));++++ // ── A synchronous run is killed at its deadline ──++ const long = path.join(data, 'long.wav');++ ff('-f', 'lavfi', '-i', 'sine=frequency=220:duration=600', '-ar', '44100', long);++ const res = Object.assign(new EventEmitter(), { writableFinished: false });++ const t0 = Date.now();++ await assert.rejects(runSync(res, 'reverb', long, {}, { timeoutMs: 150 }), (e) => e.status === 504 && e.code === 'tools.run.timeout' && e.guardReason === 'timeout');++ assert.ok(Date.now() - t0 < 5000, `stopped quickly (${Date.now() - t0} ms)`);++ // A client that goes away stops it too.++ const gone = Object.assign(new EventEmitter(), { writableFinished: false });++ const run = runSync(gone, 'reverb', long, {});++ setTimeout(() => gone.emit('close'), 150);++ await assert.rejects(run, (e) => e.code !== 'tools.run.timeout');++++ // ── Through the app: sniffing turns the playlist away at the door ──++ const env = { DATA_DIR: data, UPLOADS_DIR: path.join(data, 'uploads'), OUTPUT_DIR: path.join(data, 'output') };++ const app = await startApp('audio', env);++ try {++ const ctx = await fetch(`${app.base}/api/context`);++ const cookie = String(ctx.headers.get('set-cookie') || '').split(';')[0];++ assert.match(cookie, /^ov_tools_jobs=/, 'the page\'s context call starts the session audio tools need');++ const post = (buf, name, type, tool = 'convert') => {++ const f = new FormData();++ f.append('file', new Blob([buf], { type }), name);++ f.append('tool', tool);++ f.append('format', 'mp3');++ return fetch(`${app.base}/api/process`, { method: 'POST', body: f, headers: { cookie } });++ };++ let r = await post(fs.readFileSync(hls), 'song.mp3', 'audio/mpeg');++ assert.strictEqual(r.status, 415);++ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');++ r = await post(fs.readFileSync(secret), 'take.wav', 'application/octet-stream');++ const body = await r.json();++ assert.strictEqual(r.status, 200, JSON.stringify(body));++ assert.strictEqual(body.output.ext, 'mp3');++ // A job with the playlist is refused the same way.++ const f = new FormData();++ f.append('type', 'audio.process');++ f.append('input', JSON.stringify({ tool: 'convert', format: 'mp3' }));++ f.append('file', new Blob([fs.readFileSync(concat)], { type: 'audio/mpeg' }), 'list.mp3');++ r = await fetch(`${app.base}/api/v1/jobs`, { method: 'POST', body: f, headers: { cookie } });++ assert.strictEqual(r.status, 415);++ } finally {++ await app.kill();++ fs.rmSync(data, { recursive: true, force: true });++ }++ console.log('audio hardening: playlist and concat refused (sniffing, probe, ffmpeg whitelists), duration cap and -t, sync deadline and client-gone kill: all checks passed');++})().catch((err) => { console.error(err); process.exit(1); });ddeleted file mode 100644@@@ -1,120 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Docs.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)--// Lightweight auth — no DB, no linked_accounts.--// Verifies the shared ov_token cookie offline against the--// Network's public key and attaches decoded claims to req.user.--// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise--// fetched from the Network's /api/.well-known/jwks endpoint.--// ═══════════════════════════════════════════════════════════════----const jwt = require('jsonwebtoken');--const fs = require('fs');--const path = require('path');--const config = require('./config');----let publicKey = null;--let retryTimer = null;----const ISSUER = config.networkUrl;--const JWKS_PATH = '/api/.well-known/jwks';--const FETCH_TIMEOUT_MS = 5000;--const RETRY_INTERVAL_MS = 60 * 1000;----function loadKeyFromFile() {-- for (const p of config.publicKeyPaths) {-- try {-- const resolved = path.resolve(p);-- if (fs.existsSync(resolved)) {-- publicKey = fs.readFileSync(resolved, 'utf8');-- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);-- return true;-- }-- } catch { /* try next */ }-- }-- return false;--}----async function fetchKeyFromNetwork() {-- for (const base of [config.networkInternalUrl, config.networkUrl]) {-- try {-- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });-- if (!res.ok) continue;-- const body = await res.json();-- if (body && body.public_key) {-- publicKey = body.public_key;-- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);-- return true;-- }-- } catch { /* try next base */ }-- }-- return false;--}----function loadPublicKey() {-- if (loadKeyFromFile()) return;-- fetchKeyFromNetwork().then((loaded) => {-- if (loaded || retryTimer) return;-- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);-- retryTimer = setInterval(async () => {-- if (await fetchKeyFromNetwork()) {-- clearInterval(retryTimer);-- retryTimer = null;-- }-- }, RETRY_INTERVAL_MS);-- if (retryTimer.unref) retryTimer.unref();-- });--}--loadPublicKey();----function extractToken(req) {-- const authHeader = req.headers.authorization;-- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);-- if (req.cookies?.ov_token) return req.cookies.ov_token;-- if (req.cookies?.token) return req.cookies.token;-- return null;--}----function verifyToken(token) {-- if (!publicKey || !token) return null;-- try {-- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });-- } catch {-- return null;-- }--}----/**-- * Optional auth — attaches req.user if valid token present.-- * Never blocks the request; anonymous users get req.user = null.-- */--function optionalAuth(req, _res, next) {-- const token = extractToken(req);-- if (token) {-- const decoded = verifyToken(token);-- if (decoded) {-- req.user = decoded;-- req.token = token;-- }-- }-- next();--}----/**-- * Required auth — 401 if no valid token.-- */--function requireAuth(req, res, next) {-- const token = extractToken(req);-- if (!token) return res.status(401).json({ error: 'Authentication required' });-- const decoded = verifyToken(token);-- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });-- req.user = decoded;-- req.token = token;-- next();--}----/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */--function getPublicKey() { return publicKey; }----module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };@@@ -13,19 +13,19 @@ const cookieParser = require('cookie-parser');const path = require('path');const fs = require('fs');++const fsp = require('fs/promises');++const rateLimit = require('express-rate-limit');const config = require('./config');--const auth = require('./auth');--const { optionalAuth } = auth;const { resolveContext, DOMAIN_MAP } = require('./domain-map');const { getTool, listTools, BINARIES } = require('./tools');const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');--const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');const retention = require('./retention/manager');const { buildOptions, defineJobs } = require('./process');const { hostGuard, ownHost } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');const jobsRuntime = require('../../_shared/jobs');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const contracts = require('openvibe-contracts');const sdk = require('openvibe-sdk');@@@ -39,13 +39,33 @@ const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-docs', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else++// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the++// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse. Every PDF++// tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).++const SPECS = require('./descriptors').SPECS;++const guard = createGuard({++ app: 'docs', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,++ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,++});++const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);++/** The tool a request is for before its body is read: the host's own (the hubs: merge, the first). */++const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('docs.process', req.ctx.defaultOp || 'merge', null) || { id: null }).id);++/** The tool a parsed request runs: its `tool` (operation) on this host. */++const toolOf = (req) => { const op = String((req.body && req.body.tool) || req.ctx.defaultOp || ''); const d = op ? guard.toolForJob('docs.process', op, req.ctx.toolId) : null; return d ? d.id : hostTool(req); };++/** Read the uploaded files (on disk) and delete them. */++async function readUploads(files) {++ try { return await Promise.all(files.map(f => fsp.readFile(f.path))); } finally { for (const f of files) fs.unlink(f.path, () => {}); }++}++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-docs', release: release.release,checks: [@@@ -55,19 +75,20 @@ observe({ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),-- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),++ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),...BINARIES.map(b => b.readyCheck(b.name === 'qpdf' ? 'Protect and Unlock PDF (package qpdf); the other tools work without it' : 'PDF to image (package poppler-utils); the other tools work without it')),...jobsRuntime.readyChecks(() => jobs),],jobs: () => jobs,});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));// ── Security ─────────────────────────────────────────────────--app.set('trust proxy', 2); // Cloudflare → Nginx → Node++app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only addressapp.use(helmet({contentSecurityPolicy: {directives: {@@@ -98,8 +119,9 @@ app.use(exceptRegistry(cors({credentials: true,})));--// ── Rate Limiting ────────────────────────────────────────────--app.use('/api/', apiLimiter);++// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──++app.use(guard.identify);++app.use('/api/', apiLimiter, guard.apiQuota);// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's PDF tools ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -111,9 +133,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));// ── Analytics Middleware ─────────────────────────────────────app.use(analytics.middleware());--// ── Auth (optional on all routes) ────────────────────────────--app.use(optionalAuth);--// ── Hosts ────────────────────────────────────────────────────// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the@@@ -131,6 +150,7 @@ app.use((req, _res, next) => {/** A tool's error on the synchronous endpoints: 503 problem when a tool is not set up, else { error }. */function sendToolError(req, res, err, label) {++ if (guard.toolRefused(req, res, err, toolOf(req))) return undefined;if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code || 'tools.unavailable', { detail: err.message, ctx: req.ov });if (!err.expose) console.error(`[${label}] Error:`, err.message);const status = Number.isInteger(err.status) && err.status >= 400 && err.status < 500 ? err.status : 422;@@@ -143,8 +163,11 @@ app.get('/api/health', (_req, res) => {res.json({ status: 'ok', service: 'openvibe-docs', version: '1.0.0', files: stats, jobs: jobs.stats() });});--// Domain context (frontend calls this on load to get branding)++// Domain context (frontend calls this on load to get branding). It also starts this browser's session++// (ov_tools_jobs): PDF tools need one (or a sign-in, or a token) before the first upload.app.get('/api/context', (req, res) => {++ guard.ensureSession(req, res);++ res.set('Cache-Control', 'private, no-store');const ctx = req.ctx;const tools = listTools();res.json({@@@ -166,10 +189,11 @@ app.get('/api/tools', (_req, res) => {});// ── PDF Info (no file mutation) ──────────────────────────────--app.post('/api/info', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/info', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(hostTool), guard.heavy(hostTool), async (req, res) => {try {const tool = getTool('metadata');-- const result = await tool.handler(req.file.buffer, { mode: 'view' });++ const [buffer] = await readUploads([req.file]);++ const result = await tool.handler(buffer, { mode: 'view' });res.json({ success: true, ...result });} catch (err) {sendToolError(req, res, err, 'Info');@@@ -177,16 +201,17 @@ app.post('/api/info', burstLimiter, processLimiter, uploadSingle, async (req, re});// ── Main Processing Endpoint (single file) ───────────────────--app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {try {const toolId = req.body.tool || req.ctx.defaultOp;-- if (!toolId) return res.status(400).json({ error: 'No tool specified.' });++ if (!toolId) { fs.unlink(req.file.path, () => {}); return res.status(400).json({ error: 'No tool specified.' }); }const tool = getTool(toolId);-- if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });++ if (!tool) { fs.unlink(req.file.path, () => {}); return res.status(400).json({ error: `Unknown tool: ${toolId}` }); }// Same options and code path as the docs.process job-- const result = await tool.handler(req.file.buffer, buildOptions(req.body, req.ctx, false));++ const [buffer] = await readUploads([req.file]);++ const result = await tool.handler(buffer, buildOptions(req.body, req.ctx, false));// Some tools return view-only results (like metadata view)if (result.viewOnly) {@@@ -223,16 +248,18 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,});// ── Multi-File Processing Endpoint (merge, img2pdf) ──────────--app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, async (req, res) => {++app.post('/api/process/multi', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadMultiple, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {try {const toolId = req.body.tool || req.ctx.defaultOp;-- if (!toolId) return res.status(400).json({ error: 'No tool specified.' });---- const tool = getTool(toolId);-- if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });-- if (!tool.multiFile) return res.status(400).json({ error: `Tool "${toolId}" does not support multiple files. Use /api/process instead.` });++ const tool = toolId ? getTool(toolId) : null;++ if (!tool || !tool.multiFile) {++ for (const f of req.files) fs.unlink(f.path, () => {});++ if (!toolId) return res.status(400).json({ error: 'No tool specified.' });++ if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });++ return res.status(400).json({ error: `Tool "${toolId}" does not support multiple files. Use /api/process instead.` });++ }-- const buffers = req.files.map(f => f.buffer);++ const buffers = await readUploads(req.files);const result = await tool.handler(buffers, buildOptions(req.body, req.ctx, true));// Save output@@@ -268,10 +295,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy// into data/jobs.db, followed over SSE, reattachable by id after a reload or a restart.const jobs = jobsRuntime.setupJobs({app, service: 'docs', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,-- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,++ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,define: defineJobs,receive: uploadAny,-- limiters: [burstLimiter, processLimiter],++ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],++ jobTool: (req, type, input) => { const op = String((input && input.tool) || req.ctx.defaultOp || ''); const d = op ? guard.toolForJob('docs.process', op, req.ctx.toolId) : null; return d ? d.id : null; },defaults(req, input) {const out = { ...input };if (!out.tool && req.ctx.defaultOp) out.tool = req.ctx.defaultOp;@@@ -354,6 +382,7 @@ function shutdown() {analytics.destroy();analyticsDb.close();jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}ddeleted file mode 100644@@@ -1,48 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Docs.OpenVibe — Rate Limiting Middleware--// Tiered rate limits: anonymous vs authenticated users.--// Includes anti-abuse burst protection.--// ═══════════════════════════════════════════════════════════════----const rateLimit = require('express-rate-limit');----/** General API rate limit */--const apiLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 120 : 60,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Too many requests. Please try again later.' },--});----/**-- * Processing rate limit — stricter, tiered by auth.-- * Anonymous: 10 conversions/minute-- * Authenticated: 30 conversions/minute-- */--const processLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 30 : 10,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },--});----/**-- * Anti-burst limiter — prevents rapid-fire abuse.-- * Very short window: max 4 requests per 5 seconds per IP.-- */--const burstLimiter = rateLimit({-- windowMs: 5 * 1000,-- max: 4,-- standardHeaders: false,-- legacyHeaders: false,-- keyGenerator: (req) => req.ip,-- message: { error: 'Too many requests in quick succession. Please slow down.' },--});----module.exports = { apiLimiter, processLimiter, burstLimiter };@@@ -2,16 +2,30 @@// ═══════════════════════════════════════════════════════════════// Docs.OpenVibe — Multer Upload Middleware--// Memory storage with file size + mime validation.--// Supports multiple file uploads for merge operations.++// Disk storage (up to 50 files of 100 MB for a merge must never sit in memory while they arrive) with++// size + declared-type validation; the guard then checks the bytes (apps/_shared/guard/sniff.js).++// Files land in config.uploadsDir with a random name; the routes read them and delete them, and the++// retention sweep removes anything older than retention.tempMaxAge. Job submits move them into the job.// ═══════════════════════════════════════════════════════════════const multer = require('multer');++const path = require('path');++const fs = require('fs');++const crypto = require('crypto');const config = require('../config');const ALLOWED_MIMES = new Set(config.upload.allowedMimes);--const storage = multer.memoryStorage();++const uploadsDir = path.resolve(config.uploadsDir);++fs.mkdirSync(uploadsDir, { recursive: true });++++const storage = multer.diskStorage({++ destination: (_req, _file, cb) => cb(null, uploadsDir),++ filename: (_req, file, cb) => {++ const ext = (path.extname(file.originalname || '').toLowerCase().match(/^\.[a-z0-9]{1,8}$/) || ['.bin'])[0];++ cb(null, `${crypto.randomBytes(16).toString('hex')}${ext}`);++ },++});const upload = multer({storage,@@@ -30,7 +44,7 @@ const upload = multer({/*** Single file upload on 'file' field.-- * Attaches req.file with { buffer, originalname, mimetype, size }.++ * Attaches req.file with { path, originalname, mimetype, size }.*/function uploadSingle(req, res, next) {upload.single('file')(req, res, (err) => {@@@ -49,7 +63,7 @@ function uploadSingle(req, res, next) {/*** Multiple file upload on 'files' field (for merge, img2pdf, etc.).-- * Attaches req.files as an array.++ * Attaches req.files as an array of { path, originalname, mimetype, size }.*/function uploadMultiple(req, res, next) {upload.array('files', 50)(req, res, (err) => {@@@ -8,6 +8,16 @@const { PDFDocument } = require('pdf-lib');const sharp = require('sharp');++const guardLimits = require('../../../_shared/guard/limits');++++/** No picture larger than the guard's pixel limit (40 MP, TOOLS_MAX_INPUT_PIXELS) is decoded. */++function checkPixels(metadata, i, count) {++ const limit = guardLimits.bounds().maxInputPixels;++ const w = metadata.width || 0, h = metadata.pageHeight || metadata.height || 0;++ if (w * h <= limit) return;++ const mp = (n) => `${Math.round(n / 1e5) / 10} megapixels`;++ throw Object.assign(new Error(`${count > 1 ? `Image ${i + 1}` : 'The image'} is ${w}×${h} (${mp(w * h)}); the limit is ${mp(limit)}.`), { status: 413, code: 'tools.file.too_large', guardReason: 'pixels', expose: true });++}/*** Convert images to PDF.@@@ -30,19 +40,21 @@ async function img2pdf(buffers, options = {}) {legal: { width: 612, height: 1008 },};-- for (const imgBuf of buffers) {++ const limitInputPixels = guardLimits.bounds().maxInputPixels;++ for (const [i, imgBuf] of buffers.entries()) {// Convert to PNG or JPG using sharp for consistency-- const metadata = await sharp(imgBuf).metadata();++ const metadata = await sharp(imgBuf, { limitInputPixels: false }).metadata(); // the header only; checked next++ checkPixels(metadata, i, buffers.length);const imgWidth = metadata.width || 800;const imgHeight = metadata.height || 600;// Determine if we should embed as PNG or JPGlet embeddedImage;if (metadata.format === 'jpeg' || metadata.format === 'jpg') {-- const jpgBuf = await sharp(imgBuf).jpeg({ quality: 90 }).toBuffer();++ const jpgBuf = await sharp(imgBuf, { limitInputPixels }).jpeg({ quality: 90 }).toBuffer();embeddedImage = await pdf.embedJpg(jpgBuf);} else {-- const pngBuf = await sharp(imgBuf).png().toBuffer();++ const pngBuf = await sharp(imgBuf, { limitInputPixels }).png().toBuffer();embeddedImage = await pdf.embedPng(pngBuf);}nnew file mode 100644@@@ -0,0 +1,73 @@++'use strict';++// The guard on the PDF satellite: uploads go to disk (never 50 × 100 MB in memory) and are deleted++// after the synchronous call, the bytes are checked against the tool's accept list (a picture called++// .pdf is refused, and Image to PDF refuses a PDF), the page's context call starts the session PDF++// tools need, and a caller without one is recorded in report mode (TOOLS_GUARD=enforce refuses it).++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { PDFDocument } = require('pdf-lib');++const sharp = require('sharp');++const { startApp } = require('../../_shared/test/spawn');++++async function makePdf(pages) {++ const doc = await PDFDocument.create();++ for (let i = 0; i < pages; i++) doc.addPage([200, 300]).drawText(`page ${i + 1}`, { x: 20, y: 150 });++ return Buffer.from(await doc.save());++}++++(async () => {++ const pdf = await makePdf(2);++ const png = await sharp({ create: { width: 20, height: 20, channels: 3, background: '#aa3355' } }).png().toBuffer();++ for (const mode of ['report', 'enforce']) {++ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-docs-guard-'));++ const uploads = path.join(data, 'uploads');++ const app = await startApp('docs', { DATA_DIR: data, UPLOADS_DIR: uploads, OUTPUT_DIR: path.join(data, 'output'), TOOLS_GUARD: mode });++ // Each call from its own address, so the older burst limiter (4 in 5 s per address) stays out of it.++ let n = 0;++ const post = (files, fields, cookie, p = '/api/process') => {++ const f = new FormData();++ for (const [k, v] of Object.entries(fields)) f.append(k, v);++ for (const [buf, name, type, field = 'file'] of files) f.append(field, new Blob([buf], { type }), name);++ return fetch(`${app.base}${p}`, { method: 'POST', body: f, headers: { 'X-Forwarded-For': `198.51.100.${++n}`, ...(cookie ? { cookie } : {}) } });++ };++ try {++ const ctx = await fetch(`${app.base}/api/context`, { headers: { Host: 'compresspdf.openvibe.tools' } });++ const cookie = String(ctx.headers.get('set-cookie') || '').split(';')[0];++ assert.match(cookie, /^ov_tools_jobs=/);++++ let r = await post([[pdf, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, cookie);++ let body = await r.json();++ assert.strictEqual(r.status, 200, JSON.stringify(body));++ assert.strictEqual(body.output.ext, 'pdf');++ assert.deepStrictEqual(fs.readdirSync(uploads), [], 'the upload was on disk and is gone after the call');++++ r = await post([[png, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, cookie);++ assert.strictEqual(r.status, 415, 'a PNG called doc.pdf is refused (in both modes)');++ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');++ r = await post([[png, 'a.png', 'image/png', 'files'], [pdf, 'b.png', 'image/png', 'files']], { tool: 'img2pdf' }, cookie, '/api/process/multi');++ assert.strictEqual(r.status, 415, 'Image to PDF takes pictures only: file 2 is a PDF');++ assert.match((await r.json()).detail, /File 2 is a PDF file/);++ r = await post([[png, 'a.png', 'image/png', 'files'], [png, 'b.png', 'image/png', 'files']], { tool: 'img2pdf' }, cookie, '/api/process/multi');++ assert.strictEqual(r.status, 200);++ assert.deepStrictEqual(fs.readdirSync(uploads), []);++++ // No session, no sign-in, no token: PDF tools (auth.anonymous false) record or refuse it.++ r = await post([[pdf, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, null);++ if (mode === 'report') assert.strictEqual(r.status, 200, 'report mode lets it through');++ else {++ assert.strictEqual(r.status, 401);++ assert.strictEqual((await r.json()).code, 'tools.session_required');++ }++ const m = await (await fetch(`${app.base}/metrics`)).text();++ assert.match(m, /tools_guard_refused_total\{reason="session",tool="compresspdf"\} 1/);++ assert.match(m, /tools_guard_refused_total\{reason="sniff",tool="compresspdf"\} 1/);++ assert.match(m, /tools_guard_refused_total\{reason="sniff",tool="image2pdf"\} 1/);++ } finally {++ await app.kill();++ fs.rmSync(data, { recursive: true, force: true });++ }++ }++ console.log('docs guard: disk uploads cleaned up, sniffing per tool (PDF vs pictures), context session, session rule report/enforce: all checks passed');++})().catch((err) => { console.error(err); process.exit(1); });@@@ -19,12 +19,24 @@ const { internalOk } = require('../../_shared/internal-auth');const { hostGuard, stampedPage } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-food', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota and++// the abuse log (data/guard.db). The data lookups are counted by maps (tools-map), which gets the++// visitor's address from here. TOOLS_GUARD=report (default) records what it would refuse.++const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';++const guard = createGuard({++ app: 'food', dataDir: path.join(__dirname, '..', 'data'), Database, specs: require('../../maps/server/descriptors').SPECS.filter(s => s.id === 'food'),++ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',++ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),++});++const PORT = parseInt(process.env.PORT) || 4011;const MAPS_API = process.env.MAPS_API || 'http://127.0.0.1:4010';@@@ -34,7 +46,7 @@ const release = require('openvibe-shared/release').createRelease({ service: 'too// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-food', release: release.release,checks: [@@@ -43,14 +55,15 @@ observe({ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only' }),],});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));--// Only loopback hops (the host's nginx, the gateway) are proxies: req.ip is the first address before--// them, which is what maps is told below. A client's own X-Forwarded-For is never believed.--app.set('trust proxy', 'loopback');++// One hop on loopback (the host's nginx, or the gateway) is the proxy: req.ip is the address it gives,++// which is what maps is told below. A client's own X-Forwarded-For is never believed.++app.set('trust proxy', TRUST_PROXY);app.use(exceptRegistry(cors({origin(origin, callback) {if (!origin) return callback(null, true);@@@ -74,7 +87,11 @@ app.use(helmet({},},}));--app.use(rateLimit({ windowMs: 60000, max: 60 }));++// Sign-in first, so limits know who is asking; everyone else counts by address (IPv6 /64).++app.use(cookieParser());++app.use(guard.identify);++app.use(guard.legacyLimiter(rateLimit, { windowMs: 60000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }));++app.use('/api/', guard.apiQuota);// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's food finder ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -82,7 +99,6 @@ app.use(rateLimit({ windowMs: 60000, max: 60 }));// The food finder is built on the maps backend; its descriptor lives with the maps app's.const toolRegistry = createLocalRegistry({ specs: require('../../maps/server/descriptors').SPECS.filter(s => s.id === 'food') });app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));--app.use(cookieParser());// ── Analytics Middleware ─────────────────────────────────────app.use(analytics.middleware());@@@ -145,6 +161,7 @@ function shutdown() {console.log('[Food.OpenVibe] Shutting down...');analytics.destroy();analyticsDb.close();++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}@@@ -1,7 +1,7 @@'use strict';// Food proxies every API call to maps. It used to call maps without the visitor's address, so maps'// per-IP limit (30/min) was one bucket shared by every food visitor (127.0.0.1). Now food forwards--// X-Forwarded-For / X-Real-IP from req.ip, and maps believes X-Forwarded-For only from loopback hops.++// X-Forwarded-For / X-Real-IP from req.ip, and maps believes X-Forwarded-For from one loopback hop only.const assert = require('assert');const fs = require('fs');const path = require('path');@@@ -21,7 +21,8 @@ const { startApp } = require('../../_shared/test/spawn');assert.strictEqual(other.status, 200, 'a second visitor has their own bucket');const src = (app) => fs.readFileSync(path.join(__dirname, '..', '..', app, 'server', 'index.js'), 'utf8');-- assert.match(src('maps'), /app\.set\('trust proxy', 'loopback'\)/, 'maps trusts forwarded addresses from loopback only');++ assert.match(src('maps'), /app\.set\('trust proxy', TRUST_PROXY\)/, 'maps trusts one forwarded hop, on loopback only');++ assert.match(src('food'), /app\.set\('trust proxy', TRUST_PROXY\)/);assert.match(src('food'), /'X-Forwarded-For': req\.ip/);} finally {await food.kill();@@@ -9,6 +9,7 @@"version": "2.0.0","license": "MIT","dependencies": {++ "better-sqlite3": "^11.10.0","cookie-parser": "^1.4.7","cors": "^2.8.5","dotenv": "^16.4.7",@@@ -72,6 +73,57 @@"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==","license": "MIT"},++ "node_modules/base64-js": {++ "version": "1.5.1",++ "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",++ "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",++ "funding": [++ {++ "type": "github",++ "url": "https://github.com/sponsors/feross"++ },++ {++ "type": "patreon",++ "url": "https://www.patreon.com/feross"++ },++ {++ "type": "consulting",++ "url": "https://feross.org/support"++ }++ ],++ "license": "MIT"++ },++ "node_modules/better-sqlite3": {++ "version": "11.10.0",++ "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",++ "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",++ "hasInstallScript": true,++ "license": "MIT",++ "dependencies": {++ "bindings": "^1.5.0",++ "prebuild-install": "^7.1.1"++ }++ },++ "node_modules/bindings": {++ "version": "1.5.0",++ "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",++ "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",++ "license": "MIT",++ "dependencies": {++ "file-uri-to-path": "1.0.0"++ }++ },++ "node_modules/bl": {++ "version": "4.1.0",++ "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",++ "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",++ "license": "MIT",++ "dependencies": {++ "buffer": "^5.5.0",++ "inherits": "^2.0.4",++ "readable-stream": "^3.4.0"++ }++ },"node_modules/body-parser": {"version": "1.20.6","resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.6.tgz",@@@ -96,6 +148,30 @@"npm": "1.2.8000 || >= 1.4.16"}},++ "node_modules/buffer": {++ "version": "5.7.1",++ "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",++ "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",++ "funding": [++ {++ "type": "github",++ "url": "https://github.com/sponsors/feross"++ },++ {++ "type": "patreon",++ "url": "https://www.patreon.com/feross"++ },++ {++ "type": "consulting",++ "url": "https://feross.org/support"++ }++ ],++ "license": "MIT",++ "dependencies": {++ "base64-js": "^1.3.1",++ "ieee754": "^1.1.13"++ }++ },"node_modules/buffer-equal-constant-time": {"version": "1.0.1","resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",@@@ -140,6 +216,12 @@"url": "https://github.com/sponsors/ljharb"}},++ "node_modules/chownr": {++ "version": "1.1.4",++ "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",++ "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",++ "license": "ISC"++ },"node_modules/content-disposition": {"version": "0.5.4","resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",@@@ -215,6 +297,30 @@"ms": "2.0.0"}},++ "node_modules/decompress-response": {++ "version": "6.0.0",++ "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",++ "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",++ "license": "MIT",++ "dependencies": {++ "mimic-response": "^3.1.0"++ },++ "engines": {++ "node": ">=10"++ },++ "funding": {++ "url": "https://github.com/sponsors/sindresorhus"++ }++ },++ "node_modules/deep-extend": {++ "version": "0.6.0",++ "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",++ "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",++ "license": "MIT",++ "engines": {++ "node": ">=4.0.0"++ }++ },"node_modules/depd": {"version": "2.0.0","resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",@@@ -234,6 +340,15 @@"npm": "1.2.8000 || >= 1.4.16"}},++ "node_modules/detect-libc": {++ "version": "2.1.2",++ "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",++ "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",++ "license": "Apache-2.0",++ "engines": {++ "node": ">=8"++ }++ },"node_modules/dotenv": {"version": "16.6.1","resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",@@@ -284,6 +399,15 @@"node": ">= 0.8"}},++ "node_modules/end-of-stream": {++ "version": "1.4.5",++ "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",++ "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",++ "license": "MIT",++ "dependencies": {++ "once": "^1.4.0"++ }++ },"node_modules/es-define-property": {"version": "1.0.1","resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",@@@ -329,6 +453,15 @@"node": ">= 0.6"}},++ "node_modules/expand-template": {++ "version": "2.0.3",++ "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",++ "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",++ "license": "(MIT OR WTFPL)",++ "engines": {++ "node": ">=6"++ }++ },"node_modules/express": {"version": "4.22.2","resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz",@@@ -412,6 +545,12 @@],"license": "BSD-3-Clause"},++ "node_modules/file-uri-to-path": {++ "version": "1.0.0",++ "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",++ "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",++ "license": "MIT"++ },"node_modules/finalhandler": {"version": "1.3.2","resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",@@@ -448,6 +587,12 @@"node": ">= 0.6"}},++ "node_modules/fs-constants": {++ "version": "1.0.0",++ "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",++ "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",++ "license": "MIT"++ },"node_modules/function-bind": {"version": "1.1.2","resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",@@@ -494,6 +639,12 @@"node": ">= 0.4"}},++ "node_modules/github-from-package": {++ "version": "0.0.0",++ "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",++ "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",++ "license": "MIT"++ },"node_modules/gopd": {"version": "1.2.0","resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",@@@ -574,12 +725,38 @@"node": ">=0.10.0"}},++ "node_modules/ieee754": {++ "version": "1.2.1",++ "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",++ "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",++ "funding": [++ {++ "type": "github",++ "url": "https://github.com/sponsors/feross"++ },++ {++ "type": "patreon",++ "url": "https://www.patreon.com/feross"++ },++ {++ "type": "consulting",++ "url": "https://feross.org/support"++ }++ ],++ "license": "BSD-3-Clause"++ },"node_modules/inherits": {"version": "2.0.4","resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz","integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==","license": "ISC"},++ "node_modules/ini": {++ "version": "1.3.8",++ "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",++ "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",++ "license": "ISC"++ },"node_modules/ipaddr.js": {"version": "1.9.1","resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",@@@ -755,12 +932,45 @@"node": ">= 0.6"}},++ "node_modules/mimic-response": {++ "version": "3.1.0",++ "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",++ "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",++ "license": "MIT",++ "engines": {++ "node": ">=10"++ },++ "funding": {++ "url": "https://github.com/sponsors/sindresorhus"++ }++ },++ "node_modules/minimist": {++ "version": "1.2.8",++ "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",++ "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",++ "license": "MIT",++ "funding": {++ "url": "https://github.com/sponsors/ljharb"++ }++ },++ "node_modules/mkdirp-classic": {++ "version": "0.5.3",++ "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",++ "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",++ "license": "MIT"++ },"node_modules/ms": {"version": "2.0.0","resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz","integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==","license": "MIT"},++ "node_modules/napi-build-utils": {++ "version": "2.0.0",++ "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",++ "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",++ "license": "MIT"++ },"node_modules/negotiator": {"version": "0.6.3","resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",@@@ -770,6 +980,18 @@"node": ">= 0.6"}},++ "node_modules/node-abi": {++ "version": "3.96.0",++ "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",++ "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",++ "license": "MIT",++ "dependencies": {++ "semver": "^7.3.5"++ },++ "engines": {++ "node": ">=10"++ }++ },"node_modules/object-assign": {"version": "4.1.1","resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",@@@ -803,6 +1025,15 @@"node": ">= 0.8"}},++ "node_modules/once": {++ "version": "1.4.0",++ "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",++ "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",++ "license": "ISC",++ "dependencies": {++ "wrappy": "1"++ }++ },"node_modules/openvibe-contracts": {"version": "0.33.0","resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.33.0",@@@ -843,6 +1074,33 @@"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==","license": "MIT"},++ "node_modules/prebuild-install": {++ "version": "7.1.3",++ "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",++ "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",++ "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",++ "license": "MIT",++ "dependencies": {++ "detect-libc": "^2.0.0",++ "expand-template": "^2.0.3",++ "github-from-package": "0.0.0",++ "minimist": "^1.2.3",++ "mkdirp-classic": "^0.5.3",++ "napi-build-utils": "^2.0.0",++ "node-abi": "^3.3.0",++ "pump": "^3.0.0",++ "rc": "^1.2.7",++ "simple-get": "^4.0.0",++ "tar-fs": "^2.0.0",++ "tunnel-agent": "^0.6.0"++ },++ "bin": {++ "prebuild-install": "bin.js"++ },++ "engines": {++ "node": ">=10"++ }++ },"node_modules/proxy-addr": {"version": "2.0.7","resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",@@@ -856,6 +1114,16 @@"node": ">= 0.10"}},++ "node_modules/pump": {++ "version": "3.0.4",++ "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",++ "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",++ "license": "MIT",++ "dependencies": {++ "end-of-stream": "^1.1.0",++ "once": "^1.3.1"++ }++ },"node_modules/qs": {"version": "6.15.3","resolved": "https://registry.npmjs.org/qs/-/qs-6.15.3.tgz",@@@ -896,6 +1164,35 @@"node": ">= 0.8"}},++ "node_modules/rc": {++ "version": "1.2.8",++ "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",++ "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",++ "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",++ "dependencies": {++ "deep-extend": "^0.6.0",++ "ini": "~1.3.0",++ "minimist": "^1.2.0",++ "strip-json-comments": "~2.0.1"++ },++ "bin": {++ "rc": "cli.js"++ }++ },++ "node_modules/readable-stream": {++ "version": "3.6.2",++ "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",++ "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",++ "license": "MIT",++ "dependencies": {++ "inherits": "^2.0.3",++ "string_decoder": "^1.1.1",++ "util-deprecate": "^1.0.1"++ },++ "engines": {++ "node": ">= 6"++ }++ },"node_modules/require-from-string": {"version": "2.0.2","resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",@@@ -1066,6 +1363,51 @@"url": "https://github.com/sponsors/ljharb"}},++ "node_modules/simple-concat": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",++ "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",++ "funding": [++ {++ "type": "github",++ "url": "https://github.com/sponsors/feross"++ },++ {++ "type": "patreon",++ "url": "https://www.patreon.com/feross"++ },++ {++ "type": "consulting",++ "url": "https://feross.org/support"++ }++ ],++ "license": "MIT"++ },++ "node_modules/simple-get": {++ "version": "4.0.1",++ "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",++ "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",++ "funding": [++ {++ "type": "github",++ "url": "https://github.com/sponsors/feross"++ },++ {++ "type": "patreon",++ "url": "https://www.patreon.com/feross"++ },++ {++ "type": "consulting",++ "url": "https://feross.org/support"++ }++ ],++ "license": "MIT",++ "dependencies": {++ "decompress-response": "^6.0.0",++ "once": "^1.3.1",++ "simple-concat": "^1.0.0"++ }++ },"node_modules/statuses": {"version": "2.0.2","resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",@@@ -1075,6 +1417,52 @@"node": ">= 0.8"}},++ "node_modules/string_decoder": {++ "version": "1.3.0",++ "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",++ "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",++ "license": "MIT",++ "dependencies": {++ "safe-buffer": "~5.2.0"++ }++ },++ "node_modules/strip-json-comments": {++ "version": "2.0.1",++ "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",++ "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/tar-fs": {++ "version": "2.1.5",++ "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",++ "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",++ "license": "MIT",++ "dependencies": {++ "chownr": "^1.1.1",++ "mkdirp-classic": "^0.5.2",++ "pump": "^3.0.0",++ "tar-stream": "^2.1.4"++ }++ },++ "node_modules/tar-stream": {++ "version": "2.2.0",++ "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",++ "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",++ "license": "MIT",++ "dependencies": {++ "bl": "^4.0.3",++ "end-of-stream": "^1.4.1",++ "fs-constants": "^1.0.0",++ "inherits": "^2.0.3",++ "readable-stream": "^3.1.1"++ },++ "engines": {++ "node": ">=6"++ }++ },"node_modules/toidentifier": {"version": "1.0.1","resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",@@@ -1084,6 +1472,18 @@"node": ">=0.6"}},++ "node_modules/tunnel-agent": {++ "version": "0.6.0",++ "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",++ "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",++ "license": "Apache-2.0",++ "dependencies": {++ "safe-buffer": "^5.0.1"++ },++ "engines": {++ "node": "*"++ }++ },"node_modules/type-is": {"version": "1.6.18","resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",@@@ -1106,6 +1506,12 @@"node": ">= 0.8"}},++ "node_modules/util-deprecate": {++ "version": "1.0.2",++ "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",++ "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",++ "license": "MIT"++ },"node_modules/utils-merge": {"version": "1.0.1","resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",@@@ -1123,6 +1529,12 @@"engines": {"node": ">= 0.8"}++ },++ "node_modules/wrappy": {++ "version": "1.0.2",++ "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",++ "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",++ "license": "ISC"}}}@@@ -10,6 +10,7 @@},"license": "MIT","dependencies": {++ "better-sqlite3": "^11.10.0","cookie-parser": "^1.4.7","cors": "^2.8.5","dotenv": "^16.4.7",@@@ -378,10 +378,10 @@ P.curlconvert=(s,m)=>DevEngine.curlconvert(s,m);P.webhook=async()=>{const o=$('out');-- const r=await fetch(API+'/webhook/bins',{method:'POST'});const d=await r.json();if(!d.ok)throw new Error(d.error||'Failed');++ const r=await fetch(API+'/webhook/bins',{method:'POST',credentials:'include'});const d=await r.json();if(!d.ok)throw new Error(d.error||'Failed');const binUrl=API+'/webhook/bins/'+d.binId+'/in';o.innerHTML='<div style="margin-bottom:12px"><div class="jl">Your Webhook URL</div><div style="font-size:1rem;word-break:break-all;color:var(--ac);cursor:pointer" onclick="navigator.clipboard.writeText(\''+binUrl+"');toast('Copied!')\">"+binUrl+'</div><div style="color:var(--tx3);font-size:.78rem;margin-top:4px">Click to copy \u00b7 Polling for requests...</div></div><div id="whr">Waiting for requests...</div>';-- const iv=setInterval(async()=>{try{const r2=await fetch(API+'/webhook/bins/'+d.binId);const d2=await r2.json();++ const iv=setInterval(async()=>{try{const r2=await fetch(API+'/webhook/bins/'+d.binId,{credentials:'include'});const d2=await r2.json();if(d2.ok&&d2.requests.length){$('whr').innerHTML=d2.requests.map((rq,i)=>'<div class="js"><div class="jl">#'+(i+1)+' '+rq.method+' \u00b7 '+new Date(rq.timestamp).toLocaleTimeString()+'</div>Headers:\n'+Object.entries(rq.headers||{}).map(([k,v])=>k+': '+v).join('\n')+'\n\nBody:\n'+H(typeof rq.body==='object'?JSON.stringify(rq.body,null,2):String(rq.body||'(empty)'))+'</div>').join('')}}catch{}},2000);setTimeout(()=>clearInterval(iv),3600000);return null;};@@@ -4,18 +4,24 @@// Dev.OpenVibe — API Routes// Server-side endpoints for webhook bins, Open Graph fetching,// and tool listing. Most dev tools are client-side only.++//++// Guard (apps/_shared/guard, opts.guard): the Open Graph fetcher counts against tools-fetch and a++// per-target throttle; a webhook bin belongs to whoever made it (the browser session cookie, a++// sign-in or a token): only they can read or delete it, anyone may post to its /in URL. Bins per++// owner and per address are capped (limits.js webhook), so one address cannot fill the global cap.// ═══════════════════════════════════════════════════════════════const { Router } = require('express');const crypto = require('crypto');const { DEV_TOOLS } = require('./config');const { createEgress, TargetRefused } = require('../../../_shared/egress');++const { createCallerResolver } = require('../../../_shared/guard/caller');++const guardLimits = require('../../../_shared/guard/limits');// ── In-memory webhook bin storage ────────────────────────────--const webhookBins = new Map();++const webhookBins = new Map(); // binId → { created, requests, owner, ipKey }const WEBHOOK_BIN_TTL = 60 * 60 * 1000; // 1 hourconst WEBHOOK_MAX_REQUESTS = 200;--const WEBHOOK_MAX_BINS = 500;function cleanupBins() {const now = Date.now();@@@ -51,21 +57,19 @@ function extractOGTags(html) {// ═════════════════════════════════════════════════════════════// `opts.egress` is for tests: the shared SSRF guard with a mock resolver and transports.++// `opts.guard` is the gateway's guard (identify() has run before these routes; quotas, the per-target++// throttle, the abuse log). Without one (tests), bins are still owned: by the session cookie.module.exports = function createDevRoutes(db, requireAuth, opts = {}) {const router = Router();const egress = opts.egress || createEgress();++ const guard = opts.guard || null;++ const callers = guard ? null : createCallerResolver({ getPublicKey: () => null });++ const whoIs = (req, res, create) => (guard ? guard.resolveCaller(req, res, { create }) : callers.resolve(req, res, { create }));++ const caps = guardLimits.bounds().webhook;++ const refuse = (req, res, r) => (guard ? guard.refuse(req, res, r) : (res.status(r.status).json({ error: r.detail, code: r.code }), true));-- // Optional auth — attaches req.user if token present, but doesn't block.-- // Verifies the shared ov_token OFFLINE against the Network public key.-- function optionalAuth(req, res, next) {-- const authHeader = req.headers.authorization;-- const token = authHeader?.startsWith('Bearer ') ? authHeader.slice(7) : req.cookies?.ov_token;-- const auth = req.app.locals.auth;-- if (!token || !auth) return next();-- Promise.resolve(auth.verify(token))-- .then(claims => { if (claims) req.user = claims; next(); })-- .catch(() => next());-- }++ // Every dev API call (and the Open Graph fetcher's own tools-fetch quota, below).++ if (guard) router.use(guard.toolQuota((req) => (req.method === 'GET' && req.path === '/opengraph' ? 'opengraph' : null)));// ── List all dev tools ───────────────────────────────────router.get('/tools', (_req, res) => {@@@ -80,7 +84,7 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {});// ── Open Graph / Meta tag fetcher ────────────────────────-- router.get('/opengraph', optionalAuth, async (req, res) => {++ router.get('/opengraph', async (req, res) => {let targetUrl = req.query.url || req.query.target;if (!targetUrl) return res.status(400).json({ error: 'Missing ?url= parameter' });@@@ -92,6 +96,8 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {} catch {return res.status(400).json({ error: 'Invalid URL' });}++ // Per-target throttle (descriptor opengraph: limits.perTargetPerMinute), across every caller.++ if (guard && !guard.target(req, res, { tool: 'opengraph', target: targetUrl })) return;try {// Public addresses only, checked after DNS and dialled as checked; each redirect hop is@@@ -147,17 +153,32 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {// ── Webhook Bins ─────────────────────────────────────────-- // Create a new bin-- router.post('/webhook/bins', optionalAuth, (_req, res) => {++ /** The bin, if it exists AND belongs to the caller; otherwise null (the same 404 either way). */++ function ownBin(req, res) {++ const bin = webhookBins.get(req.params.binId);++ const who = whoIs(req, res, false);++ return bin && who.owner && bin.owner === who.owner ? bin : null;++ }++++ // Create a new bin. Its owner is the caller: a browser gets the session cookie here (the page++ // calls with credentials), a signed-in person or a token is itself.++ router.post('/webhook/bins', (req, res) => {cleanupBins();-- if (webhookBins.size >= WEBHOOK_MAX_BINS) {++ if (webhookBins.size >= caps.total) {return res.status(429).json({ error: 'Too many active bins. Try again later.' });}++ const who = whoIs(req, res, true);++ let mine = 0, fromHere = 0;++ for (const b of webhookBins.values()) { if (b.owner === who.owner) mine++; if (b.ipKey === who.ipKey) fromHere++; }++ if (mine >= caps.perOwner && refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'webhook', tool: 'webhook', retryAfter: 300, detail: `At most ${caps.perOwner} webhook bins at a time; old ones expire after an hour.`, extra: { scope: 'webhook' } })) return undefined;++ if (fromHere >= caps.perIp && refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'webhook', tool: 'webhook', retryAfter: 300, detail: `At most ${caps.perIp} webhook bins from one address at a time.`, extra: { scope: 'address' } })) return undefined;const binId = crypto.randomBytes(12).toString('hex');webhookBins.set(binId, {created: Date.now(),requests: [],++ owner: who.owner,++ ipKey: who.ipKey,});res.json({@@@ -168,9 +189,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {});});-- // Get bin requests-- router.get('/webhook/bins/:binId', optionalAuth, (req, res) => {-- const bin = webhookBins.get(req.params.binId);++ // Get bin requests (its owner only)++ router.get('/webhook/bins/:binId', (req, res) => {++ res.set('Cache-Control', 'private, no-store');++ const bin = ownBin(req, res);if (!bin) return res.status(404).json({ error: 'Bin not found or expired' });res.json({@@@ -213,9 +235,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {res.status(200).json({ ok: true, message: 'Received' });});-- // Delete bin-- router.delete('/webhook/bins/:binId', optionalAuth, (req, res) => {-- const deleted = webhookBins.delete(req.params.binId);++ // Delete bin (its owner only; anyone else learns nothing)++ router.delete('/webhook/bins/:binId', (req, res) => {++ const bin = ownBin(req, res);++ const deleted = bin ? webhookBins.delete(req.params.binId) : false;res.json({ ok: true, deleted });});@@@ -27,6 +27,7 @@ const createNetRoutes = require('./net/routes');const { NET_TOOL_MAP, NET_ALIASES } = require('./net/config');const createDevRoutes = require('./dev/routes');const { DEV_TOOL_MAP, DEV_ALIASES } = require('./dev/config');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.@@@ -51,6 +52,19 @@ const registry = require('./registry');const toolRegistry = require('./registry/descriptors');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');let auth = null; // created below; the key check reads it at request time++++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else++// the address), the tools-api quota, the net and dev tools' quotas by descriptor, the per-target++// throttle, the port-scan cap, webhook bin caps and the abuse log (data/guard.db). The Network key is++// the OAuth client's (loaded and retried by it). TOOLS_GUARD=report (default) records what it would refuse.++const guard = createGuard({++ app: 'gateway', dataDir: require('path').resolve(__dirname, '..', process.env.DATA_DIR || 'data'),++ Database: require('better-sqlite3'), contracts: require('openvibe-contracts'),++ specs: [...require('./net/descriptors').SPECS, ...require('./dev/descriptors').SPECS],++ issuer: config.networkUrl,++ keys: { get: () => (auth && auth.client.publicKey) || null, ensure: () => (auth ? auth.ensureKey() : Promise.resolve(null)) },++});const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools', release: release.release, mountReady: false,@@@ -76,6 +90,7 @@ const obs = observe({...Object.entries(SATELLITES).map(([name, port]) => ready.upstream(`satellite_${name}`, `http://127.0.0.1:${port}/api/ready`, { description: `${name} satellite (port ${port}); its hosts fail without it` })),],});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);function getRequestHost(req) {@@@ -84,9 +99,10 @@ function getRequestHost(req) {// ── Security ─────────────────────────────────────────────────// One hop: the host's nginx, which applies Cloudflare's real IP (real_ip_header CF-Connecting-IP) and--// sets X-Forwarded-For to $remote_addr. req.ip is that address; anything a client put in its own--// X-Forwarded-For stays to the left of it and is never believed (myip, rate limits, forwarded IPs).--app.set('trust proxy', 1);++// sets X-Forwarded-For to $remote_addr, on loopback (TRUST_PROXY). req.ip is that address; anything a++// client put in its own X-Forwarded-For is never believed (myip, rate limits, forwarded IPs). The++// satellites this gateway proxies to get req.ip as their one hop (registry/host-middleware.js).++app.set('trust proxy', TRUST_PROXY);app.use(helmet({contentSecurityPolicy: {@@@ -155,9 +171,10 @@ app.use(exceptRegistry(cors({credentials: true,})));--// ── Rate Limiting ────────────────────────────────────────────--app.use('/api/', rateLimit({ windowMs: 60_000, max: 120 }));--app.use('/auth/', rateLimit({ windowMs: 15 * 60_000, max: 60 }));++// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──++app.use(guard.identify);++app.use('/api/', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 120, signedIn: 240, message: 'Too many requests. Please try again later.' }), guard.apiQuota);++app.use('/auth/', rateLimit({ windowMs: 15 * 60_000, max: 60, keyGenerator: (req) => guard.caller(req).ipKey }));// ── Tool registry (ADR-027, capability tools.tool.read) ──────// GET /api/v1/tools (tools.tool-list@1), /api/v1/tools/:id (tools.tool@1), /api/v1/tools/:id/schema.@@@ -195,7 +212,7 @@ app.get('/api/brand', (_req, res) => res.json(BRAND));// which owns pastes (roadmap Wave 5). The visitor's Network JWT goes along: Community verifies it and// files writes under the person's canonical subject, so no site has to translate ids any more. The// client address goes along too, for Community's anonymous-write limits.--app.use('/api/pastes', rateLimit({ windowMs: 60_000, max: 120 }), async (req, res) => {++app.use('/api/pastes', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 120, signedIn: 240, message: 'Too many requests. Please try again later.' }), async (req, res) => {try {const target = `${config.communityUrl}/api/pastes${req.url}`;const fetchOpts = { method: req.method, headers: {} };@@@ -222,10 +239,10 @@ app.use('/api/pastes', rateLimit({ windowMs: 60_000, max: 120 }), async (req, re});// ── Net.OpenVibe — Network Tools API ─────────────────────────--app.use('/api/net', rateLimit({ windowMs: 60_000, max: 60 }), createNetRoutes(null, requireAuth));++app.use('/api/net', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }), createNetRoutes(null, requireAuth, { guard }));// ── Dev.OpenVibe — Developer & SEO Tools API ─────────────────--app.use('/api/dev', rateLimit({ windowMs: 60_000, max: 60 }), createDevRoutes(null, requireAuth));++app.use('/api/dev', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }), createDevRoutes(null, requireAuth, { guard }));// ── Host-header subdomain routing ────────────────────────────function subdomainOf(req) {@@@ -24,7 +24,40 @@ const IPAPI_FIELDS = 'status,message,country,countryCode,region,regionName,city,* Every tool here that connects to the target (ssl, headers, redirects, port, ping, lookup, and a* custom DNS server) goes through the shared SSRF guard: only public addresses, checked after DNS* and dialled as checked. `opts.egress` is for tests (a guard with a mock resolver and transports).++ *++ * `opts.guard` (apps/_shared/guard, the gateway's): each call is charged to its tool's descriptor++ * (tools-fetch or tools-probe × the caller's tier, weighted by cost), a principal needs the tool's++ * capability (tools.net.probe for port, ping, latency), every tool that reaches a target is throttled++ * per target across all callers (limits.perTargetPerMinute), and the port checker is capped per++ * request and per caller so it cannot sweep hosts or ranges. The throttle and the port caps apply in++ * report mode too. The pages call these routes without a session: people are counted by address.*/++const { SPECS: NET_SPECS } = require('./descriptors');++++// endpoint ('/dns') → the descriptors served by it (dns, dig, nslookup, mx, txt, …)++const BY_ENDPOINT = new Map();++for (const spec of NET_SPECS) {++ if (!spec.route) continue;++ const ep = spec.route.path.replace(/^\/api\/net/, '');++ if (!BY_ENDPOINT.has(ep)) BY_ENDPOINT.set(ep, []);++ BY_ENDPOINT.get(ep).push(spec);++}++++/** The tool a /api/net request runs: the host's own tool when it uses this endpoint, else the one named after it. */++function netToolOf(req) {++ const ep = `/${String(req.path || '').split('/')[1] || ''}`;++ const list = BY_ENDPOINT.get(ep);++ if (!list) return null;++ const host = req.ovHost && req.ovHost.tool;++ return (list.find(s => s.id === host) || list.find(s => s.id === ep.slice(1)) || list[0]).id;++}++++/** The target a request names: /api/net/<endpoint>/<target> or ?target= (or ?url=). */++function netTargetOf(req) {++ const m = /^\/[a-z0-9]+\/(.+)$/i.exec(String(req.path || ''));++ if (m) { try { return decodeURIComponent(m[1]); } catch { return m[1]; } }++ return String(req.query.target || req.query.url || '');++}// ── IPv4 / IPv6 arithmetic ─────────────────────────────────const ipv4ToInt = (ip) => ip.split('.').reduce((n, o) => n * 256 + Number(o), 0) >>> 0;@@@ -117,6 +150,7 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {const cache = opts.cache || createCache({ max: 5000, ttlMs: HOUR });const checks = createChecks({ egress, cache, ...(opts.resolverFor && { resolverFor: opts.resolverFor }), ...(opts.tlsUpgrade && { tlsUpgrade: opts.tlsUpgrade }) });const reverseDns = opts.reverse || ((ip) => dns.reverse(ip));++ const guard = opts.guard || null;// ── Helpers ──────────────────────────────────────────────@@@ -226,20 +260,25 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {/** Get config (live — reads DB each time for admin changes) */function cfg() { return getNetConfig(db); }-- /** Optional auth — attaches user if token present, but doesn't block.-- * Verifies the shared ov_token OFFLINE against the Network public key. */-- function optionalAuth(req, res, next) {-- const authHeader = req.headers.authorization;-- const token = authHeader?.startsWith('Bearer ') ? authHeader.slice(7) : req.cookies?.ov_token;-- const auth = req.app.locals.auth;-- if (!token || !auth) return next();-- Promise.resolve(auth.verify(token))-- .then(claims => { if (claims) req.user = claims; next(); })-- .catch(() => next());++ // ── Guard: who may run it, its quota, the per-target throttle (sign-in was read by the app) ──++ if (guard) {++ router.use((req, res, next) => {++ const id = netToolOf(req);++ if (!id) return next();++ req.netTool = id;++ if (!guard.capabilityRule(req, res, guard.tool(id))) return undefined;++ return next();++ });++ router.use(guard.toolQuota((req) => req.netTool));++ router.use((req, res, next) => {++ const d = req.netTool && guard.tool(req.netTool);++ if (!d || !d.egress || !(d.limits && d.limits.perTargetPerMinute)) return next();++ const t = netTargetOf(req);++ if (t && !guard.target(req, res, { tool: d.id, target: t })) return undefined;++ return next();++ });}-- router.use(optionalAuth);--// ── Tool list endpoint ───────────────────────────────────router.get('/tools', (_req, res) => {ok(res, { tools: NET_TOOLS, regions: cfg().probeRegions });@@@ -699,9 +738,12 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {const target = parseTarget(req.params.target || req.query.target);if (!target) return fail(res, 'Please provide a host');-- const portsStr = req.query.ports || '80,443,22,21,25,53,3306,5432,8080,8443';-- const ports = portsStr.split(',').map(p => parseInt(p.trim())).filter(p => p > 0 && p <= 65535).slice(0, 20);++ const portsStr = String(req.query.ports || '80,443,22,21,25,53,3306,5432,8080,8443');++ const ports = [...new Set(portsStr.split(',').map(p => parseInt(p.trim(), 10)).filter(p => p > 0 && p <= 65535))];if (!ports.length) return fail(res, 'No valid ports specified');++ if (ports.length > 20) return fail(res, 'At most 20 ports per check');++ // Per caller: at most so many ports and so many different hosts in ten minutes (a hard limit).++ if (guard && !guard.portScan(req, res, { tool: req.netTool || 'port', target, ports })) return undefined;// Resolve + check; every probe dials the checked address.const dest = await egress.resolve(target, { prefer: 4 });@@@ -31,8 +31,12 @@ function proxyTo(port, info, req, res) {headers['x-ov-short-host'] = info.shortHost;headers['x-forwarded-host'] = info.host;headers['x-forwarded-proto'] = 'https';-- const prior = req.headers['x-forwarded-for'];-- headers['x-forwarded-for'] = (prior ? prior + ', ' : '') + (req.socket.remoteAddress || '');++ // The gateway is the satellite's one trusted hop (TRUST_PROXY): it passes on the address it++ // resolved itself (req.ip, from nginx), never a chain a client could have started.++ const client = String(req.ip || req.socket.remoteAddress || '');++ headers['x-forwarded-for'] = client;++ headers['x-real-ip'] = client;++ headers['cf-connecting-ip'] = client;const up = http.request({ host: '127.0.0.1', port, method: req.method, path: req.originalUrl || req.url, headers }, (r) => {const out = {};for (const [k, v] of Object.entries(r.headers)) if (!HOP.has(k)) out[k] = v;nnew file mode 100644@@@ -0,0 +1,156 @@++'use strict';++// The guard on the gateway's own tools: the net tools count against their descriptor's class and++// cost, every tool that reaches a target is throttled per target across all callers, the port++// checker cannot sweep (per request and per caller, in report mode too), a webhook bin belongs to++// whoever made it (session cookie or token) and one address cannot fill the global cap, a principal++// needs tools.net.probe for probes, and the proxy to a satellite passes the one address it resolved.++// Mock resolver and transports: nothing touches a real network.++const assert = require('assert');++const crypto = require('crypto');++const http = require('http');++const express = require('express');++const cookieParser = require('cookie-parser');++const { EventEmitter } = require('events');++const contracts = require('openvibe-contracts');++const { createEgress } = require('../../_shared/egress');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');++const createNetRoutes = require('../server/net/routes');++const createDevRoutes = require('../server/dev/routes');++const { proxyTo } = require('../server/registry/host-middleware');++++const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });++const ISSUER = 'https://openvibe.network';++const t = () => Math.floor(Date.now() / 1000);++const svc = (cap) => contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: 'svc:partner', actor_type: 'service', aud: ['openvibe.tools'], cap, ns: [], iat: t(), exp: t() + 900, jti: `tok_${crypto.randomBytes(12).toString('hex')}` }, privateKey);++++const lookup = (host, _opts, cb) => cb(null, [{ address: '93.184.216.34', family: 4 }]);++const probes = [];++const egress = createEgress({++ lookup,++ tcpConnect: ({ host, port }) => { probes.push(`${host}:${port}`); const s = new EventEmitter(); s.setTimeout = () => {}; s.destroy = () => {}; s.end = () => {}; process.nextTick(() => s.emit('connect')); return s; },++ tlsConnect: () => { throw new Error('unused'); },++ httpRequest: () => { throw new Error('unused'); },++ httpsRequest: () => { throw new Error('unused'); },++});++++async function serve(app) {++ const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });++ return { base: `http://127.0.0.1:${srv.address().port}`, srv, close: () => new Promise(r => srv.close(r)) };++}++++(async () => {++ const guard = createGuard({++ app: 'gateway', contracts, issuer: ISSUER, keys: { get: () => publicKey },++ specs: [...require('../server/net/descriptors').SPECS, ...require('../server/dev/descriptors').SPECS],++ env: { TOOLS_GUARD: 'report' }, log: { log() {}, warn() {}, error() {} }, pruneIntervalMs: 0,++ });++ const app = express();++ app.set('trust proxy', TRUST_PROXY);++ app.use(cookieParser());++ app.use(guard.identify);++ app.use('/api/net', createNetRoutes(null, null, { egress, guard }));++ app.use('/api/dev', createDevRoutes(null, null, { egress, guard }));++ const s = await serve(app);++ const get = (p, headers = {}) => fetch(s.base + p, { headers });++ const from = (ip, extra = {}) => ({ 'X-Forwarded-For': ip, ...extra });++ try {++ // ── Quota by descriptor: a ping is tools-probe at cost 3 ──++ let r = await get('/api/net/ping/one.example?count=1', from('198.51.100.1'));++ assert.strictEqual(r.status, 200);++ assert.strictEqual(r.headers.get('ratelimit-limit'), '15', 'tools-probe, anonymous: a bucket of 15');++ assert.strictEqual(r.headers.get('ratelimit-remaining'), '12', 'one ping costs 3');++++ // ── Per-target throttle: ping 6 a minute against one host, whoever asks, in report mode too ──++ const codes = [];++ for (let i = 2; i <= 7; i++) codes.push((await get('/api/net/ping/ONE.example.?count=1', from(`198.51.100.${i}`))).status);++ assert.deepStrictEqual(codes, [200, 200, 200, 200, 200, 429]);++ r = await get('/api/net/headers/https%3A%2F%2Fone.example%2Fpath', from('198.51.100.30'));++ assert.strictEqual(r.status, 429, 'other tools share the host\'s budget');++ const body = await r.json();++ assert.deepStrictEqual([body.code, body.scope], ['tools.quota.exceeded', 'target']);++ assert.ok(Number(r.headers.get('retry-after')) >= 1);++ assert.strictEqual((await get('/api/net/ping/two.example?count=1', from('198.51.100.2'))).status, 200, 'another host is fine');++++ // ── The port checker: per request, per caller ──++ const ports = (n, base = 1000) => Array.from({ length: n }, (_, i) => base + i).join(',');++ r = await get(`/api/net/port/three.example?ports=${ports(21)}`, from('198.51.100.50'));++ assert.strictEqual(r.status, 400, 'more than 20 ports is refused, not silently cut');++ probes.length = 0;++ for (let i = 0; i < 5; i++) {++ r = await get(`/api/net/port/h${i}.example?ports=${ports(20)}`, from('198.51.100.51'));++ assert.strictEqual(r.status, 200, `check ${i + 1}`);++ }++ assert.strictEqual(probes.length, 100);++ r = await get('/api/net/port/h9.example?ports=22', from('198.51.100.51'));++ assert.strictEqual(r.status, 429, 'the 101st port in ten minutes is refused (a hard limit)');++ assert.strictEqual((await r.json()).scope, 'ports');++ assert.strictEqual(probes.length, 100, 'nothing was dialled');++++ // ── Probes through the API need tools.net.probe (report mode records, enforce refuses) ──++ r = await get('/api/net/ping/four.example?count=1', { Authorization: `Bearer ${svc(['tools.tool.run'])}` });++ assert.strictEqual(r.status, 200, 'report mode: recorded, not refused');++ assert.ok(guard.store.abuseRows().some(x => x.reason === 'capability' && x.principal === 'svc:partner' && x.tool === 'ping'));++ r = await get('/api/net/ping/five.example?count=1', { Authorization: `Bearer ${svc(['tools.net.probe'])}` });++ assert.strictEqual(r.status, 200);++ assert.strictEqual(r.headers.get('ratelimit-limit'), '300', 'the service tier');++++ // ── Webhook bins belong to their maker ──++ r = await fetch(`${s.base}/api/dev/webhook/bins`, { method: 'POST', headers: from('203.0.113.10') });++ const made = await r.json();++ const cookie = String(r.headers.get('set-cookie') || '').split(';')[0];++ assert.match(cookie, /^ov_tools_jobs=/, 'the maker gets a session');++ assert.ok(made.ok && made.binId);++ await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}/in`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{"hello":1}' });++ r = await get(`/api/dev/webhook/bins/${made.binId}`, { cookie });++ assert.strictEqual(r.status, 200);++ assert.strictEqual((await r.json()).requestCount, 1, 'the maker reads it');++ r = await get(`/api/dev/webhook/bins/${made.binId}`, from('203.0.113.10'));++ assert.strictEqual(r.status, 404, 'anyone else (even from the same address) gets 404');++ r = await get(`/api/dev/webhook/bins/${made.binId}`, { cookie: `ov_tools_jobs=${crypto.randomBytes(24).toString('base64url')}` });++ assert.strictEqual(r.status, 404, 'another session gets 404');++ r = await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}`, { method: 'DELETE', headers: from('203.0.113.99') });++ assert.strictEqual((await r.json()).deleted, false, 'and cannot delete it');++ assert.strictEqual((await get(`/api/dev/webhook/bins/${made.binId}`, { cookie })).status, 200, 'still there');++ r = await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}`, { method: 'DELETE', headers: { cookie } });++ assert.strictEqual((await r.json()).deleted, true, 'the maker can');++++ // Caps: per owner and per address (report mode records; a guard in enforce mode refuses).++ const enforce = createGuard({ app: 'gateway', contracts, issuer: ISSUER, keys: { get: () => publicKey }, specs: require('../server/dev/descriptors').SPECS, env: { TOOLS_GUARD: 'enforce' }, log: { log() {}, warn() {}, error() {} }, pruneIntervalMs: 0 });++ const app2 = express();++ app2.set('trust proxy', TRUST_PROXY);++ app2.use(cookieParser());++ app2.use('/api/dev', createDevRoutes(null, null, { egress, guard: enforce }));++ const s2 = await serve(app2);++ try {++ const make = (headers) => fetch(`${s2.base}/api/dev/webhook/bins`, { method: 'POST', headers });++ const first = await make(from('203.0.113.20'));++ const c1 = String(first.headers.get('set-cookie')).split(';')[0];++ const mine = [first.status];++ for (let i = 0; i < 5; i++) mine.push((await make({ ...from('203.0.113.20'), cookie: c1 })).status);++ assert.deepStrictEqual(mine, [200, 200, 200, 200, 200, 429], 'five bins per owner');++ const fresh = [];++ for (let i = 0; i < 6; i++) fresh.push((await make(from('203.0.113.20'))).status); // a new cookie every time++ assert.deepStrictEqual(fresh, [200, 200, 200, 200, 200, 429], 'ten per address, whatever cookie is shown');++ assert.strictEqual((await make(from('203.0.113.21'))).status, 200, 'another address is fine');++ } finally { await s2.close(); enforce.close(); }++++ // ── The proxy to a satellite: the one address the gateway resolved, nothing a client sent ──++ let seen = null;++ const sat = http.createServer((req, res) => { seen = req.headers; res.end('ok'); });++ await new Promise(r => sat.listen(0, '127.0.0.1', r));++ const front = express();++ front.set('trust proxy', TRUST_PROXY);++ front.use((req, res) => proxyTo(sat.address().port, { tool: 'png', role: 'canonical', canonicalHost: 'png.openvibe.tools', shortHost: 'png.openvibe.tools', host: 'png.openvibe.tools' }, req, res));++ const f = await serve(front);++ try {++ await fetch(`${f.base}/x`, { headers: { 'X-Forwarded-For': '6.6.6.6, 203.0.113.5', 'CF-Connecting-IP': '6.6.6.6' } });++ assert.strictEqual(seen['x-forwarded-for'], '203.0.113.5', 'one address: the one nginx gave');++ assert.strictEqual(seen['x-real-ip'], '203.0.113.5');++ assert.strictEqual(seen['cf-connecting-ip'], '203.0.113.5');++ } finally { await f.close(); await new Promise(r => sat.close(r)); }++ } finally {++ await s.close();++ guard.close();++ }++ console.log('gateway guard: net quotas by descriptor, per-target throttle across tools and callers, port cap, probe capability, webhook bin ownership and caps, proxy address: all checks passed');++})().catch((err) => { console.error(err); process.exit(1); });@@@ -11,6 +11,7 @@ const { EventEmitter } = require('events');const { Readable } = require('stream');const { createEgress } = require('../../_shared/egress');const createNetRoutes = require('../server/net/routes');++const { TRUST_PROXY } = require('../../_shared/guard');const { createCache } = require('../server/net/cache');const { parseRobots, testRobots, analyseSitemap } = require('../server/net/checks');const { NET_TOOLS } = require('../server/net/config');@@@ -182,7 +183,7 @@ async function fakeFetch(url) {// ── Through the routes ──const egress = createEgress({ lookup, tcpConnect, tlsConnect: () => { throw new Error('unused'); }, httpRequest: fakeHttp, httpsRequest: fakeHttp });const app = express();-- app.set('trust proxy', 1); // what the gateway sets: the host's nginx is the one hop++ app.set('trust proxy', TRUST_PROXY); // what the gateway sets: the host's nginx on loopback is the one hopapp.use('/api/net', createNetRoutes(null, null, { egress, resolverFor, tlsUpgrade, fetch: fakeFetch, reverse: async (ip) => (ip === '2606:4700:4700::1111' ? ['one.one.one.one'] : []) }));const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });const base = `http://127.0.0.1:${srv.address().port}`;@@@ -197,7 +198,7 @@ async function fakeFetch(url) {r = await get('/api/net/myip');assert.strictEqual(r.body.ip, '127.0.0.1');const gatewaySrc = fs.readFileSync(path.join(__dirname, '..', 'server', 'index.js'), 'utf8');-- assert.match(gatewaySrc, /app\.set\('trust proxy', 1\)/, 'the gateway trusts exactly one proxy hop');++ assert.match(gatewaySrc, /app\.set\('trust proxy', TRUST_PROXY\)/, 'the gateway trusts exactly one proxy hop, on loopback');// Upstream cache: ip-api once per IP, RDAP and DoH once per question.fetched.length = 0;ddeleted file mode 100644@@@ -1,120 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Img.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)--// Lightweight auth — no DB, no linked_accounts.--// Verifies the shared ov_token cookie offline against the--// Network's public key and attaches decoded claims to req.user.--// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise--// fetched from the Network's /api/.well-known/jwks endpoint.--// ═══════════════════════════════════════════════════════════════----const jwt = require('jsonwebtoken');--const fs = require('fs');--const path = require('path');--const config = require('./config');----let publicKey = null;--let retryTimer = null;----const ISSUER = config.networkUrl;--const JWKS_PATH = '/api/.well-known/jwks';--const FETCH_TIMEOUT_MS = 5000;--const RETRY_INTERVAL_MS = 60 * 1000;----function loadKeyFromFile() {-- for (const p of config.publicKeyPaths) {-- try {-- const resolved = path.resolve(p);-- if (fs.existsSync(resolved)) {-- publicKey = fs.readFileSync(resolved, 'utf8');-- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);-- return true;-- }-- } catch { /* try next */ }-- }-- return false;--}----async function fetchKeyFromNetwork() {-- for (const base of [config.networkInternalUrl, config.networkUrl]) {-- try {-- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });-- if (!res.ok) continue;-- const body = await res.json();-- if (body && body.public_key) {-- publicKey = body.public_key;-- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);-- return true;-- }-- } catch { /* try next base */ }-- }-- return false;--}----function loadPublicKey() {-- if (loadKeyFromFile()) return;-- fetchKeyFromNetwork().then((loaded) => {-- if (loaded || retryTimer) return;-- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);-- retryTimer = setInterval(async () => {-- if (await fetchKeyFromNetwork()) {-- clearInterval(retryTimer);-- retryTimer = null;-- }-- }, RETRY_INTERVAL_MS);-- if (retryTimer.unref) retryTimer.unref();-- });--}--loadPublicKey();----function extractToken(req) {-- const authHeader = req.headers.authorization;-- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);-- if (req.cookies?.ov_token) return req.cookies.ov_token;-- if (req.cookies?.token) return req.cookies.token;-- return null;--}----function verifyToken(token) {-- if (!publicKey || !token) return null;-- try {-- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });-- } catch {-- return null;-- }--}----/**-- * Optional auth — attaches req.user if valid token present.-- * Never blocks the request; anonymous users get req.user = null.-- */--function optionalAuth(req, _res, next) {-- const token = extractToken(req);-- if (token) {-- const decoded = verifyToken(token);-- if (decoded) {-- req.user = decoded;-- req.token = token;-- }-- }-- next();--}----/**-- * Required auth — 401 if no valid token.-- */--function requireAuth(req, res, next) {-- const token = extractToken(req);-- if (!token) return res.status(401).json({ error: 'Authentication required' });-- const decoded = verifyToken(token);-- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });-- req.user = decoded;-- req.token = token;-- next();--}----/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */--function getPublicKey() { return publicKey; }----module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };@@@ -11,6 +11,7 @@// ═══════════════════════════════════════════════════════════════const { DOMAIN_MAP } = require('./domain-map');++const guardLimits = require('../../_shared/guard/limits');// tools/convert.js FORMAT_CONFIG: what convert can write, and the media type each one is.const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', webp: 'image/webp', avif: 'image/avif', tiff: 'image/tiff', bmp: 'image/bmp', gif: 'image/gif', ico: 'image/x-icon' };@@@ -18,8 +19,9 @@ const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', wconst ACCEPT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/bmp', 'image/gif', 'image/heic', 'image/heif', 'image/svg+xml', 'image/x-icon', 'image/vnd.microsoft.icon'];const MAX_BYTES = 50 * 1024 * 1024; // config.js upload.maxFileSizeconst TIMEOUT_MS = 5 * 60 * 1000; // process.js img.process timeoutMs--const SHARP_PIXELS = 268402689; // sharp's default limitInputPixels (0x3FFF²), not raised here--const CODEC_PIXELS = 100 * 1024 * 1024; // tools/codec.js MAX_PIXELS: BMP written or read by our codec++// tools/codec.js inputPixels(): no upload larger than this is decoded (sharp's limitInputPixels, and the++// BMP, ICO and HEIC readers), 40 megapixels unless TOOLS_MAX_INPUT_PIXELS says otherwise.++const MAX_PIXELS = guardLimits.bounds().maxInputPixels;const SAME_AS_INPUT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/gif', 'image/bmp'];const RESULT = {@@@ -74,7 +76,7 @@ function tool(id, extra = {}) {input: INPUT[op](ctx.defaultFormat),files: { min: 1, max: 1, accept: ACCEPT, maxBytes: MAX_BYTES },output: { kind: 'file', mime: out, schema: RESULT },-- limits: { timeoutMs: TIMEOUT_MS, maxPixels: ctx.defaultFormat === 'bmp' ? CODEC_PIXELS : SHARP_PIXELS },++ limits: { timeoutMs: TIMEOUT_MS, maxPixels: MAX_PIXELS },auth: { anonymous: true, capability: 'tools.tool.run' },quotaClass: 'tools-job', cost: op === 'convert' && ['avif', 'ico'].includes(ctx.defaultFormat) ? 8 : 5, egress: false,example: { input: {} },@@@ -13,13 +13,11 @@ const cookieParser = require('cookie-parser');const path = require('path');const fs = require('fs');++const rateLimit = require('express-rate-limit');const config = require('./config');--const auth = require('./auth');--const { optionalAuth } = auth;const { resolveContext, DOMAIN_MAP } = require('./domain-map');const { getTool, listTools } = require('./tools');const { uploadSingle } = require('./middleware/upload');--const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');const retention = require('./retention/manager');const { buildOptions, describe, processBuffer, defineJobs } = require('./process');const codec = require('./tools/codec');@@@ -27,6 +25,7 @@ const { hostGuard, ownHost } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');const jobsRuntime = require('../../_shared/jobs');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const contracts = require('openvibe-contracts');const sdk = require('openvibe-sdk');@@@ -40,13 +39,28 @@ const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-img', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else++// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the++// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.++const SPECS = require('./descriptors').SPECS;++const guard = createGuard({++ app: 'img', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,++ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,++});++const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);++/** The tool a request is for before its body is read: the host's own, else the operation it defaults to. */++const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('img.process', req.ctx.defaultOp || 'convert', null) || { id: null }).id);++/** The tool a parsed request runs: its `tool` (operation) on this host. */++const toolOf = (req) => { const d = guard.toolForJob('img.process', String((req.body && req.body.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : hostTool(req); };++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-img', release: release.release,checks: [@@@ -56,19 +70,20 @@ observe({ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),-- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),++ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),...jobsRuntime.readyChecks(() => jobs),codec.heif.readyCheck('HEIC (iPhone) photos: heif-dec or heif-convert with an HEVC decoder plugin (libheif-examples + libheif-plugin-libde265); other formats work without it'),],jobs: () => jobs,});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));// ── Security ─────────────────────────────────────────────────--app.set('trust proxy', 2); // Cloudflare → Nginx → Node++app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only addressapp.use(helmet({contentSecurityPolicy: {directives: {@@@ -100,8 +115,9 @@ app.use(exceptRegistry(cors({credentials: true,})));--// ── Rate Limiting ────────────────────────────────────────────--app.use('/api/', apiLimiter);++// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──++app.use(guard.identify);++app.use('/api/', apiLimiter, guard.apiQuota);// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's image tools ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -112,9 +128,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));// ── Analytics Middleware ─────────────────────────────────────app.use(analytics.middleware());--// ── Auth (optional on all routes) ────────────────────────────--app.use(optionalAuth);--// ── Hosts ────────────────────────────────────────────────────// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the@@@ -136,8 +149,11 @@ app.get('/api/health', (_req, res) => {res.json({ status: 'ok', service: 'openvibe-img', version: '1.0.0', files: stats, jobs: jobs.stats() });});--// Domain context (frontend calls this on load to get branding)++// Domain context (frontend calls this on load to get branding). It also starts this browser's session++// (ov_tools_jobs), so the page's first job already counts as a session rather than an address.app.get('/api/context', (req, res) => {++ guard.ensureSession(req, res);++ res.set('Cache-Control', 'private, no-store');const ctx = req.ctx;const tools = listTools();res.json({@@@ -161,7 +177,7 @@ app.get('/api/tools', (_req, res) => {});// ── Main Processing Endpoint ─────────────────────────────────--app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {try {const toolId = req.body.tool || req.ctx.defaultOp || 'convert';const tool = getTool(toolId);@@@ -183,6 +199,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,res.json({ success: true, download: saved, ...describe(toolId, result) });} catch (err) {++ if (guard.toolRefused(req, res, err, toolOf(req))) return;if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });console.error('[Process] Error:', err.message);res.status(422).json({ error: err.message || 'Image processing failed' });@@@ -190,7 +207,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,});// ── Direct Download (inline preview) ─────────────────────────--app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, async (req, res) => {++app.post('/api/process/direct', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {try {const toolId = req.body.tool || req.ctx.defaultOp || 'convert';const tool = getTool(toolId);@@@ -206,6 +223,7 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn});res.send(result.buffer);} catch (err) {++ if (guard.toolRefused(req, res, err, toolOf(req))) return;if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });console.error('[Process/Direct] Error:', err.message);res.status(422).json({ error: err.message || 'Image processing failed' });@@@ -217,10 +235,11 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn// followed over SSE, reattachable by id after a reload or a restart (apps/_shared/jobs).const jobs = jobsRuntime.setupJobs({app, service: 'img', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,-- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,++ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,define: defineJobs,receive: uploadSingle,-- limiters: [burstLimiter, processLimiter],++ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],++ jobTool: (req, type, input) => { const d = guard.toolForJob('img.process', String((input && input.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : null; },defaults(req, input) {const out = { ...input };if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';@@@ -301,6 +320,7 @@ function shutdown() {analytics.destroy();analyticsDb.close();jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}ddeleted file mode 100644@@@ -1,48 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// Img.OpenVibe — Rate Limiting Middleware--// Tiered rate limits: anonymous vs authenticated users.--// Includes anti-abuse burst protection.--// ═══════════════════════════════════════════════════════════════----const rateLimit = require('express-rate-limit');----/** General API rate limit */--const apiLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 120 : 60,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Too many requests. Please try again later.' },--});----/**-- * Processing rate limit — stricter, tiered by auth.-- * Anonymous: 10 conversions/minute-- * Authenticated: 30 conversions/minute-- */--const processLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: (req) => req.user ? 30 : 10,-- standardHeaders: true,-- legacyHeaders: false,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },--});----/**-- * Anti-burst limiter — prevents rapid-fire abuse (bot attacks).-- * Very short window: max 4 requests per 5 seconds per IP.-- */--const burstLimiter = rateLimit({-- windowMs: 5 * 1000,-- max: 4,-- standardHeaders: false,-- legacyHeaders: false,-- keyGenerator: (req) => req.ip,-- message: { error: 'Too many requests in quick succession. Please slow down.' },--});----module.exports = { apiLimiter, processLimiter, burstLimiter };@@@ -12,7 +12,9 @@// it a HEIC upload answers 503 tools.unavailable. AVIF and other HEIF files go to sharp.//// open(buffer) is the one way the tools read an upload: it decodes once and hands out fresh--// sharp pipelines over the decoded pixels.++// sharp pipelines over the decoded pixels. No input larger than the guard's pixel limit (40++// megapixels, TOOLS_MAX_INPUT_PIXELS; apps/_shared/guard/limits.js) is ever decoded: the size is++// read from the header first, and every sharp pipeline carries limitInputPixels as the backstop.// ═══════════════════════════════════════════════════════════════const { spawn, spawnSync } = require('child_process');@@@ -21,8 +23,9 @@ const os = require('os');const path = require('path');const sharp = require('sharp');const { createBinary, Unavailable } = require('../../../_shared/binaries');++const guardLimits = require('../../../_shared/guard/limits');--// Largest picture read from or written to BMP (and ICO): 100 megapixels, 400 MB of RGBA.++// Largest picture written to BMP: 100 megapixels, 400 MB of RGBA (what is read is limited by inputPixels()).const MAX_PIXELS = 100 * 1024 * 1024;const HEIF_TIMEOUT_MS = 60_000;@@@ -30,6 +33,22 @@ class ImageError extends Error {constructor(message) { super(message); this.status = 422; this.expose = true; }}++/** The largest input picture (width × height) any tool decodes. */++const inputPixels = () => guardLimits.bounds().maxInputPixels;++++/** A picture over the pixel limit: 413, recorded by the guard as a hard refusal ('pixels'). */++function tooManyPixels(width, height) {++ const mp = (n) => `${Math.round(n / 1e5) / 10} megapixels`;++ const err = new ImageError(`This picture is ${width}×${height} (${mp(width * height)}); the limit is ${mp(inputPixels())}.`);++ err.status = 413;++ err.code = 'tools.file.too_large';++ err.guardReason = 'pixels';++ return err;++}++function checkPixels(width, height) {++ if (width * height > inputPixels()) throw tooManyPixels(width, height);++}++// ── Format sniffing ──────────────────────────────────────────const HEVC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);@@@ -96,7 +115,7 @@ function decodeDib(buf, dib, pixelsAt, ico) {const topDown = height < 0;height = Math.abs(height);if (!(width > 0 && height > 0)) throw new ImageError('This BMP file has no pixels');-- if (width * height > MAX_PIXELS) throw new ImageError(`This BMP picture is larger than ${MAX_PIXELS / 1024 / 1024} megapixels`);++ checkPixels(width, height);if (![1, 4, 8, 16, 24, 32].includes(bpp)) throw new ImageError(`${bpp}-bit BMP files are not supported`);if (![0, 1, 2, 3, 6].includes(compression)) throw new ImageError('This BMP compression (JPEG/PNG inside BMP) is not supported');@@@ -206,6 +225,17 @@ function decodeBmp(buf) {// ── ICO reader ───────────────────────────────────────────────++/** The largest entry's size as the directory states it (0 means 256). */++function icoLargest(buf) {++ let best = null;++ for (let i = 0; i < buf.readUInt16LE(4) && 6 + i * 16 + 16 <= buf.length; i++) {++ const e = 6 + i * 16;++ const width = buf[e] || 256, height = buf[e + 1] || 256;++ if (!best || width * height > best.width * best.height) best = { width, height };++ }++ return best;++}++/** The largest (then deepest) entry of an .ico → a sharp input: PNG bytes, or RGBA pixels. */function decodeIco(buf) {const n = buf.readUInt16LE(4);@@@ -272,20 +302,30 @@ function decodeHeic(buf) {* Decode an upload once. → { format: 'bmp'|'ico'|'heic'|<sharp format>, sharp(): a fresh pipeline }*/async function open(buf) {++ const limitInputPixels = inputPixels();++ /** Header size first (metadata reads the header only): a picture over the limit is refused before anything is decoded. */++ const sized = async (input) => {++ let meta;++ try { meta = await sharp(input, { limitInputPixels: false }).metadata(); } catch { throw new ImageError('This file is not an image this tool can read'); }++ checkPixels(meta.width || 0, meta.pageHeight || meta.height || 0);++ return meta;++ };const kind = sniff(buf);if (kind === 'bmp' || kind === 'ico') {++ // An ICO entry's header says its size before it is decoded; PNG entries are checked by sharp's header.++ if (kind === 'ico') { const e = icoLargest(buf); if (e) checkPixels(e.width, e.height); }const d = kind === 'bmp' ? decodeBmp(buf) : decodeIco(buf);-- if (d.png) return { format: kind, sharp: () => sharp(d.png) };++ if (d.png) { await sized(d.png); return { format: kind, sharp: () => sharp(d.png, { limitInputPixels }) }; }const raw = { width: d.width, height: d.height, channels: 4 };-- return { format: kind, sharp: () => sharp(d.data, { raw }) };++ return { format: kind, sharp: () => sharp(d.data, { raw, limitInputPixels }) };}if (kind === 'heic') {const png = await decodeHeic(buf);-- return { format: 'heic', sharp: () => sharp(png) };++ await sized(png);++ return { format: 'heic', sharp: () => sharp(png, { limitInputPixels }) };}-- let format = null;-- try { format = (await sharp(buf).metadata()).format || null; } catch { throw new ImageError('This file is not an image this tool can read'); }-- return { format, sharp: () => sharp(buf) };++ const meta = await sized(buf);++ return { format: meta.format || null, sharp: () => sharp(buf, { limitInputPixels }) };}// ── BMP writer ───────────────────────────────────────────────@@@ -334,4 +374,4 @@ async function toBmp(pipeline) {return { buffer: encodeBmp(data, info.width, info.height), width: info.width, height: info.height };}--module.exports = { open, sniff, decodeBmp, decodeIco, encodeBmp, toBmp, heif, ImageError, MAX_PIXELS, HEIC_SETTING_UP };++module.exports = { open, sniff, decodeBmp, decodeIco, encodeBmp, toBmp, heif, ImageError, MAX_PIXELS, HEIC_SETTING_UP, inputPixels, checkPixels };nnew file mode 100644@@@ -0,0 +1,80 @@++'use strict';++// The guard on the image satellite: no picture over the pixel limit is decoded (sharp's++// limitInputPixels, checked from the header first; BMP and ICO headers too), sniffing refuses a file++// that is not what its name and type say, the sync endpoint's refusals are problem+json, and the++// descriptor tells the same limit. TOOLS_MAX_INPUT_PIXELS is lowered here so small pictures show it.++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++++process.env.TOOLS_MAX_INPUT_PIXELS = '10000'; // before anything reads the limit++const sharp = require('sharp');++const codec = require('../server/tools/codec');++const convert = require('../server/tools/convert');++const { startApp } = require('../../_shared/test/spawn');++++(async () => {++ // ── The codec: the header's size decides, before anything is decoded ──++ const small = await sharp({ create: { width: 100, height: 100, channels: 3, background: '#336699' } }).png().toBuffer();++ const big = await sharp({ create: { width: 200, height: 100, channels: 3, background: '#336699' } }).png().toBuffer();++ assert.strictEqual(codec.inputPixels(), 10000);++ const ok = await convert(small, { format: 'webp' });++ assert.strictEqual(ok.mime, 'image/webp', '100×100 is within 10 000 pixels');++ await assert.rejects(convert(big, { format: 'webp' }), (e) => e.status === 413 && e.code === 'tools.file.too_large' && e.guardReason === 'pixels' && /200×100/.test(e.message));++ // A BMP header claiming 30 000 × 30 000 is refused without allocating its pixels.++ const bmp = Buffer.alloc(54);++ bmp.write('BM', 0, 'latin1'); bmp.writeUInt32LE(54, 10); bmp.writeUInt32LE(40, 14);++ bmp.writeInt32LE(30000, 18); bmp.writeInt32LE(30000, 22); bmp.writeUInt16LE(1, 26); bmp.writeUInt16LE(24, 28);++ await assert.rejects(codec.open(bmp), (e) => e.guardReason === 'pixels');++ // An ICO whose directory says 256×256 (65 536 pixels) is refused from the directory.++ const ico = Buffer.concat([Buffer.from([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 32, 0]), Buffer.from([8, 0, 0, 0, 22, 0, 0, 0]), small.subarray(0, 8)]);++ await assert.rejects(codec.open(ico), (e) => e.guardReason === 'pixels');++ // The descriptors say the same limit.++ const { SPECS } = require('../server/descriptors');++ assert.ok(SPECS.every(s => s.limits.maxPixels === 10000), 'every image tool\'s limits.maxPixels is the guard\'s');++++ // ── Through the app ──++ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-img-guard-'));++ const app = await startApp('img', { DATA_DIR: data, UPLOADS_DIR: path.join(data, 'uploads'), OUTPUT_DIR: path.join(data, 'output'), TOOLS_MAX_INPUT_PIXELS: '10000' });++ const post = (buf, name, type, p = '/api/process') => {++ const f = new FormData();++ f.append('file', new Blob([buf], { type }), name);++ f.append('tool', 'convert');++ f.append('format', 'png');++ return fetch(`${app.base}${p}`, { method: 'POST', body: f });++ };++ try {++ let r = await post(big, 'big.png', 'image/png');++ assert.strictEqual(r.status, 413, 'the pixel limit applies in report mode too');++ let body = await r.json();++ assert.strictEqual(r.headers.get('content-type'), 'application/problem+json');++ assert.strictEqual(body.code, 'tools.file.too_large');++ r = await post(Buffer.from('%PDF-1.4\n%%EOF\n'), 'photo.png', 'image/png', '/api/process/direct');++ assert.strictEqual(r.status, 415, 'a PDF called photo.png is refused');++ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');++ r = await post(small, 'photo.gif', 'image/gif');++ body = await r.json();++ assert.strictEqual(r.status, 200, JSON.stringify(body));++ assert.strictEqual(body.output.ext, 'png');++ // The job API refuses the same picture the same way (hard limit): the job fails with the code.++ const f = new FormData();++ f.append('type', 'img.process');++ f.append('input', JSON.stringify({ tool: 'convert', format: 'png' }));++ f.append('file', new Blob([Buffer.from('not an image at all')], { type: 'image/png' }), 'x.png');++ r = await fetch(`${app.base}/api/v1/jobs`, { method: 'POST', body: f });++ assert.strictEqual(r.status, 415, 'a job upload is sniffed before it is accepted');++ // Answers carry RateLimit headers and the sync endpoint is behind the semaphore.++ assert.ok(r.headers.get('ratelimit-limit'), 'RateLimit-Limit');++ const metrics = await (await fetch(`${app.base}/metrics`)).text();++ assert.match(metrics, /tools_guard_refused_total\{reason="pixels",tool="convert"\} 1/);++ assert.match(metrics, /tools_guard_refused_total\{reason="sniff",tool="convert"\} [12]/);++ assert.match(metrics, /tools_guard_sync\{kind="limit"\} 2/);++ assert.match(metrics, /tools_guard_enforcing 0/);++ assert.ok(fs.existsSync(path.join(data, 'guard.db')), 'guard.db in the data directory');++ } finally {++ await app.kill();++ fs.rmSync(data, { recursive: true, force: true });++ }++ console.log('img guard: pixel limit from headers (PNG, BMP, ICO) and in descriptors, sniffing on sync and job uploads, problem+json, metrics: all checks passed');++})().catch((err) => { console.error(err); process.exit(1); });@@@ -24,19 +24,32 @@ const { internalOk } = require('../../_shared/internal-auth');const { hostGuard, stampedPage } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');++const { createPacer, Busy } = require('../../_shared/guard/semaphore');const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-maps', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, else the address — the food app forwards its++// visitor's), the tools-map quota on the data routes (descriptor maps: cost 2 a call) and the abuse log++// (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.++const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';++const guard = createGuard({++ app: 'maps', dataDir: path.join(__dirname, '..', 'data'), Database, specs: require('./descriptors').SPECS,++ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',++ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),++});++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-maps', release: release.release,checks: [@@@ -44,6 +57,7 @@ observe({],// The public data sources maps queries (OSM and others) are not probed here: each request says when one fails.});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.@@@ -51,10 +65,10 @@ app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301,const cache = new NodeCache({ stdTTL: config.cache.search, checkperiod: 60 });// ── Middleware ──────────────────────────────────────────────--// X-Forwarded-For is believed only from loopback hops: the host's nginx, the gateway and the food--// app (which forwards its visitor's address). Maps listens on 0.0.0.0 by default, so a direct++// X-Forwarded-For is believed from exactly one hop on loopback: the host's nginx, the gateway or the++// food app (which forwards its visitor's address). Maps listens on 0.0.0.0 by default, so a direct// caller's own header must not count.--app.set('trust proxy', 'loopback');++app.set('trust proxy', TRUST_PROXY);app.use(helmet({contentSecurityPolicy: {directives: {@@@ -73,13 +87,12 @@ app.use(exceptRegistry(cors({ origin: ['https://maps.openvibe.tools', 'https://fapp.use(cookieParser());app.use(express.json());--// Rate limiter--const apiLimiter = rateLimit({-- windowMs: 60 * 1000,-- max: 30,-- message: { error: 'Too many requests — slow down, traveler.' },--});--app.use('/api/', apiLimiter);++// Rate limits: sign-in first, so limits know who is asking; everyone else counts by address (IPv6 /64).++app.use(guard.identify);++const apiLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60 * 1000, anonymous: 30, signedIn: 60, message: 'Too many requests — slow down, traveler.' });++app.use('/api/', apiLimiter, guard.apiQuota);++// The map's data lookups (OpenStreetMap, weather, food banks…): the tools-map quota, by descriptor.++app.use(['/api/geocode', '/api/search', '/api/weather', '/api/terrain', '/api/food-banks', '/api/stores', '/api/foods', '/api/meal-plan'], guard.toolQuota('maps'));// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's survival map ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -129,6 +142,9 @@ const terrain = require('./sources/terrain');const grocery = require('./sources/grocery');// ── Geocode endpoint ───────────────────────────────────────++// Nominatim's usage policy is one request a second from us in all: answers are cached, and the misses++// are paced (at most 20 waiting, none longer than 15 s; beyond that 503 with Retry-After).++const nominatim = createPacer({ minIntervalMs: 1100, queue: 20, maxWaitMs: 15_000 });app.get('/api/geocode', async (req, res) => {const { q } = req.query;if (!q) return res.status(400).json({ error: 'Missing query' });@@@ -139,11 +155,11 @@ app.get('/api/geocode', async (req, res) => {try {const axios = require('axios');-- const resp = await axios.get('https://nominatim.openstreetmap.org/search', {++ const resp = await nominatim.run(() => axios.get('https://nominatim.openstreetmap.org/search', {params: { q, format: 'json', limit: 5, countrycodes: 'us,ca,mx' },headers: { 'User-Agent': 'Maps.OpenVibe/1.0 (maps.openvibe.tools)' },timeout: 8000,-- });++ }));const results = (resp.data || []).map(r => ({lat: parseFloat(r.lat),lon: parseFloat(r.lon),@@@ -153,6 +169,7 @@ app.get('/api/geocode', async (req, res) => {cache.set(cacheKey, results, config.cache.geocode);res.json(results);} catch (err) {++ if (err instanceof Busy) return res.status(503).set('Retry-After', '15').json({ error: 'Place search is busy; try again in a few seconds.' });res.status(502).json({ error: 'Geocode failed' });}});@@@ -475,6 +492,7 @@ function shutdown() {console.log('[Maps.OpenVibe] Shutting down...');analytics.destroy();analyticsDb.close();++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}@@@ -13,6 +13,7 @@ const OVERPASS_URL = 'https://overpass-api.de/api/interpreter';const _overpassQueue = [];let _overpassRunning = 0;const MAX_CONCURRENT_OVERPASS = 2; // Max parallel Overpass requests++const MAX_QUEUED_OVERPASS = 60; // More waiting than this: refuse (a search asks a handful at most)const OVERPASS_RETRY_DELAYS = [2000, 5000, 12000]; // Backoff per retryasync function _processOverpassQueue() {@@@ -61,6 +62,9 @@ async function _runOverpassJob({ query, timeout, resolve, reject, attempt = 0 })*/function overpassQuery(query, timeout = 45000) {return new Promise((resolve, reject) => {++ if (_overpassQueue.length >= MAX_QUEUED_OVERPASS) {++ return reject(Object.assign(new Error('OpenStreetMap is busy with other searches; try again in a minute.'), { status: 503, busy: true }));++ }_overpassQueue.push({ query, timeout, resolve, reject });_processOverpassQueue();});@@@ -17,32 +17,44 @@ const { internalOk } = require('../../_shared/internal-auth');const { hostGuard } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');const analyticsDbPath = path.resolve(__dirname, '..', process.env.DATA_DIR || 'data', 'analytics.db');fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-text', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota on++// /api/ and the abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.++const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';++const guard = createGuard({++ app: 'text', dataDir: path.resolve(__dirname, '..', process.env.DATA_DIR || 'data'), Database, specs: require('./descriptors').SPECS,++ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',++ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),++});++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-text', release: release.release,checks: [ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; the text tools run in the browser and on this process' }),],});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));// ── Security ─────────────────────────────────────────────────--app.set('trust proxy', 2);++app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only addressapp.use(helmet({contentSecurityPolicy: {directives: {@@@ -73,8 +85,10 @@ app.use(exceptRegistry(cors({credentials: true,})));--// ── Rate Limiting ────────────────────────────────────────────--app.use(rateLimit({ windowMs: 60_000, max: 200 }));++// ── Who is asking, then rate limits ──────────────────────────++app.use(guard.identify);++app.use(guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 200, signedIn: 400, message: 'Too many requests. Please try again later.' }));++app.use('/api/', guard.apiQuota);// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's text and logo tools ──// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The@@@ -161,6 +175,7 @@ function shutdown() {console.log('[Text.OpenVibe] Shutting down...');analytics.destroy();analyticsDb.close();++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}ddeleted file mode 100644@@@ -1,120 +0,0 @@--'use strict';----// ═══════════════════════════════════════════════════════════════--// YT.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)--// Lightweight auth — no DB, no linked_accounts.--// Verifies the shared ov_token cookie offline against the--// Network's public key and attaches decoded claims to req.user.--// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise--// fetched from the Network's /api/.well-known/jwks endpoint.--// ═══════════════════════════════════════════════════════════════----const jwt = require('jsonwebtoken');--const fs = require('fs');--const path = require('path');--const config = require('./config');----let publicKey = null;--let retryTimer = null;----const ISSUER = config.networkUrl;--const JWKS_PATH = '/api/.well-known/jwks';--const FETCH_TIMEOUT_MS = 5000;--const RETRY_INTERVAL_MS = 60 * 1000;----function loadKeyFromFile() {-- for (const p of config.publicKeyPaths) {-- try {-- const resolved = path.resolve(p);-- if (fs.existsSync(resolved)) {-- publicKey = fs.readFileSync(resolved, 'utf8');-- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);-- return true;-- }-- } catch { /* try next */ }-- }-- return false;--}----async function fetchKeyFromNetwork() {-- for (const base of [config.networkInternalUrl, config.networkUrl]) {-- try {-- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });-- if (!res.ok) continue;-- const body = await res.json();-- if (body && body.public_key) {-- publicKey = body.public_key;-- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);-- return true;-- }-- } catch { /* try next base */ }-- }-- return false;--}----function loadPublicKey() {-- if (loadKeyFromFile()) return;-- fetchKeyFromNetwork().then((loaded) => {-- if (loaded || retryTimer) return;-- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);-- retryTimer = setInterval(async () => {-- if (await fetchKeyFromNetwork()) {-- clearInterval(retryTimer);-- retryTimer = null;-- }-- }, RETRY_INTERVAL_MS);-- if (retryTimer.unref) retryTimer.unref();-- });--}--loadPublicKey();----function extractToken(req) {-- const authHeader = req.headers.authorization;-- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);-- if (req.cookies?.ov_token) return req.cookies.ov_token;-- if (req.cookies?.token) return req.cookies.token;-- return null;--}----function verifyToken(token) {-- if (!publicKey || !token) return null;-- try {-- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });-- } catch {-- return null;-- }--}----/**-- * Optional auth — attaches req.user if valid token present.-- * Never blocks the request; anonymous users get req.user = null.-- */--function optionalAuth(req, _res, next) {-- const token = extractToken(req);-- if (token) {-- const decoded = verifyToken(token);-- if (decoded) {-- req.user = decoded;-- req.token = token;-- }-- }-- next();--}----/**-- * Required auth — 401 if no valid token.-- */--function requireAuth(req, res, next) {-- const token = extractToken(req);-- if (!token) return res.status(401).json({ error: 'Authentication required' });-- const decoded = verifyToken(token);-- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });-- req.user = decoded;-- req.token = token;-- next();--}----/** The Network public key (PEM) once loaded (readiness reports it). */--function getPublicKey() { return publicKey; }----module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey };@@@ -14,12 +14,12 @@ const path = require('path');const fs = require('fs');const config = require('./config');--const { optionalAuth } = require('./auth');const downloader = require('./downloader');const seo = require('./seo');const { hostGuard } = require('../../_shared/host-role');const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');++const { createGuard, TRUST_PROXY } = require('../../_shared/guard');// ── Analytics ────────────────────────────────────────────────const Database = require('better-sqlite3');@@@ -31,13 +31,22 @@ const analyticsDb = new Database(analyticsDbPath);analyticsDb.pragma('journal_mode = WAL');const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-yt', { retention: { days: 30 } });++// ── Guard (apps/_shared/guard) ───────────────────────────────++// Who is asking (Network sign-in with aud openvibe.tools, the browser session, else the address),++// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (data/guard.db).++// TOOLS_GUARD=report (default) records what it would refuse; the older limiters below stay until enforce.++const guard = createGuard({++ app: 'yt', dataDir: path.resolve(__dirname, '..', config.dataDir || 'data'), Database, specs: require('./descriptors').SPECS,++ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,++});++const app = express();// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real// dependencies (roadmap Track O). First, so the HTTP metrics see every request.const { observe, checks: ready, which } = require('../../_shared/observe');--observe({++const obs = observe({app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),service: 'tools-yt', release: release.release,checks: [@@@ -46,19 +55,20 @@ observe({ready.binary('yt_dlp', config.ytdlpPath, { description: 'every download runs yt-dlp' }),ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'merging video+audio and audio conversion' }),...(process.env.YT_COOKIES_FILE ? [ready.readableFile('yt_cookies', process.env.YT_COOKIES_FILE, { description: 'YT_COOKIES_FILE is set; yt-dlp is given these cookies' })] : []),-- ready.networkKey('network_key', require('./auth').getPublicKey, { description: 'recognises signed-in visitors (higher limits); anonymous use works without it' }),++ ready.networkKey('network_key', guard.keys.get, { description: 'recognises signed-in visitors (higher limits); anonymous use works without it' }),],// Download progress streams last as long as the download.skip: (req) => /^\/api\/status\/[^/]+\/stream$/.test(req.path),details: () => ({ yt_proxy_configured: !!String(process.env.YT_PROXY || '').trim() }),});++guard.attachMetrics(obs.registry);app.get('/release.json', release.handler);// Legal documents live on the apex; every tool host points there instead of answering 404.app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));// ── Security ─────────────────────────────────────────────────--app.set('trust proxy', 2);++app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address// API answers are live state, never revalidated: a 304 on /api/status froze the progress poll.app.set('etag', false);app.use(helmet({@@@ -93,9 +103,12 @@ app.use(exceptRegistry(cors({// ── Rate Limiting ────────────────────────────────────────────// Progress is one SSE connection, or a poll every 1–3 s when SSE is unavailable; the poll must// not eat the general budget (60/min would cut a download off after a minute).++// Sign-in is read first, so limits know who is asking; everyone else counts by address (IPv6 by /64).const isStatusRoute = (req) => req.method === 'GET' && /^\/status\/[a-f0-9]+(\/stream)?$/.test(req.path);--app.use('/api/', rateLimit({ windowMs: 60_000, max: 60, skip: isStatusRoute }));--app.use('/api/status/', rateLimit({ windowMs: 60_000, max: 240 }));++app.use(guard.identify);++const apiLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' });++app.use('/api/', (req, res, next) => (isStatusRoute(req) ? next() : apiLimiter(req, res, (err) => (err ? next(err) : guard.apiQuota(req, res, next)))));++app.use('/api/status/', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 240, signedIn: 240, message: 'Too many requests. Please try again later.' }));app.use('/api/', (_req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); });// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's YouTube downloader ──@@@ -107,15 +120,18 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));// ── Analytics Middleware ─────────────────────────────────────app.use(analytics.middleware());--const downloadLimiter = rateLimit({-- windowMs: 60 * 60 * 1000, // 1 hour-- max: (req) => req.user ? config.rateLimit.authedPerHour : config.rateLimit.anonPerHour,-- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,-- message: { error: 'Download limit reached. Sign in for more downloads or wait an hour.' },--});----// ── Auth ─────────────────────────────────────────────────────--app.use(optionalAuth);++// Downloads: the older hourly limit (5 anonymous, 20 signed in) until enforce mode, and the guard's++// tools-download quota (descriptor yt: cost 50) in both. A download belongs to whoever started it (the++// browser session cookie, minted on the start, or a sign-in): its status, progress stream, file and++// cancel answer 404 to anyone else, the same as for an id that does not exist.++const owners = new Map(); // download id → owner++const ownsDownload = (req, res, id) => {++ const owner = owners.get(String(id || ''));++ const who = guard.resolveCaller(req, res);++ return !!owner && !!who.owner && owner === who.owner;++};++setInterval(() => { for (const id of owners.keys()) if (!downloader.getStatus(id) && !downloader.getFile(id)) owners.delete(id); }, 10 * 60_000).unref();++const downloadLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60 * 60 * 1000, anonymous: config.rateLimit.anonPerHour, signedIn: config.rateLimit.authedPerHour, message: 'Download limit reached. Sign in for more downloads or wait an hour.' });// ── Hosts ────────────────────────────────────────────────────// Pages are only rendered for hosts this tool serves (or that the gateway vouches for with@@@ -150,7 +166,7 @@ app.post('/api/info', async (req, res) => {});// Start download--app.post('/api/download', downloadLimiter, async (req, res) => {++app.post('/api/download', downloadLimiter, guard.toolQuota('yt'), async (req, res) => {const { url, quality, title } = req.body;if (!url) return res.status(400).json({ error: 'URL is required' });if (!downloader.isValidUrl(url)) return res.status(400).json({ error: 'Only YouTube URLs are supported' });@@@ -159,7 +175,9 @@ app.post('/api/download', downloadLimiter, async (req, res) => {if (known && known.downloadable === false) return res.status(422).json({ error: known.reason });try {++ const who = guard.resolveCaller(req, res, { create: true });const { id } = await downloader.startDownload(url, quality || 'best', { title: typeof title === 'string' ? title : '' });++ owners.set(id, who.owner);res.json({ success: true, id, statusUrl: `/api/status/${id}`, streamUrl: `/api/status/${id}/stream` });} catch (err) {console.error('[Download] Error:', err.message);@@@ -169,7 +187,7 @@ app.post('/api/download', downloadLimiter, async (req, res) => {// Download status (poll for progress)app.get('/api/status/:id', (req, res) => {-- const status = downloader.getStatus(req.params.id);++ const status = ownsDownload(req, res, req.params.id) ? downloader.getStatus(req.params.id) : null;if (!status) return res.status(404).json({ error: 'Download not found' });res.json(status);});@@@ -186,6 +204,7 @@ const SSE_EVENT = { downloading: 'progress', done: 'complete', error: 'failed' }app.get('/api/status/:id/stream', (req, res) => {const id = req.params.id;++ if (!ownsDownload(req, res, id)) return res.status(404).json({ error: 'Download not found' });res.writeHead(200, {'Content-Type': 'text/event-stream; charset=utf-8',@@@ -235,14 +254,14 @@ app.get('/api/status/:id/stream', (req, res) => {// Cancel a running download (kills yt-dlp) or discard a finished fileapp.delete('/api/download/:id', (req, res) => {-- const result = downloader.cancelDownload(req.params.id);++ const result = ownsDownload(req, res, req.params.id) ? downloader.cancelDownload(req.params.id) : null;if (!result) return res.status(404).json({ error: 'Download not found' });res.json({ success: true, ...result });});// Serve downloaded fileapp.get('/api/download/:id', (req, res) => {-- const entry = downloader.getFile(req.params.id);++ const entry = ownsDownload(req, res, req.params.id) ? downloader.getFile(req.params.id) : null;if (!entry) return res.status(404).json({ error: 'File not found or expired' });res.set({@@@ -306,6 +325,7 @@ function shutdown() {console.log('[YT.OpenVibe] Shutting down...');analytics.destroy();analyticsDb.close();++ guard.close();server.close(() => process.exit(0));setTimeout(() => process.exit(1), 5000);}@@@ -89,8 +89,14 @@ process.exit(0); // the match filter skipped it: no filer = await post('/api/download', { url: 'https://www.youtube.com/watch?v=SHORTxx1234', quality: 'best' });body = await r.json();assert.strictEqual(r.status, 200, JSON.stringify(body));++ // The download belongs to the browser session the start minted: only it sees the status.++ const cookie = String(r.headers.get('set-cookie') || '').split(';')[0];++ assert.match(cookie, /^ov_tools_jobs=/, 'starting a download starts a session');++ assert.strictEqual((await fetch(`${app.base}/api/status/${body.id}`)).status, 404, 'someone else gets 404');++ assert.strictEqual((await fetch(`${app.base}/api/status/${body.id}/stream`)).status, 404);++ assert.strictEqual((await fetch(`${app.base}/api/download/${body.id}`, { method: 'DELETE' })).status, 404, 'and cannot cancel it');let st;-- for (let i = 0; i < 50; i++) { st = await (await fetch(`${app.base}/api/status/${body.id}`)).json(); if (st.status !== 'downloading') break; await sleep(50); }++ for (let i = 0; i < 50; i++) { st = await (await fetch(`${app.base}/api/status/${body.id}`, { headers: { cookie } })).json(); if (st.status !== 'downloading') break; await sleep(50); }const args = JSON.parse(fs.readFileSync(path.join(data, 'args.json'), 'utf8'));assert.strictEqual(args[args.indexOf('--match-filter') + 1], 'duration<=600 & !is_live');assert.strictEqual(args[args.indexOf('--max-filesize') + 1], '100M');
new file mode 100644
new file mode 100644
@@ -0,0 +1,152 @@
@@ -0,0 +1,152 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// Who is asking — the one resolver for the gateway and every satellite (it replaces the four copied
+// auth.js files and the job routes' own owner logic).
+//
+// principal Authorization: Bearer <service or app token>, aud openvibe.tools (tier service; a
+// developer app's sandbox token: tier sandbox, job routes only take it from apps)
+// user a Network sign-in (the ov_token cookie or a Bearer user token, aud openvibe.tools)
+// session this browser's ov_tools_jobs cookie (only its hash is ever kept)
+// anonymous nobody: counted by address, an IPv6 address by its /64
+//
+// Addresses are req.ip only (`trust proxy` is one loopback hop, ./ip.js). A first-party service on
+// loopback is not special: it is identified by its service token, and without one it is anonymous,
+// counted by the address it forwarded (as the one trusted hop) or else by its own.
+//
+// → { kind, tier, key, ipKey, ipHash, owner, id, claims, env, error? }
+// key what quotas count (session hash, user:usr_…, the principal's sub, ip:<hash>)
+// ipKey ip:<HMAC-SHA256(today's salt, address or /64)>, never the address itself
+// owner the job owner (user:usr_…, the principal's sub, session:<hash>) or null
+// error [status, code, detail] for a bad principal token on the job routes
+// ═══════════════════════════════════════════════════════════════
+
+const crypto = require('crypto');
+const { ipBucket } = require('./ip');
+const tokens = require('./tokens');
+
+const SESSION_RE = /^[A-Za-z0-9_-]{32,64}$/;
+const USER_ID_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
+const JOB_CAPS = { create: 'tools.job.create', read: 'tools.job.read', cancel: 'tools.job.cancel' };
+const COOKIE = 'ov_tools_jobs';
+
+/**
+ * @param {object} o
+ * @param {() => string|null} o.getPublicKey
+ * @param {string} o.issuer
+ * @param {string} [o.audience='openvibe.tools']
+ * @param {object} [o.contracts] openvibe-contracts, when the app has it
+ * @param {() => string} [o.salt] today's salt (guard.db); a process-local one otherwise
+ * @param {string} [o.cookieName='ov_tools_jobs']
+ * @param {boolean} [o.secureCookie]
+ */
+function createCallerResolver(o) {
+ const audience = o.audience || tokens.AUDIENCE;
+ const cookieName = o.cookieName || COOKIE;
+ const contracts = o.contracts || null;
+ let localSalt = null;
+ const salt = o.salt || (() => {
+ const day = new Date().toISOString().slice(0, 10);
+ if (!localSalt || localSalt.day !== day) localSalt = { day, salt: crypto.randomBytes(32).toString('hex') };
+ return localSalt.salt;
+ });
+ const isUserId = (sid) => (contracts ? contracts.ids.isSubjectId('user', sid) : USER_ID_RE.test(String(sid || '')));
+
+ function hashIp(ip) {
+ return crypto.createHmac('sha256', salt()).update(ipBucket(ip) || 'unknown').digest('hex').slice(0, 32);
+ }
+
+ function sessionOwner(value) {
+ return `session:${crypto.createHash('sha256').update(value).digest('hex').slice(0, 40)}`;
+ }
+
+ /** Mint the browser-session cookie now (pages call this so their first job already has one). */
+ function ensureSession(req, res) {
+ let v = req.cookies && req.cookies[cookieName];
+ if (SESSION_RE.test(String(v || ''))) return v;
+ v = crypto.randomBytes(24).toString('base64url');
+ res.cookie(cookieName, v, {
+ httpOnly: true, sameSite: 'lax', path: '/', maxAge: 7 * 24 * 60 * 60 * 1000,
+ secure: o.secureCookie != null ? o.secureCookie : process.env.NODE_ENV === 'production',
+ });
+ if (req.cookies) req.cookies[cookieName] = v; else req.cookies = { [cookieName]: v };
+ return v;
+ }
+
+ /**
+ * @param {object} req
+ * @param {object} [res] needed with create
+ * @param {object} [opt]
+ * @param {boolean} [opt.create] mint a session cookie for an anonymous caller
+ * @param {string} [opt.action] job routes: create | read | cancel (a principal needs tools.job.<action>)
+ * @param {boolean} [opt.jobs] job routes: a bad principal token is an error, not anonymity
+ */
+ function resolve(req, res, opt = {}) {
+ const ipHash = hashIp(req.ip || (req.socket && req.socket.remoteAddress));
+ const base = { ipKey: `ip:${ipHash}`, ipHash, claims: null, env: 'production' };
+
+ // 1. A principal's token (service or developer app).
+ const h = String((req.headers && req.headers.authorization) || '');
+ if (h.startsWith('Bearer ')) {
+ const token = h.slice(7).trim();
+ if (tokens.looksLikePrincipal(tokens.peek(token))) {
+ const r = tokens.verifyServiceToken(token, { publicKey: o.getPublicKey(), issuer: o.issuer, audience, acceptSandbox: true, contracts });
+ let error = null;
+ if (!r.ok) error = [401, r.code, r.reason];
+ else if (r.claims.env === 'sandbox' && !(r.claims.actor_type === 'app' && /^app:/.test(String(r.claims.sub || '')))) error = [401, 'token.sandbox_refused', 'sandbox tokens are accepted only from developer apps'];
+ if (!error && opt.action) {
+ const cap = JOB_CAPS[opt.action] || JOB_CAPS.read;
+ const denied = tokens.capabilityDenied(r.claims, cap, contracts);
+ if (denied) error = [403, denied, `${cap} not granted`];
+ }
+ if (error && opt.jobs) return { ...base, kind: 'anonymous', tier: 'anonymous', key: base.ipKey, owner: null, id: null, error };
+ if (!error) {
+ const sandbox = r.claims.env === 'sandbox';
+ return { ...base, kind: 'principal', tier: sandbox ? 'sandbox' : 'service', key: r.claims.sub, owner: r.claims.sub, id: r.claims.sub, claims: r.claims, env: sandbox ? 'sandbox' : 'production' };
+ }
+ // Elsewhere a token that does not verify is simply not believed: the caller is anonymous.
+ }
+ }
+
+ // 2. A signed-in person (req.user comes from the guard's identify(): signature, issuer, audience).
+ const user = req.user || null;
+ const sid = user && user.subject_id;
+ let session = req.cookies && req.cookies[cookieName];
+ if (!SESSION_RE.test(String(session || ''))) session = null;
+ if (user) {
+ const id = isUserId(sid) ? `user:${sid}` : `user:${String(user.sub || user.id || 'unknown')}`;
+ // A job belongs to the person's subject; a token without one (older sign-ins) keeps its jobs in the browser session.
+ let owner = isUserId(sid) ? `user:${sid}` : null;
+ if (!owner) {
+ if (!session && opt.create && res) session = ensureSession(req, res);
+ owner = session ? sessionOwner(session) : null;
+ }
+ return { ...base, kind: 'user', tier: 'user', key: id, owner, id, claims: user };
+ }
+
+ // 3. This browser's session, 4. nobody.
+ if (!session && opt.create && res) session = ensureSession(req, res);
+ if (session) {
+ const owner = sessionOwner(session);
+ return { ...base, kind: 'session', tier: 'session', key: owner, owner, id: null };
+ }
+ return { ...base, kind: 'anonymous', tier: 'anonymous', key: base.ipKey, owner: null, id: null };
+ }
+
+ return { resolve, ensureSession, hashIp, sessionOwner, cookieName };
+}
+
+/**
+ * The job routes' owner resolver (apps/_shared/jobs/http.js), on top of the one caller resolver:
+ * → { owner, kind: 'principal'|'user'|'session', claims?, env } | { error } | null (anonymous, not creating).
+ */
+function jobOwnerResolver(callers) {
+ return function resolveOwner(req, res, { create = false, action = 'read' } = {}) {
+ const c = callers.resolve(req, res, { create, action, jobs: true });
+ if (c.error) return { error: c.error };
+ if (!c.owner) return null;
+ const kind = c.kind === 'principal' ? 'principal' : c.owner.startsWith('user:') ? 'user' : 'session';
+ return { owner: c.owner, kind, claims: c.claims || undefined, env: c.env, caller: c };
+ };
+}
+
+module.exports = { createCallerResolver, jobOwnerResolver, JOB_CAPS, SESSION_RE, COOKIE };
new file mode 100644
new file mode 100644
@@ -0,0 +1,30 @@
@@ -0,0 +1,30 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// The challenge hook (decision 4: Turnstile later, not now).
+//
+// A challenge provider decides whether a caller about to be refused for a quota may instead prove it
+// is a person, and checks that proof. The guard asks it before a quota refusal of an anonymous or
+// session caller; with the default (none) nobody is ever challenged and nothing changes.
+//
+// {
+// name: 'turnstile',
+// required(req, caller, { reason, tool }) → boolean challenge instead of refusing?
+// verify(req, caller) → Promise<boolean> the proof the request carries is good
+// }
+//
+// A request that passes verify() is let through once (its quota is still counted); one that needs a
+// challenge and carries no good proof gets 403 problem tools.challenge.required, with the provider's
+// name in `challenge` so the page knows which widget to show.
+// ═══════════════════════════════════════════════════════════════
+
+const NO_CHALLENGE = Object.freeze({
+ name: 'none',
+ required() { return false; },
+ async verify() { return true; },
+});
+
+function validChallenge(c) {
+ return !!c && typeof c.required === 'function' && typeof c.verify === 'function';
+}
+
+module.exports = { NO_CHALLENGE, validChallenge };
new file mode 100644
new file mode 100644
@@ -0,0 +1,103 @@
@@ -0,0 +1,103 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// ffmpeg hardening (hard limits: they apply in report mode too).
+//
+// An upload is a file on disk, but ffmpeg decides by the bytes what to do with it: an HLS playlist or an
+// ffconcat list makes it open other files (anything the service user can read) or URLs. So every
+// input of every command and every probe gets
+// -protocol_whitelist file,pipe nothing but local files and pipes
+// -format_whitelist <the operation's demuxers> no hls, concat, image2, lavfi, sdp, tty, …
+// -t <limits.maxDurationSec> never more than the descriptor's longest input is read
+// and probeInputs() refuses an input longer than maxDurationSec before any work starts (the -t is the
+// backstop for a file whose header lies). A synchronous run gets a deadline (limits.timeoutMs) and is
+// killed like a cancelled job when it passes (the audio app runs sync calls in its job context).
+//
+// No dependencies: the app passes fluent-ffmpeg.
+// ═══════════════════════════════════════════════════════════════
+
+// Demuxer names (ffmpeg -formats): audio containers, and the video containers only the extractor takes.
+const AUDIO_DEMUXERS = ['mp3', 'wav', 'flac', 'ogg', 'mov', 'aac', 'asf', 'aiff', 'ac3', 'eac3', 'matroska', 'amr'];
+const VIDEO_DEMUXERS = ['avi', 'flv'];
+const PROTOCOLS = 'file,pipe';
+
+/** The demuxers an operation may open (extract takes video files too). */
+function formatsFor(operation) {
+ return operation === 'extract' ? [...AUDIO_DEMUXERS, ...VIDEO_DEMUXERS] : AUDIO_DEMUXERS;
+}
+
+/** Input options for one input. */
+function inputArgs({ formats = AUDIO_DEMUXERS, maxDurationSec = null } = {}) {
+ return [
+ '-protocol_whitelist', PROTOCOLS,
+ '-format_whitelist', formats.join(','),
+ ...(maxDurationSec > 0 ? ['-t', String(maxDurationSec)] : []),
+ ];
+}
+
+/** Options for ffprobe (no -t: a probe reads headers). */
+function probeArgs({ formats = AUDIO_DEMUXERS } = {}) {
+ return ['-protocol_whitelist', PROTOCOLS, '-format_whitelist', formats.join(',')];
+}
+
+/** Put the hardening options on every input of a fluent-ffmpeg command (once). */
+function applyToCommand(cmd, opts) {
+ for (const input of cmd._inputs || []) {
+ if (input.__ovHardened) continue;
+ input.options(...inputArgs(opts));
+ input.__ovHardened = true;
+ }
+}
+
+class InputRefused extends Error {
+ constructor(message, reason) { super(message); this.status = reason === 'duration' ? 413 : 415; this.code = reason === 'duration' ? 'tools.file.too_large' : 'tools.file.unsupported_type'; this.guardReason = `ffmpeg.${reason}`; this.expose = true; }
+}
+
+/**
+ * Wrap fluent-ffmpeg once: every command's inputs and every ffprobe get the options above.
+ * @param {Function} ffmpeg require('fluent-ffmpeg')
+ * @param {() => object} optionsNow { formats, maxDurationSec } for the command being started
+ * (the app reads its AsyncLocalStorage context)
+ */
+function harden(ffmpeg, optionsNow) {
+ const proto = ffmpeg.prototype;
+ if (proto.__ovHardened) return;
+ const run = proto.run;
+ proto.run = function hardenedRun(...args) {
+ applyToCommand(this, optionsNow() || {});
+ return run.apply(this, args);
+ };
+ const probe = proto.ffprobe;
+ proto.ffprobe = function hardenedProbe(...args) {
+ // ffprobe([index], [options], callback): the options go first; ours are added to them.
+ const cb = args[args.length - 1];
+ let index = null, options = [];
+ if (args.length === 3) { index = args[0]; options = args[1] || []; } else if (args.length === 2) { if (typeof args[0] === 'number') index = args[0]; else if (Array.isArray(args[0])) options = args[0]; }
+ const all = [...probeArgs(optionsNow() || {}), ...options];
+ return index === null ? probe.call(this, all, cb) : probe.call(this, index, all, cb);
+ };
+ proto.__ovHardened = true;
+}
+
+/**
+ * Probe every input (hardened) before work starts: each must open with the allowed demuxers, and
+ * together they must fit maxDurationSec. → the total duration in seconds (0 when unknown).
+ * @param {Function} probe (path) → Promise<ffprobe data> (the app's helper; hardened by harden())
+ */
+async function probeInputs(probe, paths, { maxDurationSec = null } = {}) {
+ let total = 0;
+ for (let i = 0; i < paths.length; i++) {
+ let info;
+ try { info = await probe(paths[i]); } catch {
+ throw new InputRefused(paths.length > 1 ? `File ${i + 1} is not an audio or video file this tool can read.` : 'This is not an audio or video file this tool can read.', 'format');
+ }
+ const d = parseFloat(info && info.format && info.format.duration);
+ if (Number.isFinite(d) && d > 0) total += d;
+ }
+ if (maxDurationSec > 0 && total > maxDurationSec) {
+ const len = (s) => (s >= 120 ? `${Math.round(s / 60)} minutes` : `${Math.round(s)} seconds`);
+ throw new InputRefused(`The audio is ${len(total)} long; this tool takes at most ${len(maxDurationSec)}.`, 'duration');
+ }
+ return total;
+}
+
+module.exports = { AUDIO_DEMUXERS, VIDEO_DEMUXERS, PROTOCOLS, formatsFor, inputArgs, probeArgs, applyToCommand, harden, probeInputs, InputRefused };
new file mode 100644
new file mode 100644
@@ -0,0 +1,449 @@
@@ -0,0 +1,449 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// The Tools guard (apps/_shared/guard): anti-abuse for the gateway and every satellite, driven by
+// the tools' descriptors (tools.tool@1: quotaClass, cost, auth, files.accept, limits) and the host-wide
+// numbers in ./limits.js. Like the rest of apps/_shared it has no dependencies: the app passes its
+// better-sqlite3 class, openvibe-contracts (when it has it) and its descriptor specs.
+//
+// const guard = createGuard({ app: 'img', dataDir, Database, contracts, specs, networkUrl, … });
+// app.set('trust proxy', TRUST_PROXY); // ./ip.js: one loopback hop
+// app.use(guard.identify); // req.user (Network sign-in, aud checked)
+// app.use('/api/', legacyLimiter, guard.apiQuota); // every /api/ request (tools-api)
+// app.post('/api/process', guard.toolQuota(hostTool), upload, guard.admitUpload(toolOf), guard.heavy(), …)
+//
+// What it does:
+// callers one resolver (./caller.js): anonymous (address, IPv6 /64) < session < user < service;
+// sandbox tokens keep their small allowance
+// quotas token buckets per class × tier weighted by the tool's cost (./quota.js), day allowances
+// in guard.db; RateLimit-Limit/Remaining/Reset on every counted answer, and on refusal
+// 429 problem+json tools.quota.exceeded with Retry-After
+// heavy a semaphore for synchronous heavy calls (503 tools.busy + Retry-After when it is full)
+// uploads the bytes are checked against the descriptor's files.accept (415 tools.file.unsupported_type);
+// heavy tools (auth.anonymous false) need a session cookie, a sign-in or a token
+// egress a per-target throttle across all callers (limits.perTargetPerMinute) and the port-scan cap
+// log every refusal (and every would-be refusal in report mode) in guard.db's abuse log:
+// HMAC(address, today's salt), principal or user id, tool, reason — 30 days, no raw
+// address anywhere; metric tools_guard_refused_total{reason,tool}
+// challenge a hook for a person-check (Turnstile later; ./challenge.js, a no-op now)
+//
+// Mode: TOOLS_GUARD=report (default) logs and counts what it would refuse and refuses nothing, except
+// the hard limits that apply in both modes: the image pixel limit, ffmpeg's protocol and format
+// whitelists and duration cap, upload sniffing, the port-scan cap and the per-target throttle.
+// TOOLS_GUARD=enforce refuses. The apps' older per-route limiters keep working in report mode and step
+// aside in enforce mode (guard.enforcing), where the tiered quotas replace them.
+// ═══════════════════════════════════════════════════════════════
+
+const fs = require('fs');
+const path = require('path');
+const limits = require('./limits');
+const ip = require('./ip');
+const tokens = require('./tokens');
+const sniff = require('./sniff');
+const ffmpeg = require('./ffmpeg');
+const { createGuardStore, dayOf, DAY_MS } = require('./store');
+const { createQuotas, setHeaders } = require('./quota');
+const { createSemaphore, Busy } = require('./semaphore');
+const { createTargetThrottle, createPortScanCap, normalizeTarget } = require('./targets');
+const { createCallerResolver, jobOwnerResolver } = require('./caller');
+const { NO_CHALLENGE, validChallenge } = require('./challenge');
+
+const TITLES = { 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found', 413: 'Payload Too Large', 415: 'Unsupported Media Type', 422: 'Unprocessable Content', 429: 'Too Many Requests', 503: 'Service Unavailable', 504: 'Gateway Timeout' };
+
+/** RFC 9457 problem+json in the contracts' shape (errors.problem@1), with the legacy { error } field. */
+function sendProblem(req, res, status, code, detail, extra, contracts) {
+ if (contracts && contracts.http) return contracts.http.sendProblem(res, status, code, { detail, extra, ctx: req.ov });
+ const body = { type: `https://openvibe.network/problems/${code}`, title: TITLES[status] || 'Error', status, code };
+ if (detail) body.detail = detail;
+ if (req.ov) { body.request_id = req.ov.requestId; body.trace_id = req.ov.traceId; }
+ body.error = detail || body.title;
+ Object.assign(body, extra);
+ res.statusCode = status;
+ res.setHeader('Content-Type', 'application/problem+json');
+ res.end(JSON.stringify(body));
+ return body;
+}
+
+/** Every uploaded file on a request (multer single, array or fields). */
+function uploadsOf(req) {
+ if (Array.isArray(req.files)) return req.files;
+ if (req.files && typeof req.files === 'object') return Object.values(req.files).flat();
+ return req.file ? [req.file] : [];
+}
+
+/** Delete the files multer wrote to disk for a request that is refused. */
+function discardUploads(req) {
+ for (const f of uploadsOf(req)) if (f.path) fs.unlink(f.path, () => {});
+}
+
+/**
+ * @param {object} o
+ * @param {string} o.app 'gateway' | 'img' | … (logs, metrics)
+ * @param {string} [o.dataDir] guard.db goes here (with o.Database)
+ * @param {Function} [o.Database] require('better-sqlite3'); without it everything stays in memory
+ * @param {object} [o.contracts] openvibe-contracts, when the app has it
+ * @param {object[]} [o.specs] the app's descriptor specs (server/descriptors.js SPECS)
+ * @param {object} [o.keys] { get(), ensure() } — or networkUrl/networkInternalUrl/publicKeyFiles
+ * @param {string} o.issuer the Network issuer URL
+ * @param {object} [o.challenge] a challenge provider (./challenge.js); default: none
+ * @param {object} [o.env] [o.log] [o.now] [o.secureCookie]
+ */
+function createGuard(o = {}) {
+ const env = o.env || process.env;
+ const log = o.log || console;
+ const now = o.now || Date.now;
+ const appName = o.app || 'tools';
+ const mode = String(env.TOOLS_GUARD || 'report').trim().toLowerCase() === 'enforce' ? 'enforce' : 'report';
+ const enforcing = mode === 'enforce';
+ const bounds = limits.bounds(env);
+ const table = limits.quotas(env, log);
+ const store = createGuardStore({ Database: o.Database, dataDir: o.dataDir, now, log });
+
+ let saltDay = null, saltValue = null;
+ const salt = () => {
+ const d = dayOf(now());
+ if (d !== saltDay) { saltValue = store.salt(d); saltDay = d; }
+ return saltValue;
+ };
+
+ const keys = o.keys || tokens.createKeySource({ networkUrl: o.networkUrl, networkInternalUrl: o.networkInternalUrl, files: o.publicKeyFiles || [], log });
+ const issuer = o.issuer;
+ const audience = env.OV_TOOLS_AUDIENCE || tokens.AUDIENCE;
+ const callers = createCallerResolver({ getPublicKey: () => keys.get(), issuer, audience, contracts: o.contracts, salt, secureCookie: o.secureCookie });
+ const quotas = createQuotas({ store, quotas: () => table, now });
+ const targets = createTargetThrottle({ now });
+ const ports = createPortScanCap({ ...bounds.ports, now });
+ const sync = createSemaphore({ max: bounds.sync.concurrency, queue: bounds.sync.queue, waitMs: bounds.sync.waitMs });
+ const challenge = validChallenge(o.challenge) ? o.challenge : NO_CHALLENGE;
+
+ // ── Descriptors ──────────────────────────────────────────
+ const specs = new Map((o.specs || []).map(s => [s.id, s]));
+ const tool = (id) => (id && specs.get(id)) || null;
+ /** The descriptor of a job run: the host's own tool when it runs this operation, else the tool named after it, else the first. */
+ function toolForJob(type, operation, preferId) {
+ const pref = tool(preferId);
+ if (pref && pref.job && pref.job.type === type && (!operation || pref.job.operation === operation)) return pref;
+ let first = null;
+ for (const s of specs.values()) {
+ if (!s.job || s.job.type !== type || (operation && s.job.operation !== operation)) continue;
+ if (s.id === operation) return s;
+ if (!first) first = s;
+ }
+ return first;
+ }
+ const label = (id) => (id && specs.has(id) ? id : id ? 'other' : '-');
+
+ // ── Metrics and the log ──────────────────────────────────
+ let refused = null;
+ function attachMetrics(registry) {
+ if (!registry || refused) return;
+ refused = registry.counter({ name: 'tools_guard_refused_total', help: 'Requests the guard refused, or in report mode would have refused, by reason and tool', labelNames: ['reason', 'tool'], maxSeries: 2000 });
+ registry.gauge({ name: 'tools_guard_enforcing', help: '1 when TOOLS_GUARD=enforce, 0 in report mode (hard limits refuse in both)', collect: () => [{ labels: {}, value: enforcing ? 1 : 0 }] });
+ registry.gauge({
+ name: 'tools_guard_sync', help: 'Synchronous heavy calls running and waiting, and the limit', labelNames: ['kind'],
+ collect: () => { const s = sync.stats(); return [{ labels: { kind: 'running' }, value: s.active }, { labels: { kind: 'waiting' }, value: s.waiting }, { labels: { kind: 'limit' }, value: s.max }]; },
+ });
+ }
+ const said = new Map();
+ function record(req, c, { reason, tool: toolId, enforced }) {
+ try {
+ store.logAbuse({ at: now(), ipHash: c ? c.ipHash : callers.hashIp(req.ip), principal: (c && c.id) || '', tool: toolId || '', reason, enforced });
+ } catch (err) { log.error(`[Guard] ${appName}: abuse log write failed: ${err.message}`); }
+ if (refused) refused.inc({ reason, tool: label(toolId) });
+ const k = `${reason}|${toolId || ''}|${enforced}`;
+ const t = now();
+ if (!said.has(k) || t - said.get(k) > 60_000) {
+ said.set(k, t);
+ if (said.size > 1000) said.clear();
+ log.warn(`[Guard] ${appName}: ${enforced ? 'refused' : 'would refuse (report mode)'} ${reason}${toolId ? ` on ${toolId}` : ''} (${c ? c.tier : 'anonymous'})`);
+ }
+ }
+
+ // ── Callers ──────────────────────────────────────────────
+ /** The request's caller, resolved once (before any cookie is minted during the request). */
+ function caller(req, res) {
+ if (!req._ovCaller) req._ovCaller = callers.resolve(req, res);
+ return req._ovCaller;
+ }
+
+ /** Attach req.user for a valid Network sign-in (aud openvibe.tools); a principal's token is left to the resolver. */
+ function identify(req, _res, next) {
+ const { token } = tokens.extractToken(req);
+ if (!token || tokens.looksLikePrincipal(tokens.peek(token))) return next();
+ const verify = () => {
+ const claims = tokens.verifyUserToken(token, { publicKey: keys.get(), issuer, audience, now: now() });
+ if (claims) { req.user = claims; req.token = token; }
+ };
+ if (keys.get() || typeof keys.ensure !== 'function') { verify(); return next(); }
+ Promise.resolve(keys.ensure()).then(verify, () => {}).finally(() => next());
+ }
+
+ // ── Refusals ─────────────────────────────────────────────
+ /**
+ * Refuse (enforce mode, or a hard limit) or only record it (report mode).
+ * → true when the answer was sent, false when the request may go on.
+ */
+ function refuse(req, res, r) {
+ const c = caller(req, res);
+ const enforced = !!r.hard || enforcing;
+ record(req, c, { reason: r.reason, tool: r.tool, enforced });
+ if (!enforced) return false;
+ if (r.discard !== false) discardUploads(req);
+ if (r.retryAfter) res.setHeader('Retry-After', String(Math.max(1, Math.ceil(r.retryAfter))));
+ sendProblem(req, res, r.status, r.code, r.detail, r.extra, o.contracts);
+ return true;
+ }
+
+ // ── Quotas ───────────────────────────────────────────────
+ async function charge(req, res, { quotaClass, cost, tool: toolId }) {
+ const c = caller(req, res);
+ const r = quotas.check(c, { quotaClass, cost }, { report: !enforcing });
+ // In report mode an older limiter that ran on this request (req.rateLimit) is the one in force:
+ // its RateLimit-* headers stay. Otherwise the guard's describe the allowance closest to running out.
+ if (enforcing || !req.rateLimit) setHeaders(res, r);
+ if (r.ok) return true;
+ if (enforcing && (c.tier === 'anonymous' || c.tier === 'session') && challenge.required(req, c, { reason: 'quota', tool: toolId })) {
+ if (await challenge.verify(req, c)) return true;
+ record(req, c, { reason: 'challenge', tool: toolId, enforced: true });
+ discardUploads(req);
+ sendProblem(req, res, 403, 'tools.challenge.required', 'Show that you are a person to go on.', { challenge: challenge.name }, o.contracts);
+ return false;
+ }
+ const detail = r.binding === 'day'
+ ? `You have used today's allowance for ${quotaClass.replace(/^tools-/, '')} requests; it renews at midnight UTC.`
+ : 'Too many requests; slow down a little.';
+ return !refuse(req, res, { status: 429, code: 'tools.quota.exceeded', detail, reason: 'quota', tool: toolId, retryAfter: r.retryAfter, extra: { quota_class: quotaClass, scope: r.scope, tier: c.tier, retry_after: r.retryAfter } });
+ }
+
+ /** Middleware: charge a fixed class and cost (tools-api for every /api/ request). */
+ function quota({ quotaClass, cost = 1, skip = null }) {
+ return (req, res, next) => {
+ if (skip && skip(req)) return next();
+ charge(req, res, { quotaClass, cost }).then(ok => { if (ok) next(); }, next);
+ };
+ }
+ // Liveness and readiness probes (the gateway polls every satellite's) are never counted.
+ const PROBES = new Set(['/api/health', '/api/ready']);
+ const isProbe = (req) => req.method === 'GET' && PROBES.has(`${req.baseUrl || ''}${req.path}`);
+
+ /**
+ * Middleware: charge the tool's own class and cost (its descriptor). toolOf(req) → a tool id. Records
+ * what was taken on req._ovCharged, so admit()/admitJob() can settle the difference once the real
+ * tool is known (a multipart body names it only after the upload).
+ */
+ function toolQuota(toolOf) {
+ return (req, res, next) => {
+ const d = tool(typeof toolOf === 'function' ? toolOf(req) : toolOf);
+ if (!d) return next();
+ req._ovCharged = { quotaClass: d.quotaClass, cost: d.cost || 1, tool: d.id };
+ charge(req, res, { quotaClass: d.quotaClass, cost: d.cost || 1, tool: d.id }).then(ok => { if (ok) next(); }, next);
+ };
+ }
+
+ /** Settle a pre-charge against the real tool: charge the difference or give it back. → ok */
+ async function settle(req, res, d) {
+ const was = req._ovCharged;
+ const cost = d.cost || 1;
+ if (!was || was.quotaClass !== d.quotaClass) {
+ req._ovCharged = { quotaClass: d.quotaClass, cost, tool: d.id };
+ return charge(req, res, { quotaClass: d.quotaClass, cost, tool: d.id });
+ }
+ if (cost > was.cost) { req._ovCharged = { ...was, cost }; return charge(req, res, { quotaClass: d.quotaClass, cost: cost - was.cost, tool: d.id }); }
+ if (cost < was.cost) { quotas.refund(caller(req, res), { quotaClass: d.quotaClass, cost: was.cost - cost }); req._ovCharged = { ...was, cost }; }
+ return true;
+ }
+
+ // ── Who may run it ───────────────────────────────────────
+ /** Heavy tools (descriptor auth.anonymous false) need a session that existed before this request, a sign-in or a token. → ok */
+ function sessionRule(req, res, d) {
+ if (!d || !d.auth || d.auth.anonymous !== false) return true;
+ const c = caller(req, res);
+ if (c.kind !== 'anonymous') return true;
+ return !refuse(req, res, { status: 401, code: 'tools.session_required', detail: 'This tool needs a browser session (open its page first), a sign-in or an API token.', reason: 'session', tool: d.id });
+ }
+
+ /** A principal must hold the tool's capability (tools.net.probe for probes). People use the pages. → ok */
+ function capabilityRule(req, res, d) {
+ if (!d || !d.auth) return true;
+ const c = caller(req, res);
+ if (c.kind !== 'principal') return true;
+ if (tokens.hasCapability(c.claims, d.auth.capability, o.contracts)) return true;
+ return !refuse(req, res, { status: 403, code: 'capability.denied', detail: `${d.auth.capability} not granted`, reason: 'capability', tool: d.id });
+ }
+
+ // ── Uploads ──────────────────────────────────────────────
+ /**
+ * The bytes of every file against `accept` (hard limit). Fixes each file's type (and a misleading
+ * extension) to what the bytes are. files: multer files ({ buffer|path, originalname, mimetype })
+ * or job files ({ buffer|path, name, mime }). → ok
+ */
+ function checkFiles(req, res, files, accept, toolId) {
+ if (!Array.isArray(accept) || !accept.length) return true;
+ for (let i = 0; i < files.length; i++) {
+ const f = files[i];
+ let found = null;
+ try { found = sniff.detect(sniff.head(f)); } catch { found = null; }
+ const name = f.originalname != null ? f.originalname : f.name;
+ if (!sniff.accepts(accept, found)) {
+ const what = found ? `a ${found.ext.toUpperCase()} file` : 'not a file type this tool knows';
+ refuse(req, res, { status: 415, code: 'tools.file.unsupported_type', hard: true, reason: 'sniff', tool: toolId, detail: `${files.length > 1 ? `File ${i + 1}` : 'The file'} is ${what}; this tool takes ${sniff.describe(accept)}.`, extra: { detected: found ? found.mime : null } });
+ return false;
+ }
+ if ('mimetype' in f) f.mimetype = found.mime; else f.mime = found.mime;
+ if (!sniff.extFits(name, found)) {
+ const base = String(name || 'file').replace(/\.[A-Za-z0-9]{1,8}$/, '') || 'file';
+ if (f.originalname != null) f.originalname = `${base}.${found.ext}`; else f.name = `${base}.${found.ext}`;
+ if (f.path && !sniff.extFits(f.path, found)) {
+ const to = `${f.path.replace(/\.[A-Za-z0-9]{1,8}$/, '')}.${found.ext}`;
+ try { fs.renameSync(f.path, to); f.path = to; if (f.filename) f.filename = path.basename(to); } catch { /* keep the old name */ }
+ }
+ }
+ }
+ return true;
+ }
+
+ /**
+ * After the upload middleware: the real tool (toolOf(req) once the body is parsed) — its session rule,
+ * the bytes against its files.accept, and the quota difference. → middleware
+ */
+ function admitUpload(toolOf) {
+ return (req, res, next) => {
+ const d = tool(toolOf(req));
+ if (!d) return next();
+ if (!sessionRule(req, res, d)) return;
+ if (!checkFiles(req, res, uploadsOf(req), d.files && d.files.accept, d.id)) return;
+ settle(req, res, d).then(ok => { if (ok) next(); }, next);
+ };
+ }
+
+ /** The job routes' admission hook (apps/_shared/jobs/http.js o.admit): the same checks for a job submit. → ok */
+ async function admitJob(req, res, { type, input, files, toolId }) {
+ const d = tool(toolId) || toolForJob(type, input && input.tool, null);
+ if (!d) return true;
+ if (!sessionRule(req, res, d)) return false;
+ if (!checkFiles(req, res, files, d.files && d.files.accept, d.id)) return false;
+ return settle(req, res, d);
+ }
+
+ // ── Heavy work ───────────────────────────────────────────
+ /** Middleware: hold one of the synchronous heavy slots until the answer is sent. */
+ function heavy(toolOf) {
+ return (req, res, next) => {
+ sync.acquire().then((release) => {
+ let done = false;
+ const free = () => { if (!done) { done = true; release(); } };
+ res.on('finish', free);
+ res.on('close', free);
+ next();
+ }, (err) => {
+ if (!(err instanceof Busy)) return next(err);
+ const toolId = typeof toolOf === 'function' ? toolOf(req) : toolOf;
+ if (refuse(req, res, { status: 503, code: 'tools.busy', detail: 'The server is busy with other files right now; try again in a moment.', reason: `busy.${err.reason === 'queue' ? 'sync' : 'wait'}`, tool: toolId, retryAfter: 10 })) return;
+ next(); // report mode: runs without a slot
+ });
+ };
+ }
+
+ /** The job store is over its queue or disk bound (system.busy()): refuse or record. → ok */
+ function jobsBusy(req, res, busy, toolId) {
+ if (!busy) return true;
+ return !refuse(req, res, { status: 503, code: 'tools.busy', detail: busy.detail, reason: `busy.${busy.reason}`, tool: toolId, retryAfter: busy.retryAfter || 30 });
+ }
+
+ // ── Egress ───────────────────────────────────────────────
+ /** Per-target throttle (hard). → ok */
+ function target(req, res, { tool: toolId, target: t, perMinute }) {
+ const d = tool(toolId);
+ const per = perMinute || (d && d.limits && d.limits.perTargetPerMinute);
+ const r = targets.take(t, per);
+ if (r.ok) return true;
+ refuse(req, res, { status: 429, code: 'tools.quota.exceeded', hard: true, reason: 'target', tool: toolId, retryAfter: r.retryAfter, detail: `${r.target} was checked too often in the last minute; try again in ${r.retryAfter} s.`, extra: { scope: 'target', retry_after: r.retryAfter } });
+ return false;
+ }
+
+ /** The port checker's caps (hard). Browser sessions count by address, so a new cookie is no new allowance. → ok */
+ function portScan(req, res, { tool: toolId, target: t, ports: list }) {
+ const c = caller(req, res);
+ const key = c.tier === 'anonymous' || c.tier === 'session' ? c.ipKey : c.key;
+ const r = ports.take(key, t, list);
+ if (r.ok) return true;
+ if (r.reason === 'ports.per_request') {
+ refuse(req, res, { status: 400, code: 'tools.run.invalid', hard: true, reason: 'ports', tool: toolId, detail: r.detail });
+ } else {
+ refuse(req, res, { status: 429, code: 'tools.quota.exceeded', hard: true, reason: 'ports', tool: toolId, retryAfter: r.retryAfter, detail: `${r.detail}: the port checker is for your own servers, not for scanning.`, extra: { scope: 'ports', retry_after: r.retryAfter } });
+ }
+ return false;
+ }
+
+ // ── Older limiters, and tool errors ──────────────────────
+ /**
+ * One of the apps' older express-rate-limit limiters, keyed by the resolved caller: a signed-in
+ * person or a principal by who they are (their higher number applies — these limiters used to run
+ * before sign-in was read), everyone else by address (IPv6 /64, hashed). Skipped in enforce mode.
+ * @param {Function} rateLimit require('express-rate-limit')
+ */
+ function legacyLimiter(rateLimit, { windowMs, anonymous, signedIn = anonymous, message, headers = true }) {
+ const signed = (c) => c.tier === 'user' || c.tier === 'service';
+ return rateLimit({
+ windowMs,
+ max: (req) => (signed(caller(req)) ? signedIn : anonymous),
+ standardHeaders: headers,
+ legacyHeaders: false,
+ keyGenerator: (req) => { const c = caller(req); return signed(c) || c.tier === 'sandbox' ? c.key : c.ipKey; },
+ skip: () => enforcing,
+ message: { error: message },
+ });
+ }
+ /** The three limiters an upload app had (LEGACY numbers in ./limits.js). */
+ function legacyLimiters(rateLimit, numbers) {
+ const n = numbers || limits.LEGACY[appName];
+ return {
+ apiLimiter: legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: n.api[0], signedIn: n.api[1], message: 'Too many requests. Please try again later.' }),
+ processLimiter: legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: n.process[0], signedIn: n.process[1], message: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' }),
+ burstLimiter: legacyLimiter(rateLimit, { windowMs: 5_000, anonymous: n.burst, signedIn: n.burst, headers: false, message: 'Too many requests in quick succession. Please slow down.' }),
+ };
+ }
+
+ /**
+ * A tool refused its input on a hard limit (err.guardReason: 'pixels', 'ffmpeg.format',
+ * 'ffmpeg.duration', 'timeout'): record it and answer its problem. → true when answered.
+ */
+ function toolRefused(req, res, err, toolId) {
+ if (!err || !err.guardReason || res.headersSent) return false;
+ refuse(req, res, { status: err.status || 422, code: err.code || 'tools.input.invalid', hard: true, reason: err.guardReason, tool: toolId, detail: err.message });
+ return true;
+ }
+
+ // ── Upkeep ───────────────────────────────────────────────
+ function prune() {
+ try {
+ const cutoff = now() - bounds.abuseRetentionDays * DAY_MS;
+ store.pruneAbuse(cutoff);
+ store.pruneDays(dayOf(now() - DAY_MS));
+ } catch (err) { log.error(`[Guard] ${appName}: prune failed: ${err.message}`); }
+ }
+ prune();
+ const pruneTimer = o.pruneIntervalMs === 0 ? null : setInterval(prune, o.pruneIntervalMs || 6 * 60 * 60 * 1000);
+ if (pruneTimer && pruneTimer.unref) pruneTimer.unref();
+
+ const api = {
+ app: appName, mode, enforcing, bounds, table, store, keys, challenge,
+ issuer, audience,
+ identify, caller, resolveCaller: (req, res, opt) => callers.resolve(req, res, opt),
+ ensureSession: (req, res) => callers.ensureSession(req, res),
+ ownerResolver: () => jobOwnerResolver(callers),
+ hashIp: (addr) => callers.hashIp(addr),
+ tool, toolForJob,
+ refuse, record, sendProblem: (req, res, status, code, detail, extra) => sendProblem(req, res, status, code, detail, extra, o.contracts),
+ quota, apiQuota: quota({ quotaClass: 'tools-api', cost: 1, skip: isProbe }), toolQuota, charge, settle, quotas,
+ sessionRule, capabilityRule,
+ checkFiles, admitUpload, admitJob, uploadsOf, discardUploads,
+ heavy, sync, jobsBusy,
+ target, portScan, targets, ports, normalizeTarget,
+ legacyLimiter, legacyLimiters, toolRefused,
+ attachMetrics, prune,
+ close() { if (pruneTimer) clearInterval(pruneTimer); if (keys.stop) keys.stop(); store.close(); },
+ };
+ return api;
+}
+
+module.exports = { createGuard, sendProblem, uploadsOf, discardUploads, TRUST_PROXY: ip.TRUST_PROXY, limits, ip, tokens, sniff, ffmpeg, NO_CHALLENGE };
new file mode 100644
new file mode 100644
@@ -0,0 +1,63 @@
@@ -0,0 +1,63 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// Client addresses (decision 5).
+//
+// The host's nginx applies Cloudflare's real IP (real_ip_header CF-Connecting-IP, Cloudflare ranges
+// only) and every proxied location sets X-Forwarded-For to $remote_addr. So there is exactly one
+// proxy hop, and it is on loopback: `trust proxy` is TRUST_PROXY below in every app, req.ip is the
+// only address anyone reads, and a client's own X-Forwarded-For is never believed. A first-party
+// process that calls a Tools app on loopback on behalf of a visitor (the gateway's proxy to a
+// satellite, food → maps) is that one hop too, and forwards the visitor's address the same way.
+// An app listening beyond loopback (maps binds 0.0.0.0 by default) believes nobody else's header.
+// ═══════════════════════════════════════════════════════════════
+
+const net = require('net');
+
+/** '::ffff:1.2.3.4' → '1.2.3.4'; zone ids dropped; lower case. */
+function normalizeIp(ip) {
+ let s = String(ip || '').trim().toLowerCase();
+ const pct = s.indexOf('%');
+ if (pct >= 0) s = s.slice(0, pct);
+ const mapped = /^::ffff:(\d+\.\d+\.\d+\.\d+)$/.exec(s);
+ if (mapped) s = mapped[1];
+ return s;
+}
+
+function isLoopback(ip) {
+ const s = normalizeIp(ip);
+ return s === '::1' || /^127\./.test(s);
+}
+
+/** An IPv6 address → its eight groups, expanded. */
+function expand6(ip) {
+ let s = ip;
+ const v4 = /(\d+\.\d+\.\d+\.\d+)$/.exec(s);
+ if (v4) {
+ const n = v4[1].split('.').reduce((a, o) => a * 256 + Number(o), 0) >>> 0;
+ s = s.slice(0, -v4[1].length) + `${(n >>> 16).toString(16)}:${(n & 0xffff).toString(16)}`;
+ }
+ const [head, tail] = s.split('::');
+ const left = head ? head.split(':') : [];
+ const right = tail !== undefined && tail ? tail.split(':') : [];
+ const groups = tail !== undefined ? [...left, ...Array(Math.max(0, 8 - left.length - right.length)).fill('0'), ...right] : left;
+ return groups.map(g => g.padStart(4, '0'));
+}
+
+/**
+ * The unit an address is counted as: an IPv4 address itself, an IPv6 address by its /64 (one
+ * customer's network: a host can pick any of its 2^64 addresses). Not an IP → '' (never throws).
+ */
+function ipBucket(ip) {
+ const s = normalizeIp(ip);
+ const v = net.isIP(s);
+ if (v === 4) return s;
+ if (v === 6) return `${expand6(s).slice(0, 4).join(':')}::/64`;
+ return '';
+}
+
+/** Express `trust proxy`: exactly one hop, and only when that hop is on loopback. */
+function TRUST_PROXY(addr, i) {
+ return i === 0 && isLoopback(addr);
+}
+
+module.exports = { normalizeIp, isLoopback, ipBucket, expand6, TRUST_PROXY };
new file mode 100644
new file mode 100644
@@ -0,0 +1,131 @@
@@ -0,0 +1,131 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// The guard's numbers, in one place. Everything per tool comes from its descriptor (tools.tool@1:
+// cost, quotaClass, limits.perTargetPerMinute, limits.maxDurationSec, limits.timeoutMs, files.accept);
+// everything per caller tier and every host-wide bound is here. Pure data, no dependencies.
+//
+// Quotas count weight, not requests: a run costs its descriptor's `cost` (1 = a cheap lookup, 5 an
+// image conversion, 10 an audio effect, 50 a video download). Each quota class × caller tier has
+// perMinute a token bucket refilled at this many units a minute (kept in memory)
+// burst the bucket's size: what may be spent at once after a quiet spell
+// perDay units per UTC day, persisted in the app's guard.db (0 = no day allowance)
+// Tiers: anonymous (keyed by IP, IPv6 by /64) < session (the ov_tools_jobs browser cookie) < user
+// (a Network sign-in) < service (a first-party service or a developer app's production token).
+// sandbox is a developer app's sandbox token: the smallest allowance, keyed by the app.
+//
+// The anonymous numbers start from the per-app limits the satellites had (img 10 conversions a
+// minute, audio 6, docs 10, net/dev 60 lookups, maps 30 calls, yt 5 downloads an hour); signed-in
+// and service tiers get multiples. Normal use of a page stays well inside them.
+//
+// Class names describe the work, never a tier or a price (decision 6): a paid tier later adds a
+// column, not a class. tools-api is the guard's own class for every /api/ request (cost 1).
+//
+// Overrides (one /etc/openvibe/tools.env for every unit):
+// TOOLS_GUARD_LIMITS='{"tools-job":{"anonymous":{"perDay":800}}}' merged into QUOTAS
+// and the TOOLS_* variables named below for the host-wide bounds.
+// ═══════════════════════════════════════════════════════════════
+
+const TIERS = ['anonymous', 'session', 'user', 'service', 'sandbox'];
+
+const q = (perMinute, burst, perDay) => ({ perMinute, burst, perDay });
+
+const QUOTAS = {
+ // Every /api/ request of an app (registry reads, job polls, page context…).
+ 'tools-api': {
+ anonymous: q(120, 60, 0), session: q(180, 90, 0), user: q(480, 240, 0), service: q(2400, 1200, 0), sandbox: q(60, 30, 0),
+ },
+ // Pure transforms (text and dev engines): cheap, run inline.
+ 'tools-run': {
+ anonymous: q(120, 60, 5000), session: q(180, 90, 10000), user: q(480, 240, 50000), service: q(2400, 1200, 500000), sandbox: q(60, 30, 1000),
+ },
+ // Lookups that reach a host the caller chose (DNS, WHOIS, headers, Open Graph…); also throttled per target.
+ 'tools-fetch': {
+ anonymous: q(60, 30, 2000), session: q(90, 45, 4000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 15, 500),
+ },
+ // Network probes (port, ping, latency): traffic to the target itself; also throttled per target.
+ 'tools-probe': {
+ anonymous: q(30, 15, 600), session: q(45, 25, 1000), user: q(120, 60, 5000), service: q(600, 300, 50000), sandbox: q(15, 10, 200),
+ },
+ // Uploads processed by sharp, ffmpeg, pdf-lib, qpdf, poppler (sync /api/process and job submits).
+ 'tools-job': {
+ anonymous: q(60, 30, 1500), session: q(90, 45, 3000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 15, 300),
+ },
+ // The map and food finder pages' own lookups (OpenStreetMap, weather), cost 2 a call.
+ 'tools-map': {
+ anonymous: q(60, 40, 6000), session: q(90, 60, 8000), user: q(240, 120, 20000), service: q(1200, 600, 200000), sandbox: q(30, 20, 500),
+ },
+ // Server downloads (yt: cost 50 each, so anonymous = two at once, one a minute, 30 a day).
+ 'tools-download': {
+ anonymous: q(50, 100, 1500), session: q(75, 150, 2500), user: q(150, 300, 5000), service: q(300, 600, 10000), sandbox: q(25, 50, 100),
+ },
+};
+
+// All browser sessions from one address together may use this many times one session's allowance:
+// dropping the cookie starts a new session, never a new allowance.
+const SESSION_IP_SHARE = 3;
+
+// The apps' older per-route limiters (express-rate-limit, requests per window per address, or per
+// person when signed in). They stay in force in report mode and step aside with TOOLS_GUARD=enforce,
+// where the tiered quotas above take over. [anonymous or session, signed in or service].
+const LEGACY = {
+ img: { api: [60, 120], process: [10, 30], burst: 4 },
+ audio: { api: [60, 120], process: [6, 20], burst: 3 },
+ docs: { api: [60, 120], process: [10, 30], burst: 4 },
+};
+
+function envInt(name, fallback, env = process.env) {
+ const v = parseInt(env[name], 10);
+ return Number.isFinite(v) && v >= 0 ? v : fallback;
+}
+
+/** The host-wide bounds, read from the environment at call time (tests set it per process). */
+function bounds(env = process.env) {
+ return {
+ // Synchronous heavy calls (img, audio, docs /api/process…) running at once per satellite, and
+ // how many may wait for a slot, and for how long, before 503 tools.busy.
+ sync: {
+ concurrency: Math.max(1, envInt('TOOLS_SYNC_CONCURRENCY', 2, env)),
+ queue: envInt('TOOLS_SYNC_QUEUE', 8, env),
+ waitMs: envInt('TOOLS_SYNC_WAIT_MS', 30_000, env),
+ },
+ // Every satellite's job store: queued jobs (all owners) and bytes under <dataDir>/jobs.
+ jobs: {
+ maxQueued: envInt('TOOLS_JOBS_MAX_QUEUED', 200, env),
+ diskBudgetBytes: envInt('TOOLS_DISK_BUDGET_MB', 8192, env) * 1024 * 1024,
+ },
+ // Largest image input decoded by sharp (width × height); OpenVibe.Media uses the same value.
+ maxInputPixels: envInt('TOOLS_MAX_INPUT_PIXELS', 40_000_000, env),
+ // The port checker: ports per request, and per caller in a rolling window.
+ ports: {
+ perRequest: 20,
+ windowMs: 10 * 60_000,
+ perCaller: envInt('TOOLS_PORTS_PER_CALLER', 100, env),
+ targetsPerCaller: envInt('TOOLS_PORT_TARGETS_PER_CALLER', 10, env),
+ },
+ // Webhook request bins (in memory, one hour): per owner, per address, in all.
+ webhook: { perOwner: 5, perIp: 10, total: 500 },
+ // The abuse log (guard.db) keeps rows this long.
+ abuseRetentionDays: 30,
+ };
+}
+
+function merge(base, over) {
+ const out = JSON.parse(JSON.stringify(base));
+ for (const [cls, tiers] of Object.entries(over || {})) {
+ if (!out[cls]) out[cls] = {};
+ for (const [tier, v] of Object.entries(tiers || {})) out[cls][tier] = { ...(out[cls][tier] || q(0, 0, 0)), ...v };
+ }
+ return out;
+}
+
+/** QUOTAS with TOOLS_GUARD_LIMITS merged in (a bad value is reported and ignored). */
+function quotas(env = process.env, log = console) {
+ const raw = env.TOOLS_GUARD_LIMITS;
+ if (!raw) return QUOTAS;
+ try { return merge(QUOTAS, JSON.parse(raw)); } catch (err) {
+ log.warn(`[Guard] TOOLS_GUARD_LIMITS is not valid JSON (${err.message}); using the defaults`);
+ return QUOTAS;
+ }
+}
+
+module.exports = { TIERS, QUOTAS, SESSION_IP_SHARE, LEGACY, bounds, quotas, merge };
new file mode 100644
new file mode 100644
@@ -0,0 +1,143 @@
@@ -0,0 +1,143 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// Tiered quotas: a token bucket per (key, class) in memory, and a UTC-day allowance per (key, class)
+// in guard.db. A run takes its descriptor's `cost` from both, all or nothing.
+//
+// Who is charged (keys come from the caller resolver, addresses already hashed):
+// anonymous its address (IPv6: its /64) at the anonymous tier
+// session the session at the session tier, AND every session from that address together at
+// SESSION_IP_SHARE × the session tier — so dropping the cookie never resets anything
+// user the person; service / sandbox: the principal
+//
+// check() → { ok, limit, remaining, reset, retryAfter, scope, binding }: `limit`/`remaining`/`reset`
+// describe whichever allowance is closest to running out (the RateLimit-* headers), `retryAfter` the
+// seconds until this run would fit. With { report: true } a refused run is still counted (the work
+// happens), which keeps report mode's numbers honest.
+// ═══════════════════════════════════════════════════════════════
+
+const { SESSION_IP_SHARE } = require('./limits');
+const { dayOf, DAY_MS } = require('./store');
+
+/**
+ * @param {object} o
+ * @param {object} o.store guard store (dayUsed, dayAdd)
+ * @param {() => object} o.quotas the QUOTAS table (limits.quotas())
+ * @param {() => number} [o.now]
+ * @param {number} [o.maxBuckets]
+ */
+function createQuotas(o) {
+ const now = o.now || Date.now;
+ const buckets = new Map(); // `${cls}|${key}` → { tokens, at }
+ const maxBuckets = o.maxBuckets || 200_000;
+ let lastSweep = now();
+
+ function sweep(t) {
+ // Buckets that are full again carry no information: drop them.
+ if (t - lastSweep < 60_000 && buckets.size < maxBuckets) return;
+ lastSweep = t;
+ for (const [k, b] of buckets) if (t - b.at > 10 * 60_000) buckets.delete(k);
+ }
+
+ /** A bucket's tokens right now (refilled since it was last touched). */
+ function level(id, lim, t) {
+ const b = buckets.get(id);
+ if (!b) return lim.burst;
+ const rate = lim.perMinute / 60_000;
+ return Math.min(lim.burst, b.tokens + (t - b.at) * rate);
+ }
+
+ function chargesFor(caller, cls, table) {
+ const tiers = table[cls];
+ if (!tiers) return [];
+ const tier = tiers[caller.tier] || tiers.anonymous;
+ if (caller.tier === 'anonymous') return [{ key: caller.ipKey, lim: tier, scope: 'address' }];
+ if (caller.tier === 'session') {
+ const share = { perMinute: tier.perMinute * SESSION_IP_SHARE, burst: tier.burst * SESSION_IP_SHARE, perDay: tier.perDay * SESSION_IP_SHARE };
+ return [{ key: caller.key, lim: tier, scope: 'session' }, { key: `sessions@${caller.ipKey}`, lim: share, scope: 'address' }];
+ }
+ return [{ key: caller.key, lim: tier, scope: caller.tier }];
+ }
+
+ /**
+ * @param {object} caller { tier, key, ipKey }
+ * @param {object} q { quotaClass, cost }
+ * @param {object} [opt] { report: charge even when refused }
+ */
+ function check(caller, { quotaClass, cost = 1 }, opt = {}) {
+ const t = now();
+ sweep(t);
+ const table = o.quotas();
+ const charges = chargesFor(caller, quotaClass, table);
+ if (!charges.length) return { ok: true, limit: null, remaining: null, reset: null, retryAfter: 0 };
+ const day = dayOf(t);
+ const nextDay = Math.ceil((Math.floor(t / DAY_MS) + 1) * DAY_MS / 1000) - Math.floor(t / 1000);
+ let ok = true, retryAfter = 0, binding = null, scope = null;
+ const views = [];
+ for (const c of charges) {
+ const { lim } = c;
+ if (!(lim.burst > 0) || !(lim.perMinute > 0)) continue;
+ const need = Math.min(cost, lim.burst); // one run never costs more than a full bucket
+ const tokens = level(`${quotaClass}|${c.key}`, lim, t);
+ const rate = lim.perMinute / 60_000;
+ const minuteOk = tokens >= need;
+ const used = lim.perDay > 0 ? o.store.dayUsed(day, c.key, quotaClass) : 0;
+ const dayOk = !(lim.perDay > 0) || used + cost <= lim.perDay;
+ const after = Math.max(0, tokens - need);
+ views.push({
+ c, need, tokens, rate, dayUsed: used,
+ minute: { limit: lim.burst, remaining: Math.floor(after), reset: Math.ceil((lim.burst - after) / rate / 1000) },
+ day: lim.perDay > 0 ? { limit: lim.perDay, remaining: Math.max(0, Math.floor(lim.perDay - used - cost)), reset: nextDay } : null,
+ });
+ if (!minuteOk) {
+ ok = false;
+ const wait = Math.ceil((need - tokens) / rate / 1000);
+ if (wait > retryAfter) { retryAfter = wait; binding = 'minute'; scope = c.scope; }
+ }
+ if (!dayOk) {
+ ok = false;
+ if (nextDay > retryAfter) { retryAfter = nextDay; binding = 'day'; scope = c.scope; }
+ }
+ }
+ if (ok || opt.report) {
+ for (const v of views) {
+ buckets.set(`${quotaClass}|${v.c.key}`, { tokens: Math.max(0, v.tokens - v.need), at: t });
+ if (v.c.lim.perDay > 0) o.store.dayAdd(day, v.c.key, quotaClass, cost);
+ }
+ }
+ // The headers describe the allowance closest to running out, in units of this run's cost.
+ let head = null;
+ for (const v of views) {
+ for (const w of [v.minute, v.day]) {
+ if (!w) continue;
+ const runsLeft = w.remaining / Math.max(1, cost);
+ if (!head || runsLeft < head.runsLeft) head = { ...w, runsLeft };
+ }
+ }
+ return {
+ ok, retryAfter: ok ? 0 : Math.max(1, retryAfter), binding, scope,
+ limit: head ? head.limit : null, remaining: head ? (ok ? head.remaining : 0) : null, reset: head ? head.reset : null,
+ };
+ }
+
+ /** Give tokens back (a run that turned out to cost less than was taken up front). Day units stay counted. */
+ function refund(caller, { quotaClass, cost }) {
+ if (!(cost > 0)) return;
+ const t = now();
+ for (const c of chargesFor(caller, quotaClass, o.quotas())) {
+ const id = `${quotaClass}|${c.key}`;
+ if (buckets.has(id)) buckets.set(id, { tokens: Math.min(c.lim.burst, level(id, c.lim, t) + cost), at: t });
+ }
+ }
+
+ return { check, refund, size: () => buckets.size };
+}
+
+/** RateLimit-* headers (IETF draft) from a check() answer. */
+function setHeaders(res, r) {
+ if (!r || r.limit == null || res.headersSent) return;
+ res.setHeader('RateLimit-Limit', String(r.limit));
+ res.setHeader('RateLimit-Remaining', String(Math.max(0, r.remaining)));
+ res.setHeader('RateLimit-Reset', String(Math.max(0, r.reset)));
+}
+
+module.exports = { createQuotas, setHeaders };
new file mode 100644
new file mode 100644
@@ -0,0 +1,61 @@
@@ -0,0 +1,61 @@
+'use strict';
+// A counting semaphore with a bounded wait list: at most `max` holders, at most `queue` waiting, each
+// for at most `waitMs`. acquire() → a release function, or a Busy error ('queue' | 'timeout').
+// createPacer spaces calls to an upstream that asks for it (below).
+
+class Busy extends Error {
+ constructor(reason) { super(reason === 'queue' ? 'Too many requests are waiting' : 'Waited too long for a free slot'); this.reason = reason; }
+}
+
+function createSemaphore({ max = 2, queue = 8, waitMs = 30_000 } = {}) {
+ let active = 0;
+ const waiting = [];
+
+ function release() {
+ const next = waiting.shift();
+ if (next) { clearTimeout(next.timer); next.resolve(once()); } else active = Math.max(0, active - 1);
+ }
+ function once() {
+ let done = false;
+ return () => { if (!done) { done = true; release(); } };
+ }
+
+ function acquire() {
+ if (active < max) { active++; return Promise.resolve(once()); }
+ if (waiting.length >= queue) return Promise.reject(new Busy('queue'));
+ return new Promise((resolve, reject) => {
+ const w = { resolve, timer: null };
+ w.timer = setTimeout(() => {
+ const i = waiting.indexOf(w);
+ if (i >= 0) waiting.splice(i, 1);
+ reject(new Busy('timeout'));
+ }, waitMs);
+ waiting.push(w);
+ });
+ }
+
+ return { acquire, stats: () => ({ active, waiting: waiting.length, max, queue }) };
+}
+
+/**
+ * A pacer for an upstream with a usage policy (Nominatim: one request a second): run() starts each
+ * call at least `minIntervalMs` after the previous one, with at most `queue` calls waiting; a call
+ * that would wait longer than `maxWaitMs` is refused with Busy('queue') rather than piling up.
+ */
+function createPacer({ minIntervalMs = 1000, queue = 20, maxWaitMs = 15_000 } = {}) {
+ let next = 0, waiting = 0;
+ async function run(fn) {
+ const t = Date.now();
+ const at = Math.max(t, next);
+ if (waiting >= queue || at - t > maxWaitMs) throw new Busy('queue');
+ next = at + minIntervalMs;
+ waiting++;
+ try {
+ if (at > t) await new Promise(r => setTimeout(r, at - t));
+ } finally { waiting--; }
+ return fn();
+ }
+ return { run, stats: () => ({ waiting }) };
+}
+
+module.exports = { createSemaphore, createPacer, Busy };
new file mode 100644
new file mode 100644
@@ -0,0 +1,169 @@
@@ -0,0 +1,169 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// What an upload really is, from its first bytes (never the client's Content-Type or file name), and
+// whether a tool's descriptor accepts it (files.accept: media types, `type/*` allowed).
+//
+// detect(buf) → { mime, ext, also: [equivalent media types] } | null. Text formats a demuxer would
+// follow (HLS playlists, ffconcat lists, SDP) are never detected as media, so they never pass.
+// ═══════════════════════════════════════════════════════════════
+
+const fs = require('fs');
+
+const HEAD_BYTES = 4096;
+
+const EQUIV = {
+ 'image/png': [], 'image/gif': [], 'image/webp': [], 'image/avif': [], 'image/tiff': [], 'image/svg+xml': [],
+ 'image/jpeg': ['image/jpg', 'image/pjpeg'],
+ 'image/bmp': ['image/x-bmp', 'image/x-ms-bmp'],
+ 'image/x-icon': ['image/vnd.microsoft.icon', 'image/ico'],
+ 'image/heic': ['image/heif', 'image/heic-sequence', 'image/heif-sequence'],
+ 'image/heif': ['image/heic'],
+ 'application/pdf': ['application/x-pdf'],
+ 'audio/mpeg': ['audio/mp3', 'audio/mpeg3', 'audio/x-mpeg'],
+ 'audio/wav': ['audio/x-wav', 'audio/wave', 'audio/vnd.wave'],
+ 'audio/flac': ['audio/x-flac'],
+ 'audio/ogg': ['application/ogg', 'audio/vorbis', 'video/ogg'],
+ 'audio/opus': ['audio/ogg', 'application/ogg'],
+ 'audio/aac': ['audio/x-aac', 'audio/aacp'],
+ 'audio/mp4': ['audio/x-m4a', 'audio/m4a', 'video/mp4'],
+ 'video/mp4': ['audio/mp4', 'audio/x-m4a'],
+ 'video/quicktime': ['video/mp4'],
+ 'video/3gpp': ['audio/3gpp', 'video/mp4'],
+ 'audio/x-ms-wma': ['video/x-ms-asf', 'audio/x-ms-asf', 'video/x-ms-wmv'],
+ 'audio/aiff': ['audio/x-aiff'],
+ 'audio/ac3': ['audio/x-ac3'],
+ 'audio/amr': [],
+ 'video/webm': ['audio/webm', 'video/x-matroska'],
+ 'video/x-matroska': ['video/webm', 'audio/webm', 'audio/x-matroska'],
+ 'video/x-msvideo': ['video/avi', 'video/msvideo'],
+ 'video/x-flv': [],
+};
+
+const EXT = {
+ 'image/png': 'png', 'image/jpeg': 'jpg', 'image/gif': 'gif', 'image/webp': 'webp', 'image/avif': 'avif', 'image/tiff': 'tiff',
+ 'image/bmp': 'bmp', 'image/x-icon': 'ico', 'image/heic': 'heic', 'image/heif': 'heif', 'image/svg+xml': 'svg',
+ 'application/pdf': 'pdf',
+ 'audio/mpeg': 'mp3', 'audio/wav': 'wav', 'audio/flac': 'flac', 'audio/ogg': 'ogg', 'audio/opus': 'opus', 'audio/aac': 'aac',
+ 'audio/mp4': 'm4a', 'video/mp4': 'mp4', 'video/quicktime': 'mov', 'video/3gpp': '3gp', 'audio/x-ms-wma': 'wma',
+ 'audio/aiff': 'aiff', 'audio/ac3': 'ac3', 'audio/amr': 'amr', 'video/webm': 'webm', 'video/x-matroska': 'mkv',
+ 'video/x-msvideo': 'avi', 'video/x-flv': 'flv',
+};
+
+// File name extensions that plausibly go with each type. An upload whose name says something else
+// (song.m3u8 holding MP3 bytes) is renamed to the canonical extension, because some tools derive the
+// output format from the input's extension and ffmpeg takes extensions as hints.
+const FITTING = {
+ 'image/jpeg': ['jpg', 'jpeg', 'jpe', 'jfif'], 'image/tiff': ['tif', 'tiff'], 'image/bmp': ['bmp', 'dib'], 'image/x-icon': ['ico'],
+ 'image/heic': ['heic', 'heif', 'hif'], 'image/heif': ['heif', 'heic', 'hif'],
+ 'audio/mpeg': ['mp3', 'mpga', 'mp2'], 'audio/wav': ['wav', 'wave'], 'audio/ogg': ['ogg', 'oga', 'ogv', 'opus', 'spx'],
+ 'audio/opus': ['opus', 'ogg', 'oga'], 'audio/aac': ['aac', 'adts'], 'audio/mp4': ['m4a', 'm4b', 'm4p', 'm4r', 'mp4'],
+ 'video/mp4': ['mp4', 'm4a', 'm4v', 'm4b', 'm4r', 'mov'], 'video/quicktime': ['mov', 'qt', 'mp4', 'm4a'], 'video/3gpp': ['3gp', '3g2', '3gpp'],
+ 'audio/x-ms-wma': ['wma', 'wmv', 'asf'], 'audio/aiff': ['aiff', 'aif', 'aifc'], 'video/webm': ['webm', 'weba', 'mkv'],
+ 'video/x-matroska': ['mkv', 'mka', 'webm'],
+};
+
+/** Is `name`'s extension one that fits the detected type? */
+function extFits(name, detected) {
+ const ext = String(name || '').toLowerCase().match(/\.([a-z0-9]{1,8})$/);
+ if (!ext || !detected) return false;
+ return (FITTING[detected.mime] || [detected.ext]).includes(ext[1]);
+}
+
+const HEIC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);
+
+const at = (buf, off, str) => buf.length >= off + str.length && buf.toString('latin1', off, off + str.length) === str;
+const found = (mime) => ({ mime, ext: EXT[mime], also: EQUIV[mime] || [] });
+
+function ftyp(buf) {
+ const size = Math.min(buf.readUInt32BE(0) || 0, buf.length);
+ const brands = [buf.toString('latin1', 8, 12)];
+ for (let o = 16; o + 4 <= size; o += 4) brands.push(buf.toString('latin1', o, o + 4));
+ if (brands.includes('avif') || brands.includes('avis')) return 'image/avif';
+ if (brands.some(b => HEIC_BRANDS.has(b))) return 'image/heic';
+ if (brands.includes('mif1') || brands.includes('msf1')) return 'image/heif';
+ const major = brands[0];
+ if (/^M4[ABP] $/.test(major)) return 'audio/mp4';
+ if (major === 'qt ') return 'video/quicktime';
+ if (/^3g[p2]/.test(major)) return 'video/3gpp';
+ return 'video/mp4';
+}
+
+/** Plausible MPEG audio frame header at `o` (layer I–III, a real bitrate and sample rate). */
+function mpegFrame(buf, o = 0) {
+ if (buf.length < o + 4 || buf[o] !== 0xff || (buf[o + 1] & 0xe0) !== 0xe0) return false;
+ const version = (buf[o + 1] >> 3) & 3, layer = (buf[o + 1] >> 1) & 3, bitrate = buf[o + 2] >> 4, rate = (buf[o + 2] >> 2) & 3;
+ return version !== 1 && layer !== 0 && bitrate !== 0 && bitrate !== 15 && rate !== 3;
+}
+
+function isSvg(buf) {
+ let s = buf.toString('utf8', 0, Math.min(buf.length, HEAD_BYTES)).replace(/^/, '').trimStart();
+ // An XML declaration, doctype and comments may come before the root element.
+ for (let i = 0; i < 20; i++) {
+ const m = /^(<\?xml[\s\S]*?\?>|<!--[\s\S]*?-->|<!DOCTYPE[^>]*>)\s*/i.exec(s);
+ if (!m) break;
+ s = s.slice(m[0].length);
+ }
+ return /^<svg[\s>]/i.test(s);
+}
+
+/** The media type of the bytes, or null. */
+function detect(buf) {
+ if (!Buffer.isBuffer(buf) || buf.length < 4) return null;
+ if (at(buf, 0, '\x89PNG\r\n\x1a\n')) return found('image/png');
+ if (buf[0] === 0xff && buf[1] === 0xd8 && buf[2] === 0xff) return found('image/jpeg');
+ if (at(buf, 0, 'GIF87a') || at(buf, 0, 'GIF89a')) return found('image/gif');
+ if (at(buf, 0, 'RIFF') && at(buf, 8, 'WEBP')) return found('image/webp');
+ if (at(buf, 0, 'RIFF') && at(buf, 8, 'WAVE')) return found('audio/wav');
+ if (at(buf, 0, 'RIFF') && at(buf, 8, 'AVI ')) return found('video/x-msvideo');
+ if (at(buf, 0, 'BM') && buf.length >= 18 && [12, 40, 52, 56, 64, 108, 124].includes(buf.readUInt32LE(14))) return found('image/bmp');
+ if (at(buf, 0, 'II*\0') || at(buf, 0, 'MM\0*') || at(buf, 0, 'II+\0') || at(buf, 0, 'MM\0+')) return found('image/tiff');
+ if (buf.length >= 22 && buf.readUInt16LE(0) === 0 && buf.readUInt16LE(2) === 1 && buf.readUInt16LE(4) > 0 && buf[9] === 0) return found('image/x-icon');
+ if (buf.length >= 12 && at(buf, 4, 'ftyp')) return found(ftyp(buf));
+ if (buf.indexOf('%PDF-', 0, 'latin1') >= 0 && buf.indexOf('%PDF-', 0, 'latin1') < 1024) return found('application/pdf');
+ if (at(buf, 0, 'fLaC')) return found('audio/flac');
+ if (at(buf, 0, 'OggS')) return found(buf.indexOf('OpusHead', 0, 'latin1') > 0 && buf.indexOf('OpusHead', 0, 'latin1') < 100 ? 'audio/opus' : 'audio/ogg');
+ if (at(buf, 0, 'ID3')) return found('audio/mpeg');
+ if (buf[0] === 0xff && (buf[1] & 0xf6) === 0xf0) return found('audio/aac'); // ADTS: sync + layer 00
+ if (mpegFrame(buf, 0)) return found('audio/mpeg');
+ if (at(buf, 0, '\x30\x26\xb2\x75\x8e\x66\xcf\x11')) return found('audio/x-ms-wma');
+ if (at(buf, 0, 'FORM') && (at(buf, 8, 'AIFF') || at(buf, 8, 'AIFC'))) return found('audio/aiff');
+ if (buf[0] === 0x0b && buf[1] === 0x77) return found('audio/ac3');
+ if (at(buf, 0, '#!AMR')) return found('audio/amr');
+ if (at(buf, 0, '\x1a\x45\xdf\xa3')) return found(buf.indexOf('webm', 0, 'latin1') > 0 && buf.indexOf('webm', 0, 'latin1') < 64 ? 'video/webm' : 'video/x-matroska');
+ if (at(buf, 0, 'FLV\x01')) return found('video/x-flv');
+ if (isSvg(buf)) return found('image/svg+xml');
+ return null;
+}
+
+/** Does `accept` (media types, type/* allowed) cover what was detected? */
+function accepts(accept, detected) {
+ if (!detected) return false;
+ const list = Array.isArray(accept) ? accept : [];
+ return [detected.mime, ...detected.also].some(m => list.includes(m) || list.includes(`${m.split('/')[0]}/*`));
+}
+
+/** An accept list for people: 'PNG, JPG, WEBP…' (aliases folded into the type they stand for). */
+function describe(accept) {
+ const out = [];
+ for (const m of Array.isArray(accept) ? accept : []) {
+ if (/octet-stream/.test(m)) continue;
+ const canon = EXT[m] ? m : Object.keys(EQUIV).find(c => EQUIV[c].includes(m) && EXT[c]);
+ const word = (canon ? EXT[canon] : m.split('/')[1].replace(/^(x-|vnd\.)/, '').replace(/\+xml$/, '')).toUpperCase();
+ if (!out.includes(word)) out.push(word);
+ }
+ return out.join(', ');
+}
+
+/** The first HEAD_BYTES of an upload (multer memory `buffer` or disk `path`). */
+function head(file) {
+ if (file.buffer) return file.buffer.subarray(0, HEAD_BYTES);
+ if (!file.path) return Buffer.alloc(0);
+ const fd = fs.openSync(file.path, 'r');
+ try {
+ const b = Buffer.alloc(HEAD_BYTES);
+ const n = fs.readSync(fd, b, 0, HEAD_BYTES, 0);
+ return b.subarray(0, n);
+ } finally { fs.closeSync(fd); }
+}
+
+module.exports = { detect, accepts, head, extFits, describe, EQUIV, EXT, HEAD_BYTES };
new file mode 100644
new file mode 100644
@@ -0,0 +1,152 @@
@@ -0,0 +1,152 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// guard.db — the guard's own small SQLite file in each app's data directory (better-sqlite3 handle
+// class passed in, like the job runtime). Three tables:
+//
+// guard_salt today's random salt (one row; the previous day's is deleted when the day turns). Every
+// address the guard counts or logs is HMAC-SHA256(salt, address) — never the address.
+// Kept on disk so a restart keeps today's counts; gone the next day, so older hashes
+// can no longer be tied to anything (pseudonymous, ADR-021).
+// guard_day quota units used per UTC day, key and quota class (the day allowances survive restarts;
+// rows older than yesterday are pruned).
+// guard_abuse the abuse log: time, hashed address, principal or user id, tool, reason, whether it was
+// refused (enforce) or only reported; repeats of the same row within a minute add to
+// `count`. Kept 30 days.
+//
+// Without a Database class (an app that has none) the same API is kept in memory.
+// ═══════════════════════════════════════════════════════════════
+
+const crypto = require('crypto');
+const fs = require('fs');
+const path = require('path');
+
+const DAY_MS = 24 * 60 * 60 * 1000;
+const dayOf = (ms) => new Date(ms).toISOString().slice(0, 10);
+
+const SCHEMA = `
+CREATE TABLE IF NOT EXISTS guard_salt (
+ day TEXT PRIMARY KEY,
+ salt TEXT NOT NULL
+);
+CREATE TABLE IF NOT EXISTS guard_day (
+ day TEXT NOT NULL,
+ key TEXT NOT NULL,
+ class TEXT NOT NULL,
+ used REAL NOT NULL DEFAULT 0,
+ PRIMARY KEY (day, key, class)
+);
+CREATE TABLE IF NOT EXISTS guard_abuse (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ at INTEGER NOT NULL,
+ minute INTEGER NOT NULL,
+ ip_hash TEXT NOT NULL DEFAULT '',
+ principal TEXT NOT NULL DEFAULT '',
+ tool TEXT NOT NULL DEFAULT '',
+ reason TEXT NOT NULL,
+ enforced INTEGER NOT NULL,
+ count INTEGER NOT NULL DEFAULT 1
+);
+CREATE UNIQUE INDEX IF NOT EXISTS guard_abuse_once ON guard_abuse(minute, ip_hash, principal, tool, reason, enforced);
+CREATE INDEX IF NOT EXISTS guard_abuse_at ON guard_abuse(at);
+`;
+
+/**
+ * @param {object} o
+ * @param {Function} [o.Database] require('better-sqlite3'); omitted → in memory
+ * @param {string} [o.dataDir]
+ * @param {string} [o.file='guard.db']
+ * @param {() => number} [o.now]
+ */
+function createGuardStore(o = {}) {
+ const now = o.now || Date.now;
+ if (!o.Database || !o.dataDir) return memoryStore(now);
+ try { return sqliteStore(o, now); } catch (err) {
+ // An unwritable data directory must not take the app down: quotas and the log stay in memory.
+ (o.log || console).error(`[Guard] ${path.join(o.dataDir, o.file || 'guard.db')} cannot be opened (${err.message}); keeping guard state in memory`);
+ return memoryStore(now);
+ }
+}
+
+function sqliteStore(o, now) {
+ fs.mkdirSync(o.dataDir, { recursive: true });
+ const db = new o.Database(path.join(o.dataDir, o.file || 'guard.db'));
+ db.pragma('journal_mode = WAL');
+ db.pragma('synchronous = NORMAL');
+ db.pragma('busy_timeout = 5000');
+ db.exec(SCHEMA);
+ const st = {
+ saltGet: db.prepare('SELECT salt FROM guard_salt WHERE day = ?'),
+ saltPut: db.prepare('INSERT OR IGNORE INTO guard_salt (day, salt) VALUES (?, ?)'),
+ saltDrop: db.prepare('DELETE FROM guard_salt WHERE day <> ?'),
+ dayGet: db.prepare('SELECT used FROM guard_day WHERE day = ? AND key = ? AND class = ?'),
+ dayAdd: db.prepare('INSERT INTO guard_day (day, key, class, used) VALUES (?, ?, ?, ?) ON CONFLICT(day, key, class) DO UPDATE SET used = used + excluded.used'),
+ dayPrune: db.prepare('DELETE FROM guard_day WHERE day < ?'),
+ abuse: db.prepare(`INSERT INTO guard_abuse (at, minute, ip_hash, principal, tool, reason, enforced) VALUES (?, ?, ?, ?, ?, ?, ?)
+ ON CONFLICT(minute, ip_hash, principal, tool, reason, enforced) DO UPDATE SET count = count + 1, at = excluded.at`),
+ abusePrune: db.prepare('DELETE FROM guard_abuse WHERE at < ?'),
+ abuseAll: db.prepare('SELECT at, ip_hash, principal, tool, reason, enforced, count FROM guard_abuse ORDER BY id'),
+ };
+ const cache = new Map(); // `${day}|${key}|${cls}` → used (today only)
+ let cacheDay = null;
+ const saltTx = db.transaction((day) => {
+ st.saltPut.run(day, crypto.randomBytes(32).toString('hex'));
+ st.saltDrop.run(day);
+ return st.saltGet.get(day).salt;
+ });
+ return {
+ kind: 'sqlite',
+ db,
+ salt(day = dayOf(now())) {
+ const row = st.saltGet.get(day);
+ return row ? row.salt : saltTx(day);
+ },
+ dayUsed(day, key, cls) {
+ if (cacheDay !== day) { cache.clear(); cacheDay = day; }
+ const k = `${day}|${key}|${cls}`;
+ if (!cache.has(k)) { const r = st.dayGet.get(day, key, cls); cache.set(k, r ? r.used : 0); }
+ return cache.get(k);
+ },
+ dayAdd(day, key, cls, n) {
+ if (!(n > 0)) return;
+ const used = this.dayUsed(day, key, cls) + n;
+ cache.set(`${day}|${key}|${cls}`, used);
+ st.dayAdd.run(day, key, cls, n);
+ },
+ pruneDays(keepFromDay) { return st.dayPrune.run(keepFromDay).changes; },
+ logAbuse(r) {
+ const at = r.at || now();
+ st.abuse.run(at, Math.floor(at / 60_000), r.ipHash || '', r.principal || '', r.tool || '', String(r.reason), r.enforced ? 1 : 0);
+ },
+ pruneAbuse(beforeMs) { return st.abusePrune.run(beforeMs).changes; },
+ abuseRows() { return st.abuseAll.all(); },
+ close() { try { db.close(); } catch { /* already closed */ } },
+ };
+}
+
+function memoryStore(now) {
+ let salt = null;
+ const days = new Map();
+ const abuse = [];
+ return {
+ kind: 'memory',
+ db: null,
+ salt(day = dayOf(now())) {
+ if (!salt || salt.day !== day) salt = { day, salt: crypto.randomBytes(32).toString('hex') };
+ return salt.salt;
+ },
+ dayUsed(day, key, cls) { return days.get(`${day}|${key}|${cls}`) || 0; },
+ dayAdd(day, key, cls, n) { if (n > 0) days.set(`${day}|${key}|${cls}`, this.dayUsed(day, key, cls) + n); },
+ pruneDays(keepFromDay) { let n = 0; for (const k of days.keys()) if (k.slice(0, 10) < keepFromDay) { days.delete(k); n++; } return n; },
+ logAbuse(r) {
+ const at = r.at || now();
+ const row = { at, minute: Math.floor(at / 60_000), ip_hash: r.ipHash || '', principal: r.principal || '', tool: r.tool || '', reason: String(r.reason), enforced: r.enforced ? 1 : 0, count: 1 };
+ const same = abuse.find(x => x.minute === row.minute && x.ip_hash === row.ip_hash && x.principal === row.principal && x.tool === row.tool && x.reason === row.reason && x.enforced === row.enforced);
+ if (same) { same.count++; same.at = at; } else { abuse.push(row); if (abuse.length > 10000) abuse.shift(); }
+ },
+ pruneAbuse(beforeMs) { const n = abuse.length; for (let i = abuse.length - 1; i >= 0; i--) if (abuse[i].at < beforeMs) abuse.splice(i, 1); return n - abuse.length; },
+ abuseRows() { return abuse.map(({ minute, ...r }) => r); },
+ close() {},
+ };
+}
+
+module.exports = { createGuardStore, dayOf, DAY_MS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,83 @@
@@ -0,0 +1,83 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// Egress bounds (hard limits: they apply in report mode too).
+//
+// Per-target throttle: one bucket per target host or address, across every caller and every tool.
+// It holds one minute of work; a run of a tool costs 1 / its descriptor's limits.perTargetPerMinute of
+// it, so headers (10 a minute) and SMTP (3 a minute) against one host share one budget, and nobody
+// can point the network tools at a host faster than the strictest of them allows.
+//
+// Port-scan cap: the port checker takes at most `perRequest` ports a request, and one caller at most
+// `perCaller` ports and `targetsPerCaller` distinct targets in a rolling window, so it can check a
+// server's own ports but cannot sweep a range.
+// ═══════════════════════════════════════════════════════════════
+
+const { ipBucket } = require('./ip');
+
+/** 'https://Example.COM:8443/x' → 'example.com'; an IPv6 address → its /64. '' when there is nothing. */
+function normalizeTarget(raw) {
+ let t = String(raw || '').trim().toLowerCase();
+ if (!t) return '';
+ t = t.replace(/^[a-z][a-z0-9+.-]*:\/\//, '');
+ t = t.replace(/^[^@/]*@/, ''); // user:pass@
+ const bracket = /^\[([0-9a-f:.]+)\]/.exec(t);
+ if (bracket) t = bracket[1];
+ else {
+ t = t.replace(/[/?#].*$/, '');
+ if ((t.match(/:/g) || []).length === 1) t = t.replace(/:\d*$/, '');
+ }
+ t = t.replace(/\.+$/, '');
+ return ipBucket(t) || t.slice(0, 253);
+}
+
+function createTargetThrottle({ now = Date.now } = {}) {
+ const buckets = new Map(); // target → { units, at }
+ const FULL = 1; // one minute of work
+
+ function take(target, perMinute) {
+ const key = normalizeTarget(target);
+ if (!key || !(perMinute > 0)) return { ok: true, retryAfter: 0, target: key };
+ const t = now();
+ if (buckets.size > 50_000) for (const [k, b] of buckets) if (t - b.at > 60_000) buckets.delete(k);
+ const b = buckets.get(key);
+ const units = b ? Math.min(FULL, b.units + (t - b.at) / 60_000) : FULL;
+ const cost = 1 / perMinute;
+ if (units + 1e-9 < cost) return { ok: false, retryAfter: Math.max(1, Math.ceil((cost - units) * 60)), target: key };
+ buckets.set(key, { units: units - cost, at: t });
+ return { ok: true, retryAfter: 0, target: key };
+ }
+
+ return { take, size: () => buckets.size };
+}
+
+function createPortScanCap({ perRequest = 20, windowMs = 10 * 60_000, perCaller = 100, targetsPerCaller = 10, now = Date.now } = {}) {
+ const callers = new Map(); // caller key → [{ at, target, ports }]
+
+ /** → { ok, reason?, detail?, retryAfter? }; records the probe when it is allowed. */
+ function take(callerKey, target, ports) {
+ const n = Array.isArray(ports) ? ports.length : Number(ports) || 0;
+ if (n > perRequest) return { ok: false, reason: 'ports.per_request', detail: `At most ${perRequest} ports per check` };
+ const t = now();
+ const key = normalizeTarget(target);
+ const list = (callers.get(callerKey) || []).filter(e => t - e.at < windowMs);
+ const used = list.reduce((s, e) => s + e.ports, 0);
+ const targets = new Set(list.map(e => e.target));
+ const retryAfter = list.length ? Math.max(1, Math.ceil((list[0].at + windowMs - t) / 1000)) : 1;
+ if (used + n > perCaller) {
+ callers.set(callerKey, list);
+ return { ok: false, reason: 'ports.per_caller', detail: `At most ${perCaller} ports every ${Math.round(windowMs / 60_000)} minutes`, retryAfter };
+ }
+ if (!targets.has(key) && targets.size >= targetsPerCaller) {
+ callers.set(callerKey, list);
+ return { ok: false, reason: 'ports.targets', detail: `At most ${targetsPerCaller} different hosts every ${Math.round(windowMs / 60_000)} minutes`, retryAfter };
+ }
+ list.push({ at: t, target: key, ports: n });
+ callers.set(callerKey, list);
+ if (callers.size > 50_000) for (const [k, l] of callers) if (!l.length || t - l[l.length - 1].at > windowMs) callers.delete(k);
+ return { ok: true };
+ }
+
+ return { take, perRequest };
+}
+
+module.exports = { normalizeTarget, createTargetThrottle, createPortScanCap };
new file mode 100644
new file mode 100644
@@ -0,0 +1,171 @@
@@ -0,0 +1,171 @@
+'use strict';
+// ═══════════════════════════════════════════════════════════════
+// Network tokens, verified offline against the Network's RS256 key (no dependencies: node:crypto).
+//
+// What the Network issues (OpenVibe.Network server/auth, checked 2026-09-23):
+// user (browser sign-in, OAuth code or FedCM exchange, and the Network's own session tokens)
+// iss = the Network, aud = [openvibe.live, openvibe.tools, openvibe.games, openvibe.media,
+// openvibe.network] (+ registry hosts on white-label installs), 24 h; claims sub/id (the
+// numeric account id), subject_id (usr_…), username, display_name, role, avatar_url,
+// profile_color. No cap.
+// service / app principal (client_credentials, ADR-014 developer apps)
+// aud = [the one audience asked for], sub svc:<client> | app:<id>, actor_type, cap[], ns[],
+// env (apps: production | sandbox), 15 min.
+// /internal/issue-token (Network-side cross-service features) carries no aud and no subject_id:
+// not for Tools, and refused here.
+// A user token is accepted only with aud openvibe.tools (the copies this replaces checked the
+// issuer alone). A token shaped like a principal (cap[] or an svc:/app:/mod: subject) is never a user.
+// ═══════════════════════════════════════════════════════════════
+
+const crypto = require('crypto');
+const fs = require('fs');
+const path = require('path');
+
+const AUDIENCE = 'openvibe.tools';
+const JWKS_PATH = '/api/.well-known/jwks';
+
+const fromB64url = (s) => Buffer.from(String(s).replace(/-/g, '+').replace(/_/g, '/'), 'base64');
+const json = (part) => { try { const v = JSON.parse(fromB64url(part).toString('utf8')); return v && typeof v === 'object' ? v : null; } catch { return null; } };
+
+/** A JWT's payload, unverified (to route it to the right check). */
+function peek(token) {
+ const parts = typeof token === 'string' ? token.split('.') : [];
+ return parts.length === 3 ? json(parts[1]) : null;
+}
+
+function looksLikePrincipal(claims) {
+ return !!claims && (Array.isArray(claims.cap) || /^(svc|app|mod):/.test(String(claims.sub || '')));
+}
+
+/** RS256 signature check → { header, claims } or null. */
+function verifySignature(token, publicKey) {
+ if (!publicKey) return null;
+ const parts = typeof token === 'string' ? token.split('.') : [];
+ if (parts.length !== 3) return null;
+ const header = json(parts[0]);
+ const claims = json(parts[1]);
+ if (!header || !claims || header.alg !== 'RS256') return null;
+ let good = false;
+ try { good = crypto.verify('RSA-SHA256', Buffer.from(`${parts[0]}.${parts[1]}`), publicKey, fromB64url(parts[2])); } catch { good = false; }
+ return good ? { header, claims } : null;
+}
+
+function audiences(claims) { return Array.isArray(claims.aud) ? claims.aud : claims.aud == null ? [] : [claims.aud]; }
+
+function timely(claims, now, skewSec) {
+ const t = Math.floor(now / 1000);
+ if (typeof claims.exp !== 'number' || claims.exp + skewSec < t) return false;
+ if (typeof claims.nbf === 'number' && claims.nbf - skewSec > t) return false;
+ if (typeof claims.iat === 'number' && claims.iat - skewSec > t) return false;
+ return true;
+}
+
+/**
+ * A signed-in person's token → its claims, or null (bad signature, expired, wrong issuer, not for
+ * openvibe.tools, or a principal's token).
+ */
+function verifyUserToken(token, { publicKey, issuer, audience = AUDIENCE, now = Date.now(), skewSec = 30 } = {}) {
+ const v = verifySignature(token, publicKey);
+ if (!v) return null;
+ const c = v.claims;
+ if (looksLikePrincipal(c)) return null;
+ if (issuer && c.iss !== issuer) return null;
+ if (!audiences(c).includes(audience)) return null;
+ if (!timely(c, now, skewSec)) return null;
+ return c;
+}
+
+/**
+ * A service or app principal's token → { ok: true, claims } | { ok: false, code, reason }.
+ * With openvibe-contracts it is contracts.serviceAuth.verifyServiceToken (claims checked against
+ * identity.service-token-claims@1); apps without contracts get the same checks minus the schema.
+ */
+function verifyServiceToken(token, { publicKey, issuer, audience = AUDIENCE, acceptSandbox = false, contracts = null, now = Date.now() } = {}) {
+ if (contracts && contracts.serviceAuth) return contracts.serviceAuth.verifyServiceToken(token, { publicKey, issuer, audience, acceptSandbox, now });
+ const fail = (code, reason) => ({ ok: false, code, reason });
+ if (typeof token !== 'string' || token.split('.').length !== 3) return fail('token.malformed', 'not a JWT');
+ const v = verifySignature(token, publicKey);
+ if (!v) return fail('token.bad_signature', 'signature does not verify');
+ const c = v.claims;
+ if (!timely(c, now, 30)) return fail('token.expired', 'expired');
+ if (issuer && c.iss !== issuer) return fail('token.wrong_issuer', `issuer ${c.iss}`);
+ if (!audiences(c).includes(audience)) return fail('token.wrong_audience', `not for ${audience}`);
+ if (!/^(svc|app|mod):[A-Za-z0-9_.-]+$/.test(String(c.sub || '')) || !Array.isArray(c.cap) || !c.cap.every(x => typeof x === 'string')) return fail('token.invalid_claims', 'sub or cap');
+ if (c.env === 'sandbox' && !acceptSandbox) return fail('token.sandbox_refused', 'sandbox tokens are not accepted here');
+ return { ok: true, claims: c };
+}
+
+/**
+ * Why a principal may not use `cap` → a problem code (capability.denied, or contracts' own, e.g. an
+ * audience or namespace mismatch), or null when it may. contracts' manifest-aware check when available.
+ */
+function capabilityDenied(claims, cap, contracts = null) {
+ if (contracts && contracts.capabilities) {
+ const c = contracts.capabilities.check(claims, cap);
+ if (c.allowed) return null;
+ // A capability the installed contracts do not know yet: an explicit grant (only Network signs) is enough.
+ if (c.code === 'capability.unknown' && contracts.capabilities.grants(claims.cap, cap)) return null;
+ return c.code === 'capability.unknown' ? 'capability.denied' : c.code;
+ }
+ return Array.isArray(claims.cap) && claims.cap.includes(cap) ? null : 'capability.denied';
+}
+const hasCapability = (claims, cap, contracts = null) => !capabilityDenied(claims, cap, contracts);
+
+/** The token a request carries: Authorization: Bearer, else the shared ov_token cookie (or the older token cookie). */
+function extractToken(req) {
+ const h = String((req.headers && req.headers.authorization) || '');
+ if (h.startsWith('Bearer ')) return { token: h.slice(7).trim(), from: 'header' };
+ const c = req.cookies || {};
+ if (c.ov_token) return { token: c.ov_token, from: 'cookie' };
+ if (c.token) return { token: c.token, from: 'cookie' };
+ return { token: null, from: null };
+}
+
+/**
+ * The Network's public key: a PEM file (OV_NETWORK_PUBLIC_KEY, air-gapped setups) or the Network's
+ * JWKS endpoint (internal URL first), fetched at start and every `retryMs` until it loads. Until
+ * then every caller is anonymous (nothing fails because the Network is down).
+ *
+ * @returns {{ get(): string|null, ensure(): Promise<string|null>, stop(): void }}
+ */
+function createKeySource({ networkUrl, networkInternalUrl, files = [], fetchImpl = (...a) => globalThis.fetch(...a), retryMs = 60_000, timeoutMs = 5000, log = console, start = true } = {}) {
+ let key = null, timer = null, inflight = null, lastTry = 0;
+ for (const p of files.filter(Boolean)) {
+ try {
+ const resolved = path.resolve(p);
+ if (fs.existsSync(resolved)) { key = fs.readFileSync(resolved, 'utf8'); log.log(`[Auth] Network public key from ${resolved}`); break; }
+ } catch { /* next */ }
+ }
+ async function fetchKey() {
+ for (const base of [networkInternalUrl, networkUrl].filter(Boolean)) {
+ try {
+ const res = await fetchImpl(`${String(base).replace(/\/+$/, '')}${JWKS_PATH}`, { signal: AbortSignal.timeout(timeoutMs) });
+ if (!res.ok) continue;
+ const body = await res.json();
+ if (body && body.public_key) { key = body.public_key; log.log(`[Auth] Network public key from ${base}${JWKS_PATH}`); return key; }
+ } catch { /* next base */ }
+ }
+ return null;
+ }
+ // Requests carrying a token may ask for the key while it is missing; the Network is asked at most
+ // every 30 s for them (the retry timer below keeps trying on its own).
+ function ensure() {
+ if (key) return Promise.resolve(key);
+ if (inflight) return inflight;
+ if (Date.now() - lastTry < 30_000) return Promise.resolve(null);
+ lastTry = Date.now();
+ inflight = fetchKey().finally(() => { inflight = null; });
+ return inflight;
+ }
+ if (!key && start) {
+ ensure().then((k) => {
+ if (k) return;
+ log.warn(`[Auth] OpenVibe Network unreachable — retrying every ${retryMs / 1000}s (everyone is anonymous until the key loads)`);
+ timer = setInterval(async () => { if (await ensure()) { clearInterval(timer); timer = null; } }, retryMs);
+ if (timer.unref) timer.unref();
+ });
+ }
+ return { get: () => key, ensure, stop() { if (timer) clearInterval(timer); timer = null; } };
+}
+
+module.exports = { AUDIENCE, peek, looksLikePrincipal, verifySignature, verifyUserToken, verifyServiceToken, hasCapability, capabilityDenied, extractToken, createKeySource };
@@ -28,16 +28,13 @@
@@ -28,16 +28,13 @@
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
-const crypto = require('crypto');
const { Readable } = require('stream');
const { Readable } = require('stream');
+const { createCallerResolver, jobOwnerResolver, JOB_CAPS: CAPS } = require('../guard/caller');
+
const ID_RE = /^job_[0-9A-HJKMNP-TV-Z]{26}$/;
const ID_RE = /^job_[0-9A-HJKMNP-TV-Z]{26}$/;
-const SESSION_RE = /^[A-Za-z0-9_-]{32,64}$/;
const TERMINAL = new Set(['succeeded', 'failed', 'cancelled']);
const TERMINAL = new Set(['succeeded', 'failed', 'cancelled']);
const CLIENT_JS = path.join(__dirname, 'client.js');
const CLIENT_JS = path.join(__dirname, 'client.js');
-const CAPS = { create: 'tools.job.create', read: 'tools.job.read', cancel: 'tools.job.cancel' };
-
-const b64json = (part) => { try { return JSON.parse(Buffer.from(String(part).replace(/-/g, '+').replace(/_/g, '/'), 'base64').toString('utf8')); } catch { return null; } };
function contentDisposition(kind, name) {
function contentDisposition(kind, name) {
const ascii = String(name || 'result').replace(/[^\x20-\x7e]/g, '_').replace(/["\\]/g, '_');
const ascii = String(name || 'result').replace(/[^\x20-\x7e]/g, '_').replace(/["\\]/g, '_');
@@ -45,7 +42,9 @@ function contentDisposition(kind, name) {
@@ -45,7 +42,9 @@ function contentDisposition(kind, name) {
}
}
/**
/**
- * Who is asking. → { owner, kind: 'principal'|'user'|'session', claims? } | { error: [status, code, detail] } | null
+ * Who is asking. → { owner, kind: 'principal'|'user'|'session', claims?, env } | { error: [status, code, detail] } | null
+ * The guard's one caller resolver (apps/_shared/guard/caller.js); an app with a guard passes
+ * guard.ownerResolver() instead, which is the same resolver with the guard's daily address salt.
*
*
* @param {object} o
* @param {object} o
* @param {object} o.contracts
* @param {object} o.contracts
@@ -56,54 +55,7 @@ function contentDisposition(kind, name) {
@@ -56,54 +55,7 @@ function contentDisposition(kind, name) {
* @param {boolean} [o.secureCookie]
* @param {boolean} [o.secureCookie]
*/
*/
function createOwnerResolver(o) {
function createOwnerResolver(o) {
- const audience = o.audience || 'openvibe.tools';
- const cookieName = o.cookieName || 'ov_tools_jobs';
- const { contracts } = o;
-
- function principalCapability(claims, cap) {
- const c = contracts.capabilities.check(claims, cap);
- if (c.allowed) return null;
- // The tools.job.* manifests are proposed for the next contracts release; until the installed
- // version knows them, an explicit grant in the token (which only Network can sign) is enough.
- if (c.code === 'capability.unknown' && contracts.capabilities.grants(claims.cap, cap)) return null;
- return [403, c.code === 'capability.unknown' ? 'capability.denied' : c.code, `${cap} not granted`];
- }
-
- return function resolveOwner(req, res, { create = false, action = 'read' } = {}) {
- const auth = String(req.headers.authorization || '');
- if (auth.startsWith('Bearer ')) {
- const token = auth.slice(7).trim();
- const parts = token.split('.');
- const claims = parts.length === 3 ? b64json(parts[1]) : null;
- const looksLikePrincipal = claims && (Array.isArray(claims.cap) || /^(svc|app|mod):/.test(String(claims.sub || '')));
- if (looksLikePrincipal) {
- // Job routes are the only Tools routes that take sandbox tokens (developer apps, ADR-014), and
- // only from apps: a sandbox token that is not an app's is refused.
- const r = contracts.serviceAuth.verifyServiceToken(token, { publicKey: o.getPublicKey(), issuer: o.issuer, audience, acceptSandbox: true });
- if (r.ok && r.claims.env === 'sandbox' && !(r.claims.actor_type === 'app' && /^app:/.test(String(r.claims.sub || '')))) {
- return { error: [401, 'token.sandbox_refused', 'sandbox tokens are accepted only from developer apps'] };
- }
- if (!r.ok) return { error: [401, r.code, r.reason] };
- const denied = principalCapability(r.claims, CAPS[action] || CAPS.read);
- if (denied) return { error: denied };
- return { owner: r.claims.sub, kind: 'principal', claims: r.claims, env: r.claims.env === 'sandbox' ? 'sandbox' : 'production' };
- }
- }
- const sid = req.user && req.user.subject_id;
- if (sid && contracts.ids.isSubjectId('user', sid)) return { owner: `user:${sid}`, kind: 'user' };
-
- let session = req.cookies && req.cookies[cookieName];
- if (!SESSION_RE.test(String(session || ''))) {
- if (!create) return null;
- session = crypto.randomBytes(24).toString('base64url');
- res.cookie(cookieName, session, {
- httpOnly: true, sameSite: 'lax', path: '/', maxAge: 7 * 24 * 60 * 60 * 1000,
- secure: o.secureCookie != null ? o.secureCookie : process.env.NODE_ENV === 'production',
- });
- }
- // Only a hash of the cookie is stored, so the database cannot be used to take over a session.
- return { owner: `session:${crypto.createHash('sha256').update(session).digest('hex').slice(0, 40)}`, kind: 'session' };
- };
+ return jobOwnerResolver(createCallerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer, audience: o.audience, cookieName: o.cookieName, secureCookie: o.secureCookie }));
}
}
/**
/**
@@ -118,6 +70,13 @@ function createOwnerResolver(o) {
@@ -118,6 +70,13 @@ function createOwnerResolver(o) {
* @param {function[]} [o.limiters] middleware run before accepting a submit (rate limits)
* @param {function[]} [o.limiters] middleware run before accepting a submit (rate limits)
* @param {(req, who) => number} [o.ttlMs] how long a finished job is kept
* @param {(req, who) => number} [o.ttlMs] how long a finished job is kept
* @param {(req, input) => object} [o.defaults] fills input from the host (e.g. the format a png.* host implies)
* @param {(req, input) => object} [o.defaults] fills input from the host (e.g. the format a png.* host implies)
+ * @param {(req, res, { type, input, files }) => Promise<boolean>} [o.admit] the guard's admission of a
+ * submit once its body is parsed (session rule, the bytes against the tool's accept, the quota
+ * difference); false = it answered
+ * @param {(req, res, busy) => boolean} [o.onBusy] system.busy() said so: true = go on (report mode);
+ * without it a busy store answers 503 tools.busy
+ * @param {(req, res, full) => boolean} [o.onAddressFull] a browser session's address already has its
+ * unfinished jobs (system.addressFull): true = go on (report mode); without it 429
*/
*/
function mountJobRoutes(app, o) {
function mountJobRoutes(app, o) {
const { system, contracts } = o;
const { system, contracts } = o;
@@ -158,7 +117,16 @@ function mountJobRoutes(app, o) {
@@ -158,7 +117,16 @@ function mountJobRoutes(app, o) {
receive(req, res, (err) => { res.json = json; next(err); });
receive(req, res, (err) => { res.json = json; next(err); });
};
};
- app.post('/api/v1/jobs', ...(o.limiters || []), multipartOnly, async (req, res) => {
+ // Over the store's bounds (queued jobs, disk): answer before an upload is accepted.
+ const notBusy = (req, res, next) => {
+ const busy = system.busy ? system.busy() : null;
+ if (!busy) return next();
+ if (o.onBusy) return o.onBusy(req, res, busy) ? next() : undefined;
+ res.set('Retry-After', String(busy.retryAfter || 30));
+ return send(res, 503, 'tools.busy', busy.detail);
+ };
+
+ app.post('/api/v1/jobs', ...(o.limiters || []), notBusy, multipartOnly, async (req, res) => {
noStore(res);
noStore(res);
const w = who(req, res, { create: true, action: 'create' });
const w = who(req, res, { create: true, action: 'create' });
if (!w) return;
if (!w) return;
@@ -173,8 +141,22 @@ function mountJobRoutes(app, o) {
@@ -173,8 +141,22 @@ function mountJobRoutes(app, o) {
const list = Array.isArray(req.files) ? req.files : req.files && typeof req.files === 'object' ? Object.values(req.files).flat() : req.file ? [req.file] : [];
const list = Array.isArray(req.files) ? req.files : req.files && typeof req.files === 'object' ? Object.values(req.files).flat() : req.file ? [req.file] : [];
const files = list.map(f => ({ path: f.path, buffer: f.path ? undefined : f.buffer, name: f.originalname, mime: f.mimetype, size: f.size }));
const files = list.map(f => ({ path: f.path, buffer: f.path ? undefined : f.buffer, name: f.originalname, mime: f.mimetype, size: f.size }));
const key = req.headers['idempotency-key'] != null ? String(req.headers['idempotency-key']) : (b.idempotency_key != null ? String(b.idempotency_key) : null);
const key = req.headers['idempotency-key'] != null ? String(req.headers['idempotency-key']) : (b.idempotency_key != null ? String(b.idempotency_key) : null);
+ // Sessions from one address share a bound on unfinished jobs: a new cookie is no new allowance.
+ const full = w.caller && system.addressFull ? system.addressFull(w.owner, w.caller.ipKey) : null;
+ if (full) {
+ const goOn = o.onAddressFull ? o.onAddressFull(req, res, full) : (send(res, 429, 'tools.job.too_many_active', `At most ${full.limit} unfinished jobs from one address at a time; wait for one to finish`), false);
+ if (!goOn) { for (const f of files) if (f.path) fs.unlink(f.path, () => {}); return; }
+ }
+ if (o.admit) {
+ let ok = false;
+ try { ok = await o.admit(req, res, { type: String(b.type || ''), input, files }); } catch (err) {
+ console.error('[Jobs] admission failed:', err.message);
+ if (!res.headersSent) send(res, 500, 'tools.job.submit_failed', 'The job could not be accepted');
+ }
+ if (!ok) { for (const f of files) if (f.path) fs.unlink(f.path, () => {}); return; }
+ }
try {
try {
- const { job, replayed } = await system.submit({ owner: w.owner, type: String(b.type || ''), input, files, idempotencyKey: key, ttlMs: ttlMs(req, w), env: w.env || 'production' });
+ const { job, replayed } = await system.submit({ owner: w.owner, type: String(b.type || ''), input, files, idempotencyKey: key, ttlMs: ttlMs(req, w), env: w.env || 'production', ipKey: w.caller ? w.caller.ipKey : null });
res.set('Location', `/api/v1/jobs/${job.id}`);
res.set('Location', `/api/v1/jobs/${job.id}`);
if (replayed) res.set('Idempotent-Replayed', 'true');
if (replayed) res.set('Idempotent-Replayed', 'true');
return res.status(replayed ? 200 : 202).json(system.view(job));
return res.status(replayed ? 200 : 202).json(system.view(job));
@@ -201,7 +183,7 @@ function mountJobRoutes(app, o) {
@@ -201,7 +183,7 @@ function mountJobRoutes(app, o) {
});
});
// Retry and references are writes: a principal needs tools.job.create, like a submit.
// Retry and references are writes: a principal needs tools.job.create, like a submit.
- app.post('/api/v1/jobs/:id/retry', ...(o.limiters || []), (req, res) => {
+ app.post('/api/v1/jobs/:id/retry', ...(o.limiters || []), notBusy, (req, res) => {
noStore(res);
noStore(res);
const row = load(req, res, 'create');
const row = load(req, res, 'create');
if (!row) return;
if (!row) return;
@@ -9,6 +9,10 @@
@@ -9,6 +9,10 @@
// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)
// TOOLS_JOBS_CONCURRENCY jobs running at once per satellite (default 2)
// TOOLS_JOBS_CONCURRENCY_<APP> per-satellite override, e.g. TOOLS_JOBS_CONCURRENCY_AUDIO=1
// TOOLS_JOBS_CONCURRENCY_<APP> per-satellite override, e.g. TOOLS_JOBS_CONCURRENCY_AUDIO=1
// TOOLS_JOBS_MAX_ACTIVE unfinished jobs per owner (default 10)
// TOOLS_JOBS_MAX_ACTIVE unfinished jobs per owner (default 10)
+// TOOLS_JOBS_MAX_ACTIVE_PER_ADDRESS unfinished browser-session jobs per address, all sessions (default 3 × the above)
+// TOOLS_JOBS_MAX_QUEUED queued jobs (all owners) per satellite before 503 tools.busy (default 200)
+// TOOLS_DISK_BUDGET_MB MB under <dataDir>/jobs per satellite before 503 tools.busy (default 8192)
+// (both through the guard: refused with TOOLS_GUARD=enforce, recorded in report mode)
// TOOLS_JOB_RESULTS local (default) | media — where result files go
// TOOLS_JOB_RESULTS local (default) | media — where result files go
// TOOLS_MEDIA_NAMESPACE Media namespace for results (default tools)
// TOOLS_MEDIA_NAMESPACE Media namespace for results (default tools)
// OV_MEDIA_INTERNAL_URL Media on this host (default http://127.0.0.1:4100; MEDIA_URL is read as a fallback)
// OV_MEDIA_INTERNAL_URL Media on this host (default http://127.0.0.1:4100; MEDIA_URL is read as a fallback)
@@ -76,6 +80,11 @@ function mediaOrigin(env = process.env) {
@@ -76,6 +80,11 @@ function mediaOrigin(env = process.env) {
* @param {Function} [o.receive] multipart middleware (the app's multer)
* @param {Function} [o.receive] multipart middleware (the app's multer)
* @param {Function[]} [o.limiters]
* @param {Function[]} [o.limiters]
* @param {Function} [o.defaults] (req, input) → input with host defaults
* @param {Function} [o.defaults] (req, input) → input with host defaults
+ * @param {object} [o.guard] apps/_shared/guard: its caller resolver, admission (session rule,
+ * sniffing, quota) and busy handling; without it the job routes
+ * resolve owners themselves and a busy store is always refused
+ * @param {(req, type, input) => string|null} [o.jobTool] the descriptor id a submit runs (the host's
+ * tool, or the one its input.tool names)
*/
*/
function setupJobs(o) {
function setupJobs(o) {
fs.mkdirSync(o.dataDir, { recursive: true });
fs.mkdirSync(o.dataDir, { recursive: true });
@@ -89,12 +98,23 @@ function setupJobs(o) {
@@ -89,12 +98,23 @@ function setupJobs(o) {
db, contracts: o.contracts, service: o.service, dataDir: o.dataDir,
db, contracts: o.contracts, service: o.service, dataDir: o.dataDir,
concurrency: envInt(`TOOLS_JOBS_CONCURRENCY_${o.service.toUpperCase()}`, envInt('TOOLS_JOBS_CONCURRENCY', 2)),
concurrency: envInt(`TOOLS_JOBS_CONCURRENCY_${o.service.toUpperCase()}`, envInt('TOOLS_JOBS_CONCURRENCY', 2)),
maxActivePerOwner: envInt('TOOLS_JOBS_MAX_ACTIVE', 10) || 10,
maxActivePerOwner: envInt('TOOLS_JOBS_MAX_ACTIVE', 10) || 10,
+ maxActivePerAddress: envInt('TOOLS_JOBS_MAX_ACTIVE_PER_ADDRESS', 0),
+ maxQueued: envInt('TOOLS_JOBS_MAX_QUEUED', 200),
+ diskBudgetBytes: envInt('TOOLS_DISK_BUDGET_MB', 8192) * 1024 * 1024,
media: results.media,
media: results.media,
outbox: events.outbox,
outbox: events.outbox,
});
});
o.define(system);
o.define(system);
- const resolveOwner = createOwnerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer });
- mountJobRoutes(o.app, { system, contracts: o.contracts, resolveOwner, receive: o.receive, limiters: o.limiters, defaults: o.defaults });
+ const g = o.guard || null;
+ const resolveOwner = g ? g.ownerResolver() : createOwnerResolver({ contracts: o.contracts, getPublicKey: o.getPublicKey, issuer: o.issuer });
+ mountJobRoutes(o.app, {
+ system, contracts: o.contracts, resolveOwner, receive: o.receive, limiters: o.limiters, defaults: o.defaults,
+ ...(g && {
+ admit: (req, res, job) => g.admitJob(req, res, { ...job, toolId: o.jobTool ? o.jobTool(req, job.type, job.input) : null }),
+ onBusy: (req, res, busy) => g.jobsBusy(req, res, busy, o.jobTool ? o.jobTool(req, null, {}) : null),
+ onAddressFull: (req, res, full) => !g.refuse(req, res, { status: 429, code: 'tools.job.too_many_active', reason: 'jobs.address', tool: o.jobTool ? o.jobTool(req, null, {}) : null, retryAfter: 30, detail: `At most ${full.limit} unfinished jobs from one address at a time; wait for one to finish.`, extra: { scope: 'address' } }),
+ }),
+ });
system.start();
system.start();
let pruneOutbox = null;
let pruneOutbox = null;
if (events.outbox) {
if (events.outbox) {
@@ -65,6 +65,8 @@ const ADDED_COLUMNS = [
@@ -65,6 +65,8 @@ const ADDED_COLUMNS = [
['env', "TEXT NOT NULL DEFAULT 'production' CHECK (env IN ('production','sandbox'))"], // developer-app sandboxes
['env', "TEXT NOT NULL DEFAULT 'production' CHECK (env IN ('production','sandbox'))"], // developer-app sandboxes
['retry_of', 'TEXT'], // the failed job this one retries
['retry_of', 'TEXT'], // the failed job this one retries
['retried_by', 'TEXT'], // set on a failed job once it has been retried: the retry's id
['retried_by', 'TEXT'], // set on a failed job once it has been retried: the retry's id
+ ['ip_key', 'TEXT'], // a browser session's job: the guard's hashed address key (never the address), so
+ // unfinished jobs are also bounded per address (a new cookie is no new allowance)
];
];
const parse = (s, fallback) => { if (s == null) return fallback; try { return JSON.parse(s); } catch { return fallback; } };
const parse = (s, fallback) => { if (s == null) return fallback; try { return JSON.parse(s); } catch { return fallback; } };
@@ -75,10 +77,11 @@ function createStore(db) {
@@ -75,10 +77,11 @@ function createStore(db) {
db.exec(SCHEMA);
db.exec(SCHEMA);
const have = new Set(db.prepare('PRAGMA table_info(tool_jobs)').all().map((c) => c.name));
const have = new Set(db.prepare('PRAGMA table_info(tool_jobs)').all().map((c) => c.name));
for (const [name, decl] of ADDED_COLUMNS) if (!have.has(name)) db.exec(`ALTER TABLE tool_jobs ADD COLUMN ${name} ${decl}`);
for (const [name, decl] of ADDED_COLUMNS) if (!have.has(name)) db.exec(`ALTER TABLE tool_jobs ADD COLUMN ${name} ${decl}`);
+ db.exec("CREATE INDEX IF NOT EXISTS tool_jobs_ip ON tool_jobs(ip_key, state) WHERE ip_key IS NOT NULL");
const q = {
const q = {
- insert: db.prepare(`INSERT INTO tool_jobs (id, type, type_version, owner, state, input_json, files_json, idempotency_key, request_hash, max_attempts, ttl_ms, created_at, updated_at, env, retry_of)
- VALUES (@id, @type, @type_version, @owner, 'queued', @input_json, @files_json, @idempotency_key, @request_hash, @max_attempts, @ttl_ms, @now, @now, @env, @retry_of)`),
+ insert: db.prepare(`INSERT INTO tool_jobs (id, type, type_version, owner, state, input_json, files_json, idempotency_key, request_hash, max_attempts, ttl_ms, created_at, updated_at, env, retry_of, ip_key)
+ VALUES (@id, @type, @type_version, @owner, 'queued', @input_json, @files_json, @idempotency_key, @request_hash, @max_attempts, @ttl_ms, @now, @now, @env, @retry_of, @ip_key)`),
markRetried: db.prepare("UPDATE tool_jobs SET retried_by = @next, updated_at = @now WHERE id = @id AND state = 'failed' AND retried_by IS NULL"),
markRetried: db.prepare("UPDATE tool_jobs SET retried_by = @next, updated_at = @now WHERE id = @id AND state = 'failed' AND retried_by IS NULL"),
get: db.prepare('SELECT * FROM tool_jobs WHERE id = ?'),
get: db.prepare('SELECT * FROM tool_jobs WHERE id = ?'),
byIdem: db.prepare('SELECT * FROM tool_jobs WHERE owner = ? AND idempotency_key = ?'),
byIdem: db.prepare('SELECT * FROM tool_jobs WHERE owner = ? AND idempotency_key = ?'),
@@ -86,6 +89,7 @@ function createStore(db) {
@@ -86,6 +89,7 @@ function createStore(db) {
claim: db.prepare("UPDATE tool_jobs SET state = 'running', attempts = attempts + 1, started_at = @now, updated_at = @now, progress = NULL, progress_message = NULL WHERE id = @id AND state = 'queued'"),
claim: db.prepare("UPDATE tool_jobs SET state = 'running', attempts = attempts + 1, started_at = @now, updated_at = @now, progress = NULL, progress_message = NULL WHERE id = @id AND state = 'queued'"),
running: db.prepare("SELECT * FROM tool_jobs WHERE state = 'running'"),
running: db.prepare("SELECT * FROM tool_jobs WHERE state = 'running'"),
activeForOwner: db.prepare("SELECT COUNT(*) AS n FROM tool_jobs WHERE owner = ? AND state IN ('queued','running')"),
activeForOwner: db.prepare("SELECT COUNT(*) AS n FROM tool_jobs WHERE owner = ? AND state IN ('queued','running')"),
+ activeForIp: db.prepare("SELECT COUNT(*) AS n FROM tool_jobs WHERE ip_key = ? AND state IN ('queued','running')"),
progress: db.prepare('UPDATE tool_jobs SET progress = @progress, progress_message = @message, updated_at = @now WHERE id = @id AND state = \'running\''),
progress: db.prepare('UPDATE tool_jobs SET progress = @progress, progress_message = @message, updated_at = @now WHERE id = @id AND state = \'running\''),
requestCancel: db.prepare("UPDATE tool_jobs SET cancel_requested = 1, updated_at = @now WHERE id = @id AND state IN ('queued','running')"),
requestCancel: db.prepare("UPDATE tool_jobs SET cancel_requested = 1, updated_at = @now WHERE id = @id AND state IN ('queued','running')"),
requeue: db.prepare("UPDATE tool_jobs SET state = 'queued', started_at = NULL, progress = NULL, progress_message = NULL, updated_at = @now WHERE id = @id AND state = 'running'"),
requeue: db.prepare("UPDATE tool_jobs SET state = 'queued', started_at = NULL, progress = NULL, progress_message = NULL, updated_at = @now WHERE id = @id AND state = 'running'"),
@@ -120,7 +124,7 @@ function createStore(db) {
@@ -120,7 +124,7 @@ function createStore(db) {
return {
return {
db,
db,
TERMINAL, STATES,
TERMINAL, STATES,
- insert(row) { q.insert.run({ retry_of: null, ...row }); },
+ insert(row) { q.insert.run({ retry_of: null, ip_key: null, ...row }); },
markRetried: (id, next, now = Date.now()) => q.markRetried.run({ id, next, now }).changes === 1,
markRetried: (id, next, now = Date.now()) => q.markRetried.run({ id, next, now }).changes === 1,
get: (id) => q.get.get(id) || null,
get: (id) => q.get.get(id) || null,
byIdempotencyKey: (owner, key) => q.byIdem.get(owner, key) || null,
byIdempotencyKey: (owner, key) => q.byIdem.get(owner, key) || null,
@@ -128,6 +132,7 @@ function createStore(db) {
@@ -128,6 +132,7 @@ function createStore(db) {
claim: (id, now = Date.now()) => q.claim.run({ id, now }).changes === 1,
claim: (id, now = Date.now()) => q.claim.run({ id, now }).changes === 1,
running: () => q.running.all(),
running: () => q.running.all(),
activeForOwner: (owner) => q.activeForOwner.get(owner).n,
activeForOwner: (owner) => q.activeForOwner.get(owner).n,
+ activeForIp: (ipKey) => (ipKey ? q.activeForIp.get(ipKey).n : 0),
setProgress: (id, progress, message, now = Date.now()) => q.progress.run({ id, progress, message, now }).changes === 1,
setProgress: (id, progress, message, now = Date.now()) => q.progress.run({ id, progress, message, now }).changes === 1,
requestCancel: (id, now = Date.now()) => q.requestCancel.run({ id, now }).changes === 1,
requestCancel: (id, now = Date.now()) => q.requestCancel.run({ id, now }).changes === 1,
requeue: (id, now = Date.now()) => q.requeue.run({ id, now }).changes === 1,
requeue: (id, now = Date.now()) => q.requeue.run({ id, now }).changes === 1,
@@ -10,7 +10,10 @@
@@ -10,7 +10,10 @@
// (onRestart: 'fail'), per job type. Queued rows simply start again.
// (onRestart: 'fail'), per job type. Queued rows simply start again.
//
//
// Bounded: at most `concurrency` jobs run at once in this process, and each owner may have at
// Bounded: at most `concurrency` jobs run at once in this process, and each owner may have at
-// most `maxActivePerOwner` queued + running jobs. Finished jobs expire (ttl chosen at submit),
+// most `maxActivePerOwner` queued + running jobs. busy() says when the whole store is over its
+// bounds — `maxQueued` jobs waiting (all owners), or `diskBudgetBytes` under <dataDir>/jobs — so the
+// HTTP layer can answer 503 tools.busy with Retry-After before it accepts an upload (through the
+// guard: refused in enforce mode, recorded in report mode). Finished jobs expire (ttl chosen at submit),
// and the pruner deletes the row, its events, its files and any Media objects it made — except
// and the pruner deletes the row, its events, its files and any Media objects it made — except
// while something references the result (reference(), e.g. a paste or a project that points at the
// while something references the result (reference(), e.g. a paste or a project that points at the
// Media object), and never while Media keeps an object (a retention hold, or Media unreachable):
// Media object), and never while Media keeps an object (a retention hold, or Media unreachable):
@@ -75,6 +78,10 @@ const scrub = (msg) => String(msg || '').replace(/(?:\/[\w.-]+){2,}/g, '[file]')
@@ -75,6 +78,10 @@ const scrub = (msg) => String(msg || '').replace(/(?:\/[\w.-]+){2,}/g, '[file]')
* @param {string} o.dataDir where job files live (<dataDir>/jobs/<id>/…)
* @param {string} o.dataDir where job files live (<dataDir>/jobs/<id>/…)
* @param {number} [o.concurrency=2]
* @param {number} [o.concurrency=2]
* @param {number} [o.maxActivePerOwner=10]
* @param {number} [o.maxActivePerOwner=10]
+ * @param {number} [o.maxActivePerAddress] unfinished browser-session jobs per address (ipKey), all
+ * sessions together (default 3 × maxActivePerOwner); see addressFull()
+ * @param {number} [o.maxQueued=0] queued jobs (all owners) before busy() says so; 0 = no bound
+ * @param {number} [o.diskBudgetBytes=0] bytes under <dataDir>/jobs before busy() says so; 0 = no bound
* @param {object} [o.media] result store from ./media (null = keep results on local disk)
* @param {object} [o.media] result store from ./media (null = keep results on local disk)
* @param {object} [o.outbox] openvibe-sdk outbox on `db` (./events outboxFromEnv); null = no platform events
* @param {object} [o.outbox] openvibe-sdk outbox on `db` (./events outboxFromEnv); null = no platform events
* @param {number} [o.pruneIntervalMs=300000]
* @param {number} [o.pruneIntervalMs=300000]
@@ -91,6 +98,10 @@ function createJobSystem(o) {
@@ -91,6 +98,10 @@ function createJobSystem(o) {
const throttleMs = o.progressThrottleMs == null ? 250 : o.progressThrottleMs;
const throttleMs = o.progressThrottleMs == null ? 250 : o.progressThrottleMs;
const root = path.resolve(o.dataDir, 'jobs');
const root = path.resolve(o.dataDir, 'jobs');
fs.mkdirSync(root, { recursive: true });
fs.mkdirSync(root, { recursive: true });
+ const maxQueued = o.maxQueued > 0 ? o.maxQueued : 0;
+ const maxActivePerAddress = o.maxActivePerAddress > 0 ? o.maxActivePerAddress : 3 * maxActivePerOwner;
+ const diskBudget = o.diskBudgetBytes > 0 ? o.diskBudgetBytes : 0;
+ const disk = { bytes: 0, at: 0, scanning: null };
const announce = createJobEvents({ contracts, service: o.service, outbox: o.outbox || null, referenceCount: (id) => store.referenceCount(id), log });
const announce = createJobEvents({ contracts, service: o.service, outbox: o.outbox || null, referenceCount: (id) => store.referenceCount(id), log });
const types = new Map();
const types = new Map();
@@ -191,7 +202,7 @@ function createJobSystem(o) {
@@ -191,7 +202,7 @@ function createJobSystem(o) {
// Developer-app sandbox jobs: fewer at a time, kept briefly, results never leave this server.
// Developer-app sandbox jobs: fewer at a time, kept briefly, results never leave this server.
const SANDBOX_MAX_ACTIVE = Math.max(1, o.sandboxMaxActivePerOwner || 2);
const SANDBOX_MAX_ACTIVE = Math.max(1, o.sandboxMaxActivePerOwner || 2);
const SANDBOX_TTL_MS = 30 * 60 * 1000;
const SANDBOX_TTL_MS = 30 * 60 * 1000;
- async function submit({ owner, type, input = {}, files = [], idempotencyKey = null, ttlMs = HOUR, env = 'production' }) {
+ async function submit({ owner, type, input = {}, files = [], idempotencyKey = null, ttlMs = HOUR, env = 'production', ipKey = null }) {
if (env !== 'sandbox') env = 'production';
if (env !== 'sandbox') env = 'production';
if (env === 'sandbox') ttlMs = Math.min(ttlMs, SANDBOX_TTL_MS);
if (env === 'sandbox') ttlMs = Math.min(ttlMs, SANDBOX_TTL_MS);
if (stopped) throw new JobError(503, 'tools.job.unavailable', 'The job system is shutting down');
if (stopped) throw new JobError(503, 'tools.job.unavailable', 'The job system is shutting down');
@@ -240,6 +251,7 @@ function createJobSystem(o) {
@@ -240,6 +251,7 @@ function createJobSystem(o) {
id, type, type_version: def.version, owner, input_json: JSON.stringify(input), files_json: JSON.stringify(stored),
id, type, type_version: def.version, owner, input_json: JSON.stringify(input), files_json: JSON.stringify(stored),
idempotency_key: idempotencyKey == null ? null : String(idempotencyKey), request_hash: requestHash,
idempotency_key: idempotencyKey == null ? null : String(idempotencyKey), request_hash: requestHash,
max_attempts: def.maxAttempts, ttl_ms: ttlMs, now: Date.now(), env,
max_attempts: def.maxAttempts, ttl_ms: ttlMs, now: Date.now(), env,
+ ip_key: String(owner).startsWith('session:') && ipKey ? String(ipKey) : null,
});
});
announce.created(store.get(id));
announce.created(store.get(id));
})();
})();
@@ -253,6 +265,7 @@ function createJobSystem(o) {
@@ -253,6 +265,7 @@ function createJobSystem(o) {
}
}
throw err;
throw err;
}
}
+ disk.bytes += stored.reduce((n, f) => n + (f.size || 0), 0);
emit(id, 'job.queued');
emit(id, 'job.queued');
kick();
kick();
return { job: store.get(id), replayed: false };
return { job: store.get(id), replayed: false };
@@ -461,7 +474,7 @@ function createJobSystem(o) {
@@ -461,7 +474,7 @@ function createJobSystem(o) {
store.insert({
store.insert({
id: next, type: row.type, type_version: def.version, owner: row.owner, input_json: row.input_json, files_json: JSON.stringify(stored),
id: next, type: row.type, type_version: def.version, owner: row.owner, input_json: row.input_json, files_json: JSON.stringify(stored),
idempotency_key: null, request_hash: row.request_hash, max_attempts: def.maxAttempts, ttl_ms: row.ttl_ms, now: Date.now(),
idempotency_key: null, request_hash: row.request_hash, max_attempts: def.maxAttempts, ttl_ms: row.ttl_ms, now: Date.now(),
- env: row.env, retry_of: row.id,
+ env: row.env, retry_of: row.id, ip_key: row.ip_key || null,
});
});
announce.created(store.get(next));
announce.created(store.get(next));
})();
})();
@@ -573,9 +586,60 @@ function createJobSystem(o) {
@@ -573,9 +586,60 @@ function createJobSystem(o) {
return n;
return n;
}
}
+ // ── Host-wide bounds ───────────────────────────────────────
+ /** Bytes under <dataDir>/jobs (inputs and results), walked in the background at most every 30 s. */
+ async function scanDisk() {
+ let total = 0;
+ const walk = async (dir) => {
+ let entries = [];
+ try { entries = await fsp.readdir(dir, { withFileTypes: true }); } catch { return; }
+ for (const e of entries) {
+ const p = path.join(dir, e.name);
+ if (e.isDirectory()) await walk(p);
+ else if (e.isFile()) { try { total += (await fsp.stat(p)).size; } catch { /* gone meanwhile */ } }
+ }
+ };
+ await walk(root);
+ disk.bytes = total;
+ disk.at = Date.now();
+ return total;
+ }
+ function refreshDisk() {
+ if (!disk.scanning) disk.scanning = scanDisk().catch(() => disk.bytes).finally(() => { disk.scanning = null; });
+ return disk.scanning;
+ }
+
+ /**
+ * Is the store over its bounds? → null, or { reason: 'queue'|'disk', detail, retryAfter } for a
+ * 503 tools.busy. Cheap: one COUNT and the last disk walk (a stale walk is refreshed in the background).
+ */
+ function busy() {
+ if (maxQueued) {
+ const queued = store.counts().queued || 0;
+ if (queued >= maxQueued) return { reason: 'queue', detail: `${queued} jobs are already waiting on this server; try again in a minute.`, retryAfter: 60, queued, limit: maxQueued };
+ }
+ if (diskBudget) {
+ if (Date.now() - disk.at > 30_000) refreshDisk();
+ if (disk.bytes >= diskBudget) return { reason: 'disk', detail: 'This server is holding as many files as it can right now; try again in a few minutes.', retryAfter: 300, bytes: disk.bytes, limit: diskBudget };
+ }
+ return null;
+ }
+
+ /**
+ * Browser sessions from one address together have maxActivePerAddress unfinished jobs: → null, or
+ * { active, limit } when a new session job from `ipKey` would pass it (the HTTP layer answers 429
+ * tools.job.too_many_active through the guard). Principals and people are bounded per owner only.
+ */
+ function addressFull(owner, ipKey) {
+ if (!ipKey || !String(owner || '').startsWith('session:')) return null;
+ const active = store.activeForIp(ipKey);
+ return active >= maxActivePerAddress ? { active, limit: maxActivePerAddress } : null;
+ }
+
function start() {
function start() {
if (started) return api;
if (started) return api;
started = true;
started = true;
+ if (diskBudget) refreshDisk();
const r = recover();
const r = recover();
if (r.requeued || r.failed) log.log(`[Jobs] ${o.service}: ${r.requeued} job(s) re-queued, ${r.failed} failed after restart`);
if (r.requeued || r.failed) log.log(`[Jobs] ${o.service}: ${r.requeued} job(s) re-queued, ${r.failed} failed after restart`);
prune().catch(() => {});
prune().catch(() => {});
@@ -610,6 +674,7 @@ function createJobSystem(o) {
@@ -610,6 +674,7 @@ function createJobSystem(o) {
const api = {
const api = {
define, submit, cancel, retry, reference, unreference, subscribe, start, stop, onStop, prune, recover, view, resultFile,
define, submit, cancel, retry, reference, unreference, subscribe, start, stop, onStop, prune, recover, view, resultFile,
+ busy, refreshDisk, addressFull, bounds: { maxQueued, diskBudgetBytes: diskBudget, maxActivePerAddress },
get: (id) => store.get(id),
get: (id) => store.get(id),
eventsAfter: (id, seq) => store.eventsAfter(id, seq),
eventsAfter: (id, seq) => store.eventsAfter(id, seq),
lastSeq: (id) => store.lastSeq(id),
lastSeq: (id) => store.lastSeq(id),
new file mode 100644
new file mode 100644
@@ -0,0 +1,470 @@
@@ -0,0 +1,470 @@
+'use strict';
+// The shared guard (apps/_shared/guard), end to end where it matters:
+// callers and keys (IPv6 /64, one loopback hop, loopback services by their token, sessions, users
+// with the audience check, sandbox apps), token-bucket math and cost weighting from descriptors,
+// session-per-address sharing, report vs enforce, day allowances persisted across a restart,
+// RateLimit-* headers and problem+json, upload sniffing (a renamed file), the per-target throttle,
+// the port-scan cap, the abuse log (no raw address, pruned after 30 days) and the metric, the sync
+// semaphore, the jobs' queue and disk bounds, and the challenge hook.
+const assert = require('assert');
+const crypto = require('crypto');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { dep } = require('./deps');
+
+const express = dep('express');
+const cookieParser = dep('cookie-parser');
+const multer = dep('multer');
+const Database = dep('better-sqlite3');
+const contracts = dep('openvibe-contracts');
+const metrics = dep('openvibe-shared/metrics');
+const guardLib = require('../guard');
+const { createGuard, TRUST_PROXY } = guardLib;
+const { createSemaphore, Busy } = require('../guard/semaphore');
+const { createQuotas } = require('../guard/quota');
+const { createGuardStore } = require('../guard/store');
+const { ipBucket } = require('../guard/ip');
+const sniff = require('../guard/sniff');
+const jobs = require('../jobs');
+
+const ISSUER = 'https://openvibe.network';
+const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });
+const b64 = (o) => Buffer.from(JSON.stringify(o)).toString('base64url');
+function jwt(claims) {
+ const input = `${b64({ alg: 'RS256', typ: 'JWT' })}.${b64(claims)}`;
+ return `${input}.${crypto.sign('RSA-SHA256', Buffer.from(input), privateKey).toString('base64url')}`;
+}
+const now0 = Date.UTC(2026, 8, 23, 12, 0, 0);
+const t = () => Math.floor(Date.now() / 1000);
+const USER = 'usr_01JAAAAAAAAAAAAAAAAAAAAAAA';
+const userToken = (extra = {}) => jwt({ iss: ISSUER, sub: '42', id: 42, subject_id: USER, username: 'alice', aud: ['openvibe.live', 'openvibe.tools', 'openvibe.network'], iat: t(), exp: t() + 3600, ...extra });
+const svcToken = (name, extra = {}) => contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: `svc:${name}`, actor_type: 'service', aud: ['openvibe.tools'], cap: ['tools.tool.run'], ns: [], iat: t(), exp: t() + 900, jti: `tok_${crypto.randomBytes(12).toString('hex')}`, ...extra }, privateKey);
+const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'ov-guard-'));
+const quiet = { log() {}, warn() {}, error() {} };
+
+// Pure-data descriptors of the real apps (nothing loads sharp or ffmpeg).
+const IMG = require('../../img/server/descriptors').SPECS;
+const AUDIO = require('../../audio/server/descriptors').SPECS;
+const NET = require('../../gateway/server/net/descriptors').SPECS;
+
+function makeGuard(o = {}) {
+ return createGuard({
+ app: o.app || 'test', dataDir: o.dataDir, Database: o.dataDir ? Database : undefined, contracts,
+ specs: o.specs || [...IMG, ...AUDIO, ...NET], issuer: ISSUER,
+ keys: { get: () => publicKey, ensure: async () => publicKey },
+ env: { TOOLS_GUARD: o.mode || 'report', ...(o.env || {}) }, log: quiet, now: o.now, pruneIntervalMs: 0,
+ challenge: o.challenge,
+ });
+}
+
+async function serve(app) {
+ const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });
+ return { base: `http://127.0.0.1:${srv.address().port}`, close: () => new Promise(r => srv.close(r)) };
+}
+
+(async () => {
+ // ── IPv6 by /64, IPv4-mapped as IPv4 ──
+ assert.strictEqual(ipBucket('2001:db8:1:2:aaaa::1'), '2001:0db8:0001:0002::/64');
+ assert.strictEqual(ipBucket('2001:db8:1:2:bbbb:cccc:dddd:9'), ipBucket('2001:db8:1:2::77'));
+ assert.notStrictEqual(ipBucket('2001:db8:1:3::1'), ipBucket('2001:db8:1:2::1'));
+ assert.strictEqual(ipBucket('::ffff:203.0.113.5'), '203.0.113.5');
+ assert.strictEqual(TRUST_PROXY('127.0.0.1', 0), true);
+ assert.strictEqual(TRUST_PROXY('::ffff:127.0.0.1', 0), true);
+ assert.strictEqual(TRUST_PROXY('127.0.0.1', 1), false, 'exactly one hop');
+ assert.strictEqual(TRUST_PROXY('203.0.113.9', 0), false, 'a non-loopback peer is never a proxy');
+
+ // ── Callers over HTTP: one loopback hop, tokens, sessions ──
+ {
+ const g = makeGuard();
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ app.use(cookieParser());
+ app.use(g.identify);
+ app.get('/who', (req, res) => { const c = g.caller(req, res); res.json({ kind: c.kind, tier: c.tier, key: c.key, ipKey: c.ipKey, user: req.user ? req.user.subject_id : null, ip: req.ip }); });
+ app.get('/job-who', (req, res) => res.json(g.ownerResolver()(req, res, { create: true, action: 'create' }) || {}));
+ const s = await serve(app);
+ const who = async (headers = {}, p = '/who') => (await fetch(s.base + p, { headers })).json();
+ try {
+ const direct = await who();
+ assert.strictEqual(direct.ip, '127.0.0.1');
+ assert.strictEqual(direct.kind, 'anonymous', 'a loopback caller without a token is anonymous');
+ assert.ok(direct.ipKey.startsWith('ip:') && !direct.ipKey.includes('127.0.0.1'), 'addresses are hashed');
+ const viaNginx = await who({ 'X-Forwarded-For': '6.6.6.6, 198.51.100.7' });
+ assert.strictEqual(viaNginx.ip, '198.51.100.7', 'only the one hop nginx adds is believed');
+ assert.notStrictEqual(viaNginx.ipKey, direct.ipKey, 'a visitor forwarded by the proxy is not the loopback bucket');
+ const v6a = await who({ 'X-Forwarded-For': '2001:db8:1:2::1' });
+ const v6b = await who({ 'X-Forwarded-For': '2001:db8:1:2:ffff::2' });
+ const v6c = await who({ 'X-Forwarded-For': '2001:db8:9:9::1' });
+ assert.strictEqual(v6a.ipKey, v6b.ipKey, 'one /64 is one anonymous caller');
+ assert.notStrictEqual(v6a.ipKey, v6c.ipKey);
+
+ // Two first-party services on loopback are two callers, by their tokens.
+ const live = await who({ Authorization: `Bearer ${svcToken('live')}` });
+ const chat = await who({ Authorization: `Bearer ${svcToken('chat')}` });
+ assert.deepStrictEqual([live.kind, live.tier, live.key], ['principal', 'service', 'svc:live']);
+ assert.strictEqual(chat.key, 'svc:chat');
+ assert.strictEqual(live.ipKey, chat.ipKey, 'same loopback address, different callers');
+ const wrongAud = await who({ Authorization: `Bearer ${svcToken('live', { aud: ['openvibe.media'] })}` });
+ assert.strictEqual(wrongAud.kind, 'anonymous', 'a token for another audience is not believed');
+ const jobWrongAud = await who({ Authorization: `Bearer ${svcToken('live', { aud: ['openvibe.media'] })}` }, '/job-who');
+ assert.deepStrictEqual(jobWrongAud.error, [401, 'token.wrong_audience', 'not for openvibe.tools'], 'on the job routes it is an error');
+
+ // Users: the audience is checked now.
+ const alice = await who({ Cookie: `ov_token=${userToken()}` });
+ assert.deepStrictEqual([alice.kind, alice.tier, alice.key, alice.user], ['user', 'user', `user:${USER}`, USER]);
+ const noAud = await who({ Authorization: `Bearer ${jwt({ iss: ISSUER, sub: '42', username: 'alice', role: 'user', iat: t(), exp: t() + 3600 })}` });
+ assert.strictEqual(noAud.kind, 'anonymous', 'a Network token without aud openvibe.tools (/internal/issue-token) is not a sign-in here');
+ const otherIssuer = await who({ Cookie: `ov_token=${userToken({ iss: 'https://evil.example' })}` });
+ assert.strictEqual(otherIssuer.kind, 'anonymous');
+ const expired = await who({ Cookie: `ov_token=${userToken({ exp: t() - 120 })}` });
+ assert.strictEqual(expired.kind, 'anonymous');
+
+ // Sessions; a sandbox app keeps its small tier.
+ const sid = crypto.randomBytes(24).toString('base64url');
+ const session = await who({ Cookie: `ov_tools_jobs=${sid}` });
+ assert.strictEqual(session.tier, 'session');
+ assert.match(session.key, /^session:[0-9a-f]{40}$/);
+ const sandbox = await who({ Authorization: `Bearer ${contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: 'app:app_01JCCCCCCCCCCCCCCCCCCCCCCC', actor_type: 'app', aud: ['openvibe.tools'], cap: ['tools.job.create'], ns: ['prj_01JCCCCCCCCCCCCCCCCCCCCCCC'], project_id: 'prj_01JCCCCCCCCCCCCCCCCCCCCCCC', env: 'sandbox', iat: t(), exp: t() + 900, jti: 'tok_' + 'a'.repeat(24) }, privateKey)}` });
+ assert.deepStrictEqual([sandbox.kind, sandbox.tier], ['principal', 'sandbox']);
+ } finally { await s.close(); g.close(); }
+ }
+
+ // ── Bucket math and cost weighting from the descriptors ──
+ {
+ let clock = now0;
+ const store = createGuardStore({ now: () => clock });
+ const q = createQuotas({ store, quotas: () => guardLib.limits.QUOTAS, now: () => clock });
+ const anon = { tier: 'anonymous', key: 'ip:a', ipKey: 'ip:a' };
+ const cost = (id) => [...IMG, ...AUDIO].find(s => s.id === id).cost;
+ assert.strictEqual(cost('png'), 5);
+ assert.strictEqual(cost('avif'), 8, 'AVIF encoding weighs more');
+ assert.strictEqual(cost('merge'), 20);
+ // tools-job anonymous: a bucket of 30 refilled at 60 a minute.
+ for (let i = 0; i < 3; i++) assert.ok(q.check(anon, { quotaClass: 'tools-job', cost: cost('avif') }).ok, `avif ${i + 1}`);
+ let r = q.check(anon, { quotaClass: 'tools-job', cost: cost('png') });
+ assert.ok(r.ok, 'png: 24 + 5 = 29 of 30');
+ assert.strictEqual(r.limit, 30);
+ assert.strictEqual(r.remaining, 1);
+ r = q.check(anon, { quotaClass: 'tools-job', cost: cost('png') });
+ assert.strictEqual(r.ok, false, 'the next png does not fit');
+ assert.strictEqual(r.binding, 'minute');
+ assert.strictEqual(r.retryAfter, 4, 'four more units at one a second');
+ clock += 4000;
+ assert.ok(q.check(anon, { quotaClass: 'tools-job', cost: cost('png') }).ok, 'refilled');
+ // Tiers: a signed-in person has four times the bucket, a service forty.
+ const user = { tier: 'user', key: 'user:x', ipKey: 'ip:a' };
+ let n = 0; while (q.check(user, { quotaClass: 'tools-job', cost: 5 }).ok) n++;
+ assert.strictEqual(n, 24, 'user burst 120 / 5');
+ // Sessions from one address share SESSION_IP_SHARE × one session's allowance: dropping the cookie is no reset.
+ const sessions = [1, 2, 3, 4].map(i => ({ tier: 'session', key: `session:${i}`, ipKey: 'ip:nat' }));
+ const spent = sessions.map(sess => { let k = 0; while (q.check(sess, { quotaClass: 'tools-job', cost: 5 }).ok) k++; return k; });
+ assert.deepStrictEqual(spent, [9, 9, 9, 0], 'three sessions use the address allowance, a fourth cookie gets nothing more');
+ // A run costing more than the whole bucket still runs when the bucket is full.
+ assert.ok(q.check({ tier: 'anonymous', key: 'ip:big', ipKey: 'ip:big' }, { quotaClass: 'tools-download', cost: 500 }).ok);
+ }
+
+ // ── Report vs enforce; headers and problem+json; the metric; the challenge hook ──
+ for (const mode of ['report', 'enforce']) {
+ const dataDir = tmp();
+ const g = makeGuard({ mode, dataDir, env: { TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-api': { anonymous: { perMinute: 2, burst: 2 } } }) } });
+ const registry = metrics.createRegistry();
+ g.attachMetrics(registry);
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ app.use(cookieParser());
+ app.use(g.identify);
+ app.use('/api/', g.apiQuota);
+ app.get('/api/x', (_req, res) => res.json({ ok: true }));
+ app.get('/api/probe', (req, res) => (g.target(req, res, { tool: 'ping', target: req.query.t }) ? res.json({ ok: true }) : undefined));
+ const s = await serve(app);
+ const ip = { 'X-Forwarded-For': '203.0.113.77' };
+ try {
+ const r1 = await fetch(`${s.base}/api/x`, { headers: ip });
+ assert.strictEqual(r1.status, 200);
+ assert.strictEqual(r1.headers.get('ratelimit-limit'), '2');
+ assert.strictEqual(r1.headers.get('ratelimit-remaining'), '1');
+ await fetch(`${s.base}/api/x`, { headers: ip });
+ const r3 = await fetch(`${s.base}/api/x`, { headers: ip });
+ if (mode === 'report') {
+ assert.strictEqual(r3.status, 200, 'report mode refuses nothing');
+ } else {
+ assert.strictEqual(r3.status, 429);
+ assert.strictEqual(r3.headers.get('content-type'), 'application/problem+json');
+ assert.ok(Number(r3.headers.get('retry-after')) >= 1);
+ assert.strictEqual(r3.headers.get('ratelimit-remaining'), '0');
+ const body = await r3.json();
+ assert.strictEqual(body.code, 'tools.quota.exceeded');
+ assert.strictEqual(body.status, 429);
+ assert.strictEqual(body.type, 'https://openvibe.network/problems/tools.quota.exceeded');
+ assert.strictEqual(body.quota_class, 'tools-api');
+ assert.strictEqual(body.tier, 'anonymous');
+ assert.ok(body.error, 'the legacy { error } field is there');
+ }
+ // Another address is not affected.
+ assert.strictEqual((await fetch(`${s.base}/api/x`, { headers: { 'X-Forwarded-For': '203.0.113.78' } })).status, 200);
+ // A hard limit (the per-target throttle: ping 6 a minute) refuses in both modes.
+ const codes = [];
+ for (let i = 0; i < 7; i++) codes.push((await fetch(`${s.base}/api/probe?t=${i % 2 ? 'Example.COM' : 'https://example.com:443/x'}`, { headers: { 'X-Forwarded-For': `198.51.100.${i + 1}` } })).status);
+ assert.deepStrictEqual(codes, [200, 200, 200, 200, 200, 200, 429], `${mode}: one target, all callers, however it is written`);
+ assert.strictEqual((await fetch(`${s.base}/api/probe?t=example.org`, { headers: { 'X-Forwarded-For': '198.51.100.99' } })).status, 200, 'another target is fine');
+ const rows = g.store.abuseRows();
+ const quota = rows.find(r => r.reason === 'quota');
+ assert.ok(quota, 'the quota refusal is in the abuse log');
+ assert.strictEqual(quota.enforced, mode === 'enforce' ? 1 : 0);
+ assert.ok(rows.find(r => r.reason === 'target' && r.enforced === 1 && r.tool === 'ping'));
+ const text = registry.metrics();
+ assert.match(text, /tools_guard_refused_total\{reason="quota",tool="-"\} [1-9]/);
+ assert.match(text, /tools_guard_refused_total\{reason="target",tool="ping"\} 1/);
+ assert.match(text, new RegExp(`tools_guard_enforcing ${mode === 'enforce' ? 1 : 0}`));
+ } finally { await s.close(); g.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }
+ }
+ {
+ // The challenge hook: a provider may let a person through instead of a quota refusal.
+ let asked = 0;
+ const g = makeGuard({ mode: 'enforce', env: { TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-api': { anonymous: { perMinute: 1, burst: 1 } } }) }, challenge: { name: 'test', required: () => { asked++; return true; }, verify: async (req) => req.headers['x-proof'] === 'ok' } });
+ assert.strictEqual(makeGuard().challenge.name, 'none', 'no challenge by default');
+ const app = express();
+ app.use('/api/', g.apiQuota);
+ app.get('/api/x', (_req, res) => res.json({ ok: true }));
+ const s = await serve(app);
+ try {
+ assert.strictEqual((await fetch(`${s.base}/api/x`)).status, 200);
+ const denied = await fetch(`${s.base}/api/x`);
+ assert.strictEqual(denied.status, 403);
+ assert.deepStrictEqual([(await denied.json()).code, asked], ['tools.challenge.required', 1]);
+ assert.strictEqual((await fetch(`${s.base}/api/x`, { headers: { 'x-proof': 'ok' } })).status, 200, 'a passed challenge goes on');
+ } finally { await s.close(); g.close(); }
+ }
+
+ // ── Day allowances survive a restart (guard.db) ──
+ {
+ const dataDir = tmp();
+ const env = { TOOLS_GUARD: 'enforce', TOOLS_GUARD_LIMITS: JSON.stringify({ 'tools-fetch': { anonymous: { perMinute: 600, burst: 600, perDay: 10 } } }) };
+ const req = { ip: '198.51.100.20', headers: {}, cookies: {} };
+ let g = makeGuard({ dataDir, env });
+ const caller1 = g.resolveCaller(req);
+ assert.ok(g.quotas.check(caller1, { quotaClass: 'tools-fetch', cost: 4 }).ok);
+ assert.ok(g.quotas.check(caller1, { quotaClass: 'tools-fetch', cost: 4 }).ok);
+ g.close();
+ g = makeGuard({ dataDir, env });
+ const caller2 = g.resolveCaller(req);
+ assert.strictEqual(caller2.ipKey, caller1.ipKey, "today's salt is kept, so the same address is the same key after a restart");
+ const r = g.quotas.check(caller2, { quotaClass: 'tools-fetch', cost: 4 });
+ assert.strictEqual(r.ok, false, '8 of 10 used before the restart: 4 more do not fit');
+ assert.strictEqual(r.binding, 'day');
+ assert.ok(g.quotas.check(caller2, { quotaClass: 'tools-fetch', cost: 2 }).ok, 'the last 2 still do');
+ g.close();
+ fs.rmSync(dataDir, { recursive: true, force: true });
+ }
+
+ // ── The abuse log: no raw address anywhere, 30 days ──
+ {
+ const dataDir = tmp();
+ let clock = now0;
+ const g = makeGuard({ dataDir, now: () => clock, mode: 'enforce' });
+ const RAW = '203.0.113.201';
+ const req = { ip: RAW, headers: {}, cookies: {} };
+ const res = { headersSent: false, statusCode: 200, setHeader() {}, end() {} };
+ for (let i = 0; i < 3; i++) g.refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'quota', tool: 'png', detail: 'x' });
+ clock += 25 * 24 * 3600 * 1000;
+ g.refuse({ ip: RAW, headers: {}, cookies: {} }, res, { status: 415, code: 'tools.file.unsupported_type', reason: 'sniff', tool: 'png', detail: 'x', hard: true });
+ let rows = g.store.abuseRows();
+ assert.strictEqual(rows.length, 2);
+ assert.strictEqual(rows[0].count, 3, 'repeats within a minute are one row with a count');
+ assert.match(rows[0].ip_hash, /^[0-9a-f]{32}$/);
+ assert.notStrictEqual(rows[0].ip_hash, rows[1].ip_hash, 'the salt rotates daily: yesterday\'s hash no longer matches');
+ g.store.db.pragma('wal_checkpoint(TRUNCATE)');
+ for (const f of fs.readdirSync(dataDir)) assert.ok(!fs.readFileSync(path.join(dataDir, f)).includes(RAW), `${f} holds no raw address`);
+ assert.strictEqual(g.store.db.prepare('SELECT COUNT(*) AS n FROM guard_salt').get().n, 1, 'only today\'s salt is kept');
+ clock += 6 * 24 * 3600 * 1000; // the first row is now 31 days old, the second 6
+ g.prune();
+ rows = g.store.abuseRows();
+ assert.deepStrictEqual(rows.map(r => r.reason), ['sniff'], 'rows older than 30 days are pruned');
+ g.close();
+ fs.rmSync(dataDir, { recursive: true, force: true });
+ }
+
+ // ── Upload sniffing: the bytes, not the name or the declared type ──
+ {
+ const png = Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d4944415478da63f8ffff3f0005fe02fea7d6a4c50000000049454e44ae426082', 'hex');
+ const pdf = Buffer.from('%PDF-1.7\n1 0 obj << >> endobj\ntrailer << >>\n%%EOF\n');
+ const hls = Buffer.from('#EXTM3U\n#EXT-X-TARGETDURATION:10\n#EXTINF:2,\nfile:///etc/passwd\n#EXT-X-ENDLIST\n');
+ const concat = Buffer.from("ffconcat version 1.0\nfile '/etc/passwd'\n");
+ const wav = Buffer.concat([Buffer.from('RIFF'), Buffer.alloc(4), Buffer.from('WAVEfmt '), Buffer.alloc(32)]);
+ assert.strictEqual(sniff.detect(png).mime, 'image/png');
+ assert.strictEqual(sniff.detect(pdf).mime, 'application/pdf');
+ assert.strictEqual(sniff.detect(wav).mime, 'audio/wav');
+ assert.strictEqual(sniff.detect(hls), null, 'a playlist is never media');
+ assert.strictEqual(sniff.detect(concat), null);
+ assert.strictEqual(sniff.detect(Buffer.from('<?xml version="1.0"?>\n<!-- x -->\n<svg xmlns="http://www.w3.org/2000/svg"/>')).mime, 'image/svg+xml');
+
+ const g = makeGuard(); // report mode: sniffing refuses anyway
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ const upload = multer({ storage: multer.memoryStorage() }).single('file');
+ const dir = tmp();
+ const disk = multer({ storage: multer.diskStorage({ destination: dir, filename: (_q, f, cb) => cb(null, `${crypto.randomBytes(8).toString('hex')}${path.extname(f.originalname)}`) }) }).single('file');
+ app.post('/img', upload, g.admitUpload(() => 'png'), (req, res) => res.json({ mime: req.file.mimetype, name: req.file.originalname }));
+ app.post('/audio', disk, g.admitUpload(() => 'reverb'), (req, res) => res.json({ mime: req.file.mimetype, path: path.basename(req.file.path), name: req.file.originalname }));
+ const s = await serve(app);
+ const send = (p, buf, name, type) => { const f = new FormData(); f.append('file', new Blob([buf], { type }), name); return fetch(s.base + p, { method: 'POST', body: f }); };
+ try {
+ let r = await send('/img', pdf, 'holiday.png', 'image/png');
+ assert.strictEqual(r.status, 415, 'a PDF renamed to .png is refused');
+ let body = await r.json();
+ assert.strictEqual(body.code, 'tools.file.unsupported_type');
+ assert.strictEqual(body.detected, 'application/pdf');
+ r = await send('/img', png, 'scan.pdf', 'application/pdf');
+ assert.strictEqual(r.status, 200, 'a PNG with the wrong name and type is what its bytes say');
+ assert.deepStrictEqual(await r.json(), { mime: 'image/png', name: 'scan.png' });
+ r = await send('/audio', hls, 'song.mp3', 'audio/mpeg');
+ assert.strictEqual(r.status, 415, 'an HLS playlist named .mp3 never reaches ffmpeg');
+ assert.strictEqual(fs.readdirSync(dir).length, 0, 'and the refused upload is deleted');
+ r = await send('/audio', wav, 'take.m3u8', 'application/octet-stream');
+ body = await r.json();
+ assert.strictEqual(r.status, 200);
+ assert.strictEqual(body.mime, 'audio/wav');
+ assert.match(body.path, /\.wav$/, 'the stored upload gets the extension of what it is');
+ assert.strictEqual(body.name, 'take.wav');
+ } finally { await s.close(); g.close(); fs.rmSync(dir, { recursive: true, force: true }); }
+ }
+
+ // ── The port-scan cap (hard: report mode too) ──
+ {
+ const g = makeGuard();
+ const res = () => { const r = { statusCode: 200, headers: {}, body: null, headersSent: false, setHeader(k, v) { this.headers[k.toLowerCase()] = v; }, end(b) { this.body = JSON.parse(b); } }; return r; };
+ const reqFrom = (ip, cookie) => ({ ip, headers: {}, cookies: cookie ? { ov_tools_jobs: cookie } : {} });
+ const twenty = Array.from({ length: 20 }, (_, i) => 1000 + i);
+ let out = res();
+ assert.strictEqual(g.portScan(reqFrom('198.51.100.40'), out, { tool: 'port', target: 'a.example', ports: [...twenty, 9999] }), false);
+ assert.deepStrictEqual([out.statusCode, out.body.code], [400, 'tools.run.invalid'], 'at most 20 ports a request');
+ for (let i = 0; i < 5; i++) assert.ok(g.portScan(reqFrom('198.51.100.41'), res(), { tool: 'port', target: 'a.example', ports: twenty }), `check ${i + 1}`);
+ out = res();
+ assert.strictEqual(g.portScan(reqFrom('198.51.100.41'), out, { tool: 'port', target: 'a.example', ports: [22] }), false, '100 ports in ten minutes, then no more');
+ assert.deepStrictEqual([out.statusCode, out.body.code, out.body.scope], [429, 'tools.quota.exceeded', 'ports']);
+ assert.ok(Number(out.headers['retry-after']) > 0);
+ // Distinct hosts: ten, whatever cookie the browser shows (sessions count by address).
+ for (let i = 0; i < 10; i++) assert.ok(g.portScan(reqFrom('198.51.100.42', crypto.randomBytes(24).toString('base64url')), res(), { tool: 'port', target: `h${i}.example`, ports: [443] }));
+ out = res();
+ assert.strictEqual(g.portScan(reqFrom('198.51.100.42', crypto.randomBytes(24).toString('base64url')), out, { tool: 'port', target: 'h10.example', ports: [443] }), false, 'an eleventh host is a sweep');
+ assert.ok(g.portScan(reqFrom('198.51.100.42'), res(), { tool: 'port', target: 'h3.example', ports: [80] }), 'the same hosts again are fine');
+ g.close();
+ }
+
+ // ── The sync semaphore holds heavy calls to the cap ──
+ {
+ const sem = createSemaphore({ max: 2, queue: 3, waitMs: 200 });
+ let running = 0, peak = 0;
+ const work = async () => { const release = await sem.acquire(); running++; peak = Math.max(peak, running); await new Promise(r => setTimeout(r, 30)); running--; release(); };
+ await Promise.all([work(), work(), work(), work(), work()]);
+ assert.strictEqual(peak, 2, 'never more than two at once');
+ const held = [await sem.acquire(), await sem.acquire()];
+ const waiters = [sem.acquire(), sem.acquire(), sem.acquire()];
+ await assert.rejects(sem.acquire(), (e) => e instanceof Busy && e.reason === 'queue', 'a full wait list refuses');
+ held.forEach(r => r());
+ (await Promise.all(waiters.slice(0, 2))).forEach(r => r());
+ (await waiters[2])();
+ const again = [await sem.acquire(), await sem.acquire()];
+ await assert.rejects(sem.acquire(), (e) => e instanceof Busy && e.reason === 'timeout', 'waiting too long refuses');
+ again.forEach(r => r());
+
+ // Over HTTP: TOOLS_SYNC_CONCURRENCY=1 runs one request at a time; with no room to wait, 503 tools.busy.
+ for (const mode of ['enforce', 'report']) {
+ const g = makeGuard({ mode, env: { TOOLS_SYNC_CONCURRENCY: '1', TOOLS_SYNC_QUEUE: '1' } });
+ let now = 0, max = 0;
+ const app = express();
+ app.post('/p', g.heavy('png'), async (_req, res) => { now++; max = Math.max(max, now); await new Promise(r => setTimeout(r, 80)); now--; res.json({ ok: true }); });
+ const s = await serve(app);
+ try {
+ const codes = await Promise.all([1, 2, 3].map(() => fetch(`${s.base}/p`, { method: 'POST' }).then(r => r.status)));
+ if (mode === 'enforce') {
+ assert.deepStrictEqual(codes.sort(), [200, 200, 503], 'one runs, one waits, the third is told to come back');
+ assert.strictEqual(max, 1);
+ } else {
+ assert.deepStrictEqual(codes, [200, 200, 200], 'report mode lets the overflow run');
+ assert.ok(g.store.abuseRows().some(r => r.reason === 'busy.sync' && r.enforced === 0));
+ }
+ } finally { await s.close(); g.close(); }
+ }
+ }
+
+ // ── The job store's bounds: queued jobs and disk (system.busy) through the guard ──
+ for (const mode of ['enforce', 'report']) {
+ const dataDir = tmp();
+ const db = new Database(':memory:');
+ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, maxQueued: 2, diskBudgetBytes: 1_000_000, log: quiet });
+ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });
+ system.start();
+ const g = makeGuard({ mode });
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ app.use(express.json());
+ app.use(cookieParser());
+ jobs.mountJobRoutes(app, { system, contracts, resolveOwner: g.ownerResolver(), onBusy: (req, res, busy) => g.jobsBusy(req, res, busy) });
+ const s = await serve(app);
+ const submit = () => fetch(`${s.base}/api/v1/jobs`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ type: 'test.noop', input: {} }) });
+ try {
+ assert.strictEqual((await submit()).status, 202);
+ assert.strictEqual((await submit()).status, 202);
+ assert.strictEqual(system.busy().reason, 'queue');
+ const r = await submit();
+ if (mode === 'enforce') {
+ assert.strictEqual(r.status, 503, 'a third queued job is refused');
+ assert.strictEqual((await r.json()).code, 'tools.busy');
+ assert.strictEqual(r.headers.get('retry-after'), '60');
+ } else {
+ assert.strictEqual(r.status, 202, 'report mode accepts it');
+ assert.ok(g.store.abuseRows().some(x => x.reason === 'busy.queue' && x.enforced === 0));
+ }
+ } finally { await s.close(); g.close(); system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }
+ }
+ // Unfinished browser-session jobs are bounded per address too: a new cookie is no new allowance.
+ for (const mode of ['enforce', 'report']) {
+ const dataDir = tmp();
+ const db = new Database(':memory:');
+ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, maxActivePerOwner: 1, log: quiet });
+ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });
+ system.start();
+ assert.strictEqual(system.bounds.maxActivePerAddress, 3, '3 × the per-owner bound by default');
+ const g = makeGuard({ mode });
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ app.use(express.json());
+ app.use(cookieParser());
+ jobs.mountJobRoutes(app, {
+ system, contracts, resolveOwner: g.ownerResolver(),
+ onAddressFull: (req, res, full) => !g.refuse(req, res, { status: 429, code: 'tools.job.too_many_active', reason: 'jobs.address', retryAfter: 30, detail: `at most ${full.limit}` }),
+ });
+ const s = await serve(app);
+ const submit = (ip, cookie) => fetch(`${s.base}/api/v1/jobs`, { method: 'POST', headers: { 'content-type': 'application/json', 'X-Forwarded-For': ip, ...(cookie ? { cookie } : {}) }, body: JSON.stringify({ type: 'test.noop', input: {} }) });
+ try {
+ const first = await submit('198.51.100.60');
+ assert.strictEqual(first.status, 202);
+ const cookie = String(first.headers.get('set-cookie')).split(';')[0];
+ assert.strictEqual((await submit('198.51.100.60', cookie)).status, 429, 'the per-owner bound (1) as before');
+ const codes = [];
+ for (let i = 0; i < 3; i++) codes.push((await submit('198.51.100.60')).status); // a fresh cookie each time
+ assert.deepStrictEqual(codes, mode === 'enforce' ? [202, 202, 429] : [202, 202, 202], `${mode}: three per address`);
+ assert.strictEqual((await submit('198.51.100.61')).status, 202, 'another address is fine');
+ const row = db.prepare('SELECT ip_key FROM tool_jobs LIMIT 1').get();
+ assert.match(row.ip_key, /^ip:[0-9a-f]{32}$/, 'the job keeps the hashed key, never the address');
+ if (mode === 'report') assert.ok(g.store.abuseRows().some(x => x.reason === 'jobs.address' && x.enforced === 0));
+ } finally { await s.close(); g.close(); system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true }); }
+ }
+ {
+ const dataDir = tmp();
+ const db = new Database(':memory:');
+ const system = jobs.createJobSystem({ db, contracts, service: 'test', dataDir, concurrency: 0, diskBudgetBytes: 1000, log: quiet });
+ system.define({ type: 'test.noop', maxFiles: 1, async run() { return { files: [], data: {} }; } });
+ system.start();
+ await system.refreshDisk();
+ assert.strictEqual(system.busy(), null);
+ await system.submit({ owner: 'session:x', type: 'test.noop', input: {}, files: [{ buffer: Buffer.alloc(1500), name: 'a.bin', mime: 'application/octet-stream', size: 1500 }] });
+ assert.strictEqual(system.busy().reason, 'disk', 'accepted inputs count against the disk budget at once');
+ assert.strictEqual((await system.refreshDisk()) >= 1500, true, 'and the walk finds them');
+ system.stop(); db.close(); fs.rmSync(dataDir, { recursive: true, force: true });
+ }
+
+ console.log('guard: callers (/64, one loopback hop, services by token, aud-checked users, sessions, sandbox), bucket math + cost, session sharing, report vs enforce, headers + problem+json, day counters across restarts, sniffing, target throttle, port cap, abuse log + prune + metric, semaphore, job bounds (queue, disk, per address), challenge hook: all checks passed');
+})().catch((err) => { console.error(err); process.exit(1); });
@@ -71,7 +71,9 @@ function assertShape(body, service) {
@@ -71,7 +71,9 @@ function assertShape(body, service) {
const fd = new FormData();
const fd = new FormData();
fd.append('type', 'img.process');
fd.append('type', 'img.process');
fd.append('input', JSON.stringify({ tool: 'convert', format: 'webp' }));
fd.append('input', JSON.stringify({ tool: 'convert', format: 'webp' }));
- fd.append('file', new Blob([Buffer.from('not really a png')], { type: 'image/png' }), 'a.png');
+ // A real 1×1 PNG: uploads are sniffed (apps/_shared/guard), so bytes must be what they claim.
+ const onePx = Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d4944415478da63f8ffff3f0005fe02fea7d6a4c50000000049454e44ae426082', 'hex');
+ fd.append('file', new Blob([onePx], { type: 'image/png' }), 'a.png');
const sub = await fetch(`${img.base}/api/v1/jobs`, { method: 'POST', body: fd });
const sub = await fetch(`${img.base}/api/v1/jobs`, { method: 'POST', body: fd });
assert.strictEqual(sub.status, 202, await sub.clone().text());
assert.strictEqual(sub.status, 202, await sub.clone().text());
const job = await sub.json();
const job = await sub.json();
deleted file mode 100644
deleted file mode 100644
@@ -1,120 +0,0 @@
@@ -1,120 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Audio.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)
-// Lightweight auth — no DB, no linked_accounts.
-// Verifies the shared ov_token cookie offline against the
-// Network's public key and attaches decoded claims to req.user.
-// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise
-// fetched from the Network's /api/.well-known/jwks endpoint.
-// ═══════════════════════════════════════════════════════════════
-
-const jwt = require('jsonwebtoken');
-const fs = require('fs');
-const path = require('path');
-const config = require('./config');
-
-let publicKey = null;
-let retryTimer = null;
-
-const ISSUER = config.networkUrl;
-const JWKS_PATH = '/api/.well-known/jwks';
-const FETCH_TIMEOUT_MS = 5000;
-const RETRY_INTERVAL_MS = 60 * 1000;
-
-function loadKeyFromFile() {
- for (const p of config.publicKeyPaths) {
- try {
- const resolved = path.resolve(p);
- if (fs.existsSync(resolved)) {
- publicKey = fs.readFileSync(resolved, 'utf8');
- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);
- return true;
- }
- } catch { /* try next */ }
- }
- return false;
-}
-
-async function fetchKeyFromNetwork() {
- for (const base of [config.networkInternalUrl, config.networkUrl]) {
- try {
- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });
- if (!res.ok) continue;
- const body = await res.json();
- if (body && body.public_key) {
- publicKey = body.public_key;
- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);
- return true;
- }
- } catch { /* try next base */ }
- }
- return false;
-}
-
-function loadPublicKey() {
- if (loadKeyFromFile()) return;
- fetchKeyFromNetwork().then((loaded) => {
- if (loaded || retryTimer) return;
- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);
- retryTimer = setInterval(async () => {
- if (await fetchKeyFromNetwork()) {
- clearInterval(retryTimer);
- retryTimer = null;
- }
- }, RETRY_INTERVAL_MS);
- if (retryTimer.unref) retryTimer.unref();
- });
-}
-loadPublicKey();
-
-function extractToken(req) {
- const authHeader = req.headers.authorization;
- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);
- if (req.cookies?.ov_token) return req.cookies.ov_token;
- if (req.cookies?.token) return req.cookies.token;
- return null;
-}
-
-function verifyToken(token) {
- if (!publicKey || !token) return null;
- try {
- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });
- } catch {
- return null;
- }
-}
-
-/**
- * Optional auth — attaches req.user if valid token present.
- * Never blocks the request; anonymous users get req.user = null.
- */
-function optionalAuth(req, _res, next) {
- const token = extractToken(req);
- if (token) {
- const decoded = verifyToken(token);
- if (decoded) {
- req.user = decoded;
- req.token = token;
- }
- }
- next();
-}
-
-/**
- * Required auth — 401 if no valid token.
- */
-function requireAuth(req, res, next) {
- const token = extractToken(req);
- if (!token) return res.status(401).json({ error: 'Authentication required' });
- const decoded = verifyToken(token);
- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });
- req.user = decoded;
- req.token = token;
- next();
-}
-
-/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */
-function getPublicKey() { return publicKey; }
-
-module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };
@@ -14,11 +14,17 @@ const { DOMAIN_MAP } = require('./domain-map');
@@ -14,11 +14,17 @@ const { DOMAIN_MAP } = require('./domain-map');
// tools/convert.js FORMAT_CONFIG and tools/ffmpeg-helper.js FORMAT_MIME.
// tools/convert.js FORMAT_CONFIG and tools/ffmpeg-helper.js FORMAT_MIME.
const FORMAT_MIME = { mp3: 'audio/mpeg', wav: 'audio/wav', flac: 'audio/flac', ogg: 'audio/ogg', m4a: 'audio/mp4', aac: 'audio/aac', opus: 'audio/opus', wma: 'audio/x-ms-wma', aiff: 'audio/aiff', ac3: 'audio/ac3', webm: 'audio/webm' };
const FORMAT_MIME = { mp3: 'audio/mpeg', wav: 'audio/wav', flac: 'audio/flac', ogg: 'audio/ogg', m4a: 'audio/mp4', aac: 'audio/aac', opus: 'audio/opus', wma: 'audio/x-ms-wma', aiff: 'audio/aiff', ac3: 'audio/ac3', webm: 'audio/webm' };
const FORMATS = Object.keys(FORMAT_MIME);
const FORMATS = Object.keys(FORMAT_MIME);
-// config.js upload.allowedMimes (video types are there for the extractor; octet-stream for mis-typed uploads).
-const ACCEPT = ['audio/mpeg', 'audio/mp3', 'audio/wav', 'audio/x-wav', 'audio/wave', 'audio/flac', 'audio/x-flac', 'audio/ogg', 'audio/vorbis', 'audio/mp4', 'audio/x-m4a', 'audio/aac', 'audio/opus', 'audio/x-ms-wma', 'audio/aiff', 'audio/x-aiff', 'audio/ac3', 'audio/webm', 'audio/amr', 'video/mp4', 'video/webm', 'video/x-matroska', 'video/avi', 'video/x-msvideo', 'video/quicktime', 'video/x-flv', 'video/ogg', 'application/ogg', 'application/octet-stream'];
+// What the bytes may be (checked against the bytes by the guard, apps/_shared/guard/sniff.js; the upload
+// itself also lets application/octet-stream through, since browsers send it for audio they do not know).
+// Every operation reads audio containers, including the video containers that carry audio (MP4, WebM,
+// Matroska, QuickTime, Ogg); only the extractor also reads AVI and FLV (guard/ffmpeg.js formatsFor).
+const AUDIO_ACCEPT = ['audio/mpeg', 'audio/mp3', 'audio/wav', 'audio/x-wav', 'audio/wave', 'audio/flac', 'audio/x-flac', 'audio/ogg', 'audio/vorbis', 'audio/mp4', 'audio/x-m4a', 'audio/aac', 'audio/opus', 'audio/x-ms-wma', 'audio/aiff', 'audio/x-aiff', 'audio/ac3', 'audio/webm', 'audio/amr', 'video/mp4', 'video/webm', 'video/x-matroska', 'video/quicktime', 'video/ogg', 'application/ogg'];
+const ACCEPT = [...AUDIO_ACCEPT, 'video/avi', 'video/x-msvideo', 'video/x-flv'];
const MAX_BYTES = 100 * 1024 * 1024; // config.js upload.maxFileSize
const MAX_BYTES = 100 * 1024 * 1024; // config.js upload.maxFileSize
const TIMEOUT_MS = 15 * 60 * 1000; // process.js audio.process timeoutMs
const TIMEOUT_MS = 15 * 60 * 1000; // process.js audio.process timeoutMs
const MERGE_MAX_SECONDS = 3 * 60 * 60; // tools/merge.js MAX_TOTAL_SECONDS
const MERGE_MAX_SECONDS = 3 * 60 * 60; // tools/merge.js MAX_TOTAL_SECONDS
+// Longest input any operation reads (process.js probes it first; ffmpeg reads no more than this, -t).
+const MAX_SECONDS = Math.max(1, parseInt(process.env.AUDIO_MAX_DURATION, 10) || 3 * 60 * 60);
// An effect keeps the input's own format unless `format` names one (ffmpeg-helper getMime: amr too, anything else octet-stream).
// An effect keeps the input's own format unless `format` names one (ffmpeg-helper getMime: amr too, anything else octet-stream).
const ALL_OUT = [...new Set([...Object.values(FORMAT_MIME), 'audio/amr', 'application/octet-stream'])];
const ALL_OUT = [...new Set([...Object.values(FORMAT_MIME), 'audio/amr', 'application/octet-stream'])];
@@ -85,9 +91,9 @@ function tool(id, extra = {}) {
@@ -85,9 +91,9 @@ function tool(id, extra = {}) {
job: { type: 'audio.process', operation: op, ...(fixed && { preset: { format: ctx.defaultFormat } }) },
job: { type: 'audio.process', operation: op, ...(fixed && { preset: { format: ctx.defaultFormat } }) },
legacy: [op === 'merge' ? 'POST /api/process/multi' : 'POST /api/process'],
legacy: [op === 'merge' ? 'POST /api/process/multi' : 'POST /api/process'],
input: INPUT[op](),
input: INPUT[op](),
- files: op === 'merge' ? { min: 2, max: 5, accept: ACCEPT, maxBytes: MAX_BYTES } : { min: 1, max: 1, accept: ACCEPT, maxBytes: MAX_BYTES },
+ files: { min: op === 'merge' ? 2 : 1, max: op === 'merge' ? 5 : 1, accept: op === 'extract' ? ACCEPT : AUDIO_ACCEPT, maxBytes: MAX_BYTES },
output: { kind: 'file', mime, schema: RESULT },
output: { kind: 'file', mime, schema: RESULT },
- limits: { timeoutMs: TIMEOUT_MS, ...(op === 'merge' && { maxDurationSec: MERGE_MAX_SECONDS }) },
+ limits: { timeoutMs: TIMEOUT_MS, maxDurationSec: op === 'merge' ? Math.min(MERGE_MAX_SECONDS, MAX_SECONDS) : MAX_SECONDS },
// ffmpeg over an upload is heavy work: a browser session (the page has one), a person or a token.
// ffmpeg over an upload is heavy work: a browser session (the page has one), a person or a token.
auth: { anonymous: false, capability: 'tools.tool.run' },
auth: { anonymous: false, capability: 'tools.tool.run' },
quotaClass: 'tools-job', cost: op === 'merge' ? 20 : op === 'metadata' || op === 'waveform' ? 5 : 10, egress: false,
quotaClass: 'tools-job', cost: op === 'merge' ? 20 : op === 'metadata' || op === 'waveform' ? 5 : 10, egress: false,
@@ -100,4 +106,4 @@ function tool(id, extra = {}) {
@@ -100,4 +106,4 @@ function tool(id, extra = {}) {
const IDS = ['mp3', 'wav', 'flac', 'ogg', 'm4a', 'opus', 'aac', 'wma', 'aiff', 'ac3', 'extract', 'trim', 'merge', 'ringtone', 'speed', 'pitch', 'reverse', 'normalize', 'noise', 'silence', 'vocal', 'fade', 'loop', 'bass', 'equalizer', 'compressor', 'podcast', 'voice', 'echo', 'reverb', 'chorus', 'distortion', 'bitcrusher', 'stereo', 'waveform', 'metadata'];
const IDS = ['mp3', 'wav', 'flac', 'ogg', 'm4a', 'opus', 'aac', 'wma', 'aiff', 'ac3', 'extract', 'trim', 'merge', 'ringtone', 'speed', 'pitch', 'reverse', 'normalize', 'noise', 'silence', 'vocal', 'fade', 'loop', 'bass', 'equalizer', 'compressor', 'podcast', 'voice', 'echo', 'reverb', 'chorus', 'distortion', 'bitcrusher', 'stereo', 'waveform', 'metadata'];
const SPECS = IDS.map(id => tool(id));
const SPECS = IDS.map(id => tool(id));
-module.exports = { SPECS, FORMAT_MIME, ACCEPT, MAX_BYTES, TIMEOUT_MS };
+module.exports = { SPECS, FORMAT_MIME, ACCEPT, AUDIO_ACCEPT, MAX_BYTES, TIMEOUT_MS, MAX_SECONDS };
@@ -13,21 +13,20 @@ const cookieParser = require('cookie-parser');
@@ -13,21 +13,20 @@ const cookieParser = require('cookie-parser');
const path = require('path');
const path = require('path');
const fs = require('fs');
const fs = require('fs');
+const rateLimit = require('express-rate-limit');
const config = require('./config');
const config = require('./config');
-const auth = require('./auth');
-const { optionalAuth } = auth;
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { getTool, listTools } = require('./tools');
const { getTool, listTools } = require('./tools');
const { readMetadata } = require('./tools/metadata');
const { readMetadata } = require('./tools/metadata');
const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');
const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');
-const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');
const retention = require('./retention/manager');
const retention = require('./retention/manager');
const { probe, getDuration, cleanTmp } = require('./tools/ffmpeg-helper');
const { probe, getDuration, cleanTmp } = require('./tools/ffmpeg-helper');
-const { buildOptions, describe, defineJobs } = require('./process');
+const { buildOptions, describe, defineJobs, runSync, jobContext, limitsFor } = require('./process');
const { hostGuard, ownHost } = require('../../_shared/host-role');
const { hostGuard, ownHost } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const jobsRuntime = require('../../_shared/jobs');
const jobsRuntime = require('../../_shared/jobs');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const sdk = require('openvibe-sdk');
const sdk = require('openvibe-sdk');
@@ -41,13 +40,29 @@ const analyticsDb = new Database(analyticsDbPath);
@@ -41,13 +40,29 @@ const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-audio', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-audio', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
+// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
+// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse. Every audio
+// tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
+const SPECS = require('./descriptors').SPECS;
+const guard = createGuard({
+ app: 'audio', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,
+ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,
+});
+const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);
+/** The tool a request is for before its body is read: the host's own, else the operation it defaults to. */
+const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('audio.process', req.ctx.defaultOp || 'convert', null) || { id: null }).id);
+/** The tool a parsed request runs: its `tool` (operation) on this host. */
+const toolOf = (req) => { const d = guard.toolForJob('audio.process', String((req.body && req.body.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : hostTool(req); };
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready, which } = require('../../_shared/observe');
const { observe, checks: ready, which } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-audio', release: release.release,
service: 'tools-audio', release: release.release,
checks: [
checks: [
@@ -57,19 +72,20 @@ observe({
@@ -57,19 +72,20 @@ observe({
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
+ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
...jobsRuntime.readyChecks(() => jobs),
...jobsRuntime.readyChecks(() => jobs),
ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'every audio tool runs ffmpeg; without it jobs and conversions fail' }),
ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'every audio tool runs ffmpeg; without it jobs and conversions fail' }),
],
],
jobs: () => jobs,
jobs: () => jobs,
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
-app.set('trust proxy', 2); // Cloudflare → Nginx → Node
+app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
directives: {
directives: {
@@ -101,8 +117,9 @@ app.use(exceptRegistry(cors({
@@ -101,8 +117,9 @@ app.use(exceptRegistry(cors({
credentials: true,
credentials: true,
})));
})));
-// ── Rate Limiting ────────────────────────────────────────────
-app.use('/api/', apiLimiter);
+// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──
+app.use(guard.identify);
+app.use('/api/', apiLimiter, guard.apiQuota);
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's audio tools ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's audio tools ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -113,9 +130,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
@@ -113,9 +130,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
// ── Analytics Middleware ─────────────────────────────────────
// ── Analytics Middleware ─────────────────────────────────────
app.use(analytics.middleware());
app.use(analytics.middleware());
-// ── Auth (optional on all routes) ────────────────────────────
-app.use(optionalAuth);
-
// ── Hosts ────────────────────────────────────────────────────
// ── Hosts ────────────────────────────────────────────────────
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
@@ -137,8 +151,11 @@ app.get('/api/health', (_req, res) => {
@@ -137,8 +151,11 @@ app.get('/api/health', (_req, res) => {
res.json({ status: 'ok', service: 'openvibe-audio', version: '1.0.0', files: stats, jobs: jobs.stats() });
res.json({ status: 'ok', service: 'openvibe-audio', version: '1.0.0', files: stats, jobs: jobs.stats() });
});
});
-// Domain context (frontend calls this on load to get branding)
+// Domain context (frontend calls this on load to get branding). It also starts this browser's session
+// (ov_tools_jobs): audio tools need one (or a sign-in, or a token) before the first upload.
app.get('/api/context', (req, res) => {
app.get('/api/context', (req, res) => {
+ guard.ensureSession(req, res);
+ res.set('Cache-Control', 'private, no-store');
const ctx = req.ctx;
const ctx = req.ctx;
const tools = listTools();
const tools = listTools();
res.json({
res.json({
@@ -160,10 +177,10 @@ app.get('/api/tools', (_req, res) => {
@@ -160,10 +177,10 @@ app.get('/api/tools', (_req, res) => {
});
});
// ── Probe / Metadata Endpoint ────────────────────────────────
// ── Probe / Metadata Endpoint ────────────────────────────────
-app.post('/api/probe', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/probe', burstLimiter, processLimiter, guard.toolQuota('metadata'), uploadSingle, guard.admitUpload(() => 'metadata'), async (req, res) => {
try {
try {
- const info = await probe(req.file.path);
- const meta = await readMetadata(req.file.path);
+ // Hardened like every run: local files only, audio demuxers only.
+ const { info, meta } = await jobContext.run(limitsFor('metadata'), async () => ({ info: await probe(req.file.path), meta: await readMetadata(req.file.path) }));
cleanTmp(req.file.path);
cleanTmp(req.file.path);
res.json({ success: true, ...meta, probe: info.format });
res.json({ success: true, ...meta, probe: info.format });
} catch (err) {
} catch (err) {
@@ -173,7 +190,7 @@ app.post('/api/probe', burstLimiter, processLimiter, uploadSingle, async (req, r
@@ -173,7 +190,7 @@ app.post('/api/probe', burstLimiter, processLimiter, uploadSingle, async (req, r
});
});
// ── Main Processing Endpoint ─────────────────────────────────
// ── Main Processing Endpoint ─────────────────────────────────
-app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const tool = getTool(toolId);
const tool = getTool(toolId);
@@ -186,8 +203,8 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
@@ -186,8 +203,8 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
return res.status(400).json({ error: `${tool.label} takes several files: send them in "files" to /api/process/multi` });
return res.status(400).json({ error: `${tool.label} takes several files: send them in "files" to /api/process/multi` });
}
}
- // Execute the tool (same options and code path as the audio.process job)
- const result = await tool.handler(req.file.path, buildOptions(req.body, toolId, req.ctx));
+ // Execute the tool (same options, hardening and code path as the audio.process job; killed at its timeout)
+ const result = await runSync(res, toolId, req.file.path, buildOptions(req.body, toolId, req.ctx));
// Clean up the uploaded input file
// Clean up the uploaded input file
cleanTmp(req.file.path);
cleanTmp(req.file.path);
@@ -207,13 +224,14 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
@@ -207,13 +224,14 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize) });
res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize) });
} catch (err) {
} catch (err) {
cleanTmp(req.file?.path);
cleanTmp(req.file?.path);
+ if (guard.toolRefused(req, res, err, toolOf(req))) return;
console.error('[Process] Error:', err.message);
console.error('[Process] Error:', err.message);
res.status(422).json({ error: err.message || 'Audio processing failed' });
res.status(422).json({ error: err.message || 'Audio processing failed' });
}
}
});
});
// ── Multi-File Processing Endpoint (merge) ───────────────────
// ── Multi-File Processing Endpoint (merge) ───────────────────
-app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, async (req, res) => {
+app.post('/api/process/multi', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadMultiple, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
const paths = req.files.map(f => f.path);
const paths = req.files.map(f => f.path);
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp;
const toolId = req.body.tool || req.ctx.defaultOp;
@@ -222,7 +240,7 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
@@ -222,7 +240,7 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
cleanTmp(...paths);
cleanTmp(...paths);
return res.status(400).json({ error: tool ? `Tool "${toolId}" takes one file: use /api/process` : `Unknown tool: ${toolId}` });
return res.status(400).json({ error: tool ? `Tool "${toolId}" takes one file: use /api/process` : `Unknown tool: ${toolId}` });
}
}
- const result = await tool.handler(paths, buildOptions(req.body, toolId, req.ctx));
+ const result = await runSync(res, toolId, req.files.map(f => f.path), buildOptions(req.body, toolId, req.ctx));
cleanTmp(...paths);
cleanTmp(...paths);
const first = req.files[0].originalname || 'audio';
const first = req.files[0].originalname || 'audio';
const saved = retention.saveOutputFromFile(result.outputPath, result.ext, result.mime, !!req.user,
const saved = retention.saveOutputFromFile(result.outputPath, result.ext, result.mime, !!req.user,
@@ -230,7 +248,8 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
@@ -230,7 +248,8 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
const inputSize = req.files.reduce((n, f) => n + f.size, 0);
const inputSize = req.files.reduce((n, f) => n + f.size, 0);
res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize), fileCount: req.files.length });
res.json({ success: true, download: saved, ...describe(toolId, result, saved.size, inputSize), fileCount: req.files.length });
} catch (err) {
} catch (err) {
- cleanTmp(...paths);
+ cleanTmp(...req.files.map(f => f.path));
+ if (guard.toolRefused(req, res, err, toolOf(req))) return;
console.error('[Process/Multi] Error:', err.message);
console.error('[Process/Multi] Error:', err.message);
res.status(422).json({ error: err.message || 'Audio processing failed' });
res.status(422).json({ error: err.message || 'Audio processing failed' });
}
}
@@ -242,10 +261,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
@@ -242,10 +261,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
// after a reload or a restart (apps/_shared/jobs).
// after a reload or a restart (apps/_shared/jobs).
const jobs = jobsRuntime.setupJobs({
const jobs = jobsRuntime.setupJobs({
app, service: 'audio', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
app, service: 'audio', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,
+ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,
define: defineJobs,
define: defineJobs,
receive: uploadAny,
receive: uploadAny,
- limiters: [burstLimiter, processLimiter],
+ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],
+ jobTool: (req, type, input) => { const d = guard.toolForJob('audio.process', String((input && input.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : null; },
defaults(req, input) {
defaults(req, input) {
const out = { ...input };
const out = { ...input };
if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';
if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';
@@ -355,6 +375,7 @@ function shutdown() {
@@ -355,6 +375,7 @@ function shutdown() {
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,51 +0,0 @@
@@ -1,51 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Audio.OpenVibe — Rate Limiting Middleware
-// Tiered rate limits: anonymous vs authenticated users.
-// Includes anti-abuse burst protection and stricter anon limits.
-// ═══════════════════════════════════════════════════════════════
-
-const rateLimit = require('express-rate-limit');
-
-/** General API rate limit */
-const apiLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 120 : 60,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Too many requests. Please try again later.' },
-});
-
-/**
- * Processing rate limit — stricter, tiered by auth.
- * Audio processing is more CPU-intensive than image processing,
- * so limits are a bit tighter.
- * Anonymous: 6 conversions/minute
- * Authenticated: 20 conversions/minute
- */
-const processLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 20 : 6,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },
-});
-
-/**
- * Anti-burst limiter — prevents rapid-fire abuse (bot attacks).
- * Very short window: max 3 requests per 5 seconds per IP.
- * Applies to processing endpoints only.
- */
-const burstLimiter = rateLimit({
- windowMs: 5 * 1000,
- max: 3,
- standardHeaders: false,
- legacyHeaders: false,
- keyGenerator: (req) => req.ip,
- message: { error: 'Too many requests in quick succession. Please slow down.' },
-});
-
-module.exports = { apiLimiter, processLimiter, burstLimiter };
@@ -6,8 +6,14 @@
@@ -6,8 +6,14 @@
//
//
// Jobs can be cancelled and report progress although the tools build their own ffmpeg commands:
// Jobs can be cancelled and report progress although the tools build their own ffmpeg commands:
// FfmpegCommand#run is wrapped once, and a command started inside a job's AsyncLocalStorage
// FfmpegCommand#run is wrapped once, and a command started inside a job's AsyncLocalStorage
-// context reports ffmpeg's percent to the job and is killed when the job's signal aborts.
-// Commands started outside a job (the synchronous endpoint) are untouched.
+// context reports ffmpeg's percent to the job and is killed when the job's signal aborts. The
+// synchronous endpoint runs in the same kind of context (runSync): a deadline (the descriptor's
+// limits.timeoutMs) and a client that goes away abort it, and ffmpeg is killed.
+//
+// Hardening (apps/_shared/guard/ffmpeg.js, in both paths): every input of every command and every
+// probe reads only local files and pipes, only through the operation's demuxers (never hls, concat,
+// image2, lavfi…), and at most limits.maxDurationSec of it; inputs are probed first and a longer one
+// is refused before any work starts.
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
const fs = require('fs');
const fs = require('fs');
@@ -15,21 +21,36 @@ const path = require('path');
@@ -15,21 +21,36 @@ const path = require('path');
const { AsyncLocalStorage } = require('async_hooks');
const { AsyncLocalStorage } = require('async_hooks');
const ffmpeg = require('fluent-ffmpeg');
const ffmpeg = require('fluent-ffmpeg');
const { getTool, listTools } = require('./tools');
const { getTool, listTools } = require('./tools');
+const { probe } = require('./tools/ffmpeg-helper');
+const hardening = require('../../_shared/guard/ffmpeg');
+const { SPECS } = require('./descriptors');
const jobContext = new AsyncLocalStorage();
const jobContext = new AsyncLocalStorage();
const RESERVED = new Set(['tool', 'file', 'files']);
const RESERVED = new Set(['tool', 'file', 'files']);
+/** An operation's limits from its descriptor (every audio tool on one operation has the same). */
+const LIMITS = new Map();
+for (const s of SPECS) if (!LIMITS.has(s.job.operation)) LIMITS.set(s.job.operation, s.limits);
+function limitsFor(op) {
+ const l = LIMITS.get(op) || LIMITS.get('convert') || {};
+ return { formats: hardening.formatsFor(op), maxDurationSec: l.maxDurationSec || null, timeoutMs: l.timeoutMs || 15 * 60 * 1000 };
+}
+
+// Every command and probe: the whitelists and the duration cap of the operation running (a command
+// outside any context gets the widest audio list).
+hardening.harden(ffmpeg, () => jobContext.getStore() || { formats: hardening.formatsFor('extract') });
+
const proto = ffmpeg.prototype;
const proto = ffmpeg.prototype;
if (!proto.__ovJobs) {
if (!proto.__ovJobs) {
const run = proto.run;
const run = proto.run;
proto.run = function runInJob(...args) {
proto.run = function runInJob(...args) {
const job = jobContext.getStore();
const job = jobContext.getStore();
- if (job) {
+ if (job && job.signal) {
const kill = () => {
const kill = () => {
if (this.ffmpegProc) { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* already gone */ } } else this.once('start', () => { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* gone */ } });
if (this.ffmpegProc) { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* already gone */ } } else this.once('start', () => { try { this.ffmpegProc.kill('SIGKILL'); } catch { /* gone */ } });
};
};
if (job.signal.aborted) kill(); else job.signal.addEventListener('abort', kill, { once: true });
if (job.signal.aborted) kill(); else job.signal.addEventListener('abort', kill, { once: true });
- this.on('progress', (p) => { if (p && Number.isFinite(p.percent)) job.progress(10 + Math.min(100, Math.max(0, p.percent)) * 0.8, 'Encoding'); });
+ if (job.progress) this.on('progress', (p) => { if (p && Number.isFinite(p.percent)) job.progress(10 + Math.min(100, Math.max(0, p.percent)) * 0.8, 'Encoding'); });
}
}
return run.apply(this, args);
return run.apply(this, args);
};
};
@@ -55,6 +76,38 @@ function describe(toolId, result, size, inputSize) {
@@ -55,6 +76,38 @@ function describe(toolId, result, size, inputSize) {
};
};
}
}
+/** Probe the inputs (hardened) and run the tool inside the current context. */
+async function runTool(toolId, arg, options) {
+ const tool = getTool(toolId);
+ const lim = jobContext.getStore() || limitsFor(toolId);
+ await hardening.probeInputs(probe, Array.isArray(arg) ? arg : [arg], lim);
+ return tool.handler(arg, options);
+}
+
+/**
+ * The synchronous endpoint's run: the job context without a job — hardened, killed at the
+ * descriptor's timeout (504 tools.run.timeout) or when the client goes away.
+ */
+async function runSync(res, toolId, arg, options, override = {}) {
+ const lim = { ...limitsFor(toolId), ...override };
+ const ctrl = new AbortController();
+ let why = null;
+ const timer = setTimeout(() => { why = 'timeout'; ctrl.abort(); }, lim.timeoutMs);
+ const gone = () => { if (!res.writableFinished && !why) { why = 'client'; ctrl.abort(); } };
+ res.on('close', gone);
+ try {
+ return await jobContext.run({ signal: ctrl.signal, progress: () => {}, ...lim }, () => runTool(toolId, arg, options));
+ } catch (err) {
+ if (why === 'timeout') {
+ throw Object.assign(new Error(`This took longer than ${Math.round(lim.timeoutMs / 60000)} minutes and was stopped.`), { status: 504, code: 'tools.run.timeout', guardReason: 'timeout', expose: true });
+ }
+ throw err;
+ } finally {
+ clearTimeout(timer);
+ res.removeListener('close', gone);
+ }
+}
+
/** Job type audio.process: input { tool, …options } + one audio or video file (2–5 for merge). */
/** Job type audio.process: input { tool, …options } + one audio or video file (2–5 for merge). */
function defineJobs(system) {
function defineJobs(system) {
const known = new Set(listTools().map(t => t.id));
const known = new Set(listTools().map(t => t.id));
@@ -84,7 +137,7 @@ function defineJobs(system) {
@@ -84,7 +137,7 @@ function defineJobs(system) {
const src = files[0];
const src = files[0];
progress(5, files.length > 1 ? `Reading ${files.length} files` : 'Reading the audio');
progress(5, files.length > 1 ? `Reading ${files.length} files` : 'Reading the audio');
const arg = tool.multiFile ? files.map(f => f.path) : src.path;
const arg = tool.multiFile ? files.map(f => f.path) : src.path;
- const result = await jobContext.run({ signal, progress }, () => tool.handler(arg, buildOptions(input, toolId, null)));
+ const result = await jobContext.run({ signal, progress, ...limitsFor(toolId) }, () => runTool(toolId, arg, buildOptions(input, toolId, null)));
if (signal.aborted) { fs.rm(result.outputPath, { force: true }, () => {}); throw new Error('cancelled'); }
if (signal.aborted) { fs.rm(result.outputPath, { force: true }, () => {}); throw new Error('cancelled'); }
const size = fs.statSync(result.outputPath).size;
const size = fs.statSync(result.outputPath).size;
const base = path.basename(src.name || 'audio', path.extname(src.name || '')) || 'openvibeaudio-output';
const base = path.basename(src.name || 'audio', path.extname(src.name || '')) || 'openvibeaudio-output';
@@ -97,4 +150,4 @@ function defineJobs(system) {
@@ -97,4 +150,4 @@ function defineJobs(system) {
});
});
}
}
-module.exports = { buildOptions, describe, defineJobs, jobContext };
+module.exports = { buildOptions, describe, defineJobs, jobContext, runSync, runTool, limitsFor };
new file mode 100644
new file mode 100644
@@ -0,0 +1,111 @@
@@ -0,0 +1,111 @@
+'use strict';
+// ffmpeg hardening (apps/_shared/guard/ffmpeg.js through server/process.js): an HLS playlist or an
+// ffconcat list that points at a local file is refused — by the upload sniffing at the door, and by
+// ffmpeg's own protocol and format whitelists if one ever got further; inputs longer than the
+// descriptor's maxDurationSec are refused before work starts (and -t caps what is read); a
+// synchronous run is killed at its deadline.
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { EventEmitter } = require('events');
+const { execFileSync } = require('child_process');
+const { startApp } = require('../../_shared/test/spawn');
+
+try { execFileSync('ffmpeg', ['-version'], { stdio: 'ignore' }); } catch {
+ console.log('audio hardening: SKIPPED (no ffmpeg on this machine)');
+ process.exit(0);
+}
+
+(async () => {
+ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-audio-guard-'));
+ process.env.DATA_DIR = data; // the tools' temp files go here
+ const ff = (...args) => execFileSync('ffmpeg', ['-loglevel', 'error', '-y', ...args]);
+ const secret = path.join(data, 'secret.wav');
+ ff('-f', 'lavfi', '-i', 'sine=frequency=440:duration=2', secret);
+ const hls = path.join(data, 'evil.m3u8');
+ fs.writeFileSync(hls, `#EXTM3U\n#EXT-X-TARGETDURATION:10\n#EXTINF:2,\nfile://${secret}\n#EXT-X-ENDLIST\n`);
+ const concat = path.join(data, 'evil.ffconcat');
+ fs.writeFileSync(concat, "ffconcat version 1.0\nfile 'secret.wav'\n");
+ // The attack is real on this ffmpeg: unhardened, the playlist reads the local file.
+ const plain = execFileSync('ffprobe', ['-v', 'error', '-show_entries', 'format=format_name,duration', '-of', 'csv=p=0', hls]).toString();
+ assert.match(plain, /hls/, 'without the whitelists ffprobe follows the playlist');
+
+ const { jobContext, runTool, runSync, limitsFor } = require('../server/process');
+ const { getTool } = require('../server/tools');
+ const hardening = require('../../_shared/guard/ffmpeg');
+ const ffmpeg = require('fluent-ffmpeg');
+
+ // ── Every command's inputs carry the whitelists and the duration cap ──
+ const lim = limitsFor('convert');
+ assert.strictEqual(lim.maxDurationSec, 3 * 60 * 60, 'the descriptor\'s limits.maxDurationSec');
+ assert.ok(!lim.formats.includes('hls') && !lim.formats.includes('concat'));
+ assert.ok(limitsFor('extract').formats.includes('avi') && !lim.formats.includes('avi'), 'only the extractor opens video-only containers');
+ const cmd = ffmpeg(secret).noVideo().output(path.join(data, 'x.mp3'));
+ hardening.applyToCommand(cmd, lim);
+ const args = cmd._getArguments();
+ const at = args.indexOf('-i');
+ assert.deepStrictEqual(args.slice(at - 6, at), ['-protocol_whitelist', 'file,pipe', '-format_whitelist', lim.formats.join(','), '-t', '10800'], 'input options before -i');
+
+ // ── The playlist and the concat list are refused, at the probe and by ffmpeg itself ──
+ for (const evil of [hls, concat]) {
+ await assert.rejects(jobContext.run(lim, () => runTool('convert', evil, { format: 'mp3' })),
+ (e) => e.status === 415 && e.code === 'tools.file.unsupported_type' && e.guardReason === 'ffmpeg.format', `${path.basename(evil)}: refused before work starts`);
+ // Even the tool's own ffmpeg command (no probe first) cannot open it.
+ await assert.rejects(jobContext.run(lim, () => getTool('convert').handler(evil, { format: 'mp3' })), /whitelist|Invalid argument|Invalid data/i, `${path.basename(evil)}: ffmpeg refuses it`);
+ }
+ const ok = await jobContext.run(lim, () => runTool('convert', secret, { format: 'mp3' }));
+ assert.strictEqual(ok.ext, 'mp3', 'a real WAV converts as before');
+ fs.rmSync(ok.outputPath, { force: true });
+
+ // ── Longer than the descriptor allows: refused before work starts ──
+ await assert.rejects(jobContext.run({ ...lim, maxDurationSec: 1 }, () => runTool('convert', secret, { format: 'mp3' })),
+ (e) => e.status === 413 && e.guardReason === 'ffmpeg.duration' && /2 seconds long; this tool takes at most 1 seconds/.test(e.message));
+
+ // ── A synchronous run is killed at its deadline ──
+ const long = path.join(data, 'long.wav');
+ ff('-f', 'lavfi', '-i', 'sine=frequency=220:duration=600', '-ar', '44100', long);
+ const res = Object.assign(new EventEmitter(), { writableFinished: false });
+ const t0 = Date.now();
+ await assert.rejects(runSync(res, 'reverb', long, {}, { timeoutMs: 150 }), (e) => e.status === 504 && e.code === 'tools.run.timeout' && e.guardReason === 'timeout');
+ assert.ok(Date.now() - t0 < 5000, `stopped quickly (${Date.now() - t0} ms)`);
+ // A client that goes away stops it too.
+ const gone = Object.assign(new EventEmitter(), { writableFinished: false });
+ const run = runSync(gone, 'reverb', long, {});
+ setTimeout(() => gone.emit('close'), 150);
+ await assert.rejects(run, (e) => e.code !== 'tools.run.timeout');
+
+ // ── Through the app: sniffing turns the playlist away at the door ──
+ const env = { DATA_DIR: data, UPLOADS_DIR: path.join(data, 'uploads'), OUTPUT_DIR: path.join(data, 'output') };
+ const app = await startApp('audio', env);
+ try {
+ const ctx = await fetch(`${app.base}/api/context`);
+ const cookie = String(ctx.headers.get('set-cookie') || '').split(';')[0];
+ assert.match(cookie, /^ov_tools_jobs=/, 'the page\'s context call starts the session audio tools need');
+ const post = (buf, name, type, tool = 'convert') => {
+ const f = new FormData();
+ f.append('file', new Blob([buf], { type }), name);
+ f.append('tool', tool);
+ f.append('format', 'mp3');
+ return fetch(`${app.base}/api/process`, { method: 'POST', body: f, headers: { cookie } });
+ };
+ let r = await post(fs.readFileSync(hls), 'song.mp3', 'audio/mpeg');
+ assert.strictEqual(r.status, 415);
+ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');
+ r = await post(fs.readFileSync(secret), 'take.wav', 'application/octet-stream');
+ const body = await r.json();
+ assert.strictEqual(r.status, 200, JSON.stringify(body));
+ assert.strictEqual(body.output.ext, 'mp3');
+ // A job with the playlist is refused the same way.
+ const f = new FormData();
+ f.append('type', 'audio.process');
+ f.append('input', JSON.stringify({ tool: 'convert', format: 'mp3' }));
+ f.append('file', new Blob([fs.readFileSync(concat)], { type: 'audio/mpeg' }), 'list.mp3');
+ r = await fetch(`${app.base}/api/v1/jobs`, { method: 'POST', body: f, headers: { cookie } });
+ assert.strictEqual(r.status, 415);
+ } finally {
+ await app.kill();
+ fs.rmSync(data, { recursive: true, force: true });
+ }
+ console.log('audio hardening: playlist and concat refused (sniffing, probe, ffmpeg whitelists), duration cap and -t, sync deadline and client-gone kill: all checks passed');
+})().catch((err) => { console.error(err); process.exit(1); });
deleted file mode 100644
deleted file mode 100644
@@ -1,120 +0,0 @@
@@ -1,120 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Docs.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)
-// Lightweight auth — no DB, no linked_accounts.
-// Verifies the shared ov_token cookie offline against the
-// Network's public key and attaches decoded claims to req.user.
-// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise
-// fetched from the Network's /api/.well-known/jwks endpoint.
-// ═══════════════════════════════════════════════════════════════
-
-const jwt = require('jsonwebtoken');
-const fs = require('fs');
-const path = require('path');
-const config = require('./config');
-
-let publicKey = null;
-let retryTimer = null;
-
-const ISSUER = config.networkUrl;
-const JWKS_PATH = '/api/.well-known/jwks';
-const FETCH_TIMEOUT_MS = 5000;
-const RETRY_INTERVAL_MS = 60 * 1000;
-
-function loadKeyFromFile() {
- for (const p of config.publicKeyPaths) {
- try {
- const resolved = path.resolve(p);
- if (fs.existsSync(resolved)) {
- publicKey = fs.readFileSync(resolved, 'utf8');
- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);
- return true;
- }
- } catch { /* try next */ }
- }
- return false;
-}
-
-async function fetchKeyFromNetwork() {
- for (const base of [config.networkInternalUrl, config.networkUrl]) {
- try {
- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });
- if (!res.ok) continue;
- const body = await res.json();
- if (body && body.public_key) {
- publicKey = body.public_key;
- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);
- return true;
- }
- } catch { /* try next base */ }
- }
- return false;
-}
-
-function loadPublicKey() {
- if (loadKeyFromFile()) return;
- fetchKeyFromNetwork().then((loaded) => {
- if (loaded || retryTimer) return;
- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);
- retryTimer = setInterval(async () => {
- if (await fetchKeyFromNetwork()) {
- clearInterval(retryTimer);
- retryTimer = null;
- }
- }, RETRY_INTERVAL_MS);
- if (retryTimer.unref) retryTimer.unref();
- });
-}
-loadPublicKey();
-
-function extractToken(req) {
- const authHeader = req.headers.authorization;
- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);
- if (req.cookies?.ov_token) return req.cookies.ov_token;
- if (req.cookies?.token) return req.cookies.token;
- return null;
-}
-
-function verifyToken(token) {
- if (!publicKey || !token) return null;
- try {
- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });
- } catch {
- return null;
- }
-}
-
-/**
- * Optional auth — attaches req.user if valid token present.
- * Never blocks the request; anonymous users get req.user = null.
- */
-function optionalAuth(req, _res, next) {
- const token = extractToken(req);
- if (token) {
- const decoded = verifyToken(token);
- if (decoded) {
- req.user = decoded;
- req.token = token;
- }
- }
- next();
-}
-
-/**
- * Required auth — 401 if no valid token.
- */
-function requireAuth(req, res, next) {
- const token = extractToken(req);
- if (!token) return res.status(401).json({ error: 'Authentication required' });
- const decoded = verifyToken(token);
- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });
- req.user = decoded;
- req.token = token;
- next();
-}
-
-/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */
-function getPublicKey() { return publicKey; }
-
-module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };
@@ -13,19 +13,19 @@ const cookieParser = require('cookie-parser');
@@ -13,19 +13,19 @@ const cookieParser = require('cookie-parser');
const path = require('path');
const path = require('path');
const fs = require('fs');
const fs = require('fs');
+const fsp = require('fs/promises');
+const rateLimit = require('express-rate-limit');
const config = require('./config');
const config = require('./config');
-const auth = require('./auth');
-const { optionalAuth } = auth;
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { getTool, listTools, BINARIES } = require('./tools');
const { getTool, listTools, BINARIES } = require('./tools');
const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');
const { uploadSingle, uploadMultiple, uploadAny } = require('./middleware/upload');
-const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');
const retention = require('./retention/manager');
const retention = require('./retention/manager');
const { buildOptions, defineJobs } = require('./process');
const { buildOptions, defineJobs } = require('./process');
const { hostGuard, ownHost } = require('../../_shared/host-role');
const { hostGuard, ownHost } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const jobsRuntime = require('../../_shared/jobs');
const jobsRuntime = require('../../_shared/jobs');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const sdk = require('openvibe-sdk');
const sdk = require('openvibe-sdk');
@@ -39,13 +39,33 @@ const analyticsDb = new Database(analyticsDbPath);
@@ -39,13 +39,33 @@ const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-docs', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-docs', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
+// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
+// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse. Every PDF
+// tool needs a browser session, a sign-in or a token (descriptor auth.anonymous false).
+const SPECS = require('./descriptors').SPECS;
+const guard = createGuard({
+ app: 'docs', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,
+ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,
+});
+const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);
+/** The tool a request is for before its body is read: the host's own (the hubs: merge, the first). */
+const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('docs.process', req.ctx.defaultOp || 'merge', null) || { id: null }).id);
+/** The tool a parsed request runs: its `tool` (operation) on this host. */
+const toolOf = (req) => { const op = String((req.body && req.body.tool) || req.ctx.defaultOp || ''); const d = op ? guard.toolForJob('docs.process', op, req.ctx.toolId) : null; return d ? d.id : hostTool(req); };
+/** Read the uploaded files (on disk) and delete them. */
+async function readUploads(files) {
+ try { return await Promise.all(files.map(f => fsp.readFile(f.path))); } finally { for (const f of files) fs.unlink(f.path, () => {}); }
+}
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready } = require('../../_shared/observe');
const { observe, checks: ready } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-docs', release: release.release,
service: 'tools-docs', release: release.release,
checks: [
checks: [
@@ -55,19 +75,20 @@ observe({
@@ -55,19 +75,20 @@ observe({
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
+ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
...BINARIES.map(b => b.readyCheck(b.name === 'qpdf' ? 'Protect and Unlock PDF (package qpdf); the other tools work without it' : 'PDF to image (package poppler-utils); the other tools work without it')),
...BINARIES.map(b => b.readyCheck(b.name === 'qpdf' ? 'Protect and Unlock PDF (package qpdf); the other tools work without it' : 'PDF to image (package poppler-utils); the other tools work without it')),
...jobsRuntime.readyChecks(() => jobs),
...jobsRuntime.readyChecks(() => jobs),
],
],
jobs: () => jobs,
jobs: () => jobs,
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
-app.set('trust proxy', 2); // Cloudflare → Nginx → Node
+app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
directives: {
directives: {
@@ -98,8 +119,9 @@ app.use(exceptRegistry(cors({
@@ -98,8 +119,9 @@ app.use(exceptRegistry(cors({
credentials: true,
credentials: true,
})));
})));
-// ── Rate Limiting ────────────────────────────────────────────
-app.use('/api/', apiLimiter);
+// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──
+app.use(guard.identify);
+app.use('/api/', apiLimiter, guard.apiQuota);
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's PDF tools ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's PDF tools ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -111,9 +133,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
@@ -111,9 +133,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
// ── Analytics Middleware ─────────────────────────────────────
// ── Analytics Middleware ─────────────────────────────────────
app.use(analytics.middleware());
app.use(analytics.middleware());
-// ── Auth (optional on all routes) ────────────────────────────
-app.use(optionalAuth);
-
// ── Hosts ────────────────────────────────────────────────────
// ── Hosts ────────────────────────────────────────────────────
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
@@ -131,6 +150,7 @@ app.use((req, _res, next) => {
@@ -131,6 +150,7 @@ app.use((req, _res, next) => {
/** A tool's error on the synchronous endpoints: 503 problem when a tool is not set up, else { error }. */
/** A tool's error on the synchronous endpoints: 503 problem when a tool is not set up, else { error }. */
function sendToolError(req, res, err, label) {
function sendToolError(req, res, err, label) {
+ if (guard.toolRefused(req, res, err, toolOf(req))) return undefined;
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code || 'tools.unavailable', { detail: err.message, ctx: req.ov });
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code || 'tools.unavailable', { detail: err.message, ctx: req.ov });
if (!err.expose) console.error(`[${label}] Error:`, err.message);
if (!err.expose) console.error(`[${label}] Error:`, err.message);
const status = Number.isInteger(err.status) && err.status >= 400 && err.status < 500 ? err.status : 422;
const status = Number.isInteger(err.status) && err.status >= 400 && err.status < 500 ? err.status : 422;
@@ -143,8 +163,11 @@ app.get('/api/health', (_req, res) => {
@@ -143,8 +163,11 @@ app.get('/api/health', (_req, res) => {
res.json({ status: 'ok', service: 'openvibe-docs', version: '1.0.0', files: stats, jobs: jobs.stats() });
res.json({ status: 'ok', service: 'openvibe-docs', version: '1.0.0', files: stats, jobs: jobs.stats() });
});
});
-// Domain context (frontend calls this on load to get branding)
+// Domain context (frontend calls this on load to get branding). It also starts this browser's session
+// (ov_tools_jobs): PDF tools need one (or a sign-in, or a token) before the first upload.
app.get('/api/context', (req, res) => {
app.get('/api/context', (req, res) => {
+ guard.ensureSession(req, res);
+ res.set('Cache-Control', 'private, no-store');
const ctx = req.ctx;
const ctx = req.ctx;
const tools = listTools();
const tools = listTools();
res.json({
res.json({
@@ -166,10 +189,11 @@ app.get('/api/tools', (_req, res) => {
@@ -166,10 +189,11 @@ app.get('/api/tools', (_req, res) => {
});
});
// ── PDF Info (no file mutation) ──────────────────────────────
// ── PDF Info (no file mutation) ──────────────────────────────
-app.post('/api/info', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/info', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(hostTool), guard.heavy(hostTool), async (req, res) => {
try {
try {
const tool = getTool('metadata');
const tool = getTool('metadata');
- const result = await tool.handler(req.file.buffer, { mode: 'view' });
+ const [buffer] = await readUploads([req.file]);
+ const result = await tool.handler(buffer, { mode: 'view' });
res.json({ success: true, ...result });
res.json({ success: true, ...result });
} catch (err) {
} catch (err) {
sendToolError(req, res, err, 'Info');
sendToolError(req, res, err, 'Info');
@@ -177,16 +201,17 @@ app.post('/api/info', burstLimiter, processLimiter, uploadSingle, async (req, re
@@ -177,16 +201,17 @@ app.post('/api/info', burstLimiter, processLimiter, uploadSingle, async (req, re
});
});
// ── Main Processing Endpoint (single file) ───────────────────
// ── Main Processing Endpoint (single file) ───────────────────
-app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp;
const toolId = req.body.tool || req.ctx.defaultOp;
- if (!toolId) return res.status(400).json({ error: 'No tool specified.' });
+ if (!toolId) { fs.unlink(req.file.path, () => {}); return res.status(400).json({ error: 'No tool specified.' }); }
const tool = getTool(toolId);
const tool = getTool(toolId);
- if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });
+ if (!tool) { fs.unlink(req.file.path, () => {}); return res.status(400).json({ error: `Unknown tool: ${toolId}` }); }
// Same options and code path as the docs.process job
// Same options and code path as the docs.process job
- const result = await tool.handler(req.file.buffer, buildOptions(req.body, req.ctx, false));
+ const [buffer] = await readUploads([req.file]);
+ const result = await tool.handler(buffer, buildOptions(req.body, req.ctx, false));
// Some tools return view-only results (like metadata view)
// Some tools return view-only results (like metadata view)
if (result.viewOnly) {
if (result.viewOnly) {
@@ -223,16 +248,18 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
@@ -223,16 +248,18 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
});
});
// ── Multi-File Processing Endpoint (merge, img2pdf) ──────────
// ── Multi-File Processing Endpoint (merge, img2pdf) ──────────
-app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, async (req, res) => {
+app.post('/api/process/multi', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadMultiple, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp;
const toolId = req.body.tool || req.ctx.defaultOp;
- if (!toolId) return res.status(400).json({ error: 'No tool specified.' });
-
- const tool = getTool(toolId);
- if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });
- if (!tool.multiFile) return res.status(400).json({ error: `Tool "${toolId}" does not support multiple files. Use /api/process instead.` });
+ const tool = toolId ? getTool(toolId) : null;
+ if (!tool || !tool.multiFile) {
+ for (const f of req.files) fs.unlink(f.path, () => {});
+ if (!toolId) return res.status(400).json({ error: 'No tool specified.' });
+ if (!tool) return res.status(400).json({ error: `Unknown tool: ${toolId}` });
+ return res.status(400).json({ error: `Tool "${toolId}" does not support multiple files. Use /api/process instead.` });
+ }
- const buffers = req.files.map(f => f.buffer);
+ const buffers = await readUploads(req.files);
const result = await tool.handler(buffers, buildOptions(req.body, req.ctx, true));
const result = await tool.handler(buffers, buildOptions(req.body, req.ctx, true));
// Save output
// Save output
@@ -268,10 +295,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
@@ -268,10 +295,11 @@ app.post('/api/process/multi', burstLimiter, processLimiter, uploadMultiple, asy
// into data/jobs.db, followed over SSE, reattachable by id after a reload or a restart.
// into data/jobs.db, followed over SSE, reattachable by id after a reload or a restart.
const jobs = jobsRuntime.setupJobs({
const jobs = jobsRuntime.setupJobs({
app, service: 'docs', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
app, service: 'docs', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,
+ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,
define: defineJobs,
define: defineJobs,
receive: uploadAny,
receive: uploadAny,
- limiters: [burstLimiter, processLimiter],
+ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],
+ jobTool: (req, type, input) => { const op = String((input && input.tool) || req.ctx.defaultOp || ''); const d = op ? guard.toolForJob('docs.process', op, req.ctx.toolId) : null; return d ? d.id : null; },
defaults(req, input) {
defaults(req, input) {
const out = { ...input };
const out = { ...input };
if (!out.tool && req.ctx.defaultOp) out.tool = req.ctx.defaultOp;
if (!out.tool && req.ctx.defaultOp) out.tool = req.ctx.defaultOp;
@@ -354,6 +382,7 @@ function shutdown() {
@@ -354,6 +382,7 @@ function shutdown() {
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,48 +0,0 @@
@@ -1,48 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Docs.OpenVibe — Rate Limiting Middleware
-// Tiered rate limits: anonymous vs authenticated users.
-// Includes anti-abuse burst protection.
-// ═══════════════════════════════════════════════════════════════
-
-const rateLimit = require('express-rate-limit');
-
-/** General API rate limit */
-const apiLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 120 : 60,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Too many requests. Please try again later.' },
-});
-
-/**
- * Processing rate limit — stricter, tiered by auth.
- * Anonymous: 10 conversions/minute
- * Authenticated: 30 conversions/minute
- */
-const processLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 30 : 10,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },
-});
-
-/**
- * Anti-burst limiter — prevents rapid-fire abuse.
- * Very short window: max 4 requests per 5 seconds per IP.
- */
-const burstLimiter = rateLimit({
- windowMs: 5 * 1000,
- max: 4,
- standardHeaders: false,
- legacyHeaders: false,
- keyGenerator: (req) => req.ip,
- message: { error: 'Too many requests in quick succession. Please slow down.' },
-});
-
-module.exports = { apiLimiter, processLimiter, burstLimiter };
@@ -2,16 +2,30 @@
@@ -2,16 +2,30 @@
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
// Docs.OpenVibe — Multer Upload Middleware
// Docs.OpenVibe — Multer Upload Middleware
-// Memory storage with file size + mime validation.
-// Supports multiple file uploads for merge operations.
+// Disk storage (up to 50 files of 100 MB for a merge must never sit in memory while they arrive) with
+// size + declared-type validation; the guard then checks the bytes (apps/_shared/guard/sniff.js).
+// Files land in config.uploadsDir with a random name; the routes read them and delete them, and the
+// retention sweep removes anything older than retention.tempMaxAge. Job submits move them into the job.
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
const multer = require('multer');
const multer = require('multer');
+const path = require('path');
+const fs = require('fs');
+const crypto = require('crypto');
const config = require('../config');
const config = require('../config');
const ALLOWED_MIMES = new Set(config.upload.allowedMimes);
const ALLOWED_MIMES = new Set(config.upload.allowedMimes);
-const storage = multer.memoryStorage();
+const uploadsDir = path.resolve(config.uploadsDir);
+fs.mkdirSync(uploadsDir, { recursive: true });
+
+const storage = multer.diskStorage({
+ destination: (_req, _file, cb) => cb(null, uploadsDir),
+ filename: (_req, file, cb) => {
+ const ext = (path.extname(file.originalname || '').toLowerCase().match(/^\.[a-z0-9]{1,8}$/) || ['.bin'])[0];
+ cb(null, `${crypto.randomBytes(16).toString('hex')}${ext}`);
+ },
+});
const upload = multer({
const upload = multer({
storage,
storage,
@@ -30,7 +44,7 @@ const upload = multer({
@@ -30,7 +44,7 @@ const upload = multer({
/**
/**
* Single file upload on 'file' field.
* Single file upload on 'file' field.
- * Attaches req.file with { buffer, originalname, mimetype, size }.
+ * Attaches req.file with { path, originalname, mimetype, size }.
*/
*/
function uploadSingle(req, res, next) {
function uploadSingle(req, res, next) {
upload.single('file')(req, res, (err) => {
upload.single('file')(req, res, (err) => {
@@ -49,7 +63,7 @@ function uploadSingle(req, res, next) {
@@ -49,7 +63,7 @@ function uploadSingle(req, res, next) {
/**
/**
* Multiple file upload on 'files' field (for merge, img2pdf, etc.).
* Multiple file upload on 'files' field (for merge, img2pdf, etc.).
- * Attaches req.files as an array.
+ * Attaches req.files as an array of { path, originalname, mimetype, size }.
*/
*/
function uploadMultiple(req, res, next) {
function uploadMultiple(req, res, next) {
upload.array('files', 50)(req, res, (err) => {
upload.array('files', 50)(req, res, (err) => {
@@ -8,6 +8,16 @@
@@ -8,6 +8,16 @@
const { PDFDocument } = require('pdf-lib');
const { PDFDocument } = require('pdf-lib');
const sharp = require('sharp');
const sharp = require('sharp');
+const guardLimits = require('../../../_shared/guard/limits');
+
+/** No picture larger than the guard's pixel limit (40 MP, TOOLS_MAX_INPUT_PIXELS) is decoded. */
+function checkPixels(metadata, i, count) {
+ const limit = guardLimits.bounds().maxInputPixels;
+ const w = metadata.width || 0, h = metadata.pageHeight || metadata.height || 0;
+ if (w * h <= limit) return;
+ const mp = (n) => `${Math.round(n / 1e5) / 10} megapixels`;
+ throw Object.assign(new Error(`${count > 1 ? `Image ${i + 1}` : 'The image'} is ${w}×${h} (${mp(w * h)}); the limit is ${mp(limit)}.`), { status: 413, code: 'tools.file.too_large', guardReason: 'pixels', expose: true });
+}
/**
/**
* Convert images to PDF.
* Convert images to PDF.
@@ -30,19 +40,21 @@ async function img2pdf(buffers, options = {}) {
@@ -30,19 +40,21 @@ async function img2pdf(buffers, options = {}) {
legal: { width: 612, height: 1008 },
legal: { width: 612, height: 1008 },
};
};
- for (const imgBuf of buffers) {
+ const limitInputPixels = guardLimits.bounds().maxInputPixels;
+ for (const [i, imgBuf] of buffers.entries()) {
// Convert to PNG or JPG using sharp for consistency
// Convert to PNG or JPG using sharp for consistency
- const metadata = await sharp(imgBuf).metadata();
+ const metadata = await sharp(imgBuf, { limitInputPixels: false }).metadata(); // the header only; checked next
+ checkPixels(metadata, i, buffers.length);
const imgWidth = metadata.width || 800;
const imgWidth = metadata.width || 800;
const imgHeight = metadata.height || 600;
const imgHeight = metadata.height || 600;
// Determine if we should embed as PNG or JPG
// Determine if we should embed as PNG or JPG
let embeddedImage;
let embeddedImage;
if (metadata.format === 'jpeg' || metadata.format === 'jpg') {
if (metadata.format === 'jpeg' || metadata.format === 'jpg') {
- const jpgBuf = await sharp(imgBuf).jpeg({ quality: 90 }).toBuffer();
+ const jpgBuf = await sharp(imgBuf, { limitInputPixels }).jpeg({ quality: 90 }).toBuffer();
embeddedImage = await pdf.embedJpg(jpgBuf);
embeddedImage = await pdf.embedJpg(jpgBuf);
} else {
} else {
- const pngBuf = await sharp(imgBuf).png().toBuffer();
+ const pngBuf = await sharp(imgBuf, { limitInputPixels }).png().toBuffer();
embeddedImage = await pdf.embedPng(pngBuf);
embeddedImage = await pdf.embedPng(pngBuf);
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,73 @@
@@ -0,0 +1,73 @@
+'use strict';
+// The guard on the PDF satellite: uploads go to disk (never 50 × 100 MB in memory) and are deleted
+// after the synchronous call, the bytes are checked against the tool's accept list (a picture called
+// .pdf is refused, and Image to PDF refuses a PDF), the page's context call starts the session PDF
+// tools need, and a caller without one is recorded in report mode (TOOLS_GUARD=enforce refuses it).
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { PDFDocument } = require('pdf-lib');
+const sharp = require('sharp');
+const { startApp } = require('../../_shared/test/spawn');
+
+async function makePdf(pages) {
+ const doc = await PDFDocument.create();
+ for (let i = 0; i < pages; i++) doc.addPage([200, 300]).drawText(`page ${i + 1}`, { x: 20, y: 150 });
+ return Buffer.from(await doc.save());
+}
+
+(async () => {
+ const pdf = await makePdf(2);
+ const png = await sharp({ create: { width: 20, height: 20, channels: 3, background: '#aa3355' } }).png().toBuffer();
+ for (const mode of ['report', 'enforce']) {
+ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-docs-guard-'));
+ const uploads = path.join(data, 'uploads');
+ const app = await startApp('docs', { DATA_DIR: data, UPLOADS_DIR: uploads, OUTPUT_DIR: path.join(data, 'output'), TOOLS_GUARD: mode });
+ // Each call from its own address, so the older burst limiter (4 in 5 s per address) stays out of it.
+ let n = 0;
+ const post = (files, fields, cookie, p = '/api/process') => {
+ const f = new FormData();
+ for (const [k, v] of Object.entries(fields)) f.append(k, v);
+ for (const [buf, name, type, field = 'file'] of files) f.append(field, new Blob([buf], { type }), name);
+ return fetch(`${app.base}${p}`, { method: 'POST', body: f, headers: { 'X-Forwarded-For': `198.51.100.${++n}`, ...(cookie ? { cookie } : {}) } });
+ };
+ try {
+ const ctx = await fetch(`${app.base}/api/context`, { headers: { Host: 'compresspdf.openvibe.tools' } });
+ const cookie = String(ctx.headers.get('set-cookie') || '').split(';')[0];
+ assert.match(cookie, /^ov_tools_jobs=/);
+
+ let r = await post([[pdf, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, cookie);
+ let body = await r.json();
+ assert.strictEqual(r.status, 200, JSON.stringify(body));
+ assert.strictEqual(body.output.ext, 'pdf');
+ assert.deepStrictEqual(fs.readdirSync(uploads), [], 'the upload was on disk and is gone after the call');
+
+ r = await post([[png, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, cookie);
+ assert.strictEqual(r.status, 415, 'a PNG called doc.pdf is refused (in both modes)');
+ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');
+ r = await post([[png, 'a.png', 'image/png', 'files'], [pdf, 'b.png', 'image/png', 'files']], { tool: 'img2pdf' }, cookie, '/api/process/multi');
+ assert.strictEqual(r.status, 415, 'Image to PDF takes pictures only: file 2 is a PDF');
+ assert.match((await r.json()).detail, /File 2 is a PDF file/);
+ r = await post([[png, 'a.png', 'image/png', 'files'], [png, 'b.png', 'image/png', 'files']], { tool: 'img2pdf' }, cookie, '/api/process/multi');
+ assert.strictEqual(r.status, 200);
+ assert.deepStrictEqual(fs.readdirSync(uploads), []);
+
+ // No session, no sign-in, no token: PDF tools (auth.anonymous false) record or refuse it.
+ r = await post([[pdf, 'doc.pdf', 'application/pdf']], { tool: 'compress' }, null);
+ if (mode === 'report') assert.strictEqual(r.status, 200, 'report mode lets it through');
+ else {
+ assert.strictEqual(r.status, 401);
+ assert.strictEqual((await r.json()).code, 'tools.session_required');
+ }
+ const m = await (await fetch(`${app.base}/metrics`)).text();
+ assert.match(m, /tools_guard_refused_total\{reason="session",tool="compresspdf"\} 1/);
+ assert.match(m, /tools_guard_refused_total\{reason="sniff",tool="compresspdf"\} 1/);
+ assert.match(m, /tools_guard_refused_total\{reason="sniff",tool="image2pdf"\} 1/);
+ } finally {
+ await app.kill();
+ fs.rmSync(data, { recursive: true, force: true });
+ }
+ }
+ console.log('docs guard: disk uploads cleaned up, sniffing per tool (PDF vs pictures), context session, session rule report/enforce: all checks passed');
+})().catch((err) => { console.error(err); process.exit(1); });
@@ -19,12 +19,24 @@ const { internalOk } = require('../../_shared/internal-auth');
@@ -19,12 +19,24 @@ const { internalOk } = require('../../_shared/internal-auth');
const { hostGuard, stampedPage } = require('../../_shared/host-role');
const { hostGuard, stampedPage } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');
const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
const analyticsDb = new Database(analyticsDbPath);
const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-food', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-food', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota and
+// the abuse log (data/guard.db). The data lookups are counted by maps (tools-map), which gets the
+// visitor's address from here. TOOLS_GUARD=report (default) records what it would refuse.
+const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
+const guard = createGuard({
+ app: 'food', dataDir: path.join(__dirname, '..', 'data'), Database, specs: require('../../maps/server/descriptors').SPECS.filter(s => s.id === 'food'),
+ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',
+ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),
+});
+
const PORT = parseInt(process.env.PORT) || 4011;
const PORT = parseInt(process.env.PORT) || 4011;
const MAPS_API = process.env.MAPS_API || 'http://127.0.0.1:4010';
const MAPS_API = process.env.MAPS_API || 'http://127.0.0.1:4010';
@@ -34,7 +46,7 @@ const release = require('openvibe-shared/release').createRelease({ service: 'too
@@ -34,7 +46,7 @@ const release = require('openvibe-shared/release').createRelease({ service: 'too
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready } = require('../../_shared/observe');
const { observe, checks: ready } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-food', release: release.release,
service: 'tools-food', release: release.release,
checks: [
checks: [
@@ -43,14 +55,15 @@ observe({
@@ -43,14 +55,15 @@ observe({
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only' }),
],
],
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
-// Only loopback hops (the host's nginx, the gateway) are proxies: req.ip is the first address before
-// them, which is what maps is told below. A client's own X-Forwarded-For is never believed.
-app.set('trust proxy', 'loopback');
+// One hop on loopback (the host's nginx, or the gateway) is the proxy: req.ip is the address it gives,
+// which is what maps is told below. A client's own X-Forwarded-For is never believed.
+app.set('trust proxy', TRUST_PROXY);
app.use(exceptRegistry(cors({
app.use(exceptRegistry(cors({
origin(origin, callback) {
origin(origin, callback) {
if (!origin) return callback(null, true);
if (!origin) return callback(null, true);
@@ -74,7 +87,11 @@ app.use(helmet({
@@ -74,7 +87,11 @@ app.use(helmet({
},
},
},
},
}));
}));
-app.use(rateLimit({ windowMs: 60000, max: 60 }));
+// Sign-in first, so limits know who is asking; everyone else counts by address (IPv6 /64).
+app.use(cookieParser());
+app.use(guard.identify);
+app.use(guard.legacyLimiter(rateLimit, { windowMs: 60000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }));
+app.use('/api/', guard.apiQuota);
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's food finder ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's food finder ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -82,7 +99,6 @@ app.use(rateLimit({ windowMs: 60000, max: 60 }));
@@ -82,7 +99,6 @@ app.use(rateLimit({ windowMs: 60000, max: 60 }));
// The food finder is built on the maps backend; its descriptor lives with the maps app's.
// The food finder is built on the maps backend; its descriptor lives with the maps app's.
const toolRegistry = createLocalRegistry({ specs: require('../../maps/server/descriptors').SPECS.filter(s => s.id === 'food') });
const toolRegistry = createLocalRegistry({ specs: require('../../maps/server/descriptors').SPECS.filter(s => s.id === 'food') });
app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
-app.use(cookieParser());
// ── Analytics Middleware ─────────────────────────────────────
// ── Analytics Middleware ─────────────────────────────────────
app.use(analytics.middleware());
app.use(analytics.middleware());
@@ -145,6 +161,7 @@ function shutdown() {
@@ -145,6 +161,7 @@ function shutdown() {
console.log('[Food.OpenVibe] Shutting down...');
console.log('[Food.OpenVibe] Shutting down...');
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
'use strict';
'use strict';
// Food proxies every API call to maps. It used to call maps without the visitor's address, so maps'
// Food proxies every API call to maps. It used to call maps without the visitor's address, so maps'
// per-IP limit (30/min) was one bucket shared by every food visitor (127.0.0.1). Now food forwards
// per-IP limit (30/min) was one bucket shared by every food visitor (127.0.0.1). Now food forwards
-// X-Forwarded-For / X-Real-IP from req.ip, and maps believes X-Forwarded-For only from loopback hops.
+// X-Forwarded-For / X-Real-IP from req.ip, and maps believes X-Forwarded-For from one loopback hop only.
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
@@ -21,7 +21,8 @@ const { startApp } = require('../../_shared/test/spawn');
@@ -21,7 +21,8 @@ const { startApp } = require('../../_shared/test/spawn');
assert.strictEqual(other.status, 200, 'a second visitor has their own bucket');
assert.strictEqual(other.status, 200, 'a second visitor has their own bucket');
const src = (app) => fs.readFileSync(path.join(__dirname, '..', '..', app, 'server', 'index.js'), 'utf8');
const src = (app) => fs.readFileSync(path.join(__dirname, '..', '..', app, 'server', 'index.js'), 'utf8');
- assert.match(src('maps'), /app\.set\('trust proxy', 'loopback'\)/, 'maps trusts forwarded addresses from loopback only');
+ assert.match(src('maps'), /app\.set\('trust proxy', TRUST_PROXY\)/, 'maps trusts one forwarded hop, on loopback only');
+ assert.match(src('food'), /app\.set\('trust proxy', TRUST_PROXY\)/);
assert.match(src('food'), /'X-Forwarded-For': req\.ip/);
assert.match(src('food'), /'X-Forwarded-For': req\.ip/);
} finally {
} finally {
await food.kill();
await food.kill();
@@ -9,6 +9,7 @@
@@ -9,6 +9,7 @@
"version": "2.0.0",
"version": "2.0.0",
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
+ "better-sqlite3": "^11.10.0",
"cookie-parser": "^1.4.7",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"cors": "^2.8.5",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
@@ -72,6 +73,57 @@
@@ -72,6 +73,57 @@
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
"integrity": "sha512-PCVAQswWemu6UdxsDFFX/+gVeYqKAod3D3UVm91jHwynguOwAvYPhx8nNlM++NqRcK6CxxpUafjmhIdKiHibqg==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/base64-js": {
+ "version": "1.5.1",
+ "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
+ "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/feross"
+ },
+ {
+ "type": "patreon",
+ "url": "https://www.patreon.com/feross"
+ },
+ {
+ "type": "consulting",
+ "url": "https://feross.org/support"
+ }
+ ],
+ "license": "MIT"
+ },
+ "node_modules/better-sqlite3": {
+ "version": "11.10.0",
+ "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.10.0.tgz",
+ "integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",
+ "hasInstallScript": true,
+ "license": "MIT",
+ "dependencies": {
+ "bindings": "^1.5.0",
+ "prebuild-install": "^7.1.1"
+ }
+ },
+ "node_modules/bindings": {
+ "version": "1.5.0",
+ "resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
+ "integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
+ "license": "MIT",
+ "dependencies": {
+ "file-uri-to-path": "1.0.0"
+ }
+ },
+ "node_modules/bl": {
+ "version": "4.1.0",
+ "resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
+ "integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
+ "license": "MIT",
+ "dependencies": {
+ "buffer": "^5.5.0",
+ "inherits": "^2.0.4",
+ "readable-stream": "^3.4.0"
+ }
+ },
"node_modules/body-parser": {
"node_modules/body-parser": {
"version": "1.20.6",
"version": "1.20.6",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.6.tgz",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.6.tgz",
@@ -96,6 +148,30 @@
@@ -96,6 +148,30 @@
"npm": "1.2.8000 || >= 1.4.16"
"npm": "1.2.8000 || >= 1.4.16"
}
}
},
},
+ "node_modules/buffer": {
+ "version": "5.7.1",
+ "resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
+ "integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/feross"
+ },
+ {
+ "type": "patreon",
+ "url": "https://www.patreon.com/feross"
+ },
+ {
+ "type": "consulting",
+ "url": "https://feross.org/support"
+ }
+ ],
+ "license": "MIT",
+ "dependencies": {
+ "base64-js": "^1.3.1",
+ "ieee754": "^1.1.13"
+ }
+ },
"node_modules/buffer-equal-constant-time": {
"node_modules/buffer-equal-constant-time": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
@@ -140,6 +216,12 @@
@@ -140,6 +216,12 @@
"url": "https://github.com/sponsors/ljharb"
"url": "https://github.com/sponsors/ljharb"
}
}
},
},
+ "node_modules/chownr": {
+ "version": "1.1.4",
+ "resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
+ "integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
+ "license": "ISC"
+ },
"node_modules/content-disposition": {
"node_modules/content-disposition": {
"version": "0.5.4",
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@ -215,6 +297,30 @@
@@ -215,6 +297,30 @@
"ms": "2.0.0"
"ms": "2.0.0"
}
}
},
},
+ "node_modules/decompress-response": {
+ "version": "6.0.0",
+ "resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
+ "integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
+ "license": "MIT",
+ "dependencies": {
+ "mimic-response": "^3.1.0"
+ },
+ "engines": {
+ "node": ">=10"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/sindresorhus"
+ }
+ },
+ "node_modules/deep-extend": {
+ "version": "0.6.0",
+ "resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
+ "integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
"node_modules/depd": {
"node_modules/depd": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -234,6 +340,15 @@
@@ -234,6 +340,15 @@
"npm": "1.2.8000 || >= 1.4.16"
"npm": "1.2.8000 || >= 1.4.16"
}
}
},
},
+ "node_modules/detect-libc": {
+ "version": "2.1.2",
+ "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
+ "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=8"
+ }
+ },
"node_modules/dotenv": {
"node_modules/dotenv": {
"version": "16.6.1",
"version": "16.6.1",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-16.6.1.tgz",
@@ -284,6 +399,15 @@
@@ -284,6 +399,15 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
+ "node_modules/end-of-stream": {
+ "version": "1.4.5",
+ "resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
+ "integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
+ "license": "MIT",
+ "dependencies": {
+ "once": "^1.4.0"
+ }
+ },
"node_modules/es-define-property": {
"node_modules/es-define-property": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
@@ -329,6 +453,15 @@
@@ -329,6 +453,15 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
+ "node_modules/expand-template": {
+ "version": "2.0.3",
+ "resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
+ "integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
+ "license": "(MIT OR WTFPL)",
+ "engines": {
+ "node": ">=6"
+ }
+ },
"node_modules/express": {
"node_modules/express": {
"version": "4.22.2",
"version": "4.22.2",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz",
"resolved": "https://registry.npmjs.org/express/-/express-4.22.2.tgz",
@@ -412,6 +545,12 @@
@@ -412,6 +545,12 @@
],
],
"license": "BSD-3-Clause"
"license": "BSD-3-Clause"
},
},
+ "node_modules/file-uri-to-path": {
+ "version": "1.0.0",
+ "resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
+ "integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
+ "license": "MIT"
+ },
"node_modules/finalhandler": {
"node_modules/finalhandler": {
"version": "1.3.2",
"version": "1.3.2",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
"resolved": "https://registry.npmjs.org/finalhandler/-/finalhandler-1.3.2.tgz",
@@ -448,6 +587,12 @@
@@ -448,6 +587,12 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
+ "node_modules/fs-constants": {
+ "version": "1.0.0",
+ "resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
+ "integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
+ "license": "MIT"
+ },
"node_modules/function-bind": {
"node_modules/function-bind": {
"version": "1.1.2",
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
@@ -494,6 +639,12 @@
@@ -494,6 +639,12 @@
"node": ">= 0.4"
"node": ">= 0.4"
}
}
},
},
+ "node_modules/github-from-package": {
+ "version": "0.0.0",
+ "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
+ "integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
+ "license": "MIT"
+ },
"node_modules/gopd": {
"node_modules/gopd": {
"version": "1.2.0",
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
@@ -574,12 +725,38 @@
@@ -574,12 +725,38 @@
"node": ">=0.10.0"
"node": ">=0.10.0"
}
}
},
},
+ "node_modules/ieee754": {
+ "version": "1.2.1",
+ "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
+ "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/feross"
+ },
+ {
+ "type": "patreon",
+ "url": "https://www.patreon.com/feross"
+ },
+ {
+ "type": "consulting",
+ "url": "https://feross.org/support"
+ }
+ ],
+ "license": "BSD-3-Clause"
+ },
"node_modules/inherits": {
"node_modules/inherits": {
"version": "2.0.4",
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/ini": {
+ "version": "1.3.8",
+ "resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
+ "integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
+ "license": "ISC"
+ },
"node_modules/ipaddr.js": {
"node_modules/ipaddr.js": {
"version": "1.9.1",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -755,12 +932,45 @@
@@ -755,12 +932,45 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
+ "node_modules/mimic-response": {
+ "version": "3.1.0",
+ "resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
+ "integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=10"
+ },
+ "funding": {
+ "url": "https://github.com/sponsors/sindresorhus"
+ }
+ },
+ "node_modules/minimist": {
+ "version": "1.2.8",
+ "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
+ "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
+ "license": "MIT",
+ "funding": {
+ "url": "https://github.com/sponsors/ljharb"
+ }
+ },
+ "node_modules/mkdirp-classic": {
+ "version": "0.5.3",
+ "resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
+ "integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
+ "license": "MIT"
+ },
"node_modules/ms": {
"node_modules/ms": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/napi-build-utils": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
+ "integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
+ "license": "MIT"
+ },
"node_modules/negotiator": {
"node_modules/negotiator": {
"version": "0.6.3",
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
"resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.3.tgz",
@@ -770,6 +980,18 @@
@@ -770,6 +980,18 @@
"node": ">= 0.6"
"node": ">= 0.6"
}
}
},
},
+ "node_modules/node-abi": {
+ "version": "3.96.0",
+ "resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",
+ "integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",
+ "license": "MIT",
+ "dependencies": {
+ "semver": "^7.3.5"
+ },
+ "engines": {
+ "node": ">=10"
+ }
+ },
"node_modules/object-assign": {
"node_modules/object-assign": {
"version": "4.1.1",
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
@@ -803,6 +1025,15 @@
@@ -803,6 +1025,15 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
+ "node_modules/once": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
+ "integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
+ "license": "ISC",
+ "dependencies": {
+ "wrappy": "1"
+ }
+ },
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
"version": "0.33.0",
"version": "0.33.0",
"resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.33.0",
"resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.33.0",
@@ -843,6 +1074,33 @@
@@ -843,6 +1074,33 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/prebuild-install": {
+ "version": "7.1.3",
+ "resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
+ "integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
+ "deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
+ "license": "MIT",
+ "dependencies": {
+ "detect-libc": "^2.0.0",
+ "expand-template": "^2.0.3",
+ "github-from-package": "0.0.0",
+ "minimist": "^1.2.3",
+ "mkdirp-classic": "^0.5.3",
+ "napi-build-utils": "^2.0.0",
+ "node-abi": "^3.3.0",
+ "pump": "^3.0.0",
+ "rc": "^1.2.7",
+ "simple-get": "^4.0.0",
+ "tar-fs": "^2.0.0",
+ "tunnel-agent": "^0.6.0"
+ },
+ "bin": {
+ "prebuild-install": "bin.js"
+ },
+ "engines": {
+ "node": ">=10"
+ }
+ },
"node_modules/proxy-addr": {
"node_modules/proxy-addr": {
"version": "2.0.7",
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -856,6 +1114,16 @@
@@ -856,6 +1114,16 @@
"node": ">= 0.10"
"node": ">= 0.10"
}
}
},
},
+ "node_modules/pump": {
+ "version": "3.0.4",
+ "resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
+ "integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
+ "license": "MIT",
+ "dependencies": {
+ "end-of-stream": "^1.1.0",
+ "once": "^1.3.1"
+ }
+ },
"node_modules/qs": {
"node_modules/qs": {
"version": "6.15.3",
"version": "6.15.3",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.15.3.tgz",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.15.3.tgz",
@@ -896,6 +1164,35 @@
@@ -896,6 +1164,35 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
+ "node_modules/rc": {
+ "version": "1.2.8",
+ "resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
+ "integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
+ "license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
+ "dependencies": {
+ "deep-extend": "^0.6.0",
+ "ini": "~1.3.0",
+ "minimist": "^1.2.0",
+ "strip-json-comments": "~2.0.1"
+ },
+ "bin": {
+ "rc": "cli.js"
+ }
+ },
+ "node_modules/readable-stream": {
+ "version": "3.6.2",
+ "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
+ "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
+ "license": "MIT",
+ "dependencies": {
+ "inherits": "^2.0.3",
+ "string_decoder": "^1.1.1",
+ "util-deprecate": "^1.0.1"
+ },
+ "engines": {
+ "node": ">= 6"
+ }
+ },
"node_modules/require-from-string": {
"node_modules/require-from-string": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@ -1066,6 +1363,51 @@
@@ -1066,6 +1363,51 @@
"url": "https://github.com/sponsors/ljharb"
"url": "https://github.com/sponsors/ljharb"
}
}
},
},
+ "node_modules/simple-concat": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
+ "integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/feross"
+ },
+ {
+ "type": "patreon",
+ "url": "https://www.patreon.com/feross"
+ },
+ {
+ "type": "consulting",
+ "url": "https://feross.org/support"
+ }
+ ],
+ "license": "MIT"
+ },
+ "node_modules/simple-get": {
+ "version": "4.0.1",
+ "resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
+ "integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
+ "funding": [
+ {
+ "type": "github",
+ "url": "https://github.com/sponsors/feross"
+ },
+ {
+ "type": "patreon",
+ "url": "https://www.patreon.com/feross"
+ },
+ {
+ "type": "consulting",
+ "url": "https://feross.org/support"
+ }
+ ],
+ "license": "MIT",
+ "dependencies": {
+ "decompress-response": "^6.0.0",
+ "once": "^1.3.1",
+ "simple-concat": "^1.0.0"
+ }
+ },
"node_modules/statuses": {
"node_modules/statuses": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -1075,6 +1417,52 @@
@@ -1075,6 +1417,52 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
+ "node_modules/string_decoder": {
+ "version": "1.3.0",
+ "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
+ "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
+ "license": "MIT",
+ "dependencies": {
+ "safe-buffer": "~5.2.0"
+ }
+ },
+ "node_modules/strip-json-comments": {
+ "version": "2.0.1",
+ "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
+ "integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/tar-fs": {
+ "version": "2.1.5",
+ "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",
+ "integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",
+ "license": "MIT",
+ "dependencies": {
+ "chownr": "^1.1.1",
+ "mkdirp-classic": "^0.5.2",
+ "pump": "^3.0.0",
+ "tar-stream": "^2.1.4"
+ }
+ },
+ "node_modules/tar-stream": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
+ "integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
+ "license": "MIT",
+ "dependencies": {
+ "bl": "^4.0.3",
+ "end-of-stream": "^1.4.1",
+ "fs-constants": "^1.0.0",
+ "inherits": "^2.0.3",
+ "readable-stream": "^3.1.1"
+ },
+ "engines": {
+ "node": ">=6"
+ }
+ },
"node_modules/toidentifier": {
"node_modules/toidentifier": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz",
@@ -1084,6 +1472,18 @@
@@ -1084,6 +1472,18 @@
"node": ">=0.6"
"node": ">=0.6"
}
}
},
},
+ "node_modules/tunnel-agent": {
+ "version": "0.6.0",
+ "resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
+ "integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
+ "license": "Apache-2.0",
+ "dependencies": {
+ "safe-buffer": "^5.0.1"
+ },
+ "engines": {
+ "node": "*"
+ }
+ },
"node_modules/type-is": {
"node_modules/type-is": {
"version": "1.6.18",
"version": "1.6.18",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.18.tgz",
@@ -1106,6 +1506,12 @@
@@ -1106,6 +1506,12 @@
"node": ">= 0.8"
"node": ">= 0.8"
}
}
},
},
+ "node_modules/util-deprecate": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
+ "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
+ "license": "MIT"
+ },
"node_modules/utils-merge": {
"node_modules/utils-merge": {
"version": "1.0.1",
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
@@ -1123,6 +1529,12 @@
@@ -1123,6 +1529,12 @@
"engines": {
"engines": {
"node": ">= 0.8"
"node": ">= 0.8"
}
}
+ },
+ "node_modules/wrappy": {
+ "version": "1.0.2",
+ "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
+ "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
+ "license": "ISC"
}
}
}
}
}
}
@@ -10,6 +10,7 @@
@@ -10,6 +10,7 @@
},
},
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
+ "better-sqlite3": "^11.10.0",
"cookie-parser": "^1.4.7",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"cors": "^2.8.5",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
@@ -378,10 +378,10 @@ P.curlconvert=(s,m)=>DevEngine.curlconvert(s,m);
@@ -378,10 +378,10 @@ P.curlconvert=(s,m)=>DevEngine.curlconvert(s,m);
P.webhook=async()=>{
P.webhook=async()=>{
const o=$('out');
const o=$('out');
- const r=await fetch(API+'/webhook/bins',{method:'POST'});const d=await r.json();if(!d.ok)throw new Error(d.error||'Failed');
+ const r=await fetch(API+'/webhook/bins',{method:'POST',credentials:'include'});const d=await r.json();if(!d.ok)throw new Error(d.error||'Failed');
const binUrl=API+'/webhook/bins/'+d.binId+'/in';
const binUrl=API+'/webhook/bins/'+d.binId+'/in';
o.innerHTML='<div style="margin-bottom:12px"><div class="jl">Your Webhook URL</div><div style="font-size:1rem;word-break:break-all;color:var(--ac);cursor:pointer" onclick="navigator.clipboard.writeText(\''+binUrl+"');toast('Copied!')\">"+binUrl+'</div><div style="color:var(--tx3);font-size:.78rem;margin-top:4px">Click to copy \u00b7 Polling for requests...</div></div><div id="whr">Waiting for requests...</div>';
o.innerHTML='<div style="margin-bottom:12px"><div class="jl">Your Webhook URL</div><div style="font-size:1rem;word-break:break-all;color:var(--ac);cursor:pointer" onclick="navigator.clipboard.writeText(\''+binUrl+"');toast('Copied!')\">"+binUrl+'</div><div style="color:var(--tx3);font-size:.78rem;margin-top:4px">Click to copy \u00b7 Polling for requests...</div></div><div id="whr">Waiting for requests...</div>';
- const iv=setInterval(async()=>{try{const r2=await fetch(API+'/webhook/bins/'+d.binId);const d2=await r2.json();
+ const iv=setInterval(async()=>{try{const r2=await fetch(API+'/webhook/bins/'+d.binId,{credentials:'include'});const d2=await r2.json();
if(d2.ok&&d2.requests.length){$('whr').innerHTML=d2.requests.map((rq,i)=>'<div class="js"><div class="jl">#'+(i+1)+' '+rq.method+' \u00b7 '+new Date(rq.timestamp).toLocaleTimeString()+'</div>Headers:\n'+Object.entries(rq.headers||{}).map(([k,v])=>k+': '+v).join('\n')+'\n\nBody:\n'+H(typeof rq.body==='object'?JSON.stringify(rq.body,null,2):String(rq.body||'(empty)'))+'</div>').join('')}}catch{}},2000);
if(d2.ok&&d2.requests.length){$('whr').innerHTML=d2.requests.map((rq,i)=>'<div class="js"><div class="jl">#'+(i+1)+' '+rq.method+' \u00b7 '+new Date(rq.timestamp).toLocaleTimeString()+'</div>Headers:\n'+Object.entries(rq.headers||{}).map(([k,v])=>k+': '+v).join('\n')+'\n\nBody:\n'+H(typeof rq.body==='object'?JSON.stringify(rq.body,null,2):String(rq.body||'(empty)'))+'</div>').join('')}}catch{}},2000);
setTimeout(()=>clearInterval(iv),3600000);return null;
setTimeout(()=>clearInterval(iv),3600000);return null;
};
};
@@ -4,18 +4,24 @@
@@ -4,18 +4,24 @@
// Dev.OpenVibe — API Routes
// Dev.OpenVibe — API Routes
// Server-side endpoints for webhook bins, Open Graph fetching,
// Server-side endpoints for webhook bins, Open Graph fetching,
// and tool listing. Most dev tools are client-side only.
// and tool listing. Most dev tools are client-side only.
+//
+// Guard (apps/_shared/guard, opts.guard): the Open Graph fetcher counts against tools-fetch and a
+// per-target throttle; a webhook bin belongs to whoever made it (the browser session cookie, a
+// sign-in or a token): only they can read or delete it, anyone may post to its /in URL. Bins per
+// owner and per address are capped (limits.js webhook), so one address cannot fill the global cap.
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
const { Router } = require('express');
const { Router } = require('express');
const crypto = require('crypto');
const crypto = require('crypto');
const { DEV_TOOLS } = require('./config');
const { DEV_TOOLS } = require('./config');
const { createEgress, TargetRefused } = require('../../../_shared/egress');
const { createEgress, TargetRefused } = require('../../../_shared/egress');
+const { createCallerResolver } = require('../../../_shared/guard/caller');
+const guardLimits = require('../../../_shared/guard/limits');
// ── In-memory webhook bin storage ────────────────────────────
// ── In-memory webhook bin storage ────────────────────────────
-const webhookBins = new Map();
+const webhookBins = new Map(); // binId → { created, requests, owner, ipKey }
const WEBHOOK_BIN_TTL = 60 * 60 * 1000; // 1 hour
const WEBHOOK_BIN_TTL = 60 * 60 * 1000; // 1 hour
const WEBHOOK_MAX_REQUESTS = 200;
const WEBHOOK_MAX_REQUESTS = 200;
-const WEBHOOK_MAX_BINS = 500;
function cleanupBins() {
function cleanupBins() {
const now = Date.now();
const now = Date.now();
@@ -51,21 +57,19 @@ function extractOGTags(html) {
@@ -51,21 +57,19 @@ function extractOGTags(html) {
// ═════════════════════════════════════════════════════════════
// ═════════════════════════════════════════════════════════════
// `opts.egress` is for tests: the shared SSRF guard with a mock resolver and transports.
// `opts.egress` is for tests: the shared SSRF guard with a mock resolver and transports.
+// `opts.guard` is the gateway's guard (identify() has run before these routes; quotas, the per-target
+// throttle, the abuse log). Without one (tests), bins are still owned: by the session cookie.
module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
const router = Router();
const router = Router();
const egress = opts.egress || createEgress();
const egress = opts.egress || createEgress();
+ const guard = opts.guard || null;
+ const callers = guard ? null : createCallerResolver({ getPublicKey: () => null });
+ const whoIs = (req, res, create) => (guard ? guard.resolveCaller(req, res, { create }) : callers.resolve(req, res, { create }));
+ const caps = guardLimits.bounds().webhook;
+ const refuse = (req, res, r) => (guard ? guard.refuse(req, res, r) : (res.status(r.status).json({ error: r.detail, code: r.code }), true));
- // Optional auth — attaches req.user if token present, but doesn't block.
- // Verifies the shared ov_token OFFLINE against the Network public key.
- function optionalAuth(req, res, next) {
- const authHeader = req.headers.authorization;
- const token = authHeader?.startsWith('Bearer ') ? authHeader.slice(7) : req.cookies?.ov_token;
- const auth = req.app.locals.auth;
- if (!token || !auth) return next();
- Promise.resolve(auth.verify(token))
- .then(claims => { if (claims) req.user = claims; next(); })
- .catch(() => next());
- }
+ // Every dev API call (and the Open Graph fetcher's own tools-fetch quota, below).
+ if (guard) router.use(guard.toolQuota((req) => (req.method === 'GET' && req.path === '/opengraph' ? 'opengraph' : null)));
// ── List all dev tools ───────────────────────────────────
// ── List all dev tools ───────────────────────────────────
router.get('/tools', (_req, res) => {
router.get('/tools', (_req, res) => {
@@ -80,7 +84,7 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
@@ -80,7 +84,7 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
});
});
// ── Open Graph / Meta tag fetcher ────────────────────────
// ── Open Graph / Meta tag fetcher ────────────────────────
- router.get('/opengraph', optionalAuth, async (req, res) => {
+ router.get('/opengraph', async (req, res) => {
let targetUrl = req.query.url || req.query.target;
let targetUrl = req.query.url || req.query.target;
if (!targetUrl) return res.status(400).json({ error: 'Missing ?url= parameter' });
if (!targetUrl) return res.status(400).json({ error: 'Missing ?url= parameter' });
@@ -92,6 +96,8 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
@@ -92,6 +96,8 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
} catch {
} catch {
return res.status(400).json({ error: 'Invalid URL' });
return res.status(400).json({ error: 'Invalid URL' });
}
}
+ // Per-target throttle (descriptor opengraph: limits.perTargetPerMinute), across every caller.
+ if (guard && !guard.target(req, res, { tool: 'opengraph', target: targetUrl })) return;
try {
try {
// Public addresses only, checked after DNS and dialled as checked; each redirect hop is
// Public addresses only, checked after DNS and dialled as checked; each redirect hop is
@@ -147,17 +153,32 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
@@ -147,17 +153,32 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
// ── Webhook Bins ─────────────────────────────────────────
// ── Webhook Bins ─────────────────────────────────────────
- // Create a new bin
- router.post('/webhook/bins', optionalAuth, (_req, res) => {
+ /** The bin, if it exists AND belongs to the caller; otherwise null (the same 404 either way). */
+ function ownBin(req, res) {
+ const bin = webhookBins.get(req.params.binId);
+ const who = whoIs(req, res, false);
+ return bin && who.owner && bin.owner === who.owner ? bin : null;
+ }
+
+ // Create a new bin. Its owner is the caller: a browser gets the session cookie here (the page
+ // calls with credentials), a signed-in person or a token is itself.
+ router.post('/webhook/bins', (req, res) => {
cleanupBins();
cleanupBins();
- if (webhookBins.size >= WEBHOOK_MAX_BINS) {
+ if (webhookBins.size >= caps.total) {
return res.status(429).json({ error: 'Too many active bins. Try again later.' });
return res.status(429).json({ error: 'Too many active bins. Try again later.' });
}
}
+ const who = whoIs(req, res, true);
+ let mine = 0, fromHere = 0;
+ for (const b of webhookBins.values()) { if (b.owner === who.owner) mine++; if (b.ipKey === who.ipKey) fromHere++; }
+ if (mine >= caps.perOwner && refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'webhook', tool: 'webhook', retryAfter: 300, detail: `At most ${caps.perOwner} webhook bins at a time; old ones expire after an hour.`, extra: { scope: 'webhook' } })) return undefined;
+ if (fromHere >= caps.perIp && refuse(req, res, { status: 429, code: 'tools.quota.exceeded', reason: 'webhook', tool: 'webhook', retryAfter: 300, detail: `At most ${caps.perIp} webhook bins from one address at a time.`, extra: { scope: 'address' } })) return undefined;
const binId = crypto.randomBytes(12).toString('hex');
const binId = crypto.randomBytes(12).toString('hex');
webhookBins.set(binId, {
webhookBins.set(binId, {
created: Date.now(),
created: Date.now(),
requests: [],
requests: [],
+ owner: who.owner,
+ ipKey: who.ipKey,
});
});
res.json({
res.json({
@@ -168,9 +189,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
@@ -168,9 +189,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
});
});
});
});
- // Get bin requests
- router.get('/webhook/bins/:binId', optionalAuth, (req, res) => {
- const bin = webhookBins.get(req.params.binId);
+ // Get bin requests (its owner only)
+ router.get('/webhook/bins/:binId', (req, res) => {
+ res.set('Cache-Control', 'private, no-store');
+ const bin = ownBin(req, res);
if (!bin) return res.status(404).json({ error: 'Bin not found or expired' });
if (!bin) return res.status(404).json({ error: 'Bin not found or expired' });
res.json({
res.json({
@@ -213,9 +235,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
@@ -213,9 +235,10 @@ module.exports = function createDevRoutes(db, requireAuth, opts = {}) {
res.status(200).json({ ok: true, message: 'Received' });
res.status(200).json({ ok: true, message: 'Received' });
});
});
- // Delete bin
- router.delete('/webhook/bins/:binId', optionalAuth, (req, res) => {
- const deleted = webhookBins.delete(req.params.binId);
+ // Delete bin (its owner only; anyone else learns nothing)
+ router.delete('/webhook/bins/:binId', (req, res) => {
+ const bin = ownBin(req, res);
+ const deleted = bin ? webhookBins.delete(req.params.binId) : false;
res.json({ ok: true, deleted });
res.json({ ok: true, deleted });
});
});
@@ -27,6 +27,7 @@ const createNetRoutes = require('./net/routes');
@@ -27,6 +27,7 @@ const createNetRoutes = require('./net/routes');
const { NET_TOOL_MAP, NET_ALIASES } = require('./net/config');
const { NET_TOOL_MAP, NET_ALIASES } = require('./net/config');
const createDevRoutes = require('./dev/routes');
const createDevRoutes = require('./dev/routes');
const { DEV_TOOL_MAP, DEV_ALIASES } = require('./dev/config');
const { DEV_TOOL_MAP, DEV_ALIASES } = require('./dev/config');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
@@ -51,6 +52,19 @@ const registry = require('./registry');
@@ -51,6 +52,19 @@ const registry = require('./registry');
const toolRegistry = require('./registry/descriptors');
const toolRegistry = require('./registry/descriptors');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
let auth = null; // created below; the key check reads it at request time
let auth = null; // created below; the key check reads it at request time
+
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
+// the address), the tools-api quota, the net and dev tools' quotas by descriptor, the per-target
+// throttle, the port-scan cap, webhook bin caps and the abuse log (data/guard.db). The Network key is
+// the OAuth client's (loaded and retried by it). TOOLS_GUARD=report (default) records what it would refuse.
+const guard = createGuard({
+ app: 'gateway', dataDir: require('path').resolve(__dirname, '..', process.env.DATA_DIR || 'data'),
+ Database: require('better-sqlite3'), contracts: require('openvibe-contracts'),
+ specs: [...require('./net/descriptors').SPECS, ...require('./dev/descriptors').SPECS],
+ issuer: config.networkUrl,
+ keys: { get: () => (auth && auth.client.publicKey) || null, ensure: () => (auth ? auth.ensureKey() : Promise.resolve(null)) },
+});
const obs = observe({
const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools', release: release.release, mountReady: false,
service: 'tools', release: release.release, mountReady: false,
@@ -76,6 +90,7 @@ const obs = observe({
@@ -76,6 +90,7 @@ const obs = observe({
...Object.entries(SATELLITES).map(([name, port]) => ready.upstream(`satellite_${name}`, `http://127.0.0.1:${port}/api/ready`, { description: `${name} satellite (port ${port}); its hosts fail without it` })),
...Object.entries(SATELLITES).map(([name, port]) => ready.upstream(`satellite_${name}`, `http://127.0.0.1:${port}/api/ready`, { description: `${name} satellite (port ${port}); its hosts fail without it` })),
],
],
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
function getRequestHost(req) {
function getRequestHost(req) {
@@ -84,9 +99,10 @@ function getRequestHost(req) {
@@ -84,9 +99,10 @@ function getRequestHost(req) {
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
// One hop: the host's nginx, which applies Cloudflare's real IP (real_ip_header CF-Connecting-IP) and
// One hop: the host's nginx, which applies Cloudflare's real IP (real_ip_header CF-Connecting-IP) and
-// sets X-Forwarded-For to $remote_addr. req.ip is that address; anything a client put in its own
-// X-Forwarded-For stays to the left of it and is never believed (myip, rate limits, forwarded IPs).
-app.set('trust proxy', 1);
+// sets X-Forwarded-For to $remote_addr, on loopback (TRUST_PROXY). req.ip is that address; anything a
+// client put in its own X-Forwarded-For is never believed (myip, rate limits, forwarded IPs). The
+// satellites this gateway proxies to get req.ip as their one hop (registry/host-middleware.js).
+app.set('trust proxy', TRUST_PROXY);
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
@@ -155,9 +171,10 @@ app.use(exceptRegistry(cors({
@@ -155,9 +171,10 @@ app.use(exceptRegistry(cors({
credentials: true,
credentials: true,
})));
})));
-// ── Rate Limiting ────────────────────────────────────────────
-app.use('/api/', rateLimit({ windowMs: 60_000, max: 120 }));
-app.use('/auth/', rateLimit({ windowMs: 15 * 60_000, max: 60 }));
+// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──
+app.use(guard.identify);
+app.use('/api/', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 120, signedIn: 240, message: 'Too many requests. Please try again later.' }), guard.apiQuota);
+app.use('/auth/', rateLimit({ windowMs: 15 * 60_000, max: 60, keyGenerator: (req) => guard.caller(req).ipKey }));
// ── Tool registry (ADR-027, capability tools.tool.read) ──────
// ── Tool registry (ADR-027, capability tools.tool.read) ──────
// GET /api/v1/tools (tools.tool-list@1), /api/v1/tools/:id (tools.tool@1), /api/v1/tools/:id/schema.
// GET /api/v1/tools (tools.tool-list@1), /api/v1/tools/:id (tools.tool@1), /api/v1/tools/:id/schema.
@@ -195,7 +212,7 @@ app.get('/api/brand', (_req, res) => res.json(BRAND));
@@ -195,7 +212,7 @@ app.get('/api/brand', (_req, res) => res.json(BRAND));
// which owns pastes (roadmap Wave 5). The visitor's Network JWT goes along: Community verifies it and
// which owns pastes (roadmap Wave 5). The visitor's Network JWT goes along: Community verifies it and
// files writes under the person's canonical subject, so no site has to translate ids any more. The
// files writes under the person's canonical subject, so no site has to translate ids any more. The
// client address goes along too, for Community's anonymous-write limits.
// client address goes along too, for Community's anonymous-write limits.
-app.use('/api/pastes', rateLimit({ windowMs: 60_000, max: 120 }), async (req, res) => {
+app.use('/api/pastes', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 120, signedIn: 240, message: 'Too many requests. Please try again later.' }), async (req, res) => {
try {
try {
const target = `${config.communityUrl}/api/pastes${req.url}`;
const target = `${config.communityUrl}/api/pastes${req.url}`;
const fetchOpts = { method: req.method, headers: {} };
const fetchOpts = { method: req.method, headers: {} };
@@ -222,10 +239,10 @@ app.use('/api/pastes', rateLimit({ windowMs: 60_000, max: 120 }), async (req, re
@@ -222,10 +239,10 @@ app.use('/api/pastes', rateLimit({ windowMs: 60_000, max: 120 }), async (req, re
});
});
// ── Net.OpenVibe — Network Tools API ─────────────────────────
// ── Net.OpenVibe — Network Tools API ─────────────────────────
-app.use('/api/net', rateLimit({ windowMs: 60_000, max: 60 }), createNetRoutes(null, requireAuth));
+app.use('/api/net', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }), createNetRoutes(null, requireAuth, { guard }));
// ── Dev.OpenVibe — Developer & SEO Tools API ─────────────────
// ── Dev.OpenVibe — Developer & SEO Tools API ─────────────────
-app.use('/api/dev', rateLimit({ windowMs: 60_000, max: 60 }), createDevRoutes(null, requireAuth));
+app.use('/api/dev', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' }), createDevRoutes(null, requireAuth, { guard }));
// ── Host-header subdomain routing ────────────────────────────
// ── Host-header subdomain routing ────────────────────────────
function subdomainOf(req) {
function subdomainOf(req) {
@@ -24,7 +24,40 @@ const IPAPI_FIELDS = 'status,message,country,countryCode,region,regionName,city,
@@ -24,7 +24,40 @@ const IPAPI_FIELDS = 'status,message,country,countryCode,region,regionName,city,
* Every tool here that connects to the target (ssl, headers, redirects, port, ping, lookup, and a
* Every tool here that connects to the target (ssl, headers, redirects, port, ping, lookup, and a
* custom DNS server) goes through the shared SSRF guard: only public addresses, checked after DNS
* custom DNS server) goes through the shared SSRF guard: only public addresses, checked after DNS
* and dialled as checked. `opts.egress` is for tests (a guard with a mock resolver and transports).
* and dialled as checked. `opts.egress` is for tests (a guard with a mock resolver and transports).
+ *
+ * `opts.guard` (apps/_shared/guard, the gateway's): each call is charged to its tool's descriptor
+ * (tools-fetch or tools-probe × the caller's tier, weighted by cost), a principal needs the tool's
+ * capability (tools.net.probe for port, ping, latency), every tool that reaches a target is throttled
+ * per target across all callers (limits.perTargetPerMinute), and the port checker is capped per
+ * request and per caller so it cannot sweep hosts or ranges. The throttle and the port caps apply in
+ * report mode too. The pages call these routes without a session: people are counted by address.
*/
*/
+const { SPECS: NET_SPECS } = require('./descriptors');
+
+// endpoint ('/dns') → the descriptors served by it (dns, dig, nslookup, mx, txt, …)
+const BY_ENDPOINT = new Map();
+for (const spec of NET_SPECS) {
+ if (!spec.route) continue;
+ const ep = spec.route.path.replace(/^\/api\/net/, '');
+ if (!BY_ENDPOINT.has(ep)) BY_ENDPOINT.set(ep, []);
+ BY_ENDPOINT.get(ep).push(spec);
+}
+
+/** The tool a /api/net request runs: the host's own tool when it uses this endpoint, else the one named after it. */
+function netToolOf(req) {
+ const ep = `/${String(req.path || '').split('/')[1] || ''}`;
+ const list = BY_ENDPOINT.get(ep);
+ if (!list) return null;
+ const host = req.ovHost && req.ovHost.tool;
+ return (list.find(s => s.id === host) || list.find(s => s.id === ep.slice(1)) || list[0]).id;
+}
+
+/** The target a request names: /api/net/<endpoint>/<target> or ?target= (or ?url=). */
+function netTargetOf(req) {
+ const m = /^\/[a-z0-9]+\/(.+)$/i.exec(String(req.path || ''));
+ if (m) { try { return decodeURIComponent(m[1]); } catch { return m[1]; } }
+ return String(req.query.target || req.query.url || '');
+}
// ── IPv4 / IPv6 arithmetic ─────────────────────────────────
// ── IPv4 / IPv6 arithmetic ─────────────────────────────────
const ipv4ToInt = (ip) => ip.split('.').reduce((n, o) => n * 256 + Number(o), 0) >>> 0;
const ipv4ToInt = (ip) => ip.split('.').reduce((n, o) => n * 256 + Number(o), 0) >>> 0;
@@ -117,6 +150,7 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
@@ -117,6 +150,7 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
const cache = opts.cache || createCache({ max: 5000, ttlMs: HOUR });
const cache = opts.cache || createCache({ max: 5000, ttlMs: HOUR });
const checks = createChecks({ egress, cache, ...(opts.resolverFor && { resolverFor: opts.resolverFor }), ...(opts.tlsUpgrade && { tlsUpgrade: opts.tlsUpgrade }) });
const checks = createChecks({ egress, cache, ...(opts.resolverFor && { resolverFor: opts.resolverFor }), ...(opts.tlsUpgrade && { tlsUpgrade: opts.tlsUpgrade }) });
const reverseDns = opts.reverse || ((ip) => dns.reverse(ip));
const reverseDns = opts.reverse || ((ip) => dns.reverse(ip));
+ const guard = opts.guard || null;
// ── Helpers ──────────────────────────────────────────────
// ── Helpers ──────────────────────────────────────────────
@@ -226,20 +260,25 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
@@ -226,20 +260,25 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
/** Get config (live — reads DB each time for admin changes) */
/** Get config (live — reads DB each time for admin changes) */
function cfg() { return getNetConfig(db); }
function cfg() { return getNetConfig(db); }
- /** Optional auth — attaches user if token present, but doesn't block.
- * Verifies the shared ov_token OFFLINE against the Network public key. */
- function optionalAuth(req, res, next) {
- const authHeader = req.headers.authorization;
- const token = authHeader?.startsWith('Bearer ') ? authHeader.slice(7) : req.cookies?.ov_token;
- const auth = req.app.locals.auth;
- if (!token || !auth) return next();
- Promise.resolve(auth.verify(token))
- .then(claims => { if (claims) req.user = claims; next(); })
- .catch(() => next());
+ // ── Guard: who may run it, its quota, the per-target throttle (sign-in was read by the app) ──
+ if (guard) {
+ router.use((req, res, next) => {
+ const id = netToolOf(req);
+ if (!id) return next();
+ req.netTool = id;
+ if (!guard.capabilityRule(req, res, guard.tool(id))) return undefined;
+ return next();
+ });
+ router.use(guard.toolQuota((req) => req.netTool));
+ router.use((req, res, next) => {
+ const d = req.netTool && guard.tool(req.netTool);
+ if (!d || !d.egress || !(d.limits && d.limits.perTargetPerMinute)) return next();
+ const t = netTargetOf(req);
+ if (t && !guard.target(req, res, { tool: d.id, target: t })) return undefined;
+ return next();
+ });
}
}
- router.use(optionalAuth);
-
// ── Tool list endpoint ───────────────────────────────────
// ── Tool list endpoint ───────────────────────────────────
router.get('/tools', (_req, res) => {
router.get('/tools', (_req, res) => {
ok(res, { tools: NET_TOOLS, regions: cfg().probeRegions });
ok(res, { tools: NET_TOOLS, regions: cfg().probeRegions });
@@ -699,9 +738,12 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
@@ -699,9 +738,12 @@ module.exports = function createNetRoutes(db, requireAuth, opts = {}) {
const target = parseTarget(req.params.target || req.query.target);
const target = parseTarget(req.params.target || req.query.target);
if (!target) return fail(res, 'Please provide a host');
if (!target) return fail(res, 'Please provide a host');
- const portsStr = req.query.ports || '80,443,22,21,25,53,3306,5432,8080,8443';
- const ports = portsStr.split(',').map(p => parseInt(p.trim())).filter(p => p > 0 && p <= 65535).slice(0, 20);
+ const portsStr = String(req.query.ports || '80,443,22,21,25,53,3306,5432,8080,8443');
+ const ports = [...new Set(portsStr.split(',').map(p => parseInt(p.trim(), 10)).filter(p => p > 0 && p <= 65535))];
if (!ports.length) return fail(res, 'No valid ports specified');
if (!ports.length) return fail(res, 'No valid ports specified');
+ if (ports.length > 20) return fail(res, 'At most 20 ports per check');
+ // Per caller: at most so many ports and so many different hosts in ten minutes (a hard limit).
+ if (guard && !guard.portScan(req, res, { tool: req.netTool || 'port', target, ports })) return undefined;
// Resolve + check; every probe dials the checked address.
// Resolve + check; every probe dials the checked address.
const dest = await egress.resolve(target, { prefer: 4 });
const dest = await egress.resolve(target, { prefer: 4 });
@@ -31,8 +31,12 @@ function proxyTo(port, info, req, res) {
@@ -31,8 +31,12 @@ function proxyTo(port, info, req, res) {
headers['x-ov-short-host'] = info.shortHost;
headers['x-ov-short-host'] = info.shortHost;
headers['x-forwarded-host'] = info.host;
headers['x-forwarded-host'] = info.host;
headers['x-forwarded-proto'] = 'https';
headers['x-forwarded-proto'] = 'https';
- const prior = req.headers['x-forwarded-for'];
- headers['x-forwarded-for'] = (prior ? prior + ', ' : '') + (req.socket.remoteAddress || '');
+ // The gateway is the satellite's one trusted hop (TRUST_PROXY): it passes on the address it
+ // resolved itself (req.ip, from nginx), never a chain a client could have started.
+ const client = String(req.ip || req.socket.remoteAddress || '');
+ headers['x-forwarded-for'] = client;
+ headers['x-real-ip'] = client;
+ headers['cf-connecting-ip'] = client;
const up = http.request({ host: '127.0.0.1', port, method: req.method, path: req.originalUrl || req.url, headers }, (r) => {
const up = http.request({ host: '127.0.0.1', port, method: req.method, path: req.originalUrl || req.url, headers }, (r) => {
const out = {};
const out = {};
for (const [k, v] of Object.entries(r.headers)) if (!HOP.has(k)) out[k] = v;
for (const [k, v] of Object.entries(r.headers)) if (!HOP.has(k)) out[k] = v;
new file mode 100644
new file mode 100644
@@ -0,0 +1,156 @@
@@ -0,0 +1,156 @@
+'use strict';
+// The guard on the gateway's own tools: the net tools count against their descriptor's class and
+// cost, every tool that reaches a target is throttled per target across all callers, the port
+// checker cannot sweep (per request and per caller, in report mode too), a webhook bin belongs to
+// whoever made it (session cookie or token) and one address cannot fill the global cap, a principal
+// needs tools.net.probe for probes, and the proxy to a satellite passes the one address it resolved.
+// Mock resolver and transports: nothing touches a real network.
+const assert = require('assert');
+const crypto = require('crypto');
+const http = require('http');
+const express = require('express');
+const cookieParser = require('cookie-parser');
+const { EventEmitter } = require('events');
+const contracts = require('openvibe-contracts');
+const { createEgress } = require('../../_shared/egress');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
+const createNetRoutes = require('../server/net/routes');
+const createDevRoutes = require('../server/dev/routes');
+const { proxyTo } = require('../server/registry/host-middleware');
+
+const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });
+const ISSUER = 'https://openvibe.network';
+const t = () => Math.floor(Date.now() / 1000);
+const svc = (cap) => contracts.serviceAuth.signServiceToken({ iss: ISSUER, sub: 'svc:partner', actor_type: 'service', aud: ['openvibe.tools'], cap, ns: [], iat: t(), exp: t() + 900, jti: `tok_${crypto.randomBytes(12).toString('hex')}` }, privateKey);
+
+const lookup = (host, _opts, cb) => cb(null, [{ address: '93.184.216.34', family: 4 }]);
+const probes = [];
+const egress = createEgress({
+ lookup,
+ tcpConnect: ({ host, port }) => { probes.push(`${host}:${port}`); const s = new EventEmitter(); s.setTimeout = () => {}; s.destroy = () => {}; s.end = () => {}; process.nextTick(() => s.emit('connect')); return s; },
+ tlsConnect: () => { throw new Error('unused'); },
+ httpRequest: () => { throw new Error('unused'); },
+ httpsRequest: () => { throw new Error('unused'); },
+});
+
+async function serve(app) {
+ const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });
+ return { base: `http://127.0.0.1:${srv.address().port}`, srv, close: () => new Promise(r => srv.close(r)) };
+}
+
+(async () => {
+ const guard = createGuard({
+ app: 'gateway', contracts, issuer: ISSUER, keys: { get: () => publicKey },
+ specs: [...require('../server/net/descriptors').SPECS, ...require('../server/dev/descriptors').SPECS],
+ env: { TOOLS_GUARD: 'report' }, log: { log() {}, warn() {}, error() {} }, pruneIntervalMs: 0,
+ });
+ const app = express();
+ app.set('trust proxy', TRUST_PROXY);
+ app.use(cookieParser());
+ app.use(guard.identify);
+ app.use('/api/net', createNetRoutes(null, null, { egress, guard }));
+ app.use('/api/dev', createDevRoutes(null, null, { egress, guard }));
+ const s = await serve(app);
+ const get = (p, headers = {}) => fetch(s.base + p, { headers });
+ const from = (ip, extra = {}) => ({ 'X-Forwarded-For': ip, ...extra });
+ try {
+ // ── Quota by descriptor: a ping is tools-probe at cost 3 ──
+ let r = await get('/api/net/ping/one.example?count=1', from('198.51.100.1'));
+ assert.strictEqual(r.status, 200);
+ assert.strictEqual(r.headers.get('ratelimit-limit'), '15', 'tools-probe, anonymous: a bucket of 15');
+ assert.strictEqual(r.headers.get('ratelimit-remaining'), '12', 'one ping costs 3');
+
+ // ── Per-target throttle: ping 6 a minute against one host, whoever asks, in report mode too ──
+ const codes = [];
+ for (let i = 2; i <= 7; i++) codes.push((await get('/api/net/ping/ONE.example.?count=1', from(`198.51.100.${i}`))).status);
+ assert.deepStrictEqual(codes, [200, 200, 200, 200, 200, 429]);
+ r = await get('/api/net/headers/https%3A%2F%2Fone.example%2Fpath', from('198.51.100.30'));
+ assert.strictEqual(r.status, 429, 'other tools share the host\'s budget');
+ const body = await r.json();
+ assert.deepStrictEqual([body.code, body.scope], ['tools.quota.exceeded', 'target']);
+ assert.ok(Number(r.headers.get('retry-after')) >= 1);
+ assert.strictEqual((await get('/api/net/ping/two.example?count=1', from('198.51.100.2'))).status, 200, 'another host is fine');
+
+ // ── The port checker: per request, per caller ──
+ const ports = (n, base = 1000) => Array.from({ length: n }, (_, i) => base + i).join(',');
+ r = await get(`/api/net/port/three.example?ports=${ports(21)}`, from('198.51.100.50'));
+ assert.strictEqual(r.status, 400, 'more than 20 ports is refused, not silently cut');
+ probes.length = 0;
+ for (let i = 0; i < 5; i++) {
+ r = await get(`/api/net/port/h${i}.example?ports=${ports(20)}`, from('198.51.100.51'));
+ assert.strictEqual(r.status, 200, `check ${i + 1}`);
+ }
+ assert.strictEqual(probes.length, 100);
+ r = await get('/api/net/port/h9.example?ports=22', from('198.51.100.51'));
+ assert.strictEqual(r.status, 429, 'the 101st port in ten minutes is refused (a hard limit)');
+ assert.strictEqual((await r.json()).scope, 'ports');
+ assert.strictEqual(probes.length, 100, 'nothing was dialled');
+
+ // ── Probes through the API need tools.net.probe (report mode records, enforce refuses) ──
+ r = await get('/api/net/ping/four.example?count=1', { Authorization: `Bearer ${svc(['tools.tool.run'])}` });
+ assert.strictEqual(r.status, 200, 'report mode: recorded, not refused');
+ assert.ok(guard.store.abuseRows().some(x => x.reason === 'capability' && x.principal === 'svc:partner' && x.tool === 'ping'));
+ r = await get('/api/net/ping/five.example?count=1', { Authorization: `Bearer ${svc(['tools.net.probe'])}` });
+ assert.strictEqual(r.status, 200);
+ assert.strictEqual(r.headers.get('ratelimit-limit'), '300', 'the service tier');
+
+ // ── Webhook bins belong to their maker ──
+ r = await fetch(`${s.base}/api/dev/webhook/bins`, { method: 'POST', headers: from('203.0.113.10') });
+ const made = await r.json();
+ const cookie = String(r.headers.get('set-cookie') || '').split(';')[0];
+ assert.match(cookie, /^ov_tools_jobs=/, 'the maker gets a session');
+ assert.ok(made.ok && made.binId);
+ await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}/in`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{"hello":1}' });
+ r = await get(`/api/dev/webhook/bins/${made.binId}`, { cookie });
+ assert.strictEqual(r.status, 200);
+ assert.strictEqual((await r.json()).requestCount, 1, 'the maker reads it');
+ r = await get(`/api/dev/webhook/bins/${made.binId}`, from('203.0.113.10'));
+ assert.strictEqual(r.status, 404, 'anyone else (even from the same address) gets 404');
+ r = await get(`/api/dev/webhook/bins/${made.binId}`, { cookie: `ov_tools_jobs=${crypto.randomBytes(24).toString('base64url')}` });
+ assert.strictEqual(r.status, 404, 'another session gets 404');
+ r = await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}`, { method: 'DELETE', headers: from('203.0.113.99') });
+ assert.strictEqual((await r.json()).deleted, false, 'and cannot delete it');
+ assert.strictEqual((await get(`/api/dev/webhook/bins/${made.binId}`, { cookie })).status, 200, 'still there');
+ r = await fetch(`${s.base}/api/dev/webhook/bins/${made.binId}`, { method: 'DELETE', headers: { cookie } });
+ assert.strictEqual((await r.json()).deleted, true, 'the maker can');
+
+ // Caps: per owner and per address (report mode records; a guard in enforce mode refuses).
+ const enforce = createGuard({ app: 'gateway', contracts, issuer: ISSUER, keys: { get: () => publicKey }, specs: require('../server/dev/descriptors').SPECS, env: { TOOLS_GUARD: 'enforce' }, log: { log() {}, warn() {}, error() {} }, pruneIntervalMs: 0 });
+ const app2 = express();
+ app2.set('trust proxy', TRUST_PROXY);
+ app2.use(cookieParser());
+ app2.use('/api/dev', createDevRoutes(null, null, { egress, guard: enforce }));
+ const s2 = await serve(app2);
+ try {
+ const make = (headers) => fetch(`${s2.base}/api/dev/webhook/bins`, { method: 'POST', headers });
+ const first = await make(from('203.0.113.20'));
+ const c1 = String(first.headers.get('set-cookie')).split(';')[0];
+ const mine = [first.status];
+ for (let i = 0; i < 5; i++) mine.push((await make({ ...from('203.0.113.20'), cookie: c1 })).status);
+ assert.deepStrictEqual(mine, [200, 200, 200, 200, 200, 429], 'five bins per owner');
+ const fresh = [];
+ for (let i = 0; i < 6; i++) fresh.push((await make(from('203.0.113.20'))).status); // a new cookie every time
+ assert.deepStrictEqual(fresh, [200, 200, 200, 200, 200, 429], 'ten per address, whatever cookie is shown');
+ assert.strictEqual((await make(from('203.0.113.21'))).status, 200, 'another address is fine');
+ } finally { await s2.close(); enforce.close(); }
+
+ // ── The proxy to a satellite: the one address the gateway resolved, nothing a client sent ──
+ let seen = null;
+ const sat = http.createServer((req, res) => { seen = req.headers; res.end('ok'); });
+ await new Promise(r => sat.listen(0, '127.0.0.1', r));
+ const front = express();
+ front.set('trust proxy', TRUST_PROXY);
+ front.use((req, res) => proxyTo(sat.address().port, { tool: 'png', role: 'canonical', canonicalHost: 'png.openvibe.tools', shortHost: 'png.openvibe.tools', host: 'png.openvibe.tools' }, req, res));
+ const f = await serve(front);
+ try {
+ await fetch(`${f.base}/x`, { headers: { 'X-Forwarded-For': '6.6.6.6, 203.0.113.5', 'CF-Connecting-IP': '6.6.6.6' } });
+ assert.strictEqual(seen['x-forwarded-for'], '203.0.113.5', 'one address: the one nginx gave');
+ assert.strictEqual(seen['x-real-ip'], '203.0.113.5');
+ assert.strictEqual(seen['cf-connecting-ip'], '203.0.113.5');
+ } finally { await f.close(); await new Promise(r => sat.close(r)); }
+ } finally {
+ await s.close();
+ guard.close();
+ }
+ console.log('gateway guard: net quotas by descriptor, per-target throttle across tools and callers, port cap, probe capability, webhook bin ownership and caps, proxy address: all checks passed');
+})().catch((err) => { console.error(err); process.exit(1); });
@@ -11,6 +11,7 @@ const { EventEmitter } = require('events');
@@ -11,6 +11,7 @@ const { EventEmitter } = require('events');
const { Readable } = require('stream');
const { Readable } = require('stream');
const { createEgress } = require('../../_shared/egress');
const { createEgress } = require('../../_shared/egress');
const createNetRoutes = require('../server/net/routes');
const createNetRoutes = require('../server/net/routes');
+const { TRUST_PROXY } = require('../../_shared/guard');
const { createCache } = require('../server/net/cache');
const { createCache } = require('../server/net/cache');
const { parseRobots, testRobots, analyseSitemap } = require('../server/net/checks');
const { parseRobots, testRobots, analyseSitemap } = require('../server/net/checks');
const { NET_TOOLS } = require('../server/net/config');
const { NET_TOOLS } = require('../server/net/config');
@@ -182,7 +183,7 @@ async function fakeFetch(url) {
@@ -182,7 +183,7 @@ async function fakeFetch(url) {
// ── Through the routes ──
// ── Through the routes ──
const egress = createEgress({ lookup, tcpConnect, tlsConnect: () => { throw new Error('unused'); }, httpRequest: fakeHttp, httpsRequest: fakeHttp });
const egress = createEgress({ lookup, tcpConnect, tlsConnect: () => { throw new Error('unused'); }, httpRequest: fakeHttp, httpsRequest: fakeHttp });
const app = express();
const app = express();
- app.set('trust proxy', 1); // what the gateway sets: the host's nginx is the one hop
+ app.set('trust proxy', TRUST_PROXY); // what the gateway sets: the host's nginx on loopback is the one hop
app.use('/api/net', createNetRoutes(null, null, { egress, resolverFor, tlsUpgrade, fetch: fakeFetch, reverse: async (ip) => (ip === '2606:4700:4700::1111' ? ['one.one.one.one'] : []) }));
app.use('/api/net', createNetRoutes(null, null, { egress, resolverFor, tlsUpgrade, fetch: fakeFetch, reverse: async (ip) => (ip === '2606:4700:4700::1111' ? ['one.one.one.one'] : []) }));
const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });
const srv = await new Promise(r => { const s = app.listen(0, '127.0.0.1', () => r(s)); });
const base = `http://127.0.0.1:${srv.address().port}`;
const base = `http://127.0.0.1:${srv.address().port}`;
@@ -197,7 +198,7 @@ async function fakeFetch(url) {
@@ -197,7 +198,7 @@ async function fakeFetch(url) {
r = await get('/api/net/myip');
r = await get('/api/net/myip');
assert.strictEqual(r.body.ip, '127.0.0.1');
assert.strictEqual(r.body.ip, '127.0.0.1');
const gatewaySrc = fs.readFileSync(path.join(__dirname, '..', 'server', 'index.js'), 'utf8');
const gatewaySrc = fs.readFileSync(path.join(__dirname, '..', 'server', 'index.js'), 'utf8');
- assert.match(gatewaySrc, /app\.set\('trust proxy', 1\)/, 'the gateway trusts exactly one proxy hop');
+ assert.match(gatewaySrc, /app\.set\('trust proxy', TRUST_PROXY\)/, 'the gateway trusts exactly one proxy hop, on loopback');
// Upstream cache: ip-api once per IP, RDAP and DoH once per question.
// Upstream cache: ip-api once per IP, RDAP and DoH once per question.
fetched.length = 0;
fetched.length = 0;
deleted file mode 100644
deleted file mode 100644
@@ -1,120 +0,0 @@
@@ -1,120 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Img.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)
-// Lightweight auth — no DB, no linked_accounts.
-// Verifies the shared ov_token cookie offline against the
-// Network's public key and attaches decoded claims to req.user.
-// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise
-// fetched from the Network's /api/.well-known/jwks endpoint.
-// ═══════════════════════════════════════════════════════════════
-
-const jwt = require('jsonwebtoken');
-const fs = require('fs');
-const path = require('path');
-const config = require('./config');
-
-let publicKey = null;
-let retryTimer = null;
-
-const ISSUER = config.networkUrl;
-const JWKS_PATH = '/api/.well-known/jwks';
-const FETCH_TIMEOUT_MS = 5000;
-const RETRY_INTERVAL_MS = 60 * 1000;
-
-function loadKeyFromFile() {
- for (const p of config.publicKeyPaths) {
- try {
- const resolved = path.resolve(p);
- if (fs.existsSync(resolved)) {
- publicKey = fs.readFileSync(resolved, 'utf8');
- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);
- return true;
- }
- } catch { /* try next */ }
- }
- return false;
-}
-
-async function fetchKeyFromNetwork() {
- for (const base of [config.networkInternalUrl, config.networkUrl]) {
- try {
- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });
- if (!res.ok) continue;
- const body = await res.json();
- if (body && body.public_key) {
- publicKey = body.public_key;
- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);
- return true;
- }
- } catch { /* try next base */ }
- }
- return false;
-}
-
-function loadPublicKey() {
- if (loadKeyFromFile()) return;
- fetchKeyFromNetwork().then((loaded) => {
- if (loaded || retryTimer) return;
- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);
- retryTimer = setInterval(async () => {
- if (await fetchKeyFromNetwork()) {
- clearInterval(retryTimer);
- retryTimer = null;
- }
- }, RETRY_INTERVAL_MS);
- if (retryTimer.unref) retryTimer.unref();
- });
-}
-loadPublicKey();
-
-function extractToken(req) {
- const authHeader = req.headers.authorization;
- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);
- if (req.cookies?.ov_token) return req.cookies.ov_token;
- if (req.cookies?.token) return req.cookies.token;
- return null;
-}
-
-function verifyToken(token) {
- if (!publicKey || !token) return null;
- try {
- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });
- } catch {
- return null;
- }
-}
-
-/**
- * Optional auth — attaches req.user if valid token present.
- * Never blocks the request; anonymous users get req.user = null.
- */
-function optionalAuth(req, _res, next) {
- const token = extractToken(req);
- if (token) {
- const decoded = verifyToken(token);
- if (decoded) {
- req.user = decoded;
- req.token = token;
- }
- }
- next();
-}
-
-/**
- * Required auth — 401 if no valid token.
- */
-function requireAuth(req, res, next) {
- const token = extractToken(req);
- if (!token) return res.status(401).json({ error: 'Authentication required' });
- const decoded = verifyToken(token);
- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });
- req.user = decoded;
- req.token = token;
- next();
-}
-
-/** The Network public key (PEM) once loaded; the job routes verify service tokens with it. */
-function getPublicKey() { return publicKey; }
-
-module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey, ISSUER };
@@ -11,6 +11,7 @@
@@ -11,6 +11,7 @@
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
const { DOMAIN_MAP } = require('./domain-map');
const { DOMAIN_MAP } = require('./domain-map');
+const guardLimits = require('../../_shared/guard/limits');
// tools/convert.js FORMAT_CONFIG: what convert can write, and the media type each one is.
// tools/convert.js FORMAT_CONFIG: what convert can write, and the media type each one is.
const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', webp: 'image/webp', avif: 'image/avif', tiff: 'image/tiff', bmp: 'image/bmp', gif: 'image/gif', ico: 'image/x-icon' };
const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', webp: 'image/webp', avif: 'image/avif', tiff: 'image/tiff', bmp: 'image/bmp', gif: 'image/gif', ico: 'image/x-icon' };
@@ -18,8 +19,9 @@ const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', w
@@ -18,8 +19,9 @@ const FORMAT_MIME = { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', w
const ACCEPT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/bmp', 'image/gif', 'image/heic', 'image/heif', 'image/svg+xml', 'image/x-icon', 'image/vnd.microsoft.icon'];
const ACCEPT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/bmp', 'image/gif', 'image/heic', 'image/heif', 'image/svg+xml', 'image/x-icon', 'image/vnd.microsoft.icon'];
const MAX_BYTES = 50 * 1024 * 1024; // config.js upload.maxFileSize
const MAX_BYTES = 50 * 1024 * 1024; // config.js upload.maxFileSize
const TIMEOUT_MS = 5 * 60 * 1000; // process.js img.process timeoutMs
const TIMEOUT_MS = 5 * 60 * 1000; // process.js img.process timeoutMs
-const SHARP_PIXELS = 268402689; // sharp's default limitInputPixels (0x3FFF²), not raised here
-const CODEC_PIXELS = 100 * 1024 * 1024; // tools/codec.js MAX_PIXELS: BMP written or read by our codec
+// tools/codec.js inputPixels(): no upload larger than this is decoded (sharp's limitInputPixels, and the
+// BMP, ICO and HEIC readers), 40 megapixels unless TOOLS_MAX_INPUT_PIXELS says otherwise.
+const MAX_PIXELS = guardLimits.bounds().maxInputPixels;
const SAME_AS_INPUT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/gif', 'image/bmp'];
const SAME_AS_INPUT = ['image/png', 'image/jpeg', 'image/webp', 'image/avif', 'image/tiff', 'image/gif', 'image/bmp'];
const RESULT = {
const RESULT = {
@@ -74,7 +76,7 @@ function tool(id, extra = {}) {
@@ -74,7 +76,7 @@ function tool(id, extra = {}) {
input: INPUT[op](ctx.defaultFormat),
input: INPUT[op](ctx.defaultFormat),
files: { min: 1, max: 1, accept: ACCEPT, maxBytes: MAX_BYTES },
files: { min: 1, max: 1, accept: ACCEPT, maxBytes: MAX_BYTES },
output: { kind: 'file', mime: out, schema: RESULT },
output: { kind: 'file', mime: out, schema: RESULT },
- limits: { timeoutMs: TIMEOUT_MS, maxPixels: ctx.defaultFormat === 'bmp' ? CODEC_PIXELS : SHARP_PIXELS },
+ limits: { timeoutMs: TIMEOUT_MS, maxPixels: MAX_PIXELS },
auth: { anonymous: true, capability: 'tools.tool.run' },
auth: { anonymous: true, capability: 'tools.tool.run' },
quotaClass: 'tools-job', cost: op === 'convert' && ['avif', 'ico'].includes(ctx.defaultFormat) ? 8 : 5, egress: false,
quotaClass: 'tools-job', cost: op === 'convert' && ['avif', 'ico'].includes(ctx.defaultFormat) ? 8 : 5, egress: false,
example: { input: {} },
example: { input: {} },
@@ -13,13 +13,11 @@ const cookieParser = require('cookie-parser');
@@ -13,13 +13,11 @@ const cookieParser = require('cookie-parser');
const path = require('path');
const path = require('path');
const fs = require('fs');
const fs = require('fs');
+const rateLimit = require('express-rate-limit');
const config = require('./config');
const config = require('./config');
-const auth = require('./auth');
-const { optionalAuth } = auth;
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { resolveContext, DOMAIN_MAP } = require('./domain-map');
const { getTool, listTools } = require('./tools');
const { getTool, listTools } = require('./tools');
const { uploadSingle } = require('./middleware/upload');
const { uploadSingle } = require('./middleware/upload');
-const { apiLimiter, processLimiter, burstLimiter } = require('./middleware/rate-limit');
const retention = require('./retention/manager');
const retention = require('./retention/manager');
const { buildOptions, describe, processBuffer, defineJobs } = require('./process');
const { buildOptions, describe, processBuffer, defineJobs } = require('./process');
const codec = require('./tools/codec');
const codec = require('./tools/codec');
@@ -27,6 +25,7 @@ const { hostGuard, ownHost } = require('../../_shared/host-role');
@@ -27,6 +25,7 @@ const { hostGuard, ownHost } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const jobsRuntime = require('../../_shared/jobs');
const jobsRuntime = require('../../_shared/jobs');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const sdk = require('openvibe-sdk');
const sdk = require('openvibe-sdk');
@@ -40,13 +39,28 @@ const analyticsDb = new Database(analyticsDbPath);
@@ -40,13 +39,28 @@ const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-img', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-img', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, service tokens, the browser session, else
+// the address), tiered quotas by each tool's descriptor, upload sniffing, the sync semaphore and the
+// abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.
+const SPECS = require('./descriptors').SPECS;
+const guard = createGuard({
+ app: 'img', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, specs: SPECS,
+ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,
+});
+const { apiLimiter, processLimiter, burstLimiter } = guard.legacyLimiters(rateLimit);
+/** The tool a request is for before its body is read: the host's own, else the operation it defaults to. */
+const hostTool = (req) => (guard.tool(req.ctx.toolId) ? req.ctx.toolId : (guard.toolForJob('img.process', req.ctx.defaultOp || 'convert', null) || { id: null }).id);
+/** The tool a parsed request runs: its `tool` (operation) on this host. */
+const toolOf = (req) => { const d = guard.toolForJob('img.process', String((req.body && req.body.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : hostTool(req); };
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready } = require('../../_shared/observe');
const { observe, checks: ready } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-img', release: release.release,
service: 'tools-img', release: release.release,
checks: [
checks: [
@@ -56,19 +70,20 @@ observe({
@@ -56,19 +70,20 @@ observe({
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('uploads_dir', path.resolve(config.uploadsDir), { description: 'synchronous uploads' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.writableDir('output_dir', path.resolve(config.outputDir), { description: 'synchronous results' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; tools work without it' }),
- ready.networkKey('network_key', auth.getPublicKey, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
+ ready.networkKey('network_key', guard.keys.get, { description: 'verifies signed-in users and service tokens on the job API; anonymous use works without it' }),
...jobsRuntime.readyChecks(() => jobs),
...jobsRuntime.readyChecks(() => jobs),
codec.heif.readyCheck('HEIC (iPhone) photos: heif-dec or heif-convert with an HEVC decoder plugin (libheif-examples + libheif-plugin-libde265); other formats work without it'),
codec.heif.readyCheck('HEIC (iPhone) photos: heif-dec or heif-convert with an HEVC decoder plugin (libheif-examples + libheif-plugin-libde265); other formats work without it'),
],
],
jobs: () => jobs,
jobs: () => jobs,
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
-app.set('trust proxy', 2); // Cloudflare → Nginx → Node
+app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
directives: {
directives: {
@@ -100,8 +115,9 @@ app.use(exceptRegistry(cors({
@@ -100,8 +115,9 @@ app.use(exceptRegistry(cors({
credentials: true,
credentials: true,
})));
})));
-// ── Rate Limiting ────────────────────────────────────────────
-app.use('/api/', apiLimiter);
+// ── Who is asking, then rate limits (sign-in first, so a signed-in tier applies) ──
+app.use(guard.identify);
+app.use('/api/', apiLimiter, guard.apiQuota);
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's image tools ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's image tools ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -112,9 +128,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
@@ -112,9 +128,6 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
// ── Analytics Middleware ─────────────────────────────────────
// ── Analytics Middleware ─────────────────────────────────────
app.use(analytics.middleware());
app.use(analytics.middleware());
-// ── Auth (optional on all routes) ────────────────────────────
-app.use(optionalAuth);
-
// ── Hosts ────────────────────────────────────────────────────
// ── Hosts ────────────────────────────────────────────────────
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// Through the gateway the X-OV-* headers name the tool and its canonical host (a custom domain
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
// included); aliases the gateway missed are redirected. Hosts this app does not serve go to the
@@ -136,8 +149,11 @@ app.get('/api/health', (_req, res) => {
@@ -136,8 +149,11 @@ app.get('/api/health', (_req, res) => {
res.json({ status: 'ok', service: 'openvibe-img', version: '1.0.0', files: stats, jobs: jobs.stats() });
res.json({ status: 'ok', service: 'openvibe-img', version: '1.0.0', files: stats, jobs: jobs.stats() });
});
});
-// Domain context (frontend calls this on load to get branding)
+// Domain context (frontend calls this on load to get branding). It also starts this browser's session
+// (ov_tools_jobs), so the page's first job already counts as a session rather than an address.
app.get('/api/context', (req, res) => {
app.get('/api/context', (req, res) => {
+ guard.ensureSession(req, res);
+ res.set('Cache-Control', 'private, no-store');
const ctx = req.ctx;
const ctx = req.ctx;
const tools = listTools();
const tools = listTools();
res.json({
res.json({
@@ -161,7 +177,7 @@ app.get('/api/tools', (_req, res) => {
@@ -161,7 +177,7 @@ app.get('/api/tools', (_req, res) => {
});
});
// ── Main Processing Endpoint ─────────────────────────────────
// ── Main Processing Endpoint ─────────────────────────────────
-app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/process', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const tool = getTool(toolId);
const tool = getTool(toolId);
@@ -183,6 +199,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
@@ -183,6 +199,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
res.json({ success: true, download: saved, ...describe(toolId, result) });
res.json({ success: true, download: saved, ...describe(toolId, result) });
} catch (err) {
} catch (err) {
+ if (guard.toolRefused(req, res, err, toolOf(req))) return;
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });
console.error('[Process] Error:', err.message);
console.error('[Process] Error:', err.message);
res.status(422).json({ error: err.message || 'Image processing failed' });
res.status(422).json({ error: err.message || 'Image processing failed' });
@@ -190,7 +207,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
@@ -190,7 +207,7 @@ app.post('/api/process', burstLimiter, processLimiter, uploadSingle, async (req,
});
});
// ── Direct Download (inline preview) ─────────────────────────
// ── Direct Download (inline preview) ─────────────────────────
-app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, async (req, res) => {
+app.post('/api/process/direct', burstLimiter, processLimiter, guard.toolQuota(hostTool), uploadSingle, guard.admitUpload(toolOf), guard.heavy(toolOf), async (req, res) => {
try {
try {
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const toolId = req.body.tool || req.ctx.defaultOp || 'convert';
const tool = getTool(toolId);
const tool = getTool(toolId);
@@ -206,6 +223,7 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn
@@ -206,6 +223,7 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn
});
});
res.send(result.buffer);
res.send(result.buffer);
} catch (err) {
} catch (err) {
+ if (guard.toolRefused(req, res, err, toolOf(req))) return;
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });
if (err.status === 503) return contracts.http.sendProblem(res, 503, err.code, { detail: err.message, ctx: req.ov });
console.error('[Process/Direct] Error:', err.message);
console.error('[Process/Direct] Error:', err.message);
res.status(422).json({ error: err.message || 'Image processing failed' });
res.status(422).json({ error: err.message || 'Image processing failed' });
@@ -217,10 +235,11 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn
@@ -217,10 +235,11 @@ app.post('/api/process/direct', burstLimiter, processLimiter, uploadSingle, asyn
// followed over SSE, reattachable by id after a reload or a restart (apps/_shared/jobs).
// followed over SSE, reattachable by id after a reload or a restart (apps/_shared/jobs).
const jobs = jobsRuntime.setupJobs({
const jobs = jobsRuntime.setupJobs({
app, service: 'img', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
app, service: 'img', dataDir: path.resolve(__dirname, '..', config.dataDir), Database, contracts, sdk,
- getPublicKey: auth.getPublicKey, issuer: auth.ISSUER,
+ getPublicKey: guard.keys.get, issuer: config.networkUrl, guard,
define: defineJobs,
define: defineJobs,
receive: uploadSingle,
receive: uploadSingle,
- limiters: [burstLimiter, processLimiter],
+ limiters: [burstLimiter, processLimiter, guard.toolQuota(hostTool)],
+ jobTool: (req, type, input) => { const d = guard.toolForJob('img.process', String((input && input.tool) || req.ctx.defaultOp || 'convert'), req.ctx.toolId); return d ? d.id : null; },
defaults(req, input) {
defaults(req, input) {
const out = { ...input };
const out = { ...input };
if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';
if (!out.tool) out.tool = req.ctx.defaultOp || 'convert';
@@ -301,6 +320,7 @@ function shutdown() {
@@ -301,6 +320,7 @@ function shutdown() {
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
jobs.close(); // running jobs stay 'running' in data/jobs.db; the next boot re-queues them
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,48 +0,0 @@
@@ -1,48 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// Img.OpenVibe — Rate Limiting Middleware
-// Tiered rate limits: anonymous vs authenticated users.
-// Includes anti-abuse burst protection.
-// ═══════════════════════════════════════════════════════════════
-
-const rateLimit = require('express-rate-limit');
-
-/** General API rate limit */
-const apiLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 120 : 60,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Too many requests. Please try again later.' },
-});
-
-/**
- * Processing rate limit — stricter, tiered by auth.
- * Anonymous: 10 conversions/minute
- * Authenticated: 30 conversions/minute
- */
-const processLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: (req) => req.user ? 30 : 10,
- standardHeaders: true,
- legacyHeaders: false,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Processing rate limit reached. Sign in for higher limits or wait a moment.' },
-});
-
-/**
- * Anti-burst limiter — prevents rapid-fire abuse (bot attacks).
- * Very short window: max 4 requests per 5 seconds per IP.
- */
-const burstLimiter = rateLimit({
- windowMs: 5 * 1000,
- max: 4,
- standardHeaders: false,
- legacyHeaders: false,
- keyGenerator: (req) => req.ip,
- message: { error: 'Too many requests in quick succession. Please slow down.' },
-});
-
-module.exports = { apiLimiter, processLimiter, burstLimiter };
@@ -12,7 +12,9 @@
@@ -12,7 +12,9 @@
// it a HEIC upload answers 503 tools.unavailable. AVIF and other HEIF files go to sharp.
// it a HEIC upload answers 503 tools.unavailable. AVIF and other HEIF files go to sharp.
//
//
// open(buffer) is the one way the tools read an upload: it decodes once and hands out fresh
// open(buffer) is the one way the tools read an upload: it decodes once and hands out fresh
-// sharp pipelines over the decoded pixels.
+// sharp pipelines over the decoded pixels. No input larger than the guard's pixel limit (40
+// megapixels, TOOLS_MAX_INPUT_PIXELS; apps/_shared/guard/limits.js) is ever decoded: the size is
+// read from the header first, and every sharp pipeline carries limitInputPixels as the backstop.
// ═══════════════════════════════════════════════════════════════
// ═══════════════════════════════════════════════════════════════
const { spawn, spawnSync } = require('child_process');
const { spawn, spawnSync } = require('child_process');
@@ -21,8 +23,9 @@ const os = require('os');
@@ -21,8 +23,9 @@ const os = require('os');
const path = require('path');
const path = require('path');
const sharp = require('sharp');
const sharp = require('sharp');
const { createBinary, Unavailable } = require('../../../_shared/binaries');
const { createBinary, Unavailable } = require('../../../_shared/binaries');
+const guardLimits = require('../../../_shared/guard/limits');
-// Largest picture read from or written to BMP (and ICO): 100 megapixels, 400 MB of RGBA.
+// Largest picture written to BMP: 100 megapixels, 400 MB of RGBA (what is read is limited by inputPixels()).
const MAX_PIXELS = 100 * 1024 * 1024;
const MAX_PIXELS = 100 * 1024 * 1024;
const HEIF_TIMEOUT_MS = 60_000;
const HEIF_TIMEOUT_MS = 60_000;
@@ -30,6 +33,22 @@ class ImageError extends Error {
@@ -30,6 +33,22 @@ class ImageError extends Error {
constructor(message) { super(message); this.status = 422; this.expose = true; }
constructor(message) { super(message); this.status = 422; this.expose = true; }
}
}
+/** The largest input picture (width × height) any tool decodes. */
+const inputPixels = () => guardLimits.bounds().maxInputPixels;
+
+/** A picture over the pixel limit: 413, recorded by the guard as a hard refusal ('pixels'). */
+function tooManyPixels(width, height) {
+ const mp = (n) => `${Math.round(n / 1e5) / 10} megapixels`;
+ const err = new ImageError(`This picture is ${width}×${height} (${mp(width * height)}); the limit is ${mp(inputPixels())}.`);
+ err.status = 413;
+ err.code = 'tools.file.too_large';
+ err.guardReason = 'pixels';
+ return err;
+}
+function checkPixels(width, height) {
+ if (width * height > inputPixels()) throw tooManyPixels(width, height);
+}
+
// ── Format sniffing ──────────────────────────────────────────
// ── Format sniffing ──────────────────────────────────────────
const HEVC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);
const HEVC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);
@@ -96,7 +115,7 @@ function decodeDib(buf, dib, pixelsAt, ico) {
@@ -96,7 +115,7 @@ function decodeDib(buf, dib, pixelsAt, ico) {
const topDown = height < 0;
const topDown = height < 0;
height = Math.abs(height);
height = Math.abs(height);
if (!(width > 0 && height > 0)) throw new ImageError('This BMP file has no pixels');
if (!(width > 0 && height > 0)) throw new ImageError('This BMP file has no pixels');
- if (width * height > MAX_PIXELS) throw new ImageError(`This BMP picture is larger than ${MAX_PIXELS / 1024 / 1024} megapixels`);
+ checkPixels(width, height);
if (![1, 4, 8, 16, 24, 32].includes(bpp)) throw new ImageError(`${bpp}-bit BMP files are not supported`);
if (![1, 4, 8, 16, 24, 32].includes(bpp)) throw new ImageError(`${bpp}-bit BMP files are not supported`);
if (![0, 1, 2, 3, 6].includes(compression)) throw new ImageError('This BMP compression (JPEG/PNG inside BMP) is not supported');
if (![0, 1, 2, 3, 6].includes(compression)) throw new ImageError('This BMP compression (JPEG/PNG inside BMP) is not supported');
@@ -206,6 +225,17 @@ function decodeBmp(buf) {
@@ -206,6 +225,17 @@ function decodeBmp(buf) {
// ── ICO reader ───────────────────────────────────────────────
// ── ICO reader ───────────────────────────────────────────────
+/** The largest entry's size as the directory states it (0 means 256). */
+function icoLargest(buf) {
+ let best = null;
+ for (let i = 0; i < buf.readUInt16LE(4) && 6 + i * 16 + 16 <= buf.length; i++) {
+ const e = 6 + i * 16;
+ const width = buf[e] || 256, height = buf[e + 1] || 256;
+ if (!best || width * height > best.width * best.height) best = { width, height };
+ }
+ return best;
+}
+
/** The largest (then deepest) entry of an .ico → a sharp input: PNG bytes, or RGBA pixels. */
/** The largest (then deepest) entry of an .ico → a sharp input: PNG bytes, or RGBA pixels. */
function decodeIco(buf) {
function decodeIco(buf) {
const n = buf.readUInt16LE(4);
const n = buf.readUInt16LE(4);
@@ -272,20 +302,30 @@ function decodeHeic(buf) {
@@ -272,20 +302,30 @@ function decodeHeic(buf) {
* Decode an upload once. → { format: 'bmp'|'ico'|'heic'|<sharp format>, sharp(): a fresh pipeline }
* Decode an upload once. → { format: 'bmp'|'ico'|'heic'|<sharp format>, sharp(): a fresh pipeline }
*/
*/
async function open(buf) {
async function open(buf) {
+ const limitInputPixels = inputPixels();
+ /** Header size first (metadata reads the header only): a picture over the limit is refused before anything is decoded. */
+ const sized = async (input) => {
+ let meta;
+ try { meta = await sharp(input, { limitInputPixels: false }).metadata(); } catch { throw new ImageError('This file is not an image this tool can read'); }
+ checkPixels(meta.width || 0, meta.pageHeight || meta.height || 0);
+ return meta;
+ };
const kind = sniff(buf);
const kind = sniff(buf);
if (kind === 'bmp' || kind === 'ico') {
if (kind === 'bmp' || kind === 'ico') {
+ // An ICO entry's header says its size before it is decoded; PNG entries are checked by sharp's header.
+ if (kind === 'ico') { const e = icoLargest(buf); if (e) checkPixels(e.width, e.height); }
const d = kind === 'bmp' ? decodeBmp(buf) : decodeIco(buf);
const d = kind === 'bmp' ? decodeBmp(buf) : decodeIco(buf);
- if (d.png) return { format: kind, sharp: () => sharp(d.png) };
+ if (d.png) { await sized(d.png); return { format: kind, sharp: () => sharp(d.png, { limitInputPixels }) }; }
const raw = { width: d.width, height: d.height, channels: 4 };
const raw = { width: d.width, height: d.height, channels: 4 };
- return { format: kind, sharp: () => sharp(d.data, { raw }) };
+ return { format: kind, sharp: () => sharp(d.data, { raw, limitInputPixels }) };
}
}
if (kind === 'heic') {
if (kind === 'heic') {
const png = await decodeHeic(buf);
const png = await decodeHeic(buf);
- return { format: 'heic', sharp: () => sharp(png) };
+ await sized(png);
+ return { format: 'heic', sharp: () => sharp(png, { limitInputPixels }) };
}
}
- let format = null;
- try { format = (await sharp(buf).metadata()).format || null; } catch { throw new ImageError('This file is not an image this tool can read'); }
- return { format, sharp: () => sharp(buf) };
+ const meta = await sized(buf);
+ return { format: meta.format || null, sharp: () => sharp(buf, { limitInputPixels }) };
}
}
// ── BMP writer ───────────────────────────────────────────────
// ── BMP writer ───────────────────────────────────────────────
@@ -334,4 +374,4 @@ async function toBmp(pipeline) {
@@ -334,4 +374,4 @@ async function toBmp(pipeline) {
return { buffer: encodeBmp(data, info.width, info.height), width: info.width, height: info.height };
return { buffer: encodeBmp(data, info.width, info.height), width: info.width, height: info.height };
}
}
-module.exports = { open, sniff, decodeBmp, decodeIco, encodeBmp, toBmp, heif, ImageError, MAX_PIXELS, HEIC_SETTING_UP };
+module.exports = { open, sniff, decodeBmp, decodeIco, encodeBmp, toBmp, heif, ImageError, MAX_PIXELS, HEIC_SETTING_UP, inputPixels, checkPixels };
new file mode 100644
new file mode 100644
@@ -0,0 +1,80 @@
@@ -0,0 +1,80 @@
+'use strict';
+// The guard on the image satellite: no picture over the pixel limit is decoded (sharp's
+// limitInputPixels, checked from the header first; BMP and ICO headers too), sniffing refuses a file
+// that is not what its name and type say, the sync endpoint's refusals are problem+json, and the
+// descriptor tells the same limit. TOOLS_MAX_INPUT_PIXELS is lowered here so small pictures show it.
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+
+process.env.TOOLS_MAX_INPUT_PIXELS = '10000'; // before anything reads the limit
+const sharp = require('sharp');
+const codec = require('../server/tools/codec');
+const convert = require('../server/tools/convert');
+const { startApp } = require('../../_shared/test/spawn');
+
+(async () => {
+ // ── The codec: the header's size decides, before anything is decoded ──
+ const small = await sharp({ create: { width: 100, height: 100, channels: 3, background: '#336699' } }).png().toBuffer();
+ const big = await sharp({ create: { width: 200, height: 100, channels: 3, background: '#336699' } }).png().toBuffer();
+ assert.strictEqual(codec.inputPixels(), 10000);
+ const ok = await convert(small, { format: 'webp' });
+ assert.strictEqual(ok.mime, 'image/webp', '100×100 is within 10 000 pixels');
+ await assert.rejects(convert(big, { format: 'webp' }), (e) => e.status === 413 && e.code === 'tools.file.too_large' && e.guardReason === 'pixels' && /200×100/.test(e.message));
+ // A BMP header claiming 30 000 × 30 000 is refused without allocating its pixels.
+ const bmp = Buffer.alloc(54);
+ bmp.write('BM', 0, 'latin1'); bmp.writeUInt32LE(54, 10); bmp.writeUInt32LE(40, 14);
+ bmp.writeInt32LE(30000, 18); bmp.writeInt32LE(30000, 22); bmp.writeUInt16LE(1, 26); bmp.writeUInt16LE(24, 28);
+ await assert.rejects(codec.open(bmp), (e) => e.guardReason === 'pixels');
+ // An ICO whose directory says 256×256 (65 536 pixels) is refused from the directory.
+ const ico = Buffer.concat([Buffer.from([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 32, 0]), Buffer.from([8, 0, 0, 0, 22, 0, 0, 0]), small.subarray(0, 8)]);
+ await assert.rejects(codec.open(ico), (e) => e.guardReason === 'pixels');
+ // The descriptors say the same limit.
+ const { SPECS } = require('../server/descriptors');
+ assert.ok(SPECS.every(s => s.limits.maxPixels === 10000), 'every image tool\'s limits.maxPixels is the guard\'s');
+
+ // ── Through the app ──
+ const data = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-img-guard-'));
+ const app = await startApp('img', { DATA_DIR: data, UPLOADS_DIR: path.join(data, 'uploads'), OUTPUT_DIR: path.join(data, 'output'), TOOLS_MAX_INPUT_PIXELS: '10000' });
+ const post = (buf, name, type, p = '/api/process') => {
+ const f = new FormData();
+ f.append('file', new Blob([buf], { type }), name);
+ f.append('tool', 'convert');
+ f.append('format', 'png');
+ return fetch(`${app.base}${p}`, { method: 'POST', body: f });
+ };
+ try {
+ let r = await post(big, 'big.png', 'image/png');
+ assert.strictEqual(r.status, 413, 'the pixel limit applies in report mode too');
+ let body = await r.json();
+ assert.strictEqual(r.headers.get('content-type'), 'application/problem+json');
+ assert.strictEqual(body.code, 'tools.file.too_large');
+ r = await post(Buffer.from('%PDF-1.4\n%%EOF\n'), 'photo.png', 'image/png', '/api/process/direct');
+ assert.strictEqual(r.status, 415, 'a PDF called photo.png is refused');
+ assert.strictEqual((await r.json()).code, 'tools.file.unsupported_type');
+ r = await post(small, 'photo.gif', 'image/gif');
+ body = await r.json();
+ assert.strictEqual(r.status, 200, JSON.stringify(body));
+ assert.strictEqual(body.output.ext, 'png');
+ // The job API refuses the same picture the same way (hard limit): the job fails with the code.
+ const f = new FormData();
+ f.append('type', 'img.process');
+ f.append('input', JSON.stringify({ tool: 'convert', format: 'png' }));
+ f.append('file', new Blob([Buffer.from('not an image at all')], { type: 'image/png' }), 'x.png');
+ r = await fetch(`${app.base}/api/v1/jobs`, { method: 'POST', body: f });
+ assert.strictEqual(r.status, 415, 'a job upload is sniffed before it is accepted');
+ // Answers carry RateLimit headers and the sync endpoint is behind the semaphore.
+ assert.ok(r.headers.get('ratelimit-limit'), 'RateLimit-Limit');
+ const metrics = await (await fetch(`${app.base}/metrics`)).text();
+ assert.match(metrics, /tools_guard_refused_total\{reason="pixels",tool="convert"\} 1/);
+ assert.match(metrics, /tools_guard_refused_total\{reason="sniff",tool="convert"\} [12]/);
+ assert.match(metrics, /tools_guard_sync\{kind="limit"\} 2/);
+ assert.match(metrics, /tools_guard_enforcing 0/);
+ assert.ok(fs.existsSync(path.join(data, 'guard.db')), 'guard.db in the data directory');
+ } finally {
+ await app.kill();
+ fs.rmSync(data, { recursive: true, force: true });
+ }
+ console.log('img guard: pixel limit from headers (PNG, BMP, ICO) and in descriptors, sniffing on sync and job uploads, problem+json, metrics: all checks passed');
+})().catch((err) => { console.error(err); process.exit(1); });
@@ -24,19 +24,32 @@ const { internalOk } = require('../../_shared/internal-auth');
@@ -24,19 +24,32 @@ const { internalOk } = require('../../_shared/internal-auth');
const { hostGuard, stampedPage } = require('../../_shared/host-role');
const { hostGuard, stampedPage } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
+const { createPacer, Busy } = require('../../_shared/guard/semaphore');
const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');
const analyticsDbPath = path.join(__dirname, '..', 'data', 'analytics.db');
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
const analyticsDb = new Database(analyticsDbPath);
const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-maps', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-maps', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, else the address — the food app forwards its
+// visitor's), the tools-map quota on the data routes (descriptor maps: cost 2 a call) and the abuse log
+// (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.
+const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
+const guard = createGuard({
+ app: 'maps', dataDir: path.join(__dirname, '..', 'data'), Database, specs: require('./descriptors').SPECS,
+ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',
+ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),
+});
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready } = require('../../_shared/observe');
const { observe, checks: ready } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-maps', release: release.release,
service: 'tools-maps', release: release.release,
checks: [
checks: [
@@ -44,6 +57,7 @@ observe({
@@ -44,6 +57,7 @@ observe({
],
],
// The public data sources maps queries (OSM and others) are not probed here: each request says when one fails.
// The public data sources maps queries (OSM and others) are not probed here: each request says when one fails.
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
@@ -51,10 +65,10 @@ app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301,
@@ -51,10 +65,10 @@ app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301,
const cache = new NodeCache({ stdTTL: config.cache.search, checkperiod: 60 });
const cache = new NodeCache({ stdTTL: config.cache.search, checkperiod: 60 });
// ── Middleware ──────────────────────────────────────────────
// ── Middleware ──────────────────────────────────────────────
-// X-Forwarded-For is believed only from loopback hops: the host's nginx, the gateway and the food
-// app (which forwards its visitor's address). Maps listens on 0.0.0.0 by default, so a direct
+// X-Forwarded-For is believed from exactly one hop on loopback: the host's nginx, the gateway or the
+// food app (which forwards its visitor's address). Maps listens on 0.0.0.0 by default, so a direct
// caller's own header must not count.
// caller's own header must not count.
-app.set('trust proxy', 'loopback');
+app.set('trust proxy', TRUST_PROXY);
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
directives: {
directives: {
@@ -73,13 +87,12 @@ app.use(exceptRegistry(cors({ origin: ['https://maps.openvibe.tools', 'https://f
@@ -73,13 +87,12 @@ app.use(exceptRegistry(cors({ origin: ['https://maps.openvibe.tools', 'https://f
app.use(cookieParser());
app.use(cookieParser());
app.use(express.json());
app.use(express.json());
-// Rate limiter
-const apiLimiter = rateLimit({
- windowMs: 60 * 1000,
- max: 30,
- message: { error: 'Too many requests — slow down, traveler.' },
-});
-app.use('/api/', apiLimiter);
+// Rate limits: sign-in first, so limits know who is asking; everyone else counts by address (IPv6 /64).
+app.use(guard.identify);
+const apiLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60 * 1000, anonymous: 30, signedIn: 60, message: 'Too many requests — slow down, traveler.' });
+app.use('/api/', apiLimiter, guard.apiQuota);
+// The map's data lookups (OpenStreetMap, weather, food banks…): the tools-map quota, by descriptor.
+app.use(['/api/geocode', '/api/search', '/api/weather', '/api/terrain', '/api/food-banks', '/api/stores', '/api/foods', '/api/meal-plan'], guard.toolQuota('maps'));
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's survival map ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's survival map ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -129,6 +142,9 @@ const terrain = require('./sources/terrain');
@@ -129,6 +142,9 @@ const terrain = require('./sources/terrain');
const grocery = require('./sources/grocery');
const grocery = require('./sources/grocery');
// ── Geocode endpoint ───────────────────────────────────────
// ── Geocode endpoint ───────────────────────────────────────
+// Nominatim's usage policy is one request a second from us in all: answers are cached, and the misses
+// are paced (at most 20 waiting, none longer than 15 s; beyond that 503 with Retry-After).
+const nominatim = createPacer({ minIntervalMs: 1100, queue: 20, maxWaitMs: 15_000 });
app.get('/api/geocode', async (req, res) => {
app.get('/api/geocode', async (req, res) => {
const { q } = req.query;
const { q } = req.query;
if (!q) return res.status(400).json({ error: 'Missing query' });
if (!q) return res.status(400).json({ error: 'Missing query' });
@@ -139,11 +155,11 @@ app.get('/api/geocode', async (req, res) => {
@@ -139,11 +155,11 @@ app.get('/api/geocode', async (req, res) => {
try {
try {
const axios = require('axios');
const axios = require('axios');
- const resp = await axios.get('https://nominatim.openstreetmap.org/search', {
+ const resp = await nominatim.run(() => axios.get('https://nominatim.openstreetmap.org/search', {
params: { q, format: 'json', limit: 5, countrycodes: 'us,ca,mx' },
params: { q, format: 'json', limit: 5, countrycodes: 'us,ca,mx' },
headers: { 'User-Agent': 'Maps.OpenVibe/1.0 (maps.openvibe.tools)' },
headers: { 'User-Agent': 'Maps.OpenVibe/1.0 (maps.openvibe.tools)' },
timeout: 8000,
timeout: 8000,
- });
+ }));
const results = (resp.data || []).map(r => ({
const results = (resp.data || []).map(r => ({
lat: parseFloat(r.lat),
lat: parseFloat(r.lat),
lon: parseFloat(r.lon),
lon: parseFloat(r.lon),
@@ -153,6 +169,7 @@ app.get('/api/geocode', async (req, res) => {
@@ -153,6 +169,7 @@ app.get('/api/geocode', async (req, res) => {
cache.set(cacheKey, results, config.cache.geocode);
cache.set(cacheKey, results, config.cache.geocode);
res.json(results);
res.json(results);
} catch (err) {
} catch (err) {
+ if (err instanceof Busy) return res.status(503).set('Retry-After', '15').json({ error: 'Place search is busy; try again in a few seconds.' });
res.status(502).json({ error: 'Geocode failed' });
res.status(502).json({ error: 'Geocode failed' });
}
}
});
});
@@ -475,6 +492,7 @@ function shutdown() {
@@ -475,6 +492,7 @@ function shutdown() {
console.log('[Maps.OpenVibe] Shutting down...');
console.log('[Maps.OpenVibe] Shutting down...');
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
@@ -13,6 +13,7 @@ const OVERPASS_URL = 'https://overpass-api.de/api/interpreter';
@@ -13,6 +13,7 @@ const OVERPASS_URL = 'https://overpass-api.de/api/interpreter';
const _overpassQueue = [];
const _overpassQueue = [];
let _overpassRunning = 0;
let _overpassRunning = 0;
const MAX_CONCURRENT_OVERPASS = 2; // Max parallel Overpass requests
const MAX_CONCURRENT_OVERPASS = 2; // Max parallel Overpass requests
+const MAX_QUEUED_OVERPASS = 60; // More waiting than this: refuse (a search asks a handful at most)
const OVERPASS_RETRY_DELAYS = [2000, 5000, 12000]; // Backoff per retry
const OVERPASS_RETRY_DELAYS = [2000, 5000, 12000]; // Backoff per retry
async function _processOverpassQueue() {
async function _processOverpassQueue() {
@@ -61,6 +62,9 @@ async function _runOverpassJob({ query, timeout, resolve, reject, attempt = 0 })
@@ -61,6 +62,9 @@ async function _runOverpassJob({ query, timeout, resolve, reject, attempt = 0 })
*/
*/
function overpassQuery(query, timeout = 45000) {
function overpassQuery(query, timeout = 45000) {
return new Promise((resolve, reject) => {
return new Promise((resolve, reject) => {
+ if (_overpassQueue.length >= MAX_QUEUED_OVERPASS) {
+ return reject(Object.assign(new Error('OpenStreetMap is busy with other searches; try again in a minute.'), { status: 503, busy: true }));
+ }
_overpassQueue.push({ query, timeout, resolve, reject });
_overpassQueue.push({ query, timeout, resolve, reject });
_processOverpassQueue();
_processOverpassQueue();
});
});
@@ -17,32 +17,44 @@ const { internalOk } = require('../../_shared/internal-auth');
@@ -17,32 +17,44 @@ const { internalOk } = require('../../_shared/internal-auth');
const { hostGuard } = require('../../_shared/host-role');
const { hostGuard } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
const analyticsDbPath = path.resolve(__dirname, '..', process.env.DATA_DIR || 'data', 'analytics.db');
const analyticsDbPath = path.resolve(__dirname, '..', process.env.DATA_DIR || 'data', 'analytics.db');
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
fs.mkdirSync(path.dirname(analyticsDbPath), { recursive: true });
const analyticsDb = new Database(analyticsDbPath);
const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-text', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-text', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, else the address), the tools-api quota on
+// /api/ and the abuse log (data/guard.db). TOOLS_GUARD=report (default) records what it would refuse.
+const NETWORK_URL = process.env.OV_NETWORK_URL || 'https://openvibe.network';
+const guard = createGuard({
+ app: 'text', dataDir: path.resolve(__dirname, '..', process.env.DATA_DIR || 'data'), Database, specs: require('./descriptors').SPECS,
+ issuer: NETWORK_URL, networkUrl: NETWORK_URL, networkInternalUrl: process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000',
+ publicKeyFiles: [process.env.OV_NETWORK_PUBLIC_KEY].filter(Boolean),
+});
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready } = require('../../_shared/observe');
const { observe, checks: ready } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-text', release: release.release,
service: 'tools-text', release: release.release,
checks: [
checks: [
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; the text tools run in the browser and on this process' }),
ready.sqlite('analytics_db', analyticsDb, { required: false, description: 'visit analytics only; the text tools run in the browser and on this process' }),
],
],
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
-app.set('trust proxy', 2);
+app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address
app.use(helmet({
app.use(helmet({
contentSecurityPolicy: {
contentSecurityPolicy: {
directives: {
directives: {
@@ -73,8 +85,10 @@ app.use(exceptRegistry(cors({
@@ -73,8 +85,10 @@ app.use(exceptRegistry(cors({
credentials: true,
credentials: true,
})));
})));
-// ── Rate Limiting ────────────────────────────────────────────
-app.use(rateLimit({ windowMs: 60_000, max: 200 }));
+// ── Who is asking, then rate limits ──────────────────────────
+app.use(guard.identify);
+app.use(guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 200, signedIn: 400, message: 'Too many requests. Please try again later.' }));
+app.use('/api/', guard.apiQuota);
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's text and logo tools ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's text and logo tools ──
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
// Public (Access-Control-Allow-Origin *), cacheable (ETag), counted by the limiter above. The
@@ -161,6 +175,7 @@ function shutdown() {
@@ -161,6 +175,7 @@ function shutdown() {
console.log('[Text.OpenVibe] Shutting down...');
console.log('[Text.OpenVibe] Shutting down...');
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,120 +0,0 @@
@@ -1,120 +0,0 @@
-'use strict';
-
-// ═══════════════════════════════════════════════════════════════
-// YT.OpenVibe — JWT Auth (OpenVibe Network RS256 verification)
-// Lightweight auth — no DB, no linked_accounts.
-// Verifies the shared ov_token cookie offline against the
-// Network's public key and attaches decoded claims to req.user.
-// Key source: OV_NETWORK_PUBLIC_KEY file override, otherwise
-// fetched from the Network's /api/.well-known/jwks endpoint.
-// ═══════════════════════════════════════════════════════════════
-
-const jwt = require('jsonwebtoken');
-const fs = require('fs');
-const path = require('path');
-const config = require('./config');
-
-let publicKey = null;
-let retryTimer = null;
-
-const ISSUER = config.networkUrl;
-const JWKS_PATH = '/api/.well-known/jwks';
-const FETCH_TIMEOUT_MS = 5000;
-const RETRY_INTERVAL_MS = 60 * 1000;
-
-function loadKeyFromFile() {
- for (const p of config.publicKeyPaths) {
- try {
- const resolved = path.resolve(p);
- if (fs.existsSync(resolved)) {
- publicKey = fs.readFileSync(resolved, 'utf8');
- console.log(`[Auth] Loaded OpenVibe Network public key from ${resolved}`);
- return true;
- }
- } catch { /* try next */ }
- }
- return false;
-}
-
-async function fetchKeyFromNetwork() {
- for (const base of [config.networkInternalUrl, config.networkUrl]) {
- try {
- const res = await fetch(`${base}${JWKS_PATH}`, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) });
- if (!res.ok) continue;
- const body = await res.json();
- if (body && body.public_key) {
- publicKey = body.public_key;
- console.log(`[Auth] Loaded OpenVibe Network public key from ${base}${JWKS_PATH}`);
- return true;
- }
- } catch { /* try next base */ }
- }
- return false;
-}
-
-function loadPublicKey() {
- if (loadKeyFromFile()) return;
- fetchKeyFromNetwork().then((loaded) => {
- if (loaded || retryTimer) return;
- console.warn(`[Auth] OpenVibe Network unreachable — will retry every ${RETRY_INTERVAL_MS / 1000}s (users treated as anonymous until the key loads)`);
- retryTimer = setInterval(async () => {
- if (await fetchKeyFromNetwork()) {
- clearInterval(retryTimer);
- retryTimer = null;
- }
- }, RETRY_INTERVAL_MS);
- if (retryTimer.unref) retryTimer.unref();
- });
-}
-loadPublicKey();
-
-function extractToken(req) {
- const authHeader = req.headers.authorization;
- if (authHeader && authHeader.startsWith('Bearer ')) return authHeader.slice(7);
- if (req.cookies?.ov_token) return req.cookies.ov_token;
- if (req.cookies?.token) return req.cookies.token;
- return null;
-}
-
-function verifyToken(token) {
- if (!publicKey || !token) return null;
- try {
- return jwt.verify(token, publicKey, { algorithms: ['RS256'], issuer: ISSUER });
- } catch {
- return null;
- }
-}
-
-/**
- * Optional auth — attaches req.user if valid token present.
- * Never blocks the request; anonymous users get req.user = null.
- */
-function optionalAuth(req, _res, next) {
- const token = extractToken(req);
- if (token) {
- const decoded = verifyToken(token);
- if (decoded) {
- req.user = decoded;
- req.token = token;
- }
- }
- next();
-}
-
-/**
- * Required auth — 401 if no valid token.
- */
-function requireAuth(req, res, next) {
- const token = extractToken(req);
- if (!token) return res.status(401).json({ error: 'Authentication required' });
- const decoded = verifyToken(token);
- if (!decoded) return res.status(401).json({ error: 'Invalid or expired token' });
- req.user = decoded;
- req.token = token;
- next();
-}
-
-/** The Network public key (PEM) once loaded (readiness reports it). */
-function getPublicKey() { return publicKey; }
-
-module.exports = { extractToken, verifyToken, optionalAuth, requireAuth, loadPublicKey, getPublicKey };
@@ -14,12 +14,12 @@ const path = require('path');
@@ -14,12 +14,12 @@ const path = require('path');
const fs = require('fs');
const fs = require('fs');
const config = require('./config');
const config = require('./config');
-const { optionalAuth } = require('./auth');
const downloader = require('./downloader');
const downloader = require('./downloader');
const seo = require('./seo');
const seo = require('./seo');
const { hostGuard } = require('../../_shared/host-role');
const { hostGuard } = require('../../_shared/host-role');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createToolsApi, exceptRegistry } = require('../../_shared/tools/http');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
const { createLocalRegistry, requiresStatus } = require('../../_shared/tools/local');
+const { createGuard, TRUST_PROXY } = require('../../_shared/guard');
// ── Analytics ────────────────────────────────────────────────
// ── Analytics ────────────────────────────────────────────────
const Database = require('better-sqlite3');
const Database = require('better-sqlite3');
@@ -31,13 +31,22 @@ const analyticsDb = new Database(analyticsDbPath);
@@ -31,13 +31,22 @@ const analyticsDb = new Database(analyticsDbPath);
analyticsDb.pragma('journal_mode = WAL');
analyticsDb.pragma('journal_mode = WAL');
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-yt', { retention: { days: 30 } });
const analytics = new AnalyticsTracker(analyticsDb, 'openvibe-yt', { retention: { days: 30 } });
+// ── Guard (apps/_shared/guard) ───────────────────────────────
+// Who is asking (Network sign-in with aud openvibe.tools, the browser session, else the address),
+// the tools-download quota (a download costs 50, descriptor yt) and the abuse log (data/guard.db).
+// TOOLS_GUARD=report (default) records what it would refuse; the older limiters below stay until enforce.
+const guard = createGuard({
+ app: 'yt', dataDir: path.resolve(__dirname, '..', config.dataDir || 'data'), Database, specs: require('./descriptors').SPECS,
+ issuer: config.networkUrl, networkUrl: config.networkUrl, networkInternalUrl: config.networkInternalUrl, publicKeyFiles: config.publicKeyPaths,
+});
+
const app = express();
const app = express();
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
// What this deploy runs (ADR-016); the shared navbar's release-watch polls it on every tool host.
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'tools', root: require('path').join(__dirname, '..', '..', '..') });
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// Metrics (GET /metrics, direct loopback callers only) and GET /api/ready from this server's real
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
// dependencies (roadmap Track O). First, so the HTTP metrics see every request.
const { observe, checks: ready, which } = require('../../_shared/observe');
const { observe, checks: ready, which } = require('../../_shared/observe');
-observe({
+const obs = observe({
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
app, metrics: require('openvibe-shared/metrics'), ready: require('openvibe-shared/ready'),
service: 'tools-yt', release: release.release,
service: 'tools-yt', release: release.release,
checks: [
checks: [
@@ -46,19 +55,20 @@ observe({
@@ -46,19 +55,20 @@ observe({
ready.binary('yt_dlp', config.ytdlpPath, { description: 'every download runs yt-dlp' }),
ready.binary('yt_dlp', config.ytdlpPath, { description: 'every download runs yt-dlp' }),
ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'merging video+audio and audio conversion' }),
ready.binary('ffmpeg', process.env.FFMPEG_PATH || 'ffmpeg', { description: 'merging video+audio and audio conversion' }),
...(process.env.YT_COOKIES_FILE ? [ready.readableFile('yt_cookies', process.env.YT_COOKIES_FILE, { description: 'YT_COOKIES_FILE is set; yt-dlp is given these cookies' })] : []),
...(process.env.YT_COOKIES_FILE ? [ready.readableFile('yt_cookies', process.env.YT_COOKIES_FILE, { description: 'YT_COOKIES_FILE is set; yt-dlp is given these cookies' })] : []),
- ready.networkKey('network_key', require('./auth').getPublicKey, { description: 'recognises signed-in visitors (higher limits); anonymous use works without it' }),
+ ready.networkKey('network_key', guard.keys.get, { description: 'recognises signed-in visitors (higher limits); anonymous use works without it' }),
],
],
// Download progress streams last as long as the download.
// Download progress streams last as long as the download.
skip: (req) => /^\/api\/status\/[^/]+\/stream$/.test(req.path),
skip: (req) => /^\/api\/status\/[^/]+\/stream$/.test(req.path),
details: () => ({ yt_proxy_configured: !!String(process.env.YT_PROXY || '').trim() }),
details: () => ({ yt_proxy_configured: !!String(process.env.YT_PROXY || '').trim() }),
});
});
+guard.attachMetrics(obs.registry);
app.get('/release.json', release.handler);
app.get('/release.json', release.handler);
// Legal documents live on the apex; every tool host points there instead of answering 404.
// Legal documents live on the apex; every tool host points there instead of answering 404.
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
app.get(['/terms', '/privacy', '/dmca', '/tos'], (req, res) => res.redirect(301, 'https://openvibe.tools' + (req.path === '/tos' ? '/terms' : req.path)));
// ── Security ─────────────────────────────────────────────────
// ── Security ─────────────────────────────────────────────────
-app.set('trust proxy', 2);
+app.set('trust proxy', TRUST_PROXY); // one hop: the host's nginx (or the gateway) on loopback; req.ip is the only address
// API answers are live state, never revalidated: a 304 on /api/status froze the progress poll.
// API answers are live state, never revalidated: a 304 on /api/status froze the progress poll.
app.set('etag', false);
app.set('etag', false);
app.use(helmet({
app.use(helmet({
@@ -93,9 +103,12 @@ app.use(exceptRegistry(cors({
@@ -93,9 +103,12 @@ app.use(exceptRegistry(cors({
// ── Rate Limiting ────────────────────────────────────────────
// ── Rate Limiting ────────────────────────────────────────────
// Progress is one SSE connection, or a poll every 1–3 s when SSE is unavailable; the poll must
// Progress is one SSE connection, or a poll every 1–3 s when SSE is unavailable; the poll must
// not eat the general budget (60/min would cut a download off after a minute).
// not eat the general budget (60/min would cut a download off after a minute).
+// Sign-in is read first, so limits know who is asking; everyone else counts by address (IPv6 by /64).
const isStatusRoute = (req) => req.method === 'GET' && /^\/status\/[a-f0-9]+(\/stream)?$/.test(req.path);
const isStatusRoute = (req) => req.method === 'GET' && /^\/status\/[a-f0-9]+(\/stream)?$/.test(req.path);
-app.use('/api/', rateLimit({ windowMs: 60_000, max: 60, skip: isStatusRoute }));
-app.use('/api/status/', rateLimit({ windowMs: 60_000, max: 240 }));
+app.use(guard.identify);
+const apiLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 60, signedIn: 120, message: 'Too many requests. Please try again later.' });
+app.use('/api/', (req, res, next) => (isStatusRoute(req) ? next() : apiLimiter(req, res, (err) => (err ? next(err) : guard.apiQuota(req, res, next)))));
+app.use('/api/status/', guard.legacyLimiter(rateLimit, { windowMs: 60_000, anonymous: 240, signedIn: 240, message: 'Too many requests. Please try again later.' }));
app.use('/api/', (_req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); });
app.use('/api/', (_req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); });
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's YouTube downloader ──
// ── Tool registry (ADR-027): GET /api/v1/tools[/:id[/schema]] for this app's YouTube downloader ──
@@ -107,15 +120,18 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
@@ -107,15 +120,18 @@ app.use(createToolsApi({ snapshot: toolRegistry.snapshot }));
// ── Analytics Middleware ─────────────────────────────────────
// ── Analytics Middleware ─────────────────────────────────────
app.use(analytics.middleware());
app.use(analytics.middleware());
-const downloadLimiter = rateLimit({
- windowMs: 60 * 60 * 1000, // 1 hour
- max: (req) => req.user ? config.rateLimit.authedPerHour : config.rateLimit.anonPerHour,
- keyGenerator: (req) => req.user?.sub || req.user?.id || req.ip,
- message: { error: 'Download limit reached. Sign in for more downloads or wait an hour.' },
-});
-
-// ── Auth ─────────────────────────────────────────────────────
-app.use(optionalAuth);
+// Downloads: the older hourly limit (5 anonymous, 20 signed in) until enforce mode, and the guard's
+// tools-download quota (descriptor yt: cost 50) in both. A download belongs to whoever started it (the
+// browser session cookie, minted on the start, or a sign-in): its status, progress stream, file and
+// cancel answer 404 to anyone else, the same as for an id that does not exist.
+const owners = new Map(); // download id → owner
+const ownsDownload = (req, res, id) => {
+ const owner = owners.get(String(id || ''));
+ const who = guard.resolveCaller(req, res);
+ return !!owner && !!who.owner && owner === who.owner;
+};
+setInterval(() => { for (const id of owners.keys()) if (!downloader.getStatus(id) && !downloader.getFile(id)) owners.delete(id); }, 10 * 60_000).unref();
+const downloadLimiter = guard.legacyLimiter(rateLimit, { windowMs: 60 * 60 * 1000, anonymous: config.rateLimit.anonPerHour, signedIn: config.rateLimit.authedPerHour, message: 'Download limit reached. Sign in for more downloads or wait an hour.' });
// ── Hosts ────────────────────────────────────────────────────
// ── Hosts ────────────────────────────────────────────────────
// Pages are only rendered for hosts this tool serves (or that the gateway vouches for with
// Pages are only rendered for hosts this tool serves (or that the gateway vouches for with
@@ -150,7 +166,7 @@ app.post('/api/info', async (req, res) => {
@@ -150,7 +166,7 @@ app.post('/api/info', async (req, res) => {
});
});
// Start download
// Start download
-app.post('/api/download', downloadLimiter, async (req, res) => {
+app.post('/api/download', downloadLimiter, guard.toolQuota('yt'), async (req, res) => {
const { url, quality, title } = req.body;
const { url, quality, title } = req.body;
if (!url) return res.status(400).json({ error: 'URL is required' });
if (!url) return res.status(400).json({ error: 'URL is required' });
if (!downloader.isValidUrl(url)) return res.status(400).json({ error: 'Only YouTube URLs are supported' });
if (!downloader.isValidUrl(url)) return res.status(400).json({ error: 'Only YouTube URLs are supported' });
@@ -159,7 +175,9 @@ app.post('/api/download', downloadLimiter, async (req, res) => {
@@ -159,7 +175,9 @@ app.post('/api/download', downloadLimiter, async (req, res) => {
if (known && known.downloadable === false) return res.status(422).json({ error: known.reason });
if (known && known.downloadable === false) return res.status(422).json({ error: known.reason });
try {
try {
+ const who = guard.resolveCaller(req, res, { create: true });
const { id } = await downloader.startDownload(url, quality || 'best', { title: typeof title === 'string' ? title : '' });
const { id } = await downloader.startDownload(url, quality || 'best', { title: typeof title === 'string' ? title : '' });
+ owners.set(id, who.owner);
res.json({ success: true, id, statusUrl: `/api/status/${id}`, streamUrl: `/api/status/${id}/stream` });
res.json({ success: true, id, statusUrl: `/api/status/${id}`, streamUrl: `/api/status/${id}/stream` });
} catch (err) {
} catch (err) {
console.error('[Download] Error:', err.message);
console.error('[Download] Error:', err.message);
@@ -169,7 +187,7 @@ app.post('/api/download', downloadLimiter, async (req, res) => {
@@ -169,7 +187,7 @@ app.post('/api/download', downloadLimiter, async (req, res) => {
// Download status (poll for progress)
// Download status (poll for progress)
app.get('/api/status/:id', (req, res) => {
app.get('/api/status/:id', (req, res) => {
- const status = downloader.getStatus(req.params.id);
+ const status = ownsDownload(req, res, req.params.id) ? downloader.getStatus(req.params.id) : null;
if (!status) return res.status(404).json({ error: 'Download not found' });
if (!status) return res.status(404).json({ error: 'Download not found' });
res.json(status);
res.json(status);
});
});
@@ -186,6 +204,7 @@ const SSE_EVENT = { downloading: 'progress', done: 'complete', error: 'failed' }
@@ -186,6 +204,7 @@ const SSE_EVENT = { downloading: 'progress', done: 'complete', error: 'failed' }
app.get('/api/status/:id/stream', (req, res) => {
app.get('/api/status/:id/stream', (req, res) => {
const id = req.params.id;
const id = req.params.id;
+ if (!ownsDownload(req, res, id)) return res.status(404).json({ error: 'Download not found' });
res.writeHead(200, {
res.writeHead(200, {
'Content-Type': 'text/event-stream; charset=utf-8',
'Content-Type': 'text/event-stream; charset=utf-8',
@@ -235,14 +254,14 @@ app.get('/api/status/:id/stream', (req, res) => {
@@ -235,14 +254,14 @@ app.get('/api/status/:id/stream', (req, res) => {
// Cancel a running download (kills yt-dlp) or discard a finished file
// Cancel a running download (kills yt-dlp) or discard a finished file
app.delete('/api/download/:id', (req, res) => {
app.delete('/api/download/:id', (req, res) => {
- const result = downloader.cancelDownload(req.params.id);
+ const result = ownsDownload(req, res, req.params.id) ? downloader.cancelDownload(req.params.id) : null;
if (!result) return res.status(404).json({ error: 'Download not found' });
if (!result) return res.status(404).json({ error: 'Download not found' });
res.json({ success: true, ...result });
res.json({ success: true, ...result });
});
});
// Serve downloaded file
// Serve downloaded file
app.get('/api/download/:id', (req, res) => {
app.get('/api/download/:id', (req, res) => {
- const entry = downloader.getFile(req.params.id);
+ const entry = ownsDownload(req, res, req.params.id) ? downloader.getFile(req.params.id) : null;
if (!entry) return res.status(404).json({ error: 'File not found or expired' });
if (!entry) return res.status(404).json({ error: 'File not found or expired' });
res.set({
res.set({
@@ -306,6 +325,7 @@ function shutdown() {
@@ -306,6 +325,7 @@ function shutdown() {
console.log('[YT.OpenVibe] Shutting down...');
console.log('[YT.OpenVibe] Shutting down...');
analytics.destroy();
analytics.destroy();
analyticsDb.close();
analyticsDb.close();
+ guard.close();
server.close(() => process.exit(0));
server.close(() => process.exit(0));
setTimeout(() => process.exit(1), 5000);
setTimeout(() => process.exit(1), 5000);
}
}
@@ -89,8 +89,14 @@ process.exit(0); // the match filter skipped it: no file
@@ -89,8 +89,14 @@ process.exit(0); // the match filter skipped it: no file
r = await post('/api/download', { url: 'https://www.youtube.com/watch?v=SHORTxx1234', quality: 'best' });
r = await post('/api/download', { url: 'https://www.youtube.com/watch?v=SHORTxx1234', quality: 'best' });
body = await r.json();
body = await r.json();
assert.strictEqual(r.status, 200, JSON.stringify(body));
assert.strictEqual(r.status, 200, JSON.stringify(body));
+ // The download belongs to the browser session the start minted: only it sees the status.
+ const cookie = String(r.headers.get('set-cookie') || '').split(';')[0];
+ assert.match(cookie, /^ov_tools_jobs=/, 'starting a download starts a session');
+ assert.strictEqual((await fetch(`${app.base}/api/status/${body.id}`)).status, 404, 'someone else gets 404');
+ assert.strictEqual((await fetch(`${app.base}/api/status/${body.id}/stream`)).status, 404);
+ assert.strictEqual((await fetch(`${app.base}/api/download/${body.id}`, { method: 'DELETE' })).status, 404, 'and cannot cancel it');
let st;
let st;
- for (let i = 0; i < 50; i++) { st = await (await fetch(`${app.base}/api/status/${body.id}`)).json(); if (st.status !== 'downloading') break; await sleep(50); }
+ for (let i = 0; i < 50; i++) { st = await (await fetch(`${app.base}/api/status/${body.id}`, { headers: { cookie } })).json(); if (st.status !== 'downloading') break; await sleep(50); }
const args = JSON.parse(fs.readFileSync(path.join(data, 'args.json'), 'utf8'));
const args = JSON.parse(fs.readFileSync(path.join(data, 'args.json'), 'utf8'));
assert.strictEqual(args[args.indexOf('--match-filter') + 1], 'duration<=600 & !is_live');
assert.strictEqual(args[args.indexOf('--match-filter') + 1], 'duration<=600 & !is_live');
assert.strictEqual(args[args.indexOf('--max-filesize') + 1], '100M');
assert.strictEqual(args[args.indexOf('--max-filesize') + 1], '100M');