Commits

main

object.hash job: sha256 for local copies in bounded batches, recorded on the object (content_hash, with metadata.hash_basis = the file and size it read) and on its local location (checksum, verified_at). Only 14 of 2,839 ready objects had a hash, so scheduled verification's hash check and hash reconciliation were inert. Queued per tenant with work every MEDIA_HASH_INTERVAL_MIN (15; 0 = on demand), light lane, 50 objects and MEDIA_HASH_BUDGET_MB (4096) per run (the first file always fits, up to MEDIA_HASH_MAX_FILE_MB 20480), or one object through the jobs API. It skips a file changed in the last MEDIA_HASH_SETTLE_S (120 s) or while it was read, and a local copy whose size differs from the object's (another version), and writes only if the object still has no hash and the location still points at the file it read. A re-projection that finds the local copy at another path or size (a remux in place, a re-cut, a regenerated thumbnail) now drops a hash this job made and clears the location checksum, so a hash never goes stale and verification never condemns a legitimately rewritten file; a copy that only moved to B2/R2 keeps it. Finished object.hash jobs are pruned like thumbnail jobs. test/content-hash.test.js: recording, skips, count and byte bounds, stale-hash invalidation, the B2 move, verification catching same-size corruption, scheduling and pruning.OpenVibers · 24 Sept 2026 e5e6f09
trust proxy is loopback only (was true): req.ip is the X-Forwarded-For that the local nginx set when a request came through it, and the socket address for anyone else, so a caller reaching port 4100 directly (HOST defaults to 0.0.0.0) can no longer choose its IP. The view counter read CF-Connecting-IP / X-Forwarded-For itself, from any peer; it now keys on req.ip through server/client-ip.js, like paste limits and the live-frame rate limit. The unused getRequesterIp header reader in public/routes.js is gone. test/trust-proxy.test.js checks the setting on an app (a loopback proxy's forwarded address is believed, a non-loopback peer's X-Forwarded-For and CF-Connecting-IP are not), and boots the real server: forwarded clients get their own live-frame budgets, while a direct caller rotating X-Forwarded-For is rate-limited as one address (it fails with trust proxy true). README: client addresses, and the new jobs and scripts in the layout.OpenVibers · 24 Sept 2026 47b4349
vod.duration.reconcile: stored VOD durations are compared with a measurement of the real file in bounded batches, offloaded VODs included, and clearly wrong values are repaired (the health scanner only repaired durations <= 0 and skipped the 674 of 906 VODs on B2/R2). A local file is probed where it is; a B2/R2 copy through a presigned GET URL, which ffprobe range-reads (nothing is downloaded). Verdicts: ok (within 2 s), mismatch (reported only), wrong (more than 60 s and 2% off), missing (nothing stored, the file measures), unmeasurable, skipped. A wrong or missing value is written only when a second measurement confirms it: a stream-copy pass over a local file's packets (the packets win when the header disagrees, duration_source remux), or for a remote copy the container agreeing with its own streams (confirm_remote reads the whole copy instead). A repair writes duration_seconds, probe_duration_seconds and duration_source only while the row still holds the value read, and re-projects the object. Every run writes a JSON report under <data>/reports. The job (heavy lane; params limit 1-200, after_id, apply, confirm_remote) walks each tenant's library from a kept cursor; it is not scheduled unless MEDIA_DURATION_RECONCILE_HOURS > 0 and scheduled runs only report unless MEDIA_DURATION_RECONCILE_APPLY=1. scripts/vod-duration-reconcile.js is the operator's: a dry run by default (--all walks every batch), --apply --backup <file.json> takes a checked online database copy first and keeps the report as the rollback file, --rollback <file> [--apply] restores rows that still hold what the repair wrote. scripts/vods-orphans-report.js reports on the B2 vods-orphans/ prefix (hazard H3): each object matched to a VOD or clip by file name, storage key or the recorder's vod-<app>-<id>-<ms> name, its canonical copy HEADed and its local file checked, with a recommendation (keep_only_copy, keep_until_offloaded, delete_duplicate, review_size_differs, delete_row_gone, review_row_mismatch, review_unknown) and totals in objects and bytes. It only lists and HEADs; nothing is deleted, moved or written. vod-storage gains a read-only listObjects. test/duration-reconcile.test.js runs both against a fake S3 through the real SDK with real ffmpeg files.OpenVibers · 24 Sept 2026 20a602e
vod.finalize job: a recording whose finalize failed or never ran is retried on media_jobs with backoff, instead of a one-shot boot sweep whose failure was swallowed and a stat failure that only cleared is_recording. The orphan sweep (in the worker: at boot and every MEDIA_FINALIZE_SWEEP_S, 120 s) queues one job per vods row still is_recording = 1 that no recorder, chunk upload or finalize holds and whose file has been idle for MEDIA_FINALIZE_ORPHAN_GRACE_S (180 s; a browser chunk upload resuming after a restart keeps writing its file, so it is no longer finalized under the uploader). Finalize queues one itself (first run in 5 min) when it cannot settle a recording: nothing measurable, the file could not be stat'ed, or it threw. The job re-runs finalize (fromJob, so it queues nothing more), succeeds with the outcome (ready with duration and source, needs_review for short, corrupt, deleted for an empty file, skipped when a recording or chunk upload still holds it or it is already final) and otherwise fails with a retryable code (unmeasurable, stat_failed, finalize_threw): 5 min, 15 min, 45 min, 2 h 15, 6 h, 6 attempts, after which the VOD stays needs_review and hidden. It never stops a live recording. New finalize lane (MEDIA_JOBS_FINALIZE_CONCURRENCY, 1) that runs while recording, like the recorder's own finalize. Duplicate requests join the queued or running job. Without the worker (MEDIA_JOBS_ENABLED=0) the boot sweep finalizes directly, as before. The v2 jobs API accepts vod.finalize for the tenant's own VODs. test/vod-finalize-job.test.js.OpenVibers · 24 Sept 2026 24f1962
VOD durations are measured, never estimated: finalize stores what the file says and vods.duration_source says where it came from. probe = ffprobe's container duration of the finished file; remux = the last packet time of a stream-copy pass (the finalize remux reports it, or a separate pass into the null muxer when the remux gave none), used when the header has no duration or claims far more than the packets (1.5x + 30 s: the header is corrupt); unknown = nothing measurable, stored as 0 with health_status needs_review (probe_failed or inflated_duration) and hidden, where the old code kept the wall clock. The recorder's own start time only bounds a value (longer than 1.5x its run + 30 s is refused); an orphan finalized after a restart has no bound and is never measured as now - created_at (it used to be, so a boot-time finalize of a 2-day-old row could store 2 days). The short, corrupt and stat-failure outcomes now also replace the recorder's live elapsed-time estimate (they left it in duration_seconds), as does the recorder's failed-finalize quarantine. A later finalize that measures a VOD lifts the quarantine a failed attempt left and restores the owner's visibility. Merge/remux failures are recorded as health issues (merge_failed, remux_failed) instead of being ignored. The health job's duration repair and the GET /vods/:id self-heal record source probe. test/vod-duration.test.js covers the choice, every outcome and a real ffmpeg orphan.OpenVibers · 24 Sept 2026 cc84bf1
objects-verify test: read the re-upload from the run that first checked the object, not from media_verifications, which keeps only the latest verdict; when batch rotation re-verified the object in a later run (same-second verified_at ties), the detail had no re-upload left and the test failed intermittently in CIOpenVibers · 23 Sept 2026 0724119
Opening the database no longer fails a clip being cut: getDb() ran the restart recovery ('processing' clips → failed) every time any process opened media.db, so a backfill or operator script run on the host while the service cut a clip failed that clip under it. The recovery stays where the server boots (vod/clip-jobs.start(), which also queues the re-cut); test/db-open-keeps-clips.test.js opens the database from a second process and checks the clip is untouchedOpenVibers · 23 Sept 2026 6c2d384
gitleaks: allow the owner-subject test's made-up INTERNAL_API_KEY, matched on path and valueOpenVibers · 23 Sept 2026 ce6a02a
Owner subjects: every media object that names an app-local owner (owner_app + owner_user_id) gets the owner's canonical Network subject in owner_subject, so Live's lineage resolver and other consumers read owner.subject instead of translating Live user ids (roadmap D01/D20; 0 of 3,032 user-owned objects had one). scripts/backfill-owner-subject.js resolves the owners through Network's POST /internal/identity/resolve-batch (500 ids per call). It is a dry run by default and prints per tenant the objects to fill, already set, unresolvable, in an unsupported tenant and with no owner. --apply requires --backup <file.json>: it takes an online backup to <file>.media.db and checks it with integrity_check, writes the rows it will change to <file.json>, then fills them 500 per transaction. It never writes over a non-null owner_subject or a row whose owner changed. --rollback <file.json> [--apply] clears only the rows that still carry the subject the backfill wrote. The script opens SQLite directly, so none of the service's boot work (marking in-progress clip cuts failed) runs. New objects: X-OV-Subject is still stored at creation, and a reconcile job (every MEDIA_OWNER_SUBJECT_INTERVAL_MIN, default 10; MEDIA_OWNER_SUBJECT_SYNC=0 stops it) fills every object that names only an owner id: projections, and v2 uploads that send only user_id. It asks Network only when something is missing, and uploads never wait on it. A re-projection to a different owner drops the subject until the job resolves the new one. Network is asked with a service token (identity.subject.resolve, Network c3b7cf3), and with INTERNAL_API_KEY until Network grants it. Only live ids are resolved (SOURCE_SYSTEMS); no other tenant stores owner ids todayOpenVibers · 23 Sept 2026 31c2899
openvibe-shared v1.5.1 (Track R release manifest), openvibe-contracts v0.33.0, openvibe-sdk v0.5.0OpenVibers · 23 Sept 2026 0c27f0b
Restore drills: MEDIA_DRILL=1 makes Media a read-only instance on a restored copy of media.db, so `ovhost drill media` can boot one next to production. It refuses to start unless DB_PATH is set and outside the checkout and /opt/openvibe.media, HOST is loopback and PORT is not 4100, before the database is opened. It then starts only its HTTP server: no app seeding or JWKS refresh, no tiering sweep, health job, junk sweep, clip re-cuts, copy verification, jobs worker, disk guardian, thumbnail cleanup, object purge, backfill, orphan-recording finalize or Events relay, and webhooks are never sent (their URLs in the apps table are production's). It writes no file and creates no directory, and never opens a file path from the database: every byte route (/v and /c bytes, /t, /a, /f, /o, paste screenshots, live frames, legacy thumbnails) answers 503 while watch pages and /browse still render. Everything but GET/HEAD/OPTIONS answers 403, /auth/* 503, and net/fetch/child_process/dgram/listen are guarded so nothing leaves the process (only git runs: no ffmpeg or ffprobe). /api/ready drops the storage and remote-tier checks and says mode: drill. test/drill-mode.test.js boots the real server that way against a copy whose app has a webhook URL and checks each pointOpenVibers · 23 Sept 2026 6e74eb0
Clips say who made them: POST /clips keeps auto_generated and the description when the app itself asks (an app-key call with no X-OV-User-Id), and never when the call acts for one of the app's users. Both fields were dropped, so every AI auto-clip Live cut since the split was stored as a person's clip. PUT /clips/:id lets the app mark an existing clip machine-made or not (403 when acting for a user), so Live can correct the ones it already made from its own logOpenVibers · 23 Sept 2026 ee39be1
Job events carry a projection, not the job: media.job.<transition> payloads are queue.jobEvent (id, app_id, object_id, type, status, attempts, max_attempts, error_code, cancel_requested, has_result, run_after, decided_at, created_at, updated_at, started_at, finished_at), times in ISO 8601 UTC or nullOpenVibers · 23 Sept 2026 ad2217b
Docs: the single-part PUT's refusals include the multipart size, content type and open-session answersOpenVibers · 23 Sept 2026 5d44df5
Multipart complete is idempotent: completing a session that already completed (a retry after its answer was lost) returns the ready object instead of 409, as the single-part complete does; the SDK objects client (openvibe-sdk 0.5.0) retries it. Tenant CORS allows the SDK's traceparent and X-OpenVibe-Request-Id headers next to X-Upload-Token, X-Content-SHA256 and Idempotency-Key, so a browser can send parts to presigned URLsOpenVibers · 23 Sept 2026 0ffbde5
Wave 4: media job system, multipart and presigned uploads, R2 eviction drillOpenVibers · 23 Sept 2026 d006749
/p/:slug/screenshot: a slug Media doesn't hold (pastes made in Community since the move) goes to Community like /p/:slug and /raw; a private one answers the same way. Hero-moment thumbnails linked here and showed broken imagesOpenVibers · 23 Sept 2026 9de4b25
CI: run the shared security workflow (gitleaks secret scan + dependency audit, Track Q)OpenVibers · 23 Sept 2026 454dc09
Media's Events outbox row is written in the state change's transaction: webhooks.announce() runs the change (VOD ready/quarantine/delete, clip ready/failed, object ready + invariant row) and events.record() in ONE SQLite transaction, then wakes the relay and sends the webhook, so an outcome is never lost (committed without its event) or a phantom (event for a rolled-back change); a failed outbox insert rolls the change back. The thumbnail is generated before the ready commit so the payload still carries it. Webhook bodies carry event_id = the durable event's id, so an app reading both paths (Live's webhook → Events transition) handles each outcome once. Lifecycle events are priority important (roadmap §6.3; storage.recovered stays low). test/outbox-transaction.test.js covers announce, outbox-insert failure, change failure, and the clip-jobs and finalize call sitesOpenVibers · 23 Sept 2026 cbf8410
Scheduled copy verification: every 10 min verify the 50 least recently verified ready objects (local stat + sha256 up to 64 MB, B2/R2 HEAD + size), record verdicts on media_locations, media_verifications and media_verify_runs, restore a missing remote copy from a verified-good local copy (2 per run; corrupt remote only with MEDIA_VERIFY_REPAIR_CORRUPT=1), never delete anything; media_objects_no_good_copy on /metrics and an optional object_copies check in /api/ready (degraded, never not-ready); scripts/no-good-copy-report.js lists those objects with owners, references and locations, read-onlyOpenVibers · 23 Sept 2026 e42af9e
Docs: STATUS.json and README match production (2026-09-23)OpenVibers · 23 Sept 2026 22ba4aa
Private items answer exactly like missing ones: /v and /c returned 403 "This media is private", which confirmed that a private recording (or, through the basename form of /v, a file name) exists; now the same 404 and body as an unknown id, for the bytes and the watch page. Same rule on /v/:id/transcript.json (legacy rows with no visibility and is_public 0 count as private, clips-only sources are hidden like on /v), the legacy /api/thumbnails redirect (no longer hands out a private row's current thumbnail URL), /p/:slug/raw (no screenshot bounce or Community redirect that only real private pastes got), /o/:id (a deleted private object said 410 before the signature check), and the v1 detail routes when an app acts for a user via X-OV-User-Id (GET /vods/:id, /clips/:id, cutting a clip from someone else's private VOD, GET /pastes/:slug, which also no longer counts a view before refusing). test/private-not-found.test.js compares every refused answer with the missing-id answerOpenVibers · 23 Sept 2026 dfb2172
systemd: TimeoutStopSec 30 → 80 so a stop during a recording lets the recorder's 60 s trailer grace (and the 70 s forced exit) finish instead of SIGKILL cutting the fileOpenVibers · 23 Sept 2026 2601b0e
nginx: client address headers only from $remote_addr (realip). X-Forwarded-For was appended to and CF-Connecting-IP passed through from the client, so a request reaching the origin without Cloudflare (DNS-only host or the bare IP) chose the address the service bans and rate-limits byOpenVibers · 23 Sept 2026 4b99e02
Developer-project tenants: no inline HTML/SVG from /f, race-free files quota, retained deletes countOpenVibers · 23 Sept 2026 20189e9
openvibe-contracts v0.28.0OpenVibers · 23 Sept 2026 1c8f67d
Developer-project tenants (Wave 20, ADR-014): app tokens get a Media tenant per projectOpenVibers · 23 Sept 2026 bfc9e1b
Token-only tenants for wiki and blog (10 GB each)OpenVibers · 23 Sept 2026 d3f81b9
Token-only tenants for tools (20 GB, job results) and games (10 GB, map-editor assets)OpenVibers · 23 Sept 2026 3007461
Truthful GET /api/ready and loopback-only GET /metrics (openvibe-shared v1.3.0)OpenVibers · 23 Sept 2026 e5525bd

More commits