- JavaScript 98.5%
- PLpgSQL 1.2%
- Shell 0.3%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
|
||
| .github/workflows | ||
| deploy | ||
| docs | ||
| migrations | ||
| scripts | ||
| server | ||
| test | ||
| .env.example | ||
| .gitignore | ||
| .gitleaks.toml | ||
| CHANGELOG.md | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| SECURITY.md | ||
| STATUS.json | ||
OpenVibe.Media
Purpose
Standalone multi-tenant media service (port 4100) extracted from the
OpenVibe.Live predecessor. Owns VOD ingest/recording, storage tiering
(local → Backblaze B2 → Cloudflare R2), clips, generic files, the canonical
object model and thumbnails, for all OpenVibe apps (live, games, tools,
network, community) and developer projects. Pastes moved to OpenVibe.Community
on 2026-09-22; Media's read-only paste API is gone (T10 step 2), /p/:slug only
redirects there and legacy screenshots redirect to media objects (Pastes).
Implements Media API v1 from ../CONTRACTS.md.
npm install
cp .env.example .env # the first-party tenants are created at boot (token-only)
npm start # node server/index.js
Requires Node ≥ 20, ffmpeg/ffprobe on PATH. PostgreSQL 18 and Valkey 9 (OpenVibe.Host roles/data/, ADR-035);
without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
Owns
- VODs (recording from RTMP or RTP, chunk ingest, finalize), clips (cutting), thumbnails, generic files
and avatars, in one PostgreSQL database (
ov_mediaon the host's data role; schema in migrations/) and the local media directories - the canonical object model (
/api/v2: objects, locations, variants, holds, derivatives, lifecycle, namespaces) and its jobs (media_jobs) - storage tiering (local, Backblaze B2, Cloudflare R2), copy verification and the disk guardian
- tenants (
apps), developer-project tenants, public serving (/v,/c,/t,/f,/o) and the object explorer (/me) - the
media.*events and Media's Search documents
Does not own
- pastes since 2026-09-22 (OpenVibe.Community; Media keeps legacy screenshot files through media objects)
- channels, streams and who a VOD belongs to (OpenVibe.Live answers lineage), identity (OpenVibe.Network)
- AI analysis of recordings (OpenVibe.AI reads them through signed URLs)
Depends on
- OpenVibe.Network (JWKS, visitor SSO as OAuth client
media,identity.subject.resolve, account export and deletion), OpenVibe.Live (live.lineage.resolve), OpenVibe.Events (the outbox relay and the account and revocation subscriptions) - Backblaze B2 and Cloudflare R2 (S3 API) when configured;
ffmpeg/ffprobeon the host openvibe-contractsv0.216.0,openvibe-sdkv0.65.0 (tokens, events outbox, usage readings, per-actor limits),openvibe-sharedv4.15.2, pinned by release tarball
Capabilities
Implemented here (the service manifest's capabilities, audience openvibe.media, checked per
namespace against the token's ns): media.object.upload, media.object.read, media.object.list,
media.object.delete, media.upload.create, media.derivative.create, media.derivative.read,
media.lifecycle.read and media.lifecycle.transition. First-party apps also use their tenant API
key (Tenancy & auth).
| Capability | Audience | Route | Notes |
|---|---|---|---|
media.resource.read |
first-party | GET /api/v1/resources, GET /api/v1/resources/:ovrn |
the authority resource index (Resource index). Active in openvibe-contracts since 0.108.0; the route enforces it (a Network service token holding it, loopback, audience openvibe.media) |
media.resource.control |
first-party | POST /api/v1/resources/control |
delete a person's (or their project's) object: the soft delete, restorable until its retention ends (Resource index). Active in openvibe-contracts since 0.169.0 |
media.object.s3 |
public (an access key) | the S3 surface at s3.openvibe.media (docs/s3.md) |
PutObject, GetObject, HeadObject, DeleteObject(s), ListObjects(V2), ListBuckets, HeadBucket, presigned GET/PUT; a bucket is a zone |
media.access_key.manage |
first-party | POST/GET/DELETE /api/v1/projects/:project/access-keys |
S3 access keys for a project admin, the secret shown once (Zone's console, Services) |
media.zone.provision |
first-party (Zone) | PUT/GET /internal/zones/:id |
OpenVibe.Zone's zone records, applied by revision; usage and deletion progress |
Called elsewhere, as the service principal media (the OAuth client media):
| Service | Grant | Why |
|---|---|---|
| OpenVibe.Network | identity.subject.resolve; network.account.export.contribute, network.account.deletion.confirm |
owner subjects for objects; account export and deletion (ADR-033) |
| OpenVibe.Live | live.lineage.resolve |
which channel and person a recording belongs to |
| OpenVibe.Events | events.event.publish, events.subscription.manage |
the outbox relay; its subscriptions |
Media announces outcomes as OpenVibe.Events events (media.vod.ready, …) and sends no webhooks since 2026-10-11 (C-75).
Layout
server/
index.js boot, route mounts, background jobs, graceful shutdown
config.js env config (PORT, MEDIA_DATA_DIR, *_PATH, DATABASE_URL/DATABASE_DIRECT_URL, MEDIA_B2_*/R2_*,
RTP pool, OV_NETWORK_URL)
auth.js tenancy + auth middleware, JWKS fetch, app seeding
announce.js transactional OpenVibe.Events outcome announcements
drill.js MEDIA_DRILL: restore-drill mode (reads only, no bytes, no jobs, nothing leaves)
db/database.js PostgreSQL helpers (openvibe-sdk/db, async), all app_id-scoped
db/init.js applies migrations/ (DATABASE_URL/DATABASE_DIRECT_URL, or embedded PGlite)
migrations/NNNN_*.sql apps, vods, clips, pastes(+likes/comments), files, content_views, media_settings,
media_objects/locations/relationships/variants/jobs/holds/invariant_violations
vod/recorder.js ffmpeg recording: RTMP pull + RTP (SDP) ingest, codec passthrough
vod/media-tools.js probes, seekable remux, DVR sidecar, chunk-segment concat
vod/finalize.js finalize pipeline (remux → probe → master recovery → thumbnail → event)
vod/routes.js /api/v1/:app/vods (+ ingest + chunks)
vod/clip-cutter.js re-encoding clip cutter (local file or presigned URL source)
vod/clips-routes.js /api/v1/:app/clips
vod/vod-storage.js local/B2/R2 tiering, presigned playback, sweep, CLI
vod/health-scanner.js probe/decode/master-recovery primitives
vod/health-job.js background health scan + quarantine cleanup + master sweep
files/routes.js /api/v1/:app/files
admin/routes.js /api/v1/:app/admin/storage (disk, tiers, buckets, bulk ops)
registry/resource-index.js the authority resource index (/api/v1/resources; common.resource-summary@1, ADR-048)
registry/resource-control.js the control API (POST /api/v1/resources/control; delete an object, ADR-048)
registry/projects.js whether a person administers a project (Network's internal membership read)
thumbnails/ thumbnail service + /api/v1/:app/thumbnails
public/routes.js public /v /c /p /t /f
me/ the object explorer: /me and /api/v2/me (a signed-in person's own objects, uploads,
derivatives and usage) and the operator views /me/ops — see docs/object-model.md#object-explorer
objects/ canonical object model: model (projections, holds), routes (/api/v2 + /o),
reconcile, invariant, signing (presigned URLs), multipart,
content-type, verify-job (scheduled copy verification), copy-report,
readiness (metadata / bytes_verified / playable), popularity (unique viewers
per day, no identifiers), tier-policy + tiering (native objects in the R2 cache) —
see docs/object-model.md
jobs/ media_jobs: queue (states, idempotency, media.job.* events in the state
change's transaction; payload queue.jobEvent: ids, state, ISO times,
has_result, never params/result/error text/creator), worker (light/heavy lanes, leases, retries,
cancellation), routes (/api/v2/:app/jobs), types: thumbnail.regenerate,
invariant.scan (proposes split/remux), object.split, object.remux, object.waveform, object.sprite,
object.cmaf (CMAF/HLS segments of a finished video into media_timeline; MEDIA_HLS_ENABLED),
object.pack (durable segments packed into ~60 s chunk objects; MEDIA_HLS_ENABLED),
rendition.create (an on-demand 720p H.264/AAC rendition of a media object into media_timeline;
MEDIA_HLS_ENABLED + MEDIA_RENDITIONS),
clip.cut (a clip's cut/re-cut; with MEDIA_HLS_ENABLED + MEDIA_MATERIALIZED_CLIPS and a source
timeline it materializes over the source's segments, only the edges re-encoded —
docs/materialized-clips.md),
vod.finalize (orphans and failed finalizes, with backoff),
vod.duration.reconcile (stored vs measured durations, local and B2/R2),
storage.orphans.scan (monthly storage orphan report, service-wide, report only),
storage.move.cleanup (the alert for placement moves whose delete keeps failing, service-wide)
client-ip.js trust proxy = loopback; req.ip is the only client address
(openvibe-shared v4.15.2, openvibe-contracts v0.216.0, openvibe-sdk v0.65.0: pinned release tarballs, installed by npm)
scripts/smoke-test.sh end-to-end smoke test (boots a temp instance)
scripts/reconcile-objects.js / object-invariant.js / no-good-copy-report.js object-model operator tools
scripts/media-jobs.js list jobs, run the size-invariant scan (dry run by default), approve/cancel proposals
scripts/r2-eviction-drill.js evict one VOD's R2 copy, prove B2 serves it, re-warm R2; JSON artifact (dry run by default)
scripts/vod-duration-reconcile.js stored VOD durations vs the real files (dry run by default; --apply --backup; --rollback)
scripts/vods-orphans-report.js read-only report on the B2 vods-orphans/ prefix (matches, sizes, recommendations);
--storage: the whole storage against the database (files and keys no row names,
copies that are not there, open multipart uploads), as the monthly job does
Visitor sign-in
The public pages use the shared navbar and footer. Visitors sign in through the Network as OAuth
client media (server/user-auth.js, routes under /auth, host-only cookies); set
OV_OAUTH_CLIENT_SECRET in the unit's environment. /terms, /privacy and /dmca come from
openvibe-shared/legal (profile hosting). This is separate from tenant auth below, which is how
other OpenVibe services call the API.
Your media (/me). A signed-in person sees the objects whose owner_subject is their Network
subject, in every tenant: uploads in progress, copies, derivatives, visibility, lifecycle and their own
usage per tenant and namespace (GET /api/v2/me/objects, /objects/:id, /usage). Read-only: changes
stay with the apps. Network staff with staff.site.view get the operator views at /me/ops (failed
jobs, missing media, projection, tiering and usage recompute with staff.site.configure). Details in
docs/object-model.md.
Tenancy & auth
Tenant = the :app path segment. Rows in the apps table define tenants:
| column | meaning |
|---|---|
app_id |
tenant id (live, games, …) |
api_key_hash |
sha256 of the app API key (never stored in plaintext) |
webhook_url / webhook_secret |
retained for a separate schema migration; unused for delivery |
allowed_origins |
JSON array; guards browser (user-JWT) calls |
quota_bytes |
storage quota: v1 files + native v2 objects (0 = unlimited) |
project_id / env |
set only on developer-project tenants (below) |
Credential types on /api/v1/:app/...:
-
App API key —
Authorization: Bearer <app_api_key>. Compared in constant time against the app's stored hash. A key is only valid for its own:appsegment; presenting another app's key returns 403. -
Network user JWT (browser endpoints only — chunks/complete, clips, files, thumbnails) — RS256, verified offline against the JWKS public key fetched from
OV_NETWORK_URL/api/.well-known/jwksat boot (cached, refreshed every 6 h).audmust includeopenvibe.mediawhen present. If the request carries anOriginheader it must be in the app'sallowed_origins(CORS is reflected for allow-listed origins only). -
Network principal token (
sub svc:…,app:…,mod:…; RS256, audienceopenvibe.media) — accepted only on routes that name a verb, each checked for the namespace it works in against the token'sns: read (media.object.read), list (media.object.list, or read), write (media.object.upload), delete (media.object.delete, or upload) and transform (media.derivative.create, or upload). The tenant's root namespace is:app; children, quotas and policy are in docs/object-model.md.
Developer-project tenants (ADR-014)
A developer app from OpenVibe.Network (token sub app:app_<ULID>, project_id prj_<ULID>,
env sandbox|production, ns [project_id, app.<project_id>.*]; an older ns [project_id] means the
same) holding media verbs gets a tenant keyed by its project id, created on first use. Its
namespaces are app.<project_id> (production) and app.<project_id>.sandbox, with children below. The URL always names the project
(/api/v1/prj_<ULID>/files, /api/v2/prj_<ULID>/objects, so openvibe-sdk's
createMediaClient({ app: projectId }) works); the token's env picks one of two separate tenants:
| env | tenant (apps.app_id) |
default quota |
|---|---|---|
| production | prj_<ULID> |
MEDIA_APP_TENANT_QUOTA_MB (1024) |
| sandbox | prj_<ULID>-sandbox |
MEDIA_APP_SANDBOX_QUOTA_MB (100) |
- Both rows carry
project_idandenv, and have no API key (api_key_hashempty; the seeding env refusesprj_…ids). Only the project's own app tokens reach them: other projects' tokens, first-party service tokens, app keys and user JWTs get 403/404, and the-sandboxid cannot be named in a URL. Nothing is created unless the token'sproject_idis the path's project and the token holds the route's capability. - App tokens reach only these tenants and only the capability routes (files, objects v2 without retention holds). VODs, clips, thumbnails, assets, stats and admin refuse them.
- Sandbox tokens (
env: sandbox) are accepted on these routes only; everywhere else they get401 token.sandbox_refused. - Sandbox content is never public.
/f/:keyand/o/:idanswer 404 for it unless the URL carries a valid signature; the Files API returns a signed, short-livedurl(+url_expires_at,sandbox: true), objects havepublic_url: nulland/downloadalways signs, whatever the visibility. Sandbox tenants emit no platform events. - The quota counts v1 files and native v2 objects together (files uploads now use the same count as objects for every tenant). v1 file keys in project tenants carry a tenant tag, so two tenants never collide on identical uploads. Project tenants' files are not listed in the public gallery. Soft-deleted objects keep counting against a project tenant's quota until their bytes are purged.
/f/:key(like/o/:id) sendsnosniffand serves only images (not SVG), video, audio, PDF and plain text inline; anything else an uploader labels (HTML, SVG, XML…) is anattachment.- Not yet: reading quotas set in Network (
network.project.read), and removing a project's tenants when the project is archived.
First-party tenants
Boot creates any missing first-party tenant: live, community, tools, games, wiki, blog, inventory and
media-hub (server/auth.js ensureTokenOnlyApps), with its quota and browser origins, and leaves an existing row as
it is. Each is reached only with its service's Network service token, with grants per namespace in Network. No app
key is issued or seeded: MEDIA_APPS_SEED and MEDIA_APP_KEYS are gone (compatibility register C-71), and migration
0006 emptied the key hashes the tenants still carried. App keys now exist only as test fixtures (db.upsertApp)
and in scripts/smoke-test.sh.
API (all under /api/v1/:app)
VODs
| method | path | notes |
|---|---|---|
| POST | /vods |
{ title?, stream_id?, stream_key?, managed_stream_id?, user_id?, meta?, visibility?, clips_only? } → { id } |
| POST | /vods/:id/ingest/rtmp |
{ rtmp_url } → 202; ffmpeg pulls the URL, lossless stream-copy → fragmented .mp4. Only allow-listed RTMP servers (MEDIA_RTMP_PULL_ALLOW, a list of host:port or host:low-high, default 127.0.0.1:1935,localhost:1935,[::1]:1935,127.0.0.1:19360-19399: Live's RTMP server and OpenRestream's loopback play ports on this host) and a plain rtmp[s]://host[:port]/path shape (no user info, whitespace or backslash); anything else is 400 and starts nothing |
| POST | /vods/:id/ingest/rtp/start |
{ video: {payloadType, codec, clockRate, ssrc?, parameters?}, audio?: {…} } → { videoPort, audioPort } from UDP 12000-12199; point PlainRtpTransports at 127.0.0.1 (RTCP = port+1) |
| POST | /vods/:id/ingest/rtp/stop |
finalizes the recording |
| POST | /vods/:id/chunks |
multipart chunk (+segmentId), user JWT ok — browser MediaRecorder append flow |
| POST | /vods/:id/complete |
finalize chunked upload (user JWT ok) |
| POST | /vods/:id/finalize |
close recording; remux, probe, thumbnail, event |
| GET | /vods/:id |
{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … } (readiness: below; hls_url, with hls_expires_at when signed: the object's master playlist with MEDIA_HLS_ENABLED, docs/media-fabric.md §3) |
| GET | /vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since |
list; include_private app-key only; order = newest|oldest|views; since = created at or after (ISO 8601 or YYYY-MM-DD HH:MM:SS, UTC) |
| PUT | /vods/:id |
{ title?, description?, visibility? } |
| DELETE | /vods/:id |
deletes local + B2 + R2 objects + row |
status: pending → recording → ready | failed (derived; failures come from
health quarantine: corrupt / zero-byte / missing file).
Readiness. VOD and clip answers (and v2 objects) also carry readiness
{ metadata, bytes_verified, playable, hash_verified, verified_copies, reason }, read from the object
model: bytes_verified needs a copy whose bytes a check found (local file, B2/R2 HEAD, sha256 when
known), and playable adds a finished, not failed, not deleted VOD or clip. A B2 copy nobody has
checked yet is not verified. The watch pages offer a player only when playable (see
docs/object-model.md).
Durations are measured, never estimated. Finalize stores what the file says and
records where it came from in duration_source: probe (ffprobe's container
duration), remux (the last packet time of a stream-copy pass, used when the header
has none or is far longer than the packets) or unknown (nothing measurable: the
duration is stored as 0, health_status is needs_review with probe_failed or
inflated_duration, and the VOD is hidden until a later finalize measures it). The
wall clock only bounds a recording the recorder saw start (a value longer than 1.5x
its run + 30 s is refused); an orphan finalized after a restart has no bound and is
never measured as now - created_at. While recording, duration is the live elapsed
time; finalize replaces it.
Recording formats (inherited codec-passthrough behavior — VODs are not
always .webm): RTMP and RTP-H.264 record by lossless stream copy into a
fragmented .mp4 (audio → AAC on the RTP path); RTP-VP8/VP9 copies into
.webm (Opus copied); exotic codecs fall back to a libvpx re-encode with a
lossless .master.mkv recovery archive. A .seekable sidecar is remuxed every
60 s during recording so /v/:id is DVR-seekable while live.
Clips
| method | path | notes |
|---|---|---|
| POST | /clips |
{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? } → over a VOD with a CMAF timeline (MEDIA_HLS_ENABLED): a virtual clip, 201 ready at once (the clip, storage_provider: 'timeline'), played as HLS over the source's segments (/c/:id redirects to its playlist); with materialize: true or no timeline → 202 { id, status: 'processing', job_id }; cut (clip.cut) runs in background — with MEDIA_HLS_ENABLED + MEDIA_MATERIALIZED_CLIPS=1 and a source timeline it is materialized over the source's bytes (the clip's own media_timeline rows copy the interior segments; only the two window edges are re-encoded, docs/materialized-clips.md), otherwise the full re-encode (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart video = direct upload of an already-cut blob → 201 ready |
| GET | /clips/:id |
status: `processing |
| POST | /clips/:id/recut |
{ materialize? } → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or materialize: true) 202 { id, status, job_id } (clip.cut; a virtual clip stays ready while it is cut) |
| POST | /clips/:id/trim |
{ start_s, end_s }, seconds into the clip (at least 1 s apart) → the clip narrows in place: same id and title, its window of the source shrinks and it is cut again from the source (202 { id, status: 'processing', job_id }, clip.cut, clip.ready again; the superseded file is removed), or a virtual clip plays the narrower window at once (200, the clip); the whole clip → 200 unchanged: true; a file clip is processing from the answer until its cut is in; still being cut (or a queued/running clip.cut) → 409; under a retention hold (its own or its VOD's) → 409 media.object.held; no source VOD (a browser upload), a file moved to cloud storage or an unreadable source recording → 422; a cut that fails for good gives the clip its previous window and file back (clip.ready again) |
| GET | /clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since |
list; channel_user_id = clipped-channel owner; include_private app-key only; auto_generated=1|0 = cut by the app's automation (AI auto-clips) or made by a person; status=ready = playable clips only; since as for VODs |
| PUT | /clips/:id |
{ title?, visibility?, auto_generated? } (auto_generated app key only, 403 when acting for a user) |
| DELETE | /clips/:id |
local + offloaded objects + row; a timeline clip's own rows and edge bytes go with it, while every location the source still names — a materialized clip's interior — stays (docs/materialized-clips.md) |
Pastes
The API is gone: OpenVibe.Community owns pastes (since 2026-09-22) and Media's
read-only paste app API was retired in T10 step 2 — /api/v1/:app/pastes is
unmounted, so every method there answers 404. No paste is created, read, listed or
deleted through Media any more.
/p/:slug and /p/:slug/raw are unconditional 301s to Community: the target
is config.pastes.movedTo (PASTES_MOVED_TO, default https://openvibe.community,
so the redirect never depends on an env var) and no row is read — a private or
missing slug redirects exactly like a public one. Legacy screenshot and avatar objects are
found by legacy_ref and /p/:slug/screenshot redirects to /o/<id> when they are
public or unlisted and ready. Other slugs redirect to Community. Avatar ingest stores
new avatars as native objects in Live's avatars namespace. Legacy files remain under
PASTES_PATH and are named by their objects' local locations. Legacy screenshot and
avatar objects are deleted through account deletion; direct v2 deletion answers 409.
| method | path | notes |
|---|---|---|
| GET | /p/:slug |
always 301 to OpenVibe.Community — pastes live there; no row is read |
| GET | /p/:slug/raw |
always 301 to Community's /p/:slug/raw; the suffix is preserved |
| GET | /p/:slug/screenshot |
301 to the legacy object when public or unlisted and ready; otherwise 301 to Community |
Legacy paste tables remain in the database until the later contract migration, but Media does not query them.
Files
| method | path | notes |
|---|---|---|
| POST | /files |
multipart file → { key, url, size, mime }; key = <sha256[0:12]>-<name>; per-app quota_bytes enforced (413 on exceed) |
| GET | /files |
list + used_bytes/quota_bytes |
| GET | /files/:key |
meta |
| DELETE | /files/:key |
delete |
Thumbnails
| method | path | notes |
|---|---|---|
| POST | /thumbnails/:kind/:id |
kind vod/clip: multipart thumbnail (or {image: base64}) uploads a custom image, or with no body (re)generates from the media (VOD @10%, clip near first frame); kind live: upload a broadcaster frame, stored under the stable name stream-<app>-<id>.jpg → { url } |
Admin storage (/admin/storage, app-key auth only)
Storage-management endpoints ported from the predecessor's admin panel. Auth
is the app API key only (no user JWTs) — admins reach these through their
app's own server, which holds the key. DB-derived stats are scoped to the
calling app; disk totals and directory sizes are host-wide (the data
directories are shared across apps) and responses carry a note saying so.
| method | path | notes |
|---|---|---|
| GET | /admin/storage |
{ disk (df of VOD volume), database.bytes, breakdown [vods/clips/pastes/thumbnails/files dirs], vodStats/clipStats/fileStats (app-scoped), byProvider.{vods,clips} (count+bytes per local/b2/r2) } |
| GET | /admin/storage/vods?limit&offset&sort&order&provider&tier |
detailed app VOD listing: id, title, size, provider, health, views, last_accessed_at, created + fileExists/diskSize/actualTier (disk-reconciled: local/b2/r2/missing), per-user summary. sort = size|date|duration|tier|views|accessed, order = asc|desc, provider = local|b2|r2 (tier accepts legacy hot/cold aliases) |
| DELETE | /admin/storage/vods/bulk |
{ ids: [...] } (max 200) — deletes each VOD everywhere (local + B2 + R2 + sidecars/master + thumbnail + row), app-scoped → { deleted, freed, results: [{id, ok, error?}] } |
| GET | /admin/storage/tiers |
tiering status: settings, provider health, local disk, service-wide tier counts (tiers/clipTiers), sweepRunning, plus app.{tiers, pendingOffload} scoped to the caller |
| PUT | /admin/storage/tiers/settings |
update any storage_tier.* knob (see vod-storage.js DEFAULTS: enabled, minAgeDays, maxViewsForCold, minLastAccessDays, sweepIntervalMs, hotDiskPressurePct, localLowWaterPct, maxPerSweep, r2Enabled, r2MinViews, …); persisted in media_settings, sweep timer restarted → { ok, settings } |
| POST | /admin/storage/tiers/sweep |
run the tiering sweep now → sweep summary |
| POST | /admin/storage/tiers/move |
{ vod_id, target: local|hot|b2|cold|r2 } — reuses the storage engine's verified move logic; VOD must belong to the app |
| POST | /admin/storage/tiers/bulk-move |
{ ids: [...], target } (max 50) → { moved, bytes, errors? } |
| GET | /admin/storage/tiers/policy |
read-only R2 policy: each threshold (r2Enabled, r2MinViews, r2RecentAccessDays, r2MaxIdleDays, r2MaxPerSweep) as { value, default, source: default|setting }, the promote and demote rules in words, the R2/B2 provider status, and this app's decisions: last_24h counts per action and outcome and the 20 most recent |
| GET | /admin/storage/tiers/decisions?vod_id&action&outcome&limit&before_id |
this app's R2 promote/demote decisions, newest first: { id, decided_at, vod_id, object_id, action, from_provider, to_provider, outcome: done|already|refused|failed, trigger: sweep|admin|clip|drill|manual, reason, inputs, thresholds, error } → { decisions, next_before_id } |
| GET | /admin/storage/tiers/objects/policy |
native v2 objects' R2 tiering (below): the activation gate, each media.object_tier threshold as { value, default, source }, the rules, this app's promotion candidates (with what would refuse each) and its decisions (24-hour counts and the 20 most recent) |
| GET | /admin/storage/tiers/objects/decisions?object_id&action&outcome&limit&before_id |
this app's native object decisions, newest first (outcome: done|already|refused|failed|dry_run) → { decisions, next_before_id } |
| GET / POST | /admin/storage/config[/media.storage_tier|/media.object_tier[/history|/rollback]] |
the revisioned tier policies (openvibe-shared/config): read, apply { values, merge, reason }, history, roll back |
| GET | /admin/storage/holds?all&object_id&vod_id&clip_id&limit&before_id |
this app's retention holds, newest first (active only unless all=1) |
| POST | /admin/storage/holds |
{ object_id | vod_id | clip_id, reason, kind?, note?, placed_by? } → 201: the object is kept (no delete, no tier move) and so are the clips cut from a held VOD. Not for calls acting for a user (403) |
| POST | /admin/storage/holds/:holdId/release |
{ released_by } (also DELETE /admin/storage/holds/:holdId); 409 when already released. Holds stay on record |
| GET | /admin/storage/buckets |
sanitized bucket status per provider: { configured, endpoint, bucket, region, healthy, reachable } via a live HeadBucket probe — credentials are never returned |
| GET | /admin/storage/ops |
this app's operator report (as /me/ops shows it to Network staff): failed jobs, missing media, projection, tiering, the namespaces' usage snapshot. Not for calls acting for a user (403) |
| POST | /admin/storage/ops/recompute |
refresh this app's namespaces' usage snapshot from the rows |
Outcome events
Media announces outcomes as OpenVibe.Events events (media.vod.ready, …) and sends no webhooks since 2026-10-11 (C-75).
Each outcome is a durable OpenVibe.Events event (media.vod.ready|failed,
media.clip.ready|failed, media.object.uploaded, media.storage.alert|recovered;
server/events.js). The outbox row is written in the same transaction as the state
change it describes (announce()), so an outcome is never lost or announced for a
rolled-back change.
Two additional object events go to OpenVibe.Events: media.object.deleted
{ object_id, app_id, kind, legacy_ref, deleted_at } and media.object.visibility_changed
{ object_id, app_id, kind, legacy_ref, visibility, previous_visibility, changed_at } (subject
object <id>, important, times ISO 8601 UTC, legacy_ref null for native objects). Nothing
else about the object is in them (no title, owner, keys or sizes). Triggers on media_objects
stage every deletion and visibility change in the transaction that makes it, whichever path made it
(a v1 visibility change, a row delete, a v2 soft delete, operator SQL); the staged rows become outbox
envelopes inside Media's own transactions (projection sync, announce(), soft delete), so those
commit with their change, and the relay drains anything else. No visibility event for an object
that is deleted; none for sandbox tenants. test/object-events.test.js.
Resource index (/api/v1/resources, first-party)
Media's authority resource index (ADR-048 §3, plan T13 step 8): the resources Media owns, as
common.resource-summary@1 — the shape OpenVibe.Services fans out over and merges. It lists Media's
objects alone (kind media.object, id med_<ULID>, the media_objects table): v1 vods, clips and
files are projections over objects (bigint ids) and are never listed.
GET /api/v1/resources[?project=<prj_ id>&kind=<kind>&cursor=<opaque>&limit=<n>] → common.resource-list-result@1
GET /api/v1/resources/:ovrn → common.resource-summary@1
| Param | Meaning |
|---|---|
project |
the tenancy boundary: a prj_ id. Only that project's rows answer (its production and sandbox tenants); never another project's. Omitted, the first-party caller sees every Media resource |
kind |
media.object (the only kind Media owns). An unknown kind is an empty page, not an error |
cursor |
opaque keyset cursor from the previous page's next_cursor, ordered by id |
limit |
page size, default 100, max 1000 |
Auth is a Network service token holding media.resource.read, loopback only (audience openvibe.media;
server/service-guard.js) — the same first-party check as the other internal routes. No token is
401, a token without the capability 403. A bad query is 400 resources.bad_query; an unknown or
unnameable resource is 404 resources.unknown_resource.
An object's project_id is its tenant's (apps.project_id); a first-party tenant (live, …) has none, so
its objects carry no project_id and no ovrn. The summary's ovrn is composed by openvibe-contracts'
contracts.resources.nameOf — the one formatter — so it is present exactly when the object has a project
(ovrn:media:<project_id>:object/<med_…>); GET /api/v1/resources/:ovrn reads one by that name, and only a
resource whose computed name equals the one asked for answers. owner is a subject-ref when the object
stores a usr_ owner. test/resource-index.test.js.
Control (media.resource.control, plan T13 step 10): POST /api/v1/resources/control takes a
common.resource-control-request@1 from OpenVibe.Services (same loopback service-token check, and the caller must
be a svc: service) and answers a common.resource-control-result@1. The one action is delete: the API's soft
delete (restorable_until in the result; restore through POST /api/v1/objects/:id/restore). A held object and a
legacy-managed one (a VOD, clip or file) are refused. The object's owner may act, and so may an admin or owner of its
project (asked of Network with network.project.member.read, server/registry/projects.js); anyone else gets
resources.unknown_resource. Dry runs; the first answer per (caller, project, key) is kept in media_control_log and
returned for a repeat. test/resource-control.test.js.
Restore drills (MEDIA_DRILL=1)
A restore drill starts a second Media from this checkout on 127.0.0.1:14100 with MEDIA_DRILL=1 and
DATABASE_URL naming a restored copy of the database (pgBackRest restores the cluster on a spare port). In that
mode (server/drill.js) Media:
- refuses to start unless
DATABASE_URLis set and does not name production's database (ov_mediaon 5432, or PgBouncer's 6432),DATABASE_DIRECT_URLis unset (a drill never migrates),HOSTis loopback andPORTis not 4100 (before the database is opened), and opens the copy without migrating it; - starts only its HTTP server: no app seeding, JWKS refresh, tiering sweep, health job, junk sweep, clip re-cuts, copy verification, jobs worker, disk guardian, thumbnail cleanup, object purge, orphan-recording finalize or Events relay;
- writes no file and creates no directory (the storage checks leave
/api/ready, as do the remote tiers; it reports"mode": "drill"); - never opens a file path from the database: every byte route (
/vand/cbytes,/t,/a,/f,/o,/p/:slug/screenshot,/live/:sel/frame.jpg) answers 503; watch pages and the/browseindex still render from the copy; - connects to nothing and runs no program but
git(no ffmpeg or ffprobe), opens no UDP socket and listens on nothing but its port; - answers 403 to every method but GET, HEAD and OPTIONS, and 503 to
/auth/*.
test/drill-mode.test.js boots the real server that way against a restored copy
and checks each point.
N-1: the previous release against this one
test/n-1.test.js (roadmap WS-P task 11, in npm test) boots this checkout in the drill sandbox
(with writes let through) on an embedded PGlite database and replays test/fixtures/n-1/: every call the previous release's clients make (the
openvibe-sdk media and objects clients with an app key, the object explorer, and every player URL,
link, script and form of the /, /browse, /v, /c, /p, /me and /updates pages it served),
each answered compatibly (status, JSON, the response fields the client reads). Every PostgreSQL migration
the previous release ran must still be present and unchanged. After each deploy, record the
release now in production as the next N-1 (npm run n-1:record [ref]) and commit the fixtures.
Client addresses
Express trust proxy is loopback (server/client-ip.js): req.ip is the X-Forwarded-For that the
local nginx set when the request came through it, and the socket address for anyone else. A caller
that reaches port 4100 directly cannot choose its IP by sending X-Forwarded-For or
CF-Connecting-IP itself. View counting and the live-frame rate limit read req.ip,
never the headers (test/trust-proxy.test.js).
Segment-native video settings
| Environment variable | Default | Purpose |
|---|---|---|
MEDIA_HLS_ENABLED |
0 |
Enable timeline generation and HLS routes. |
MEDIA_CMAF_TRANSCODE |
0 |
Encode incompatible source streams to H.264/AAC for the source timeline; uses CPU. |
MEDIA_CMAF_TRANSCODE_MAX_SECONDS |
14400 |
Maximum duration of a source needing CMAF transcoding (four hours). |
MEDIA_CMAF_TRANSCODE_THREADS |
2 |
libx264 threads for one CMAF transcode, which also runs at nice 15 so live streams keep the CPU. |
MEDIA_TIMELINE_LOCAL_MAX_BYTES |
10737418240 |
Maximum local timeline cache (10 GiB; 0 keeps no durable copies locally). |
MEDIA_TIMELINE_LOCAL_MIN_FREE_BYTES |
8589934592 |
Below this free disk level, new verified timeline bytes remain durable only. |
MEDIA_HLS_BACKFILL_PER_HOUR |
20 |
Maximum lazy object.cmaf queues for older VODs per rolling hour on a single Media host; finalize is exempt. |
MEDIA_JOBS_HEAVY_CONCURRENCY |
1 |
Keep at 1 while enabling HLS in production. |
The hourly sweep evicts the least recently served verified local timeline files. A segment whose local copy was
evicted is served from its durable provider. Watch timeline_cache on /api/ready and job durations during backfill
(docs/media-fabric.md §3).
Per-actor limits
nginx limits by address; the capability routes also limit by who calls (server/actor-limits.js,
openvibe-sdk/limits, roadmap WS-R task 4): the Network token's principal (svc:community, app:app_…),
the person signed in to /api/v2/me, or an app acting for one of its users (X-OV-User-Id). An app's
own API key is not counted (a first-party server speaks for all its visitors); a browser with only an
upload token is counted by address. Past a limit: 429 problem+json rate_limited with Retry-After,
before the route does any work, and one [Limits] log line and media_rate_limited_total{limit,window}.
| Routes | Per caller |
|---|---|
Reads: objects, files, jobs, namespaces, /api/v2/me |
MEDIA_LIMITS_MINUTE / MEDIA_LIMITS_HOUR (120 / 3000) |
Object init, /upload-url, /multipart, multipart complete; file upload |
30 a minute, 600 an hour |
| Object content PUT, complete; object and file delete, restore, multipart abort; job cancel | 60 / 1200 |
| Multipart parts | 600 / 20000 (the session already bounds them) |
| Job create, approve | 10 / 200 |
POST /api/v2/me/ops/recompute |
6 / 60 |
Never limited: /healthz, /api/ready, /release.json, /metrics, the signed /internal/events deliveries.
Counters are per process (a restart forgets them). test/actor-limits.test.js.
Usage readings → Billing
Media's storage usage goes to OpenVibe.Billing as one platform.usage-sample@1 reading per (project, subject,
tier) where a project's work is billable (server/billing.js, plan T5 step 14):
metric (resource) |
what | period | unit | provider |
|---|---|---|---|---|
gb-month |
bytes stored: every present copy in media_locations, summed by tier, counted once when more than one object names a location and billed to the oldest owner, as that day's share of a GB-month |
closed UTC day | GB |
the tier (local/b2/r2) |
Media emits gb-month only. gib-delivered is deliberately not emitted: demand.record() counts route()
calls, not bytes, so reads × size_bytes would bill aborted downloads and range reads in full, would bill
presigned B2/R2 redirects Media never serves, and would miss HLS segment reads. It comes back once responses are
metered by the bytes actually written (sendSlice/streamFileWithRange) and redirected reads are metered by CDN
logs. Nothing here bills delivery.
Only the newest closed UTC day is aggregated, because the query reads media_locations' current state: a day
missed while the service was down is not billed (a reading never claims today's state for an older day).
First-party traffic (a tenant with no project) and sandbox tenants are never recorded; an object without an
owner_subject is still stored with its project (Billing leaves it unrated until it names a user subject).
The SDK reporter (openvibe-sdk/usage createUsageReporter) queues each reading idempotency-keyed in
billing_readings inside the aggregation's transaction and relays it to Billing's POST /api/v1/usage
(billing.usage.record) with Media's Network service token (clientId media, audience openvibe.billing;
OpenVibe.Network grants exactly that). A relay that cannot reach Billing (no grant, a missing route, a timeout,
backpressure, a 401/403/404/408/425/429 or a 5xx) stays pending with backoff across restarts; only Billing
refusing the reading itself (any other 4xx) is marked rejected and never retried, so nothing is billed twice.
A stored reading is never edited (a trigger). Re-aggregating a day inserts nothing
(billing_periods).
Off by default. MEDIA_BILLING_INTERVAL_MS=0 (the default) starts no timer and aggregates nothing; a
restore drill starts nothing at all. Turn it on with MEDIA_BILLING_INTERVAL_MS plus OV_BILLING_URL and,
only if it differs, OV_BILLING_AUDIENCE; without a billing URL or OV_OAUTH_CLIENT_SECRET readings are
aggregated and stay queued. The runbook is docs/cutover-media-billing-readings.md; test/billing.test.js.
Health, readiness and metrics
| path | behavior |
|---|---|
GET /healthz |
liveness: the process answers (unchanged; it checks nothing else) |
GET /api/ready |
readiness from real checks, each with status, required, latency_ms, checked_at. Required (503 when one fails): db (a real round trip that names the store, postgresql or pglite, and counts the tenants), valkey optional (skipped without VALKEY_URL), storage_vods, storage_clips, storage_pastes, storage_thumbnails, storage_files, storage_objects (a probe file is written and removed). Optional (listed in degraded, still 200): network_jwks, remote_b2 / remote_r2, events_outbox, and timeline_cache (bytes vs budget; free disk vs floor). Also recordings_in_progress. |
GET /metrics |
Prometheus text (openvibe-shared/metrics) for direct loopback callers only; 404 through nginx. HTTP golden signals by route template, process metrics, release_info, release_client_updates_total{outcome,reason}, plus media_recordings_in_progress, media_timeline_local_bytes, media_timeline_local_evictions, media_object_locations{provider,state}, media_objects{lifecycle_status}, media_events_outbox{status} (only while the outbox runs), media_job_stale_completions_total{action} (job writes refused by the lease-token fence). |
GET /release.json |
release manifest (registry.release-manifest@1 1.1.0, openvibe-shared/release, ADR-016): deployed commit, package versions, components, contract ranges, metrics_url. The shared navbar's release-watch polls it. No components are declared, so every release still prompts open tabs to reload. |
POST /release-metrics |
open tabs' update reports (release-watch beacons, at most 4 KB; 30 a minute per connecting address, which behind nginx is the proxy, so one budget for all tabs; Sec-GPC/DNT dropped) into release_client_updates_total on /metrics. 403 in a restore drill, like every write. |
Public serving (no auth unless the item is private)
| path | behavior |
|---|---|
GET /v/:id |
VOD playback — local stream with Range support, live-DVR .seekable sidecar while recording, or 302 to a presigned B2/R2 URL. X-Robots-Tag: noindex. By numeric id only: the file-name form was retired with C-75 (2026-10-11) |
GET /c/:id |
clip playback, same logic, noindex |
GET /p/:slug |
301 to OpenVibe.Community, the canonical home for pastes since 2026-09-22 (every slug, found or not) |
GET /p/:slug/raw |
301 to Community's raw text |
GET /p/:slug/screenshot |
301 to a ready, nonprivate legacy screenshot or avatar object; otherwise 301 to Community |
GET /t/:id |
thumbnail (id = filename), noindex. ?w= and/or ?fmt= answer a variant (Media Fabric F4, server/thumbnails/variants.js): the width snaps up to 64, 96, 160, 320, 480 or 640 (never wider than the source), the format is webp (the default), avif or jpeg, named in the URL and never negotiated (Cloudflare ignores Vary). Made on the first request and kept in a bounded cache (THUMBNAIL_VARIANTS_PATH, default thumbnail-variants beside the thumbnails; MEDIA_THUMB_VARIANTS_MAX_BYTES 512 MiB, least recently used dropped; MEDIA_THUMB_VARIANTS_CONCURRENCY 2). The key holds the source's modification time, so a rewritten live thumbnail gets a fresh variant. Same caching as the original; a busy cache answers the original with max-age=60; a bad w or fmt is a 400. A 640 px VOD thumbnail of about 24 KB is about 13 KB as a 480 px WebP |
GET /f/:key |
file with stored Content-Type + Range, noindex |
GET /live/:sel/transcript.json |
transcript + AI timeline API — full audio-transcription log and AI overview timeline. :sel = slot id / slot slug → slot-scoped sessions; @username → user-scoped (all their slots, works while offline). Returns { live, current, sessions[] (each: ai_overview, transcript, duration…), streamer (overview + stream memories), user }. ?limit=1..50 sessions (default 10), ?app=. Cached 30s (that's the rate limit), CORS-open. |
GET /live/:sel/chat-insight.json |
chat insight API — a user's chat-related AI insight/timeline (:sel = @username or numeric user id): today-vs-alltime chat overviews, condensed memory, event timeline, plus their streamer overview + stream memories when they stream. Proxied from the app's public chat-AI API over loopback; cached 30s, CORS-open. |
GET /v/:id/transcript.json |
VOD transcript API — transcript + AI overview for one existing VOD id ({ vod_id, title, duration_seconds, ai_overview, transcript, ai_analyzed_at }). Private VODs → 404. Cached 30s, CORS-open. |
GET /o/:id/master.m3u8, /o/:id/source/index.m3u8, /o/:id/source/{init.mp4,init-head.mp4,init-tail.mp4,NNNNNN.m4s}, /o/:id/<rendition>/… |
segment-native video (F3.1), only with MEDIA_HLS_ENABLED=1 (off by default; off, these 404): HLS playlists written from the object's media_timeline rows (VOD, #EXT-X-ENDLIST) and its CMAF segments (Range; a segment packed by object.pack (F3.3) is a byte range of its chunk). The master playlist lists one variant per rendition whose rows exist — the source cut plus, with MEDIA_RENDITIONS=1, an on-demand 720p rung cut by rendition.create and queued lazily by this route and by download?format=json (F4 slice 1, docs/media-fabric.md §5). Same check as GET /o/:id (and, like it, no Access-Control-Allow-Origin on a private or sandbox answer): private and sandbox objects only with the object's ?exp&sig (/api/v2/:app/objects/:id/download?format=json → hls_url, a playlist token valid MEDIA_HLS_PLAYLIST_TTL_S (6 h; 60 s to 12 h) that no other route accepts; a download signature works too), carried onto every URI; deleted 410. The object.cmaf job makes the timeline; a recording's finalize queues it, and download?format=json queues it for an object that has none yet; the v1 VOD and clip answers carry the same hls_url (and hls_expires_at when signed). A materialized clip (F3.6, MEDIA_MATERIALIZED_CLIPS=1) plays its own persisted rows: the playlist carries #EXT-X-DISCONTINUITY + a fresh #EXT-X-MAP where a re-encoded edge's init-head.mp4/init-tail.mp4 replaces the source init (docs/materialized-clips.md). noindex |
GET /robots.txt |
crawler policy (openvibe-shared/seo.robotsTxt: AI and search crawlers named, /api/, /auth/, /internal/, /o/, /metrics, /live/, /me disallowed) and the sitemap |
GET /sitemap.xml |
the media index and the watch pages Media is the canonical home of: public, playable VODs and clips of apps other than Live (Live lists its own /vod and /clip pages), never AI clips, private, unlisted, sandbox, recording or failed items; thumbnails as image entries. Every listed page renders index, follow |
GET /llms.txt |
a map of the site for language-model crawlers |
GET /llms-full.txt |
the same map plus the full text of the index and the newest watch pages Media is the canonical home of (openvibe-shared/seo.llmsFull; rendering stops at 512 KB, and any further pages become its "truncated" line) |
GET /live/:sel/frame.jpg |
live frame API — near-realtime JPEG frame of an actively-live stream slot, extracted from its in-progress recording. :sel = slot id (1), slot slug (whip), or @username (that streamer's top-viewed live slot; slug/username resolve via the app's /api/streams listing, cached 5s). Optional ?w=64..1920 scales the width, ?app= selects the tenant (default live; internal base URLs from APP_INTERNAL_URLS JSON env or LIVE_APP_INTERNAL_URL). Cached 5s per slot (that cache is the rate limit), CORS-open for external APIs/bots/dashboards. Not live → 404 with a styled OFFLINE card (real JPEG bytes — dev pipelines decode the body as image/jpeg) so <img> embeds degrade nicely (?format=json for JSON errors); 503 + card when live but a frame can't be cut. |
Sitemap entries carry lastmod from the release commit for the index and each watch item's recorded time.
Watch pages (/v/:id, /c/:id on a browser navigation) offer a player only when the item's
readiness is playable; otherwise they say why (still being recorded, still processing,
not available) and are noindex. They carry a schema.org VideoObject when the item is public and
playable (none on unlisted or private pages). AI clips (auto_generated) are noindex, follow
wherever they belong, labelled "AI clip", attributed to no person (creator OpenVibe AI) and
isBasedOn their source VOD, as on Live.
Private items (and legacy rows with no visibility and is_public = 0) answer
exactly like a missing id — the same 404 and body — unless the request bears the
owning app's API key, optionally acting for the owner via X-OV-User-Id. That holds
for the watch page, the bytes, transcript.json and /p/:slug/raw; the v1 detail routes (GET /vods/:id, /clips/:id)
apply it to an app acting for someone other than the owner. A call acting for a user writes
only that user's VODs and clips (update, delete, ingest, chunks, finalize, re-cut, thumbnail):
someone else's private one is the missing answer, anything else 403. The media index lists a
thumbnail only while its VOD or clip is public. Tests: test/security-*.test.js.
GET /o/:id serves object bytes by canonical id (med_…): public/unlisted
objects openly (and, for a JPEG, PNG, WebP, AVIF or GIF with a local copy, ?w=&fmt= answers a resized variant exactly
as /t/:id does, from its own cache OBJECT_VARIANTS_PATH, default object-variants; Network's /avatar/:username?s=
and the shared navbar ask for avatars this way; a private image takes a variant only with its signature, served
private and no-store like the original, which is how openvibe.download shows its previews; without a local copy, or when the cache is busy, the original), private ones only with a valid signature from
/api/v2/:app/objects/:id/download, deleted ones 410 (a deleted private object
without a signature is a 404 like any other private one). A native object in the
R2 popularity cache redirects to its verified R2 copy; a GET of a native object
counts its viewer for the day, identifying no one (see Storage tiering).
S3 surface and zones
Media answers a subset of S3 at https://s3.openvibe.media (ADR-031; DNS-only since 2026-10-21, so an object may be larger than Cloudflare's 100 MB; https://s3.openvibe.network is the same surface behind Cloudflare). Each bucket is a zone of a project environment, and zones are managed by OpenVibe.Zone. Media issues and verifies the ovk_ access keys, and every object stays an ordinary Media object.
See docs/s3.md for the operations, the rules and the tests.
Object model and API v2
Every stored blob is a media object (med_<ULID>, media_objects) with one
media_locations row per copy (local / B2 canonical / R2 cache, each present,
missing, pending or corrupt). The vods, clips and files rows are typed projections over objects (object_id column); every
existing route and response is unchanged, and the old write paths keep the
model current. Full reference: docs/object-model.md.
- Object projection — C-75 was retired on 2026-10-10 after production drift was 0 of 3036 rows. Every write makes its media object in the same transaction (
server/objects/model.jswithObject). Operators retainscripts/reconcile-objects.jsandscripts/object-invariant.jsas checks (details). - API —
/api/v2/:app/objects: init →PUT /:id/content(sha256, size, quota) →/:id/complete;GET /:id, cursorGET /, softDELETE /:id(bytes keptMEDIA_DELETE_RETENTION_DAYS),/:id/restore,/:id/download(302 for public, HMAC-signed short-lived URL for private —MEDIA_SIGNING_SECRET). App key, or a Network service token withmedia.object.upload/media.object.readfor namespace:app;X-OV-Subjectsets the owner. - Retention holds —
media_holds(moderation,dmca,creator_pin,admin,evidence; reason, placed by/at, released by/at, note;/:id/holdsand the staff routes/api/v1/:app/admin/storage/holds, app key only): a held object cannot be deleted by any path (409 / DB trigger) or moved between tiers (moveToHotincluded), and a clip cut from a held VOD is held too (docs). - Reconciliation —
node scripts/reconcile-objects.js [--verify] [--hash]: read-only by default;--verifyHEADs B2/R2 and records location states. Reports missing canonical copies, lost local files, size/hash mismatches, orphan locations, deleted-but-still-served objects and unprojected rows. - Orphan report — monthly job
storage.orphans.scan(MEDIA_ORPHAN_SCAN_DAYS, 30) andnode scripts/vods-orphans-report.js --storage [--no-remote]: local files and B2/R2 keys no row names, recorded copies that are not there, open multipart uploads (Media's and the buckets'). Report only: it never deletes anything (docs). - Public object-size invariant —
MEDIA_PUBLIC_OBJECT_MAX_MB(500) /_TARGET_MB(256) /_WARN_MB(384): v2 refuses oversized public playback uploads;node scripts/object-invariant.jslists offenders and records them inmedia_invariant_violations. No automatic re-encoding.
Storage tiering
vod/vod-storage.js (ported nearly as-is): local → B2 canonical cold tier →
R2 popularity cache; verified uploads before any deletion; periodic sweep
(offload by age/views/last-access, aggressive drain under disk pressure, R2
promote/demote); presigned-302 playback. Knobs live in media_settings under
storage_tier.* (JSON values). Every promotion to and demotion from R2 (the sweep, an admin move, a
clip cut fetching an R2 VOD home, the eviction drill) is logged in media_tier_decisions with the
inputs it saw (views, last access, provider, recording, hold, size), the thresholds in force and their
source, the reason and the outcome; /admin/storage/tiers/policy shows the policy and the log, and
media_tier_decisions_24h{target="vod",action,outcome} counts them on /metrics (test/tier-decisions.test.js).
Native v2 objects (roadmap WS-G task 11; docs) keep
their canonical copy and, when popular, get a verified copy in R2 that /o/:id plays from. Popularity is
unique viewers per object per UTC day, counted on /o/:id without storing an IP address or subject id
(a daily-salted hash, deleted with its salt when the day ends; only the counts stay, 30 days), summed over
7 days. The policy is the revisioned namespace media.object_tier: promote at promoteMinUniqueViewers7d
(500) unique viewers, viewed within 3 days, 16 MB to 4 GB, at most 3 per sweep; demote after 14 idle days
or when the object is no longer ready. Its activation gate active is off by default: the sweep (step 4
of this one) then moves nothing and records what it would do as dry_run decisions. Holds block every
move, only a verified canonical copy is promoted (re-hashed; the R2 copy's size and sha256 are checked
before it is used), and a demotion never removes the last good copy. Every object classifies into one of the six
storage classes (video, image, download, game-asset, attachment, backup): kind and mime type decide
video and image, and an app may declare any class in the object's metadata (metadata.class, or media_class /
placement_class), so each class's budgets, hysteresis band and R2 storage ceiling act on that class alone.
Decisions:
media_object_tier_decisions, /admin/storage/tiers/objects/*, /me/ops, and
media_tier_decisions_24h{target="object"} (test/object-tiering.test.js).
The storage engine's CLI:
node server/vod/vod-storage.js check|drain [targetPct]
Background jobs wired in index.js: tiering sweep, VOD health job (probe scan,
master-recovery repair, quarantine + cleanup, orphaned-master sweep), disk
guardian (refuses new recordings when free space is critical), stale live-thumb
cleanup, and an on-boot sweep that finalizes recordings orphaned by an unclean
shutdown.
Not ported
- Song-request media queue (
media/yt-dlp downloader/queue): deeply coupled to Live's channels/chat/coins subsystems; Live keeps it local. - Server-side live thumbnail grabbers (RTMP-FLV / JSMPEG-WS / SFU PlainRTP) and JSMPEG stdin recording: sources live inside the apps now; apps POST frames / use RTMP-RTP-chunk ingest instead.
- VOD part rotation + live-recording reconciler: tied to the predecessor's stream registry; apps re-trigger ingest on their side.
- Comments on VODs/clips, notifications, permissions/moderation ranks: owned by the apps.
VOD storage tiering & disk safety
server/vod/vod-storage.js keeps the local VOD volume healthy without anyone watching it:
- Drain policy — a drain starts when the disk is above
hotDiskPressurePct(70%) or has less thanminFreeGb(25 GB) free, and stops only when it is underlocalLowWaterPct(60%) and hastargetFreeGb(40 GB) free. AbovecriticalDiskPct(90%) recordings finished more thancriticalMinAgeHours(2 h) ago are eligible; otherwise 1 day. Least-recently-watched VODs go first. A pass that frees nothing raisesstorage.alert drain_stalledonly when eligible VODs could not be moved, or when free space is under the recorder'sVOD_DISK_WARN_GB(recorderWarnGb, 15 GB); a disk shared with other services that stays above the targets with nothing old enough to move is logged once, when that starts. All values live inmedia_settingsasstorage_tier.*(PUT /api/v1/:app/admin/storage/tiers/settings). - Only real files are candidates. Rows without a file (a recording that was refused or never produced output, legacy paths) are quarantined as
health_status='missing_file'and excluded — they used to fill the candidate list and starve the drain. - Cadence — while a drain is still needed the sweep re-runs every
pressureRetryMs(2 min), otherwise everysweepIntervalMs(15 min). A VOD whose upload failed is skipped for 30 min so one bad file cannot pin the drain. - Bounded uploads — each upload is aborted past
uploadTimeoutFloorMs + size / uploadMinThroughputMBps; a watchdog logs a sweep that has run for hours. - Stall alerts — after
alertAfterStalledPasses(2) consecutive pressure passes that freed nothing, and whenever the disk is critical, Media queues amedia.storage.alertevent;media.storage.recoveredfollows once the drain works again. Each pass under pressure logs candidates / uploaded / failed / backed-off / quarantined. - Status —
GET /api/v1/:app/admin/storageincludessweep(last result, next run, stalled flag, needsDrain/critical).
Run npm test for the policy regression tests.
Acceptance
npm test runs every test/*.test.js in turn (each process gets a migrated PGlite database, or the
PostgreSQL containers under npm run test:pg; local stubs, no network). What they prove includes: tenancy
and developer-project tenants (app-tenants, app-tenant-abuse); the object
model, multipart, reconcile, verification and tiering (objects-*, object-tiering, tier-*); jobs
with leases and fencing (jobs*); VOD finalize, durations and storage policy (vod-*); private items
stay private and nobody acts on another's ids (security-private, security-idor,
private-not-found); SSRF, secrets and open redirects (security-ssrf, security-secrets,
open-redirect); client addresses (trust-proxy); revocation and account data (revocations,
account-data); events in the change's transaction (outbox-transaction, object-events); the
restore-drill mode (drill-mode); per-actor limits (actor-limits); and the previous release against
this one (n-1). npm run smoke boots a temp instance end to end.
Security
Reporting a vulnerability: SECURITY.md. The rules the code keeps:
- Auth. App API keys are stored as sha256 hashes and compared in constant time, valid only for
their own tenant. Network user JWTs and principal tokens are verified offline (RS256, audience
openvibe.media); principal tokens are checked per verb and namespace; sandbox tokens reach only their project's sandbox tenant. - Private data. Anyone who may not see a private item gets the answer a missing id gets
(
test/private-not-found.test.js); sandbox items need a valid signature; uploads a client labels as HTML, SVG or XML are served as attachments withnosniff; view counts store no identifiers. - Network exposure.
trust proxyis loopback only, so a direct caller cannot choose its address;/metricsanswers direct loopback callers only;/internal/eventsdeliveries are signed (MEDIA_INBOUND_EVENTS_SECRET). - Egress. Recording pulls only from allow-listed RTMP servers (
MEDIA_RTMP_PULL_ALLOW,test/rtmp-allowlist.test.js); avatar ingestion fetches through the SSRF guard; other calls go to the configured Network, Live, Events, B2 and R2 endpoints. - Secrets.
OV_OAUTH_CLIENT_SECRET,MEDIA_SIGNING_SECRET,MEDIA_SECRET,MEDIA_B2_*,MEDIA_R2_*,MEDIA_INBOUND_EVENTS_SECRETandVIEW_HASH_SECRET(the secret visitor hashes are derived from; it falls back toMEDIA_SECRET) live in/etc/openvibe/media.env(0600), by name only.
Deploy
Production deploys with sudo ovhost deploy media on the host (strategy git-checkout: fetch,
fast-forward /opt/openvibe.media, install on a lockfile change, restart, wait for /api/ready, 60 s).
ovhost refuses the restart while a VOD is recording (recordings_in_progress on /api/ready); --wait-idle
holds it until none is. The unit is openvibe-media.service (deploy/systemd/,
runs as ubuntu, media under /opt/openvibe.media/data) on 127.0.0.1:4100, the env file
/etc/openvibe/media.env. The listener belongs to openvibe-media.socket (with the
openvibe-media.service.d/socket.conf drop-in): requests that arrive during a restart wait for the new process
instead of getting 502s; ovhost restarts the service, never the socket. The database is ov_media on the host's data role (sudo /opt/openvibe.host/roles/data/add-service.sh media
writes its settings); the release migrates it at boot; nginx serves openvibe.media from
deploy/nginx/openvibe.media.conf. After a deploy, record the N-1
fixtures (npm run n-1:record).
Rollback: ovhost puts the previous sha back by itself when /api/ready does not answer 2xx after the
restart; afterwards sudo ovhost rollback media --to <sha>.
- openvibe-contracts: v0.216.0
- openvibe-sdk: v0.65.0
- openvibe-shared: v4.15.2