openvibers/openvibe.media Commits
Expose the timeline HLS URL on the v1 VOD and clip API (#36)
Changes
@@@ -233,7 +233,7 @@ MEDIA_APP_KEYS="live:key1,games:key2"| POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow || POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) || POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, webhook |--| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness)) |++| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness); `hls_url`, with `hls_expires_at` when signed: the object's master playlist with `MEDIA_HLS_ENABLED`, docs/media-fabric.md §3) || GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) || PUT | `/vods/:id` | `{ title?, description?, visibility? }` || DELETE | `/vods/:id` | deletes local + B2 + R2 objects + row |@@@ -271,7 +271,7 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every| method | path | notes ||---|---|---|| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |--| GET | `/clips/:id` | status: `processing | ready | failed` |++| GET | `/clips/:id` | status: `processing | ready | failed`; with `MEDIA_HLS_ENABLED` also `hls_url` (and `hls_expires_at` when signed) — the clip's playlist over its own timeline or, virtual, its source's window (docs/media-fabric.md §4) || POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) || GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs || PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |@@@ -463,7 +463,7 @@ Counters are per process (a restart forgets them). `test/actor-limits.test.js`.| `GET /live/:sel/transcript.json` | **transcript + AI timeline API** — full audio-transcription log and AI overview timeline. `:sel` = slot id / slot slug → slot-scoped sessions; **`@username`** → user-scoped (all their slots, works while offline). Returns `{ live, current, sessions[] (each: ai_overview, transcript, duration…), streamer (overview + stream memories), user }`. `?limit=1..50` sessions (default 10), `?app=`. Cached **30s** (that's the rate limit), CORS-open. || `GET /live/:sel/chat-insight.json` | **chat insight API** — a user's chat-related AI insight/timeline (`:sel` = `@username` or numeric user id): today-vs-alltime chat overviews, condensed memory, event timeline, plus their streamer overview + stream memories when they stream. Proxied from the app's public chat-AI API over loopback; cached **30s**, CORS-open. || `GET /v/:id/transcript.json` | **VOD transcript API** — transcript + AI overview for one existing VOD id (`{ vod_id, title, duration_seconds, ai_overview, transcript, ai_analyzed_at }`). Private VODs → 404. Cached 30s, CORS-open. |--| `GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8`, `/o/:id/source/{init.mp4,NNNNNN.m4s}` | segment-native video (F3.1), only with `MEDIA_HLS_ENABLED=1` (off by default; off, these 404): HLS playlists written from the object's `media_timeline` rows (VOD, `#EXT-X-ENDLIST`) and its CMAF segments (Range; a segment packed by `object.pack` (F3.3) is a byte range of its chunk). Same check as `GET /o/:id` (and, like it, no `Access-Control-Allow-Origin` on a private or sandbox answer): private and sandbox objects only with the object's `?exp&sig` (`/api/v2/:app/objects/:id/download?format=json` → `hls_url`, a playlist token valid `MEDIA_HLS_PLAYLIST_TTL_S` (6 h; 60 s to 12 h) that no other route accepts; a download signature works too), carried onto every URI; deleted 410. The `object.cmaf` job makes the timeline; a recording's finalize queues it, and `download?format=json` queues it for an object that has none yet. `noindex` |++| `GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8`, `/o/:id/source/{init.mp4,NNNNNN.m4s}` | segment-native video (F3.1), only with `MEDIA_HLS_ENABLED=1` (off by default; off, these 404): HLS playlists written from the object's `media_timeline` rows (VOD, `#EXT-X-ENDLIST`) and its CMAF segments (Range; a segment packed by `object.pack` (F3.3) is a byte range of its chunk). Same check as `GET /o/:id` (and, like it, no `Access-Control-Allow-Origin` on a private or sandbox answer): private and sandbox objects only with the object's `?exp&sig` (`/api/v2/:app/objects/:id/download?format=json` → `hls_url`, a playlist token valid `MEDIA_HLS_PLAYLIST_TTL_S` (6 h; 60 s to 12 h) that no other route accepts; a download signature works too), carried onto every URI; deleted 410. The `object.cmaf` job makes the timeline; a recording's finalize queues it, and `download?format=json` queues it for an object that has none yet; the v1 VOD and clip answers carry the same `hls_url` (and `hls_expires_at` when signed). `noindex` || `GET /robots.txt` | crawler policy (`openvibe-shared/seo.robotsTxt`: AI and search crawlers named, `/api/`, `/auth/`, `/internal/`, `/o/`, `/metrics`, `/live/`, `/me` disallowed) and the sitemap || `GET /sitemap.xml` | the media index and the watch pages Media is the canonical home of: public, playable VODs and clips of apps other than Live (Live lists its own `/vod` and `/clip` pages), never AI clips, private, unlisted, sandbox, recording or failed items; thumbnails as image entries. Every listed page renders `index, follow` || `GET /llms.txt` | a map of the site for language-model crawlers |@@@ -40,7 +40,8 @@"every storage class in the native-object sweep: kind and mime type classify video/image/download, an app may declare game-asset/attachment/backup (or any of the six) in the object's metadata (metadata.class, or media_class/placement_class), and each class spends its own per-sweep budget, hysteresis band and R2 storage ceiling","segment-native video (F3.1, behind MEDIA_HLS_ENABLED): the media_timeline index, the object.cmaf job (CMAF/fMP4 segments of a finished video on its own keyframes, sha256 and durability per row, idempotent) and HLS playlists written from the timeline at /o/:id/master.m3u8 under the object's access checks","packing after finalize (F3.3, behind MEDIA_HLS_ENABLED): the object.pack job concatenates durable segments into ~60 s chunk objects verified by sha256, each media_timeline row naming its byte range; the segment route serves a packed segment as a ranged read of its chunk; the per-segment keys are deleted once the chunk is durable; segment providers come from the placement router; a finished object.cmaf queues object.pack by itself (dedupeActive: a rerun joins an active pack, a later cut queues a new one)",-- "timeline sprites (F3.1, behind MEDIA_HLS_ENABLED): object.sprite cuts its sheet from the object's media_timeline rows — each sample time picks the row covering it and one frame is decoded from that row's bytes (this node's segment, or a packed row's byte range in its chunk), checked against the row's sha256 — with the same layout and metadata as the seek path, so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame"++ "timeline sprites (F3.1, behind MEDIA_HLS_ENABLED): object.sprite cuts its sheet from the object's media_timeline rows — each sample time picks the row covering it and one frame is decoded from that row's bytes (this node's segment, or a packed row's byte range in its chunk), checked against the row's sha256 — with the same layout and metadata as the seek path, so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame",++ "HLS discovery on the v1 shapes (F3.c, behind MEDIA_HLS_ENABLED): the VOD and clip answers (vodPublic/clipPublic — get, list, create, update) carry hls_url (and hls_expires_at when the object is private or sandbox), the same field and access rules as GET /o/:id/download?format=json, read through one shared discovery (server/objects/hls.js); an object without a timeline gets object.cmaf queued and the field appears once it ran, and a virtual clip plays its source's window without being cut a timeline of its own"],"updated": "2026-10-04"}@@@ -159,7 +159,12 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}(`hls`), valid for `MEDIA_HLS_PLAYLIST_TTL_S` (6 h by default, 60 s to 12 h) so a viewer plays a long VOD on one pageload, carried onto every playlist and segment URI, and accepted only by the HLS routes (never by `GET /o/:id` or anyother route); the HLS routes still accept the object's download signature, so earlier URLs keep working. A public-- object's `hls_url` stays unsigned. `GET …/download?format=mp4` answers the object's `remux` variant (`object.remux`,++ object's `hls_url` stays unsigned. The v1 VOD and clip shapes carry the same field (`GET /api/v1/:app/vods/:id` and++ `/clips/:id`, their lists, creates and updates through `vodPublic`/`clipPublic`), from the shared discovery++ (`server/objects/hls.js`; the same presence and access rules as `…/download?format=json`), so an app renders a player++ from the record it already has: a VOD with a timeline, a materialized clip with one, or a virtual clip playing its++ source's window; one without gets `object.cmaf` queued and the field appears once it ran. `GET …/download?format=mp4`++ answers the object's `remux` variant (`object.remux`,`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux(`dedupeActive`) and answers `202 { job_id }` while it is missing.- **F3.5, shipped: virtual and materialized clips over the timeline (§4).**@@@ -194,9 +199,11 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}timeline of its own uses it. A virtual clip's source must be the same tenant's ready object; a deleted source stopsit. `…/download?format=json` names its `hls_url` and never queues `object.cmaf` for it; `/c/:id` (public, the owner,or a `getc` URL from `GET /clips/:id/signed-url`) answers a virtual clip with a 302 to its master playlist (signed-- with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); `playback_url` is-- unchanged. Its object's metadata carries `virtual: true`: readiness counts it `bytes_verified`/`playable` without a-- location of its own, and the verify job skips it. **Materialized** = today's `clip.cut` full++ with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); the clip answers++ (`GET /api/v1/:app/clips/:id` and the list/create/update shapes) carry its `hls_url` (the clip's own token when++ closed), and `playback_url` is unchanged. Its object's metadata carries `virtual: true`: readiness counts it++ `bytes_verified`/`playable` without a location of its own, and the verify job skips it. **Materialized** = today's++ `clip.cut` fullre-encode into the clip's own file: with `materialize: true` on `POST /clips` or `POST /clips/:id/recut`, or whenthe source has no timeline (or the flag is off). A virtual clip keeps playing while it is materialized and staysvirtual if the cut fails; a clip with a file is never turned virtual (its recut stays a cut). The retry sweepernnew file mode 100644@@@ -0,0 +1,190 @@++/**++ * OpenVibe.Media — HLS discovery for an object (docs/media-fabric.md §3, F3.c)++ *++ * One place answers "does this object have a serviceable HLS timeline, and what is its playlist URL?": the v2++ * /o/:id/download?format=json route (objects/routes.js) and the v1 VOD and clip shapes (vod/finalize.js vodPublic,++ * vod/clips-routes.js clipPublic) all use it, so the field and its access rules cannot drift between them.++ *++ * A playlist exists when MEDIA_HLS_ENABLED is on and the object's source timeline has a media row (seq > 0) — its own++ * rows, or for a virtual clip (F3.5) its source's window rows. With none, object.cmaf is queued (idempotent) and no URL++ * is answered, unless the object is a virtual clip (its source's window plays it; it is never cut a timeline of its own).++ * A single-item read takes that path (deduped in-process); a list read uses discoveryMany, which answers a whole page in++ * a constant number of queries and never queues anything.++ *++ * The URL is built exactly as the v2 route does: an open object (not private, not in a sandbox tenant) gets the plain++ * /o/<id>/master.m3u8; a closed one gets a signed playlist (purpose 'hls', signing.signedPlaylistUrl) and its expiry.++ */++'use strict';++++const config = require('../config');++const db = require('../db/database');++const model = require('./model');++const signing = require('./signing');++++/**++ * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata++ * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.++ */++async function clipWindow(obj) {++ if (!obj || obj.kind !== 'clip') return null;++ const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);++ if (!rel) return null;++ const src = await model.getObject(rel.to_object_id);++ if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;++ const md = model.parseJson(rel.metadata, {});++ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);++ const endMs = Math.round(Number(md.end_time) * 1000) || 0;++ return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;++}++++/**++ * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a++ * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's++ * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.++ */++async function timelineOf(obj) {++ const timeline = require('./timeline');++ const own = await timeline.list(obj.id, timeline.SOURCE);++ if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };++ const window = await clipWindow(obj);++ if (!window) return { rows: [], window: null };++ return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };++}++++/** Whether an object is served openly: not private and not in a developer-project sandbox tenant (GET /o/:id's rule). */++async function isOpen(obj) {++ return obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id);++}++++// The lazy object.cmaf one read can queue (discoveryForObject) is deduped per process: objectId → last queued ms.++// A bounded Map (QUEUE_MAX entries, expired ones swept first) keeps it from growing without limit; the TTL is a++// window, not correctness — timeline-queue.queueCmaf is itself idempotent, this only spares a hot read its queries.++const QUEUE_TTL_MS = 10 * 60 * 1000;++const QUEUE_MAX = 1000;++const _queuedCmaf = new Map();++++/** Queue object.cmaf for an object at most once per QUEUE_TTL_MS in this process. Never throws. */++async function queueCmafOnce(appId, objectId) {++ const now = Date.now();++ const last = _queuedCmaf.get(objectId);++ if (last != null && now - last < QUEUE_TTL_MS) return;++ if (_queuedCmaf.size >= QUEUE_MAX) {++ for (const [id, at] of _queuedCmaf) if (now - at >= QUEUE_TTL_MS) _queuedCmaf.delete(id);++ while (_queuedCmaf.size >= QUEUE_MAX) _queuedCmaf.delete(_queuedCmaf.keys().next().value);++ }++ _queuedCmaf.set(objectId, now);++ try { await require('./timeline-queue').queueCmaf(appId, objectId); } catch (err) { console.warn(`[HLS] Timeline queue failed for ${objectId}:`, err.message); }++}++++/** What an object with a serviceable timeline is discovered by: { hls_url, hls_expires_at? } (open stays unsigned). */++function playlistFor(obj, open) {++ if (open) return { hls_url: `${config.publicUrl}/o/${obj.id}/master.m3u8` };++ const signed = signing.signedPlaylistUrl(obj.id);++ return { hls_url: signed.url, hls_expires_at: signed.expires_at };++}++++/**++ * The HLS answer for an already-loaded ready object, or null: { hls_url, hls_expires_at? } once the timeline exists;++ * otherwise object.cmaf is queued (idempotent) for an object that could still get one and nothing is answered. Never++ * throws — a lookup or queue failure is logged and the object simply has no hls_url this time.++ */++async function discoveryForObject(obj) {++ if (!config.hls.enabled || !obj || obj.lifecycle_status !== 'ready') return null;++ try {++ const tl = await timelineOf(obj);++ if (!tl.rows.some((r) => Number(r.seq) > 0)) {++ if (!tl.window) await queueCmafOnce(obj.app_id, obj.id);++ return null;++ }++ return playlistFor(obj, await isOpen(obj));++ } catch (err) {++ console.warn(`[HLS] Discovery failed for ${obj.id}:`, err.message);++ return null;++ }++}++++/** discoveryForObject for a v1 row's object_id (null/absent id answers nothing). */++async function discovery(objectId) {++ if (!config.hls.enabled || !objectId) return null;++ try {++ return await discoveryForObject(await model.getObject(String(objectId)));++ } catch (err) {++ console.warn(`[HLS] Discovery failed for ${objectId}:`, err.message);++ return null;++ }++}++++/**++ * The HLS answers for a whole page of v1 rows (the VOD/clip lists), in a constant number of queries: the objects,++ * which of them have a timeline, their sandbox tenants, and — only for clip objects with no timeline of their own —++ * the source window they play over (F3.5). discoveryForObject's per-row path would run all of that once per row,++ * and queue object.cmaf for each row without a timeline; a list read must do neither.++ *++ * Returns a Map(object_id → { hls_url, hls_expires_at? }). A row with no serviceable timeline (and any lookup that++ * fails) simply has no entry. Never queues object.cmaf. Never throws.++ */++async function discoveryMany(rows) {++ const out = new Map();++ if (!config.hls.enabled || !Array.isArray(rows) || !rows.length) return out;++ try {++ const objectIds = [...new Set(rows.map((r) => r && r.object_id).filter(Boolean).map(String))];++ if (!objectIds.length) return out;++ const timeline = require('./timeline');++++ const ready = (await db.all('SELECT * FROM media_objects WHERE id = ANY(?)', [objectIds]))++ .filter((o) => o.lifecycle_status === 'ready');++ if (!ready.length) return out;++ const readyIds = ready.map((o) => String(o.id));++ const withTimeline = new Set((await db.all(++ 'SELECT DISTINCT object_id FROM media_timeline WHERE rendition = ? AND seq > 0 AND object_id = ANY(?)',++ [timeline.SOURCE, readyIds])).map((r) => String(r.object_id)));++++ // A clip object without a timeline of its own is a virtual clip: its playlist is its source's window.++ const virtual = new Set();++ const needWindow = ready.filter((o) => o.kind === 'clip' && !withTimeline.has(String(o.id)));++ if (needWindow.length) {++ const rels = await db.all("SELECT from_object_id, to_object_id, metadata FROM media_relationships WHERE relation = 'clip_of' AND from_object_id = ANY(?) ORDER BY id",++ [needWindow.map((o) => String(o.id))]);++ const relOf = new Map();++ for (const r of rels) if (!relOf.has(String(r.from_object_id))) relOf.set(String(r.from_object_id), r);++ const sourceIds = [...new Set(rels.map((r) => String(r.to_object_id)))];++ const sourceOf = new Map((sourceIds.length ? await db.all('SELECT * FROM media_objects WHERE id = ANY(?)', [sourceIds]) : [])++ .map((s) => [String(s.id), s]));++ const sourceRows = new Map();++ if (sourceIds.length) {++ for (const r of await db.all('SELECT * FROM media_timeline WHERE rendition = ? AND object_id = ANY(?) ORDER BY object_id, seq', [timeline.SOURCE, sourceIds])) {++ const k = String(r.object_id);++ if (!sourceRows.has(k)) sourceRows.set(k, []);++ sourceRows.get(k).push(r);++ }++ }++ for (const clip of needWindow) {++ const rel = relOf.get(String(clip.id));++ const src = rel && sourceOf.get(String(rel.to_object_id));++ if (!src || src.app_id !== clip.app_id || src.lifecycle_status !== 'ready') continue;++ const md = model.parseJson(rel.metadata, {});++ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);++ const endMs = Math.round(Number(md.end_time) * 1000) || 0;++ if (!(endMs > startMs)) continue;++ const clipped = timeline.clipRows(sourceRows.get(String(src.id)) || [], startMs, endMs);++ if (clipped.some((r) => Number(r.seq) > 0)) virtual.add(String(clip.id));++ }++ }++++ // isSandboxTenant, resolved once per distinct app of the page instead of once per row.++ const appIds = [...new Set(ready.map((o) => String(o.app_id)))];++ const sandbox = new Set((await db.all("SELECT app_id FROM apps WHERE env = 'sandbox' AND app_id = ANY(?)", [appIds]))++ .map((r) => String(r.app_id)));++++ for (const obj of ready) {++ const id = String(obj.id);++ if (!withTimeline.has(id) && !virtual.has(id)) continue;++ out.set(id, playlistFor(obj, obj.visibility !== 'private' && !sandbox.has(String(obj.app_id))));++ }++ } catch (err) {++ console.warn('[HLS] Batch discovery failed:', err.message);++ }++ return out;++}++++module.exports = { timelineOf, discoveryForObject, discovery, discoveryMany };@@@ -51,6 +51,7 @@ const config = require('../config');const model = require('./model');const invariant = require('./invariant');const signing = require('./signing');++const hls = require('./hls');const ctypes = require('./content-type');const multipart = require('./multipart');const namespaces = require('./namespaces');@@@ -673,19 +674,15 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.// A clip without its own timeline plays its window of the source's (F3.5) and is never cut a CMAF timeline here.-- const tl = json && config.hls.enabled ? await timelineOf(obj) : null;-- const hls = !!tl && tl.rows.some((r) => Number(r.seq) > 0);-- if (tl && !hls && !tl.window) {-- try { await require('./timeline-queue').queueCmaf(obj.app_id, obj.id); } catch (err) { console.warn(`[Objects] Timeline queue failed for ${obj.id}:`, err.message); }-- }++ // The same answer shapes the v1 VOD and clip (objects/hls.js discovery; vodPublic/clipPublic).++ const hlsInfo = json ? await hls.discoveryForObject(obj) : null;if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {const url = model.legacyPublicUrl(obj) || `${config.publicUrl}/o/${obj.id}`;-- return json ? res.json({ url, expires_at: null, public: true, ...(hls && { hls_url: `${config.publicUrl}/o/${obj.id}/master.m3u8` }) }) : res.redirect(302, url);++ return json ? res.json({ url, expires_at: null, public: true, ...(hlsInfo || {}) }) : res.redirect(302, url);}const signed = signing.signedDownloadUrl(obj.id, req.query.ttl);if (['1', 'true'].includes(String(req.query.redirect || ''))) return res.redirect(302, signed.url);-- const playlist = hls && signing.signedPlaylistUrl(obj.id);-- res.json({ url: signed.url, expires_at: signed.expires_at, public: false, ...(playlist && { hls_url: playlist.url, hls_expires_at: playlist.expires_at }) });++ res.json({ url: signed.url, expires_at: signed.expires_at, public: false, ...(hlsInfo || {}) });});/**@@@ -783,36 +780,8 @@ const publicRouter = express.Router();const HLS_TYPE = 'application/vnd.apple.mpegurl';const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2----/**-- * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata-- * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.-- */--async function clipWindow(obj) {-- if (!obj || obj.kind !== 'clip') return null;-- const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);-- if (!rel) return null;-- const src = await model.getObject(rel.to_object_id);-- if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;-- const md = model.parseJson(rel.metadata, {});-- const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);-- const endMs = Math.round(Number(md.end_time) * 1000) || 0;-- return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;--}----/**-- * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a-- * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's-- * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.-- */--async function timelineOf(obj) {-- const timeline = require('./timeline');-- const own = await timeline.list(obj.id, timeline.SOURCE);-- if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };-- const window = await clipWindow(obj);-- if (!window) return { rows: [], window: null };-- return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };--}++// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in++// objects/hls.js, shared with the v1 VOD/clip discovery./** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */async function hlsObject(req, res, next) {@@@ -846,14 +815,14 @@ const hlsRoute = (fn) => async (req, res, next) => {publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {const timeline = require('./timeline');-- const { rows } = await timelineOf(obj);++ const { rows } = await hls.timelineOf(obj);if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));}));publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {const timeline = require('./timeline');-- const { rows } = await timelineOf(obj);++ const { rows } = await hls.timelineOf(obj);if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));}));@@@ -908,7 +877,7 @@ publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed })let row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;// A virtual clip serves only the source segments inside its window (timelineOf): any other name is a 404, whatever// the signature, so a clip's token never reaches the rest of its source.-- if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await timelineOf(obj)).rows.find((r) => r.name === name) || null;++ if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await hls.timelineOf(obj)).rows.find((r) => r.name === name) || null;if (!row) return res.status(404).json({ error: 'Not found' });const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };@@@ -40,7 +40,7 @@ let _timer = null, _busy = false;function freeBytes() {try { const st = fs.statfsSync(path.resolve(config.vod.path)); return Number(st.bavail) * Number(st.bsize); } catch { return 0; }}--async function _clipPublic(clip) { try { return await require('./clips-routes').clipPublic(clip); } catch { return clip; } }++async function _clipPublic(clip) { try { return await require('./clips-routes').clipPublic(clip, { hls: false }); } catch { return clip; } }const RECUT_CAP_MS = 30 * 60 * 1000; // one clip can never hold the queue longer than this@@@ -54,8 +54,10 @@ const clipUpload = multer({});// { readiness: true } (API responses) adds the object's readiness levels (server/objects/readiness.js);--// the clip.ready / clip.failed payloads leave them out, as vodPublic's do.--async function clipPublic(clip, { readiness = false } = {}) {++// the clip.ready / clip.failed payloads leave them out, as vodPublic's do. `hls` selects the playlist++// fields: undefined = discover them for this row (a single-item read), false = omit them (an event payload++// must never carry a signed URL into the outbox), an object = the answer a list route already batched.++async function clipPublic(clip, { readiness = false, hls } = {}) {if (!clip) return null;const out = {unique_views: clip.unique_views || 0,@@@ -91,6 +93,13 @@ async function clipPublic(clip, { readiness = false } = {}) {view_count: clip.view_count || 0,created_at: clip.created_at,};++ // The HLS master playlist of the clip's object, when it has one (objects/hls.js): the same field and access rules as++ // GET /o/:id/download?format=json — unsigned for an open object, a signed playlist (+ hls_expires_at) for a private++ // or sandbox one. Absent with MEDIA_HLS_ENABLED off, before the object is ready, and until its timeline (or, for a++ // virtual clip, its source's window) has a segment; object.cmaf is queued for a materialized clip that has none.++ // Never throws.++ if (hls === undefined) Object.assign(out, await require('../objects/hls').discovery(clip.object_id) || {});++ else if (hls) Object.assign(out, hls);if (readiness) out.readiness = await require('../objects/readiness').forRow(clip);return out;}@@@ -329,7 +338,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)};const clips = await db.listClips(req.appId, filters);const total = await db.countClips(req.appId, filters);-- res.json({ clips: (await Promise.all(clips.map(async c => await clipPublic(c, { readiness: true })))), total, limit, offset, hasMore: offset + clips.length < total });++ const hls = await require('../objects/hls').discoveryMany(clips);++ res.json({ clips: await Promise.all(clips.map(async c => await clipPublic(c, { readiness: true, hls: hls.get(String(c.object_id)) || null }))), total, limit, offset, hasMore: offset + clips.length < total });} catch (err) {console.error('[Clips] List error:', err.message);res.status(500).json({ error: 'Failed to list clips' });@@@ -37,8 +37,10 @@ function isFinalizing(vodId) {// Public shape of a vod row for API responses + webhooks. { readiness: true } (API responses) adds the// object's readiness levels (server/objects/readiness.js); webhook payloads leave them out (their shape--// predates the object model and stays as it is).--async function vodPublic(vod, { readiness = false } = {}) {++// predates the object model and stays as it is). `hls` selects the playlist fields: undefined = discover++// them for this row (a single-item read), false = omit them (an event payload must never carry a signed++// URL into the outbox), an object = the answer the list route already batched (objects/hls.discoveryMany).++async function vodPublic(vod, { readiness = false, hls } = {}) {if (!vod) return null;const out = {unique_views: vod.unique_views || 0,@@@ -78,6 +80,12 @@ async function vodPublic(vod, { readiness = false } = {}) {created_at: vod.created_at,meta: (() => { try { return JSON.parse(vod.meta_json || '{}'); } catch { return {}; } })(),};++ // The HLS master playlist of the VOD's object, when it has one (objects/hls.js): the same field and access rules as++ // GET /o/:id/download?format=json — unsigned for an open object, a signed playlist (+ hls_expires_at) for a private++ // or sandbox one. Absent with MEDIA_HLS_ENABLED off, before the object is ready, and until its timeline exists++ // (object.cmaf is queued for it, idempotently). Never throws.++ if (hls === undefined) Object.assign(out, await require('../objects/hls').discovery(vod.object_id) || {});++ else if (hls) Object.assign(out, hls);if (readiness) out.readiness = await require('../objects/readiness').forRow(vod);return out;}@@@ -118,7 +126,7 @@ function rebuildWebmFromMaster(masterPath, webmPath) {*/async function _commitVodOutcome(vod, event, change, row = async () => await db.getVodById(vod.id)) {try {-- return (await announce(vod.app_id, event, { change: async () => await objects().withObject('vod', vod.id, change), payload: async () => await vodPublic(row()) })).data;++ return (await announce(vod.app_id, event, { change: async () => await objects().withObject('vod', vod.id, change), payload: async () => await vodPublic(row(), { hls: false }) })).data;} catch (err) {console.error(`[VOD] vod ${vod.id}: ${event} not committed:`, err.message);return null;@@@ -383,7 +391,7 @@ async function _doFinalize(vodId, opts) {await announce(vod.app_id, 'vod.ready', {change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),-- payload: async () => await vodPublic(await db.getVodById(vodId)),++ payload: async () => await vodPublic(await db.getVodById(vodId), { hls: false }),});const ready = await db.getVodById(vodId);// Segment-native video (MEDIA_HLS_ENABLED): the finished recording's timeline is cut by object.cmaf. A queue error@@@ -292,7 +292,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)};const vods = await db.listVods(req.appId, filters);const total = await db.countVods(req.appId, filters);-- res.json({ vods: (await Promise.all(vods.map(async v => await vodPublic(v, { readiness: true })))), total, limit, offset, hasMore: offset + vods.length < total });++ const hls = await require('../objects/hls').discoveryMany(vods);++ res.json({ vods: (await Promise.all(vods.map(async v => await vodPublic(v, { readiness: true, hls: hls.get(String(v.object_id)) || null })))), total, limit, offset, hasMore: offset + vods.length < total });} catch (err) {console.error('[VOD] List error:', err.message);res.status(500).json({ error: 'Failed to list VODs' });@@@ -96,6 +96,10 @@ const express = require('express');assert.strictEqual((await call('POST', `/api/v1/live/vods/${vodId}/chunks`, f)).body.status, 'created');const vodObj = (await finalizeVod(vodId)).object_id;const vodRow = await model.getObject(vodObj);++ // Before the timeline exists: no hls_url, and the VOD answer queues object.cmaf (idempotent with the finalize's).++ r = await call('GET', `/api/v1/live/vods/${vodId}`);++ assert.ok(!('hls_url' in r.body), 'no hls_url before the timeline exists');++ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${vodObj}'`)).length, 1, 'the VOD answer queues object.cmaf once');const bytes = {};const rows = src.map((s) => {const p = timeline.localPathFor(vodRow, 'source', s.name, 'v1');@@@ -109,6 +113,103 @@ const express = require('express');});await timeline.replace(vodObj, 'source', rows);++ // ── The v1 VOD names its HLS playlist: a playlist token (private) that plays through /o/<object>/master.m3u8 ──++ r = await call('GET', `/api/v1/live/vods/${vodId}`);++ assert.strictEqual(r.status, 200, JSON.stringify(r.body));++ const vhu = new URL(r.body.hls_url);++ assert.strictEqual(vhu.pathname, `/o/${vodObj}/master.m3u8`, 'hls_url names the object\'s master playlist');++ assert.ok(signing.verifyPlaylist(vodObj, vhu.searchParams.get('exp'), vhu.searchParams.get('sig')), 'a private VOD\'s hls_url carries a playlist token');++ assert.ok(r.body.hls_expires_at, 'and its expiry');++ assert.strictEqual((await get(r.body.hls_url)).status, 200, 'the VOD hls_url serves 200');++ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${vodObj}'`)).length, 1, 'no extra object.cmaf once the timeline exists');++ // An open VOD's hls_url stays unsigned; put it back for the clip paths below.++ assert.strictEqual((await call('PUT', `/api/v1/live/vods/${vodId}`, { visibility: 'public' })).status, 200);++ const vpub = await call('GET', `/api/v1/live/vods/${vodId}`);++ assert.strictEqual(vpub.body.hls_url, `${config.publicUrl}/o/${vodObj}/master.m3u8`, 'a public VOD\'s hls_url is the plain master URL');++ assert.ok(!('hls_expires_at' in vpub.body), 'and carries no expiry');++ assert.strictEqual((await call('PUT', `/api/v1/live/vods/${vodId}`, { visibility: 'private' })).status, 200);++ const vlist = await call('GET', '/api/v1/live/vods?limit=5&include_private=1');++ assert.ok(vlist.body.vods.find((v) => v.id === vodId).hls_url, 'the VOD list carries hls_url too');++ console.log('✅ GET /vods/:id: hls_url signed for a private VOD, plain for a public one, and it plays');++++ // ── A list read is a constant number of timeline queries and never queues object.cmaf (F3.c review) ──++ // The HLS fields of a page come from objects/hls.discoveryMany: one batched timeline query, not one per row,++ // and a list is not the place to queue object.cmaf (finalize and /download already do).++ {++ const timelineQueue = require('../server/objects/timeline-queue');++ const realQueueCmaf = timelineQueue.queueCmaf;++ const realAll = db.all;++ let queued = 0;++ let timelineQueries = 0;++ timelineQueue.queueCmaf = async (...a) => { queued++; return await realQueueCmaf(...a); };++ db.all = async (sql, params) => { if (/media_timeline/.test(sql)) timelineQueries++; return await realAll(sql, params); };++ const bulk = [];++ try {++ for (let i = 0; i < 6; i++) {++ const oid = await model.createObject({ app_id: 'live', owner_user_id: 5, visibility: 'public', lifecycle_status: 'ready', kind: 'vod', mime_type: 'video/mp4', size_bytes: 1 });++ const vid = (await db.run('INSERT INTO vods (app_id, object_id, title, visibility, is_public) VALUES (?, ?, ?, ?, 1) RETURNING id', ['live', oid, `Bulk ${i}`, 'public'])).lastInsertRowid;++ bulk.push({ vid, oid });++ }++ await timeline.replace(bulk[0].oid, 'source', src); // one of them has a timeline++++ timelineQueries = 0;++ const one = await call('GET', '/api/v1/live/vods?limit=1&include_private=1');++ assert.strictEqual(one.status, 200, JSON.stringify(one.body));++ const q1 = timelineQueries;++ timelineQueries = 0;++ const many = await call('GET', '/api/v1/live/vods?limit=50&include_private=1');++ assert.ok(many.body.vods.length >= 4, `the page lists the bulk VODs (${many.body.vods.length})`);++ assert.strictEqual(timelineQueries, q1, `a page of ${many.body.vods.length} VODs runs the same number of timeline queries as one`);++ assert.strictEqual(timelineQueries, 1, 'exactly one batched timeline query answers the page');++ assert.strictEqual(queued, 0, 'a list request never queues object.cmaf');++ const withHls = many.body.vods.find((v) => v.id === bulk[0].vid);++ assert.strictEqual(withHls.hls_url, `${config.publicUrl}/o/${bulk[0].oid}/master.m3u8`, 'the batched answer reaches the row');++ } finally {++ db.all = realAll;++ timelineQueue.queueCmaf = realQueueCmaf;++ }++ }++++ // ── A single-item read still queues object.cmaf for a timeline-less object, deduped per process ──++ {++ const timelineQueue = require('../server/objects/timeline-queue');++ const realQueueCmaf = timelineQueue.queueCmaf;++ let calls = 0;++ timelineQueue.queueCmaf = async (...a) => { calls++; return await realQueueCmaf(...a); };++ try {++ const fresh = await db.createVod({ app_id: 'live', title: 'No timeline', visibility: 'private' });++ const freshId = fresh.lastInsertRowid;++ await db.run("UPDATE media_objects SET lifecycle_status = 'ready' WHERE id = ?", [(await db.getVodById(freshId, 'live')).object_id]);++ assert.strictEqual((await call('GET', `/api/v1/live/vods/${freshId}`)).status, 200);++ assert.strictEqual((await call('GET', `/api/v1/live/vods/${freshId}`)).status, 200);++ assert.strictEqual(calls, 1, 'two immediate reads queue object.cmaf once (10-minute in-process dedupe)');++ } finally {++ timelineQueue.queueCmaf = realQueueCmaf;++ }++ }++++ // ── An event payload never carries a playlist URL (F3.c review): announce() feeds the durable outbox ──++ // vodPublic also shapes vod.ready and _clipPublic shapes clip.ready/clip.failed; a private VOD's signed++ // hls_url would otherwise sit in the outbox for up to 12 h and reach every subscriber.++ {++ const events = require('../server/events');++ const realRecord = events.record;++ const captured = [];++ events.record = async (event, appId, data) => { captured.push({ event, data }); return await realRecord(event, appId, data); };++ try {++ // The private source's timeline now exists, so a second finalize announces vod.ready with it present.++ assert.ok(await finalizeVod(vodId), 'the second finalize settles');++ const vodReady = captured.filter((e) => e.event === 'vod.ready').pop();++ assert.ok(vodReady, 'vod.ready was announced');++ assert.ok(!('hls_url' in vodReady.data) && !('hls_expires_at' in vodReady.data), 'a vod.ready payload never carries a playlist URL');++ } finally {++ events.record = realRecord;++ }++ // Positive control: the private VOD's own API shape still answers a signed playlist.++ const api = await call('GET', `/api/v1/live/vods/${vodId}`);++ assert.ok(api.body.hls_url && new URL(api.body.hls_url).searchParams.get('sig'), 'the API shape still answers a signed hls_url');++ }++// ── A clip over it is virtual: ready at once, no clip.cut ──r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 5, end_s: 11, visibility: 'private', title: 'Virtual' });assert.strictEqual(r.status, 201, JSON.stringify(r.body));@@@ -129,6 +230,18 @@ const express = require('express');assert.ok(!(await db.get('SELECT 1 FROM media_verifications WHERE object_id = ?', [clipObj])), 'and reports no no_good_copy for it');console.log('✅ POST /clips over a timeline: ready at once (201), storage_provider timeline, playable, no clip.cut, not verified');++ // ── The v1 clip names its playlist too: its own token, over its source window, serving 200 ──++ r = await call('GET', `/api/v1/live/clips/${clipId}`);++ assert.strictEqual(r.status, 200, JSON.stringify(r.body));++ const chu = new URL(r.body.hls_url);++ assert.strictEqual(chu.pathname, `/o/${clipObj}/master.m3u8`, 'the clip\'s hls_url names its own master playlist');++ assert.ok(signing.verifyPlaylist(clipObj, chu.searchParams.get('exp'), chu.searchParams.get('sig')), 'and carries the clip\'s playlist token');++ assert.strictEqual((await get(r.body.hls_url)).status, 200, 'the clip hls_url serves 200');++ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${clipObj}'`)).length, 0, 'a virtual clip is never queued object.cmaf');++ const clist = await call('GET', `/api/v1/live/clips?vod_id=${vodId}&limit=5&include_private=1`);++ assert.ok(clist.body.clips.find((c) => c.id === clipId).hls_url, 'the clip list carries hls_url too');++ console.log('✅ GET /clips/:id: hls_url is the clip\'s playlist token over its source window, and it plays');++// ── Its playlists: hlsObject() rules, the source's segments inside the window ──for (const p of ['master.m3u8', 'source/index.m3u8', 'source/init.mp4', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${clipObj}/${p}`)).status, 404, `unsigned ${p}`);const tok = new URL(signing.signedPlaylistUrl(clipObj).url).search;@@@ -217,9 +330,35 @@ const express = require('express');assert.strictEqual(r.status, 202, JSON.stringify(r.body));assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'MEDIA_HLS_ENABLED off: clip.cut');assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 404, 'and the HLS routes are off');++ assert.ok(!('hls_url' in (await call('GET', `/api/v1/live/vods/${vodId}`)).body), 'flag off: the VOD omits hls_url');++ assert.ok(!('hls_url' in (await call('GET', `/api/v1/live/clips/${clipId}`)).body), 'flag off: the clip omits hls_url');config.hls.enabled = true;console.log('✅ MEDIA_HLS_ENABLED off: clips are cut as before');++ // ── A clip.ready payload likewise never carries the (signed) playlist URL ──++ {++ const clipJobs = require('../server/vod/clip-jobs');++ const payloadClip = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Payload', file_path: null, start_time: 16.5, end_time: 18.5,++ duration_seconds: 2, is_public: 0, visibility: 'private', status: 'processing' })).lastInsertRowid;++ const win = await clipJobs.timelineWindow(await db.getVodById(vodId), 16.5, 18.5);++ assert.ok(win, 'a window over the source timeline');++ const events = require('../server/events');++ const realRecord = events.record;++ const captured = [];++ events.record = async (event, appId, data) => { captured.push({ event, data }); return await realRecord(event, appId, data); };++ try {++ await clipJobs.makeVirtual(payloadClip, win);++ } finally {++ events.record = realRecord;++ }++ const clipReady = captured.filter((e) => e.event === 'clip.ready').pop();++ assert.ok(clipReady, 'clip.ready was announced');++ assert.ok(!('hls_url' in clipReady.data) && !('hls_expires_at' in clipReady.data), 'a clip.ready payload never carries a playlist URL');++ // Positive control: the clip's API shape still answers a signed playlist.++ const api = await require('../server/vod/clips-routes').clipPublic(await db.getClipById(payloadClip), { readiness: true });++ assert.ok(api.hls_url && new URL(api.hls_url).searchParams.get('sig'), 'the clip API shape still answers a signed hls_url');++ }++server.close();fs.rmSync(tmp, { recursive: true, force: true });console.log('✅ All clip timeline tests passed');@@@ -468,6 +468,35 @@ const { spawn, spawnSync } = require('child_process');assert.strictEqual((await get(`/o/${pub}/master.m3u8${signed.search}`)).status, 200, 'a public object needs no signature');assert.strictEqual((await call('GET', `/api/v2/live/objects/${pub}/download?format=json`)).body.hls_url, `${config.publicUrl}/o/${pub}/master.m3u8`);++ // ── The shared discovery (objects/hls.js) the v1 VOD/clip shapes read answers exactly as /download does ──++ const hls = require('../server/objects/hls');++ assert.deepStrictEqual(await hls.discovery(pub), { hls_url: `${config.publicUrl}/o/${pub}/master.m3u8` }, 'an open object with a timeline: the plain master URL');++ const privHls = new URL((await hls.discovery(priv)).hls_url);++ assert.strictEqual(privHls.pathname, `/o/${priv}/master.m3u8`, 'a private object: the signed master URL');++ assert.ok(privHls.searchParams.get('exp') && privHls.searchParams.get('sig'), 'with a playlist token');++ assert.strictEqual(await hls.discovery('med_NOPE'), null, 'a missing object answers nothing');++++ // ── A sandbox tenant's object: signed, never the plain URL — per-row and batched alike ──++ {++ await db.ensureProjectTenant('prj_hls_sandbox', 'sandbox', 1024 * 1024);++ const sb = await model.createObject({ app_id: 'prj_hls_sandbox-sandbox', visibility: 'public', lifecycle_status: 'ready', kind: 'file', mime_type: 'video/mp4', size_bytes: 1 });++ await timeline.replace(sb, 'source', [row(0, 0, 0, 500), row(1, 0, 3000, 30000)]);++ const plain = `${config.publicUrl}/o/${sb}/master.m3u8`;++ const signing = require('../server/objects/signing');++ const one = await hls.discovery(sb);++ const u = new URL(one.hls_url);++ assert.strictEqual(u.pathname, `/o/${sb}/master.m3u8`, 'the sandbox object\'s playlist URL names its master');++ assert.ok(u.searchParams.get('exp') && u.searchParams.get('sig'), 'a sandbox object is signed');++ assert.ok(signing.verifyPlaylist(sb, u.searchParams.get('exp'), u.searchParams.get('sig')), 'the signature verifies');++ assert.notStrictEqual(one.hls_url, plain, 'a sandbox object never gets the plain URL');++ assert.ok(one.hls_expires_at, 'and carries its expiry');++ const many = await hls.discoveryMany([{ object_id: sb }, { object_id: pub }]);++ assert.strictEqual(many.get(String(pub)).hls_url, `${config.publicUrl}/o/${pub}/master.m3u8`, 'the batch leaves a production object plain');++ const bu = new URL(many.get(String(sb)).hls_url);++ assert.ok(bu.searchParams.get('sig') && many.get(String(sb)).hls_url !== plain, 'and signs the sandbox object in the batch too');++ assert.ok(many.get(String(sb)).hls_expires_at, 'the batched sandbox answer carries its expiry');++ }++// ── MEDIA_HLS_ENABLED off: the routes 404, the job is refused, the download answer is as before ──config.hls.enabled = false;assert.strictEqual((await get(`/o/${pub}/master.m3u8`)).status, 404);
@@ -233,7 +233,7 @@ MEDIA_APP_KEYS="live:key1,games:key2"
@@ -233,7 +233,7 @@ MEDIA_APP_KEYS="live:key1,games:key2"
| POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow |
| POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow |
| POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) |
| POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) |
| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, webhook |
| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, webhook |
-| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness)) |
+| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness); `hls_url`, with `hls_expires_at` when signed: the object's master playlist with `MEDIA_HLS_ENABLED`, docs/media-fabric.md §3) |
| GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) |
| GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) |
| PUT | `/vods/:id` | `{ title?, description?, visibility? }` |
| PUT | `/vods/:id` | `{ title?, description?, visibility? }` |
| DELETE | `/vods/:id` | deletes local + B2 + R2 objects + row |
| DELETE | `/vods/:id` | deletes local + B2 + R2 objects + row |
@@ -271,7 +271,7 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
@@ -271,7 +271,7 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
| method | path | notes |
| method | path | notes |
|---|---|---|
|---|---|---|
| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
-| GET | `/clips/:id` | status: `processing | ready | failed` |
+| GET | `/clips/:id` | status: `processing | ready | failed`; with `MEDIA_HLS_ENABLED` also `hls_url` (and `hls_expires_at` when signed) — the clip's playlist over its own timeline or, virtual, its source's window (docs/media-fabric.md §4) |
| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) |
| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
@@ -463,7 +463,7 @@ Counters are per process (a restart forgets them). `test/actor-limits.test.js`.
@@ -463,7 +463,7 @@ Counters are per process (a restart forgets them). `test/actor-limits.test.js`.
| `GET /live/:sel/transcript.json` | **transcript + AI timeline API** — full audio-transcription log and AI overview timeline. `:sel` = slot id / slot slug → slot-scoped sessions; **`@username`** → user-scoped (all their slots, works while offline). Returns `{ live, current, sessions[] (each: ai_overview, transcript, duration…), streamer (overview + stream memories), user }`. `?limit=1..50` sessions (default 10), `?app=`. Cached **30s** (that's the rate limit), CORS-open. |
| `GET /live/:sel/transcript.json` | **transcript + AI timeline API** — full audio-transcription log and AI overview timeline. `:sel` = slot id / slot slug → slot-scoped sessions; **`@username`** → user-scoped (all their slots, works while offline). Returns `{ live, current, sessions[] (each: ai_overview, transcript, duration…), streamer (overview + stream memories), user }`. `?limit=1..50` sessions (default 10), `?app=`. Cached **30s** (that's the rate limit), CORS-open. |
| `GET /live/:sel/chat-insight.json` | **chat insight API** — a user's chat-related AI insight/timeline (`:sel` = `@username` or numeric user id): today-vs-alltime chat overviews, condensed memory, event timeline, plus their streamer overview + stream memories when they stream. Proxied from the app's public chat-AI API over loopback; cached **30s**, CORS-open. |
| `GET /live/:sel/chat-insight.json` | **chat insight API** — a user's chat-related AI insight/timeline (`:sel` = `@username` or numeric user id): today-vs-alltime chat overviews, condensed memory, event timeline, plus their streamer overview + stream memories when they stream. Proxied from the app's public chat-AI API over loopback; cached **30s**, CORS-open. |
| `GET /v/:id/transcript.json` | **VOD transcript API** — transcript + AI overview for one existing VOD id (`{ vod_id, title, duration_seconds, ai_overview, transcript, ai_analyzed_at }`). Private VODs → 404. Cached 30s, CORS-open. |
| `GET /v/:id/transcript.json` | **VOD transcript API** — transcript + AI overview for one existing VOD id (`{ vod_id, title, duration_seconds, ai_overview, transcript, ai_analyzed_at }`). Private VODs → 404. Cached 30s, CORS-open. |
-| `GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8`, `/o/:id/source/{init.mp4,NNNNNN.m4s}` | segment-native video (F3.1), only with `MEDIA_HLS_ENABLED=1` (off by default; off, these 404): HLS playlists written from the object's `media_timeline` rows (VOD, `#EXT-X-ENDLIST`) and its CMAF segments (Range; a segment packed by `object.pack` (F3.3) is a byte range of its chunk). Same check as `GET /o/:id` (and, like it, no `Access-Control-Allow-Origin` on a private or sandbox answer): private and sandbox objects only with the object's `?exp&sig` (`/api/v2/:app/objects/:id/download?format=json` → `hls_url`, a playlist token valid `MEDIA_HLS_PLAYLIST_TTL_S` (6 h; 60 s to 12 h) that no other route accepts; a download signature works too), carried onto every URI; deleted 410. The `object.cmaf` job makes the timeline; a recording's finalize queues it, and `download?format=json` queues it for an object that has none yet. `noindex` |
+| `GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8`, `/o/:id/source/{init.mp4,NNNNNN.m4s}` | segment-native video (F3.1), only with `MEDIA_HLS_ENABLED=1` (off by default; off, these 404): HLS playlists written from the object's `media_timeline` rows (VOD, `#EXT-X-ENDLIST`) and its CMAF segments (Range; a segment packed by `object.pack` (F3.3) is a byte range of its chunk). Same check as `GET /o/:id` (and, like it, no `Access-Control-Allow-Origin` on a private or sandbox answer): private and sandbox objects only with the object's `?exp&sig` (`/api/v2/:app/objects/:id/download?format=json` → `hls_url`, a playlist token valid `MEDIA_HLS_PLAYLIST_TTL_S` (6 h; 60 s to 12 h) that no other route accepts; a download signature works too), carried onto every URI; deleted 410. The `object.cmaf` job makes the timeline; a recording's finalize queues it, and `download?format=json` queues it for an object that has none yet; the v1 VOD and clip answers carry the same `hls_url` (and `hls_expires_at` when signed). `noindex` |
| `GET /robots.txt` | crawler policy (`openvibe-shared/seo.robotsTxt`: AI and search crawlers named, `/api/`, `/auth/`, `/internal/`, `/o/`, `/metrics`, `/live/`, `/me` disallowed) and the sitemap |
| `GET /robots.txt` | crawler policy (`openvibe-shared/seo.robotsTxt`: AI and search crawlers named, `/api/`, `/auth/`, `/internal/`, `/o/`, `/metrics`, `/live/`, `/me` disallowed) and the sitemap |
| `GET /sitemap.xml` | the media index and the watch pages Media is the canonical home of: public, playable VODs and clips of apps other than Live (Live lists its own `/vod` and `/clip` pages), never AI clips, private, unlisted, sandbox, recording or failed items; thumbnails as image entries. Every listed page renders `index, follow` |
| `GET /sitemap.xml` | the media index and the watch pages Media is the canonical home of: public, playable VODs and clips of apps other than Live (Live lists its own `/vod` and `/clip` pages), never AI clips, private, unlisted, sandbox, recording or failed items; thumbnails as image entries. Every listed page renders `index, follow` |
| `GET /llms.txt` | a map of the site for language-model crawlers |
| `GET /llms.txt` | a map of the site for language-model crawlers |
@@ -40,7 +40,8 @@
@@ -40,7 +40,8 @@
"every storage class in the native-object sweep: kind and mime type classify video/image/download, an app may declare game-asset/attachment/backup (or any of the six) in the object's metadata (metadata.class, or media_class/placement_class), and each class spends its own per-sweep budget, hysteresis band and R2 storage ceiling",
"every storage class in the native-object sweep: kind and mime type classify video/image/download, an app may declare game-asset/attachment/backup (or any of the six) in the object's metadata (metadata.class, or media_class/placement_class), and each class spends its own per-sweep budget, hysteresis band and R2 storage ceiling",
"segment-native video (F3.1, behind MEDIA_HLS_ENABLED): the media_timeline index, the object.cmaf job (CMAF/fMP4 segments of a finished video on its own keyframes, sha256 and durability per row, idempotent) and HLS playlists written from the timeline at /o/:id/master.m3u8 under the object's access checks",
"segment-native video (F3.1, behind MEDIA_HLS_ENABLED): the media_timeline index, the object.cmaf job (CMAF/fMP4 segments of a finished video on its own keyframes, sha256 and durability per row, idempotent) and HLS playlists written from the timeline at /o/:id/master.m3u8 under the object's access checks",
"packing after finalize (F3.3, behind MEDIA_HLS_ENABLED): the object.pack job concatenates durable segments into ~60 s chunk objects verified by sha256, each media_timeline row naming its byte range; the segment route serves a packed segment as a ranged read of its chunk; the per-segment keys are deleted once the chunk is durable; segment providers come from the placement router; a finished object.cmaf queues object.pack by itself (dedupeActive: a rerun joins an active pack, a later cut queues a new one)",
"packing after finalize (F3.3, behind MEDIA_HLS_ENABLED): the object.pack job concatenates durable segments into ~60 s chunk objects verified by sha256, each media_timeline row naming its byte range; the segment route serves a packed segment as a ranged read of its chunk; the per-segment keys are deleted once the chunk is durable; segment providers come from the placement router; a finished object.cmaf queues object.pack by itself (dedupeActive: a rerun joins an active pack, a later cut queues a new one)",
- "timeline sprites (F3.1, behind MEDIA_HLS_ENABLED): object.sprite cuts its sheet from the object's media_timeline rows — each sample time picks the row covering it and one frame is decoded from that row's bytes (this node's segment, or a packed row's byte range in its chunk), checked against the row's sha256 — with the same layout and metadata as the seek path, so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame"
+ "timeline sprites (F3.1, behind MEDIA_HLS_ENABLED): object.sprite cuts its sheet from the object's media_timeline rows — each sample time picks the row covering it and one frame is decoded from that row's bytes (this node's segment, or a packed row's byte range in its chunk), checked against the row's sha256 — with the same layout and metadata as the seek path, so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame",
+ "HLS discovery on the v1 shapes (F3.c, behind MEDIA_HLS_ENABLED): the VOD and clip answers (vodPublic/clipPublic — get, list, create, update) carry hls_url (and hls_expires_at when the object is private or sandbox), the same field and access rules as GET /o/:id/download?format=json, read through one shared discovery (server/objects/hls.js); an object without a timeline gets object.cmaf queued and the field appears once it ran, and a virtual clip plays its source's window without being cut a timeline of its own"
],
],
"updated": "2026-10-04"
"updated": "2026-10-04"
}
}
@@ -159,7 +159,12 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
@@ -159,7 +159,12 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
(`hls`), valid for `MEDIA_HLS_PLAYLIST_TTL_S` (6 h by default, 60 s to 12 h) so a viewer plays a long VOD on one page
(`hls`), valid for `MEDIA_HLS_PLAYLIST_TTL_S` (6 h by default, 60 s to 12 h) so a viewer plays a long VOD on one page
load, carried onto every playlist and segment URI, and accepted only by the HLS routes (never by `GET /o/:id` or any
load, carried onto every playlist and segment URI, and accepted only by the HLS routes (never by `GET /o/:id` or any
other route); the HLS routes still accept the object's download signature, so earlier URLs keep working. A public
other route); the HLS routes still accept the object's download signature, so earlier URLs keep working. A public
- object's `hls_url` stays unsigned. `GET …/download?format=mp4` answers the object's `remux` variant (`object.remux`,
+ object's `hls_url` stays unsigned. The v1 VOD and clip shapes carry the same field (`GET /api/v1/:app/vods/:id` and
+ `/clips/:id`, their lists, creates and updates through `vodPublic`/`clipPublic`), from the shared discovery
+ (`server/objects/hls.js`; the same presence and access rules as `…/download?format=json`), so an app renders a player
+ from the record it already has: a VOD with a timeline, a materialized clip with one, or a virtual clip playing its
+ source's window; one without gets `object.cmaf` queued and the field appears once it ran. `GET …/download?format=mp4`
+ answers the object's `remux` variant (`object.remux`,
`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux
`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux
(`dedupeActive`) and answers `202 { job_id }` while it is missing.
(`dedupeActive`) and answers `202 { job_id }` while it is missing.
- **F3.5, shipped: virtual and materialized clips over the timeline (§4).**
- **F3.5, shipped: virtual and materialized clips over the timeline (§4).**
@@ -194,9 +199,11 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
@@ -194,9 +199,11 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
timeline of its own uses it. A virtual clip's source must be the same tenant's ready object; a deleted source stops
timeline of its own uses it. A virtual clip's source must be the same tenant's ready object; a deleted source stops
it. `…/download?format=json` names its `hls_url` and never queues `object.cmaf` for it; `/c/:id` (public, the owner,
it. `…/download?format=json` names its `hls_url` and never queues `object.cmaf` for it; `/c/:id` (public, the owner,
or a `getc` URL from `GET /clips/:id/signed-url`) answers a virtual clip with a 302 to its master playlist (signed
or a `getc` URL from `GET /clips/:id/signed-url`) answers a virtual clip with a 302 to its master playlist (signed
- with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); `playback_url` is
- unchanged. Its object's metadata carries `virtual: true`: readiness counts it `bytes_verified`/`playable` without a
- location of its own, and the verify job skips it. **Materialized** = today's `clip.cut` full
+ with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); the clip answers
+ (`GET /api/v1/:app/clips/:id` and the list/create/update shapes) carry its `hls_url` (the clip's own token when
+ closed), and `playback_url` is unchanged. Its object's metadata carries `virtual: true`: readiness counts it
+ `bytes_verified`/`playable` without a location of its own, and the verify job skips it. **Materialized** = today's
+ `clip.cut` full
re-encode into the clip's own file: with `materialize: true` on `POST /clips` or `POST /clips/:id/recut`, or when
re-encode into the clip's own file: with `materialize: true` on `POST /clips` or `POST /clips/:id/recut`, or when
the source has no timeline (or the flag is off). A virtual clip keeps playing while it is materialized and stays
the source has no timeline (or the flag is off). A virtual clip keeps playing while it is materialized and stays
virtual if the cut fails; a clip with a file is never turned virtual (its recut stays a cut). The retry sweeper
virtual if the cut fails; a clip with a file is never turned virtual (its recut stays a cut). The retry sweeper
new file mode 100644
new file mode 100644
@@ -0,0 +1,190 @@
@@ -0,0 +1,190 @@
+/**
+ * OpenVibe.Media — HLS discovery for an object (docs/media-fabric.md §3, F3.c)
+ *
+ * One place answers "does this object have a serviceable HLS timeline, and what is its playlist URL?": the v2
+ * /o/:id/download?format=json route (objects/routes.js) and the v1 VOD and clip shapes (vod/finalize.js vodPublic,
+ * vod/clips-routes.js clipPublic) all use it, so the field and its access rules cannot drift between them.
+ *
+ * A playlist exists when MEDIA_HLS_ENABLED is on and the object's source timeline has a media row (seq > 0) — its own
+ * rows, or for a virtual clip (F3.5) its source's window rows. With none, object.cmaf is queued (idempotent) and no URL
+ * is answered, unless the object is a virtual clip (its source's window plays it; it is never cut a timeline of its own).
+ * A single-item read takes that path (deduped in-process); a list read uses discoveryMany, which answers a whole page in
+ * a constant number of queries and never queues anything.
+ *
+ * The URL is built exactly as the v2 route does: an open object (not private, not in a sandbox tenant) gets the plain
+ * /o/<id>/master.m3u8; a closed one gets a signed playlist (purpose 'hls', signing.signedPlaylistUrl) and its expiry.
+ */
+'use strict';
+
+const config = require('../config');
+const db = require('../db/database');
+const model = require('./model');
+const signing = require('./signing');
+
+/**
+ * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata
+ * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.
+ */
+async function clipWindow(obj) {
+ if (!obj || obj.kind !== 'clip') return null;
+ const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);
+ if (!rel) return null;
+ const src = await model.getObject(rel.to_object_id);
+ if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;
+ const md = model.parseJson(rel.metadata, {});
+ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);
+ const endMs = Math.round(Number(md.end_time) * 1000) || 0;
+ return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;
+}
+
+/**
+ * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a
+ * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's
+ * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.
+ */
+async function timelineOf(obj) {
+ const timeline = require('./timeline');
+ const own = await timeline.list(obj.id, timeline.SOURCE);
+ if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };
+ const window = await clipWindow(obj);
+ if (!window) return { rows: [], window: null };
+ return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };
+}
+
+/** Whether an object is served openly: not private and not in a developer-project sandbox tenant (GET /o/:id's rule). */
+async function isOpen(obj) {
+ return obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id);
+}
+
+// The lazy object.cmaf one read can queue (discoveryForObject) is deduped per process: objectId → last queued ms.
+// A bounded Map (QUEUE_MAX entries, expired ones swept first) keeps it from growing without limit; the TTL is a
+// window, not correctness — timeline-queue.queueCmaf is itself idempotent, this only spares a hot read its queries.
+const QUEUE_TTL_MS = 10 * 60 * 1000;
+const QUEUE_MAX = 1000;
+const _queuedCmaf = new Map();
+
+/** Queue object.cmaf for an object at most once per QUEUE_TTL_MS in this process. Never throws. */
+async function queueCmafOnce(appId, objectId) {
+ const now = Date.now();
+ const last = _queuedCmaf.get(objectId);
+ if (last != null && now - last < QUEUE_TTL_MS) return;
+ if (_queuedCmaf.size >= QUEUE_MAX) {
+ for (const [id, at] of _queuedCmaf) if (now - at >= QUEUE_TTL_MS) _queuedCmaf.delete(id);
+ while (_queuedCmaf.size >= QUEUE_MAX) _queuedCmaf.delete(_queuedCmaf.keys().next().value);
+ }
+ _queuedCmaf.set(objectId, now);
+ try { await require('./timeline-queue').queueCmaf(appId, objectId); } catch (err) { console.warn(`[HLS] Timeline queue failed for ${objectId}:`, err.message); }
+}
+
+/** What an object with a serviceable timeline is discovered by: { hls_url, hls_expires_at? } (open stays unsigned). */
+function playlistFor(obj, open) {
+ if (open) return { hls_url: `${config.publicUrl}/o/${obj.id}/master.m3u8` };
+ const signed = signing.signedPlaylistUrl(obj.id);
+ return { hls_url: signed.url, hls_expires_at: signed.expires_at };
+}
+
+/**
+ * The HLS answer for an already-loaded ready object, or null: { hls_url, hls_expires_at? } once the timeline exists;
+ * otherwise object.cmaf is queued (idempotent) for an object that could still get one and nothing is answered. Never
+ * throws — a lookup or queue failure is logged and the object simply has no hls_url this time.
+ */
+async function discoveryForObject(obj) {
+ if (!config.hls.enabled || !obj || obj.lifecycle_status !== 'ready') return null;
+ try {
+ const tl = await timelineOf(obj);
+ if (!tl.rows.some((r) => Number(r.seq) > 0)) {
+ if (!tl.window) await queueCmafOnce(obj.app_id, obj.id);
+ return null;
+ }
+ return playlistFor(obj, await isOpen(obj));
+ } catch (err) {
+ console.warn(`[HLS] Discovery failed for ${obj.id}:`, err.message);
+ return null;
+ }
+}
+
+/** discoveryForObject for a v1 row's object_id (null/absent id answers nothing). */
+async function discovery(objectId) {
+ if (!config.hls.enabled || !objectId) return null;
+ try {
+ return await discoveryForObject(await model.getObject(String(objectId)));
+ } catch (err) {
+ console.warn(`[HLS] Discovery failed for ${objectId}:`, err.message);
+ return null;
+ }
+}
+
+/**
+ * The HLS answers for a whole page of v1 rows (the VOD/clip lists), in a constant number of queries: the objects,
+ * which of them have a timeline, their sandbox tenants, and — only for clip objects with no timeline of their own —
+ * the source window they play over (F3.5). discoveryForObject's per-row path would run all of that once per row,
+ * and queue object.cmaf for each row without a timeline; a list read must do neither.
+ *
+ * Returns a Map(object_id → { hls_url, hls_expires_at? }). A row with no serviceable timeline (and any lookup that
+ * fails) simply has no entry. Never queues object.cmaf. Never throws.
+ */
+async function discoveryMany(rows) {
+ const out = new Map();
+ if (!config.hls.enabled || !Array.isArray(rows) || !rows.length) return out;
+ try {
+ const objectIds = [...new Set(rows.map((r) => r && r.object_id).filter(Boolean).map(String))];
+ if (!objectIds.length) return out;
+ const timeline = require('./timeline');
+
+ const ready = (await db.all('SELECT * FROM media_objects WHERE id = ANY(?)', [objectIds]))
+ .filter((o) => o.lifecycle_status === 'ready');
+ if (!ready.length) return out;
+ const readyIds = ready.map((o) => String(o.id));
+ const withTimeline = new Set((await db.all(
+ 'SELECT DISTINCT object_id FROM media_timeline WHERE rendition = ? AND seq > 0 AND object_id = ANY(?)',
+ [timeline.SOURCE, readyIds])).map((r) => String(r.object_id)));
+
+ // A clip object without a timeline of its own is a virtual clip: its playlist is its source's window.
+ const virtual = new Set();
+ const needWindow = ready.filter((o) => o.kind === 'clip' && !withTimeline.has(String(o.id)));
+ if (needWindow.length) {
+ const rels = await db.all("SELECT from_object_id, to_object_id, metadata FROM media_relationships WHERE relation = 'clip_of' AND from_object_id = ANY(?) ORDER BY id",
+ [needWindow.map((o) => String(o.id))]);
+ const relOf = new Map();
+ for (const r of rels) if (!relOf.has(String(r.from_object_id))) relOf.set(String(r.from_object_id), r);
+ const sourceIds = [...new Set(rels.map((r) => String(r.to_object_id)))];
+ const sourceOf = new Map((sourceIds.length ? await db.all('SELECT * FROM media_objects WHERE id = ANY(?)', [sourceIds]) : [])
+ .map((s) => [String(s.id), s]));
+ const sourceRows = new Map();
+ if (sourceIds.length) {
+ for (const r of await db.all('SELECT * FROM media_timeline WHERE rendition = ? AND object_id = ANY(?) ORDER BY object_id, seq', [timeline.SOURCE, sourceIds])) {
+ const k = String(r.object_id);
+ if (!sourceRows.has(k)) sourceRows.set(k, []);
+ sourceRows.get(k).push(r);
+ }
+ }
+ for (const clip of needWindow) {
+ const rel = relOf.get(String(clip.id));
+ const src = rel && sourceOf.get(String(rel.to_object_id));
+ if (!src || src.app_id !== clip.app_id || src.lifecycle_status !== 'ready') continue;
+ const md = model.parseJson(rel.metadata, {});
+ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);
+ const endMs = Math.round(Number(md.end_time) * 1000) || 0;
+ if (!(endMs > startMs)) continue;
+ const clipped = timeline.clipRows(sourceRows.get(String(src.id)) || [], startMs, endMs);
+ if (clipped.some((r) => Number(r.seq) > 0)) virtual.add(String(clip.id));
+ }
+ }
+
+ // isSandboxTenant, resolved once per distinct app of the page instead of once per row.
+ const appIds = [...new Set(ready.map((o) => String(o.app_id)))];
+ const sandbox = new Set((await db.all("SELECT app_id FROM apps WHERE env = 'sandbox' AND app_id = ANY(?)", [appIds]))
+ .map((r) => String(r.app_id)));
+
+ for (const obj of ready) {
+ const id = String(obj.id);
+ if (!withTimeline.has(id) && !virtual.has(id)) continue;
+ out.set(id, playlistFor(obj, obj.visibility !== 'private' && !sandbox.has(String(obj.app_id))));
+ }
+ } catch (err) {
+ console.warn('[HLS] Batch discovery failed:', err.message);
+ }
+ return out;
+}
+
+module.exports = { timelineOf, discoveryForObject, discovery, discoveryMany };
@@ -51,6 +51,7 @@ const config = require('../config');
@@ -51,6 +51,7 @@ const config = require('../config');
const model = require('./model');
const model = require('./model');
const invariant = require('./invariant');
const invariant = require('./invariant');
const signing = require('./signing');
const signing = require('./signing');
+const hls = require('./hls');
const ctypes = require('./content-type');
const ctypes = require('./content-type');
const multipart = require('./multipart');
const multipart = require('./multipart');
const namespaces = require('./namespaces');
const namespaces = require('./namespaces');
@@ -673,19 +674,15 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r
@@ -673,19 +674,15 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r
// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private
// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private
// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.
// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.
// A clip without its own timeline plays its window of the source's (F3.5) and is never cut a CMAF timeline here.
// A clip without its own timeline plays its window of the source's (F3.5) and is never cut a CMAF timeline here.
- const tl = json && config.hls.enabled ? await timelineOf(obj) : null;
- const hls = !!tl && tl.rows.some((r) => Number(r.seq) > 0);
- if (tl && !hls && !tl.window) {
- try { await require('./timeline-queue').queueCmaf(obj.app_id, obj.id); } catch (err) { console.warn(`[Objects] Timeline queue failed for ${obj.id}:`, err.message); }
- }
+ // The same answer shapes the v1 VOD and clip (objects/hls.js discovery; vodPublic/clipPublic).
+ const hlsInfo = json ? await hls.discoveryForObject(obj) : null;
if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {
if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {
const url = model.legacyPublicUrl(obj) || `${config.publicUrl}/o/${obj.id}`;
const url = model.legacyPublicUrl(obj) || `${config.publicUrl}/o/${obj.id}`;
- return json ? res.json({ url, expires_at: null, public: true, ...(hls && { hls_url: `${config.publicUrl}/o/${obj.id}/master.m3u8` }) }) : res.redirect(302, url);
+ return json ? res.json({ url, expires_at: null, public: true, ...(hlsInfo || {}) }) : res.redirect(302, url);
}
}
const signed = signing.signedDownloadUrl(obj.id, req.query.ttl);
const signed = signing.signedDownloadUrl(obj.id, req.query.ttl);
if (['1', 'true'].includes(String(req.query.redirect || ''))) return res.redirect(302, signed.url);
if (['1', 'true'].includes(String(req.query.redirect || ''))) return res.redirect(302, signed.url);
- const playlist = hls && signing.signedPlaylistUrl(obj.id);
- res.json({ url: signed.url, expires_at: signed.expires_at, public: false, ...(playlist && { hls_url: playlist.url, hls_expires_at: playlist.expires_at }) });
+ res.json({ url: signed.url, expires_at: signed.expires_at, public: false, ...(hlsInfo || {}) });
});
});
/**
/**
@@ -783,36 +780,8 @@ const publicRouter = express.Router();
@@ -783,36 +780,8 @@ const publicRouter = express.Router();
const HLS_TYPE = 'application/vnd.apple.mpegurl';
const HLS_TYPE = 'application/vnd.apple.mpegurl';
const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;
const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
-
-/**
- * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata
- * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.
- */
-async function clipWindow(obj) {
- if (!obj || obj.kind !== 'clip') return null;
- const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);
- if (!rel) return null;
- const src = await model.getObject(rel.to_object_id);
- if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;
- const md = model.parseJson(rel.metadata, {});
- const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);
- const endMs = Math.round(Number(md.end_time) * 1000) || 0;
- return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;
-}
-
-/**
- * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a
- * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's
- * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.
- */
-async function timelineOf(obj) {
- const timeline = require('./timeline');
- const own = await timeline.list(obj.id, timeline.SOURCE);
- if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };
- const window = await clipWindow(obj);
- if (!window) return { rows: [], window: null };
- return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };
-}
+// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in
+// objects/hls.js, shared with the v1 VOD/clip discovery.
/** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */
/** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */
async function hlsObject(req, res, next) {
async function hlsObject(req, res, next) {
@@ -846,14 +815,14 @@ const hlsRoute = (fn) => async (req, res, next) => {
@@ -846,14 +815,14 @@ const hlsRoute = (fn) => async (req, res, next) => {
publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {
publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {
const timeline = require('./timeline');
const timeline = require('./timeline');
- const { rows } = await timelineOf(obj);
+ const { rows } = await hls.timelineOf(obj);
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));
res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));
}));
}));
publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {
publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {
const timeline = require('./timeline');
const timeline = require('./timeline');
- const { rows } = await timelineOf(obj);
+ const { rows } = await hls.timelineOf(obj);
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));
res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));
}));
}));
@@ -908,7 +877,7 @@ publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed })
@@ -908,7 +877,7 @@ publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed })
let row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;
let row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;
// A virtual clip serves only the source segments inside its window (timelineOf): any other name is a 404, whatever
// A virtual clip serves only the source segments inside its window (timelineOf): any other name is a 404, whatever
// the signature, so a clip's token never reaches the rest of its source.
// the signature, so a clip's token never reaches the rest of its source.
- if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await timelineOf(obj)).rows.find((r) => r.name === name) || null;
+ if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await hls.timelineOf(obj)).rows.find((r) => r.name === name) || null;
if (!row) return res.status(404).json({ error: 'Not found' });
if (!row) return res.status(404).json({ error: 'Not found' });
const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';
const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';
const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };
const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };
@@ -40,7 +40,7 @@ let _timer = null, _busy = false;
@@ -40,7 +40,7 @@ let _timer = null, _busy = false;
function freeBytes() {
function freeBytes() {
try { const st = fs.statfsSync(path.resolve(config.vod.path)); return Number(st.bavail) * Number(st.bsize); } catch { return 0; }
try { const st = fs.statfsSync(path.resolve(config.vod.path)); return Number(st.bavail) * Number(st.bsize); } catch { return 0; }
}
}
-async function _clipPublic(clip) { try { return await require('./clips-routes').clipPublic(clip); } catch { return clip; } }
+async function _clipPublic(clip) { try { return await require('./clips-routes').clipPublic(clip, { hls: false }); } catch { return clip; } }
const RECUT_CAP_MS = 30 * 60 * 1000; // one clip can never hold the queue longer than this
const RECUT_CAP_MS = 30 * 60 * 1000; // one clip can never hold the queue longer than this
@@ -54,8 +54,10 @@ const clipUpload = multer({
@@ -54,8 +54,10 @@ const clipUpload = multer({
});
});
// { readiness: true } (API responses) adds the object's readiness levels (server/objects/readiness.js);
// { readiness: true } (API responses) adds the object's readiness levels (server/objects/readiness.js);
-// the clip.ready / clip.failed payloads leave them out, as vodPublic's do.
-async function clipPublic(clip, { readiness = false } = {}) {
+// the clip.ready / clip.failed payloads leave them out, as vodPublic's do. `hls` selects the playlist
+// fields: undefined = discover them for this row (a single-item read), false = omit them (an event payload
+// must never carry a signed URL into the outbox), an object = the answer a list route already batched.
+async function clipPublic(clip, { readiness = false, hls } = {}) {
if (!clip) return null;
if (!clip) return null;
const out = {
const out = {
unique_views: clip.unique_views || 0,
unique_views: clip.unique_views || 0,
@@ -91,6 +93,13 @@ async function clipPublic(clip, { readiness = false } = {}) {
@@ -91,6 +93,13 @@ async function clipPublic(clip, { readiness = false } = {}) {
view_count: clip.view_count || 0,
view_count: clip.view_count || 0,
created_at: clip.created_at,
created_at: clip.created_at,
};
};
+ // The HLS master playlist of the clip's object, when it has one (objects/hls.js): the same field and access rules as
+ // GET /o/:id/download?format=json — unsigned for an open object, a signed playlist (+ hls_expires_at) for a private
+ // or sandbox one. Absent with MEDIA_HLS_ENABLED off, before the object is ready, and until its timeline (or, for a
+ // virtual clip, its source's window) has a segment; object.cmaf is queued for a materialized clip that has none.
+ // Never throws.
+ if (hls === undefined) Object.assign(out, await require('../objects/hls').discovery(clip.object_id) || {});
+ else if (hls) Object.assign(out, hls);
if (readiness) out.readiness = await require('../objects/readiness').forRow(clip);
if (readiness) out.readiness = await require('../objects/readiness').forRow(clip);
return out;
return out;
}
}
@@ -329,7 +338,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)
@@ -329,7 +338,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)
};
};
const clips = await db.listClips(req.appId, filters);
const clips = await db.listClips(req.appId, filters);
const total = await db.countClips(req.appId, filters);
const total = await db.countClips(req.appId, filters);
- res.json({ clips: (await Promise.all(clips.map(async c => await clipPublic(c, { readiness: true })))), total, limit, offset, hasMore: offset + clips.length < total });
+ const hls = await require('../objects/hls').discoveryMany(clips);
+ res.json({ clips: await Promise.all(clips.map(async c => await clipPublic(c, { readiness: true, hls: hls.get(String(c.object_id)) || null }))), total, limit, offset, hasMore: offset + clips.length < total });
} catch (err) {
} catch (err) {
console.error('[Clips] List error:', err.message);
console.error('[Clips] List error:', err.message);
res.status(500).json({ error: 'Failed to list clips' });
res.status(500).json({ error: 'Failed to list clips' });
@@ -37,8 +37,10 @@ function isFinalizing(vodId) {
@@ -37,8 +37,10 @@ function isFinalizing(vodId) {
// Public shape of a vod row for API responses + webhooks. { readiness: true } (API responses) adds the
// Public shape of a vod row for API responses + webhooks. { readiness: true } (API responses) adds the
// object's readiness levels (server/objects/readiness.js); webhook payloads leave them out (their shape
// object's readiness levels (server/objects/readiness.js); webhook payloads leave them out (their shape
-// predates the object model and stays as it is).
-async function vodPublic(vod, { readiness = false } = {}) {
+// predates the object model and stays as it is). `hls` selects the playlist fields: undefined = discover
+// them for this row (a single-item read), false = omit them (an event payload must never carry a signed
+// URL into the outbox), an object = the answer the list route already batched (objects/hls.discoveryMany).
+async function vodPublic(vod, { readiness = false, hls } = {}) {
if (!vod) return null;
if (!vod) return null;
const out = {
const out = {
unique_views: vod.unique_views || 0,
unique_views: vod.unique_views || 0,
@@ -78,6 +80,12 @@ async function vodPublic(vod, { readiness = false } = {}) {
@@ -78,6 +80,12 @@ async function vodPublic(vod, { readiness = false } = {}) {
created_at: vod.created_at,
created_at: vod.created_at,
meta: (() => { try { return JSON.parse(vod.meta_json || '{}'); } catch { return {}; } })(),
meta: (() => { try { return JSON.parse(vod.meta_json || '{}'); } catch { return {}; } })(),
};
};
+ // The HLS master playlist of the VOD's object, when it has one (objects/hls.js): the same field and access rules as
+ // GET /o/:id/download?format=json — unsigned for an open object, a signed playlist (+ hls_expires_at) for a private
+ // or sandbox one. Absent with MEDIA_HLS_ENABLED off, before the object is ready, and until its timeline exists
+ // (object.cmaf is queued for it, idempotently). Never throws.
+ if (hls === undefined) Object.assign(out, await require('../objects/hls').discovery(vod.object_id) || {});
+ else if (hls) Object.assign(out, hls);
if (readiness) out.readiness = await require('../objects/readiness').forRow(vod);
if (readiness) out.readiness = await require('../objects/readiness').forRow(vod);
return out;
return out;
}
}
@@ -118,7 +126,7 @@ function rebuildWebmFromMaster(masterPath, webmPath) {
@@ -118,7 +126,7 @@ function rebuildWebmFromMaster(masterPath, webmPath) {
*/
*/
async function _commitVodOutcome(vod, event, change, row = async () => await db.getVodById(vod.id)) {
async function _commitVodOutcome(vod, event, change, row = async () => await db.getVodById(vod.id)) {
try {
try {
- return (await announce(vod.app_id, event, { change: async () => await objects().withObject('vod', vod.id, change), payload: async () => await vodPublic(row()) })).data;
+ return (await announce(vod.app_id, event, { change: async () => await objects().withObject('vod', vod.id, change), payload: async () => await vodPublic(row(), { hls: false }) })).data;
} catch (err) {
} catch (err) {
console.error(`[VOD] vod ${vod.id}: ${event} not committed:`, err.message);
console.error(`[VOD] vod ${vod.id}: ${event} not committed:`, err.message);
return null;
return null;
@@ -383,7 +391,7 @@ async function _doFinalize(vodId, opts) {
@@ -383,7 +391,7 @@ async function _doFinalize(vodId, opts) {
await announce(vod.app_id, 'vod.ready', {
await announce(vod.app_id, 'vod.ready', {
change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,
change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,
[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),
[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),
- payload: async () => await vodPublic(await db.getVodById(vodId)),
+ payload: async () => await vodPublic(await db.getVodById(vodId), { hls: false }),
});
});
const ready = await db.getVodById(vodId);
const ready = await db.getVodById(vodId);
// Segment-native video (MEDIA_HLS_ENABLED): the finished recording's timeline is cut by object.cmaf. A queue error
// Segment-native video (MEDIA_HLS_ENABLED): the finished recording's timeline is cut by object.cmaf. A queue error
@@ -292,7 +292,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)
@@ -292,7 +292,8 @@ router.get('/', tenantAuth({ verb: 'list', allowUser: true }), async (req, res)
};
};
const vods = await db.listVods(req.appId, filters);
const vods = await db.listVods(req.appId, filters);
const total = await db.countVods(req.appId, filters);
const total = await db.countVods(req.appId, filters);
- res.json({ vods: (await Promise.all(vods.map(async v => await vodPublic(v, { readiness: true })))), total, limit, offset, hasMore: offset + vods.length < total });
+ const hls = await require('../objects/hls').discoveryMany(vods);
+ res.json({ vods: (await Promise.all(vods.map(async v => await vodPublic(v, { readiness: true, hls: hls.get(String(v.object_id)) || null })))), total, limit, offset, hasMore: offset + vods.length < total });
} catch (err) {
} catch (err) {
console.error('[VOD] List error:', err.message);
console.error('[VOD] List error:', err.message);
res.status(500).json({ error: 'Failed to list VODs' });
res.status(500).json({ error: 'Failed to list VODs' });
@@ -96,6 +96,10 @@ const express = require('express');
@@ -96,6 +96,10 @@ const express = require('express');
assert.strictEqual((await call('POST', `/api/v1/live/vods/${vodId}/chunks`, f)).body.status, 'created');
assert.strictEqual((await call('POST', `/api/v1/live/vods/${vodId}/chunks`, f)).body.status, 'created');
const vodObj = (await finalizeVod(vodId)).object_id;
const vodObj = (await finalizeVod(vodId)).object_id;
const vodRow = await model.getObject(vodObj);
const vodRow = await model.getObject(vodObj);
+ // Before the timeline exists: no hls_url, and the VOD answer queues object.cmaf (idempotent with the finalize's).
+ r = await call('GET', `/api/v1/live/vods/${vodId}`);
+ assert.ok(!('hls_url' in r.body), 'no hls_url before the timeline exists');
+ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${vodObj}'`)).length, 1, 'the VOD answer queues object.cmaf once');
const bytes = {};
const bytes = {};
const rows = src.map((s) => {
const rows = src.map((s) => {
const p = timeline.localPathFor(vodRow, 'source', s.name, 'v1');
const p = timeline.localPathFor(vodRow, 'source', s.name, 'v1');
@@ -109,6 +113,103 @@ const express = require('express');
@@ -109,6 +113,103 @@ const express = require('express');
});
});
await timeline.replace(vodObj, 'source', rows);
await timeline.replace(vodObj, 'source', rows);
+ // ── The v1 VOD names its HLS playlist: a playlist token (private) that plays through /o/<object>/master.m3u8 ──
+ r = await call('GET', `/api/v1/live/vods/${vodId}`);
+ assert.strictEqual(r.status, 200, JSON.stringify(r.body));
+ const vhu = new URL(r.body.hls_url);
+ assert.strictEqual(vhu.pathname, `/o/${vodObj}/master.m3u8`, 'hls_url names the object\'s master playlist');
+ assert.ok(signing.verifyPlaylist(vodObj, vhu.searchParams.get('exp'), vhu.searchParams.get('sig')), 'a private VOD\'s hls_url carries a playlist token');
+ assert.ok(r.body.hls_expires_at, 'and its expiry');
+ assert.strictEqual((await get(r.body.hls_url)).status, 200, 'the VOD hls_url serves 200');
+ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${vodObj}'`)).length, 1, 'no extra object.cmaf once the timeline exists');
+ // An open VOD's hls_url stays unsigned; put it back for the clip paths below.
+ assert.strictEqual((await call('PUT', `/api/v1/live/vods/${vodId}`, { visibility: 'public' })).status, 200);
+ const vpub = await call('GET', `/api/v1/live/vods/${vodId}`);
+ assert.strictEqual(vpub.body.hls_url, `${config.publicUrl}/o/${vodObj}/master.m3u8`, 'a public VOD\'s hls_url is the plain master URL');
+ assert.ok(!('hls_expires_at' in vpub.body), 'and carries no expiry');
+ assert.strictEqual((await call('PUT', `/api/v1/live/vods/${vodId}`, { visibility: 'private' })).status, 200);
+ const vlist = await call('GET', '/api/v1/live/vods?limit=5&include_private=1');
+ assert.ok(vlist.body.vods.find((v) => v.id === vodId).hls_url, 'the VOD list carries hls_url too');
+ console.log('✅ GET /vods/:id: hls_url signed for a private VOD, plain for a public one, and it plays');
+
+ // ── A list read is a constant number of timeline queries and never queues object.cmaf (F3.c review) ──
+ // The HLS fields of a page come from objects/hls.discoveryMany: one batched timeline query, not one per row,
+ // and a list is not the place to queue object.cmaf (finalize and /download already do).
+ {
+ const timelineQueue = require('../server/objects/timeline-queue');
+ const realQueueCmaf = timelineQueue.queueCmaf;
+ const realAll = db.all;
+ let queued = 0;
+ let timelineQueries = 0;
+ timelineQueue.queueCmaf = async (...a) => { queued++; return await realQueueCmaf(...a); };
+ db.all = async (sql, params) => { if (/media_timeline/.test(sql)) timelineQueries++; return await realAll(sql, params); };
+ const bulk = [];
+ try {
+ for (let i = 0; i < 6; i++) {
+ const oid = await model.createObject({ app_id: 'live', owner_user_id: 5, visibility: 'public', lifecycle_status: 'ready', kind: 'vod', mime_type: 'video/mp4', size_bytes: 1 });
+ const vid = (await db.run('INSERT INTO vods (app_id, object_id, title, visibility, is_public) VALUES (?, ?, ?, ?, 1) RETURNING id', ['live', oid, `Bulk ${i}`, 'public'])).lastInsertRowid;
+ bulk.push({ vid, oid });
+ }
+ await timeline.replace(bulk[0].oid, 'source', src); // one of them has a timeline
+
+ timelineQueries = 0;
+ const one = await call('GET', '/api/v1/live/vods?limit=1&include_private=1');
+ assert.strictEqual(one.status, 200, JSON.stringify(one.body));
+ const q1 = timelineQueries;
+ timelineQueries = 0;
+ const many = await call('GET', '/api/v1/live/vods?limit=50&include_private=1');
+ assert.ok(many.body.vods.length >= 4, `the page lists the bulk VODs (${many.body.vods.length})`);
+ assert.strictEqual(timelineQueries, q1, `a page of ${many.body.vods.length} VODs runs the same number of timeline queries as one`);
+ assert.strictEqual(timelineQueries, 1, 'exactly one batched timeline query answers the page');
+ assert.strictEqual(queued, 0, 'a list request never queues object.cmaf');
+ const withHls = many.body.vods.find((v) => v.id === bulk[0].vid);
+ assert.strictEqual(withHls.hls_url, `${config.publicUrl}/o/${bulk[0].oid}/master.m3u8`, 'the batched answer reaches the row');
+ } finally {
+ db.all = realAll;
+ timelineQueue.queueCmaf = realQueueCmaf;
+ }
+ }
+
+ // ── A single-item read still queues object.cmaf for a timeline-less object, deduped per process ──
+ {
+ const timelineQueue = require('../server/objects/timeline-queue');
+ const realQueueCmaf = timelineQueue.queueCmaf;
+ let calls = 0;
+ timelineQueue.queueCmaf = async (...a) => { calls++; return await realQueueCmaf(...a); };
+ try {
+ const fresh = await db.createVod({ app_id: 'live', title: 'No timeline', visibility: 'private' });
+ const freshId = fresh.lastInsertRowid;
+ await db.run("UPDATE media_objects SET lifecycle_status = 'ready' WHERE id = ?", [(await db.getVodById(freshId, 'live')).object_id]);
+ assert.strictEqual((await call('GET', `/api/v1/live/vods/${freshId}`)).status, 200);
+ assert.strictEqual((await call('GET', `/api/v1/live/vods/${freshId}`)).status, 200);
+ assert.strictEqual(calls, 1, 'two immediate reads queue object.cmaf once (10-minute in-process dedupe)');
+ } finally {
+ timelineQueue.queueCmaf = realQueueCmaf;
+ }
+ }
+
+ // ── An event payload never carries a playlist URL (F3.c review): announce() feeds the durable outbox ──
+ // vodPublic also shapes vod.ready and _clipPublic shapes clip.ready/clip.failed; a private VOD's signed
+ // hls_url would otherwise sit in the outbox for up to 12 h and reach every subscriber.
+ {
+ const events = require('../server/events');
+ const realRecord = events.record;
+ const captured = [];
+ events.record = async (event, appId, data) => { captured.push({ event, data }); return await realRecord(event, appId, data); };
+ try {
+ // The private source's timeline now exists, so a second finalize announces vod.ready with it present.
+ assert.ok(await finalizeVod(vodId), 'the second finalize settles');
+ const vodReady = captured.filter((e) => e.event === 'vod.ready').pop();
+ assert.ok(vodReady, 'vod.ready was announced');
+ assert.ok(!('hls_url' in vodReady.data) && !('hls_expires_at' in vodReady.data), 'a vod.ready payload never carries a playlist URL');
+ } finally {
+ events.record = realRecord;
+ }
+ // Positive control: the private VOD's own API shape still answers a signed playlist.
+ const api = await call('GET', `/api/v1/live/vods/${vodId}`);
+ assert.ok(api.body.hls_url && new URL(api.body.hls_url).searchParams.get('sig'), 'the API shape still answers a signed hls_url');
+ }
+
// ── A clip over it is virtual: ready at once, no clip.cut ──
// ── A clip over it is virtual: ready at once, no clip.cut ──
r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 5, end_s: 11, visibility: 'private', title: 'Virtual' });
r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 5, end_s: 11, visibility: 'private', title: 'Virtual' });
assert.strictEqual(r.status, 201, JSON.stringify(r.body));
assert.strictEqual(r.status, 201, JSON.stringify(r.body));
@@ -129,6 +230,18 @@ const express = require('express');
@@ -129,6 +230,18 @@ const express = require('express');
assert.ok(!(await db.get('SELECT 1 FROM media_verifications WHERE object_id = ?', [clipObj])), 'and reports no no_good_copy for it');
assert.ok(!(await db.get('SELECT 1 FROM media_verifications WHERE object_id = ?', [clipObj])), 'and reports no no_good_copy for it');
console.log('✅ POST /clips over a timeline: ready at once (201), storage_provider timeline, playable, no clip.cut, not verified');
console.log('✅ POST /clips over a timeline: ready at once (201), storage_provider timeline, playable, no clip.cut, not verified');
+ // ── The v1 clip names its playlist too: its own token, over its source window, serving 200 ──
+ r = await call('GET', `/api/v1/live/clips/${clipId}`);
+ assert.strictEqual(r.status, 200, JSON.stringify(r.body));
+ const chu = new URL(r.body.hls_url);
+ assert.strictEqual(chu.pathname, `/o/${clipObj}/master.m3u8`, 'the clip\'s hls_url names its own master playlist');
+ assert.ok(signing.verifyPlaylist(clipObj, chu.searchParams.get('exp'), chu.searchParams.get('sig')), 'and carries the clip\'s playlist token');
+ assert.strictEqual((await get(r.body.hls_url)).status, 200, 'the clip hls_url serves 200');
+ assert.strictEqual((await jobs('object.cmaf', `AND object_id = '${clipObj}'`)).length, 0, 'a virtual clip is never queued object.cmaf');
+ const clist = await call('GET', `/api/v1/live/clips?vod_id=${vodId}&limit=5&include_private=1`);
+ assert.ok(clist.body.clips.find((c) => c.id === clipId).hls_url, 'the clip list carries hls_url too');
+ console.log('✅ GET /clips/:id: hls_url is the clip\'s playlist token over its source window, and it plays');
+
// ── Its playlists: hlsObject() rules, the source's segments inside the window ──
// ── Its playlists: hlsObject() rules, the source's segments inside the window ──
for (const p of ['master.m3u8', 'source/index.m3u8', 'source/init.mp4', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${clipObj}/${p}`)).status, 404, `unsigned ${p}`);
for (const p of ['master.m3u8', 'source/index.m3u8', 'source/init.mp4', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${clipObj}/${p}`)).status, 404, `unsigned ${p}`);
const tok = new URL(signing.signedPlaylistUrl(clipObj).url).search;
const tok = new URL(signing.signedPlaylistUrl(clipObj).url).search;
@@ -217,9 +330,35 @@ const express = require('express');
@@ -217,9 +330,35 @@ const express = require('express');
assert.strictEqual(r.status, 202, JSON.stringify(r.body));
assert.strictEqual(r.status, 202, JSON.stringify(r.body));
assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'MEDIA_HLS_ENABLED off: clip.cut');
assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'MEDIA_HLS_ENABLED off: clip.cut');
assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 404, 'and the HLS routes are off');
assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 404, 'and the HLS routes are off');
+ assert.ok(!('hls_url' in (await call('GET', `/api/v1/live/vods/${vodId}`)).body), 'flag off: the VOD omits hls_url');
+ assert.ok(!('hls_url' in (await call('GET', `/api/v1/live/clips/${clipId}`)).body), 'flag off: the clip omits hls_url');
config.hls.enabled = true;
config.hls.enabled = true;
console.log('✅ MEDIA_HLS_ENABLED off: clips are cut as before');
console.log('✅ MEDIA_HLS_ENABLED off: clips are cut as before');
+ // ── A clip.ready payload likewise never carries the (signed) playlist URL ──
+ {
+ const clipJobs = require('../server/vod/clip-jobs');
+ const payloadClip = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Payload', file_path: null, start_time: 16.5, end_time: 18.5,
+ duration_seconds: 2, is_public: 0, visibility: 'private', status: 'processing' })).lastInsertRowid;
+ const win = await clipJobs.timelineWindow(await db.getVodById(vodId), 16.5, 18.5);
+ assert.ok(win, 'a window over the source timeline');
+ const events = require('../server/events');
+ const realRecord = events.record;
+ const captured = [];
+ events.record = async (event, appId, data) => { captured.push({ event, data }); return await realRecord(event, appId, data); };
+ try {
+ await clipJobs.makeVirtual(payloadClip, win);
+ } finally {
+ events.record = realRecord;
+ }
+ const clipReady = captured.filter((e) => e.event === 'clip.ready').pop();
+ assert.ok(clipReady, 'clip.ready was announced');
+ assert.ok(!('hls_url' in clipReady.data) && !('hls_expires_at' in clipReady.data), 'a clip.ready payload never carries a playlist URL');
+ // Positive control: the clip's API shape still answers a signed playlist.
+ const api = await require('../server/vod/clips-routes').clipPublic(await db.getClipById(payloadClip), { readiness: true });
+ assert.ok(api.hls_url && new URL(api.hls_url).searchParams.get('sig'), 'the clip API shape still answers a signed hls_url');
+ }
+
server.close();
server.close();
fs.rmSync(tmp, { recursive: true, force: true });
fs.rmSync(tmp, { recursive: true, force: true });
console.log('✅ All clip timeline tests passed');
console.log('✅ All clip timeline tests passed');
@@ -468,6 +468,35 @@ const { spawn, spawnSync } = require('child_process');
@@ -468,6 +468,35 @@ const { spawn, spawnSync } = require('child_process');
assert.strictEqual((await get(`/o/${pub}/master.m3u8${signed.search}`)).status, 200, 'a public object needs no signature');
assert.strictEqual((await get(`/o/${pub}/master.m3u8${signed.search}`)).status, 200, 'a public object needs no signature');
assert.strictEqual((await call('GET', `/api/v2/live/objects/${pub}/download?format=json`)).body.hls_url, `${config.publicUrl}/o/${pub}/master.m3u8`);
assert.strictEqual((await call('GET', `/api/v2/live/objects/${pub}/download?format=json`)).body.hls_url, `${config.publicUrl}/o/${pub}/master.m3u8`);
+ // ── The shared discovery (objects/hls.js) the v1 VOD/clip shapes read answers exactly as /download does ──
+ const hls = require('../server/objects/hls');
+ assert.deepStrictEqual(await hls.discovery(pub), { hls_url: `${config.publicUrl}/o/${pub}/master.m3u8` }, 'an open object with a timeline: the plain master URL');
+ const privHls = new URL((await hls.discovery(priv)).hls_url);
+ assert.strictEqual(privHls.pathname, `/o/${priv}/master.m3u8`, 'a private object: the signed master URL');
+ assert.ok(privHls.searchParams.get('exp') && privHls.searchParams.get('sig'), 'with a playlist token');
+ assert.strictEqual(await hls.discovery('med_NOPE'), null, 'a missing object answers nothing');
+
+ // ── A sandbox tenant's object: signed, never the plain URL — per-row and batched alike ──
+ {
+ await db.ensureProjectTenant('prj_hls_sandbox', 'sandbox', 1024 * 1024);
+ const sb = await model.createObject({ app_id: 'prj_hls_sandbox-sandbox', visibility: 'public', lifecycle_status: 'ready', kind: 'file', mime_type: 'video/mp4', size_bytes: 1 });
+ await timeline.replace(sb, 'source', [row(0, 0, 0, 500), row(1, 0, 3000, 30000)]);
+ const plain = `${config.publicUrl}/o/${sb}/master.m3u8`;
+ const signing = require('../server/objects/signing');
+ const one = await hls.discovery(sb);
+ const u = new URL(one.hls_url);
+ assert.strictEqual(u.pathname, `/o/${sb}/master.m3u8`, 'the sandbox object\'s playlist URL names its master');
+ assert.ok(u.searchParams.get('exp') && u.searchParams.get('sig'), 'a sandbox object is signed');
+ assert.ok(signing.verifyPlaylist(sb, u.searchParams.get('exp'), u.searchParams.get('sig')), 'the signature verifies');
+ assert.notStrictEqual(one.hls_url, plain, 'a sandbox object never gets the plain URL');
+ assert.ok(one.hls_expires_at, 'and carries its expiry');
+ const many = await hls.discoveryMany([{ object_id: sb }, { object_id: pub }]);
+ assert.strictEqual(many.get(String(pub)).hls_url, `${config.publicUrl}/o/${pub}/master.m3u8`, 'the batch leaves a production object plain');
+ const bu = new URL(many.get(String(sb)).hls_url);
+ assert.ok(bu.searchParams.get('sig') && many.get(String(sb)).hls_url !== plain, 'and signs the sandbox object in the batch too');
+ assert.ok(many.get(String(sb)).hls_expires_at, 'the batched sandbox answer carries its expiry');
+ }
+
// ── MEDIA_HLS_ENABLED off: the routes 404, the job is refused, the download answer is as before ──
// ── MEDIA_HLS_ENABLED off: the routes 404, the job is refused, the download answer is as before ──
config.hls.enabled = false;
config.hls.enabled = false;
assert.strictEqual((await get(`/o/${pub}/master.m3u8`)).status, 404);
assert.strictEqual((await get(`/o/${pub}/master.m3u8`)).status, 404);