openvibers/openvibe.media Commits
Serve clips as virtual windows over the source's timeline (#30)
Changes
@@@ -268,8 +268,9 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every| method | path | notes ||---|---|---|--| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated? }` → **202** `{ id, status: 'processing' }`; cut runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |++| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready || GET | `/clips/:id` | status: `processing | ready | failed` |++| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) || GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs || PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) || DELETE | `/clips/:id` | local + offloaded objects + row |@@@ -146,14 +146,39 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}object's `hls_url` stays unsigned. `GET …/download?format=mp4` answers the object's `remux` variant (`object.remux`,`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux(`dedupeActive`) and answers `202 { job_id }` while it is missing.++- **F3.5, shipped: virtual and materialized clips over the timeline (§4).****Still open in F3:** the growing live/DVR playlist written as OpenRe segments (F3.2), write-behind durability for-- live segments and its upload-lag metric, virtual and materialized clips over the timeline (chunk reference counts),-- sprites and captions on it, queuing `object.pack` after the cut by itself, segment-bucket demand, and Live's player++ live segments and its upload-lag metric, a materialized clip that references the source's chunks (chunk reference++ counts, boundary-only re-encode), sprites and captions on it, queuing `object.pack` after the cut by itself, segment-bucket demand, and Live's playermoving to HLS. Renditions are F4.## 4. Clips reuse the source- **Virtual clip** (default): `{source, start, end}` → a clipped manifest over the source's segments. No bytes copied.++- **F3.5, shipped.** `POST /api/v1/:app/clips` over a VOD whose object has a source timeline (with `MEDIA_HLS_ENABLED`)++ makes a **virtual** clip: no ffmpeg, ready at once (**201**, `clip.ready` as before), the row `status='ready'`,++ `file_path` NULL, `storage_provider='timeline'` (the marker; no schema change), its object ready with no location++ and its `clip_of` edge carrying the window (`start_time`/`end_time`, seconds; the end clamped to the timeline's).++ `timeline.clipRows(rows, startMs, endMs)` writes it: the media segments intersecting the window plus the init++ segment, `seq` renumbered 1..n, times clipped to the window; `key`/`local_path`/`durable_provider`/`packed_object_id`/++ `byte_*` and the name stay the source's, so a segment serves from the source's bytes or a slice of its packed chunk.++ A segment straddling an edge is served whole (the clip's edges are the source's segment boundaries). The clip's++ `/o/<clip>/master.m3u8`, `source/index.m3u8` and `source/<name>` go through `hlsObject()` with the **clip's** own++ visibility and signature (a playlist token over the clip's id, purpose `hls`, or its download signature); the++ segment route resolves a name on the clip's own timeline, else only among `clipRows` of its source: **a clip's token++ reaches the segments inside its window and nothing else** (the rest of the source is a 404, the source's routes++ refuse it, a source token is never needed and opens nothing of the clip, `GET /o/:id` refuses it). A clip with a++ timeline of its own uses it. A virtual clip's source must be the same tenant's ready object; a deleted source stops++ it. `…/download?format=json` names its `hls_url` and never queues `object.cmaf` for it; `/c/:id` (public, the owner,++ or a `getc` URL from `GET /clips/:id/signed-url`) answers a virtual clip with a 302 to its master playlist (signed++ with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); `playback_url` is++ unchanged. Its object's metadata carries `virtual: true`: readiness counts it `bytes_verified`/`playable` without a++ location of its own, and the verify job skips it. **Materialized** = today's `clip.cut` full++ re-encode into the clip's own file: with `materialize: true` on `POST /clips` or `POST /clips/:id/recut`, or when++ the source has no timeline (or the flag is off). A virtual clip keeps playing while it is materialized and stays++ virtual if the cut fails; a clip with a file is never turned virtual (its recut stays a cut). The retry sweeper++ recovers a failed, file-less clip the way a recut does: virtual when its source now has a timeline. Holds still inherit++ through `clip_of`.- **Materialized clip** (shared externally, downloaded, edited, popular): middle GOPs are **referenced**, not copied(the new manifest points at the source's packed chunks); only the two boundary segments are re-encoded forframe-accurate cuts. Segments and chunks are content-addressed (sha256) with reference counts, so deleting a VOD@@@ -302,8 +327,9 @@ metric. Budgets per class and provider with a forecast; `media.provider.cost_thrthree failures (§6, shipped). Still open in F2: every class (not only native objects ↔ R2) through the one sweep.- **F3 segment-native video:** the timeline index and the CMAF/HLS source representation of finished video (F3.1,shipped; §3), packing into ~60 s chunks (F3.3, shipped; §3), the timeline queued at finalize (and lazily on-- download), signed playlists outliving a download URL and the faststart MP4 fallback (F3.c, shipped; §3); still open:-- live DVR from OpenRe (CMAF/HLS recording with the growing playlist), virtual clips, sprites and captions on the++ download), signed playlists outliving a download URL and the faststart MP4 fallback (F3.c, shipped; §3), virtual and++ materialized clips (F3.5, shipped; §4); still open: live DVR from OpenRe (CMAF/HLS recording with the growing++ playlist), materialized clips referencing the source's chunks, sprites and captions on thetimeline, segment-bucket demand; Live's player moves to HLS.- **F4 reactive derivatives:** on-demand renditions and image variants, keep-vs-regenerate economics, AV1 for viralVODs, compute placement.@@@ -333,7 +333,10 @@ function clipProjection(row) {const clipsDir = path.resolve(config.vod.clipsPath);const { locs, canonical } = tieredLocations(row, [row.file_path || null, row.file_path ? path.join(clipsDir, path.basename(row.file_path)) : null]);const status = row.status || 'ready';-- const lifecycle = status === 'processing' ? 'uploading' : status === 'failed' ? 'failed' : (row.file_path ? 'ready' : 'failed');++ // A virtual clip (storage_provider 'timeline', F3.5) has no file of its own: it plays its window of the source VOD's++ // timeline (objects/routes.js timelineOf), so it is ready without one.++ const virtual = !row.file_path && row.storage_provider === 'timeline';++ const lifecycle = status === 'processing' ? 'uploading' : status === 'failed' ? 'failed' : (row.file_path || virtual ? 'ready' : 'failed');const local = locs.find(l => l.provider === 'local' && l.state === 'present');return {legacy_ref: legacyRef(row.app_id, 'clip', row.id), existingId: row.object_id,@@@ -342,7 +345,9 @@ function clipProjection(row) {mime_type: row.file_path ? mimeFor(row.file_path, 'video/webm') : null,size_bytes: local ? local.size_bytes : 0,metadata: { title: row.title || null, duration_seconds: Number(row.duration_seconds) || 0, start_time: row.start_time, end_time: row.end_time,-- channel_user_id: row.channel_user_id || null, auto_generated: !!row.auto_generated },++ channel_user_id: row.channel_user_id || null, auto_generated: !!row.auto_generated,++ // Marks the clip virtual for readiness and the verify job; undefined drops the key once it is materialized.++ virtual: virtual || undefined },created_at: row.created_at, locations: locs, canonical,};}@@@ -40,13 +40,20 @@ function isVerifiedCopy(obj, l) {return l.state === 'present' && !!l.verified_at && !(l.checksum && obj.content_hash && l.checksum !== obj.content_hash);}++/** A ready clip object with no file of its own, played as a window of its source's timeline (model.js clipProjection). */++function isVirtualClip(obj, md = metadataOf(obj)) {++ return obj.kind === 'clip' && obj.lifecycle_status === 'ready' && !!md && md.virtual === true;++}++/** Readiness of one object row (null → nothing known). `locations` may be passed when already loaded. */async function compute(obj, locations = null) {if (!obj) return { metadata: false, bytes_verified: false, playable: false, hash_verified: false, verified_copies: [], reason: 'no_object' };const locs = locations || await db.all('SELECT provider, state, checksum, verified_at FROM media_locations WHERE object_id = ? ORDER BY id', [obj.id]);const md = metadataOf(obj);const verified = locs.filter(l => isVerifiedCopy(obj, l));-- const bytesVerified = verified.length > 0;++ // A virtual clip (F3.5) has no copy of its own: its bytes are its source's timeline segments, served under the++ // source's rules, so a ready one counts as having its bytes.++ const bytesVerified = verified.length > 0 || isVirtualClip(obj, md);let reason = null;if (obj.lifecycle_status === 'deleted') reason = 'deleted';else if (!md) reason = 'metadata_unreadable';@@@ -75,8 +82,9 @@ async function forRow(row) {function playableSql(ref) {return `EXISTS (SELECT 1 FROM media_objects ro WHERE ro.id = ${ref} AND ro.lifecycle_status = 'ready'AND ro.kind IN (${PLAYBACK_KINDS.map(k => `'${k}'`).join(', ')}) AND (CASE WHEN json_valid(ro.metadata) THEN json_type(ro.metadata) END) = 'object'-- AND EXISTS (SELECT 1 FROM media_locations rl WHERE rl.object_id = ro.id AND rl.state = 'present' AND rl.verified_at IS NOT NULL-- AND (rl.checksum IS NULL OR ro.content_hash IS NULL OR rl.checksum = ro.content_hash)))`;++ AND (EXISTS (SELECT 1 FROM media_locations rl WHERE rl.object_id = ro.id AND rl.state = 'present' AND rl.verified_at IS NOT NULL++ AND (rl.checksum IS NULL OR ro.content_hash IS NULL OR rl.checksum = ro.content_hash))++ OR (ro.kind = 'clip' AND (CASE WHEN json_valid(ro.metadata) THEN json_extract(ro.metadata, '$.virtual') END) IS NOT NULL)))`;}--module.exports = { compute, forRow, playableSql, isVerifiedCopy, PLAYBACK_KINDS, REASONS };++module.exports = { compute, forRow, playableSql, isVerifiedCopy, isVirtualClip, PLAYBACK_KINDS, REASONS };@@@ -672,8 +672,10 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r// With segment-native video on (MEDIA_HLS_ENABLED) and a timeline made, the JSON answer also names the HLS master// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.-- const hls = json && config.hls.enabled && await require('./timeline').has(obj.id);-- if (json && config.hls.enabled && !hls) {++ // A clip without its own timeline plays its window of the source's (F3.5) and is never cut a CMAF timeline here.++ const tl = json && config.hls.enabled ? await timelineOf(obj) : null;++ const hls = !!tl && tl.rows.some((r) => Number(r.seq) > 0);++ if (tl && !hls && !tl.window) {try { await require('./timeline-queue').queueCmaf(obj.app_id, obj.id); } catch (err) { console.warn(`[Objects] Timeline queue failed for ${obj.id}:`, err.message); }}if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {@@@ -782,6 +784,36 @@ const HLS_TYPE = 'application/vnd.apple.mpegurl';const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2++/**++ * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata++ * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.++ */++async function clipWindow(obj) {++ if (!obj || obj.kind !== 'clip') return null;++ const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);++ if (!rel) return null;++ const src = await model.getObject(rel.to_object_id);++ if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;++ const md = model.parseJson(rel.metadata, {});++ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);++ const endMs = Math.round(Number(md.end_time) * 1000) || 0;++ return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;++}++++/**++ * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a++ * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's++ * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.++ */++async function timelineOf(obj) {++ const timeline = require('./timeline');++ const own = await timeline.list(obj.id, timeline.SOURCE);++ if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };++ const window = await clipWindow(obj);++ if (!window) return { rows: [], window: null };++ return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };++}++/** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */async function hlsObject(req, res, next) {if (!config.hls.enabled) { next(); return null; }@@@ -814,14 +846,14 @@ const hlsRoute = (fn) => async (req, res, next) => {publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {const timeline = require('./timeline');-- const rows = await timeline.list(obj.id, timeline.SOURCE);++ const { rows } = await timelineOf(obj);if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));}));publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {const timeline = require('./timeline');-- const rows = await timeline.list(obj.id, timeline.SOURCE);++ const { rows } = await timelineOf(obj);if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));}));@@@ -873,7 +905,10 @@ async function sendSlice(req, res, { file, url, offset, length, headers }) {publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed }) => {const timeline = require('./timeline');const name = String(req.params.name || '');-- const row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;++ let row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;++ // A virtual clip serves only the source segments inside its window (timelineOf): any other name is a 404, whatever++ // the signature, so a clip's token never reaches the rest of its source.++ if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await timelineOf(obj)).rows.find((r) => r.name === name) || null;if (!row) return res.status(404).json({ error: 'Not found' });const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };@@@ -255,8 +255,26 @@ function masterPlaylist(renditions, { query = '' } = {}) {return lines.join('\n');}++// ── Clips (docs/media-fabric.md §4, F3.5) ───────────────────++/**++ * A virtual clip is a manifest over its source's rows: the media segments intersecting [startMs, endMs), the source's++ * init segment, seq renumbered 1..n, each row's times clipped to the window (and made relative to its start). key,++ * local_path, durable_provider, packed_object_id and byte_* stay the source's, so a segment serves from the source's++ * bytes (or its packed chunk), and its name, the playlist URI, never changes. A segment straddling an edge is served++ * whole: a virtual clip's edges are the source's segment boundaries (a materialized clip cuts exactly).++ */++function clipRows(rows, startMs, endMs) {++ const init = rows.find((r) => Number(r.seq) === 0);++ const segs = rows++ .filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs)++ .map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs }))++ .filter((r) => r.end_ms > r.start_ms)++ .map((r, i) => ({ ...r, seq: i + 1 }));++ return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...segs] : segs;++}++module.exports = {SOURCE, INIT_NAME, FIELDS, localRoot, localPathFor, keyFor, segmentName,list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, deleteBytes, removeObject, markPacked, durableProvider, locationsOf,-- mediaPlaylist, masterPlaylist,++ mediaPlaylist, masterPlaylist, clipRows,};@@@ -157,6 +157,7 @@ async function runOnce({ batch = config.verify.batch, head = defaultHead, uploadconst n = Math.max(1, Number(batch) || 1);const objects = await db.all(`SELECT o.* FROM media_objects o LEFT JOIN media_verifications v ON v.object_id = o.idWHERE o.lifecycle_status = 'ready'++ AND NOT (o.kind = 'clip' AND (CASE WHEN json_valid(o.metadata) THEN json_extract(o.metadata, '$.virtual') END) IS NOT NULL)ORDER BY (v.verified_at IS NOT NULL), v.verified_at, o.id LIMIT ?`, [n]);const budget = { left: Math.max(0, Number(maxReuploads) || 0) };for (const obj of objects) {@@@ -137,6 +137,20 @@ async function serveMediaRecord(kind, record, req, res) {// recordings exist (and, through the basename form of /v, which file names are real).if (visibility === 'private' && !canAccessPrivate(record, req, kind)) return notFound(res);++ // A virtual clip (F3.5) has no file: its bytes are its HLS playlist over the source's timeline. Every reader allowed++ // here (public, the owner, or a 'getc' URL from GET /api/clips/:id/signed-url), a browser navigation included, is sent++ // to the master playlist, signed for a closed clip with the clip's own 'hls' token, which reaches only the segments++ // inside the clip's window. Before the watch page: its player and download link name ?raw=1 bytes it does not have.++ if (kind === 'clip' && !record.file_path && record.storage_provider === 'timeline') {++ const config = require('../config');++ if (drill.refuseBytes(res)) return;++ if (!config.hls.enabled || !record.object_id || record.status !== 'ready') return res.status(404).json({ error: 'Media file unavailable' });++ await trackUniqueView(kind, record.id, req, record.user_id);++ const closed = visibility === 'private' || await db.isSandboxTenant(record.app_id);++ res.set({ 'Cache-Control': 'private, max-age=0', 'X-Robots-Tag': 'noindex' });++ return res.redirect(302, closed ? require('../objects/signing').signedPlaylistUrl(record.object_id).url : `${config.publicUrl}/o/${record.object_id}/master.m3u8`);++ }++// A person (or a link-preview crawler) landing on the URL gets the watch// page; its <video> comes back here with ?raw=1 for the bytes. The page offers a player only when// the object is playable (server/objects/readiness.js: finished, and a copy whose bytes were@@@ -244,7 +258,7 @@ router.get('/c/:id', optionalIdentity, async (req, res) => {const clip = await db.getClipById(parseInt(req.params.id, 10));// A clip still being cut (or whose cut failed) has no file yet: its bytes are a 404, and a// browser navigation gets the watch page saying so (private clips still look missing).-- if (!clip || (!clip.file_path && !pages.wantsHtmlPage(req))) return notFound(res);++ if (!clip || (!clip.file_path && clip.storage_provider !== 'timeline' && !pages.wantsHtmlPage(req))) return notFound(res);await serveMediaRecord('clip', clip, req, res);} catch (err) {console.error('[Public] /c error:', err.message);@@@ -9,8 +9,13 @@* cut and every re-cut run as a job, so they carry media.job.* events and a* job id the UI can reattach to (GET /api/v2/:app/jobs/:id). The queue* owns the retries (backoff 2, 10, 30 min; a hopeless source fails at once).++ * timelineWindow / makeVirtual a clip over a source that has a CMAF timeline (F3.5) is virtual: no cut, no++ * file; ready at once (storage_provider 'timeline') and played as its window of++ * the source's segments. clip.cut is the materialized path (asked for, or no++ * timeline); materializing a virtual clip keeps it playing until the file is in.* start() sweeper: every 3 min, failed clips that still have-- * attempts left are re-cut. From the second attempt on, a++ * attempts left are re-cut (or turned virtual, as a recut is,++ * when the source now has a timeline). From the second attempt on, a* VOD that lives in cloud storage is first pulled back to* local disk (moveToHot) — cutting a multi-hour recording* over HTTP is what used to time out — as long as there is@@@ -62,8 +67,9 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {return { ok: false, error: 'Source VOD no longer exists' };}const attempt = (Number(clip.cut_attempts) || 0) + 1;-- // Row and object (back to uploading) together.-- await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'processing', cut_attempts = ?, cut_next_at = NULL WHERE id = ?", [attempt, clipId]));++ // Row and object (back to uploading) together. A virtual clip being materialized stays ready and playing meanwhile.++ if (isVirtual(clip)) await db.run('UPDATE clips SET cut_attempts = ?, cut_next_at = NULL WHERE id = ?', [attempt, clipId]);++ else await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'processing', cut_attempts = ?, cut_next_at = NULL WHERE id = ?", [attempt, clipId]));// Attempt 2+: bring a cloud-stored VOD home first when the disk can take it.let source = await vodStorage.resolveMediaSource(vod);@@@ -88,7 +94,8 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {// object and its event in one transaction (webhooks.announce), then the webhook.try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }await announce(clip.app_id, 'clip.ready', {-- change: async () => await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET file_path = ?, duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL WHERE id = ?",++ change: async () => await objects().withObject('clip', clipId, async () => await db.run(`UPDATE clips SET file_path = ?, duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL,++ storage_provider = CASE WHEN storage_provider = 'timeline' THEN 'local' ELSE storage_provider END WHERE id = ?`,[cut.filePath, cut.duration, startTime + cut.duration, clipId])),payload: async () => await _clipPublic(await db.getClipById(clipId)),});@@@ -105,6 +112,12 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {const more = !viaJob && attempt < MAX_ATTEMPTS && !hopeless;const mins = BACKOFF_MIN[Math.min(attempt - 1, BACKOFF_MIN.length - 1)];const nextAt = more ? new Date(Date.now() + mins * 60000).toISOString().replace('T', ' ').slice(0, 19) : null;++ // A virtual clip whose materialization failed still plays from the timeline: it keeps its status and says why.++ if (isVirtual(clip)) {++ await db.run('UPDATE clips SET cut_error = ?, cut_next_at = NULL WHERE id = ?', [msg, clipId]);++ console.warn(`[Clips] Clip ${clipId} materialization attempt ${attempt} failed: ${msg} — it stays virtual`);++ return { ok: false, error: msg, retry_at: null, hopeless };++ }await announce(clip.app_id, 'clip.failed', {change: async () => await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'failed', cut_error = ?, cut_next_at = ? WHERE id = ?", [msg, nextAt, clipId])),payload: async () => await _clipPublic(await db.getClipById(clipId)),@@@ -113,6 +126,51 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {return { ok: false, error: msg, retry_at: nextAt, hopeless };}++// ── Virtual clips (docs/media-fabric.md §4, F3.5) ────────────++++function isVirtual(clip) { return !!clip && !clip.file_path && clip.storage_provider === 'timeline' && clip.status === 'ready'; }++++/**++ * The window [startTime, endTime) seconds over the source timeline of a VOD → { objectId, endTime } (endTime clamped to++ * the timeline's end), or null: MEDIA_HLS_ENABLED off (nothing would play it), no ready object, no timeline yet, or no++ * segment inside the window. Null means the clip is cut (clip.cut).++ */++async function timelineWindow(vod, startTime, endTime) {++ if (!config.hls.enabled || !vod || !vod.object_id || vod.is_recording) return null;++ const obj = await objects().getObject(vod.object_id);++ if (!obj || obj.lifecycle_status !== 'ready') return null;++ const timeline = require('../objects/timeline');++ const rows = timeline.clipRows(await timeline.list(obj.id, timeline.SOURCE), Math.round(startTime * 1000), Math.round(endTime * 1000));++ if (!rows.some((r) => Number(r.seq) > 0)) return null;++ const last = rows[rows.length - 1];++ return { objectId: obj.id, endTime: startTime + Number(last.end_ms) / 1000 };++}++++/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction, then the webhook. */++async function makeVirtual(clipId, win) {++ const clip = await db.getClipById(clipId);++ if (!clip) return { ok: false, error: 'Clip not found' };++ const startTime = Number(clip.start_time) || 0;++ const endTime = Math.min(Number(clip.end_time) || 0, win.endTime);++ await announce(clip.app_id, 'clip.ready', {++ change: async () => await objects().withObject('clip', clipId, async () => await db.run(`UPDATE clips SET file_path = NULL, storage_provider = 'timeline',++ duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL WHERE id = ?`, [endTime - startTime, endTime, clipId])),++ payload: async () => await _clipPublic(await db.getClipById(clipId)),++ });++ console.log(`[Clips] Clip ${clipId} virtual over vod ${clip.vod_id}'s timeline (${startTime.toFixed(1)}-${endTime.toFixed(1)}s)`);++ return { ok: true, duration_seconds: endTime - startTime };++}++++/** A file-less clip over a source with a timeline → made virtual (true); otherwise false and it is cut. */++async function retryVirtual(clipId) {++ const clip = await db.getClipById(clipId);++ if (!clip || clip.file_path || !clip.vod_id) return false;++ const win = await timelineWindow(await db.getVodById(clip.vod_id), Number(clip.start_time) || 0, Number(clip.end_time) || 0);++ if (!win) return false;++ await makeVirtual(clipId, win);++ return true;++}++async function sweep() {if (_busy) return;_busy = true;@@@ -122,7 +180,12 @@ async function sweep() {AND created_at >= datetime('now', '-30 days')AND NOT EXISTS (SELECT 1 FROM media_jobs j WHERE j.job_type = 'clip.cut' AND j.app_id = clips.app_id AND CAST(json_extract(j.params, '$.clip_id') AS INTEGER) = clips.id)ORDER BY created_at DESC LIMIT 4`, [MAX_ATTEMPTS]) || [];-- for (const row of due) { try { await recutClip(row.id, { reason: 'auto-retry' }); } catch (e) { console.warn(`[Clips] auto-retry ${row.id}:`, e.message); } }++ for (const row of due) {++ try {++ // The same recovery as POST /api/clips/:id/recut: a failed clip whose source now has a timeline turns virtual.++ if (!(await retryVirtual(row.id))) await recutClip(row.id, { reason: 'auto-retry' });++ } catch (e) { console.warn(`[Clips] auto-retry ${row.id}:`, e.message); }++ }} finally { _busy = false; }}@@@ -181,4 +244,4 @@ const spec = {},};--module.exports = { recutClip, sweep, start, enqueueCut, clipObjectId, spec, MAX_ATTEMPTS };++module.exports = { recutClip, sweep, start, enqueueCut, clipObjectId, timelineWindow, makeVirtual, isVirtual, spec, MAX_ATTEMPTS };@@@ -133,7 +133,9 @@ const VALID_VIS = new Set(['public', 'unlisted', 'private']);const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].includes(String(v)) ? 0 : null);// ── Create clip ──────────────────────────────────────────────--// JSON body: cut a window out of a VOD (202, background cut + webhook).++// JSON body: a window of a VOD. Over a source with a CMAF timeline (MEDIA_HLS_ENABLED) the clip is virtual: ready at++// once (201, clip.ready webhook), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).++// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut + webhook).// Multipart `video`: direct upload of an already-cut blob (201, ready) —// inherited browser-MediaRecorder clip path.router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {@@@ -170,14 +172,16 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.singreturn res.status(200).json({ id: duplicate.id, status: duplicate.status, deduplicated: true });}-- // Resolve a source ffmpeg can read: local file, or presigned B2/R2 URL.++ const clipJobs = require('./clip-jobs');++ const win = flag(body.materialize) !== 1 && await clipJobs.timelineWindow(vod, startTime, endTime);++ // Resolve a source ffmpeg can read: local file, or presigned B2/R2 URL (a virtual clip reads no file).const vodStorage = require('./vod-storage');-- const source = await vodStorage.resolveMediaSource(vod);-- if (!source) return res.status(404).json({ error: 'VOD media unavailable' });++ const source = win ? null : await vodStorage.resolveMediaSource(vod);++ if (!win && !source) return res.status(404).json({ error: 'VOD media unavailable' });// Live recording: clamp to what's ACTUALLY on disk, not wall-clock — a// seek past end-of-file emits a header-only "broken clip".-- if (vod.is_recording && source.kind === 'file') {++ if (vod.is_recording && source && source.kind === 'file') {let recordedDur = 0;try { const pi = await tools.probeVodInfo(source.value); recordedDur = pi.duration || 0; } catch { /* */ }if (recordedDur > 0) {@@@ -221,11 +225,15 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing}); // the row and its object (uploading) in one transactionconst clipId = result.lastInsertRowid;const appId = req.appId;++ if (win) {++ await clipJobs.makeVirtual(clipId, win);++ return res.status(201).json({ ...await clipPublic(await db.getClipById(clipId, appId), { readiness: true }) });++ }// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.// clip.ready / clip.failed and the webhook come from the cut itself, as before.-- const { job } = await require('./clip-jobs').enqueueCut(appId, clipId, {++ const { job } = await clipJobs.enqueueCut(appId, clipId, {reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,});@@@ -360,6 +368,8 @@ router.get('/:id', tenantAuth({ verb: 'read', allowUser: true }), async (req, re* (vod_id, start_time, end_time), so a failure is recoverable rather than* permanent — previously a failed clip stayed broken forever with no retry path,* and the viewer was shown "the server is cutting your clip" indefinitely.++ * A clip without a file, over a source with a timeline, becomes virtual instead (200, ready); `materialize: true` (body or query)++ * always cuts its own file (clip.cut), and a virtual clip keeps playing while it is cut.*/router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {try {@@@ -368,10 +378,21 @@ router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {const clipId = clip.id;if (clip.status === 'processing') return res.status(409).json({ error: 'Clip is already being cut' });if (!clip.vod_id) return res.status(422).json({ error: 'Clip has no source VOD to re-cut from' });++ const clipJobs = require('./clip-jobs');++ const materialize = flag((req.body || {}).materialize ?? req.query.materialize) === 1;++ // A clip that has a file of its own keeps it: only a file-less one (failed, or already virtual) turns virtual.++ if (!materialize && !clip.file_path) {++ const vod = await db.getVodById(clip.vod_id, req.appId);++ const win = await clipJobs.timelineWindow(vod, Number(clip.start_time) || 0, Number(clip.end_time) || 0);++ if (win) {++ await clipJobs.makeVirtual(clipId, win);++ return res.json({ ...await clipPublic(await db.getClipById(clipId, req.appId), { readiness: true }) });++ }++ }// A manual retry resets the attempt counter so it gets the full ladder again (not projected: no object change).await db.run("UPDATE clips SET cut_attempts = 0 WHERE id = ?", [clipId]);-- const { job } = await require('./clip-jobs').enqueueCut(req.appId, clipId, { reason: 're-cut', createdBy: `app:${req.appId}` });-- res.status(202).json({ id: clipId, status: 'processing', job_id: job.id });++ const { job } = await clipJobs.enqueueCut(req.appId, clipId, { reason: materialize ? 'materialize' : 're-cut', createdBy: `app:${req.appId}` });++ res.status(202).json({ id: clipId, status: clipJobs.isVirtual(clip) ? 'ready' : 'processing', job_id: job.id });} catch (err) {console.error('[Clips] Re-cut error:', err.message);res.status(500).json({ error: 'Failed to re-cut clip' });nnew file mode 100644@@@ -0,0 +1,228 @@++'use strict';++// Clips over the timeline (F3.5; docs/media-fabric.md §4): timeline.clipRows cuts a source's rows to a window (edges,++// empty window, init segment); a clip over a VOD with a timeline is virtual — ready at once, no clip.cut — and plays as++// HLS over the source's segments through the hlsObject() rules; its playlist token reaches only the segments inside its++// window (never the rest of the source, never GET /o/:id); /c/:id sends a reader to that playlist. materialize still++// queues clip.cut, a file clip keeps serving its file, and with MEDIA_HLS_ENABLED off a clip is cut as before.++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++const http = require('http');++const express = require('express');++++(async () => {++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-clip-timeline-'));++ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };++ Object.assign(process.env, {++ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'), THUMBNAILS_PATH: dir('thumbnails'),++ PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), MEDIA_PUBLIC_URL: 'https://media.test',++ MEDIA_HLS_ENABLED: '1', MEDIA_INVARIANT_SCAN_HOURS: '0',++ });++++ const config = require('../server/config');++ const db = require('../server/db/database');++ const model = require('../server/objects/model');++ const timeline = require('../server/objects/timeline');++ const signing = require('../server/objects/signing');++ const tools = require('../server/vod/media-tools');++ const thumbService = require('../server/thumbnails/thumbnail-service');++ const vodStorage = require('../server/vod/vod-storage');++ await vodStorage.tierConfig.init(vodStorage.DEFAULTS);++ await require('../server/objects/tier-policy').init();++ const { finalizeVod } = require('../server/vod/finalize');++ await db.upsertApp({ app_id: 'live', api_key: 'live-key-clip-timeline' });++++ // No ffmpeg in these paths; the job worker is never started, so queued jobs stay queued.++ thumbService.generateVodThumbnail = async () => null;++ tools.remuxForSeekingDetailed = async () => ({ ok: true, seconds: 30, error: null });++ tools.probeDuration = async () => ({ ok: true, seconds: 30, format: {}, streams: [], error: null });++ tools.remuxForSeeking = async () => true;++ tools.probeVodInfo = async () => ({ duration: 30 });++++ // ── clipRows: the window's edges ──++ const row = (seq, start, end, extra = {}) => ({ seq, name: timeline.segmentName(seq), start_ms: start, end_ms: end, keyframe_ms: seq ? start : null,++ key: `src/source/${timeline.segmentName(seq)}`, local_path: null, durable_provider: null, packed_object_id: null, byte_offset: null,++ byte_length: 1000, sha256: String(seq).repeat(64).slice(0, 64), durability: 'local', ...extra });++ const src = [row(0, 0, 0), row(1, 0, 4000), row(2, 4000, 8000), row(3, 8000, 12000, { packed_object_id: 'p'.repeat(64), byte_offset: 5, byte_length: 10 }),++ row(4, 12000, 16000), row(5, 16000, 18500)];++ let c = timeline.clipRows(src, 5000, 11000);++ assert.deepStrictEqual(c.map((r) => [r.seq, r.name, r.start_ms, r.end_ms]), [[0, 'init.mp4', 0, 0], [1, '000002.m4s', 0, 3000], [2, '000003.m4s', 3000, 6000]],++ 'segments straddling start and end are in, renumbered, times clipped and relative to the window');++ assert.deepStrictEqual([c[2].key, c[2].packed_object_id, c[2].byte_offset, c[2].byte_length, c[2].sha256], [src[3].key, src[3].packed_object_id, 5, 10, src[3].sha256],++ 'location and bytes stay the source\'s');++ assert.deepStrictEqual(timeline.clipRows(src, 4000, 8000).map((r) => r.name), ['init.mp4', '000002.m4s'], 'a window on segment boundaries takes exactly those');++ assert.deepStrictEqual(timeline.clipRows(src, 17000, 60000).map((r) => [r.name, r.end_ms]), [['init.mp4', 0], ['000005.m4s', 1500]], 'the tail stops at the source\'s end');++ assert.deepStrictEqual(timeline.clipRows(src, 30000, 40000), [], 'an empty window has no rows, not even the init segment');++ assert.deepStrictEqual(timeline.clipRows(src, 4000, 4000), [], 'a zero-length window too');++ assert.strictEqual(src[2].seq, 2, 'the source rows are not changed');++ const mp = timeline.mediaPlaylist(c, { query: 'exp=1&sig=x' });++ assert.ok(mp.includes('#EXT-X-MEDIA-SEQUENCE:1') && mp.includes('#EXTINF:3.000,\n000002.m4s?exp=1&sig=x') && !mp.includes('000001.m4s'), mp);++ console.log('✅ clipRows: straddling edges, boundaries, tail, empty window, init segment');++++ const app = express();++ app.use(express.json());++ app.use('/api/v1/:app/vods', require('../server/vod/routes'));++ app.use('/api/v1/:app/clips', require('../server/vod/clips-routes'));++ app.use('/api/v2/:app/objects', require('../server/objects/routes'));++ app.use('/o', require('../server/objects/routes').publicRouter);++ app.use('/', require('../server/public/routes'));++ const server = http.createServer(app);++++ (async () => {++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ const api = `http://127.0.0.1:${server.address().port}`;++ const auth = { authorization: 'Bearer live-key-clip-timeline' };++ const call = async (method, p, body) => {++ const headers = { ...auth };++ if (body && !(body instanceof FormData)) headers['content-type'] = 'application/json';++ const r = await fetch(api + p, { method, headers, body: body && !(body instanceof FormData) ? JSON.stringify(body) : body, redirect: 'manual' });++ return { status: r.status, body: await r.json().catch(() => null) };++ };++ const get = async (url, headers = {}) => {++ const u = new URL(url, api);++ const r = await fetch(api + u.pathname + u.search, { headers, redirect: 'manual' });++ return { status: r.status, location: r.headers.get('location'), buf: Buffer.from(await r.arrayBuffer()) };++ };++ const jobs = async (type, where = '') => await db.all(`SELECT * FROM media_jobs WHERE job_type = ? ${where}`, [type]);++ const clipCuts = async (clipId) => (await jobs('clip.cut')).filter((j) => Number(model.parseJson(j.params, {}).clip_id) === Number(clipId));++++ // A private, finished VOD with a source timeline on this node's disk (segment 3 packed in a chunk).++ let r = await call('POST', '/api/v1/live/vods', { title: 'Source', user_id: 5, visibility: 'private' });++ assert.strictEqual(r.status, 201);++ const vodId = r.body.id;++ const f = new FormData();++ f.append('chunk', new Blob([Buffer.alloc(40000, 1)], { type: 'video/webm' }), 'a.webm');++ assert.strictEqual((await call('POST', `/api/v1/live/vods/${vodId}/chunks`, f)).body.status, 'created');++ const vodObj = (await finalizeVod(vodId)).object_id;++ const vodRow = await model.getObject(vodObj);++ const bytes = {};++ const rows = src.map((s) => {++ const p = timeline.localPathFor(vodRow, 'source', s.name, 'v1');++ fs.mkdirSync(path.dirname(p), { recursive: true });++ bytes[s.name] = Buffer.from(`${s.name}:`.padEnd(64, String(s.seq)));++ if (s.packed_object_id) {++ fs.writeFileSync(p, Buffer.concat([Buffer.from('HEAD!'), bytes[s.name].subarray(0, 10), Buffer.from('TAIL')]));++ bytes[s.name] = bytes[s.name].subarray(0, 10);++ } else fs.writeFileSync(p, bytes[s.name]);++ return { ...s, local_path: p, byte_length: s.packed_object_id ? 10 : bytes[s.name].length };++ });++ await timeline.replace(vodObj, 'source', rows);++++ // ── A clip over it is virtual: ready at once, no clip.cut ──++ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 5, end_s: 11, visibility: 'private', title: 'Virtual' });++ assert.strictEqual(r.status, 201, JSON.stringify(r.body));++ assert.deepStrictEqual([r.body.status, r.body.storage_provider, r.body.file_path, r.body.start_time, r.body.end_time, r.body.duration_seconds],++ ['ready', 'timeline', null, 5, 11, 6]);++ assert.strictEqual(r.body.playback_url, `${config.publicUrl}/c/${r.body.id}`, 'playback_url unchanged');++ const clipId = r.body.id;++ const clipObj = (await db.getClipById(clipId)).object_id;++ const co = await model.getObject(clipObj);++ assert.deepStrictEqual([co.kind, co.lifecycle_status, co.visibility], ['clip', 'ready', 'private']);++ assert.strictEqual((await clipCuts(clipId)).length, 0, 'no clip.cut for a virtual clip');++ assert.deepStrictEqual([r.body.readiness.playable, r.body.readiness.bytes_verified, r.body.readiness.reason], [true, true, null], 'a virtual clip is playable');++ assert.strictEqual(model.parseJson(co.metadata, {}).virtual, true);++ const readiness = require('../server/objects/readiness');++ assert.ok(await db.get(`SELECT 1 FROM clips WHERE id = ? AND ${readiness.playableSql('clips.object_id')}`, [clipId]), 'playableSql agrees');++ const verified = await require('../server/objects/verify-job').runOnce({ batch: 50, head: async () => undefined, upload: async () => { throw new Error('no upload in this test'); }, maxReuploads: 0 });++ assert.ok(!verified.objects.some((o) => o.object_id === clipObj), 'the verify job leaves a virtual clip alone');++ assert.ok(!(await db.get('SELECT 1 FROM media_verifications WHERE object_id = ?', [clipObj])), 'and reports no no_good_copy for it');++ console.log('✅ POST /clips over a timeline: ready at once (201), storage_provider timeline, playable, no clip.cut, not verified');++++ // ── Its playlists: hlsObject() rules, the source's segments inside the window ──++ for (const p of ['master.m3u8', 'source/index.m3u8', 'source/init.mp4', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${clipObj}/${p}`)).status, 404, `unsigned ${p}`);++ const tok = new URL(signing.signedPlaylistUrl(clipObj).url).search;++ const master = await get(`/o/${clipObj}/master.m3u8${tok}`);++ assert.strictEqual(master.status, 200);++ assert.ok(master.buf.toString().includes(`source/index.m3u8${tok}`));++ const index = (await get(`/o/${clipObj}/source/index.m3u8${tok}`)).buf.toString();++ assert.ok(index.includes(`#EXT-X-MAP:URI="init.mp4${tok}"`) && index.includes(`#EXTINF:3.000,\n000002.m4s${tok}`) && index.includes(`#EXTINF:3.000,\n000003.m4s${tok}`), index);++ assert.ok(!/00000[145]\.m4s/.test(index), 'nothing outside the window is listed');++ for (const n of ['init.mp4', '000002.m4s', '000003.m4s']) {++ const s = await get(`/o/${clipObj}/source/${n}${tok}`);++ assert.strictEqual(s.status, 200, n);++ assert.deepStrictEqual(s.buf, bytes[n], `${n}: the source's bytes${n === '000003.m4s' ? ' (a slice of its packed chunk)' : ''}`);++ }++ console.log('✅ a private virtual clip: 404 unsigned, signed playlists over the source window, source bytes and packed slices');++++ // ── Token scope: the clip's window only ──++ for (const n of ['000001.m4s', '000004.m4s', '000005.m4s']) assert.strictEqual((await get(`/o/${clipObj}/source/${n}${tok}`)).status, 404, `clip token refused for ${n}`);++ for (const p of ['master.m3u8', 'source/index.m3u8', 'source/000001.m4s', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${vodObj}/${p}${tok}`)).status, 404, `clip token never opens the source's ${p}`);++ assert.strictEqual((await get(`/o/${clipObj}${tok}`)).status, 404, 'GET /o/:id refuses a playlist token');++ const vodTok = new URL(signing.signedPlaylistUrl(vodObj).url).search;++ assert.strictEqual((await get(`/o/${clipObj}/source/000002.m4s${vodTok}`)).status, 404, 'a source token does not open the clip');++ assert.strictEqual((await get(`/o/${vodObj}/source/000001.m4s${vodTok}`)).status, 200, 'the source token still plays the source');++ console.log('✅ a clip token reaches only its window: not the source, not GET /o/:id; a source token is not needed nor accepted');++++ // ── Discovery: /download names the playlist (no object.cmaf for the clip); /c/:id redirects to it ──++ r = await call('GET', `/api/v2/live/objects/${clipObj}/download?format=json`);++ assert.strictEqual(r.status, 200, JSON.stringify(r.body));++ const hu = new URL(r.body.hls_url);++ assert.ok(hu.pathname === `/o/${clipObj}/master.m3u8` && signing.verifyPlaylist(clipObj, hu.searchParams.get('exp'), hu.searchParams.get('sig')));++ assert.strictEqual((await jobs('object.cmaf', 'AND object_id = ?'.replace('?', `'${clipObj}'`))).length, 0, 'no object.cmaf for a virtual clip');++ assert.strictEqual((await get(`/c/${clipId}`)).status, 404, '/c/:id of a private clip, unsigned: 404');++ const cr = await get(new URL(signing.signedMediaUrl('clip', clipId, 600).url).pathname + new URL(signing.signedMediaUrl('clip', clipId, 600).url).search);++ assert.strictEqual(cr.status, 302);++ const cu = new URL(cr.location);++ assert.ok(cu.pathname === `/o/${clipObj}/master.m3u8` && signing.verifyPlaylist(clipObj, cu.searchParams.get('exp'), cu.searchParams.get('sig')), cr.location);++ const signedC = new URL(signing.signedMediaUrl('clip', clipId, 600).url);++ for (const h of [{ 'sec-fetch-dest': 'document' }, { accept: 'text/html,application/xhtml+xml' }]) {++ const nav = await get(signedC.pathname + signedC.search, h);++ assert.strictEqual(nav.status, 302, `a browser navigation (${JSON.stringify(h)}) is redirected, not shown "not available"`);++ assert.strictEqual(new URL(nav.location).pathname, `/o/${clipObj}/master.m3u8`);++ }++ assert.strictEqual((await get(`/c/${clipId}`, { 'sec-fetch-dest': 'document' })).status, 404, 'unsigned navigation of a private clip: 404');++ console.log('✅ /download gives hls_url without queueing object.cmaf; /c/:id (signed, any reader) redirects to the clip\'s signed playlist');++++ // ── Materialize: still clip.cut, the virtual clip keeps playing meanwhile ──++ r = await call('POST', `/api/v1/live/clips/${clipId}/recut`, { materialize: true });++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));++ assert.ok(r.body.job_id && r.body.status === 'ready');++ assert.strictEqual((await clipCuts(clipId)).length, 1, 'recut?materialize queues clip.cut');++ assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 200, 'still playing');++ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 20, end_s: 23, materialize: true });++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));++ assert.ok(r.body.job_id && r.body.status === 'processing');++ assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'POST materialize queues clip.cut');++ console.log('✅ materialize (POST and recut) queues clip.cut; a virtual clip plays on while it is cut');++++ // ── A re-encoded clip keeps serving its file; its recut stays a cut ──++ const file = path.join(process.env.CLIPS_PATH, 'old.webm');++ fs.writeFileSync(file, Buffer.alloc(3000, 7));++ const old = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Old', file_path: file, start_time: 2, end_time: 6,++ duration_seconds: 4, is_public: 1, visibility: 'public', status: 'ready' })).lastInsertRowid;++ const oldObj = await model.getObject((await db.getClipById(old)).object_id);++ assert.strictEqual(oldObj.lifecycle_status, 'ready');++ const ob = await get(`/c/${old}?raw=1`);++ assert.deepStrictEqual([ob.status, ob.buf.length], [200, 3000], 'the file is served as before');++ r = await call('POST', `/api/v1/live/clips/${old}/recut`, {});++ assert.strictEqual(r.status, 202);++ assert.strictEqual((await clipCuts(old)).length, 1, 'a clip with a file is re-cut, not turned virtual');++ assert.strictEqual((await db.getClipById(old)).file_path, file);++ console.log('✅ a re-encoded clip keeps serving its file, and its recut is a cut');++++ // ── The retry sweeper recovers a failed, file-less clip the way a recut does: virtual over the timeline ──++ const failed = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Failed', file_path: null, start_time: 8, end_time: 12,++ duration_seconds: 4, is_public: 1, visibility: 'public', status: 'failed' })).lastInsertRowid;++ await require('../server/vod/clip-jobs').sweep();++ const fr = await db.getClipById(failed);++ assert.deepStrictEqual([fr.status, fr.storage_provider, fr.file_path], ['ready', 'timeline', null]);++ assert.strictEqual((await clipCuts(failed)).length, 0, 'no cut for it');++ console.log('✅ the retry sweeper turns a failed clip over a timeline virtual, as a recut does');++++ // ── Flag off: cut as before ──++ config.hls.enabled = false;++ await db.run("UPDATE clips SET created_at = '2000-01-01 00:00:00'"); // past the duplicate-clip window++ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 26, end_s: 29 });++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));++ assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'MEDIA_HLS_ENABLED off: clip.cut');++ assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 404, 'and the HLS routes are off');++ config.hls.enabled = true;++ console.log('✅ MEDIA_HLS_ENABLED off: clips are cut as before');++++ server.close();++ fs.rmSync(tmp, { recursive: true, force: true });++ console.log('✅ All clip timeline tests passed');++ process.exit(0);++ })().catch((err) => { console.error(err); process.exit(1); });++})().catch((err) => { console.error(err); process.exit(1); });
@@ -268,8 +268,9 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
@@ -268,8 +268,9 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
| method | path | notes |
| method | path | notes |
|---|---|---|
|---|---|---|
-| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated? }` → **202** `{ id, status: 'processing' }`; cut runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
+| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
| GET | `/clips/:id` | status: `processing | ready | failed` |
| GET | `/clips/:id` | status: `processing | ready | failed` |
+| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
| DELETE | `/clips/:id` | local + offloaded objects + row |
| DELETE | `/clips/:id` | local + offloaded objects + row |
@@ -146,14 +146,39 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
@@ -146,14 +146,39 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
object's `hls_url` stays unsigned. `GET …/download?format=mp4` answers the object's `remux` variant (`object.remux`,
object's `hls_url` stays unsigned. `GET …/download?format=mp4` answers the object's `remux` variant (`object.remux`,
`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux
`-movflags +faststart` for an MP4 source; the container stays the source's) as a signed URL, or queues the remux
(`dedupeActive`) and answers `202 { job_id }` while it is missing.
(`dedupeActive`) and answers `202 { job_id }` while it is missing.
+- **F3.5, shipped: virtual and materialized clips over the timeline (§4).**
**Still open in F3:** the growing live/DVR playlist written as OpenRe segments (F3.2), write-behind durability for
**Still open in F3:** the growing live/DVR playlist written as OpenRe segments (F3.2), write-behind durability for
- live segments and its upload-lag metric, virtual and materialized clips over the timeline (chunk reference counts),
- sprites and captions on it, queuing `object.pack` after the cut by itself, segment-bucket demand, and Live's player
+ live segments and its upload-lag metric, a materialized clip that references the source's chunks (chunk reference
+ counts, boundary-only re-encode), sprites and captions on it, queuing `object.pack` after the cut by itself, segment-bucket demand, and Live's player
moving to HLS. Renditions are F4.
moving to HLS. Renditions are F4.
## 4. Clips reuse the source
## 4. Clips reuse the source
- **Virtual clip** (default): `{source, start, end}` → a clipped manifest over the source's segments. No bytes copied.
- **Virtual clip** (default): `{source, start, end}` → a clipped manifest over the source's segments. No bytes copied.
+- **F3.5, shipped.** `POST /api/v1/:app/clips` over a VOD whose object has a source timeline (with `MEDIA_HLS_ENABLED`)
+ makes a **virtual** clip: no ffmpeg, ready at once (**201**, `clip.ready` as before), the row `status='ready'`,
+ `file_path` NULL, `storage_provider='timeline'` (the marker; no schema change), its object ready with no location
+ and its `clip_of` edge carrying the window (`start_time`/`end_time`, seconds; the end clamped to the timeline's).
+ `timeline.clipRows(rows, startMs, endMs)` writes it: the media segments intersecting the window plus the init
+ segment, `seq` renumbered 1..n, times clipped to the window; `key`/`local_path`/`durable_provider`/`packed_object_id`/
+ `byte_*` and the name stay the source's, so a segment serves from the source's bytes or a slice of its packed chunk.
+ A segment straddling an edge is served whole (the clip's edges are the source's segment boundaries). The clip's
+ `/o/<clip>/master.m3u8`, `source/index.m3u8` and `source/<name>` go through `hlsObject()` with the **clip's** own
+ visibility and signature (a playlist token over the clip's id, purpose `hls`, or its download signature); the
+ segment route resolves a name on the clip's own timeline, else only among `clipRows` of its source: **a clip's token
+ reaches the segments inside its window and nothing else** (the rest of the source is a 404, the source's routes
+ refuse it, a source token is never needed and opens nothing of the clip, `GET /o/:id` refuses it). A clip with a
+ timeline of its own uses it. A virtual clip's source must be the same tenant's ready object; a deleted source stops
+ it. `…/download?format=json` names its `hls_url` and never queues `object.cmaf` for it; `/c/:id` (public, the owner,
+ or a `getc` URL from `GET /clips/:id/signed-url`) answers a virtual clip with a 302 to its master playlist (signed
+ with the clip's playlist token when closed), a browser navigation too (no watch page for it yet); `playback_url` is
+ unchanged. Its object's metadata carries `virtual: true`: readiness counts it `bytes_verified`/`playable` without a
+ location of its own, and the verify job skips it. **Materialized** = today's `clip.cut` full
+ re-encode into the clip's own file: with `materialize: true` on `POST /clips` or `POST /clips/:id/recut`, or when
+ the source has no timeline (or the flag is off). A virtual clip keeps playing while it is materialized and stays
+ virtual if the cut fails; a clip with a file is never turned virtual (its recut stays a cut). The retry sweeper
+ recovers a failed, file-less clip the way a recut does: virtual when its source now has a timeline. Holds still inherit
+ through `clip_of`.
- **Materialized clip** (shared externally, downloaded, edited, popular): middle GOPs are **referenced**, not copied
- **Materialized clip** (shared externally, downloaded, edited, popular): middle GOPs are **referenced**, not copied
(the new manifest points at the source's packed chunks); only the two boundary segments are re-encoded for
(the new manifest points at the source's packed chunks); only the two boundary segments are re-encoded for
frame-accurate cuts. Segments and chunks are content-addressed (sha256) with reference counts, so deleting a VOD
frame-accurate cuts. Segments and chunks are content-addressed (sha256) with reference counts, so deleting a VOD
@@ -302,8 +327,9 @@ metric. Budgets per class and provider with a forecast; `media.provider.cost_thr
@@ -302,8 +327,9 @@ metric. Budgets per class and provider with a forecast; `media.provider.cost_thr
three failures (§6, shipped). Still open in F2: every class (not only native objects ↔ R2) through the one sweep.
three failures (§6, shipped). Still open in F2: every class (not only native objects ↔ R2) through the one sweep.
- **F3 segment-native video:** the timeline index and the CMAF/HLS source representation of finished video (F3.1,
- **F3 segment-native video:** the timeline index and the CMAF/HLS source representation of finished video (F3.1,
shipped; §3), packing into ~60 s chunks (F3.3, shipped; §3), the timeline queued at finalize (and lazily on
shipped; §3), packing into ~60 s chunks (F3.3, shipped; §3), the timeline queued at finalize (and lazily on
- download), signed playlists outliving a download URL and the faststart MP4 fallback (F3.c, shipped; §3); still open:
- live DVR from OpenRe (CMAF/HLS recording with the growing playlist), virtual clips, sprites and captions on the
+ download), signed playlists outliving a download URL and the faststart MP4 fallback (F3.c, shipped; §3), virtual and
+ materialized clips (F3.5, shipped; §4); still open: live DVR from OpenRe (CMAF/HLS recording with the growing
+ playlist), materialized clips referencing the source's chunks, sprites and captions on the
timeline, segment-bucket demand; Live's player moves to HLS.
timeline, segment-bucket demand; Live's player moves to HLS.
- **F4 reactive derivatives:** on-demand renditions and image variants, keep-vs-regenerate economics, AV1 for viral
- **F4 reactive derivatives:** on-demand renditions and image variants, keep-vs-regenerate economics, AV1 for viral
VODs, compute placement.
VODs, compute placement.
@@ -333,7 +333,10 @@ function clipProjection(row) {
@@ -333,7 +333,10 @@ function clipProjection(row) {
const clipsDir = path.resolve(config.vod.clipsPath);
const clipsDir = path.resolve(config.vod.clipsPath);
const { locs, canonical } = tieredLocations(row, [row.file_path || null, row.file_path ? path.join(clipsDir, path.basename(row.file_path)) : null]);
const { locs, canonical } = tieredLocations(row, [row.file_path || null, row.file_path ? path.join(clipsDir, path.basename(row.file_path)) : null]);
const status = row.status || 'ready';
const status = row.status || 'ready';
- const lifecycle = status === 'processing' ? 'uploading' : status === 'failed' ? 'failed' : (row.file_path ? 'ready' : 'failed');
+ // A virtual clip (storage_provider 'timeline', F3.5) has no file of its own: it plays its window of the source VOD's
+ // timeline (objects/routes.js timelineOf), so it is ready without one.
+ const virtual = !row.file_path && row.storage_provider === 'timeline';
+ const lifecycle = status === 'processing' ? 'uploading' : status === 'failed' ? 'failed' : (row.file_path || virtual ? 'ready' : 'failed');
const local = locs.find(l => l.provider === 'local' && l.state === 'present');
const local = locs.find(l => l.provider === 'local' && l.state === 'present');
return {
return {
legacy_ref: legacyRef(row.app_id, 'clip', row.id), existingId: row.object_id,
legacy_ref: legacyRef(row.app_id, 'clip', row.id), existingId: row.object_id,
@@ -342,7 +345,9 @@ function clipProjection(row) {
@@ -342,7 +345,9 @@ function clipProjection(row) {
mime_type: row.file_path ? mimeFor(row.file_path, 'video/webm') : null,
mime_type: row.file_path ? mimeFor(row.file_path, 'video/webm') : null,
size_bytes: local ? local.size_bytes : 0,
size_bytes: local ? local.size_bytes : 0,
metadata: { title: row.title || null, duration_seconds: Number(row.duration_seconds) || 0, start_time: row.start_time, end_time: row.end_time,
metadata: { title: row.title || null, duration_seconds: Number(row.duration_seconds) || 0, start_time: row.start_time, end_time: row.end_time,
- channel_user_id: row.channel_user_id || null, auto_generated: !!row.auto_generated },
+ channel_user_id: row.channel_user_id || null, auto_generated: !!row.auto_generated,
+ // Marks the clip virtual for readiness and the verify job; undefined drops the key once it is materialized.
+ virtual: virtual || undefined },
created_at: row.created_at, locations: locs, canonical,
created_at: row.created_at, locations: locs, canonical,
};
};
}
}
@@ -40,13 +40,20 @@ function isVerifiedCopy(obj, l) {
@@ -40,13 +40,20 @@ function isVerifiedCopy(obj, l) {
return l.state === 'present' && !!l.verified_at && !(l.checksum && obj.content_hash && l.checksum !== obj.content_hash);
return l.state === 'present' && !!l.verified_at && !(l.checksum && obj.content_hash && l.checksum !== obj.content_hash);
}
}
+/** A ready clip object with no file of its own, played as a window of its source's timeline (model.js clipProjection). */
+function isVirtualClip(obj, md = metadataOf(obj)) {
+ return obj.kind === 'clip' && obj.lifecycle_status === 'ready' && !!md && md.virtual === true;
+}
+
/** Readiness of one object row (null → nothing known). `locations` may be passed when already loaded. */
/** Readiness of one object row (null → nothing known). `locations` may be passed when already loaded. */
async function compute(obj, locations = null) {
async function compute(obj, locations = null) {
if (!obj) return { metadata: false, bytes_verified: false, playable: false, hash_verified: false, verified_copies: [], reason: 'no_object' };
if (!obj) return { metadata: false, bytes_verified: false, playable: false, hash_verified: false, verified_copies: [], reason: 'no_object' };
const locs = locations || await db.all('SELECT provider, state, checksum, verified_at FROM media_locations WHERE object_id = ? ORDER BY id', [obj.id]);
const locs = locations || await db.all('SELECT provider, state, checksum, verified_at FROM media_locations WHERE object_id = ? ORDER BY id', [obj.id]);
const md = metadataOf(obj);
const md = metadataOf(obj);
const verified = locs.filter(l => isVerifiedCopy(obj, l));
const verified = locs.filter(l => isVerifiedCopy(obj, l));
- const bytesVerified = verified.length > 0;
+ // A virtual clip (F3.5) has no copy of its own: its bytes are its source's timeline segments, served under the
+ // source's rules, so a ready one counts as having its bytes.
+ const bytesVerified = verified.length > 0 || isVirtualClip(obj, md);
let reason = null;
let reason = null;
if (obj.lifecycle_status === 'deleted') reason = 'deleted';
if (obj.lifecycle_status === 'deleted') reason = 'deleted';
else if (!md) reason = 'metadata_unreadable';
else if (!md) reason = 'metadata_unreadable';
@@ -75,8 +82,9 @@ async function forRow(row) {
@@ -75,8 +82,9 @@ async function forRow(row) {
function playableSql(ref) {
function playableSql(ref) {
return `EXISTS (SELECT 1 FROM media_objects ro WHERE ro.id = ${ref} AND ro.lifecycle_status = 'ready'
return `EXISTS (SELECT 1 FROM media_objects ro WHERE ro.id = ${ref} AND ro.lifecycle_status = 'ready'
AND ro.kind IN (${PLAYBACK_KINDS.map(k => `'${k}'`).join(', ')}) AND (CASE WHEN json_valid(ro.metadata) THEN json_type(ro.metadata) END) = 'object'
AND ro.kind IN (${PLAYBACK_KINDS.map(k => `'${k}'`).join(', ')}) AND (CASE WHEN json_valid(ro.metadata) THEN json_type(ro.metadata) END) = 'object'
- AND EXISTS (SELECT 1 FROM media_locations rl WHERE rl.object_id = ro.id AND rl.state = 'present' AND rl.verified_at IS NOT NULL
- AND (rl.checksum IS NULL OR ro.content_hash IS NULL OR rl.checksum = ro.content_hash)))`;
+ AND (EXISTS (SELECT 1 FROM media_locations rl WHERE rl.object_id = ro.id AND rl.state = 'present' AND rl.verified_at IS NOT NULL
+ AND (rl.checksum IS NULL OR ro.content_hash IS NULL OR rl.checksum = ro.content_hash))
+ OR (ro.kind = 'clip' AND (CASE WHEN json_valid(ro.metadata) THEN json_extract(ro.metadata, '$.virtual') END) IS NOT NULL)))`;
}
}
-module.exports = { compute, forRow, playableSql, isVerifiedCopy, PLAYBACK_KINDS, REASONS };
+module.exports = { compute, forRow, playableSql, isVerifiedCopy, isVirtualClip, PLAYBACK_KINDS, REASONS };
@@ -672,8 +672,10 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r
@@ -672,8 +672,10 @@ router.get('/:id/download', read, limits('media.object.download'), async (req, r
// With segment-native video on (MEDIA_HLS_ENABLED) and a timeline made, the JSON answer also names the HLS master
// With segment-native video on (MEDIA_HLS_ENABLED) and a timeline made, the JSON answer also names the HLS master
// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private
// playlist: open for a public object, signed as a playlist (purpose 'hls', MEDIA_HLS_PLAYLIST_TTL_S) for a private
// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.
// one. A media object without a timeline yet (a VOD older than F3.1) gets object.cmaf queued, and hls_url once it ran.
- const hls = json && config.hls.enabled && await require('./timeline').has(obj.id);
- if (json && config.hls.enabled && !hls) {
+ // A clip without its own timeline plays its window of the source's (F3.5) and is never cut a CMAF timeline here.
+ const tl = json && config.hls.enabled ? await timelineOf(obj) : null;
+ const hls = !!tl && tl.rows.some((r) => Number(r.seq) > 0);
+ if (tl && !hls && !tl.window) {
try { await require('./timeline-queue').queueCmaf(obj.app_id, obj.id); } catch (err) { console.warn(`[Objects] Timeline queue failed for ${obj.id}:`, err.message); }
try { await require('./timeline-queue').queueCmaf(obj.app_id, obj.id); } catch (err) { console.warn(`[Objects] Timeline queue failed for ${obj.id}:`, err.message); }
}
}
if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {
if (obj.visibility !== 'private' && !await db.isSandboxTenant(obj.app_id)) {
@@ -782,6 +784,36 @@ const HLS_TYPE = 'application/vnd.apple.mpegurl';
@@ -782,6 +784,36 @@ const HLS_TYPE = 'application/vnd.apple.mpegurl';
const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;
const SEGMENT_NAME = /^(init\.mp4|\d{6,}\.m4s)$/;
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
+/**
+ * The window of a clip object over its source (F3.5): { sourceId, startMs, endMs } from its `clip_of` edge (metadata
+ * start_time/end_time in seconds), or null — not a clip, no edge, or a source that is not this tenant's ready object.
+ */
+async function clipWindow(obj) {
+ if (!obj || obj.kind !== 'clip') return null;
+ const rel = await db.get(`SELECT to_object_id, metadata FROM media_relationships WHERE from_object_id = ? AND relation = 'clip_of' ORDER BY id LIMIT 1`, [obj.id]);
+ if (!rel) return null;
+ const src = await model.getObject(rel.to_object_id);
+ if (!src || src.app_id !== obj.app_id || src.lifecycle_status !== 'ready') return null;
+ const md = model.parseJson(rel.metadata, {});
+ const startMs = Math.max(0, Math.round(Number(md.start_time) * 1000) || 0);
+ const endMs = Math.round(Number(md.end_time) * 1000) || 0;
+ return endMs > startMs ? { sourceId: src.id, startMs, endMs } : null;
+}
+
+/**
+ * The source rendition an object's playlists are written from: its own rows, or — a clip without its own timeline (a
+ * virtual clip, F3.5) — timeline.clipRows over its source's rows, cut to the clip's window. The window is the token's
+ * scope: a clip's signature (checked against the clip's own id) reaches only the names these rows carry.
+ */
+async function timelineOf(obj) {
+ const timeline = require('./timeline');
+ const own = await timeline.list(obj.id, timeline.SOURCE);
+ if (own.some((r) => Number(r.seq) > 0)) return { rows: own, window: null };
+ const window = await clipWindow(obj);
+ if (!window) return { rows: [], window: null };
+ return { rows: timeline.clipRows(await timeline.list(window.sourceId, timeline.SOURCE), window.startMs, window.endMs), window };
+}
+
/** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */
/** The object of an HLS request, or null when answered (or passed on to the 404 while the flag is off). */
async function hlsObject(req, res, next) {
async function hlsObject(req, res, next) {
if (!config.hls.enabled) { next(); return null; }
if (!config.hls.enabled) { next(); return null; }
@@ -814,14 +846,14 @@ const hlsRoute = (fn) => async (req, res, next) => {
@@ -814,14 +846,14 @@ const hlsRoute = (fn) => async (req, res, next) => {
publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {
publicRouter.get('/:id/master.m3u8', hlsRoute(async (req, res, { obj, query }) => {
const timeline = require('./timeline');
const timeline = require('./timeline');
- const rows = await timeline.list(obj.id, timeline.SOURCE);
+ const { rows } = await timelineOf(obj);
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));
res.type(HLS_TYPE).send(timeline.masterPlaylist([{ name: timeline.SOURCE, rows }], { query }));
}));
}));
publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {
publicRouter.get('/:id/source/index.m3u8', hlsRoute(async (req, res, { obj, query }) => {
const timeline = require('./timeline');
const timeline = require('./timeline');
- const rows = await timeline.list(obj.id, timeline.SOURCE);
+ const { rows } = await timelineOf(obj);
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
if (!rows.some((r) => Number(r.seq) > 0)) return res.status(404).json({ error: 'No timeline' });
res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));
res.type(HLS_TYPE).send(timeline.mediaPlaylist(rows, { query }));
}));
}));
@@ -873,7 +905,10 @@ async function sendSlice(req, res, { file, url, offset, length, headers }) {
@@ -873,7 +905,10 @@ async function sendSlice(req, res, { file, url, offset, length, headers }) {
publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed }) => {
publicRouter.get('/:id/source/:name', hlsRoute(async (req, res, { obj, closed }) => {
const timeline = require('./timeline');
const timeline = require('./timeline');
const name = String(req.params.name || '');
const name = String(req.params.name || '');
- const row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;
+ let row = SEGMENT_NAME.test(name) ? await timeline.byName(obj.id, timeline.SOURCE, name) : null;
+ // A virtual clip serves only the source segments inside its window (timelineOf): any other name is a 404, whatever
+ // the signature, so a clip's token never reaches the rest of its source.
+ if (!row && SEGMENT_NAME.test(name) && obj.kind === 'clip') row = (await timelineOf(obj)).rows.find((r) => r.name === name) || null;
if (!row) return res.status(404).json({ error: 'Not found' });
if (!row) return res.status(404).json({ error: 'Not found' });
const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';
const mime = Number(row.seq) === 0 ? 'video/mp4' : 'video/iso.segment';
const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };
const headers = { 'Content-Type': mime, 'Cache-Control': closed ? 'private, no-store' : 'public, max-age=3600' };
@@ -255,8 +255,26 @@ function masterPlaylist(renditions, { query = '' } = {}) {
@@ -255,8 +255,26 @@ function masterPlaylist(renditions, { query = '' } = {}) {
return lines.join('\n');
return lines.join('\n');
}
}
+// ── Clips (docs/media-fabric.md §4, F3.5) ───────────────────
+/**
+ * A virtual clip is a manifest over its source's rows: the media segments intersecting [startMs, endMs), the source's
+ * init segment, seq renumbered 1..n, each row's times clipped to the window (and made relative to its start). key,
+ * local_path, durable_provider, packed_object_id and byte_* stay the source's, so a segment serves from the source's
+ * bytes (or its packed chunk), and its name, the playlist URI, never changes. A segment straddling an edge is served
+ * whole: a virtual clip's edges are the source's segment boundaries (a materialized clip cuts exactly).
+ */
+function clipRows(rows, startMs, endMs) {
+ const init = rows.find((r) => Number(r.seq) === 0);
+ const segs = rows
+ .filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs)
+ .map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs }))
+ .filter((r) => r.end_ms > r.start_ms)
+ .map((r, i) => ({ ...r, seq: i + 1 }));
+ return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...segs] : segs;
+}
+
module.exports = {
module.exports = {
SOURCE, INIT_NAME, FIELDS, localRoot, localPathFor, keyFor, segmentName,
SOURCE, INIT_NAME, FIELDS, localRoot, localPathFor, keyFor, segmentName,
list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, deleteBytes, removeObject, markPacked, durableProvider, locationsOf,
list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, deleteBytes, removeObject, markPacked, durableProvider, locationsOf,
- mediaPlaylist, masterPlaylist,
+ mediaPlaylist, masterPlaylist, clipRows,
};
};
@@ -157,6 +157,7 @@ async function runOnce({ batch = config.verify.batch, head = defaultHead, upload
@@ -157,6 +157,7 @@ async function runOnce({ batch = config.verify.batch, head = defaultHead, upload
const n = Math.max(1, Number(batch) || 1);
const n = Math.max(1, Number(batch) || 1);
const objects = await db.all(`SELECT o.* FROM media_objects o LEFT JOIN media_verifications v ON v.object_id = o.id
const objects = await db.all(`SELECT o.* FROM media_objects o LEFT JOIN media_verifications v ON v.object_id = o.id
WHERE o.lifecycle_status = 'ready'
WHERE o.lifecycle_status = 'ready'
+ AND NOT (o.kind = 'clip' AND (CASE WHEN json_valid(o.metadata) THEN json_extract(o.metadata, '$.virtual') END) IS NOT NULL)
ORDER BY (v.verified_at IS NOT NULL), v.verified_at, o.id LIMIT ?`, [n]);
ORDER BY (v.verified_at IS NOT NULL), v.verified_at, o.id LIMIT ?`, [n]);
const budget = { left: Math.max(0, Number(maxReuploads) || 0) };
const budget = { left: Math.max(0, Number(maxReuploads) || 0) };
for (const obj of objects) {
for (const obj of objects) {
@@ -137,6 +137,20 @@ async function serveMediaRecord(kind, record, req, res) {
@@ -137,6 +137,20 @@ async function serveMediaRecord(kind, record, req, res) {
// recordings exist (and, through the basename form of /v, which file names are real).
// recordings exist (and, through the basename form of /v, which file names are real).
if (visibility === 'private' && !canAccessPrivate(record, req, kind)) return notFound(res);
if (visibility === 'private' && !canAccessPrivate(record, req, kind)) return notFound(res);
+ // A virtual clip (F3.5) has no file: its bytes are its HLS playlist over the source's timeline. Every reader allowed
+ // here (public, the owner, or a 'getc' URL from GET /api/clips/:id/signed-url), a browser navigation included, is sent
+ // to the master playlist, signed for a closed clip with the clip's own 'hls' token, which reaches only the segments
+ // inside the clip's window. Before the watch page: its player and download link name ?raw=1 bytes it does not have.
+ if (kind === 'clip' && !record.file_path && record.storage_provider === 'timeline') {
+ const config = require('../config');
+ if (drill.refuseBytes(res)) return;
+ if (!config.hls.enabled || !record.object_id || record.status !== 'ready') return res.status(404).json({ error: 'Media file unavailable' });
+ await trackUniqueView(kind, record.id, req, record.user_id);
+ const closed = visibility === 'private' || await db.isSandboxTenant(record.app_id);
+ res.set({ 'Cache-Control': 'private, max-age=0', 'X-Robots-Tag': 'noindex' });
+ return res.redirect(302, closed ? require('../objects/signing').signedPlaylistUrl(record.object_id).url : `${config.publicUrl}/o/${record.object_id}/master.m3u8`);
+ }
+
// A person (or a link-preview crawler) landing on the URL gets the watch
// A person (or a link-preview crawler) landing on the URL gets the watch
// page; its <video> comes back here with ?raw=1 for the bytes. The page offers a player only when
// page; its <video> comes back here with ?raw=1 for the bytes. The page offers a player only when
// the object is playable (server/objects/readiness.js: finished, and a copy whose bytes were
// the object is playable (server/objects/readiness.js: finished, and a copy whose bytes were
@@ -244,7 +258,7 @@ router.get('/c/:id', optionalIdentity, async (req, res) => {
@@ -244,7 +258,7 @@ router.get('/c/:id', optionalIdentity, async (req, res) => {
const clip = await db.getClipById(parseInt(req.params.id, 10));
const clip = await db.getClipById(parseInt(req.params.id, 10));
// A clip still being cut (or whose cut failed) has no file yet: its bytes are a 404, and a
// A clip still being cut (or whose cut failed) has no file yet: its bytes are a 404, and a
// browser navigation gets the watch page saying so (private clips still look missing).
// browser navigation gets the watch page saying so (private clips still look missing).
- if (!clip || (!clip.file_path && !pages.wantsHtmlPage(req))) return notFound(res);
+ if (!clip || (!clip.file_path && clip.storage_provider !== 'timeline' && !pages.wantsHtmlPage(req))) return notFound(res);
await serveMediaRecord('clip', clip, req, res);
await serveMediaRecord('clip', clip, req, res);
} catch (err) {
} catch (err) {
console.error('[Public] /c error:', err.message);
console.error('[Public] /c error:', err.message);
@@ -9,8 +9,13 @@
@@ -9,8 +9,13 @@
* cut and every re-cut run as a job, so they carry media.job.* events and a
* cut and every re-cut run as a job, so they carry media.job.* events and a
* job id the UI can reattach to (GET /api/v2/:app/jobs/:id). The queue
* job id the UI can reattach to (GET /api/v2/:app/jobs/:id). The queue
* owns the retries (backoff 2, 10, 30 min; a hopeless source fails at once).
* owns the retries (backoff 2, 10, 30 min; a hopeless source fails at once).
+ * timelineWindow / makeVirtual a clip over a source that has a CMAF timeline (F3.5) is virtual: no cut, no
+ * file; ready at once (storage_provider 'timeline') and played as its window of
+ * the source's segments. clip.cut is the materialized path (asked for, or no
+ * timeline); materializing a virtual clip keeps it playing until the file is in.
* start() sweeper: every 3 min, failed clips that still have
* start() sweeper: every 3 min, failed clips that still have
- * attempts left are re-cut. From the second attempt on, a
+ * attempts left are re-cut (or turned virtual, as a recut is,
+ * when the source now has a timeline). From the second attempt on, a
* VOD that lives in cloud storage is first pulled back to
* VOD that lives in cloud storage is first pulled back to
* local disk (moveToHot) — cutting a multi-hour recording
* local disk (moveToHot) — cutting a multi-hour recording
* over HTTP is what used to time out — as long as there is
* over HTTP is what used to time out — as long as there is
@@ -62,8 +67,9 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {
@@ -62,8 +67,9 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {
return { ok: false, error: 'Source VOD no longer exists' };
return { ok: false, error: 'Source VOD no longer exists' };
}
}
const attempt = (Number(clip.cut_attempts) || 0) + 1;
const attempt = (Number(clip.cut_attempts) || 0) + 1;
- // Row and object (back to uploading) together.
- await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'processing', cut_attempts = ?, cut_next_at = NULL WHERE id = ?", [attempt, clipId]));
+ // Row and object (back to uploading) together. A virtual clip being materialized stays ready and playing meanwhile.
+ if (isVirtual(clip)) await db.run('UPDATE clips SET cut_attempts = ?, cut_next_at = NULL WHERE id = ?', [attempt, clipId]);
+ else await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'processing', cut_attempts = ?, cut_next_at = NULL WHERE id = ?", [attempt, clipId]));
// Attempt 2+: bring a cloud-stored VOD home first when the disk can take it.
// Attempt 2+: bring a cloud-stored VOD home first when the disk can take it.
let source = await vodStorage.resolveMediaSource(vod);
let source = await vodStorage.resolveMediaSource(vod);
@@ -88,7 +94,8 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {
@@ -88,7 +94,8 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {
// object and its event in one transaction (webhooks.announce), then the webhook.
// object and its event in one transaction (webhooks.announce), then the webhook.
try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }
try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }
await announce(clip.app_id, 'clip.ready', {
await announce(clip.app_id, 'clip.ready', {
- change: async () => await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET file_path = ?, duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL WHERE id = ?",
+ change: async () => await objects().withObject('clip', clipId, async () => await db.run(`UPDATE clips SET file_path = ?, duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL,
+ storage_provider = CASE WHEN storage_provider = 'timeline' THEN 'local' ELSE storage_provider END WHERE id = ?`,
[cut.filePath, cut.duration, startTime + cut.duration, clipId])),
[cut.filePath, cut.duration, startTime + cut.duration, clipId])),
payload: async () => await _clipPublic(await db.getClipById(clipId)),
payload: async () => await _clipPublic(await db.getClipById(clipId)),
});
});
@@ -105,6 +112,12 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {
@@ -105,6 +112,12 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {
const more = !viaJob && attempt < MAX_ATTEMPTS && !hopeless;
const more = !viaJob && attempt < MAX_ATTEMPTS && !hopeless;
const mins = BACKOFF_MIN[Math.min(attempt - 1, BACKOFF_MIN.length - 1)];
const mins = BACKOFF_MIN[Math.min(attempt - 1, BACKOFF_MIN.length - 1)];
const nextAt = more ? new Date(Date.now() + mins * 60000).toISOString().replace('T', ' ').slice(0, 19) : null;
const nextAt = more ? new Date(Date.now() + mins * 60000).toISOString().replace('T', ' ').slice(0, 19) : null;
+ // A virtual clip whose materialization failed still plays from the timeline: it keeps its status and says why.
+ if (isVirtual(clip)) {
+ await db.run('UPDATE clips SET cut_error = ?, cut_next_at = NULL WHERE id = ?', [msg, clipId]);
+ console.warn(`[Clips] Clip ${clipId} materialization attempt ${attempt} failed: ${msg} — it stays virtual`);
+ return { ok: false, error: msg, retry_at: null, hopeless };
+ }
await announce(clip.app_id, 'clip.failed', {
await announce(clip.app_id, 'clip.failed', {
change: async () => await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'failed', cut_error = ?, cut_next_at = ? WHERE id = ?", [msg, nextAt, clipId])),
change: async () => await objects().withObject('clip', clipId, async () => await db.run("UPDATE clips SET status = 'failed', cut_error = ?, cut_next_at = ? WHERE id = ?", [msg, nextAt, clipId])),
payload: async () => await _clipPublic(await db.getClipById(clipId)),
payload: async () => await _clipPublic(await db.getClipById(clipId)),
@@ -113,6 +126,51 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {
@@ -113,6 +126,51 @@ async function fail(clipId, clip, attempt, error, { viaJob = false } = {}) {
return { ok: false, error: msg, retry_at: nextAt, hopeless };
return { ok: false, error: msg, retry_at: nextAt, hopeless };
}
}
+// ── Virtual clips (docs/media-fabric.md §4, F3.5) ────────────
+
+function isVirtual(clip) { return !!clip && !clip.file_path && clip.storage_provider === 'timeline' && clip.status === 'ready'; }
+
+/**
+ * The window [startTime, endTime) seconds over the source timeline of a VOD → { objectId, endTime } (endTime clamped to
+ * the timeline's end), or null: MEDIA_HLS_ENABLED off (nothing would play it), no ready object, no timeline yet, or no
+ * segment inside the window. Null means the clip is cut (clip.cut).
+ */
+async function timelineWindow(vod, startTime, endTime) {
+ if (!config.hls.enabled || !vod || !vod.object_id || vod.is_recording) return null;
+ const obj = await objects().getObject(vod.object_id);
+ if (!obj || obj.lifecycle_status !== 'ready') return null;
+ const timeline = require('../objects/timeline');
+ const rows = timeline.clipRows(await timeline.list(obj.id, timeline.SOURCE), Math.round(startTime * 1000), Math.round(endTime * 1000));
+ if (!rows.some((r) => Number(r.seq) > 0)) return null;
+ const last = rows[rows.length - 1];
+ return { objectId: obj.id, endTime: startTime + Number(last.end_ms) / 1000 };
+}
+
+/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction, then the webhook. */
+async function makeVirtual(clipId, win) {
+ const clip = await db.getClipById(clipId);
+ if (!clip) return { ok: false, error: 'Clip not found' };
+ const startTime = Number(clip.start_time) || 0;
+ const endTime = Math.min(Number(clip.end_time) || 0, win.endTime);
+ await announce(clip.app_id, 'clip.ready', {
+ change: async () => await objects().withObject('clip', clipId, async () => await db.run(`UPDATE clips SET file_path = NULL, storage_provider = 'timeline',
+ duration_seconds = ?, end_time = ?, status = 'ready', cut_error = NULL, cut_next_at = NULL WHERE id = ?`, [endTime - startTime, endTime, clipId])),
+ payload: async () => await _clipPublic(await db.getClipById(clipId)),
+ });
+ console.log(`[Clips] Clip ${clipId} virtual over vod ${clip.vod_id}'s timeline (${startTime.toFixed(1)}-${endTime.toFixed(1)}s)`);
+ return { ok: true, duration_seconds: endTime - startTime };
+}
+
+/** A file-less clip over a source with a timeline → made virtual (true); otherwise false and it is cut. */
+async function retryVirtual(clipId) {
+ const clip = await db.getClipById(clipId);
+ if (!clip || clip.file_path || !clip.vod_id) return false;
+ const win = await timelineWindow(await db.getVodById(clip.vod_id), Number(clip.start_time) || 0, Number(clip.end_time) || 0);
+ if (!win) return false;
+ await makeVirtual(clipId, win);
+ return true;
+}
+
async function sweep() {
async function sweep() {
if (_busy) return;
if (_busy) return;
_busy = true;
_busy = true;
@@ -122,7 +180,12 @@ async function sweep() {
@@ -122,7 +180,12 @@ async function sweep() {
AND created_at >= datetime('now', '-30 days')
AND created_at >= datetime('now', '-30 days')
AND NOT EXISTS (SELECT 1 FROM media_jobs j WHERE j.job_type = 'clip.cut' AND j.app_id = clips.app_id AND CAST(json_extract(j.params, '$.clip_id') AS INTEGER) = clips.id)
AND NOT EXISTS (SELECT 1 FROM media_jobs j WHERE j.job_type = 'clip.cut' AND j.app_id = clips.app_id AND CAST(json_extract(j.params, '$.clip_id') AS INTEGER) = clips.id)
ORDER BY created_at DESC LIMIT 4`, [MAX_ATTEMPTS]) || [];
ORDER BY created_at DESC LIMIT 4`, [MAX_ATTEMPTS]) || [];
- for (const row of due) { try { await recutClip(row.id, { reason: 'auto-retry' }); } catch (e) { console.warn(`[Clips] auto-retry ${row.id}:`, e.message); } }
+ for (const row of due) {
+ try {
+ // The same recovery as POST /api/clips/:id/recut: a failed clip whose source now has a timeline turns virtual.
+ if (!(await retryVirtual(row.id))) await recutClip(row.id, { reason: 'auto-retry' });
+ } catch (e) { console.warn(`[Clips] auto-retry ${row.id}:`, e.message); }
+ }
} finally { _busy = false; }
} finally { _busy = false; }
}
}
@@ -181,4 +244,4 @@ const spec = {
@@ -181,4 +244,4 @@ const spec = {
},
},
};
};
-module.exports = { recutClip, sweep, start, enqueueCut, clipObjectId, spec, MAX_ATTEMPTS };
+module.exports = { recutClip, sweep, start, enqueueCut, clipObjectId, timelineWindow, makeVirtual, isVirtual, spec, MAX_ATTEMPTS };
@@ -133,7 +133,9 @@ const VALID_VIS = new Set(['public', 'unlisted', 'private']);
@@ -133,7 +133,9 @@ const VALID_VIS = new Set(['public', 'unlisted', 'private']);
const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].includes(String(v)) ? 0 : null);
const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].includes(String(v)) ? 0 : null);
// ── Create clip ──────────────────────────────────────────────
// ── Create clip ──────────────────────────────────────────────
-// JSON body: cut a window out of a VOD (202, background cut + webhook).
+// JSON body: a window of a VOD. Over a source with a CMAF timeline (MEDIA_HLS_ENABLED) the clip is virtual: ready at
+// once (201, clip.ready webhook), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).
+// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut + webhook).
// Multipart `video`: direct upload of an already-cut blob (201, ready) —
// Multipart `video`: direct upload of an already-cut blob (201, ready) —
// inherited browser-MediaRecorder clip path.
// inherited browser-MediaRecorder clip path.
router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {
router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {
@@ -170,14 +172,16 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
@@ -170,14 +172,16 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
return res.status(200).json({ id: duplicate.id, status: duplicate.status, deduplicated: true });
return res.status(200).json({ id: duplicate.id, status: duplicate.status, deduplicated: true });
}
}
- // Resolve a source ffmpeg can read: local file, or presigned B2/R2 URL.
+ const clipJobs = require('./clip-jobs');
+ const win = flag(body.materialize) !== 1 && await clipJobs.timelineWindow(vod, startTime, endTime);
+ // Resolve a source ffmpeg can read: local file, or presigned B2/R2 URL (a virtual clip reads no file).
const vodStorage = require('./vod-storage');
const vodStorage = require('./vod-storage');
- const source = await vodStorage.resolveMediaSource(vod);
- if (!source) return res.status(404).json({ error: 'VOD media unavailable' });
+ const source = win ? null : await vodStorage.resolveMediaSource(vod);
+ if (!win && !source) return res.status(404).json({ error: 'VOD media unavailable' });
// Live recording: clamp to what's ACTUALLY on disk, not wall-clock — a
// Live recording: clamp to what's ACTUALLY on disk, not wall-clock — a
// seek past end-of-file emits a header-only "broken clip".
// seek past end-of-file emits a header-only "broken clip".
- if (vod.is_recording && source.kind === 'file') {
+ if (vod.is_recording && source && source.kind === 'file') {
let recordedDur = 0;
let recordedDur = 0;
try { const pi = await tools.probeVodInfo(source.value); recordedDur = pi.duration || 0; } catch { /* */ }
try { const pi = await tools.probeVodInfo(source.value); recordedDur = pi.duration || 0; } catch { /* */ }
if (recordedDur > 0) {
if (recordedDur > 0) {
@@ -221,11 +225,15 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
@@ -221,11 +225,15 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
}); // the row and its object (uploading) in one transaction
}); // the row and its object (uploading) in one transaction
const clipId = result.lastInsertRowid;
const clipId = result.lastInsertRowid;
const appId = req.appId;
const appId = req.appId;
+ if (win) {
+ await clipJobs.makeVirtual(clipId, win);
+ return res.status(201).json({ ...await clipPublic(await db.getClipById(clipId, appId), { readiness: true }) });
+ }
// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a
// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a
// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.
// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.
// clip.ready / clip.failed and the webhook come from the cut itself, as before.
// clip.ready / clip.failed and the webhook come from the cut itself, as before.
- const { job } = await require('./clip-jobs').enqueueCut(appId, clipId, {
+ const { job } = await clipJobs.enqueueCut(appId, clipId, {
reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,
reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,
});
});
@@ -360,6 +368,8 @@ router.get('/:id', tenantAuth({ verb: 'read', allowUser: true }), async (req, re
@@ -360,6 +368,8 @@ router.get('/:id', tenantAuth({ verb: 'read', allowUser: true }), async (req, re
* (vod_id, start_time, end_time), so a failure is recoverable rather than
* (vod_id, start_time, end_time), so a failure is recoverable rather than
* permanent — previously a failed clip stayed broken forever with no retry path,
* permanent — previously a failed clip stayed broken forever with no retry path,
* and the viewer was shown "the server is cutting your clip" indefinitely.
* and the viewer was shown "the server is cutting your clip" indefinitely.
+ * A clip without a file, over a source with a timeline, becomes virtual instead (200, ready); `materialize: true` (body or query)
+ * always cuts its own file (clip.cut), and a virtual clip keeps playing while it is cut.
*/
*/
router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
try {
try {
@@ -368,10 +378,21 @@ router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
@@ -368,10 +378,21 @@ router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
const clipId = clip.id;
const clipId = clip.id;
if (clip.status === 'processing') return res.status(409).json({ error: 'Clip is already being cut' });
if (clip.status === 'processing') return res.status(409).json({ error: 'Clip is already being cut' });
if (!clip.vod_id) return res.status(422).json({ error: 'Clip has no source VOD to re-cut from' });
if (!clip.vod_id) return res.status(422).json({ error: 'Clip has no source VOD to re-cut from' });
+ const clipJobs = require('./clip-jobs');
+ const materialize = flag((req.body || {}).materialize ?? req.query.materialize) === 1;
+ // A clip that has a file of its own keeps it: only a file-less one (failed, or already virtual) turns virtual.
+ if (!materialize && !clip.file_path) {
+ const vod = await db.getVodById(clip.vod_id, req.appId);
+ const win = await clipJobs.timelineWindow(vod, Number(clip.start_time) || 0, Number(clip.end_time) || 0);
+ if (win) {
+ await clipJobs.makeVirtual(clipId, win);
+ return res.json({ ...await clipPublic(await db.getClipById(clipId, req.appId), { readiness: true }) });
+ }
+ }
// A manual retry resets the attempt counter so it gets the full ladder again (not projected: no object change).
// A manual retry resets the attempt counter so it gets the full ladder again (not projected: no object change).
await db.run("UPDATE clips SET cut_attempts = 0 WHERE id = ?", [clipId]);
await db.run("UPDATE clips SET cut_attempts = 0 WHERE id = ?", [clipId]);
- const { job } = await require('./clip-jobs').enqueueCut(req.appId, clipId, { reason: 're-cut', createdBy: `app:${req.appId}` });
- res.status(202).json({ id: clipId, status: 'processing', job_id: job.id });
+ const { job } = await clipJobs.enqueueCut(req.appId, clipId, { reason: materialize ? 'materialize' : 're-cut', createdBy: `app:${req.appId}` });
+ res.status(202).json({ id: clipId, status: clipJobs.isVirtual(clip) ? 'ready' : 'processing', job_id: job.id });
} catch (err) {
} catch (err) {
console.error('[Clips] Re-cut error:', err.message);
console.error('[Clips] Re-cut error:', err.message);
res.status(500).json({ error: 'Failed to re-cut clip' });
res.status(500).json({ error: 'Failed to re-cut clip' });
new file mode 100644
new file mode 100644
@@ -0,0 +1,228 @@
@@ -0,0 +1,228 @@
+'use strict';
+// Clips over the timeline (F3.5; docs/media-fabric.md §4): timeline.clipRows cuts a source's rows to a window (edges,
+// empty window, init segment); a clip over a VOD with a timeline is virtual — ready at once, no clip.cut — and plays as
+// HLS over the source's segments through the hlsObject() rules; its playlist token reaches only the segments inside its
+// window (never the rest of the source, never GET /o/:id); /c/:id sends a reader to that playlist. materialize still
+// queues clip.cut, a file clip keeps serving its file, and with MEDIA_HLS_ENABLED off a clip is cut as before.
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const http = require('http');
+const express = require('express');
+
+(async () => {
+ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-clip-timeline-'));
+ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };
+ Object.assign(process.env, {
+ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'), THUMBNAILS_PATH: dir('thumbnails'),
+ PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), MEDIA_PUBLIC_URL: 'https://media.test',
+ MEDIA_HLS_ENABLED: '1', MEDIA_INVARIANT_SCAN_HOURS: '0',
+ });
+
+ const config = require('../server/config');
+ const db = require('../server/db/database');
+ const model = require('../server/objects/model');
+ const timeline = require('../server/objects/timeline');
+ const signing = require('../server/objects/signing');
+ const tools = require('../server/vod/media-tools');
+ const thumbService = require('../server/thumbnails/thumbnail-service');
+ const vodStorage = require('../server/vod/vod-storage');
+ await vodStorage.tierConfig.init(vodStorage.DEFAULTS);
+ await require('../server/objects/tier-policy').init();
+ const { finalizeVod } = require('../server/vod/finalize');
+ await db.upsertApp({ app_id: 'live', api_key: 'live-key-clip-timeline' });
+
+ // No ffmpeg in these paths; the job worker is never started, so queued jobs stay queued.
+ thumbService.generateVodThumbnail = async () => null;
+ tools.remuxForSeekingDetailed = async () => ({ ok: true, seconds: 30, error: null });
+ tools.probeDuration = async () => ({ ok: true, seconds: 30, format: {}, streams: [], error: null });
+ tools.remuxForSeeking = async () => true;
+ tools.probeVodInfo = async () => ({ duration: 30 });
+
+ // ── clipRows: the window's edges ──
+ const row = (seq, start, end, extra = {}) => ({ seq, name: timeline.segmentName(seq), start_ms: start, end_ms: end, keyframe_ms: seq ? start : null,
+ key: `src/source/${timeline.segmentName(seq)}`, local_path: null, durable_provider: null, packed_object_id: null, byte_offset: null,
+ byte_length: 1000, sha256: String(seq).repeat(64).slice(0, 64), durability: 'local', ...extra });
+ const src = [row(0, 0, 0), row(1, 0, 4000), row(2, 4000, 8000), row(3, 8000, 12000, { packed_object_id: 'p'.repeat(64), byte_offset: 5, byte_length: 10 }),
+ row(4, 12000, 16000), row(5, 16000, 18500)];
+ let c = timeline.clipRows(src, 5000, 11000);
+ assert.deepStrictEqual(c.map((r) => [r.seq, r.name, r.start_ms, r.end_ms]), [[0, 'init.mp4', 0, 0], [1, '000002.m4s', 0, 3000], [2, '000003.m4s', 3000, 6000]],
+ 'segments straddling start and end are in, renumbered, times clipped and relative to the window');
+ assert.deepStrictEqual([c[2].key, c[2].packed_object_id, c[2].byte_offset, c[2].byte_length, c[2].sha256], [src[3].key, src[3].packed_object_id, 5, 10, src[3].sha256],
+ 'location and bytes stay the source\'s');
+ assert.deepStrictEqual(timeline.clipRows(src, 4000, 8000).map((r) => r.name), ['init.mp4', '000002.m4s'], 'a window on segment boundaries takes exactly those');
+ assert.deepStrictEqual(timeline.clipRows(src, 17000, 60000).map((r) => [r.name, r.end_ms]), [['init.mp4', 0], ['000005.m4s', 1500]], 'the tail stops at the source\'s end');
+ assert.deepStrictEqual(timeline.clipRows(src, 30000, 40000), [], 'an empty window has no rows, not even the init segment');
+ assert.deepStrictEqual(timeline.clipRows(src, 4000, 4000), [], 'a zero-length window too');
+ assert.strictEqual(src[2].seq, 2, 'the source rows are not changed');
+ const mp = timeline.mediaPlaylist(c, { query: 'exp=1&sig=x' });
+ assert.ok(mp.includes('#EXT-X-MEDIA-SEQUENCE:1') && mp.includes('#EXTINF:3.000,\n000002.m4s?exp=1&sig=x') && !mp.includes('000001.m4s'), mp);
+ console.log('✅ clipRows: straddling edges, boundaries, tail, empty window, init segment');
+
+ const app = express();
+ app.use(express.json());
+ app.use('/api/v1/:app/vods', require('../server/vod/routes'));
+ app.use('/api/v1/:app/clips', require('../server/vod/clips-routes'));
+ app.use('/api/v2/:app/objects', require('../server/objects/routes'));
+ app.use('/o', require('../server/objects/routes').publicRouter);
+ app.use('/', require('../server/public/routes'));
+ const server = http.createServer(app);
+
+ (async () => {
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ const api = `http://127.0.0.1:${server.address().port}`;
+ const auth = { authorization: 'Bearer live-key-clip-timeline' };
+ const call = async (method, p, body) => {
+ const headers = { ...auth };
+ if (body && !(body instanceof FormData)) headers['content-type'] = 'application/json';
+ const r = await fetch(api + p, { method, headers, body: body && !(body instanceof FormData) ? JSON.stringify(body) : body, redirect: 'manual' });
+ return { status: r.status, body: await r.json().catch(() => null) };
+ };
+ const get = async (url, headers = {}) => {
+ const u = new URL(url, api);
+ const r = await fetch(api + u.pathname + u.search, { headers, redirect: 'manual' });
+ return { status: r.status, location: r.headers.get('location'), buf: Buffer.from(await r.arrayBuffer()) };
+ };
+ const jobs = async (type, where = '') => await db.all(`SELECT * FROM media_jobs WHERE job_type = ? ${where}`, [type]);
+ const clipCuts = async (clipId) => (await jobs('clip.cut')).filter((j) => Number(model.parseJson(j.params, {}).clip_id) === Number(clipId));
+
+ // A private, finished VOD with a source timeline on this node's disk (segment 3 packed in a chunk).
+ let r = await call('POST', '/api/v1/live/vods', { title: 'Source', user_id: 5, visibility: 'private' });
+ assert.strictEqual(r.status, 201);
+ const vodId = r.body.id;
+ const f = new FormData();
+ f.append('chunk', new Blob([Buffer.alloc(40000, 1)], { type: 'video/webm' }), 'a.webm');
+ assert.strictEqual((await call('POST', `/api/v1/live/vods/${vodId}/chunks`, f)).body.status, 'created');
+ const vodObj = (await finalizeVod(vodId)).object_id;
+ const vodRow = await model.getObject(vodObj);
+ const bytes = {};
+ const rows = src.map((s) => {
+ const p = timeline.localPathFor(vodRow, 'source', s.name, 'v1');
+ fs.mkdirSync(path.dirname(p), { recursive: true });
+ bytes[s.name] = Buffer.from(`${s.name}:`.padEnd(64, String(s.seq)));
+ if (s.packed_object_id) {
+ fs.writeFileSync(p, Buffer.concat([Buffer.from('HEAD!'), bytes[s.name].subarray(0, 10), Buffer.from('TAIL')]));
+ bytes[s.name] = bytes[s.name].subarray(0, 10);
+ } else fs.writeFileSync(p, bytes[s.name]);
+ return { ...s, local_path: p, byte_length: s.packed_object_id ? 10 : bytes[s.name].length };
+ });
+ await timeline.replace(vodObj, 'source', rows);
+
+ // ── A clip over it is virtual: ready at once, no clip.cut ──
+ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 5, end_s: 11, visibility: 'private', title: 'Virtual' });
+ assert.strictEqual(r.status, 201, JSON.stringify(r.body));
+ assert.deepStrictEqual([r.body.status, r.body.storage_provider, r.body.file_path, r.body.start_time, r.body.end_time, r.body.duration_seconds],
+ ['ready', 'timeline', null, 5, 11, 6]);
+ assert.strictEqual(r.body.playback_url, `${config.publicUrl}/c/${r.body.id}`, 'playback_url unchanged');
+ const clipId = r.body.id;
+ const clipObj = (await db.getClipById(clipId)).object_id;
+ const co = await model.getObject(clipObj);
+ assert.deepStrictEqual([co.kind, co.lifecycle_status, co.visibility], ['clip', 'ready', 'private']);
+ assert.strictEqual((await clipCuts(clipId)).length, 0, 'no clip.cut for a virtual clip');
+ assert.deepStrictEqual([r.body.readiness.playable, r.body.readiness.bytes_verified, r.body.readiness.reason], [true, true, null], 'a virtual clip is playable');
+ assert.strictEqual(model.parseJson(co.metadata, {}).virtual, true);
+ const readiness = require('../server/objects/readiness');
+ assert.ok(await db.get(`SELECT 1 FROM clips WHERE id = ? AND ${readiness.playableSql('clips.object_id')}`, [clipId]), 'playableSql agrees');
+ const verified = await require('../server/objects/verify-job').runOnce({ batch: 50, head: async () => undefined, upload: async () => { throw new Error('no upload in this test'); }, maxReuploads: 0 });
+ assert.ok(!verified.objects.some((o) => o.object_id === clipObj), 'the verify job leaves a virtual clip alone');
+ assert.ok(!(await db.get('SELECT 1 FROM media_verifications WHERE object_id = ?', [clipObj])), 'and reports no no_good_copy for it');
+ console.log('✅ POST /clips over a timeline: ready at once (201), storage_provider timeline, playable, no clip.cut, not verified');
+
+ // ── Its playlists: hlsObject() rules, the source's segments inside the window ──
+ for (const p of ['master.m3u8', 'source/index.m3u8', 'source/init.mp4', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${clipObj}/${p}`)).status, 404, `unsigned ${p}`);
+ const tok = new URL(signing.signedPlaylistUrl(clipObj).url).search;
+ const master = await get(`/o/${clipObj}/master.m3u8${tok}`);
+ assert.strictEqual(master.status, 200);
+ assert.ok(master.buf.toString().includes(`source/index.m3u8${tok}`));
+ const index = (await get(`/o/${clipObj}/source/index.m3u8${tok}`)).buf.toString();
+ assert.ok(index.includes(`#EXT-X-MAP:URI="init.mp4${tok}"`) && index.includes(`#EXTINF:3.000,\n000002.m4s${tok}`) && index.includes(`#EXTINF:3.000,\n000003.m4s${tok}`), index);
+ assert.ok(!/00000[145]\.m4s/.test(index), 'nothing outside the window is listed');
+ for (const n of ['init.mp4', '000002.m4s', '000003.m4s']) {
+ const s = await get(`/o/${clipObj}/source/${n}${tok}`);
+ assert.strictEqual(s.status, 200, n);
+ assert.deepStrictEqual(s.buf, bytes[n], `${n}: the source's bytes${n === '000003.m4s' ? ' (a slice of its packed chunk)' : ''}`);
+ }
+ console.log('✅ a private virtual clip: 404 unsigned, signed playlists over the source window, source bytes and packed slices');
+
+ // ── Token scope: the clip's window only ──
+ for (const n of ['000001.m4s', '000004.m4s', '000005.m4s']) assert.strictEqual((await get(`/o/${clipObj}/source/${n}${tok}`)).status, 404, `clip token refused for ${n}`);
+ for (const p of ['master.m3u8', 'source/index.m3u8', 'source/000001.m4s', 'source/000002.m4s']) assert.strictEqual((await get(`/o/${vodObj}/${p}${tok}`)).status, 404, `clip token never opens the source's ${p}`);
+ assert.strictEqual((await get(`/o/${clipObj}${tok}`)).status, 404, 'GET /o/:id refuses a playlist token');
+ const vodTok = new URL(signing.signedPlaylistUrl(vodObj).url).search;
+ assert.strictEqual((await get(`/o/${clipObj}/source/000002.m4s${vodTok}`)).status, 404, 'a source token does not open the clip');
+ assert.strictEqual((await get(`/o/${vodObj}/source/000001.m4s${vodTok}`)).status, 200, 'the source token still plays the source');
+ console.log('✅ a clip token reaches only its window: not the source, not GET /o/:id; a source token is not needed nor accepted');
+
+ // ── Discovery: /download names the playlist (no object.cmaf for the clip); /c/:id redirects to it ──
+ r = await call('GET', `/api/v2/live/objects/${clipObj}/download?format=json`);
+ assert.strictEqual(r.status, 200, JSON.stringify(r.body));
+ const hu = new URL(r.body.hls_url);
+ assert.ok(hu.pathname === `/o/${clipObj}/master.m3u8` && signing.verifyPlaylist(clipObj, hu.searchParams.get('exp'), hu.searchParams.get('sig')));
+ assert.strictEqual((await jobs('object.cmaf', 'AND object_id = ?'.replace('?', `'${clipObj}'`))).length, 0, 'no object.cmaf for a virtual clip');
+ assert.strictEqual((await get(`/c/${clipId}`)).status, 404, '/c/:id of a private clip, unsigned: 404');
+ const cr = await get(new URL(signing.signedMediaUrl('clip', clipId, 600).url).pathname + new URL(signing.signedMediaUrl('clip', clipId, 600).url).search);
+ assert.strictEqual(cr.status, 302);
+ const cu = new URL(cr.location);
+ assert.ok(cu.pathname === `/o/${clipObj}/master.m3u8` && signing.verifyPlaylist(clipObj, cu.searchParams.get('exp'), cu.searchParams.get('sig')), cr.location);
+ const signedC = new URL(signing.signedMediaUrl('clip', clipId, 600).url);
+ for (const h of [{ 'sec-fetch-dest': 'document' }, { accept: 'text/html,application/xhtml+xml' }]) {
+ const nav = await get(signedC.pathname + signedC.search, h);
+ assert.strictEqual(nav.status, 302, `a browser navigation (${JSON.stringify(h)}) is redirected, not shown "not available"`);
+ assert.strictEqual(new URL(nav.location).pathname, `/o/${clipObj}/master.m3u8`);
+ }
+ assert.strictEqual((await get(`/c/${clipId}`, { 'sec-fetch-dest': 'document' })).status, 404, 'unsigned navigation of a private clip: 404');
+ console.log('✅ /download gives hls_url without queueing object.cmaf; /c/:id (signed, any reader) redirects to the clip\'s signed playlist');
+
+ // ── Materialize: still clip.cut, the virtual clip keeps playing meanwhile ──
+ r = await call('POST', `/api/v1/live/clips/${clipId}/recut`, { materialize: true });
+ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
+ assert.ok(r.body.job_id && r.body.status === 'ready');
+ assert.strictEqual((await clipCuts(clipId)).length, 1, 'recut?materialize queues clip.cut');
+ assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 200, 'still playing');
+ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 20, end_s: 23, materialize: true });
+ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
+ assert.ok(r.body.job_id && r.body.status === 'processing');
+ assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'POST materialize queues clip.cut');
+ console.log('✅ materialize (POST and recut) queues clip.cut; a virtual clip plays on while it is cut');
+
+ // ── A re-encoded clip keeps serving its file; its recut stays a cut ──
+ const file = path.join(process.env.CLIPS_PATH, 'old.webm');
+ fs.writeFileSync(file, Buffer.alloc(3000, 7));
+ const old = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Old', file_path: file, start_time: 2, end_time: 6,
+ duration_seconds: 4, is_public: 1, visibility: 'public', status: 'ready' })).lastInsertRowid;
+ const oldObj = await model.getObject((await db.getClipById(old)).object_id);
+ assert.strictEqual(oldObj.lifecycle_status, 'ready');
+ const ob = await get(`/c/${old}?raw=1`);
+ assert.deepStrictEqual([ob.status, ob.buf.length], [200, 3000], 'the file is served as before');
+ r = await call('POST', `/api/v1/live/clips/${old}/recut`, {});
+ assert.strictEqual(r.status, 202);
+ assert.strictEqual((await clipCuts(old)).length, 1, 'a clip with a file is re-cut, not turned virtual');
+ assert.strictEqual((await db.getClipById(old)).file_path, file);
+ console.log('✅ a re-encoded clip keeps serving its file, and its recut is a cut');
+
+ // ── The retry sweeper recovers a failed, file-less clip the way a recut does: virtual over the timeline ──
+ const failed = (await db.createClip({ app_id: 'live', vod_id: vodId, user_id: 5, title: 'Failed', file_path: null, start_time: 8, end_time: 12,
+ duration_seconds: 4, is_public: 1, visibility: 'public', status: 'failed' })).lastInsertRowid;
+ await require('../server/vod/clip-jobs').sweep();
+ const fr = await db.getClipById(failed);
+ assert.deepStrictEqual([fr.status, fr.storage_provider, fr.file_path], ['ready', 'timeline', null]);
+ assert.strictEqual((await clipCuts(failed)).length, 0, 'no cut for it');
+ console.log('✅ the retry sweeper turns a failed clip over a timeline virtual, as a recut does');
+
+ // ── Flag off: cut as before ──
+ config.hls.enabled = false;
+ await db.run("UPDATE clips SET created_at = '2000-01-01 00:00:00'"); // past the duplicate-clip window
+ r = await call('POST', '/api/v1/live/clips', { vod_id: vodId, start_s: 26, end_s: 29 });
+ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
+ assert.strictEqual((await clipCuts(r.body.id)).length, 1, 'MEDIA_HLS_ENABLED off: clip.cut');
+ assert.strictEqual((await get(`/o/${clipObj}/master.m3u8${tok}`)).status, 404, 'and the HLS routes are off');
+ config.hls.enabled = true;
+ console.log('✅ MEDIA_HLS_ENABLED off: clips are cut as before');
+
+ server.close();
+ fs.rmSync(tmp, { recursive: true, force: true });
+ console.log('✅ All clip timeline tests passed');
+ process.exit(0);
+ })().catch((err) => { console.error(err); process.exit(1); });
+})().catch((err) => { console.error(err); process.exit(1); });