openvibers/openvibe.media Commits

Clips can be trimmed in place: POST /clips/:id/trim { start_s, end_s } narrows a ready clip's window of its source and cuts it again (clip.cut, same id and title; a virtual clip plays the narrower window at once); a re-cut removes the superseded file (#153)

OP OpenVibers committed 22 days ago · 62e1e07e0c · 5 files, +191 −1

Changes

ddiff --git a/README.md b/README.md
iindex b57df1e..14fb326 100644
---- a/README.md
++++ b/README.md
@@@ -285,6 +285,7 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background — with `MEDIA_HLS_ENABLED` + `MEDIA_MATERIALIZED_CLIPS=1` and a source timeline it is **materialized** over the source's bytes (the clip's own media_timeline rows copy the interior segments; only the two window edges are re-encoded, docs/materialized-clips.md), otherwise the full re-encode (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
| GET | `/clips/:id` | status: `processing | ready | failed`; with `MEDIA_HLS_ENABLED` also `hls_url` (and `hls_expires_at` when signed) — the clip's playlist over its own timeline (a materialized clip's persisted rows) or, virtual, its source's window (docs/media-fabric.md §4) |
| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) |
++| POST | `/clips/:id/trim` | `{ start_s, end_s }`, seconds into the clip (at least 1 s apart) → the clip narrows in place: same id and title, its window of the source shrinks and it is cut again from the source (**202** `{ id, status: 'processing', job_id }`, `clip.cut`, `clip.ready` again; the superseded file is removed), or a virtual clip plays the narrower window at once (200, the clip); the whole clip → 200 `unchanged: true`; still being cut → 409; no source VOD (a browser upload) or a file moved to cloud storage → 422 |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
| DELETE | `/clips/:id` | local + offloaded objects + row; a timeline clip's own rows and edge bytes go with it, while every location the source still names — a materialized clip's interior — stays (docs/materialized-clips.md) |
ddiff --git a/docs/object-model.md b/docs/object-model.md
iindex 184c22e..b117511 100644
---- a/docs/object-model.md
++++ b/docs/object-model.md
@@@ -702,7 +702,7 @@ job over only with the token it read and, past boot, only while the lease is sti
| `invariant.scan` | light | The [size-invariant validator](#public-object-size-invariant): records violations, proposes `object.split` / `object.remux`, withdraws moot proposals. Tenant-wide (no object) |
| `object.split` | heavy | Stream-copies a vod/clip (or a video/audio object) into parts (`params.parts` 2-1000 or `segment_seconds`), each a new **private** object with a `derived_from` relationship (`job_id`, `part`, `start_seconds`, `duration_seconds`). Cuts land on keyframes, so neighbouring parts can overlap slightly. The source is never changed. Checkpointed per part |
| `object.remux` | heavy | Stream-copy remux of the whole source (seek index, duration, MP4 faststart) into one new private object; also the source's `remux` variant |
--| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips` and `/:id/recut` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` events come from the cut. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording |
++| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips`, `/:id/recut` and `/:id/trim` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` events come from the cut. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording |
| `object.waveform` | heavy | The audio waveform as a PNG (`showwavespic`, 1800×140 by default; `width`, `height`). It is a private `asset` object and the source's `waveform` variant. A remote-only source over `MEDIA_PREVIEW_REMOTE_MAX_MB` (2048) is refused, because it decodes the whole audio track. `no_audio` is permanent |
| `object.sprite` | heavy | A seek-preview sprite sheet as a JPEG: 100 frames by default, at most one per 2 s, 10 columns and 160 px tiles (`frames`, `columns`, `tile_width`). With `MEDIA_HLS_ENABLED` and a source timeline it is cut from the timeline's rows — each sample time's row decoded from this node's segment or the packed chunk's byte range, checked against the row's sha256 — so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame over the source. It is a private `asset` object and the source's `sprite` variant. The layout goes in its metadata as `sprite: { count, interval_seconds, columns, rows, tile_width, tile_height }`: frame *i* covers [*i*·interval, (*i*+1)·interval) at column *i* mod columns, row ⌊*i*/columns⌋. `no_video` is permanent |
| `object.hash` | light | sha256 of ready objects' present local copies that have no hash yet, onto the object (`content_hash`, `metadata.hash_basis`) and the local location (`checksum`). Tenant-wide batch (`params { limit (1-500, 50), budget_mb }`) or one object. Scheduled per tenant every `MEDIA_HASH_INTERVAL_MIN`. Result `{ hashed, bytes, skipped, remaining }`. See [Scheduled verification](#scheduled-verification) |
ddiff --git a/server/vod/clip-jobs.js b/server/vod/clip-jobs.js
iindex bb09cd4..911c9cc 100644
---- a/server/vod/clip-jobs.js
++++ b/server/vod/clip-jobs.js
@@@ -133,6 +133,11 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false, job = null
const tl = await require('../objects/timeline').removeObject(clip.object_id)
.catch(err => { console.warn(`[Clips] Clip ${clipId} timeline cleanup failed: ${err.message}`); return null; });
if (tl && tl.pending) console.warn(`[Clips] Clip ${clipId}: ${tl.pending} timeline segment row(s) kept — their durable copy could not be deleted; the object sweep retries`);
++ // A clip that had a file already (a trim, a re-cut of a ready clip) leaves its previous cut behind: the row names the new one now.
++ const old = clip.file_path ? path.resolve(clip.file_path) : null;
++ if (old && old !== path.resolve(cut.filePath) && path.dirname(old) === cutter.CLIPS_DIR) {
++ try { fs.unlinkSync(old); } catch { /* already gone */ }
++ }
console.log(`[Clips] Clip ${clipId} ${reason} OK from vod ${clip.vod_id} (${startTime.toFixed(1)}-${(startTime + cut.duration).toFixed(1)}s, attempt ${attempt})`);
// A public, person-made clip that just became ready is indexable: announce its watch page and the sitemap.
await require('../indexnow-notify').pingWatch('clip', await db.getClipById(clipId));
ddiff --git a/server/vod/clips-routes.js b/server/vod/clips-routes.js
iindex 36ad760..8bb0e41 100644
---- a/server/vod/clips-routes.js
++++ b/server/vod/clips-routes.js
@@@ -8,6 +8,7 @@
* VOD (clamped to flushed footage, like the predecessor).
* GET /:id clip meta (status: processing | ready | failed)
* GET / list (?limit&offset&vod_id&stream_id&user_id&auto_generated&status&since)
++ * POST /:id/trim { start_s, end_s } (seconds into the clip) → narrowed in place, cut again from the source
* PUT /:id update title/visibility (and auto_generated, app key only)
* DELETE /:id delete file + offloaded objects + row
*/
@@@ -409,6 +410,58 @@ router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
}
});
++// ── Trim ─────────────────────────────────────────────────────
++/**
++ * Narrow a ready clip in place: { start_s, end_s } are seconds into the clip (at least 1 s apart, inside it). The clip
++ * keeps its id, title and links; its window of the source VOD shrinks to that part and it is cut again from the source
++ * (202 { id, status: 'processing', job_id }, clip.cut; the superseded file is removed once the new one is in), or, a
++ * virtual clip, it plays the narrower window of the source's timeline at once (200, the clip). Asking for the whole
++ * clip changes nothing (200, the clip, unchanged: true). A clip with no source VOD (a browser upload, already cut) or
++ * whose file has moved to cloud storage is not trimmed (422). An app-key call acting for a user trims only that user's
++ * own clips, as for every write.
++ */
++router.post('/:id/trim', tenantAuth({ verb: 'write' }), async (req, res) => {
++ try {
++ const clip = await _getClipForWrite(req, res);
++ if (!clip) return;
++ if (clip.status !== 'ready') return res.status(409).json({ error: clip.status === 'processing' ? 'Clip is still being cut' : 'Clip has no footage to trim' });
++ const len = Number(clip.duration_seconds) || 0;
++ const from = Number.parseFloat((req.body || {}).start_s);
++ let to = Number.parseFloat((req.body || {}).end_s);
++ if (!Number.isFinite(from) || !Number.isFinite(to)) return res.status(400).json({ error: 'start_s and end_s are required' });
++ if (from < 0 || to > len + 0.5) return res.status(400).json({ error: `start_s and end_s must lie inside the clip (0 to ${len.toFixed(1)} s)` });
++ to = Math.min(to, len);
++ if (to - from < 1) return res.status(400).json({ error: 'Clip must be at least 1 second' });
++ if (from < 0.05 && to > len - 0.05) return res.json({ ...await clipPublic(clip, { readiness: true }), unchanged: true });
++ const clipJobs = require('./clip-jobs');
++ const vod = clip.vod_id ? await db.getVodById(clip.vod_id, req.appId) : null;
++ if (!vod) return res.status(422).json({ error: 'Clip has no source VOD to trim from' });
++ const virtual = clipJobs.isVirtual(clip);
++ if (!virtual && require('./vod-storage').isRemote(clip)) return res.status(422).json({ error: 'Clip has moved to cloud storage and can no longer be trimmed' });
++
++ const before = [clip.start_time, clip.end_time, clip.duration_seconds];
++ const start = (Number(clip.start_time) || 0) + from, end = (Number(clip.start_time) || 0) + to;
++ const setWindow = async (w) => await objects.withObject('clip', clip.id, async () => await db.run('UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = ?, cut_attempts = 0 WHERE id = ?', [...w, clip.id]));
++ await setWindow([start, end, end - start]);
++ // A plain virtual clip plays the narrower window at once; a materialized one has rows of its own, which the cut rebuilds.
++ const obj = virtual ? await objects.getObject(clip.object_id) : null;
++ const materialized = !!(obj && /"materialized":\s*true/.test(String(obj.metadata || '')));
++ const win = virtual && !materialized && await clipJobs.timelineWindow(vod, start, end);
++ if (win) {
++ await clipJobs.makeVirtual(clip.id, win);
++ return res.json({ ...await clipPublic(await db.getClipById(clip.id, req.appId), { readiness: true }) });
++ }
++ let job;
++ try { ({ job } = await clipJobs.enqueueCut(req.appId, clip.id, { reason: 'trim', createdBy: `app:${req.appId}` })); }
++ catch (err) { await setWindow(before); throw err; }
++ console.log(`[Clips] Clip ${clip.id} trimmed to ${start.toFixed(1)}-${end.toFixed(1)}s of vod ${clip.vod_id}`);
++ res.status(202).json({ id: clip.id, status: 'processing', job_id: job.id });
++ } catch (err) {
++ console.error('[Clips] Trim error:', err.message);
++ res.status(500).json({ error: 'Failed to trim clip' });
++ }
++});
++
router.put('/:id', tenantAuth({ verb: 'write' }), async (req, res) => {
try {
const clip = await _getClipForWrite(req, res);
ddiff --git a/test/clip-trim.test.js b/test/clip-trim.test.js
nnew file mode 100644
iindex 0000000..cc28a7e
---- /dev/null
++++ b/test/clip-trim.test.js
@@@ -0,0 +1,131 @@
++'use strict';
++// POST /clips/:id/trim: a ready clip narrows in place. Its window of the source VOD shrinks and it is cut again from
++// the source (202 with the clip.cut job); the id and title stay, the superseded file is removed, clip.ready is told
++// again. The whole clip is no change; a window under 1 s or outside the clip is refused; a clip still being cut is
++// 409; a clip with no source VOD (a browser upload) is 422; acting for a user, only that user's own clips.
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const http = require('http');
++const { spawnSync } = require('child_process');
++(async () => {
++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-cliptrim-'));
++ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };
++ process.env.VOD_PATH = dir('vods');
++ process.env.CLIPS_PATH = dir('clips');
++ process.env.FILES_PATH = dir('files');
++ process.env.THUMBNAILS_PATH = dir('thumbnails');
++ process.env.PASTES_PATH = dir('pastes');
++ process.env.OBJECTS_PATH = dir('objects');
++ process.env.MEDIA_JOBS_POLL_MS = '50';
++ process.env.MEDIA_INVARIANT_SCAN_HOURS = '0';
++ process.env.MEDIA_UPLOAD_MIN_FREE_MB = '0';
++ const stub = http.createServer((req, res) => {
++ req.resume();
++ req.on('end', () => {
++ res.setHeader('Content-Type', 'application/json');
++ if (req.url === '/oauth/token') return res.end(JSON.stringify({ access_token: 'tok', token_type: 'Bearer', expires_in: 300 }));
++ res.statusCode = 404; res.end('{}');
++ });
++ });
++ const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
++ const waitFor = async (fn, ms = 30000) => { const end = Date.now() + ms; while (Date.now() < end) { if (await fn()) return true; await sleep(25); } return false; };
++
++ if (spawnSync('ffmpeg', ['-version']).status !== 0) { console.log('clip-trim: skipped (ffmpeg not found)'); process.exit(0); }
++ await new Promise((r) => stub.listen(0, '127.0.0.1', r));
++ const base = `http://127.0.0.1:${stub.address().port}`;
++ const db = require('../server/db/database');
++ const conn = db.getDb();
++ await db.upsertApp({ app_id: 'live', api_key: 'live-key' });
++ const events = require('../server/events');
++ events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
++ const queue = require('../server/jobs/queue');
++ const worker = require('../server/jobs/worker');
++ const express = require('express');
++ const app = express();
++ app.use(express.json());
++ app.use('/api/v1/:app/clips', require('../server/vod/clips-routes'));
++ const server = http.createServer(app);
++ await new Promise((r) => server.listen(0, '127.0.0.1', r));
++ const api = `http://127.0.0.1:${server.address().port}`;
++ const call = async (method, p, body, headers = {}) => {
++ const res = await fetch(api + p, { method, headers: { authorization: 'Bearer live-key', 'content-type': 'application/json', ...headers }, body: body === undefined ? undefined : JSON.stringify(body) });
++ return { status: res.status, body: await res.json().catch(() => null) };
++ };
++ const readyEvents = async (id) => (await conn.prepare('SELECT envelope FROM event_outbox ORDER BY id').all())
++ .map((r) => (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope))
++ .filter((e) => e.event_type === 'media.clip.ready' && String(e.subject.id) === String(id)).length;
++ const settled = async (jobId) => await waitFor(async () => ['succeeded', 'failed'].includes((await queue.get(jobId)).status));
++ await worker.start();
++
++ const src = path.join(process.env.VOD_PATH, 'rec-30.webm');
++ const mk = spawnSync('ffmpeg', ['-v', 'error', '-y', '-f', 'lavfi', '-i', 'testsrc=duration=30:size=320x180:rate=10', '-f', 'lavfi', '-i', 'sine=frequency=440:duration=30',
++ '-c:v', 'libvpx', '-g', '10', '-b:v', '200k', '-c:a', 'libopus', '-shortest', src]);
++ assert.strictEqual(mk.status, 0, String(mk.stderr));
++ await conn.prepare("INSERT INTO vods (id, app_id, user_id, title, file_path, file_size, is_public, duration_seconds) OVERRIDING SYSTEM VALUE VALUES (30, 'live', 5, 'Thirty seconds', ?, ?, 1, 30) RETURNING id").run(src, fs.statSync(src).size);
++
++ // A 15 s clip of the VOD (5-20 s), cut.
++ let r = await call('POST', '/api/v1/live/clips', { vod_id: 30, start_s: 5, end_s: 20, title: 'Best food', user_id: 5 });
++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
++ const clipId = r.body.id;
++ assert.ok(await settled(r.body.job_id));
++ const first = await db.getClipById(clipId);
++ assert.strictEqual(first.status, 'ready');
++ const len = Number(first.duration_seconds);
++ assert.ok(Math.abs(len - 15) < 1.5, `a 15 s clip (${len})`);
++
++ // ── The whole clip: nothing changes ──
++ r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 0, end_s: len });
++ assert.strictEqual(r.status, 200, JSON.stringify(r.body));
++ assert.strictEqual(r.body.unchanged, true);
++ assert.strictEqual((await db.getClipById(clipId)).file_path, first.file_path, 'the file is untouched');
++
++ // ── Refused windows ──
++ for (const [body, why] of [[{}, 'no window'], [{ start_s: 4, end_s: 4.5 }, 'under 1 s'], [{ start_s: -1, end_s: 5 }, 'before the clip'], [{ start_s: 2, end_s: len + 5 }, 'past its end']]) {
++ assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, body)).status, 400, why);
++ }
++
++ // ── Acting for another user: not theirs ──
++ r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 3, end_s: 9 }, { 'x-ov-user-id': '6' });
++ assert.strictEqual(r.status, 403, JSON.stringify(r.body));
++
++ // ── Seconds 3-9 of the clip: cut again from the source, in place ──
++ const before = await readyEvents(clipId);
++ r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 3, end_s: 9 });
++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
++ assert.strictEqual(r.body.id, clipId);
++ assert.ok(/^mjob_/.test(r.body.job_id), 'the answer names the cut job');
++ // Still being cut: a second trim waits.
++ if ((await db.getClipById(clipId)).status === 'processing') {
++ assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 1, end_s: 3 })).status, 409);
++ }
++ assert.ok(await settled(r.body.job_id));
++ assert.strictEqual((await queue.get(r.body.job_id)).status, 'succeeded');
++ const trimmed = await db.getClipById(clipId);
++ assert.strictEqual(trimmed.status, 'ready');
++ assert.strictEqual(trimmed.title, 'Best food', 'the title stays');
++ assert.ok(Math.abs(Number(trimmed.start_time) - 8) < 0.01, `starts 3 s into the old window (${trimmed.start_time})`);
++ assert.ok(Math.abs(Number(trimmed.duration_seconds) - 6) < 1.5, `a 6 s clip (${trimmed.duration_seconds})`);
++ assert.notStrictEqual(trimmed.file_path, first.file_path, 'a new cut');
++ assert.ok(fs.existsSync(trimmed.file_path), 'the new file is there');
++ assert.ok(!fs.existsSync(first.file_path), 'the superseded file is removed');
++ assert.strictEqual(await readyEvents(clipId), before + 1, 'clip.ready is told again');
++
++ // ── A clip still being cut ──
++ await conn.prepare("UPDATE clips SET status = 'processing' WHERE id = ?").run(clipId);
++ assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 1, end_s: 3 })).status, 409);
++ await conn.prepare("UPDATE clips SET status = 'ready' WHERE id = ?").run(clipId);
++
++ // ── A browser upload has no source to trim from ──
++ const up = await db.createClip({ app_id: 'live', vod_id: null, user_id: 5, title: 'Uploaded', file_path: trimmed.file_path, start_time: 0, end_time: 6, duration_seconds: 6, status: 'ready' });
++ r = await call('POST', `/api/v1/live/clips/${up.lastInsertRowid}/trim`, { start_s: 1, end_s: 4 });
++ assert.strictEqual(r.status, 422, JSON.stringify(r.body));
++
++ worker.stop();
++ events._reset();
++ server.close();
++ stub.close();
++ console.log('clip-trim: all checks passed');
++ process.exit(0);
++})().catch((err) => { console.error(err); process.exit(1); });