openvibers/openvibe.media Commits
VODs that change shape mid-stream play each part at its own shape: one period per shape, each with its own init (#180)
Changes
@@@ -110,6 +110,23 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}honours the abort signal (which stops an upload in flight) and, before the rows commit, removes the files and keys itstaged; the stream-copy or duration-based transcode budget is honoured and its work directory removed. The masterplaylist reads CODECS from the written init segment, so an encoded source advertises its output codecs.++++ **A picture that changes shape.** A recording is a stream copy of what the broadcaster sent, so when a phone turns++ from landscape to portrait, or WebRTC lowers 720p to 360p on a weak network, its frames change size at a keyframe++ while the file's header still names the first size. One encode scales every later frame to that first size (VOD 6017:++ a portrait street stretched into 16:9), and a copy carries frames its init does not describe. With++ `MEDIA_CMAF_TRANSCODE=1`, `object.cmaf` first reads each keyframe's size (`server/jobs/shapes.js`, a keyframe-only++ decode). When there is more than one, it cuts **one period per shape**: sizes within ~4% of one aspect ratio are one++ shape (a lowered resolution is not a new period), and each period is drawn on its shape's largest size (long side at++ most 1920), with frames of another size fitted, never stretched. A period under 5 s joins a neighbour, and there are++ at most 40. Each period is its own ffmpeg run on the input's own timestamps, with its own init: the first period's is++ `init.mp4` (seq 0), period n ≥ 2's is `init-<n>.mp4`, a row at seq −(100+n) that its segments name in `init_name`++ (migration 0014). The media playlist then puts `#EXT-X-DISCONTINUITY` + `#EXT-X-MAP` between periods, so a player++ shows each part at its own shape, the way an adaptive stream switches renditions. Virtual and materialized clips carry++ the period inits their segments name, `rendition.create` cuts the same periods at its rung's height, and a clip cut++ (`server/vod/clip-cutter.js`) whose window spans a change is drawn on the window's main shape. One size throughout++ keeps the cut exactly as before. `scripts/recut-shapes.js` finds the recordings cut before this (a dry run by default)++ and queues their re-cut.Behind **`MEDIA_HLS_ENABLED`** (off by default: the job is refused with `409 media.hls.disabled` and the routes 404),`GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8` and `/o/:id/source/{init.mp4,NNNNNN.m4s}` serve the timelineunder the same check as `GET /o/:id` (private and sandbox objects only with the object's `?exp&sig`, carried ontonnew file mode 100644@@@ -0,0 +1,9 @@++-- phase: expand++-- A video whose picture changes shape mid-stream (a phone turned, a resolution lowered) is cut in periods, one init++-- segment each (server/jobs/shapes.js): the first period's is init.mp4 at seq 0, period n >= 2's is init-<n>.mp4 at seq++-- -(100 + n), named by its media rows' init_name (0005). This widens 0005's seq CHECK to admit those names. Additive++-- only: an older release reads the rows with init_name already (its playlists put a discontinuity before each init) and++-- never writes one.++ALTER TABLE media_timeline DROP CONSTRAINT IF EXISTS media_timeline_seq_check;++ALTER TABLE media_timeline ADD CONSTRAINT media_timeline_seq_check++ CHECK (seq >= 0 OR name IN ('init-head.mp4', 'init-tail.mp4') OR name ~ '^init-[0-9]{1,4}\.mp4$');nnew file mode 100755@@@ -0,0 +1,87 @@++#!/usr/bin/env node++'use strict';++/**++ * Re-cut the recordings whose source timeline was cut as one shape although their picture changes shape mid-stream++ * (server/jobs/shapes.js, docs/media-fabric.md §3: VOD 6017 played its portrait part stretched into 16:9), for the++ * operator.++ *++ * node scripts/recut-shapes.js [--days 30] [--limit 200] [--app live] [--ids 6017,6020] [--remote] [--apply] [--json]++ * Dry run (the default): for each finished VOD of the window, newest first, whose object has a source timeline++ * with one init and a local file (--remote also reads B2/R2 copies, which is egress), reads its keyframe sizes++ * and reports the ones with more than one, with the periods a re-cut would make. Changes nothing.++ * --apply queues object.cmaf for each of those (idempotency key object.cmaf:<object>:shapes, so a second run++ * answers with the same job). The running service's worker cuts them on its next poll, one at a time in the heavy++ * lane at nice 15; the new rows replace the old ones in one transaction, and playback never stops.++ * (the database is the service's: DATABASE_URL, and MEDIA_HLS_ENABLED / MEDIA_CMAF_TRANSCODE, from its env file)++ *++ * Exit codes: 0 done, 1 error, 2 bad usage.++ */++++(async () => {++ await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the service++ const args = process.argv.slice(2);++ const has = (name) => args.includes(`--${name}`);++ const arg = (name, d = null) => { const i = args.indexOf(`--${name}`); return i >= 0 && args[i + 1] && !args[i + 1].startsWith('--') ? args[i + 1] : d; };++ const usage = (msg) => { console.error(`recut-shapes: ${msg}`); process.exit(2); };++ for (const flag of ['days', 'limit', 'app', 'ids']) if (has(flag) && !arg(flag)) usage(`--${flag} needs a value`);++ const APPLY = has('apply');++ const days = Number(arg('days', 30));++ const limit = Number(arg('limit', 200));++ if (!(days > 0) || !(limit > 0)) usage('--days and --limit are positive numbers');++ const ids = arg('ids') ? arg('ids').split(',').map(Number).filter((n) => n > 0) : null;++++ const config = require('../server/config');++ const db = require('../server/db/database');++ const model = require('../server/objects/model');++ const timeline = require('../server/objects/timeline');++ const derive = require('../server/jobs/derive');++ const shapes = require('../server/jobs/shapes');++ if (!config.hls.enabled || !config.hls.transcode) usage('the re-cut needs MEDIA_HLS_ENABLED=1 and MEDIA_CMAF_TRANSCODE=1 (the service\'s env file)');++ if (APPLY) await require('../server/events').initWriter();++++ const since = new Date(Date.now() - days * 86400000).toISOString().slice(0, 19).replace('T', ' ');++ const vods = ids++ ? await db.all('SELECT id, app_id, object_id, title FROM vods WHERE id = ANY(?) ORDER BY id DESC', [ids])++ : await db.all(`SELECT id, app_id, object_id, title FROM vods WHERE is_recording = 0 AND object_id IS NOT NULL AND created_at >= ?++ ${has('app') ? 'AND app_id = ?' : ''} ORDER BY id DESC LIMIT ?`, has('app') ? [since, arg('app'), limit] : [since, limit]);++ const report = [];++ for (const v of vods) {++ const out = { vod: v.id, object: v.object_id, title: v.title };++ report.push(out);++ const obj = v.object_id && await model.getObject(v.object_id);++ if (!obj || obj.lifecycle_status !== 'ready') { out.skip = 'no ready object'; continue; }++ const rows = await timeline.list(obj.id, timeline.SOURCE);++ if (!rows.some((r) => Number(r.seq) > 0)) { out.skip = 'no source timeline (its first cut reads the shapes)'; continue; }++ if (rows.some((r) => timeline.shapeInitSeq(r.name) != null)) { out.skip = 'already cut in periods'; continue; }++ const source = await derive.resolveSource(obj);++ if (!source) { out.skip = 'no bytes'; continue; }++ if (source.remote && !has('remote')) { out.skip = 'only on B2/R2 (--remote reads it)'; continue; }++ try {++ const read = await shapes.probe(source, { timeoutMs: derive.budgetMs(obj.size_bytes) });++ if (!shapes.varies(read.changes)) { out.shapes = 1; continue; }++ const info = await derive.probeStreams(source);++ const periods = shapes.plan(read.changes, { end: read.start + info.duration });++ out.sizes = read.changes.length;++ out.periods = periods.map((p) => `${p.width}x${p.height} from ${(p.start - read.start).toFixed(1)} s`);++ if (APPLY) {++ const params = require('../server/jobs/cmaf').spec.validate({ appId: obj.app_id, obj, params: {} });++ const r = await require('../server/jobs/queue').enqueue({ appId: obj.app_id, type: 'object.cmaf', objectId: obj.id, params,++ dedupeActive: true, idempotencyKey: `object.cmaf:${obj.id}:shapes`, createdBy: `operator:${require('os').userInfo().username}` });++ out.job = r.job ? r.job.id : null;++ out.queued = !!r.created;++ }++ } catch (err) { out.error = err.message.slice(0, 200); }++ }++ if (has('json')) console.log(JSON.stringify(report, null, 2));++ else {++ for (const r of report) {++ const what = r.skip ? `skip: ${r.skip}` : r.error ? `error: ${r.error}` : r.periods++ ? `${r.sizes} sizes → ${r.periods.join(', ')}${r.job ? ` ${r.queued ? 'queued' : 'already queued'} ${r.job}` : ''}`++ : 'one shape';++ console.log(`vod ${r.vod} ${what}`);++ }++ const found = report.filter((r) => r.periods).length;++ console.log(`${report.length} checked, ${found} change shape${APPLY ? `, ${report.filter((r) => r.queued).length} queued` : found ? ' (dry run: --apply queues their re-cut)' : ''}`);++ }++ process.exit(report.some((r) => r.error) ? 1 : 0);++})().catch((err) => { console.error(err); process.exit(1); });@@@ -19,6 +19,9 @@* rendition.create (F4).* Idempotent: a rerun makes the same bytes (bitexact), reuses a row whose sha256 matches and its location as it is, and* uploads only what is not durable yet. A run that aborts or fails before the commit removes its staged files and keys.++ * A picture that changes shape mid-stream (a phone turned, a resolution lowered: jobs/shapes.js) is re-encoded with++ * MEDIA_CMAF_TRANSCODE in one period per shape, each on its own canvas with its own init segment, so the playlist++ * switches shape between periods instead of stretching every frame to the first size; `shapes` in the result counts them.* Refused: MEDIA_HLS_ENABLED off, a source that is not ready or not media (permanent), less free disk than the source* size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent), a clip whose file was* replaced (a trim or re-cut) while it ran (permanent, media.source.changed: nothing is committed).@@@ -31,6 +34,7 @@ const config = require('../config');const timeline = require('../objects/timeline');const derive = require('./derive');const segments = require('./segments');++const shapes = require('./shapes');const { JobError } = require('./queue');const DEFAULT_SEGMENT_SECONDS = 4;@@@ -39,6 +43,13 @@ function transcodeBudgetMs(duration) {return Math.min(13 * 3600 * 1000, Math.max(10 * 60000, Math.round(duration * 3000 + 10 * 60000)));}++/** The H.264 re-encode: a keyframe at every segment boundary, frame times as they are. */++function x264Args(segmentSeconds) {++ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),++ '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough'];++}++const AAC_ARGS = ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'];++function disabled() {return new JobError('media.hls.disabled', 'Segment-native video is off on this server (MEDIA_HLS_ENABLED)', { permanent: true, status: 409 });}@@@ -56,6 +67,25 @@ function validate({ obj, params }) {return p;}++/** The whole input as one HLS fMP4 rendition (one shape): → the produced segments, [{ name, seconds }]. */++async function cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness }) {++ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,++ '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',++ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',++ '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,++ '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],++ { signal: ctx.signal, timeoutMs, niceness });++ segments.aborted(ctx);++ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);++ if (!r.ok) {++ const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);++ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });++ }++ const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];++ if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);++ return list;++}++async function run(job, ctx) {if (!config.hls.enabled) throw disabled();const src = await derive.loadSource(job);@@@ -88,11 +118,28 @@ async function run(job, ctx) {throw new JobError('too_long_to_transcode', `Source duration ${info.duration}s exceeds MEDIA_CMAF_TRANSCODE_MAX_SECONDS`, { permanent: true });}timeoutMs = transcodeBudgetMs(info.duration);-- codecArgs = video === 'h264'-- ? ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),-- '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough']-- : ['-c:v', 'copy'];-- codecArgs.push(...(audio === 'aac' ? ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'] : ['-c:a', 'copy']));++ codecArgs = video === 'h264' ? x264Args(segmentSeconds) : ['-c:v', 'copy'];++ codecArgs.push(...(audio === 'aac' ? AAC_ARGS : ['-c:a', 'copy']));++ }++ }++ // Frames of more than one size (shapes.js): one period per shape, each re-encoded on its own canvas. A copy would++ // carry frames its init does not describe, and one encode scales every frame to the first size (a portrait part++ // stretched into 16:9). A probe that fails leaves the cut as it was.++ let periods = null;++ let shapeStart = 0;++ if (config.hls.transcode && info && info.video && info.duration > 0 && info.duration <= config.hls.transcodeMaxSeconds) {++ let read = null;++ try { read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }); }++ catch (err) { segments.aborted(ctx); console.warn(`[Cmaf] ${job.id}: shape probe failed (${err.message.slice(0, 200)}); cut as one shape`); }++ segments.aborted(ctx);++ if (read && shapes.varies(read.changes)) {++ periods = shapes.plan(read.changes, { end: read.start + info.duration });++ shapeStart = read.start;++ video = 'h264';++ if (audio !== 'none') audio = 'aac';++ timeoutMs = transcodeBudgetMs(info.duration);++ codecArgs = [...x264Args(segmentSeconds), ...(audio === 'aac' ? AAC_ARGS : [])];++ console.log(`[Cmaf] ${job.id}: ${read.changes.length} frame sizes → ${periods.length} period(s): ${periods.map((p) => `${p.width}x${p.height}`).join(', ')}`);}}segments.checkDisk(Number(src.size_bytes) || 0);@@@ -101,20 +148,10 @@ async function run(job, ctx) {try {const out = path.join(dir, 'out');fs.mkdirSync(out, { recursive: true });-- const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,-- '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',-- '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',-- '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,-- '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],-- { signal: ctx.signal, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });-- segments.aborted(ctx);-- const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);-- if (!r.ok) {-- const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);-- throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });-- }-- const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];-- if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);++ const list = periods++ ? await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds, audio: audio !== 'none', encode: () => codecArgs,++ signal: ctx.signal, timeoutMs, niceness: 15 })++ : await cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });const check = version == null ? null : async () => {if (await derive.fileVersion(src, { lock: true }) !== version) {@@@ -127,7 +164,7 @@ async function run(job, ctx) {return {source_id: src.id, rendition, segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,-- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio,++ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio, shapes: periods ? periods.length : 1,};} finally {derive.cleanupWork(job.id);@@@ -14,6 +14,9 @@* result { …, skipped: true, reason } when the source is already this rendition's height or smaller (a still is not* upscaled), or the rows already exist (skipped: false, already: true)*++ * A source whose picture changes shape (jobs/shapes.js) is cut as its source timeline is: one period per shape, each on++ * its own canvas scaled to the rung's height, with its own init segment.++ ** Refused: MEDIA_HLS_ENABLED or MEDIA_RENDITIONS off, a source that is not ready or not media (permanent), an object* under a hold (retried), a source with no video stream (permanent), less free disk than the source size plus* MEDIA_UPLOAD_MIN_FREE_MB (retried later). Idempotent: a re-run whose bytes match reuses the rows and locations, and@@@ -31,6 +34,7 @@ const config = require('../config');const timeline = require('../objects/timeline');const derive = require('./derive');const segments = require('./segments');++const shapes = require('./shapes');const { JobError } = require('./queue');const DEFAULT_SEGMENT_SECONDS = 4;@@@ -95,11 +99,22 @@ async function run(job, ctx) {const info = await require('../vod/media-tools').probeVodInfo(source.input);const video = (info.streams || []).find((s) => s.codec_type === 'video');if (!video) throw new JobError('no_video', 'The source has no video stream to rendition', { permanent: true });-- const height = Number(video.height) || 0;++ // Frames of more than one size: the periods the source timeline was cut in, each brought down to this rung.++ let planned = null;++ let shapeStart = 0;++ const read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }).catch(() => null);++ segments.aborted(ctx);++ if (read && shapes.varies(read.changes)) {++ planned = shapes.plan(read.changes, { end: read.start + (Number(info.duration) || 0) });++ shapeStart = read.start;++ }++ const height = planned ? Math.max(...planned.map((p) => p.height)) : Number(video.height) || 0;// A source already this rendition's height or smaller is not upscaled (a re-encode would only cost bytes).if (height > 0 && height <= targetHeight) {return { source_id: src.id, rendition, target_height: targetHeight, skipped: true, reason: `the source is ${height}p, not above ${rendition}`, segments: 0, duration_ms: 0, bytes: 0 };}++ const periods = planned && planned.map((p) => (p.height <= targetHeight ? p++ : { ...p, width: Math.max(2, Math.round(p.width * targetHeight / p.height / 2) * 2), height: targetHeight }));segments.checkDisk(Number(src.size_bytes) || 0);const srcRows = await timeline.segments(src.id, timeline.SOURCE);const segSec = segmentSeconds(job, srcRows);@@@ -108,9 +123,33 @@ async function run(job, ctx) {// and a source that long is packed long before it is re-encoded.const boundaries = srcRows.length && srcRows.length <= 2000 ? srcRows.map((r) => (Number(r.start_ms) / 1000).toFixed(3)) : [];const dir = derive.workDir(job.id);++ const publish = async (list) => {++ const published = await segments.publishRendition({ job, ctx, src, rendition, out: path.join(dir, 'out'), list, label: 'Rendition' });++ return {++ source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,++ segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,++ durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,++ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, shapes: periods ? periods.length : 1,++ };++ };try {const out = path.join(dir, 'out');fs.mkdirSync(out, { recursive: true });++ if (periods) {++ // Each period's keyframes at the source boundaries inside it, in the period's own time (its output starts at 0).++ const axis = (p, i) => (i === 0 ? 0 : p.start - shapeStart);++ const list = await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds: segSec,++ audio: (info.streams || []).some((st) => st.codec_type === 'audio'),++ encode: (p, i) => {++ const from = axis(p, i);++ const to = i + 1 < periods.length ? axis(periods[i + 1], i + 1) : Infinity;++ const own = boundaries.map(Number).filter((b) => b >= from && b < to).map((b) => (b - from).toFixed(3));++ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',++ '-c:a', 'aac', '-b:a', '128k', '-ac', '2', ...(own.length ? ['-force_key_frames', own.join(',')] : [])];++ },++ signal: ctx.signal, timeoutMs: encodeBudgetMs(src.size_bytes), niceness: 0 });++ return await publish(list);++ }const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,'-map', '0:v:0', '-map', '0:a:0?','-vf', `scale=-2:${targetHeight}`, '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',@@@ -130,13 +169,7 @@ async function run(job, ctx) {const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);-- const published = await segments.publishRendition({ job, ctx, src, rendition, out, list, label: 'Rendition' });-- return {-- source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,-- segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,-- durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,-- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true,-- };++ return await publish(list);} finally {derive.cleanupWork(job.id);}@@@ -6,8 +6,10 @@* turns that output into media_timeline rows and publishes them:** parsePlaylist ffmpeg's index.m3u8 → [{ name, seconds }], in order.-- * producedFiles those plus the init segment → [{ seq, name, file, start_ms, end_ms, keyframe_ms }], times on the-- * object's own axis (cumulative EXTINF, so each segment ends where the next starts).++ * producedFiles those plus the init segment(s) → [{ seq, name, file, start_ms, end_ms, keyframe_ms, init_name }],++ * times on the object's own axis (cumulative EXTINF, so each segment ends where the next starts). A++ * video cut in shape periods (jobs/shapes.js) has one init per period: the later ones are rows with++ * negative seq that their segments name in init_name.* publishRendition hash each file, place it content-addressed on this node (timeline.localPathFor/keyFor, versioned* by the sha prefix) and — through the placement router's durable provider (timeline.durableProvider,* purpose `durable`) — on B2/R2, then commit every row of the rendition in ONE transaction@@@ -45,16 +47,27 @@ function parsePlaylist(text) {return out;}--/** The produced files as timeline rows in the making: the init segment (seq 0) then each media segment, in order. */++/**++ * The produced files as timeline rows in the making: the init segment (seq 0), the init of each later shape period++ * (a list entry's `init`, jobs/shapes.js: negative seq, named in its segments' init_name), then each media segment, in++ * order.++ */function producedFiles(list) {const files = [{ seq: 0, name: timeline.INIT_NAME, start_ms: 0, end_ms: 0, keyframe_ms: null }];++ const media = [];let cum = 0;list.forEach((s, i) => {const start = Math.round(cum * 1000);cum += s.seconds;-- files.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start });++ const init = s.init && s.init !== timeline.INIT_NAME ? s.init : null;++ if (init && !files.some((f) => f.name === init)) {++ const seq = timeline.shapeInitSeq(init);++ if (seq == null) throw new JobError('ffmpeg_failed', `${init} is not a shape period's init`);++ files.push({ seq, name: init, start_ms: 0, end_ms: 0, keyframe_ms: null });++ }++ media.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start, init_name: init });});-- return files;++ return files.concat(media);}function checkDisk(needBytes) {@@@ -122,7 +135,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = 'const match = !!old && old.sha256 === sha;if (match && old.packed_object_id) {// Already packed into a chunk (object.pack): the same bytes, so the row and its chunk stay as they are.-- rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : old[k]])));++ rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : k === 'init_name' ? f.init_name || null : old[k]])));continue;}// Content-addressed: the same bytes always land at the same versioned location, so a rerun rewrites@@@ -143,7 +156,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = 'if (!durable && target) {try {if (!vodStorage.providerAvailable(target)) throw new Error(`${target} is unavailable`);-- await vodStorage.uploadFile(target, key, localPath, f.seq === 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });++ await vodStorage.uploadFile(target, key, localPath, f.seq <= 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });stagedRemote.push({ provider: target, key });const back = await vodStorage.sha256Object(target, key);if (!back || back.sha256 !== sha || Number(back.size) !== size) throw new Error(`${target}:${key} failed size/sha256 readback`);@@@ -161,7 +174,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = 'rows.push({seq: f.seq, name: f.name, start_ms: f.start_ms, end_ms: f.end_ms, keyframe_ms: f.keyframe_ms, key, local_path: localPath,durable_provider: durable, packed_object_id: null, byte_offset: null, byte_length: size, sha256: sha,-- durability: durable ? 'durable' : 'local',++ durability: durable ? 'durable' : 'local', init_name: f.init_name || null,});}aborted(ctx);nnew file mode 100644@@@ -0,0 +1,213 @@++/**++ * OpenVibe.Media — the picture's shapes over a video, and cutting it in one period per shape (docs/media-fabric.md §3).++ *++ * A recording is a stream copy of what the broadcaster sent. When their phone turns, or WebRTC lowers 720p to 360p on a++ * weak network, the frames change size at a keyframe while the file's header still names the first size. One encode++ * then scales every later frame to that first size (VOD 6017: a portrait street stretched into 16:9), and a stream copy++ * carries frames its init segment does not describe. So a cut that must re-encode reads the shapes first:++ *++ * probe(source) → { start, changes: [{ t, width, height }] }: the input's start time and each keyframe whose++ * size differs from the one before (one decode of the keyframes only; t in the input's own++ * seconds). `from`/`to` read only that window (a clip).++ * plan(changes, { end }) → [{ start, end, width, height }]: the periods. Sizes within SAME_SHAPE of one aspect ratio++ * are one shape (a lowered resolution is not a new period); each period is drawn on the canvas++ * of its shape's largest size (long side at most MAX_SIDE). A period shorter than MIN_PERIOD_S++ * joins a neighbour, and there are at most MAX_PERIODS.++ * fit(width, height) the filter that puts any frame on a width×height canvas without stretching it: scaled to fit,++ * centred, bars for the rest. A frame of the period's own shape fills the canvas exactly.++ * cut({ … }) one ffmpeg run per period into `out`, each with its own init segment (the first period's is++ * init.mp4; period n ≥ 2's is timeline.shapeInitName(n)) → the produced segments in order,++ * [{ name, seconds, init }] for segments.publishRendition. The playlist puts++ * #EXT-X-DISCONTINUITY + #EXT-X-MAP between periods (timeline.mediaPlaylist), so a player shows++ * each part at its own shape, the way an adaptive stream switches renditions.++ *++ * Periods are cut on the input's own timestamps (-copyts, trim/atrim at the period's edges, the next period sought to its++ * first keyframe), so no frame of one shape lands in the other's period; each period's timestamps continue where the++ * last one's ended, with no gap at the join (negative composition offsets: Chrome's native HLS stalls on a gap there).++ */++'use strict';++++const path = require('path');++const fs = require('fs');++const { spawn } = require('child_process');++const derive = require('./derive');++const timeline = require('../objects/timeline');++const { JobError } = require('./queue');++++const SAME_SHAPE = 0.04; // |ln(aspect a / aspect b)|: 1280x720 and 426x240 are one shape, 16:9 and 4:3 are not++const MIN_PERIOD_S = 5;++const MAX_PERIODS = 40;++const MAX_SIDE = 1920;++++const aspect = (s) => s.width / s.height;++const sameShape = (a, b) => Math.abs(Math.log(aspect(a) / aspect(b))) < SAME_SHAPE;++const even = (x) => Math.max(2, Math.round(x / 2) * 2);++const secs = (t) => Number(t).toFixed(6).replace(/\.?0+$/, '') || '0';++++/** Each keyframe's size, read with ffprobe (keyframes only). Resolves { start, changes }. */++function probe(source, { signal, timeoutMs = 30 * 60000, from = null, to = null } = {}) {++ return new Promise((resolve, reject) => {++ const args = ['-v', 'error', ...derive.inputArgs(source),++ ...(from != null ? ['-read_intervals', `${secs(from)}%${secs(to)}`] : []),++ '-select_streams', 'v:0', '-skip_frame', 'nokey',++ '-show_entries', 'frame=best_effort_timestamp_time,width,height:format=start_time', '-of', 'csv=p=0', source.input];++ let proc;++ try { proc = spawn('ffprobe', args, { stdio: ['ignore', 'pipe', 'pipe'] }); } catch (err) { return reject(err); }++ try { require('os').setPriority(proc.pid, 10); } catch { /* best effort */ }++ const changes = [];++ let start = null;++ let rest = '';++ let stderr = '';++ let done = false;++ const line = (l) => {++ const f = l.trim().split(',');++ if (f.length === 1) { if (f[0] && f[0] !== 'N/A') start = Number(f[0]); return; } // format=start_time++ const [t, width, height] = f.map(Number);++ if (!Number.isFinite(t) || !(width > 0) || !(height > 0)) return;++ const last = changes[changes.length - 1];++ if (!last || last.width !== width || last.height !== height) changes.push({ t, width, height });++ };++ const kill = () => { try { proc.kill('SIGKILL'); } catch { /* gone */ } };++ const timer = setTimeout(kill, timeoutMs);++ const finish = (err, value) => {++ if (done) return;++ done = true;++ clearTimeout(timer);++ if (signal) signal.removeEventListener('abort', kill);++ if (err) reject(err); else resolve(value);++ };++ if (signal) { if (signal.aborted) kill(); else signal.addEventListener('abort', kill, { once: true }); }++ proc.stdout.on('data', (d) => {++ const lines = (rest + d).split('\n');++ rest = lines.pop();++ for (const l of lines) line(l);++ });++ proc.stderr.on('data', (d) => { stderr = (stderr + d).slice(-4000); });++ proc.on('error', (err) => finish(err));++ proc.on('close', (code) => {++ if (rest) line(rest);++ if (code !== 0) return finish(new Error(`ffprobe exited ${code}: ${stderr.trim().slice(-300)}`));++ finish(null, { start: Number.isFinite(start) ? start : (changes[0] ? changes[0].t : 0), changes });++ });++ });++}++++/** The shape a set of runs mostly shows, and its largest size: { width, height } of the canvas. */++function canvasOf(runs) {++ const groups = [];++ for (const r of runs) {++ let g = groups.find((x) => sameShape(x.lead, r));++ if (!g) groups.push(g = { lead: r, seconds: 0, runs: [] });++ g.seconds += r.end - r.start;++ g.runs.push(r);++ }++ const main = groups.reduce((a, b) => (b.seconds > a.seconds ? b : a));++ const big = main.runs.reduce((a, b) => (b.width * b.height > a.width * a.height ? b : a));++ const k = Math.min(1, MAX_SIDE / Math.max(big.width, big.height));++ return { width: even(big.width * k), height: even(big.height * k) };++}++++/**++ * The periods of a video from its size changes (probe's `changes`), up to `end` (the input's end, its own seconds).++ * One size throughout → one period on that size.++ */++function plan(changes, { end } = {}) {++ if (!changes.length) return [];++ const last = Number.isFinite(end) && end > changes[changes.length - 1].t ? end : changes[changes.length - 1].t + 1;++ const runs = changes.map((c, i) => ({ start: c.t, end: i + 1 < changes.length ? changes[i + 1].t : last, width: c.width, height: c.height }));++ // Consecutive runs of one shape are one period.++ let periods = [];++ for (const r of runs) {++ const p = periods[periods.length - 1];++ if (p && sameShape(p.runs[0], r)) { p.runs.push(r); p.end = r.end; } else periods.push({ start: r.start, end: r.end, runs: [r] });++ }++ const merge = (i, j) => { // j = i ± 1: the two become one period++ const [a, b] = i < j ? [periods[i], periods[j]] : [periods[j], periods[i]];++ a.runs.push(...b.runs);++ a.end = b.end;++ periods.splice(periods.indexOf(b), 1);++ };++ const regroup = () => {++ const out = [];++ for (const p of periods) {++ const q = out[out.length - 1];++ if (q && sameShape(canvasOf(q.runs), canvasOf(p.runs))) { q.runs.push(...p.runs); q.end = p.end; } else out.push(p);++ }++ periods = out;++ };++ // A short period (a turn and back, a few seconds at another size) joins its shorter neighbour: its frames are drawn++ // on that canvas with bars rather than given a discontinuity of their own.++ const len = (p) => p.end - p.start;++ const shortest = () => periods.reduce((m, p, i) => (len(p) < len(periods[m]) ? i : m), 0);++ while (periods.length > 1) {++ const i = shortest();++ if (len(periods[i]) >= MIN_PERIOD_S && periods.length <= MAX_PERIODS) break;++ const prev = periods[i - 1];++ const next = periods[i + 1];++ merge(i, !next || (prev && len(prev) <= len(next)) ? i - 1 : i + 1);++ regroup();++ }++ return periods.map((p) => ({ start: p.start, end: p.end, ...canvasOf(p.runs) }));++}++++/** Whether the probe found more than one size: the cut must re-encode on the planned canvases. */++function varies(changes) {++ return changes.length > 1;++}++++/** Fit any frame on a width×height canvas: scaled to fit (never stretched), centred, bars for the rest. */++function fit(width, height) {++ return `scale=${width}:${height}:force_original_aspect_ratio=decrease:force_divisible_by=2,pad=${width}:${height}:(ow-iw)/2:(oh-ih)/2,setsar=1`;++}++++/**++ * Cut `periods` of `source` into `out` as HLS fMP4, one ffmpeg run per period.++ * start the input's start time (probe().start): the first period's output starts at 0++ * encode(p, i) the codec arguments of period i (video encoder + audio codec); `-vf`/`-af` are this module's (the++ * period's trim, its timestamps, the fit on its canvas: a rendition passes periods on its own canvases)++ * audio whether the input has an audio stream (mapped and trimmed with the video)++ * → [{ name, seconds, init }] in order. Throws JobError when a run fails or writes nothing.++ */++async function cut({ source, periods, start = 0, out, segmentSeconds, encode, audio, signal, timeoutMs, niceness = 15 }) {++ const list = [];++ let next = 1;++ for (const [i, p] of periods.entries()) {++ const init = i === 0 ? timeline.INIT_NAME : timeline.shapeInitName(i + 1);++ const from = i === 0 ? start : p.start;++ const edges = [i > 0 ? `start=${secs(p.start)}` : null, i + 1 < periods.length ? `end=${secs(p.end)}` : null].filter(Boolean).join(':');++ const trim = (f) => (edges ? `${f}=${edges},` : '');++ const playlist = path.join(out, `index-${i + 1}.m3u8`);++ // -ss only finds the keyframe at or before the period's start (an absolute timestamp, nothing dropped by++ // ffmpeg itself): the trim on the input's own timestamps is what cuts.++ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-copyts',++ ...(i > 0 ? ['-seek_timestamp', '1', '-noaccurate_seek', '-ss', secs(p.start)] : []), '-i', source.input,++ '-map', '0:v:0', ...(audio ? ['-map', '0:a:0?'] : []),++ '-vf', `${trim('trim')}setpts=PTS-${secs(from)}/TB,${fit(p.width, p.height)}`,++ ...(audio ? ['-af', `${trim('atrim')}asetpts=PTS-${secs(from)}/TB`] : []),++ ...encode(p, i),++ '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',++ ...(from > start ? ['-output_ts_offset', secs(from - start)] : []),++ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0', '-hls_playlist_type', 'vod',++ // Negative composition offsets (CMAF's own rule): a period's first frame is presented at its decode time.++ // Without them the encoder's B-frame delay leaves a gap at each join, which hls.js jumps and Chrome's++ // native HLS stalls on.++ '-hls_segment_type', 'fmp4', '-hls_segment_options', 'movflags=+negative_cts_offsets', '-hls_fmp4_init_filename', init,++ '-start_number', String(next), '-hls_segment_filename', path.join(out, '%06d.m4s'), playlist],++ { signal, timeoutMs, niceness });++ if (signal && signal.aborted) throw signal.reason || new Error('aborted');++ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);++ if (!r.ok) {++ const unsupported = /Unknown encoder|not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);++ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code} (period ${i + 1} of ${periods.length}): ${tail}`, { permanent: unsupported });++ }++ const made = derive.existing(playlist) ? require('./segments').parsePlaylist(fs.readFileSync(playlist, 'utf8')) : [];++ if (!made.length || !derive.existing(path.join(out, init))) {++ throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments for period ${i + 1} of ${periods.length} (${secs(p.start)}-${secs(p.end)} s): ${tail}`);++ }++ for (const s of made) list.push({ ...s, init });++ next += made.length;++ }++ return list;++}++++module.exports = { SAME_SHAPE, MIN_PERIOD_S, MAX_PERIODS, MAX_SIDE, probe, plan, varies, fit, cut, canvasOf };@@@ -806,8 +806,9 @@ const objectVariants = () => _objectVariants || (_objectVariants = require('../t// A segment's copy is the placement router's choice; a packed one (F3.3) is a ranged read of its ~60 s chunk.const HLS_TYPE = 'application/vnd.apple.mpegurl';// A rendition's seq 0 init plus the two edge inits of a materialized clip's source rendition (negative seq,--// docs/materialized-clips.md F3.6); every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.--const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|\d{6,9}\.m4s)$/;++// docs/materialized-clips.md F3.6) and the init of each shape period after the first (init-<n>.mp4, jobs/shapes.js);++// every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.++const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|init-\d{1,4}\.mp4|\d{6,9}\.m4s)$/;const RENDITION_NAME = /^[a-z0-9_-]{1,32}$/;const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in@@@ -45,6 +45,16 @@ const INIT_NAME = 'init.mp4';// negative seq (-1 head, -2 tail) so they never collide with the media segments or the rendition's seq 0 init.const HEAD_INIT = 'init-head.mp4';const TAIL_INIT = 'init-tail.mp4';++// A video whose picture changes shape (a phone turned, a resolution lowered mid-stream) is cut in periods, one init each++// (jobs/shapes.js): the first period's is init.mp4 (seq 0), period n >= 2's init-<n>.mp4 at seq -(100 + n), clear of a++// materialized clip's -1/-2 edges. Its media rows name it in init_name.++const SHAPE_INIT = /^init-(\d{1,4})\.mp4$/;++function shapeInitName(n) { return `init-${n}.mp4`; }++/** The seq of a shape period's init row, or null for any other name. */++function shapeInitSeq(name) {++ const m = SHAPE_INIT.exec(String(name || ''));++ return m && Number(m[1]) >= 2 ? -(100 + Number(m[1])) : null;++}const FIELDS = ['name', 'start_ms', 'end_ms', 'keyframe_ms', 'key', 'local_path', 'durable_provider', 'packed_object_id', 'byte_offset', 'byte_length', 'sha256', 'durability', 'init_name'];function localRoot() { return path.join(path.resolve(config.objects.path), '.timeline'); }@@@ -406,21 +416,25 @@ function masterPlaylist(renditions, { query = '' } = {}) {* A materialized clip's rows carry negative-seq edge inits and per-segment init_name: a virtual clip over it would* decode its edges with the wrong init, so building one is refused (the simplest correct answer, §Schema of* docs/materialized-clips.md). Rows made only of copies of source rows (a clip whose window sits on boundaries) carry-- * no edge init and still work.++ * no edge init and still work. A source cut in shape periods (shapeInitName) is not refused: the clip carries the++ * period inits its segments name, as they are, and each segment keeps its init_name.*/function clipRows(rows, startMs, endMs) {-- if (rows.some((r) => Number(r.seq) < 0 || (r.init_name && r.init_name !== INIT_NAME))) return [];++ const edge = (name) => name === HEAD_INIT || name === TAIL_INIT;++ if (rows.some((r) => (Number(r.seq) < 0 && shapeInitSeq(r.name) == null) || edge(r.init_name))) return [];const init = rows.find((r) => Number(r.seq) === 0);const segs = rows.filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs).map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs })).filter((r) => r.end_ms > r.start_ms).map((r, i) => ({ ...r, seq: i + 1 }));-- return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...segs] : segs;++ const named = new Set(segs.map((r) => r.init_name).filter((n) => n && n !== INIT_NAME));++ const shapeInits = rows.filter((r) => Number(r.seq) < 0 && named.has(r.name)).map((r) => ({ ...r, start_ms: 0, end_ms: 0 }));++ return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...shapeInits, ...segs] : segs;}module.exports = {-- SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName,++ SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName, shapeInitName, shapeInitSeq,list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, namedElsewhere, deleteBytes, removeObject, sweepPendingRemovals, markPacked, durableProvider, locationsOf,mediaPlaylist, masterPlaylist, clipRows,};@@@ -26,6 +26,21 @@ const MAX_CONCURRENT = parseInt(process.env.CLIP_MAX_CONCURRENT_FFMPEG || '2', 1const QUEUE_WAIT_MS = parseInt(process.env.CLIP_QUEUE_WAIT_MS || '1800000', 10);let _active = 0;++/**++ * The -vf of a cut whose window spans a change of the picture's shape (a phone turned, a resolution lowered:++ * jobs/shapes.js): the window's main shape on its canvas, the rest fitted with bars. One encode would otherwise scale++ * every frame to the first size. One size in the window (or a probe that fails) → no filter, the cut as before.++ */++async function shapeFilter(source, ss, dur, isUrl) {++ const shapes = require('../jobs/shapes');++ try {++ const read = await shapes.probe({ input: source, remote: isUrl }, { from: ss, to: ss + dur, timeoutMs: 60000 });++ if (!shapes.varies(read.changes)) return [];++ const main = shapes.plan(read.changes, { end: ss + dur }).reduce((a, b) => (b.end - b.start > a.end - a.start ? b : a));++ return ['-vf', shapes.fit(main.width, main.height)];++ } catch { return []; }++}++function _ffprobe(file) {return new Promise((resolve) => {const ff = spawn('ffprobe', ['-v', 'error', '-print_format', 'json', '-show_format', '-show_streams', file], { stdio: ['ignore', 'pipe', 'ignore'] });@@@ -76,6 +91,7 @@ async function cutClipFile({ source, startTime, duration }) {const outPath = path.join(CLIPS_DIR, filename);_active++;++ const vf = await shapeFilter(source, ss, dur, isUrl);let timedOut = false;let exitCode = null;const _stderrRef = { get: () => '' };@@@ -87,7 +103,7 @@ async function cutClipFile({ source, startTime, duration }) {// an opening of black/frozen frames.const ok = await new Promise((resolve) => {const args = [-- '-y', '-ss', String(ss), '-i', source, '-t', String(dur),++ '-y', '-ss', String(ss), '-i', source, '-t', String(dur), ...vf,'-c:v', 'libvpx', '-b:v', '2000k', '-crf', '18', '-deadline', 'realtime', '-cpu-used', '4','-force_key_frames', 'expr:gte(t,n_forced*2)', '-c:a', 'libopus', '-b:a', '128k','-avoid_negative_ts', 'make_zero', '-f', 'webm', outPath,@@@ -49,8 +49,9 @@ function locationOf(row) {* Plan a clip window over one rendition's rows.* → { originMs, endMs, durationMs, sourceInit, sourceRows, pieces } or null.* A piece is { kind: 'copy', source } or { kind: 'head'|'tail', source, start_ms, end_ms, init } (source milliseconds).-- * sourceRows are the source rows the plan copies and therefore names: the seq 0 init when an interior copy needs it-- * (init_name NULL rows decode with it), plus every copied segment. The pin compares exactly these.++ * sourceRows are the source rows the plan copies and therefore names: each init an interior copy decodes with (the++ * seq 0 init for init_name NULL rows; a shape period's own init, jobs/shapes.js, for the rows that name it), plus every++ * copied segment. The pin compares exactly these. `initOf(row)` is the source init a segment decodes with.*/function planWindow(rows, startMs, endMs) {const segs = rows.filter((r) => Number(r.seq) > 0).sort((a, b) => Number(a.seq) - Number(b.seq));@@@ -88,10 +89,15 @@ function planWindow(rows, startMs, endMs) {const snapEndMs = lastPiece.kind === 'copy' ? Number(lastPiece.source.end_ms) : lastPiece.end_ms;if (!(snapEndMs > originMs)) return null;const sourceInit = rows.find((r) => Number(r.seq) === 0) || null;++ const initOf = (r) => previews.initRowFor(rows, r);const sourceRows = [];-- if (pieces.some((p) => p.kind === 'copy') && sourceInit) sourceRows.push(sourceInit);++ for (const p of pieces) {++ if (p.kind !== 'copy') continue;++ const init = initOf(p.source);++ if (init && !sourceRows.includes(init)) sourceRows.push(init);++ }for (const p of pieces) if (p.kind === 'copy') sourceRows.push(p.source);-- return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces };++ return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces, initOf };}/** A copied source segment row as the clip's own: the location, bytes and times, seq renumbered, times relative. */@@@ -102,17 +108,20 @@ function copiedRow(source, seq, originMs) {row.start_ms = Number(source.start_ms) - originMs;row.end_ms = Number(source.end_ms) - originMs;row.keyframe_ms = source.keyframe_ms == null ? null : Number(source.keyframe_ms) - originMs;-- row.init_name = null; // decodes with the rendition's seq 0 init++ // Decodes with the rendition's seq 0 init, or with the shape period's init it names (copied beside it, same name).++ row.init_name = timeline.shapeInitSeq(source.init_name) != null ? source.init_name : null;return row;}--/** The source's seq 0 init as the clip's own row (seq 0, times 0), or null. */++/** A source init as the clip's own row (times 0): the seq 0 init at seq 0, a shape period's init at its own seq and name. */function copiedInitRow(source) {-- const row = copiedRow(source, 0, 0);-- row.name = timeline.INIT_NAME;++ const shape = timeline.shapeInitSeq(source.name);++ const row = copiedRow(source, shape != null ? shape : 0, 0);++ row.name = shape != null ? source.name : timeline.INIT_NAME;row.start_ms = 0;row.end_ms = 0;row.keyframe_ms = null;++ row.init_name = null;return row;}@@@ -266,7 +275,7 @@ async function materialize({ job, ctx, clip, vod }) {if (piece.kind === 'copy') continue;segments.aborted(ctx);const seq = i + 1;-- const made = await encodeEdge({ ctx, piece, initRow: plan.sourceInit, dir: path.join(dir, `edge-${piece.kind}-${seq}`) });++ const made = await encodeEdge({ ctx, piece, initRow: plan.initOf(piece.source), dir: path.join(dir, `edge-${piece.kind}-${seq}`) });edgeSegments.set(seq, await placeFile({ clipObj, prev, seq, name: timeline.segmentName(seq), file: made.segFile,contentType: 'video/iso.segment', target, remoteConfigured, stagedLocal, stagedRemote, failed, ctx }));edgeInits.set(piece.init, await placeFile({ clipObj, prev, seq: piece.init === timeline.HEAD_INIT ? -1 : -2, name: piece.init, file: made.initFile,@@@ -281,7 +290,7 @@ async function materialize({ job, ctx, clip, vod }) {}if (edgeInits.has(timeline.HEAD_INIT)) rows.push({ seq: -1, name: timeline.HEAD_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.HEAD_INIT), init_name: null });if (edgeInits.has(timeline.TAIL_INIT)) rows.push({ seq: -2, name: timeline.TAIL_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.TAIL_INIT), init_name: null });-- if (plan.pieces.some((p) => p.kind === 'copy') && plan.sourceInit) rows.push(copiedInitRow(plan.sourceInit));++ for (const init of plan.sourceRows.filter((r) => Number(r.seq) <= 0)) rows.push(copiedInitRow(init));rows.sort((a, b) => Number(a.seq) - Number(b.seq));segments.aborted(ctx);await pinRows({ clipObjectId: clip.object_id, sourceObjectId: vod.object_id, rows, sourceRows: plan.sourceRows, jobId: job.id });nnew file mode 100644@@@ -0,0 +1,170 @@++'use strict';++// A recording whose picture changes shape mid-stream (server/jobs/shapes.js): a phone turned from landscape to portrait,++// then WebRTC lowered the resolution. VOD 6017 was cut as one encode, and every portrait frame was stretched to the++// first 1280x720. Now the cut is one period per shape, each on its own canvas with its own init segment, and the++// playlists, virtual clips and materialized clips carry the right init for each segment.++const assert = require('assert');++const fs = require('fs');++const os = require('os');++const path = require('path');++const { spawnSync } = require('child_process');++++(async () => {++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-shapes-'));++ const dir = (name) => { const p = path.join(tmp, name); fs.mkdirSync(p, { recursive: true }); return p; };++ Object.assign(process.env, {++ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'), THUMBNAILS_PATH: dir('thumbnails'),++ PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), MEDIA_HLS_ENABLED: '1', MEDIA_CMAF_TRANSCODE: '1',++ MEDIA_UPLOAD_MIN_FREE_MB: '0', MEDIA_INVARIANT_SCAN_HOURS: '0',++ });++ for (const p of ['B2', 'R2']) for (const k of ['ENDPOINT', 'BUCKET', 'KEY_ID', 'APP_KEY', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']) process.env[`MEDIA_${p}_${k}`] = '';++ try {++ const shapes = require('../server/jobs/shapes');++ const timeline = require('../server/objects/timeline');++ const at = (list) => list.map(([t, width, height]) => ({ t, width, height }));++++ // ── Planning ──++ // VOD 6017's own keyframe sizes: 16:9 for 16 minutes, then portrait at three sizes. Two periods, the portrait one on++ // the largest portrait size; a lowered resolution is not a period of its own.++ const vod6017 = at([[0.2, 1280, 720], [976.243, 720, 1280], [1079.895, 540, 960], [1144.988, 360, 640], [1481.092, 540, 960],++ [1546.278, 360, 640], [1756.995, 540, 960], [1942.652, 360, 640], [2148.22, 540, 960], [2228.501, 360, 640],++ [2373.997, 540, 960], [2474.381, 720, 1280], [2569.747, 540, 960]]);++ assert.deepStrictEqual(shapes.plan(vod6017, { end: 2592.2 }), [++ { start: 0.2, end: 976.243, width: 1280, height: 720 },++ { start: 976.243, end: 2592.2, width: 720, height: 1280 },++ ]);++ // A two-second turn and back joins its neighbour (drawn with bars), and the two landscape runs become one.++ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [100, 720, 1280], [102, 1280, 720], [300, 960, 540]]), { end: 400 }),++ [{ start: 0, end: 400, width: 1280, height: 720 }]);++ // A canvas keeps its shape under MAX_SIDE, on even sides.++ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [50, 2161, 3841]]), { end: 100 }),++ [{ start: 0, end: 50, width: 1280, height: 720 }, { start: 50, end: 100, width: 1080, height: 1920 }]);++ // A phone turned every few seconds for an hour still has at most MAX_PERIODS periods.++ const flips = [];++ for (let t = 0; t < 3600; t += 6) flips.push([t, t % 12 ? 720 : 1280, t % 12 ? 1280 : 720]);++ const many = shapes.plan(at(flips), { end: 3600 });++ assert.ok(many.length <= shapes.MAX_PERIODS, `${many.length} periods`);++ assert.strictEqual(many[0].start, 0);++ assert.strictEqual(many[many.length - 1].end, 3600);++ for (let i = 1; i < many.length; i++) assert.strictEqual(many[i].start, many[i - 1].end, 'periods tile the video');++ assert.strictEqual(shapes.varies(at([[0, 640, 360]])), false);++ assert.strictEqual(shapes.varies(at([[0, 640, 360], [9, 320, 180]])), true);++ assert.match(shapes.fit(720, 1280), /^scale=720:1280:force_original_aspect_ratio=decrease.*pad=720:1280:\(ow-iw\)\/2:\(oh-ih\)\/2,setsar=1$/);++++ // ── Names ──++ assert.strictEqual(timeline.shapeInitName(2), 'init-2.mp4');++ assert.strictEqual(timeline.shapeInitSeq('init-2.mp4'), -102);++ for (const name of ['init.mp4', 'init-1.mp4', 'init-head.mp4', 'init-tail.mp4', '000002.m4s', null]) assert.strictEqual(timeline.shapeInitSeq(name), null, String(name));++++ // ── The cut, the playlist and the clips (needs ffmpeg with libx264, libvpx, libopus, as cmaf-transcode) ──++ if (spawnSync('ffmpeg', ['-version']).status !== 0 || spawnSync('ffprobe', ['-version']).status !== 0) {++ console.log('✅ shape periods: planning and names (cut skipped: ffmpeg not found)');++ return;++ }++ const ffmpeg = (...args) => {++ const r = spawnSync('ffmpeg', ['-v', 'error', '-y', ...args], { encoding: 'utf8' });++ assert.strictEqual(r.status, 0, r.stderr);++ };++ const frames = (file) => {++ const r = spawnSync('ffprobe', ['-v', 'error', '-select_streams', 'v:0', '-show_entries', 'frame=width,height', '-of', 'csv=p=0', file], { encoding: 'utf8' });++ assert.strictEqual(r.status, 0, r.stderr);++ return r.stdout.trim().split('\n').filter(Boolean).map((l) => l.split(',').slice(0, 2).join(','));++ };++ // The recorder's stream copy: one VP8 track whose keyframes change size, the header naming the first (320x180).++ // The landscape part is 6 s, the portrait one 6 s at 180x320 then 4 s at 90x160, all at 10 fps; the copy starts at++ // 0.2 s, as a recording's first keyframe often does.++ const part = (name, src, size, seconds, freq) => {++ const file = path.join(tmp, name);++ ffmpeg('-f', 'lavfi', '-i', `${src}=duration=${seconds}:size=${size}:rate=10`, '-f', 'lavfi', '-i', `sine=duration=${seconds}:frequency=${freq}`,++ '-c:v', 'libvpx', '-deadline', 'realtime', '-cpu-used', '8', '-g', '10', '-c:a', 'libopus', '-shortest', file);++ return file;++ };++ const parts = [part('a.webm', 'testsrc', '320x180', 6, 440), part('b.webm', 'testsrc2', '180x320', 6, 660), part('c.webm', 'testsrc2', '90x160', 4, 880)];++ fs.writeFileSync(path.join(tmp, 'list.txt'), parts.map((p) => `file '${p}'`).join('\n'));++ const mixed = path.join(tmp, 'mixed.webm');++ ffmpeg('-f', 'concat', '-safe', '0', '-i', path.join(tmp, 'list.txt'), '-c', 'copy', '-output_ts_offset', '0.2', mixed);++ const sourceFrames = frames(mixed).length;++++ const read = await shapes.probe({ input: mixed, remote: false });++ assert.deepStrictEqual(read.changes.map((c) => `${c.width}x${c.height}`), ['320x180', '180x320', '90x160']);++ assert.ok(Math.abs(read.changes[1].t - read.start - 6) < 0.1, `the turn at ~6 s (${read.changes[1].t - read.start})`);++++ const db = require('../server/db/database');++ const model = require('../server/objects/model');++ const cmaf = require('../server/jobs/cmaf');++ await db.upsertApp({ app_id: 'live', api_key: 'shapes-test' });++ const object = async (file, mime) => {++ const id = await model.createObject({ app_id: 'live', owner_user_id: 1, visibility: 'public', lifecycle_status: 'ready',++ kind: 'file', mime_type: mime, size_bytes: fs.statSync(file).size });++ await model.upsertLocation(id, { provider: 'local', key: file, state: 'present', size_bytes: fs.statSync(file).size, verified: true });++ return id;++ };++ const run = async (id, suffix) => await cmaf.spec.run({ id: `mjob_${suffix}`, app_id: 'live', object_id: id, params: { segment_seconds: 2 } },++ { signal: new AbortController().signal });++ const vod = await object(mixed, 'video/webm');++ const made = await run(vod, 'shapes');++ assert.deepStrictEqual([made.video, made.audio, made.shapes], ['h264', 'aac', 2]);++++ const rows = await timeline.list(vod, 'source');++ const init2 = rows.find((r) => r.name === 'init-2.mp4');++ assert.ok(init2, 'the portrait period has its own init');++ assert.strictEqual(Number(init2.seq), -102);++ assert.strictEqual(rows.filter((r) => Number(r.seq) <= 0).length, 2, 'two inits, one per period');++ const segs = rows.filter((r) => Number(r.seq) > 0);++ const first = segs.filter((r) => !r.init_name);++ const second = segs.filter((r) => r.init_name === 'init-2.mp4');++ assert.ok(first.length && second.length && first.length + second.length === segs.length);++ assert.ok(Number(first[first.length - 1].seq) < Number(second[0].seq), 'the periods follow each other');++ // Every frame is on its period's canvas, never the other's: the portrait part is 180x320 (its 90x160 seconds++ // scaled up into it), and no frame of either shape lands in the other's period.++ const initFile = (r) => (r.init_name ? init2 : rows.find((x) => Number(x.seq) === 0)).local_path;++ const sizes = (list) => {++ const joined = path.join(tmp, 'joined.mp4');++ const out = [];++ for (const r of list) {++ fs.writeFileSync(joined, Buffer.concat([fs.readFileSync(initFile(r)), fs.readFileSync(r.local_path)]));++ out.push(...frames(joined));++ }++ return out;++ };++ const landscape = sizes(first);++ const portrait = sizes(second);++ assert.ok(landscape.every((s) => s === '320,180'), `landscape period: ${[...new Set(landscape)]}`);++ assert.ok(portrait.every((s) => s === '180,320'), `portrait period: ${[...new Set(portrait)]}`);++ assert.ok(Math.abs(landscape.length - 60) <= 1, `${landscape.length} landscape frames, 60 sent`);++ assert.ok(Math.abs(landscape.length + portrait.length - sourceFrames) <= 1, `${landscape.length + portrait.length} frames of ${sourceFrames}`);++ // The timeline tiles the whole video.++ assert.ok(Math.abs(Number(segs[segs.length - 1].end_ms) - 16000) < 400, `ends at ${segs[segs.length - 1].end_ms} ms`);++++ // The playlist switches init with a discontinuity between the periods, exactly once.++ const playlist = timeline.mediaPlaylist(rows);++ assert.strictEqual((playlist.match(/#EXT-X-DISCONTINUITY/g) || []).length, 1);++ assert.match(playlist, /#EXT-X-MAP:URI="init\.mp4"[\s\S]*#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);++++ // A virtual clip across the turn carries init-2 and each segment's init; one inside the portrait part opens on it.++ const turnMs = Number(second[0].start_ms);++ const across = timeline.clipRows(rows, turnMs - 2500, turnMs + 2500);++ assert.ok(across.some((r) => r.name === 'init-2.mp4' && Number(r.seq) === -102));++ assert.match(timeline.mediaPlaylist(across), /#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);++ const inside = timeline.clipRows(rows, turnMs + 1000, turnMs + 5000);++ assert.match(timeline.mediaPlaylist(inside), /^#EXTM3U[\s\S]*?#EXT-X-MAP:URI="init-2\.mp4"/);++ assert.ok(!/DISCONTINUITY/.test(timeline.mediaPlaylist(inside)));++ // A materialized clip's own rows over the source (the edges keep refusing a virtual clip, as before).++ assert.deepStrictEqual(timeline.clipRows([...rows, { seq: -1, name: timeline.HEAD_INIT }], 0, 4000), []);++ const materialize = require('../server/vod/clip-materialize');++ const plan = materialize.planWindow(rows, Number(second[0].start_ms), Number(second[second.length - 1].end_ms));++ assert.ok(plan.sourceRows.some((r) => r.name === 'init-2.mp4'), 'the pin names the period init it copies');++ assert.ok(!plan.sourceRows.some((r) => Number(r.seq) === 0), 'and not the first period\'s');++ assert.strictEqual(plan.initOf(second[0]).name, 'init-2.mp4');++ const spanning = materialize.planWindow(rows, Number(first[0].start_ms), Number(second[second.length - 1].end_ms));++ assert.deepStrictEqual(spanning.sourceRows.filter((r) => Number(r.seq) <= 0).map((r) => r.name).sort(), ['init-2.mp4', 'init.mp4']);++++ // One size throughout keeps the cut as it was: a stream copy of an H.264 MP4.++ const plain = path.join(tmp, 'plain.mp4');++ ffmpeg('-f', 'lavfi', '-i', 'testsrc=duration=5:size=160x90:rate=10', '-f', 'lavfi', '-i', 'sine=duration=5',++ '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-shortest', plain);++ const copied = await run(await object(plain, 'video/mp4'), 'plain');++ assert.deepStrictEqual([copied.video, copied.audio, copied.shapes], ['copy', 'copy', 1]);++ console.log('✅ shape periods: planning, one init per shape, frames on their canvas, playlists and clips');++ } finally { fs.rmSync(tmp, { recursive: true, force: true }); }++})().catch((err) => { console.error(err); process.exitCode = 1; });
@@ -110,6 +110,23 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
@@ -110,6 +110,23 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
honours the abort signal (which stops an upload in flight) and, before the rows commit, removes the files and keys it
honours the abort signal (which stops an upload in flight) and, before the rows commit, removes the files and keys it
staged; the stream-copy or duration-based transcode budget is honoured and its work directory removed. The master
staged; the stream-copy or duration-based transcode budget is honoured and its work directory removed. The master
playlist reads CODECS from the written init segment, so an encoded source advertises its output codecs.
playlist reads CODECS from the written init segment, so an encoded source advertises its output codecs.
+
+ **A picture that changes shape.** A recording is a stream copy of what the broadcaster sent, so when a phone turns
+ from landscape to portrait, or WebRTC lowers 720p to 360p on a weak network, its frames change size at a keyframe
+ while the file's header still names the first size. One encode scales every later frame to that first size (VOD 6017:
+ a portrait street stretched into 16:9), and a copy carries frames its init does not describe. With
+ `MEDIA_CMAF_TRANSCODE=1`, `object.cmaf` first reads each keyframe's size (`server/jobs/shapes.js`, a keyframe-only
+ decode). When there is more than one, it cuts **one period per shape**: sizes within ~4% of one aspect ratio are one
+ shape (a lowered resolution is not a new period), and each period is drawn on its shape's largest size (long side at
+ most 1920), with frames of another size fitted, never stretched. A period under 5 s joins a neighbour, and there are
+ at most 40. Each period is its own ffmpeg run on the input's own timestamps, with its own init: the first period's is
+ `init.mp4` (seq 0), period n ≥ 2's is `init-<n>.mp4`, a row at seq −(100+n) that its segments name in `init_name`
+ (migration 0014). The media playlist then puts `#EXT-X-DISCONTINUITY` + `#EXT-X-MAP` between periods, so a player
+ shows each part at its own shape, the way an adaptive stream switches renditions. Virtual and materialized clips carry
+ the period inits their segments name, `rendition.create` cuts the same periods at its rung's height, and a clip cut
+ (`server/vod/clip-cutter.js`) whose window spans a change is drawn on the window's main shape. One size throughout
+ keeps the cut exactly as before. `scripts/recut-shapes.js` finds the recordings cut before this (a dry run by default)
+ and queues their re-cut.
Behind **`MEDIA_HLS_ENABLED`** (off by default: the job is refused with `409 media.hls.disabled` and the routes 404),
Behind **`MEDIA_HLS_ENABLED`** (off by default: the job is refused with `409 media.hls.disabled` and the routes 404),
`GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8` and `/o/:id/source/{init.mp4,NNNNNN.m4s}` serve the timeline
`GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8` and `/o/:id/source/{init.mp4,NNNNNN.m4s}` serve the timeline
under the same check as `GET /o/:id` (private and sandbox objects only with the object's `?exp&sig`, carried onto
under the same check as `GET /o/:id` (private and sandbox objects only with the object's `?exp&sig`, carried onto
new file mode 100644
new file mode 100644
@@ -0,0 +1,9 @@
@@ -0,0 +1,9 @@
+-- phase: expand
+-- A video whose picture changes shape mid-stream (a phone turned, a resolution lowered) is cut in periods, one init
+-- segment each (server/jobs/shapes.js): the first period's is init.mp4 at seq 0, period n >= 2's is init-<n>.mp4 at seq
+-- -(100 + n), named by its media rows' init_name (0005). This widens 0005's seq CHECK to admit those names. Additive
+-- only: an older release reads the rows with init_name already (its playlists put a discontinuity before each init) and
+-- never writes one.
+ALTER TABLE media_timeline DROP CONSTRAINT IF EXISTS media_timeline_seq_check;
+ALTER TABLE media_timeline ADD CONSTRAINT media_timeline_seq_check
+ CHECK (seq >= 0 OR name IN ('init-head.mp4', 'init-tail.mp4') OR name ~ '^init-[0-9]{1,4}\.mp4$');
new file mode 100755
new file mode 100755
@@ -0,0 +1,87 @@
@@ -0,0 +1,87 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * Re-cut the recordings whose source timeline was cut as one shape although their picture changes shape mid-stream
+ * (server/jobs/shapes.js, docs/media-fabric.md §3: VOD 6017 played its portrait part stretched into 16:9), for the
+ * operator.
+ *
+ * node scripts/recut-shapes.js [--days 30] [--limit 200] [--app live] [--ids 6017,6020] [--remote] [--apply] [--json]
+ * Dry run (the default): for each finished VOD of the window, newest first, whose object has a source timeline
+ * with one init and a local file (--remote also reads B2/R2 copies, which is egress), reads its keyframe sizes
+ * and reports the ones with more than one, with the periods a re-cut would make. Changes nothing.
+ * --apply queues object.cmaf for each of those (idempotency key object.cmaf:<object>:shapes, so a second run
+ * answers with the same job). The running service's worker cuts them on its next poll, one at a time in the heavy
+ * lane at nice 15; the new rows replace the old ones in one transaction, and playback never stops.
+ * (the database is the service's: DATABASE_URL, and MEDIA_HLS_ENABLED / MEDIA_CMAF_TRANSCODE, from its env file)
+ *
+ * Exit codes: 0 done, 1 error, 2 bad usage.
+ */
+
+(async () => {
+ await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the service
+ const args = process.argv.slice(2);
+ const has = (name) => args.includes(`--${name}`);
+ const arg = (name, d = null) => { const i = args.indexOf(`--${name}`); return i >= 0 && args[i + 1] && !args[i + 1].startsWith('--') ? args[i + 1] : d; };
+ const usage = (msg) => { console.error(`recut-shapes: ${msg}`); process.exit(2); };
+ for (const flag of ['days', 'limit', 'app', 'ids']) if (has(flag) && !arg(flag)) usage(`--${flag} needs a value`);
+ const APPLY = has('apply');
+ const days = Number(arg('days', 30));
+ const limit = Number(arg('limit', 200));
+ if (!(days > 0) || !(limit > 0)) usage('--days and --limit are positive numbers');
+ const ids = arg('ids') ? arg('ids').split(',').map(Number).filter((n) => n > 0) : null;
+
+ const config = require('../server/config');
+ const db = require('../server/db/database');
+ const model = require('../server/objects/model');
+ const timeline = require('../server/objects/timeline');
+ const derive = require('../server/jobs/derive');
+ const shapes = require('../server/jobs/shapes');
+ if (!config.hls.enabled || !config.hls.transcode) usage('the re-cut needs MEDIA_HLS_ENABLED=1 and MEDIA_CMAF_TRANSCODE=1 (the service\'s env file)');
+ if (APPLY) await require('../server/events').initWriter();
+
+ const since = new Date(Date.now() - days * 86400000).toISOString().slice(0, 19).replace('T', ' ');
+ const vods = ids
+ ? await db.all('SELECT id, app_id, object_id, title FROM vods WHERE id = ANY(?) ORDER BY id DESC', [ids])
+ : await db.all(`SELECT id, app_id, object_id, title FROM vods WHERE is_recording = 0 AND object_id IS NOT NULL AND created_at >= ?
+ ${has('app') ? 'AND app_id = ?' : ''} ORDER BY id DESC LIMIT ?`, has('app') ? [since, arg('app'), limit] : [since, limit]);
+ const report = [];
+ for (const v of vods) {
+ const out = { vod: v.id, object: v.object_id, title: v.title };
+ report.push(out);
+ const obj = v.object_id && await model.getObject(v.object_id);
+ if (!obj || obj.lifecycle_status !== 'ready') { out.skip = 'no ready object'; continue; }
+ const rows = await timeline.list(obj.id, timeline.SOURCE);
+ if (!rows.some((r) => Number(r.seq) > 0)) { out.skip = 'no source timeline (its first cut reads the shapes)'; continue; }
+ if (rows.some((r) => timeline.shapeInitSeq(r.name) != null)) { out.skip = 'already cut in periods'; continue; }
+ const source = await derive.resolveSource(obj);
+ if (!source) { out.skip = 'no bytes'; continue; }
+ if (source.remote && !has('remote')) { out.skip = 'only on B2/R2 (--remote reads it)'; continue; }
+ try {
+ const read = await shapes.probe(source, { timeoutMs: derive.budgetMs(obj.size_bytes) });
+ if (!shapes.varies(read.changes)) { out.shapes = 1; continue; }
+ const info = await derive.probeStreams(source);
+ const periods = shapes.plan(read.changes, { end: read.start + info.duration });
+ out.sizes = read.changes.length;
+ out.periods = periods.map((p) => `${p.width}x${p.height} from ${(p.start - read.start).toFixed(1)} s`);
+ if (APPLY) {
+ const params = require('../server/jobs/cmaf').spec.validate({ appId: obj.app_id, obj, params: {} });
+ const r = await require('../server/jobs/queue').enqueue({ appId: obj.app_id, type: 'object.cmaf', objectId: obj.id, params,
+ dedupeActive: true, idempotencyKey: `object.cmaf:${obj.id}:shapes`, createdBy: `operator:${require('os').userInfo().username}` });
+ out.job = r.job ? r.job.id : null;
+ out.queued = !!r.created;
+ }
+ } catch (err) { out.error = err.message.slice(0, 200); }
+ }
+ if (has('json')) console.log(JSON.stringify(report, null, 2));
+ else {
+ for (const r of report) {
+ const what = r.skip ? `skip: ${r.skip}` : r.error ? `error: ${r.error}` : r.periods
+ ? `${r.sizes} sizes → ${r.periods.join(', ')}${r.job ? ` ${r.queued ? 'queued' : 'already queued'} ${r.job}` : ''}`
+ : 'one shape';
+ console.log(`vod ${r.vod} ${what}`);
+ }
+ const found = report.filter((r) => r.periods).length;
+ console.log(`${report.length} checked, ${found} change shape${APPLY ? `, ${report.filter((r) => r.queued).length} queued` : found ? ' (dry run: --apply queues their re-cut)' : ''}`);
+ }
+ process.exit(report.some((r) => r.error) ? 1 : 0);
+})().catch((err) => { console.error(err); process.exit(1); });
@@ -19,6 +19,9 @@
@@ -19,6 +19,9 @@
* rendition.create (F4).
* rendition.create (F4).
* Idempotent: a rerun makes the same bytes (bitexact), reuses a row whose sha256 matches and its location as it is, and
* Idempotent: a rerun makes the same bytes (bitexact), reuses a row whose sha256 matches and its location as it is, and
* uploads only what is not durable yet. A run that aborts or fails before the commit removes its staged files and keys.
* uploads only what is not durable yet. A run that aborts or fails before the commit removes its staged files and keys.
+ * A picture that changes shape mid-stream (a phone turned, a resolution lowered: jobs/shapes.js) is re-encoded with
+ * MEDIA_CMAF_TRANSCODE in one period per shape, each on its own canvas with its own init segment, so the playlist
+ * switches shape between periods instead of stretching every frame to the first size; `shapes` in the result counts them.
* Refused: MEDIA_HLS_ENABLED off, a source that is not ready or not media (permanent), less free disk than the source
* Refused: MEDIA_HLS_ENABLED off, a source that is not ready or not media (permanent), less free disk than the source
* size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent), a clip whose file was
* size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent), a clip whose file was
* replaced (a trim or re-cut) while it ran (permanent, media.source.changed: nothing is committed).
* replaced (a trim or re-cut) while it ran (permanent, media.source.changed: nothing is committed).
@@ -31,6 +34,7 @@ const config = require('../config');
@@ -31,6 +34,7 @@ const config = require('../config');
const timeline = require('../objects/timeline');
const timeline = require('../objects/timeline');
const derive = require('./derive');
const derive = require('./derive');
const segments = require('./segments');
const segments = require('./segments');
+const shapes = require('./shapes');
const { JobError } = require('./queue');
const { JobError } = require('./queue');
const DEFAULT_SEGMENT_SECONDS = 4;
const DEFAULT_SEGMENT_SECONDS = 4;
@@ -39,6 +43,13 @@ function transcodeBudgetMs(duration) {
@@ -39,6 +43,13 @@ function transcodeBudgetMs(duration) {
return Math.min(13 * 3600 * 1000, Math.max(10 * 60000, Math.round(duration * 3000 + 10 * 60000)));
return Math.min(13 * 3600 * 1000, Math.max(10 * 60000, Math.round(duration * 3000 + 10 * 60000)));
}
}
+/** The H.264 re-encode: a keyframe at every segment boundary, frame times as they are. */
+function x264Args(segmentSeconds) {
+ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),
+ '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough'];
+}
+const AAC_ARGS = ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'];
+
function disabled() {
function disabled() {
return new JobError('media.hls.disabled', 'Segment-native video is off on this server (MEDIA_HLS_ENABLED)', { permanent: true, status: 409 });
return new JobError('media.hls.disabled', 'Segment-native video is off on this server (MEDIA_HLS_ENABLED)', { permanent: true, status: 409 });
}
}
@@ -56,6 +67,25 @@ function validate({ obj, params }) {
@@ -56,6 +67,25 @@ function validate({ obj, params }) {
return p;
return p;
}
}
+/** The whole input as one HLS fMP4 rendition (one shape): → the produced segments, [{ name, seconds }]. */
+async function cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness }) {
+ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
+ '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
+ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',
+ '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,
+ '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],
+ { signal: ctx.signal, timeoutMs, niceness });
+ segments.aborted(ctx);
+ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
+ if (!r.ok) {
+ const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
+ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });
+ }
+ const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
+ if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
+ return list;
+}
+
async function run(job, ctx) {
async function run(job, ctx) {
if (!config.hls.enabled) throw disabled();
if (!config.hls.enabled) throw disabled();
const src = await derive.loadSource(job);
const src = await derive.loadSource(job);
@@ -88,11 +118,28 @@ async function run(job, ctx) {
@@ -88,11 +118,28 @@ async function run(job, ctx) {
throw new JobError('too_long_to_transcode', `Source duration ${info.duration}s exceeds MEDIA_CMAF_TRANSCODE_MAX_SECONDS`, { permanent: true });
throw new JobError('too_long_to_transcode', `Source duration ${info.duration}s exceeds MEDIA_CMAF_TRANSCODE_MAX_SECONDS`, { permanent: true });
}
}
timeoutMs = transcodeBudgetMs(info.duration);
timeoutMs = transcodeBudgetMs(info.duration);
- codecArgs = video === 'h264'
- ? ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),
- '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough']
- : ['-c:v', 'copy'];
- codecArgs.push(...(audio === 'aac' ? ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'] : ['-c:a', 'copy']));
+ codecArgs = video === 'h264' ? x264Args(segmentSeconds) : ['-c:v', 'copy'];
+ codecArgs.push(...(audio === 'aac' ? AAC_ARGS : ['-c:a', 'copy']));
+ }
+ }
+ // Frames of more than one size (shapes.js): one period per shape, each re-encoded on its own canvas. A copy would
+ // carry frames its init does not describe, and one encode scales every frame to the first size (a portrait part
+ // stretched into 16:9). A probe that fails leaves the cut as it was.
+ let periods = null;
+ let shapeStart = 0;
+ if (config.hls.transcode && info && info.video && info.duration > 0 && info.duration <= config.hls.transcodeMaxSeconds) {
+ let read = null;
+ try { read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }); }
+ catch (err) { segments.aborted(ctx); console.warn(`[Cmaf] ${job.id}: shape probe failed (${err.message.slice(0, 200)}); cut as one shape`); }
+ segments.aborted(ctx);
+ if (read && shapes.varies(read.changes)) {
+ periods = shapes.plan(read.changes, { end: read.start + info.duration });
+ shapeStart = read.start;
+ video = 'h264';
+ if (audio !== 'none') audio = 'aac';
+ timeoutMs = transcodeBudgetMs(info.duration);
+ codecArgs = [...x264Args(segmentSeconds), ...(audio === 'aac' ? AAC_ARGS : [])];
+ console.log(`[Cmaf] ${job.id}: ${read.changes.length} frame sizes → ${periods.length} period(s): ${periods.map((p) => `${p.width}x${p.height}`).join(', ')}`);
}
}
}
}
segments.checkDisk(Number(src.size_bytes) || 0);
segments.checkDisk(Number(src.size_bytes) || 0);
@@ -101,20 +148,10 @@ async function run(job, ctx) {
@@ -101,20 +148,10 @@ async function run(job, ctx) {
try {
try {
const out = path.join(dir, 'out');
const out = path.join(dir, 'out');
fs.mkdirSync(out, { recursive: true });
fs.mkdirSync(out, { recursive: true });
- const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
- '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
- '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',
- '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,
- '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],
- { signal: ctx.signal, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });
- segments.aborted(ctx);
- const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
- if (!r.ok) {
- const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
- throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });
- }
- const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
- if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
+ const list = periods
+ ? await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds, audio: audio !== 'none', encode: () => codecArgs,
+ signal: ctx.signal, timeoutMs, niceness: 15 })
+ : await cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });
const check = version == null ? null : async () => {
const check = version == null ? null : async () => {
if (await derive.fileVersion(src, { lock: true }) !== version) {
if (await derive.fileVersion(src, { lock: true }) !== version) {
@@ -127,7 +164,7 @@ async function run(job, ctx) {
@@ -127,7 +164,7 @@ async function run(job, ctx) {
return {
return {
source_id: src.id, rendition, segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
source_id: src.id, rendition, segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio,
+ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio, shapes: periods ? periods.length : 1,
};
};
} finally {
} finally {
derive.cleanupWork(job.id);
derive.cleanupWork(job.id);
@@ -14,6 +14,9 @@
@@ -14,6 +14,9 @@
* result { …, skipped: true, reason } when the source is already this rendition's height or smaller (a still is not
* result { …, skipped: true, reason } when the source is already this rendition's height or smaller (a still is not
* upscaled), or the rows already exist (skipped: false, already: true)
* upscaled), or the rows already exist (skipped: false, already: true)
*
*
+ * A source whose picture changes shape (jobs/shapes.js) is cut as its source timeline is: one period per shape, each on
+ * its own canvas scaled to the rung's height, with its own init segment.
+ *
* Refused: MEDIA_HLS_ENABLED or MEDIA_RENDITIONS off, a source that is not ready or not media (permanent), an object
* Refused: MEDIA_HLS_ENABLED or MEDIA_RENDITIONS off, a source that is not ready or not media (permanent), an object
* under a hold (retried), a source with no video stream (permanent), less free disk than the source size plus
* under a hold (retried), a source with no video stream (permanent), less free disk than the source size plus
* MEDIA_UPLOAD_MIN_FREE_MB (retried later). Idempotent: a re-run whose bytes match reuses the rows and locations, and
* MEDIA_UPLOAD_MIN_FREE_MB (retried later). Idempotent: a re-run whose bytes match reuses the rows and locations, and
@@ -31,6 +34,7 @@ const config = require('../config');
@@ -31,6 +34,7 @@ const config = require('../config');
const timeline = require('../objects/timeline');
const timeline = require('../objects/timeline');
const derive = require('./derive');
const derive = require('./derive');
const segments = require('./segments');
const segments = require('./segments');
+const shapes = require('./shapes');
const { JobError } = require('./queue');
const { JobError } = require('./queue');
const DEFAULT_SEGMENT_SECONDS = 4;
const DEFAULT_SEGMENT_SECONDS = 4;
@@ -95,11 +99,22 @@ async function run(job, ctx) {
@@ -95,11 +99,22 @@ async function run(job, ctx) {
const info = await require('../vod/media-tools').probeVodInfo(source.input);
const info = await require('../vod/media-tools').probeVodInfo(source.input);
const video = (info.streams || []).find((s) => s.codec_type === 'video');
const video = (info.streams || []).find((s) => s.codec_type === 'video');
if (!video) throw new JobError('no_video', 'The source has no video stream to rendition', { permanent: true });
if (!video) throw new JobError('no_video', 'The source has no video stream to rendition', { permanent: true });
- const height = Number(video.height) || 0;
+ // Frames of more than one size: the periods the source timeline was cut in, each brought down to this rung.
+ let planned = null;
+ let shapeStart = 0;
+ const read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }).catch(() => null);
+ segments.aborted(ctx);
+ if (read && shapes.varies(read.changes)) {
+ planned = shapes.plan(read.changes, { end: read.start + (Number(info.duration) || 0) });
+ shapeStart = read.start;
+ }
+ const height = planned ? Math.max(...planned.map((p) => p.height)) : Number(video.height) || 0;
// A source already this rendition's height or smaller is not upscaled (a re-encode would only cost bytes).
// A source already this rendition's height or smaller is not upscaled (a re-encode would only cost bytes).
if (height > 0 && height <= targetHeight) {
if (height > 0 && height <= targetHeight) {
return { source_id: src.id, rendition, target_height: targetHeight, skipped: true, reason: `the source is ${height}p, not above ${rendition}`, segments: 0, duration_ms: 0, bytes: 0 };
return { source_id: src.id, rendition, target_height: targetHeight, skipped: true, reason: `the source is ${height}p, not above ${rendition}`, segments: 0, duration_ms: 0, bytes: 0 };
}
}
+ const periods = planned && planned.map((p) => (p.height <= targetHeight ? p
+ : { ...p, width: Math.max(2, Math.round(p.width * targetHeight / p.height / 2) * 2), height: targetHeight }));
segments.checkDisk(Number(src.size_bytes) || 0);
segments.checkDisk(Number(src.size_bytes) || 0);
const srcRows = await timeline.segments(src.id, timeline.SOURCE);
const srcRows = await timeline.segments(src.id, timeline.SOURCE);
const segSec = segmentSeconds(job, srcRows);
const segSec = segmentSeconds(job, srcRows);
@@ -108,9 +123,33 @@ async function run(job, ctx) {
@@ -108,9 +123,33 @@ async function run(job, ctx) {
// and a source that long is packed long before it is re-encoded.
// and a source that long is packed long before it is re-encoded.
const boundaries = srcRows.length && srcRows.length <= 2000 ? srcRows.map((r) => (Number(r.start_ms) / 1000).toFixed(3)) : [];
const boundaries = srcRows.length && srcRows.length <= 2000 ? srcRows.map((r) => (Number(r.start_ms) / 1000).toFixed(3)) : [];
const dir = derive.workDir(job.id);
const dir = derive.workDir(job.id);
+ const publish = async (list) => {
+ const published = await segments.publishRendition({ job, ctx, src, rendition, out: path.join(dir, 'out'), list, label: 'Rendition' });
+ return {
+ source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,
+ segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
+ durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
+ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, shapes: periods ? periods.length : 1,
+ };
+ };
try {
try {
const out = path.join(dir, 'out');
const out = path.join(dir, 'out');
fs.mkdirSync(out, { recursive: true });
fs.mkdirSync(out, { recursive: true });
+ if (periods) {
+ // Each period's keyframes at the source boundaries inside it, in the period's own time (its output starts at 0).
+ const axis = (p, i) => (i === 0 ? 0 : p.start - shapeStart);
+ const list = await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds: segSec,
+ audio: (info.streams || []).some((st) => st.codec_type === 'audio'),
+ encode: (p, i) => {
+ const from = axis(p, i);
+ const to = i + 1 < periods.length ? axis(periods[i + 1], i + 1) : Infinity;
+ const own = boundaries.map(Number).filter((b) => b >= from && b < to).map((b) => (b - from).toFixed(3));
+ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',
+ '-c:a', 'aac', '-b:a', '128k', '-ac', '2', ...(own.length ? ['-force_key_frames', own.join(',')] : [])];
+ },
+ signal: ctx.signal, timeoutMs: encodeBudgetMs(src.size_bytes), niceness: 0 });
+ return await publish(list);
+ }
const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
'-map', '0:v:0', '-map', '0:a:0?',
'-map', '0:v:0', '-map', '0:a:0?',
'-vf', `scale=-2:${targetHeight}`, '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',
'-vf', `scale=-2:${targetHeight}`, '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',
@@ -130,13 +169,7 @@ async function run(job, ctx) {
@@ -130,13 +169,7 @@ async function run(job, ctx) {
const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
- const published = await segments.publishRendition({ job, ctx, src, rendition, out, list, label: 'Rendition' });
- return {
- source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,
- segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
- durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true,
- };
+ return await publish(list);
} finally {
} finally {
derive.cleanupWork(job.id);
derive.cleanupWork(job.id);
}
}
@@ -6,8 +6,10 @@
@@ -6,8 +6,10 @@
* turns that output into media_timeline rows and publishes them:
* turns that output into media_timeline rows and publishes them:
*
*
* parsePlaylist ffmpeg's index.m3u8 → [{ name, seconds }], in order.
* parsePlaylist ffmpeg's index.m3u8 → [{ name, seconds }], in order.
- * producedFiles those plus the init segment → [{ seq, name, file, start_ms, end_ms, keyframe_ms }], times on the
- * object's own axis (cumulative EXTINF, so each segment ends where the next starts).
+ * producedFiles those plus the init segment(s) → [{ seq, name, file, start_ms, end_ms, keyframe_ms, init_name }],
+ * times on the object's own axis (cumulative EXTINF, so each segment ends where the next starts). A
+ * video cut in shape periods (jobs/shapes.js) has one init per period: the later ones are rows with
+ * negative seq that their segments name in init_name.
* publishRendition hash each file, place it content-addressed on this node (timeline.localPathFor/keyFor, versioned
* publishRendition hash each file, place it content-addressed on this node (timeline.localPathFor/keyFor, versioned
* by the sha prefix) and — through the placement router's durable provider (timeline.durableProvider,
* by the sha prefix) and — through the placement router's durable provider (timeline.durableProvider,
* purpose `durable`) — on B2/R2, then commit every row of the rendition in ONE transaction
* purpose `durable`) — on B2/R2, then commit every row of the rendition in ONE transaction
@@ -45,16 +47,27 @@ function parsePlaylist(text) {
@@ -45,16 +47,27 @@ function parsePlaylist(text) {
return out;
return out;
}
}
-/** The produced files as timeline rows in the making: the init segment (seq 0) then each media segment, in order. */
+/**
+ * The produced files as timeline rows in the making: the init segment (seq 0), the init of each later shape period
+ * (a list entry's `init`, jobs/shapes.js: negative seq, named in its segments' init_name), then each media segment, in
+ * order.
+ */
function producedFiles(list) {
function producedFiles(list) {
const files = [{ seq: 0, name: timeline.INIT_NAME, start_ms: 0, end_ms: 0, keyframe_ms: null }];
const files = [{ seq: 0, name: timeline.INIT_NAME, start_ms: 0, end_ms: 0, keyframe_ms: null }];
+ const media = [];
let cum = 0;
let cum = 0;
list.forEach((s, i) => {
list.forEach((s, i) => {
const start = Math.round(cum * 1000);
const start = Math.round(cum * 1000);
cum += s.seconds;
cum += s.seconds;
- files.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start });
+ const init = s.init && s.init !== timeline.INIT_NAME ? s.init : null;
+ if (init && !files.some((f) => f.name === init)) {
+ const seq = timeline.shapeInitSeq(init);
+ if (seq == null) throw new JobError('ffmpeg_failed', `${init} is not a shape period's init`);
+ files.push({ seq, name: init, start_ms: 0, end_ms: 0, keyframe_ms: null });
+ }
+ media.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start, init_name: init });
});
});
- return files;
+ return files.concat(media);
}
}
function checkDisk(needBytes) {
function checkDisk(needBytes) {
@@ -122,7 +135,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
@@ -122,7 +135,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
const match = !!old && old.sha256 === sha;
const match = !!old && old.sha256 === sha;
if (match && old.packed_object_id) {
if (match && old.packed_object_id) {
// Already packed into a chunk (object.pack): the same bytes, so the row and its chunk stay as they are.
// Already packed into a chunk (object.pack): the same bytes, so the row and its chunk stay as they are.
- rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : old[k]])));
+ rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : k === 'init_name' ? f.init_name || null : old[k]])));
continue;
continue;
}
}
// Content-addressed: the same bytes always land at the same versioned location, so a rerun rewrites
// Content-addressed: the same bytes always land at the same versioned location, so a rerun rewrites
@@ -143,7 +156,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
@@ -143,7 +156,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
if (!durable && target) {
if (!durable && target) {
try {
try {
if (!vodStorage.providerAvailable(target)) throw new Error(`${target} is unavailable`);
if (!vodStorage.providerAvailable(target)) throw new Error(`${target} is unavailable`);
- await vodStorage.uploadFile(target, key, localPath, f.seq === 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });
+ await vodStorage.uploadFile(target, key, localPath, f.seq <= 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });
stagedRemote.push({ provider: target, key });
stagedRemote.push({ provider: target, key });
const back = await vodStorage.sha256Object(target, key);
const back = await vodStorage.sha256Object(target, key);
if (!back || back.sha256 !== sha || Number(back.size) !== size) throw new Error(`${target}:${key} failed size/sha256 readback`);
if (!back || back.sha256 !== sha || Number(back.size) !== size) throw new Error(`${target}:${key} failed size/sha256 readback`);
@@ -161,7 +174,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
@@ -161,7 +174,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
rows.push({
rows.push({
seq: f.seq, name: f.name, start_ms: f.start_ms, end_ms: f.end_ms, keyframe_ms: f.keyframe_ms, key, local_path: localPath,
seq: f.seq, name: f.name, start_ms: f.start_ms, end_ms: f.end_ms, keyframe_ms: f.keyframe_ms, key, local_path: localPath,
durable_provider: durable, packed_object_id: null, byte_offset: null, byte_length: size, sha256: sha,
durable_provider: durable, packed_object_id: null, byte_offset: null, byte_length: size, sha256: sha,
- durability: durable ? 'durable' : 'local',
+ durability: durable ? 'durable' : 'local', init_name: f.init_name || null,
});
});
}
}
aborted(ctx);
aborted(ctx);
new file mode 100644
new file mode 100644
@@ -0,0 +1,213 @@
@@ -0,0 +1,213 @@
+/**
+ * OpenVibe.Media — the picture's shapes over a video, and cutting it in one period per shape (docs/media-fabric.md §3).
+ *
+ * A recording is a stream copy of what the broadcaster sent. When their phone turns, or WebRTC lowers 720p to 360p on a
+ * weak network, the frames change size at a keyframe while the file's header still names the first size. One encode
+ * then scales every later frame to that first size (VOD 6017: a portrait street stretched into 16:9), and a stream copy
+ * carries frames its init segment does not describe. So a cut that must re-encode reads the shapes first:
+ *
+ * probe(source) → { start, changes: [{ t, width, height }] }: the input's start time and each keyframe whose
+ * size differs from the one before (one decode of the keyframes only; t in the input's own
+ * seconds). `from`/`to` read only that window (a clip).
+ * plan(changes, { end }) → [{ start, end, width, height }]: the periods. Sizes within SAME_SHAPE of one aspect ratio
+ * are one shape (a lowered resolution is not a new period); each period is drawn on the canvas
+ * of its shape's largest size (long side at most MAX_SIDE). A period shorter than MIN_PERIOD_S
+ * joins a neighbour, and there are at most MAX_PERIODS.
+ * fit(width, height) the filter that puts any frame on a width×height canvas without stretching it: scaled to fit,
+ * centred, bars for the rest. A frame of the period's own shape fills the canvas exactly.
+ * cut({ … }) one ffmpeg run per period into `out`, each with its own init segment (the first period's is
+ * init.mp4; period n ≥ 2's is timeline.shapeInitName(n)) → the produced segments in order,
+ * [{ name, seconds, init }] for segments.publishRendition. The playlist puts
+ * #EXT-X-DISCONTINUITY + #EXT-X-MAP between periods (timeline.mediaPlaylist), so a player shows
+ * each part at its own shape, the way an adaptive stream switches renditions.
+ *
+ * Periods are cut on the input's own timestamps (-copyts, trim/atrim at the period's edges, the next period sought to its
+ * first keyframe), so no frame of one shape lands in the other's period; each period's timestamps continue where the
+ * last one's ended, with no gap at the join (negative composition offsets: Chrome's native HLS stalls on a gap there).
+ */
+'use strict';
+
+const path = require('path');
+const fs = require('fs');
+const { spawn } = require('child_process');
+const derive = require('./derive');
+const timeline = require('../objects/timeline');
+const { JobError } = require('./queue');
+
+const SAME_SHAPE = 0.04; // |ln(aspect a / aspect b)|: 1280x720 and 426x240 are one shape, 16:9 and 4:3 are not
+const MIN_PERIOD_S = 5;
+const MAX_PERIODS = 40;
+const MAX_SIDE = 1920;
+
+const aspect = (s) => s.width / s.height;
+const sameShape = (a, b) => Math.abs(Math.log(aspect(a) / aspect(b))) < SAME_SHAPE;
+const even = (x) => Math.max(2, Math.round(x / 2) * 2);
+const secs = (t) => Number(t).toFixed(6).replace(/\.?0+$/, '') || '0';
+
+/** Each keyframe's size, read with ffprobe (keyframes only). Resolves { start, changes }. */
+function probe(source, { signal, timeoutMs = 30 * 60000, from = null, to = null } = {}) {
+ return new Promise((resolve, reject) => {
+ const args = ['-v', 'error', ...derive.inputArgs(source),
+ ...(from != null ? ['-read_intervals', `${secs(from)}%${secs(to)}`] : []),
+ '-select_streams', 'v:0', '-skip_frame', 'nokey',
+ '-show_entries', 'frame=best_effort_timestamp_time,width,height:format=start_time', '-of', 'csv=p=0', source.input];
+ let proc;
+ try { proc = spawn('ffprobe', args, { stdio: ['ignore', 'pipe', 'pipe'] }); } catch (err) { return reject(err); }
+ try { require('os').setPriority(proc.pid, 10); } catch { /* best effort */ }
+ const changes = [];
+ let start = null;
+ let rest = '';
+ let stderr = '';
+ let done = false;
+ const line = (l) => {
+ const f = l.trim().split(',');
+ if (f.length === 1) { if (f[0] && f[0] !== 'N/A') start = Number(f[0]); return; } // format=start_time
+ const [t, width, height] = f.map(Number);
+ if (!Number.isFinite(t) || !(width > 0) || !(height > 0)) return;
+ const last = changes[changes.length - 1];
+ if (!last || last.width !== width || last.height !== height) changes.push({ t, width, height });
+ };
+ const kill = () => { try { proc.kill('SIGKILL'); } catch { /* gone */ } };
+ const timer = setTimeout(kill, timeoutMs);
+ const finish = (err, value) => {
+ if (done) return;
+ done = true;
+ clearTimeout(timer);
+ if (signal) signal.removeEventListener('abort', kill);
+ if (err) reject(err); else resolve(value);
+ };
+ if (signal) { if (signal.aborted) kill(); else signal.addEventListener('abort', kill, { once: true }); }
+ proc.stdout.on('data', (d) => {
+ const lines = (rest + d).split('\n');
+ rest = lines.pop();
+ for (const l of lines) line(l);
+ });
+ proc.stderr.on('data', (d) => { stderr = (stderr + d).slice(-4000); });
+ proc.on('error', (err) => finish(err));
+ proc.on('close', (code) => {
+ if (rest) line(rest);
+ if (code !== 0) return finish(new Error(`ffprobe exited ${code}: ${stderr.trim().slice(-300)}`));
+ finish(null, { start: Number.isFinite(start) ? start : (changes[0] ? changes[0].t : 0), changes });
+ });
+ });
+}
+
+/** The shape a set of runs mostly shows, and its largest size: { width, height } of the canvas. */
+function canvasOf(runs) {
+ const groups = [];
+ for (const r of runs) {
+ let g = groups.find((x) => sameShape(x.lead, r));
+ if (!g) groups.push(g = { lead: r, seconds: 0, runs: [] });
+ g.seconds += r.end - r.start;
+ g.runs.push(r);
+ }
+ const main = groups.reduce((a, b) => (b.seconds > a.seconds ? b : a));
+ const big = main.runs.reduce((a, b) => (b.width * b.height > a.width * a.height ? b : a));
+ const k = Math.min(1, MAX_SIDE / Math.max(big.width, big.height));
+ return { width: even(big.width * k), height: even(big.height * k) };
+}
+
+/**
+ * The periods of a video from its size changes (probe's `changes`), up to `end` (the input's end, its own seconds).
+ * One size throughout → one period on that size.
+ */
+function plan(changes, { end } = {}) {
+ if (!changes.length) return [];
+ const last = Number.isFinite(end) && end > changes[changes.length - 1].t ? end : changes[changes.length - 1].t + 1;
+ const runs = changes.map((c, i) => ({ start: c.t, end: i + 1 < changes.length ? changes[i + 1].t : last, width: c.width, height: c.height }));
+ // Consecutive runs of one shape are one period.
+ let periods = [];
+ for (const r of runs) {
+ const p = periods[periods.length - 1];
+ if (p && sameShape(p.runs[0], r)) { p.runs.push(r); p.end = r.end; } else periods.push({ start: r.start, end: r.end, runs: [r] });
+ }
+ const merge = (i, j) => { // j = i ± 1: the two become one period
+ const [a, b] = i < j ? [periods[i], periods[j]] : [periods[j], periods[i]];
+ a.runs.push(...b.runs);
+ a.end = b.end;
+ periods.splice(periods.indexOf(b), 1);
+ };
+ const regroup = () => {
+ const out = [];
+ for (const p of periods) {
+ const q = out[out.length - 1];
+ if (q && sameShape(canvasOf(q.runs), canvasOf(p.runs))) { q.runs.push(...p.runs); q.end = p.end; } else out.push(p);
+ }
+ periods = out;
+ };
+ // A short period (a turn and back, a few seconds at another size) joins its shorter neighbour: its frames are drawn
+ // on that canvas with bars rather than given a discontinuity of their own.
+ const len = (p) => p.end - p.start;
+ const shortest = () => periods.reduce((m, p, i) => (len(p) < len(periods[m]) ? i : m), 0);
+ while (periods.length > 1) {
+ const i = shortest();
+ if (len(periods[i]) >= MIN_PERIOD_S && periods.length <= MAX_PERIODS) break;
+ const prev = periods[i - 1];
+ const next = periods[i + 1];
+ merge(i, !next || (prev && len(prev) <= len(next)) ? i - 1 : i + 1);
+ regroup();
+ }
+ return periods.map((p) => ({ start: p.start, end: p.end, ...canvasOf(p.runs) }));
+}
+
+/** Whether the probe found more than one size: the cut must re-encode on the planned canvases. */
+function varies(changes) {
+ return changes.length > 1;
+}
+
+/** Fit any frame on a width×height canvas: scaled to fit (never stretched), centred, bars for the rest. */
+function fit(width, height) {
+ return `scale=${width}:${height}:force_original_aspect_ratio=decrease:force_divisible_by=2,pad=${width}:${height}:(ow-iw)/2:(oh-ih)/2,setsar=1`;
+}
+
+/**
+ * Cut `periods` of `source` into `out` as HLS fMP4, one ffmpeg run per period.
+ * start the input's start time (probe().start): the first period's output starts at 0
+ * encode(p, i) the codec arguments of period i (video encoder + audio codec); `-vf`/`-af` are this module's (the
+ * period's trim, its timestamps, the fit on its canvas: a rendition passes periods on its own canvases)
+ * audio whether the input has an audio stream (mapped and trimmed with the video)
+ * → [{ name, seconds, init }] in order. Throws JobError when a run fails or writes nothing.
+ */
+async function cut({ source, periods, start = 0, out, segmentSeconds, encode, audio, signal, timeoutMs, niceness = 15 }) {
+ const list = [];
+ let next = 1;
+ for (const [i, p] of periods.entries()) {
+ const init = i === 0 ? timeline.INIT_NAME : timeline.shapeInitName(i + 1);
+ const from = i === 0 ? start : p.start;
+ const edges = [i > 0 ? `start=${secs(p.start)}` : null, i + 1 < periods.length ? `end=${secs(p.end)}` : null].filter(Boolean).join(':');
+ const trim = (f) => (edges ? `${f}=${edges},` : '');
+ const playlist = path.join(out, `index-${i + 1}.m3u8`);
+ // -ss only finds the keyframe at or before the period's start (an absolute timestamp, nothing dropped by
+ // ffmpeg itself): the trim on the input's own timestamps is what cuts.
+ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-copyts',
+ ...(i > 0 ? ['-seek_timestamp', '1', '-noaccurate_seek', '-ss', secs(p.start)] : []), '-i', source.input,
+ '-map', '0:v:0', ...(audio ? ['-map', '0:a:0?'] : []),
+ '-vf', `${trim('trim')}setpts=PTS-${secs(from)}/TB,${fit(p.width, p.height)}`,
+ ...(audio ? ['-af', `${trim('atrim')}asetpts=PTS-${secs(from)}/TB`] : []),
+ ...encode(p, i),
+ '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
+ ...(from > start ? ['-output_ts_offset', secs(from - start)] : []),
+ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0', '-hls_playlist_type', 'vod',
+ // Negative composition offsets (CMAF's own rule): a period's first frame is presented at its decode time.
+ // Without them the encoder's B-frame delay leaves a gap at each join, which hls.js jumps and Chrome's
+ // native HLS stalls on.
+ '-hls_segment_type', 'fmp4', '-hls_segment_options', 'movflags=+negative_cts_offsets', '-hls_fmp4_init_filename', init,
+ '-start_number', String(next), '-hls_segment_filename', path.join(out, '%06d.m4s'), playlist],
+ { signal, timeoutMs, niceness });
+ if (signal && signal.aborted) throw signal.reason || new Error('aborted');
+ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
+ if (!r.ok) {
+ const unsupported = /Unknown encoder|not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
+ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code} (period ${i + 1} of ${periods.length}): ${tail}`, { permanent: unsupported });
+ }
+ const made = derive.existing(playlist) ? require('./segments').parsePlaylist(fs.readFileSync(playlist, 'utf8')) : [];
+ if (!made.length || !derive.existing(path.join(out, init))) {
+ throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments for period ${i + 1} of ${periods.length} (${secs(p.start)}-${secs(p.end)} s): ${tail}`);
+ }
+ for (const s of made) list.push({ ...s, init });
+ next += made.length;
+ }
+ return list;
+}
+
+module.exports = { SAME_SHAPE, MIN_PERIOD_S, MAX_PERIODS, MAX_SIDE, probe, plan, varies, fit, cut, canvasOf };
@@ -806,8 +806,9 @@ const objectVariants = () => _objectVariants || (_objectVariants = require('../t
@@ -806,8 +806,9 @@ const objectVariants = () => _objectVariants || (_objectVariants = require('../t
// A segment's copy is the placement router's choice; a packed one (F3.3) is a ranged read of its ~60 s chunk.
// A segment's copy is the placement router's choice; a packed one (F3.3) is a ranged read of its ~60 s chunk.
const HLS_TYPE = 'application/vnd.apple.mpegurl';
const HLS_TYPE = 'application/vnd.apple.mpegurl';
// A rendition's seq 0 init plus the two edge inits of a materialized clip's source rendition (negative seq,
// A rendition's seq 0 init plus the two edge inits of a materialized clip's source rendition (negative seq,
-// docs/materialized-clips.md F3.6); every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.
-const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|\d{6,9}\.m4s)$/;
+// docs/materialized-clips.md F3.6) and the init of each shape period after the first (init-<n>.mp4, jobs/shapes.js);
+// every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.
+const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|init-\d{1,4}\.mp4|\d{6,9}\.m4s)$/;
const RENDITION_NAME = /^[a-z0-9_-]{1,32}$/;
const RENDITION_NAME = /^[a-z0-9_-]{1,32}$/;
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in
// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in
@@ -45,6 +45,16 @@ const INIT_NAME = 'init.mp4';
@@ -45,6 +45,16 @@ const INIT_NAME = 'init.mp4';
// negative seq (-1 head, -2 tail) so they never collide with the media segments or the rendition's seq 0 init.
// negative seq (-1 head, -2 tail) so they never collide with the media segments or the rendition's seq 0 init.
const HEAD_INIT = 'init-head.mp4';
const HEAD_INIT = 'init-head.mp4';
const TAIL_INIT = 'init-tail.mp4';
const TAIL_INIT = 'init-tail.mp4';
+// A video whose picture changes shape (a phone turned, a resolution lowered mid-stream) is cut in periods, one init each
+// (jobs/shapes.js): the first period's is init.mp4 (seq 0), period n >= 2's init-<n>.mp4 at seq -(100 + n), clear of a
+// materialized clip's -1/-2 edges. Its media rows name it in init_name.
+const SHAPE_INIT = /^init-(\d{1,4})\.mp4$/;
+function shapeInitName(n) { return `init-${n}.mp4`; }
+/** The seq of a shape period's init row, or null for any other name. */
+function shapeInitSeq(name) {
+ const m = SHAPE_INIT.exec(String(name || ''));
+ return m && Number(m[1]) >= 2 ? -(100 + Number(m[1])) : null;
+}
const FIELDS = ['name', 'start_ms', 'end_ms', 'keyframe_ms', 'key', 'local_path', 'durable_provider', 'packed_object_id', 'byte_offset', 'byte_length', 'sha256', 'durability', 'init_name'];
const FIELDS = ['name', 'start_ms', 'end_ms', 'keyframe_ms', 'key', 'local_path', 'durable_provider', 'packed_object_id', 'byte_offset', 'byte_length', 'sha256', 'durability', 'init_name'];
function localRoot() { return path.join(path.resolve(config.objects.path), '.timeline'); }
function localRoot() { return path.join(path.resolve(config.objects.path), '.timeline'); }
@@ -406,21 +416,25 @@ function masterPlaylist(renditions, { query = '' } = {}) {
@@ -406,21 +416,25 @@ function masterPlaylist(renditions, { query = '' } = {}) {
* A materialized clip's rows carry negative-seq edge inits and per-segment init_name: a virtual clip over it would
* A materialized clip's rows carry negative-seq edge inits and per-segment init_name: a virtual clip over it would
* decode its edges with the wrong init, so building one is refused (the simplest correct answer, §Schema of
* decode its edges with the wrong init, so building one is refused (the simplest correct answer, §Schema of
* docs/materialized-clips.md). Rows made only of copies of source rows (a clip whose window sits on boundaries) carry
* docs/materialized-clips.md). Rows made only of copies of source rows (a clip whose window sits on boundaries) carry
- * no edge init and still work.
+ * no edge init and still work. A source cut in shape periods (shapeInitName) is not refused: the clip carries the
+ * period inits its segments name, as they are, and each segment keeps its init_name.
*/
*/
function clipRows(rows, startMs, endMs) {
function clipRows(rows, startMs, endMs) {
- if (rows.some((r) => Number(r.seq) < 0 || (r.init_name && r.init_name !== INIT_NAME))) return [];
+ const edge = (name) => name === HEAD_INIT || name === TAIL_INIT;
+ if (rows.some((r) => (Number(r.seq) < 0 && shapeInitSeq(r.name) == null) || edge(r.init_name))) return [];
const init = rows.find((r) => Number(r.seq) === 0);
const init = rows.find((r) => Number(r.seq) === 0);
const segs = rows
const segs = rows
.filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs)
.filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs)
.map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs }))
.map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs }))
.filter((r) => r.end_ms > r.start_ms)
.filter((r) => r.end_ms > r.start_ms)
.map((r, i) => ({ ...r, seq: i + 1 }));
.map((r, i) => ({ ...r, seq: i + 1 }));
- return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...segs] : segs;
+ const named = new Set(segs.map((r) => r.init_name).filter((n) => n && n !== INIT_NAME));
+ const shapeInits = rows.filter((r) => Number(r.seq) < 0 && named.has(r.name)).map((r) => ({ ...r, start_ms: 0, end_ms: 0 }));
+ return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...shapeInits, ...segs] : segs;
}
}
module.exports = {
module.exports = {
- SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName,
+ SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName, shapeInitName, shapeInitSeq,
list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, namedElsewhere, deleteBytes, removeObject, sweepPendingRemovals, markPacked, durableProvider, locationsOf,
list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, namedElsewhere, deleteBytes, removeObject, sweepPendingRemovals, markPacked, durableProvider, locationsOf,
mediaPlaylist, masterPlaylist, clipRows,
mediaPlaylist, masterPlaylist, clipRows,
};
};
@@ -26,6 +26,21 @@ const MAX_CONCURRENT = parseInt(process.env.CLIP_MAX_CONCURRENT_FFMPEG || '2', 1
@@ -26,6 +26,21 @@ const MAX_CONCURRENT = parseInt(process.env.CLIP_MAX_CONCURRENT_FFMPEG || '2', 1
const QUEUE_WAIT_MS = parseInt(process.env.CLIP_QUEUE_WAIT_MS || '1800000', 10);
const QUEUE_WAIT_MS = parseInt(process.env.CLIP_QUEUE_WAIT_MS || '1800000', 10);
let _active = 0;
let _active = 0;
+/**
+ * The -vf of a cut whose window spans a change of the picture's shape (a phone turned, a resolution lowered:
+ * jobs/shapes.js): the window's main shape on its canvas, the rest fitted with bars. One encode would otherwise scale
+ * every frame to the first size. One size in the window (or a probe that fails) → no filter, the cut as before.
+ */
+async function shapeFilter(source, ss, dur, isUrl) {
+ const shapes = require('../jobs/shapes');
+ try {
+ const read = await shapes.probe({ input: source, remote: isUrl }, { from: ss, to: ss + dur, timeoutMs: 60000 });
+ if (!shapes.varies(read.changes)) return [];
+ const main = shapes.plan(read.changes, { end: ss + dur }).reduce((a, b) => (b.end - b.start > a.end - a.start ? b : a));
+ return ['-vf', shapes.fit(main.width, main.height)];
+ } catch { return []; }
+}
+
function _ffprobe(file) {
function _ffprobe(file) {
return new Promise((resolve) => {
return new Promise((resolve) => {
const ff = spawn('ffprobe', ['-v', 'error', '-print_format', 'json', '-show_format', '-show_streams', file], { stdio: ['ignore', 'pipe', 'ignore'] });
const ff = spawn('ffprobe', ['-v', 'error', '-print_format', 'json', '-show_format', '-show_streams', file], { stdio: ['ignore', 'pipe', 'ignore'] });
@@ -76,6 +91,7 @@ async function cutClipFile({ source, startTime, duration }) {
@@ -76,6 +91,7 @@ async function cutClipFile({ source, startTime, duration }) {
const outPath = path.join(CLIPS_DIR, filename);
const outPath = path.join(CLIPS_DIR, filename);
_active++;
_active++;
+ const vf = await shapeFilter(source, ss, dur, isUrl);
let timedOut = false;
let timedOut = false;
let exitCode = null;
let exitCode = null;
const _stderrRef = { get: () => '' };
const _stderrRef = { get: () => '' };
@@ -87,7 +103,7 @@ async function cutClipFile({ source, startTime, duration }) {
@@ -87,7 +103,7 @@ async function cutClipFile({ source, startTime, duration }) {
// an opening of black/frozen frames.
// an opening of black/frozen frames.
const ok = await new Promise((resolve) => {
const ok = await new Promise((resolve) => {
const args = [
const args = [
- '-y', '-ss', String(ss), '-i', source, '-t', String(dur),
+ '-y', '-ss', String(ss), '-i', source, '-t', String(dur), ...vf,
'-c:v', 'libvpx', '-b:v', '2000k', '-crf', '18', '-deadline', 'realtime', '-cpu-used', '4',
'-c:v', 'libvpx', '-b:v', '2000k', '-crf', '18', '-deadline', 'realtime', '-cpu-used', '4',
'-force_key_frames', 'expr:gte(t,n_forced*2)', '-c:a', 'libopus', '-b:a', '128k',
'-force_key_frames', 'expr:gte(t,n_forced*2)', '-c:a', 'libopus', '-b:a', '128k',
'-avoid_negative_ts', 'make_zero', '-f', 'webm', outPath,
'-avoid_negative_ts', 'make_zero', '-f', 'webm', outPath,
@@ -49,8 +49,9 @@ function locationOf(row) {
@@ -49,8 +49,9 @@ function locationOf(row) {
* Plan a clip window over one rendition's rows.
* Plan a clip window over one rendition's rows.
* → { originMs, endMs, durationMs, sourceInit, sourceRows, pieces } or null.
* → { originMs, endMs, durationMs, sourceInit, sourceRows, pieces } or null.
* A piece is { kind: 'copy', source } or { kind: 'head'|'tail', source, start_ms, end_ms, init } (source milliseconds).
* A piece is { kind: 'copy', source } or { kind: 'head'|'tail', source, start_ms, end_ms, init } (source milliseconds).
- * sourceRows are the source rows the plan copies and therefore names: the seq 0 init when an interior copy needs it
- * (init_name NULL rows decode with it), plus every copied segment. The pin compares exactly these.
+ * sourceRows are the source rows the plan copies and therefore names: each init an interior copy decodes with (the
+ * seq 0 init for init_name NULL rows; a shape period's own init, jobs/shapes.js, for the rows that name it), plus every
+ * copied segment. The pin compares exactly these. `initOf(row)` is the source init a segment decodes with.
*/
*/
function planWindow(rows, startMs, endMs) {
function planWindow(rows, startMs, endMs) {
const segs = rows.filter((r) => Number(r.seq) > 0).sort((a, b) => Number(a.seq) - Number(b.seq));
const segs = rows.filter((r) => Number(r.seq) > 0).sort((a, b) => Number(a.seq) - Number(b.seq));
@@ -88,10 +89,15 @@ function planWindow(rows, startMs, endMs) {
@@ -88,10 +89,15 @@ function planWindow(rows, startMs, endMs) {
const snapEndMs = lastPiece.kind === 'copy' ? Number(lastPiece.source.end_ms) : lastPiece.end_ms;
const snapEndMs = lastPiece.kind === 'copy' ? Number(lastPiece.source.end_ms) : lastPiece.end_ms;
if (!(snapEndMs > originMs)) return null;
if (!(snapEndMs > originMs)) return null;
const sourceInit = rows.find((r) => Number(r.seq) === 0) || null;
const sourceInit = rows.find((r) => Number(r.seq) === 0) || null;
+ const initOf = (r) => previews.initRowFor(rows, r);
const sourceRows = [];
const sourceRows = [];
- if (pieces.some((p) => p.kind === 'copy') && sourceInit) sourceRows.push(sourceInit);
+ for (const p of pieces) {
+ if (p.kind !== 'copy') continue;
+ const init = initOf(p.source);
+ if (init && !sourceRows.includes(init)) sourceRows.push(init);
+ }
for (const p of pieces) if (p.kind === 'copy') sourceRows.push(p.source);
for (const p of pieces) if (p.kind === 'copy') sourceRows.push(p.source);
- return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces };
+ return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces, initOf };
}
}
/** A copied source segment row as the clip's own: the location, bytes and times, seq renumbered, times relative. */
/** A copied source segment row as the clip's own: the location, bytes and times, seq renumbered, times relative. */
@@ -102,17 +108,20 @@ function copiedRow(source, seq, originMs) {
@@ -102,17 +108,20 @@ function copiedRow(source, seq, originMs) {
row.start_ms = Number(source.start_ms) - originMs;
row.start_ms = Number(source.start_ms) - originMs;
row.end_ms = Number(source.end_ms) - originMs;
row.end_ms = Number(source.end_ms) - originMs;
row.keyframe_ms = source.keyframe_ms == null ? null : Number(source.keyframe_ms) - originMs;
row.keyframe_ms = source.keyframe_ms == null ? null : Number(source.keyframe_ms) - originMs;
- row.init_name = null; // decodes with the rendition's seq 0 init
+ // Decodes with the rendition's seq 0 init, or with the shape period's init it names (copied beside it, same name).
+ row.init_name = timeline.shapeInitSeq(source.init_name) != null ? source.init_name : null;
return row;
return row;
}
}
-/** The source's seq 0 init as the clip's own row (seq 0, times 0), or null. */
+/** A source init as the clip's own row (times 0): the seq 0 init at seq 0, a shape period's init at its own seq and name. */
function copiedInitRow(source) {
function copiedInitRow(source) {
- const row = copiedRow(source, 0, 0);
- row.name = timeline.INIT_NAME;
+ const shape = timeline.shapeInitSeq(source.name);
+ const row = copiedRow(source, shape != null ? shape : 0, 0);
+ row.name = shape != null ? source.name : timeline.INIT_NAME;
row.start_ms = 0;
row.start_ms = 0;
row.end_ms = 0;
row.end_ms = 0;
row.keyframe_ms = null;
row.keyframe_ms = null;
+ row.init_name = null;
return row;
return row;
}
}
@@ -266,7 +275,7 @@ async function materialize({ job, ctx, clip, vod }) {
@@ -266,7 +275,7 @@ async function materialize({ job, ctx, clip, vod }) {
if (piece.kind === 'copy') continue;
if (piece.kind === 'copy') continue;
segments.aborted(ctx);
segments.aborted(ctx);
const seq = i + 1;
const seq = i + 1;
- const made = await encodeEdge({ ctx, piece, initRow: plan.sourceInit, dir: path.join(dir, `edge-${piece.kind}-${seq}`) });
+ const made = await encodeEdge({ ctx, piece, initRow: plan.initOf(piece.source), dir: path.join(dir, `edge-${piece.kind}-${seq}`) });
edgeSegments.set(seq, await placeFile({ clipObj, prev, seq, name: timeline.segmentName(seq), file: made.segFile,
edgeSegments.set(seq, await placeFile({ clipObj, prev, seq, name: timeline.segmentName(seq), file: made.segFile,
contentType: 'video/iso.segment', target, remoteConfigured, stagedLocal, stagedRemote, failed, ctx }));
contentType: 'video/iso.segment', target, remoteConfigured, stagedLocal, stagedRemote, failed, ctx }));
edgeInits.set(piece.init, await placeFile({ clipObj, prev, seq: piece.init === timeline.HEAD_INIT ? -1 : -2, name: piece.init, file: made.initFile,
edgeInits.set(piece.init, await placeFile({ clipObj, prev, seq: piece.init === timeline.HEAD_INIT ? -1 : -2, name: piece.init, file: made.initFile,
@@ -281,7 +290,7 @@ async function materialize({ job, ctx, clip, vod }) {
@@ -281,7 +290,7 @@ async function materialize({ job, ctx, clip, vod }) {
}
}
if (edgeInits.has(timeline.HEAD_INIT)) rows.push({ seq: -1, name: timeline.HEAD_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.HEAD_INIT), init_name: null });
if (edgeInits.has(timeline.HEAD_INIT)) rows.push({ seq: -1, name: timeline.HEAD_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.HEAD_INIT), init_name: null });
if (edgeInits.has(timeline.TAIL_INIT)) rows.push({ seq: -2, name: timeline.TAIL_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.TAIL_INIT), init_name: null });
if (edgeInits.has(timeline.TAIL_INIT)) rows.push({ seq: -2, name: timeline.TAIL_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.TAIL_INIT), init_name: null });
- if (plan.pieces.some((p) => p.kind === 'copy') && plan.sourceInit) rows.push(copiedInitRow(plan.sourceInit));
+ for (const init of plan.sourceRows.filter((r) => Number(r.seq) <= 0)) rows.push(copiedInitRow(init));
rows.sort((a, b) => Number(a.seq) - Number(b.seq));
rows.sort((a, b) => Number(a.seq) - Number(b.seq));
segments.aborted(ctx);
segments.aborted(ctx);
await pinRows({ clipObjectId: clip.object_id, sourceObjectId: vod.object_id, rows, sourceRows: plan.sourceRows, jobId: job.id });
await pinRows({ clipObjectId: clip.object_id, sourceObjectId: vod.object_id, rows, sourceRows: plan.sourceRows, jobId: job.id });
new file mode 100644
new file mode 100644
@@ -0,0 +1,170 @@
@@ -0,0 +1,170 @@
+'use strict';
+// A recording whose picture changes shape mid-stream (server/jobs/shapes.js): a phone turned from landscape to portrait,
+// then WebRTC lowered the resolution. VOD 6017 was cut as one encode, and every portrait frame was stretched to the
+// first 1280x720. Now the cut is one period per shape, each on its own canvas with its own init segment, and the
+// playlists, virtual clips and materialized clips carry the right init for each segment.
+const assert = require('assert');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const { spawnSync } = require('child_process');
+
+(async () => {
+ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-shapes-'));
+ const dir = (name) => { const p = path.join(tmp, name); fs.mkdirSync(p, { recursive: true }); return p; };
+ Object.assign(process.env, {
+ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'), THUMBNAILS_PATH: dir('thumbnails'),
+ PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), MEDIA_HLS_ENABLED: '1', MEDIA_CMAF_TRANSCODE: '1',
+ MEDIA_UPLOAD_MIN_FREE_MB: '0', MEDIA_INVARIANT_SCAN_HOURS: '0',
+ });
+ for (const p of ['B2', 'R2']) for (const k of ['ENDPOINT', 'BUCKET', 'KEY_ID', 'APP_KEY', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']) process.env[`MEDIA_${p}_${k}`] = '';
+ try {
+ const shapes = require('../server/jobs/shapes');
+ const timeline = require('../server/objects/timeline');
+ const at = (list) => list.map(([t, width, height]) => ({ t, width, height }));
+
+ // ── Planning ──
+ // VOD 6017's own keyframe sizes: 16:9 for 16 minutes, then portrait at three sizes. Two periods, the portrait one on
+ // the largest portrait size; a lowered resolution is not a period of its own.
+ const vod6017 = at([[0.2, 1280, 720], [976.243, 720, 1280], [1079.895, 540, 960], [1144.988, 360, 640], [1481.092, 540, 960],
+ [1546.278, 360, 640], [1756.995, 540, 960], [1942.652, 360, 640], [2148.22, 540, 960], [2228.501, 360, 640],
+ [2373.997, 540, 960], [2474.381, 720, 1280], [2569.747, 540, 960]]);
+ assert.deepStrictEqual(shapes.plan(vod6017, { end: 2592.2 }), [
+ { start: 0.2, end: 976.243, width: 1280, height: 720 },
+ { start: 976.243, end: 2592.2, width: 720, height: 1280 },
+ ]);
+ // A two-second turn and back joins its neighbour (drawn with bars), and the two landscape runs become one.
+ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [100, 720, 1280], [102, 1280, 720], [300, 960, 540]]), { end: 400 }),
+ [{ start: 0, end: 400, width: 1280, height: 720 }]);
+ // A canvas keeps its shape under MAX_SIDE, on even sides.
+ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [50, 2161, 3841]]), { end: 100 }),
+ [{ start: 0, end: 50, width: 1280, height: 720 }, { start: 50, end: 100, width: 1080, height: 1920 }]);
+ // A phone turned every few seconds for an hour still has at most MAX_PERIODS periods.
+ const flips = [];
+ for (let t = 0; t < 3600; t += 6) flips.push([t, t % 12 ? 720 : 1280, t % 12 ? 1280 : 720]);
+ const many = shapes.plan(at(flips), { end: 3600 });
+ assert.ok(many.length <= shapes.MAX_PERIODS, `${many.length} periods`);
+ assert.strictEqual(many[0].start, 0);
+ assert.strictEqual(many[many.length - 1].end, 3600);
+ for (let i = 1; i < many.length; i++) assert.strictEqual(many[i].start, many[i - 1].end, 'periods tile the video');
+ assert.strictEqual(shapes.varies(at([[0, 640, 360]])), false);
+ assert.strictEqual(shapes.varies(at([[0, 640, 360], [9, 320, 180]])), true);
+ assert.match(shapes.fit(720, 1280), /^scale=720:1280:force_original_aspect_ratio=decrease.*pad=720:1280:\(ow-iw\)\/2:\(oh-ih\)\/2,setsar=1$/);
+
+ // ── Names ──
+ assert.strictEqual(timeline.shapeInitName(2), 'init-2.mp4');
+ assert.strictEqual(timeline.shapeInitSeq('init-2.mp4'), -102);
+ for (const name of ['init.mp4', 'init-1.mp4', 'init-head.mp4', 'init-tail.mp4', '000002.m4s', null]) assert.strictEqual(timeline.shapeInitSeq(name), null, String(name));
+
+ // ── The cut, the playlist and the clips (needs ffmpeg with libx264, libvpx, libopus, as cmaf-transcode) ──
+ if (spawnSync('ffmpeg', ['-version']).status !== 0 || spawnSync('ffprobe', ['-version']).status !== 0) {
+ console.log('✅ shape periods: planning and names (cut skipped: ffmpeg not found)');
+ return;
+ }
+ const ffmpeg = (...args) => {
+ const r = spawnSync('ffmpeg', ['-v', 'error', '-y', ...args], { encoding: 'utf8' });
+ assert.strictEqual(r.status, 0, r.stderr);
+ };
+ const frames = (file) => {
+ const r = spawnSync('ffprobe', ['-v', 'error', '-select_streams', 'v:0', '-show_entries', 'frame=width,height', '-of', 'csv=p=0', file], { encoding: 'utf8' });
+ assert.strictEqual(r.status, 0, r.stderr);
+ return r.stdout.trim().split('\n').filter(Boolean).map((l) => l.split(',').slice(0, 2).join(','));
+ };
+ // The recorder's stream copy: one VP8 track whose keyframes change size, the header naming the first (320x180).
+ // The landscape part is 6 s, the portrait one 6 s at 180x320 then 4 s at 90x160, all at 10 fps; the copy starts at
+ // 0.2 s, as a recording's first keyframe often does.
+ const part = (name, src, size, seconds, freq) => {
+ const file = path.join(tmp, name);
+ ffmpeg('-f', 'lavfi', '-i', `${src}=duration=${seconds}:size=${size}:rate=10`, '-f', 'lavfi', '-i', `sine=duration=${seconds}:frequency=${freq}`,
+ '-c:v', 'libvpx', '-deadline', 'realtime', '-cpu-used', '8', '-g', '10', '-c:a', 'libopus', '-shortest', file);
+ return file;
+ };
+ const parts = [part('a.webm', 'testsrc', '320x180', 6, 440), part('b.webm', 'testsrc2', '180x320', 6, 660), part('c.webm', 'testsrc2', '90x160', 4, 880)];
+ fs.writeFileSync(path.join(tmp, 'list.txt'), parts.map((p) => `file '${p}'`).join('\n'));
+ const mixed = path.join(tmp, 'mixed.webm');
+ ffmpeg('-f', 'concat', '-safe', '0', '-i', path.join(tmp, 'list.txt'), '-c', 'copy', '-output_ts_offset', '0.2', mixed);
+ const sourceFrames = frames(mixed).length;
+
+ const read = await shapes.probe({ input: mixed, remote: false });
+ assert.deepStrictEqual(read.changes.map((c) => `${c.width}x${c.height}`), ['320x180', '180x320', '90x160']);
+ assert.ok(Math.abs(read.changes[1].t - read.start - 6) < 0.1, `the turn at ~6 s (${read.changes[1].t - read.start})`);
+
+ const db = require('../server/db/database');
+ const model = require('../server/objects/model');
+ const cmaf = require('../server/jobs/cmaf');
+ await db.upsertApp({ app_id: 'live', api_key: 'shapes-test' });
+ const object = async (file, mime) => {
+ const id = await model.createObject({ app_id: 'live', owner_user_id: 1, visibility: 'public', lifecycle_status: 'ready',
+ kind: 'file', mime_type: mime, size_bytes: fs.statSync(file).size });
+ await model.upsertLocation(id, { provider: 'local', key: file, state: 'present', size_bytes: fs.statSync(file).size, verified: true });
+ return id;
+ };
+ const run = async (id, suffix) => await cmaf.spec.run({ id: `mjob_${suffix}`, app_id: 'live', object_id: id, params: { segment_seconds: 2 } },
+ { signal: new AbortController().signal });
+ const vod = await object(mixed, 'video/webm');
+ const made = await run(vod, 'shapes');
+ assert.deepStrictEqual([made.video, made.audio, made.shapes], ['h264', 'aac', 2]);
+
+ const rows = await timeline.list(vod, 'source');
+ const init2 = rows.find((r) => r.name === 'init-2.mp4');
+ assert.ok(init2, 'the portrait period has its own init');
+ assert.strictEqual(Number(init2.seq), -102);
+ assert.strictEqual(rows.filter((r) => Number(r.seq) <= 0).length, 2, 'two inits, one per period');
+ const segs = rows.filter((r) => Number(r.seq) > 0);
+ const first = segs.filter((r) => !r.init_name);
+ const second = segs.filter((r) => r.init_name === 'init-2.mp4');
+ assert.ok(first.length && second.length && first.length + second.length === segs.length);
+ assert.ok(Number(first[first.length - 1].seq) < Number(second[0].seq), 'the periods follow each other');
+ // Every frame is on its period's canvas, never the other's: the portrait part is 180x320 (its 90x160 seconds
+ // scaled up into it), and no frame of either shape lands in the other's period.
+ const initFile = (r) => (r.init_name ? init2 : rows.find((x) => Number(x.seq) === 0)).local_path;
+ const sizes = (list) => {
+ const joined = path.join(tmp, 'joined.mp4');
+ const out = [];
+ for (const r of list) {
+ fs.writeFileSync(joined, Buffer.concat([fs.readFileSync(initFile(r)), fs.readFileSync(r.local_path)]));
+ out.push(...frames(joined));
+ }
+ return out;
+ };
+ const landscape = sizes(first);
+ const portrait = sizes(second);
+ assert.ok(landscape.every((s) => s === '320,180'), `landscape period: ${[...new Set(landscape)]}`);
+ assert.ok(portrait.every((s) => s === '180,320'), `portrait period: ${[...new Set(portrait)]}`);
+ assert.ok(Math.abs(landscape.length - 60) <= 1, `${landscape.length} landscape frames, 60 sent`);
+ assert.ok(Math.abs(landscape.length + portrait.length - sourceFrames) <= 1, `${landscape.length + portrait.length} frames of ${sourceFrames}`);
+ // The timeline tiles the whole video.
+ assert.ok(Math.abs(Number(segs[segs.length - 1].end_ms) - 16000) < 400, `ends at ${segs[segs.length - 1].end_ms} ms`);
+
+ // The playlist switches init with a discontinuity between the periods, exactly once.
+ const playlist = timeline.mediaPlaylist(rows);
+ assert.strictEqual((playlist.match(/#EXT-X-DISCONTINUITY/g) || []).length, 1);
+ assert.match(playlist, /#EXT-X-MAP:URI="init\.mp4"[\s\S]*#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);
+
+ // A virtual clip across the turn carries init-2 and each segment's init; one inside the portrait part opens on it.
+ const turnMs = Number(second[0].start_ms);
+ const across = timeline.clipRows(rows, turnMs - 2500, turnMs + 2500);
+ assert.ok(across.some((r) => r.name === 'init-2.mp4' && Number(r.seq) === -102));
+ assert.match(timeline.mediaPlaylist(across), /#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);
+ const inside = timeline.clipRows(rows, turnMs + 1000, turnMs + 5000);
+ assert.match(timeline.mediaPlaylist(inside), /^#EXTM3U[\s\S]*?#EXT-X-MAP:URI="init-2\.mp4"/);
+ assert.ok(!/DISCONTINUITY/.test(timeline.mediaPlaylist(inside)));
+ // A materialized clip's own rows over the source (the edges keep refusing a virtual clip, as before).
+ assert.deepStrictEqual(timeline.clipRows([...rows, { seq: -1, name: timeline.HEAD_INIT }], 0, 4000), []);
+ const materialize = require('../server/vod/clip-materialize');
+ const plan = materialize.planWindow(rows, Number(second[0].start_ms), Number(second[second.length - 1].end_ms));
+ assert.ok(plan.sourceRows.some((r) => r.name === 'init-2.mp4'), 'the pin names the period init it copies');
+ assert.ok(!plan.sourceRows.some((r) => Number(r.seq) === 0), 'and not the first period\'s');
+ assert.strictEqual(plan.initOf(second[0]).name, 'init-2.mp4');
+ const spanning = materialize.planWindow(rows, Number(first[0].start_ms), Number(second[second.length - 1].end_ms));
+ assert.deepStrictEqual(spanning.sourceRows.filter((r) => Number(r.seq) <= 0).map((r) => r.name).sort(), ['init-2.mp4', 'init.mp4']);
+
+ // One size throughout keeps the cut as it was: a stream copy of an H.264 MP4.
+ const plain = path.join(tmp, 'plain.mp4');
+ ffmpeg('-f', 'lavfi', '-i', 'testsrc=duration=5:size=160x90:rate=10', '-f', 'lavfi', '-i', 'sine=duration=5',
+ '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-shortest', plain);
+ const copied = await run(await object(plain, 'video/mp4'), 'plain');
+ assert.deepStrictEqual([copied.video, copied.audio, copied.shapes], ['copy', 'copy', 1]);
+ console.log('✅ shape periods: planning, one init per shape, frames on their canvas, playlists and clips');
+ } finally { fs.rmSync(tmp, { recursive: true, force: true }); }
+})().catch((err) => { console.error(err); process.exitCode = 1; });