openvibers/openvibe.media Commits

VODs that change shape mid-stream play each part at its own shape: one period per shape, each with its own init (#180)

OP OpenVibers committed 18 days ago · 7d4152e831 · 12 files, +672 −53

A recording is a stream copy, so when a phone turns or WebRTC lowers the resolution, its frames change size at a
keyframe while the header keeps the first size. object.cmaf's single encode scaled every later frame to that first size:
VOD 6017 (1280x720, then 720x1280/540x960/360x640 from 16:16) played its portrait part stretched into 16:9.

- server/jobs/shapes.js: reads each keyframe's size (keyframe-only decode), plans periods (sizes within ~4% of one
  aspect ratio are one shape; canvas = the shape's largest size, long side <= 1920; < 5 s joins a neighbour; <= 40),
  and cuts each period with its own ffmpeg run on the input's own timestamps (-copyts + trim/atrim, so no frame of one
  shape lands in the other's period), fitted (never stretched), with negative composition offsets (no gap at a join:
  Chrome's native HLS stalled on the B-frame gap; hls.js jumped it).
- object.cmaf (MEDIA_CMAF_TRANSCODE): more than one size -> periods; period n >= 2's init is init-<n>.mp4 at seq
  -(100+n), named by its segments' init_name, so the playlist puts DISCONTINUITY + MAP between periods. One size keeps
  the cut exactly as before. Migration 0014 widens the seq CHECK for those names.
- Virtual clips (clipRows) and materialized clips carry the period inits their segments name; rendition.create cuts the
  same periods at its rung; the clip cutter draws a window that spans a change on its main shape.
- scripts/recut-shapes.js finds recordings cut before this (dry run by default) and queues their re-cut.

Checked: test/shape-periods.test.js (planning on 6017's own sizes, a real cut of a three-size VP8 recording starting at
0.2 s: two inits, every frame on its period's canvas, frame counts kept, playlist, clips), the 33 tests the change
reaches, and a 200 s slice of VOD 6017 cut and played in Chrome with hls.js and native HLS: 1280x720 then 720x1280
across the turn, one buffered range, no stall.

Changes

ddiff --git a/docs/media-fabric.md b/docs/media-fabric.md
iindex d72b0af..9d952e3 100644
---- a/docs/media-fabric.md
++++ b/docs/media-fabric.md
@@@ -110,6 +110,23 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
honours the abort signal (which stops an upload in flight) and, before the rows commit, removes the files and keys it
staged; the stream-copy or duration-based transcode budget is honoured and its work directory removed. The master
playlist reads CODECS from the written init segment, so an encoded source advertises its output codecs.
++
++ **A picture that changes shape.** A recording is a stream copy of what the broadcaster sent, so when a phone turns
++ from landscape to portrait, or WebRTC lowers 720p to 360p on a weak network, its frames change size at a keyframe
++ while the file's header still names the first size. One encode scales every later frame to that first size (VOD 6017:
++ a portrait street stretched into 16:9), and a copy carries frames its init does not describe. With
++ `MEDIA_CMAF_TRANSCODE=1`, `object.cmaf` first reads each keyframe's size (`server/jobs/shapes.js`, a keyframe-only
++ decode). When there is more than one, it cuts **one period per shape**: sizes within ~4% of one aspect ratio are one
++ shape (a lowered resolution is not a new period), and each period is drawn on its shape's largest size (long side at
++ most 1920), with frames of another size fitted, never stretched. A period under 5 s joins a neighbour, and there are
++ at most 40. Each period is its own ffmpeg run on the input's own timestamps, with its own init: the first period's is
++ `init.mp4` (seq 0), period n ≥ 2's is `init-<n>.mp4`, a row at seq −(100+n) that its segments name in `init_name`
++ (migration 0014). The media playlist then puts `#EXT-X-DISCONTINUITY` + `#EXT-X-MAP` between periods, so a player
++ shows each part at its own shape, the way an adaptive stream switches renditions. Virtual and materialized clips carry
++ the period inits their segments name, `rendition.create` cuts the same periods at its rung's height, and a clip cut
++ (`server/vod/clip-cutter.js`) whose window spans a change is drawn on the window's main shape. One size throughout
++ keeps the cut exactly as before. `scripts/recut-shapes.js` finds the recordings cut before this (a dry run by default)
++ and queues their re-cut.
Behind **`MEDIA_HLS_ENABLED`** (off by default: the job is refused with `409 media.hls.disabled` and the routes 404),
`GET /o/:id/master.m3u8`, `/o/:id/source/index.m3u8` and `/o/:id/source/{init.mp4,NNNNNN.m4s}` serve the timeline
under the same check as `GET /o/:id` (private and sandbox objects only with the object's `?exp&sig`, carried onto
ddiff --git a/migrations/0014_shape_period_inits.sql b/migrations/0014_shape_period_inits.sql
nnew file mode 100644
iindex 0000000..aaf0e5c
---- /dev/null
++++ b/migrations/0014_shape_period_inits.sql
@@@ -0,0 +1,9 @@
++-- phase: expand
++-- A video whose picture changes shape mid-stream (a phone turned, a resolution lowered) is cut in periods, one init
++-- segment each (server/jobs/shapes.js): the first period's is init.mp4 at seq 0, period n >= 2's is init-<n>.mp4 at seq
++-- -(100 + n), named by its media rows' init_name (0005). This widens 0005's seq CHECK to admit those names. Additive
++-- only: an older release reads the rows with init_name already (its playlists put a discontinuity before each init) and
++-- never writes one.
++ALTER TABLE media_timeline DROP CONSTRAINT IF EXISTS media_timeline_seq_check;
++ALTER TABLE media_timeline ADD CONSTRAINT media_timeline_seq_check
++ CHECK (seq >= 0 OR name IN ('init-head.mp4', 'init-tail.mp4') OR name ~ '^init-[0-9]{1,4}\.mp4$');
ddiff --git a/scripts/recut-shapes.js b/scripts/recut-shapes.js
nnew file mode 100755
iindex 0000000..ce0ed2f
---- /dev/null
++++ b/scripts/recut-shapes.js
@@@ -0,0 +1,87 @@
++#!/usr/bin/env node
++'use strict';
++/**
++ * Re-cut the recordings whose source timeline was cut as one shape although their picture changes shape mid-stream
++ * (server/jobs/shapes.js, docs/media-fabric.md §3: VOD 6017 played its portrait part stretched into 16:9), for the
++ * operator.
++ *
++ * node scripts/recut-shapes.js [--days 30] [--limit 200] [--app live] [--ids 6017,6020] [--remote] [--apply] [--json]
++ * Dry run (the default): for each finished VOD of the window, newest first, whose object has a source timeline
++ * with one init and a local file (--remote also reads B2/R2 copies, which is egress), reads its keyframe sizes
++ * and reports the ones with more than one, with the periods a re-cut would make. Changes nothing.
++ * --apply queues object.cmaf for each of those (idempotency key object.cmaf:<object>:shapes, so a second run
++ * answers with the same job). The running service's worker cuts them on its next poll, one at a time in the heavy
++ * lane at nice 15; the new rows replace the old ones in one transaction, and playback never stops.
++ * (the database is the service's: DATABASE_URL, and MEDIA_HLS_ENABLED / MEDIA_CMAF_TRANSCODE, from its env file)
++ *
++ * Exit codes: 0 done, 1 error, 2 bad usage.
++ */
++
++(async () => {
++ await require('../server/db/database').initDb(); // PostgreSQL (DATABASE_URL), as the service
++ const args = process.argv.slice(2);
++ const has = (name) => args.includes(`--${name}`);
++ const arg = (name, d = null) => { const i = args.indexOf(`--${name}`); return i >= 0 && args[i + 1] && !args[i + 1].startsWith('--') ? args[i + 1] : d; };
++ const usage = (msg) => { console.error(`recut-shapes: ${msg}`); process.exit(2); };
++ for (const flag of ['days', 'limit', 'app', 'ids']) if (has(flag) && !arg(flag)) usage(`--${flag} needs a value`);
++ const APPLY = has('apply');
++ const days = Number(arg('days', 30));
++ const limit = Number(arg('limit', 200));
++ if (!(days > 0) || !(limit > 0)) usage('--days and --limit are positive numbers');
++ const ids = arg('ids') ? arg('ids').split(',').map(Number).filter((n) => n > 0) : null;
++
++ const config = require('../server/config');
++ const db = require('../server/db/database');
++ const model = require('../server/objects/model');
++ const timeline = require('../server/objects/timeline');
++ const derive = require('../server/jobs/derive');
++ const shapes = require('../server/jobs/shapes');
++ if (!config.hls.enabled || !config.hls.transcode) usage('the re-cut needs MEDIA_HLS_ENABLED=1 and MEDIA_CMAF_TRANSCODE=1 (the service\'s env file)');
++ if (APPLY) await require('../server/events').initWriter();
++
++ const since = new Date(Date.now() - days * 86400000).toISOString().slice(0, 19).replace('T', ' ');
++ const vods = ids
++ ? await db.all('SELECT id, app_id, object_id, title FROM vods WHERE id = ANY(?) ORDER BY id DESC', [ids])
++ : await db.all(`SELECT id, app_id, object_id, title FROM vods WHERE is_recording = 0 AND object_id IS NOT NULL AND created_at >= ?
++ ${has('app') ? 'AND app_id = ?' : ''} ORDER BY id DESC LIMIT ?`, has('app') ? [since, arg('app'), limit] : [since, limit]);
++ const report = [];
++ for (const v of vods) {
++ const out = { vod: v.id, object: v.object_id, title: v.title };
++ report.push(out);
++ const obj = v.object_id && await model.getObject(v.object_id);
++ if (!obj || obj.lifecycle_status !== 'ready') { out.skip = 'no ready object'; continue; }
++ const rows = await timeline.list(obj.id, timeline.SOURCE);
++ if (!rows.some((r) => Number(r.seq) > 0)) { out.skip = 'no source timeline (its first cut reads the shapes)'; continue; }
++ if (rows.some((r) => timeline.shapeInitSeq(r.name) != null)) { out.skip = 'already cut in periods'; continue; }
++ const source = await derive.resolveSource(obj);
++ if (!source) { out.skip = 'no bytes'; continue; }
++ if (source.remote && !has('remote')) { out.skip = 'only on B2/R2 (--remote reads it)'; continue; }
++ try {
++ const read = await shapes.probe(source, { timeoutMs: derive.budgetMs(obj.size_bytes) });
++ if (!shapes.varies(read.changes)) { out.shapes = 1; continue; }
++ const info = await derive.probeStreams(source);
++ const periods = shapes.plan(read.changes, { end: read.start + info.duration });
++ out.sizes = read.changes.length;
++ out.periods = periods.map((p) => `${p.width}x${p.height} from ${(p.start - read.start).toFixed(1)} s`);
++ if (APPLY) {
++ const params = require('../server/jobs/cmaf').spec.validate({ appId: obj.app_id, obj, params: {} });
++ const r = await require('../server/jobs/queue').enqueue({ appId: obj.app_id, type: 'object.cmaf', objectId: obj.id, params,
++ dedupeActive: true, idempotencyKey: `object.cmaf:${obj.id}:shapes`, createdBy: `operator:${require('os').userInfo().username}` });
++ out.job = r.job ? r.job.id : null;
++ out.queued = !!r.created;
++ }
++ } catch (err) { out.error = err.message.slice(0, 200); }
++ }
++ if (has('json')) console.log(JSON.stringify(report, null, 2));
++ else {
++ for (const r of report) {
++ const what = r.skip ? `skip: ${r.skip}` : r.error ? `error: ${r.error}` : r.periods
++ ? `${r.sizes} sizes → ${r.periods.join(', ')}${r.job ? ` ${r.queued ? 'queued' : 'already queued'} ${r.job}` : ''}`
++ : 'one shape';
++ console.log(`vod ${r.vod} ${what}`);
++ }
++ const found = report.filter((r) => r.periods).length;
++ console.log(`${report.length} checked, ${found} change shape${APPLY ? `, ${report.filter((r) => r.queued).length} queued` : found ? ' (dry run: --apply queues their re-cut)' : ''}`);
++ }
++ process.exit(report.some((r) => r.error) ? 1 : 0);
++})().catch((err) => { console.error(err); process.exit(1); });
ddiff --git a/server/jobs/cmaf.js b/server/jobs/cmaf.js
iindex 021bd1b..3f0a32e 100644
---- a/server/jobs/cmaf.js
++++ b/server/jobs/cmaf.js
@@@ -19,6 +19,9 @@
* rendition.create (F4).
* Idempotent: a rerun makes the same bytes (bitexact), reuses a row whose sha256 matches and its location as it is, and
* uploads only what is not durable yet. A run that aborts or fails before the commit removes its staged files and keys.
++ * A picture that changes shape mid-stream (a phone turned, a resolution lowered: jobs/shapes.js) is re-encoded with
++ * MEDIA_CMAF_TRANSCODE in one period per shape, each on its own canvas with its own init segment, so the playlist
++ * switches shape between periods instead of stretching every frame to the first size; `shapes` in the result counts them.
* Refused: MEDIA_HLS_ENABLED off, a source that is not ready or not media (permanent), less free disk than the source
* size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent), a clip whose file was
* replaced (a trim or re-cut) while it ran (permanent, media.source.changed: nothing is committed).
@@@ -31,6 +34,7 @@ const config = require('../config');
const timeline = require('../objects/timeline');
const derive = require('./derive');
const segments = require('./segments');
++const shapes = require('./shapes');
const { JobError } = require('./queue');
const DEFAULT_SEGMENT_SECONDS = 4;
@@@ -39,6 +43,13 @@ function transcodeBudgetMs(duration) {
return Math.min(13 * 3600 * 1000, Math.max(10 * 60000, Math.round(duration * 3000 + 10 * 60000)));
}
++/** The H.264 re-encode: a keyframe at every segment boundary, frame times as they are. */
++function x264Args(segmentSeconds) {
++ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),
++ '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough'];
++}
++const AAC_ARGS = ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'];
++
function disabled() {
return new JobError('media.hls.disabled', 'Segment-native video is off on this server (MEDIA_HLS_ENABLED)', { permanent: true, status: 409 });
}
@@@ -56,6 +67,25 @@ function validate({ obj, params }) {
return p;
}
++/** The whole input as one HLS fMP4 rendition (one shape): → the produced segments, [{ name, seconds }]. */
++async function cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness }) {
++ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
++ '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
++ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',
++ '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,
++ '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],
++ { signal: ctx.signal, timeoutMs, niceness });
++ segments.aborted(ctx);
++ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
++ if (!r.ok) {
++ const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
++ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });
++ }
++ const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
++ if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
++ return list;
++}
++
async function run(job, ctx) {
if (!config.hls.enabled) throw disabled();
const src = await derive.loadSource(job);
@@@ -88,11 +118,28 @@ async function run(job, ctx) {
throw new JobError('too_long_to_transcode', `Source duration ${info.duration}s exceeds MEDIA_CMAF_TRANSCODE_MAX_SECONDS`, { permanent: true });
}
timeoutMs = transcodeBudgetMs(info.duration);
-- codecArgs = video === 'h264'
-- ? ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '23', '-pix_fmt', 'yuv420p', '-profile:v', 'high', '-threads', String(config.hls.transcodeThreads),
-- '-force_key_frames', `expr:gte(t,n_forced*${segmentSeconds})`, '-sc_threshold', '0', '-fps_mode', 'passthrough']
-- : ['-c:v', 'copy'];
-- codecArgs.push(...(audio === 'aac' ? ['-c:a', 'aac', '-b:a', '128k', '-ac', '2'] : ['-c:a', 'copy']));
++ codecArgs = video === 'h264' ? x264Args(segmentSeconds) : ['-c:v', 'copy'];
++ codecArgs.push(...(audio === 'aac' ? AAC_ARGS : ['-c:a', 'copy']));
++ }
++ }
++ // Frames of more than one size (shapes.js): one period per shape, each re-encoded on its own canvas. A copy would
++ // carry frames its init does not describe, and one encode scales every frame to the first size (a portrait part
++ // stretched into 16:9). A probe that fails leaves the cut as it was.
++ let periods = null;
++ let shapeStart = 0;
++ if (config.hls.transcode && info && info.video && info.duration > 0 && info.duration <= config.hls.transcodeMaxSeconds) {
++ let read = null;
++ try { read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }); }
++ catch (err) { segments.aborted(ctx); console.warn(`[Cmaf] ${job.id}: shape probe failed (${err.message.slice(0, 200)}); cut as one shape`); }
++ segments.aborted(ctx);
++ if (read && shapes.varies(read.changes)) {
++ periods = shapes.plan(read.changes, { end: read.start + info.duration });
++ shapeStart = read.start;
++ video = 'h264';
++ if (audio !== 'none') audio = 'aac';
++ timeoutMs = transcodeBudgetMs(info.duration);
++ codecArgs = [...x264Args(segmentSeconds), ...(audio === 'aac' ? AAC_ARGS : [])];
++ console.log(`[Cmaf] ${job.id}: ${read.changes.length} frame sizes → ${periods.length} period(s): ${periods.map((p) => `${p.width}x${p.height}`).join(', ')}`);
}
}
segments.checkDisk(Number(src.size_bytes) || 0);
@@@ -101,20 +148,10 @@ async function run(job, ctx) {
try {
const out = path.join(dir, 'out');
fs.mkdirSync(out, { recursive: true });
-- const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
-- '-map', '0:v:0?', '-map', '0:a:0?', ...codecArgs, '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
-- '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0',
-- '-hls_playlist_type', 'vod', '-hls_segment_type', 'fmp4', '-hls_fmp4_init_filename', timeline.INIT_NAME,
-- '-start_number', '1', '-hls_segment_filename', path.join(out, '%06d.m4s'), path.join(out, 'index.m3u8')],
-- { signal: ctx.signal, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });
-- segments.aborted(ctx);
-- const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
-- if (!r.ok) {
-- const unsupported = /not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
-- throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code}: ${tail}`, { permanent: unsupported });
-- }
-- const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
-- if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
++ const list = periods
++ ? await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds, audio: audio !== 'none', encode: () => codecArgs,
++ signal: ctx.signal, timeoutMs, niceness: 15 })
++ : await cutWhole({ source, out, codecArgs, segmentSeconds, ctx, timeoutMs, niceness: video === 'h264' || audio === 'aac' ? 15 : 0 });
const check = version == null ? null : async () => {
if (await derive.fileVersion(src, { lock: true }) !== version) {
@@@ -127,7 +164,7 @@ async function run(job, ctx) {
return {
source_id: src.id, rendition, segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
-- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio,
++ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, video, audio, shapes: periods ? periods.length : 1,
};
} finally {
derive.cleanupWork(job.id);
ddiff --git a/server/jobs/rendition.js b/server/jobs/rendition.js
iindex ee01944..7025e90 100644
---- a/server/jobs/rendition.js
++++ b/server/jobs/rendition.js
@@@ -14,6 +14,9 @@
* result { …, skipped: true, reason } when the source is already this rendition's height or smaller (a still is not
* upscaled), or the rows already exist (skipped: false, already: true)
*
++ * A source whose picture changes shape (jobs/shapes.js) is cut as its source timeline is: one period per shape, each on
++ * its own canvas scaled to the rung's height, with its own init segment.
++ *
* Refused: MEDIA_HLS_ENABLED or MEDIA_RENDITIONS off, a source that is not ready or not media (permanent), an object
* under a hold (retried), a source with no video stream (permanent), less free disk than the source size plus
* MEDIA_UPLOAD_MIN_FREE_MB (retried later). Idempotent: a re-run whose bytes match reuses the rows and locations, and
@@@ -31,6 +34,7 @@ const config = require('../config');
const timeline = require('../objects/timeline');
const derive = require('./derive');
const segments = require('./segments');
++const shapes = require('./shapes');
const { JobError } = require('./queue');
const DEFAULT_SEGMENT_SECONDS = 4;
@@@ -95,11 +99,22 @@ async function run(job, ctx) {
const info = await require('../vod/media-tools').probeVodInfo(source.input);
const video = (info.streams || []).find((s) => s.codec_type === 'video');
if (!video) throw new JobError('no_video', 'The source has no video stream to rendition', { permanent: true });
-- const height = Number(video.height) || 0;
++ // Frames of more than one size: the periods the source timeline was cut in, each brought down to this rung.
++ let planned = null;
++ let shapeStart = 0;
++ const read = await shapes.probe(source, { signal: ctx.signal, timeoutMs: derive.budgetMs(src.size_bytes) }).catch(() => null);
++ segments.aborted(ctx);
++ if (read && shapes.varies(read.changes)) {
++ planned = shapes.plan(read.changes, { end: read.start + (Number(info.duration) || 0) });
++ shapeStart = read.start;
++ }
++ const height = planned ? Math.max(...planned.map((p) => p.height)) : Number(video.height) || 0;
// A source already this rendition's height or smaller is not upscaled (a re-encode would only cost bytes).
if (height > 0 && height <= targetHeight) {
return { source_id: src.id, rendition, target_height: targetHeight, skipped: true, reason: `the source is ${height}p, not above ${rendition}`, segments: 0, duration_ms: 0, bytes: 0 };
}
++ const periods = planned && planned.map((p) => (p.height <= targetHeight ? p
++ : { ...p, width: Math.max(2, Math.round(p.width * targetHeight / p.height / 2) * 2), height: targetHeight }));
segments.checkDisk(Number(src.size_bytes) || 0);
const srcRows = await timeline.segments(src.id, timeline.SOURCE);
const segSec = segmentSeconds(job, srcRows);
@@@ -108,9 +123,33 @@ async function run(job, ctx) {
// and a source that long is packed long before it is re-encoded.
const boundaries = srcRows.length && srcRows.length <= 2000 ? srcRows.map((r) => (Number(r.start_ms) / 1000).toFixed(3)) : [];
const dir = derive.workDir(job.id);
++ const publish = async (list) => {
++ const published = await segments.publishRendition({ job, ctx, src, rendition, out: path.join(dir, 'out'), list, label: 'Rendition' });
++ return {
++ source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,
++ segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
++ durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
++ master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true, shapes: periods ? periods.length : 1,
++ };
++ };
try {
const out = path.join(dir, 'out');
fs.mkdirSync(out, { recursive: true });
++ if (periods) {
++ // Each period's keyframes at the source boundaries inside it, in the period's own time (its output starts at 0).
++ const axis = (p, i) => (i === 0 ? 0 : p.start - shapeStart);
++ const list = await shapes.cut({ source, periods, start: shapeStart, out, segmentSeconds: segSec,
++ audio: (info.streams || []).some((st) => st.codec_type === 'audio'),
++ encode: (p, i) => {
++ const from = axis(p, i);
++ const to = i + 1 < periods.length ? axis(periods[i + 1], i + 1) : Infinity;
++ const own = boundaries.map(Number).filter((b) => b >= from && b < to).map((b) => (b - from).toFixed(3));
++ return ['-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',
++ '-c:a', 'aac', '-b:a', '128k', '-ac', '2', ...(own.length ? ['-force_key_frames', own.join(',')] : [])];
++ },
++ signal: ctx.signal, timeoutMs: encodeBudgetMs(src.size_bytes), niceness: 0 });
++ return await publish(list);
++ }
const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-i', source.input,
'-map', '0:v:0', '-map', '0:a:0?',
'-vf', `scale=-2:${targetHeight}`, '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21', '-profile:v', 'main', '-pix_fmt', 'yuv420p',
@@@ -130,13 +169,7 @@ async function run(job, ctx) {
const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
-- const published = await segments.publishRendition({ job, ctx, src, rendition, out, list, label: 'Rendition' });
-- return {
-- source_id: src.id, rendition, target_height: targetHeight, skipped: false, already: false,
-- segments: published.segments, duration_ms: published.duration_ms, bytes: published.bytes,
-- durable: published.durable, local_only: published.local_only, rows: published.rows, uploaded: published.uploaded,
-- master_path: `/o/${src.id}/master.m3u8`, source_unchanged: true,
-- };
++ return await publish(list);
} finally {
derive.cleanupWork(job.id);
}
ddiff --git a/server/jobs/segments.js b/server/jobs/segments.js
iindex a92f910..3e1cc37 100644
---- a/server/jobs/segments.js
++++ b/server/jobs/segments.js
@@@ -6,8 +6,10 @@
* turns that output into media_timeline rows and publishes them:
*
* parsePlaylist ffmpeg's index.m3u8 → [{ name, seconds }], in order.
-- * producedFiles those plus the init segment → [{ seq, name, file, start_ms, end_ms, keyframe_ms }], times on the
-- * object's own axis (cumulative EXTINF, so each segment ends where the next starts).
++ * producedFiles those plus the init segment(s) → [{ seq, name, file, start_ms, end_ms, keyframe_ms, init_name }],
++ * times on the object's own axis (cumulative EXTINF, so each segment ends where the next starts). A
++ * video cut in shape periods (jobs/shapes.js) has one init per period: the later ones are rows with
++ * negative seq that their segments name in init_name.
* publishRendition hash each file, place it content-addressed on this node (timeline.localPathFor/keyFor, versioned
* by the sha prefix) and — through the placement router's durable provider (timeline.durableProvider,
* purpose `durable`) — on B2/R2, then commit every row of the rendition in ONE transaction
@@@ -45,16 +47,27 @@ function parsePlaylist(text) {
return out;
}
--/** The produced files as timeline rows in the making: the init segment (seq 0) then each media segment, in order. */
++/**
++ * The produced files as timeline rows in the making: the init segment (seq 0), the init of each later shape period
++ * (a list entry's `init`, jobs/shapes.js: negative seq, named in its segments' init_name), then each media segment, in
++ * order.
++ */
function producedFiles(list) {
const files = [{ seq: 0, name: timeline.INIT_NAME, start_ms: 0, end_ms: 0, keyframe_ms: null }];
++ const media = [];
let cum = 0;
list.forEach((s, i) => {
const start = Math.round(cum * 1000);
cum += s.seconds;
-- files.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start });
++ const init = s.init && s.init !== timeline.INIT_NAME ? s.init : null;
++ if (init && !files.some((f) => f.name === init)) {
++ const seq = timeline.shapeInitSeq(init);
++ if (seq == null) throw new JobError('ffmpeg_failed', `${init} is not a shape period's init`);
++ files.push({ seq, name: init, start_ms: 0, end_ms: 0, keyframe_ms: null });
++ }
++ media.push({ seq: i + 1, name: timeline.segmentName(i + 1), file: s.name, start_ms: start, end_ms: Math.round(cum * 1000), keyframe_ms: start, init_name: init });
});
-- return files;
++ return files.concat(media);
}
function checkDisk(needBytes) {
@@@ -122,7 +135,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
const match = !!old && old.sha256 === sha;
if (match && old.packed_object_id) {
// Already packed into a chunk (object.pack): the same bytes, so the row and its chunk stay as they are.
-- rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : old[k]])));
++ rows.push(Object.fromEntries(['seq', ...timeline.FIELDS].map((k) => [k, k === 'seq' ? f.seq : k === 'init_name' ? f.init_name || null : old[k]])));
continue;
}
// Content-addressed: the same bytes always land at the same versioned location, so a rerun rewrites
@@@ -143,7 +156,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
if (!durable && target) {
try {
if (!vodStorage.providerAvailable(target)) throw new Error(`${target} is unavailable`);
-- await vodStorage.uploadFile(target, key, localPath, f.seq === 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });
++ await vodStorage.uploadFile(target, key, localPath, f.seq <= 0 ? 'video/mp4' : 'video/iso.segment', { signal: ctx.signal });
stagedRemote.push({ provider: target, key });
const back = await vodStorage.sha256Object(target, key);
if (!back || back.sha256 !== sha || Number(back.size) !== size) throw new Error(`${target}:${key} failed size/sha256 readback`);
@@@ -161,7 +174,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
rows.push({
seq: f.seq, name: f.name, start_ms: f.start_ms, end_ms: f.end_ms, keyframe_ms: f.keyframe_ms, key, local_path: localPath,
durable_provider: durable, packed_object_id: null, byte_offset: null, byte_length: size, sha256: sha,
-- durability: durable ? 'durable' : 'local',
++ durability: durable ? 'durable' : 'local', init_name: f.init_name || null,
});
}
aborted(ctx);
ddiff --git a/server/jobs/shapes.js b/server/jobs/shapes.js
nnew file mode 100644
iindex 0000000..62c0f9b
---- /dev/null
++++ b/server/jobs/shapes.js
@@@ -0,0 +1,213 @@
++/**
++ * OpenVibe.Media — the picture's shapes over a video, and cutting it in one period per shape (docs/media-fabric.md §3).
++ *
++ * A recording is a stream copy of what the broadcaster sent. When their phone turns, or WebRTC lowers 720p to 360p on a
++ * weak network, the frames change size at a keyframe while the file's header still names the first size. One encode
++ * then scales every later frame to that first size (VOD 6017: a portrait street stretched into 16:9), and a stream copy
++ * carries frames its init segment does not describe. So a cut that must re-encode reads the shapes first:
++ *
++ * probe(source) → { start, changes: [{ t, width, height }] }: the input's start time and each keyframe whose
++ * size differs from the one before (one decode of the keyframes only; t in the input's own
++ * seconds). `from`/`to` read only that window (a clip).
++ * plan(changes, { end }) → [{ start, end, width, height }]: the periods. Sizes within SAME_SHAPE of one aspect ratio
++ * are one shape (a lowered resolution is not a new period); each period is drawn on the canvas
++ * of its shape's largest size (long side at most MAX_SIDE). A period shorter than MIN_PERIOD_S
++ * joins a neighbour, and there are at most MAX_PERIODS.
++ * fit(width, height) the filter that puts any frame on a width×height canvas without stretching it: scaled to fit,
++ * centred, bars for the rest. A frame of the period's own shape fills the canvas exactly.
++ * cut({ … }) one ffmpeg run per period into `out`, each with its own init segment (the first period's is
++ * init.mp4; period n ≥ 2's is timeline.shapeInitName(n)) → the produced segments in order,
++ * [{ name, seconds, init }] for segments.publishRendition. The playlist puts
++ * #EXT-X-DISCONTINUITY + #EXT-X-MAP between periods (timeline.mediaPlaylist), so a player shows
++ * each part at its own shape, the way an adaptive stream switches renditions.
++ *
++ * Periods are cut on the input's own timestamps (-copyts, trim/atrim at the period's edges, the next period sought to its
++ * first keyframe), so no frame of one shape lands in the other's period; each period's timestamps continue where the
++ * last one's ended, with no gap at the join (negative composition offsets: Chrome's native HLS stalls on a gap there).
++ */
++'use strict';
++
++const path = require('path');
++const fs = require('fs');
++const { spawn } = require('child_process');
++const derive = require('./derive');
++const timeline = require('../objects/timeline');
++const { JobError } = require('./queue');
++
++const SAME_SHAPE = 0.04; // |ln(aspect a / aspect b)|: 1280x720 and 426x240 are one shape, 16:9 and 4:3 are not
++const MIN_PERIOD_S = 5;
++const MAX_PERIODS = 40;
++const MAX_SIDE = 1920;
++
++const aspect = (s) => s.width / s.height;
++const sameShape = (a, b) => Math.abs(Math.log(aspect(a) / aspect(b))) < SAME_SHAPE;
++const even = (x) => Math.max(2, Math.round(x / 2) * 2);
++const secs = (t) => Number(t).toFixed(6).replace(/\.?0+$/, '') || '0';
++
++/** Each keyframe's size, read with ffprobe (keyframes only). Resolves { start, changes }. */
++function probe(source, { signal, timeoutMs = 30 * 60000, from = null, to = null } = {}) {
++ return new Promise((resolve, reject) => {
++ const args = ['-v', 'error', ...derive.inputArgs(source),
++ ...(from != null ? ['-read_intervals', `${secs(from)}%${secs(to)}`] : []),
++ '-select_streams', 'v:0', '-skip_frame', 'nokey',
++ '-show_entries', 'frame=best_effort_timestamp_time,width,height:format=start_time', '-of', 'csv=p=0', source.input];
++ let proc;
++ try { proc = spawn('ffprobe', args, { stdio: ['ignore', 'pipe', 'pipe'] }); } catch (err) { return reject(err); }
++ try { require('os').setPriority(proc.pid, 10); } catch { /* best effort */ }
++ const changes = [];
++ let start = null;
++ let rest = '';
++ let stderr = '';
++ let done = false;
++ const line = (l) => {
++ const f = l.trim().split(',');
++ if (f.length === 1) { if (f[0] && f[0] !== 'N/A') start = Number(f[0]); return; } // format=start_time
++ const [t, width, height] = f.map(Number);
++ if (!Number.isFinite(t) || !(width > 0) || !(height > 0)) return;
++ const last = changes[changes.length - 1];
++ if (!last || last.width !== width || last.height !== height) changes.push({ t, width, height });
++ };
++ const kill = () => { try { proc.kill('SIGKILL'); } catch { /* gone */ } };
++ const timer = setTimeout(kill, timeoutMs);
++ const finish = (err, value) => {
++ if (done) return;
++ done = true;
++ clearTimeout(timer);
++ if (signal) signal.removeEventListener('abort', kill);
++ if (err) reject(err); else resolve(value);
++ };
++ if (signal) { if (signal.aborted) kill(); else signal.addEventListener('abort', kill, { once: true }); }
++ proc.stdout.on('data', (d) => {
++ const lines = (rest + d).split('\n');
++ rest = lines.pop();
++ for (const l of lines) line(l);
++ });
++ proc.stderr.on('data', (d) => { stderr = (stderr + d).slice(-4000); });
++ proc.on('error', (err) => finish(err));
++ proc.on('close', (code) => {
++ if (rest) line(rest);
++ if (code !== 0) return finish(new Error(`ffprobe exited ${code}: ${stderr.trim().slice(-300)}`));
++ finish(null, { start: Number.isFinite(start) ? start : (changes[0] ? changes[0].t : 0), changes });
++ });
++ });
++}
++
++/** The shape a set of runs mostly shows, and its largest size: { width, height } of the canvas. */
++function canvasOf(runs) {
++ const groups = [];
++ for (const r of runs) {
++ let g = groups.find((x) => sameShape(x.lead, r));
++ if (!g) groups.push(g = { lead: r, seconds: 0, runs: [] });
++ g.seconds += r.end - r.start;
++ g.runs.push(r);
++ }
++ const main = groups.reduce((a, b) => (b.seconds > a.seconds ? b : a));
++ const big = main.runs.reduce((a, b) => (b.width * b.height > a.width * a.height ? b : a));
++ const k = Math.min(1, MAX_SIDE / Math.max(big.width, big.height));
++ return { width: even(big.width * k), height: even(big.height * k) };
++}
++
++/**
++ * The periods of a video from its size changes (probe's `changes`), up to `end` (the input's end, its own seconds).
++ * One size throughout → one period on that size.
++ */
++function plan(changes, { end } = {}) {
++ if (!changes.length) return [];
++ const last = Number.isFinite(end) && end > changes[changes.length - 1].t ? end : changes[changes.length - 1].t + 1;
++ const runs = changes.map((c, i) => ({ start: c.t, end: i + 1 < changes.length ? changes[i + 1].t : last, width: c.width, height: c.height }));
++ // Consecutive runs of one shape are one period.
++ let periods = [];
++ for (const r of runs) {
++ const p = periods[periods.length - 1];
++ if (p && sameShape(p.runs[0], r)) { p.runs.push(r); p.end = r.end; } else periods.push({ start: r.start, end: r.end, runs: [r] });
++ }
++ const merge = (i, j) => { // j = i ± 1: the two become one period
++ const [a, b] = i < j ? [periods[i], periods[j]] : [periods[j], periods[i]];
++ a.runs.push(...b.runs);
++ a.end = b.end;
++ periods.splice(periods.indexOf(b), 1);
++ };
++ const regroup = () => {
++ const out = [];
++ for (const p of periods) {
++ const q = out[out.length - 1];
++ if (q && sameShape(canvasOf(q.runs), canvasOf(p.runs))) { q.runs.push(...p.runs); q.end = p.end; } else out.push(p);
++ }
++ periods = out;
++ };
++ // A short period (a turn and back, a few seconds at another size) joins its shorter neighbour: its frames are drawn
++ // on that canvas with bars rather than given a discontinuity of their own.
++ const len = (p) => p.end - p.start;
++ const shortest = () => periods.reduce((m, p, i) => (len(p) < len(periods[m]) ? i : m), 0);
++ while (periods.length > 1) {
++ const i = shortest();
++ if (len(periods[i]) >= MIN_PERIOD_S && periods.length <= MAX_PERIODS) break;
++ const prev = periods[i - 1];
++ const next = periods[i + 1];
++ merge(i, !next || (prev && len(prev) <= len(next)) ? i - 1 : i + 1);
++ regroup();
++ }
++ return periods.map((p) => ({ start: p.start, end: p.end, ...canvasOf(p.runs) }));
++}
++
++/** Whether the probe found more than one size: the cut must re-encode on the planned canvases. */
++function varies(changes) {
++ return changes.length > 1;
++}
++
++/** Fit any frame on a width×height canvas: scaled to fit (never stretched), centred, bars for the rest. */
++function fit(width, height) {
++ return `scale=${width}:${height}:force_original_aspect_ratio=decrease:force_divisible_by=2,pad=${width}:${height}:(ow-iw)/2:(oh-ih)/2,setsar=1`;
++}
++
++/**
++ * Cut `periods` of `source` into `out` as HLS fMP4, one ffmpeg run per period.
++ * start the input's start time (probe().start): the first period's output starts at 0
++ * encode(p, i) the codec arguments of period i (video encoder + audio codec); `-vf`/`-af` are this module's (the
++ * period's trim, its timestamps, the fit on its canvas: a rendition passes periods on its own canvases)
++ * audio whether the input has an audio stream (mapped and trimmed with the video)
++ * → [{ name, seconds, init }] in order. Throws JobError when a run fails or writes nothing.
++ */
++async function cut({ source, periods, start = 0, out, segmentSeconds, encode, audio, signal, timeoutMs, niceness = 15 }) {
++ const list = [];
++ let next = 1;
++ for (const [i, p] of periods.entries()) {
++ const init = i === 0 ? timeline.INIT_NAME : timeline.shapeInitName(i + 1);
++ const from = i === 0 ? start : p.start;
++ const edges = [i > 0 ? `start=${secs(p.start)}` : null, i + 1 < periods.length ? `end=${secs(p.end)}` : null].filter(Boolean).join(':');
++ const trim = (f) => (edges ? `${f}=${edges},` : '');
++ const playlist = path.join(out, `index-${i + 1}.m3u8`);
++ // -ss only finds the keyframe at or before the period's start (an absolute timestamp, nothing dropped by
++ // ffmpeg itself): the trim on the input's own timestamps is what cuts.
++ const r = await derive.ffmpeg(['-y', '-nostdin', '-v', 'error', ...derive.inputArgs(source), '-copyts',
++ ...(i > 0 ? ['-seek_timestamp', '1', '-noaccurate_seek', '-ss', secs(p.start)] : []), '-i', source.input,
++ '-map', '0:v:0', ...(audio ? ['-map', '0:a:0?'] : []),
++ '-vf', `${trim('trim')}setpts=PTS-${secs(from)}/TB,${fit(p.width, p.height)}`,
++ ...(audio ? ['-af', `${trim('atrim')}asetpts=PTS-${secs(from)}/TB`] : []),
++ ...encode(p, i),
++ '-map_metadata', '-1', '-fflags', '+bitexact', '-flags', '+bitexact',
++ ...(from > start ? ['-output_ts_offset', secs(from - start)] : []),
++ '-f', 'hls', '-hls_time', String(segmentSeconds), '-hls_list_size', '0', '-hls_playlist_type', 'vod',
++ // Negative composition offsets (CMAF's own rule): a period's first frame is presented at its decode time.
++ // Without them the encoder's B-frame delay leaves a gap at each join, which hls.js jumps and Chrome's
++ // native HLS stalls on.
++ '-hls_segment_type', 'fmp4', '-hls_segment_options', 'movflags=+negative_cts_offsets', '-hls_fmp4_init_filename', init,
++ '-start_number', String(next), '-hls_segment_filename', path.join(out, '%06d.m4s'), playlist],
++ { signal, timeoutMs, niceness });
++ if (signal && signal.aborted) throw signal.reason || new Error('aborted');
++ const tail = String(r.stderr).trim().split('\n').slice(-2).join(' ').slice(0, 300);
++ if (!r.ok) {
++ const unsupported = /Unknown encoder|not currently supported in container|could not find tag for codec|codec not supported/i.test(r.stderr);
++ throw new JobError(unsupported ? 'codec_unsupported' : 'ffmpeg_failed', `ffmpeg exited ${r.code} (period ${i + 1} of ${periods.length}): ${tail}`, { permanent: unsupported });
++ }
++ const made = derive.existing(playlist) ? require('./segments').parsePlaylist(fs.readFileSync(playlist, 'utf8')) : [];
++ if (!made.length || !derive.existing(path.join(out, init))) {
++ throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments for period ${i + 1} of ${periods.length} (${secs(p.start)}-${secs(p.end)} s): ${tail}`);
++ }
++ for (const s of made) list.push({ ...s, init });
++ next += made.length;
++ }
++ return list;
++}
++
++module.exports = { SAME_SHAPE, MIN_PERIOD_S, MAX_PERIODS, MAX_SIDE, probe, plan, varies, fit, cut, canvasOf };
ddiff --git a/server/objects/routes.js b/server/objects/routes.js
iindex 1e86ac2..03721b0 100644
---- a/server/objects/routes.js
++++ b/server/objects/routes.js
@@@ -806,8 +806,9 @@ const objectVariants = () => _objectVariants || (_objectVariants = require('../t
// A segment's copy is the placement router's choice; a packed one (F3.3) is a ranged read of its ~60 s chunk.
const HLS_TYPE = 'application/vnd.apple.mpegurl';
// A rendition's seq 0 init plus the two edge inits of a materialized clip's source rendition (negative seq,
--// docs/materialized-clips.md F3.6); every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.
--const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|\d{6,9}\.m4s)$/;
++// docs/materialized-clips.md F3.6) and the init of each shape period after the first (init-<n>.mp4, jobs/shapes.js);
++// every init row (seq <= 0) is served as video/mp4, a media segment as iso.segment.
++const SEGMENT_NAME = /^(init\.mp4|init-head\.mp4|init-tail\.mp4|init-\d{1,4}\.mp4|\d{6,9}\.m4s)$/;
const RENDITION_NAME = /^[a-z0-9_-]{1,32}$/;
const SLICE_FETCH_MS = 15000; // a ranged read of one packed segment from B2/R2
// clipWindow/timelineOf (a clip's window over its source, and the rows its playlists are written from) live in
ddiff --git a/server/objects/timeline.js b/server/objects/timeline.js
iindex e411a7b..6a79208 100644
---- a/server/objects/timeline.js
++++ b/server/objects/timeline.js
@@@ -45,6 +45,16 @@ const INIT_NAME = 'init.mp4';
// negative seq (-1 head, -2 tail) so they never collide with the media segments or the rendition's seq 0 init.
const HEAD_INIT = 'init-head.mp4';
const TAIL_INIT = 'init-tail.mp4';
++// A video whose picture changes shape (a phone turned, a resolution lowered mid-stream) is cut in periods, one init each
++// (jobs/shapes.js): the first period's is init.mp4 (seq 0), period n >= 2's init-<n>.mp4 at seq -(100 + n), clear of a
++// materialized clip's -1/-2 edges. Its media rows name it in init_name.
++const SHAPE_INIT = /^init-(\d{1,4})\.mp4$/;
++function shapeInitName(n) { return `init-${n}.mp4`; }
++/** The seq of a shape period's init row, or null for any other name. */
++function shapeInitSeq(name) {
++ const m = SHAPE_INIT.exec(String(name || ''));
++ return m && Number(m[1]) >= 2 ? -(100 + Number(m[1])) : null;
++}
const FIELDS = ['name', 'start_ms', 'end_ms', 'keyframe_ms', 'key', 'local_path', 'durable_provider', 'packed_object_id', 'byte_offset', 'byte_length', 'sha256', 'durability', 'init_name'];
function localRoot() { return path.join(path.resolve(config.objects.path), '.timeline'); }
@@@ -406,21 +416,25 @@ function masterPlaylist(renditions, { query = '' } = {}) {
* A materialized clip's rows carry negative-seq edge inits and per-segment init_name: a virtual clip over it would
* decode its edges with the wrong init, so building one is refused (the simplest correct answer, §Schema of
* docs/materialized-clips.md). Rows made only of copies of source rows (a clip whose window sits on boundaries) carry
-- * no edge init and still work.
++ * no edge init and still work. A source cut in shape periods (shapeInitName) is not refused: the clip carries the
++ * period inits its segments name, as they are, and each segment keeps its init_name.
*/
function clipRows(rows, startMs, endMs) {
-- if (rows.some((r) => Number(r.seq) < 0 || (r.init_name && r.init_name !== INIT_NAME))) return [];
++ const edge = (name) => name === HEAD_INIT || name === TAIL_INIT;
++ if (rows.some((r) => (Number(r.seq) < 0 && shapeInitSeq(r.name) == null) || edge(r.init_name))) return [];
const init = rows.find((r) => Number(r.seq) === 0);
const segs = rows
.filter((r) => Number(r.seq) > 0 && Number(r.end_ms) > startMs && Number(r.start_ms) < endMs)
.map((r) => ({ ...r, start_ms: Math.max(Number(r.start_ms), startMs) - startMs, end_ms: Math.min(Number(r.end_ms), endMs) - startMs }))
.filter((r) => r.end_ms > r.start_ms)
.map((r, i) => ({ ...r, seq: i + 1 }));
-- return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...segs] : segs;
++ const named = new Set(segs.map((r) => r.init_name).filter((n) => n && n !== INIT_NAME));
++ const shapeInits = rows.filter((r) => Number(r.seq) < 0 && named.has(r.name)).map((r) => ({ ...r, start_ms: 0, end_ms: 0 }));
++ return segs.length && init ? [{ ...init, seq: 0, start_ms: 0, end_ms: 0 }, ...shapeInits, ...segs] : segs;
}
module.exports = {
-- SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName,
++ SOURCE, INIT_NAME, HEAD_INIT, TAIL_INIT, FIELDS, localRoot, localPathFor, keyFor, segmentName, shapeInitName, shapeInitSeq,
list, segments, byName, has, segmentAt, replace, TimelineChanged, isNamed, namedElsewhere, deleteBytes, removeObject, sweepPendingRemovals, markPacked, durableProvider, locationsOf,
mediaPlaylist, masterPlaylist, clipRows,
};
ddiff --git a/server/vod/clip-cutter.js b/server/vod/clip-cutter.js
iindex 056e9dc..af7efe7 100644
---- a/server/vod/clip-cutter.js
++++ b/server/vod/clip-cutter.js
@@@ -26,6 +26,21 @@ const MAX_CONCURRENT = parseInt(process.env.CLIP_MAX_CONCURRENT_FFMPEG || '2', 1
const QUEUE_WAIT_MS = parseInt(process.env.CLIP_QUEUE_WAIT_MS || '1800000', 10);
let _active = 0;
++/**
++ * The -vf of a cut whose window spans a change of the picture's shape (a phone turned, a resolution lowered:
++ * jobs/shapes.js): the window's main shape on its canvas, the rest fitted with bars. One encode would otherwise scale
++ * every frame to the first size. One size in the window (or a probe that fails) → no filter, the cut as before.
++ */
++async function shapeFilter(source, ss, dur, isUrl) {
++ const shapes = require('../jobs/shapes');
++ try {
++ const read = await shapes.probe({ input: source, remote: isUrl }, { from: ss, to: ss + dur, timeoutMs: 60000 });
++ if (!shapes.varies(read.changes)) return [];
++ const main = shapes.plan(read.changes, { end: ss + dur }).reduce((a, b) => (b.end - b.start > a.end - a.start ? b : a));
++ return ['-vf', shapes.fit(main.width, main.height)];
++ } catch { return []; }
++}
++
function _ffprobe(file) {
return new Promise((resolve) => {
const ff = spawn('ffprobe', ['-v', 'error', '-print_format', 'json', '-show_format', '-show_streams', file], { stdio: ['ignore', 'pipe', 'ignore'] });
@@@ -76,6 +91,7 @@ async function cutClipFile({ source, startTime, duration }) {
const outPath = path.join(CLIPS_DIR, filename);
_active++;
++ const vf = await shapeFilter(source, ss, dur, isUrl);
let timedOut = false;
let exitCode = null;
const _stderrRef = { get: () => '' };
@@@ -87,7 +103,7 @@ async function cutClipFile({ source, startTime, duration }) {
// an opening of black/frozen frames.
const ok = await new Promise((resolve) => {
const args = [
-- '-y', '-ss', String(ss), '-i', source, '-t', String(dur),
++ '-y', '-ss', String(ss), '-i', source, '-t', String(dur), ...vf,
'-c:v', 'libvpx', '-b:v', '2000k', '-crf', '18', '-deadline', 'realtime', '-cpu-used', '4',
'-force_key_frames', 'expr:gte(t,n_forced*2)', '-c:a', 'libopus', '-b:a', '128k',
'-avoid_negative_ts', 'make_zero', '-f', 'webm', outPath,
ddiff --git a/server/vod/clip-materialize.js b/server/vod/clip-materialize.js
iindex 5d90ccf..f5fb673 100644
---- a/server/vod/clip-materialize.js
++++ b/server/vod/clip-materialize.js
@@@ -49,8 +49,9 @@ function locationOf(row) {
* Plan a clip window over one rendition's rows.
* → { originMs, endMs, durationMs, sourceInit, sourceRows, pieces } or null.
* A piece is { kind: 'copy', source } or { kind: 'head'|'tail', source, start_ms, end_ms, init } (source milliseconds).
-- * sourceRows are the source rows the plan copies and therefore names: the seq 0 init when an interior copy needs it
-- * (init_name NULL rows decode with it), plus every copied segment. The pin compares exactly these.
++ * sourceRows are the source rows the plan copies and therefore names: each init an interior copy decodes with (the
++ * seq 0 init for init_name NULL rows; a shape period's own init, jobs/shapes.js, for the rows that name it), plus every
++ * copied segment. The pin compares exactly these. `initOf(row)` is the source init a segment decodes with.
*/
function planWindow(rows, startMs, endMs) {
const segs = rows.filter((r) => Number(r.seq) > 0).sort((a, b) => Number(a.seq) - Number(b.seq));
@@@ -88,10 +89,15 @@ function planWindow(rows, startMs, endMs) {
const snapEndMs = lastPiece.kind === 'copy' ? Number(lastPiece.source.end_ms) : lastPiece.end_ms;
if (!(snapEndMs > originMs)) return null;
const sourceInit = rows.find((r) => Number(r.seq) === 0) || null;
++ const initOf = (r) => previews.initRowFor(rows, r);
const sourceRows = [];
-- if (pieces.some((p) => p.kind === 'copy') && sourceInit) sourceRows.push(sourceInit);
++ for (const p of pieces) {
++ if (p.kind !== 'copy') continue;
++ const init = initOf(p.source);
++ if (init && !sourceRows.includes(init)) sourceRows.push(init);
++ }
for (const p of pieces) if (p.kind === 'copy') sourceRows.push(p.source);
-- return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces };
++ return { originMs, endMs: snapEndMs, durationMs: snapEndMs - originMs, sourceInit, sourceRows, pieces, initOf };
}
/** A copied source segment row as the clip's own: the location, bytes and times, seq renumbered, times relative. */
@@@ -102,17 +108,20 @@ function copiedRow(source, seq, originMs) {
row.start_ms = Number(source.start_ms) - originMs;
row.end_ms = Number(source.end_ms) - originMs;
row.keyframe_ms = source.keyframe_ms == null ? null : Number(source.keyframe_ms) - originMs;
-- row.init_name = null; // decodes with the rendition's seq 0 init
++ // Decodes with the rendition's seq 0 init, or with the shape period's init it names (copied beside it, same name).
++ row.init_name = timeline.shapeInitSeq(source.init_name) != null ? source.init_name : null;
return row;
}
--/** The source's seq 0 init as the clip's own row (seq 0, times 0), or null. */
++/** A source init as the clip's own row (times 0): the seq 0 init at seq 0, a shape period's init at its own seq and name. */
function copiedInitRow(source) {
-- const row = copiedRow(source, 0, 0);
-- row.name = timeline.INIT_NAME;
++ const shape = timeline.shapeInitSeq(source.name);
++ const row = copiedRow(source, shape != null ? shape : 0, 0);
++ row.name = shape != null ? source.name : timeline.INIT_NAME;
row.start_ms = 0;
row.end_ms = 0;
row.keyframe_ms = null;
++ row.init_name = null;
return row;
}
@@@ -266,7 +275,7 @@ async function materialize({ job, ctx, clip, vod }) {
if (piece.kind === 'copy') continue;
segments.aborted(ctx);
const seq = i + 1;
-- const made = await encodeEdge({ ctx, piece, initRow: plan.sourceInit, dir: path.join(dir, `edge-${piece.kind}-${seq}`) });
++ const made = await encodeEdge({ ctx, piece, initRow: plan.initOf(piece.source), dir: path.join(dir, `edge-${piece.kind}-${seq}`) });
edgeSegments.set(seq, await placeFile({ clipObj, prev, seq, name: timeline.segmentName(seq), file: made.segFile,
contentType: 'video/iso.segment', target, remoteConfigured, stagedLocal, stagedRemote, failed, ctx }));
edgeInits.set(piece.init, await placeFile({ clipObj, prev, seq: piece.init === timeline.HEAD_INIT ? -1 : -2, name: piece.init, file: made.initFile,
@@@ -281,7 +290,7 @@ async function materialize({ job, ctx, clip, vod }) {
}
if (edgeInits.has(timeline.HEAD_INIT)) rows.push({ seq: -1, name: timeline.HEAD_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.HEAD_INIT), init_name: null });
if (edgeInits.has(timeline.TAIL_INIT)) rows.push({ seq: -2, name: timeline.TAIL_INIT, start_ms: 0, end_ms: 0, keyframe_ms: null, ...edgeInits.get(timeline.TAIL_INIT), init_name: null });
-- if (plan.pieces.some((p) => p.kind === 'copy') && plan.sourceInit) rows.push(copiedInitRow(plan.sourceInit));
++ for (const init of plan.sourceRows.filter((r) => Number(r.seq) <= 0)) rows.push(copiedInitRow(init));
rows.sort((a, b) => Number(a.seq) - Number(b.seq));
segments.aborted(ctx);
await pinRows({ clipObjectId: clip.object_id, sourceObjectId: vod.object_id, rows, sourceRows: plan.sourceRows, jobId: job.id });
ddiff --git a/test/shape-periods.test.js b/test/shape-periods.test.js
nnew file mode 100644
iindex 0000000..b2a3368
---- /dev/null
++++ b/test/shape-periods.test.js
@@@ -0,0 +1,170 @@
++'use strict';
++// A recording whose picture changes shape mid-stream (server/jobs/shapes.js): a phone turned from landscape to portrait,
++// then WebRTC lowered the resolution. VOD 6017 was cut as one encode, and every portrait frame was stretched to the
++// first 1280x720. Now the cut is one period per shape, each on its own canvas with its own init segment, and the
++// playlists, virtual clips and materialized clips carry the right init for each segment.
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const { spawnSync } = require('child_process');
++
++(async () => {
++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-shapes-'));
++ const dir = (name) => { const p = path.join(tmp, name); fs.mkdirSync(p, { recursive: true }); return p; };
++ Object.assign(process.env, {
++ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'), THUMBNAILS_PATH: dir('thumbnails'),
++ PASTES_PATH: dir('pastes'), OBJECTS_PATH: dir('objects'), MEDIA_HLS_ENABLED: '1', MEDIA_CMAF_TRANSCODE: '1',
++ MEDIA_UPLOAD_MIN_FREE_MB: '0', MEDIA_INVARIANT_SCAN_HOURS: '0',
++ });
++ for (const p of ['B2', 'R2']) for (const k of ['ENDPOINT', 'BUCKET', 'KEY_ID', 'APP_KEY', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']) process.env[`MEDIA_${p}_${k}`] = '';
++ try {
++ const shapes = require('../server/jobs/shapes');
++ const timeline = require('../server/objects/timeline');
++ const at = (list) => list.map(([t, width, height]) => ({ t, width, height }));
++
++ // ── Planning ──
++ // VOD 6017's own keyframe sizes: 16:9 for 16 minutes, then portrait at three sizes. Two periods, the portrait one on
++ // the largest portrait size; a lowered resolution is not a period of its own.
++ const vod6017 = at([[0.2, 1280, 720], [976.243, 720, 1280], [1079.895, 540, 960], [1144.988, 360, 640], [1481.092, 540, 960],
++ [1546.278, 360, 640], [1756.995, 540, 960], [1942.652, 360, 640], [2148.22, 540, 960], [2228.501, 360, 640],
++ [2373.997, 540, 960], [2474.381, 720, 1280], [2569.747, 540, 960]]);
++ assert.deepStrictEqual(shapes.plan(vod6017, { end: 2592.2 }), [
++ { start: 0.2, end: 976.243, width: 1280, height: 720 },
++ { start: 976.243, end: 2592.2, width: 720, height: 1280 },
++ ]);
++ // A two-second turn and back joins its neighbour (drawn with bars), and the two landscape runs become one.
++ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [100, 720, 1280], [102, 1280, 720], [300, 960, 540]]), { end: 400 }),
++ [{ start: 0, end: 400, width: 1280, height: 720 }]);
++ // A canvas keeps its shape under MAX_SIDE, on even sides.
++ assert.deepStrictEqual(shapes.plan(at([[0, 1280, 720], [50, 2161, 3841]]), { end: 100 }),
++ [{ start: 0, end: 50, width: 1280, height: 720 }, { start: 50, end: 100, width: 1080, height: 1920 }]);
++ // A phone turned every few seconds for an hour still has at most MAX_PERIODS periods.
++ const flips = [];
++ for (let t = 0; t < 3600; t += 6) flips.push([t, t % 12 ? 720 : 1280, t % 12 ? 1280 : 720]);
++ const many = shapes.plan(at(flips), { end: 3600 });
++ assert.ok(many.length <= shapes.MAX_PERIODS, `${many.length} periods`);
++ assert.strictEqual(many[0].start, 0);
++ assert.strictEqual(many[many.length - 1].end, 3600);
++ for (let i = 1; i < many.length; i++) assert.strictEqual(many[i].start, many[i - 1].end, 'periods tile the video');
++ assert.strictEqual(shapes.varies(at([[0, 640, 360]])), false);
++ assert.strictEqual(shapes.varies(at([[0, 640, 360], [9, 320, 180]])), true);
++ assert.match(shapes.fit(720, 1280), /^scale=720:1280:force_original_aspect_ratio=decrease.*pad=720:1280:\(ow-iw\)\/2:\(oh-ih\)\/2,setsar=1$/);
++
++ // ── Names ──
++ assert.strictEqual(timeline.shapeInitName(2), 'init-2.mp4');
++ assert.strictEqual(timeline.shapeInitSeq('init-2.mp4'), -102);
++ for (const name of ['init.mp4', 'init-1.mp4', 'init-head.mp4', 'init-tail.mp4', '000002.m4s', null]) assert.strictEqual(timeline.shapeInitSeq(name), null, String(name));
++
++ // ── The cut, the playlist and the clips (needs ffmpeg with libx264, libvpx, libopus, as cmaf-transcode) ──
++ if (spawnSync('ffmpeg', ['-version']).status !== 0 || spawnSync('ffprobe', ['-version']).status !== 0) {
++ console.log('✅ shape periods: planning and names (cut skipped: ffmpeg not found)');
++ return;
++ }
++ const ffmpeg = (...args) => {
++ const r = spawnSync('ffmpeg', ['-v', 'error', '-y', ...args], { encoding: 'utf8' });
++ assert.strictEqual(r.status, 0, r.stderr);
++ };
++ const frames = (file) => {
++ const r = spawnSync('ffprobe', ['-v', 'error', '-select_streams', 'v:0', '-show_entries', 'frame=width,height', '-of', 'csv=p=0', file], { encoding: 'utf8' });
++ assert.strictEqual(r.status, 0, r.stderr);
++ return r.stdout.trim().split('\n').filter(Boolean).map((l) => l.split(',').slice(0, 2).join(','));
++ };
++ // The recorder's stream copy: one VP8 track whose keyframes change size, the header naming the first (320x180).
++ // The landscape part is 6 s, the portrait one 6 s at 180x320 then 4 s at 90x160, all at 10 fps; the copy starts at
++ // 0.2 s, as a recording's first keyframe often does.
++ const part = (name, src, size, seconds, freq) => {
++ const file = path.join(tmp, name);
++ ffmpeg('-f', 'lavfi', '-i', `${src}=duration=${seconds}:size=${size}:rate=10`, '-f', 'lavfi', '-i', `sine=duration=${seconds}:frequency=${freq}`,
++ '-c:v', 'libvpx', '-deadline', 'realtime', '-cpu-used', '8', '-g', '10', '-c:a', 'libopus', '-shortest', file);
++ return file;
++ };
++ const parts = [part('a.webm', 'testsrc', '320x180', 6, 440), part('b.webm', 'testsrc2', '180x320', 6, 660), part('c.webm', 'testsrc2', '90x160', 4, 880)];
++ fs.writeFileSync(path.join(tmp, 'list.txt'), parts.map((p) => `file '${p}'`).join('\n'));
++ const mixed = path.join(tmp, 'mixed.webm');
++ ffmpeg('-f', 'concat', '-safe', '0', '-i', path.join(tmp, 'list.txt'), '-c', 'copy', '-output_ts_offset', '0.2', mixed);
++ const sourceFrames = frames(mixed).length;
++
++ const read = await shapes.probe({ input: mixed, remote: false });
++ assert.deepStrictEqual(read.changes.map((c) => `${c.width}x${c.height}`), ['320x180', '180x320', '90x160']);
++ assert.ok(Math.abs(read.changes[1].t - read.start - 6) < 0.1, `the turn at ~6 s (${read.changes[1].t - read.start})`);
++
++ const db = require('../server/db/database');
++ const model = require('../server/objects/model');
++ const cmaf = require('../server/jobs/cmaf');
++ await db.upsertApp({ app_id: 'live', api_key: 'shapes-test' });
++ const object = async (file, mime) => {
++ const id = await model.createObject({ app_id: 'live', owner_user_id: 1, visibility: 'public', lifecycle_status: 'ready',
++ kind: 'file', mime_type: mime, size_bytes: fs.statSync(file).size });
++ await model.upsertLocation(id, { provider: 'local', key: file, state: 'present', size_bytes: fs.statSync(file).size, verified: true });
++ return id;
++ };
++ const run = async (id, suffix) => await cmaf.spec.run({ id: `mjob_${suffix}`, app_id: 'live', object_id: id, params: { segment_seconds: 2 } },
++ { signal: new AbortController().signal });
++ const vod = await object(mixed, 'video/webm');
++ const made = await run(vod, 'shapes');
++ assert.deepStrictEqual([made.video, made.audio, made.shapes], ['h264', 'aac', 2]);
++
++ const rows = await timeline.list(vod, 'source');
++ const init2 = rows.find((r) => r.name === 'init-2.mp4');
++ assert.ok(init2, 'the portrait period has its own init');
++ assert.strictEqual(Number(init2.seq), -102);
++ assert.strictEqual(rows.filter((r) => Number(r.seq) <= 0).length, 2, 'two inits, one per period');
++ const segs = rows.filter((r) => Number(r.seq) > 0);
++ const first = segs.filter((r) => !r.init_name);
++ const second = segs.filter((r) => r.init_name === 'init-2.mp4');
++ assert.ok(first.length && second.length && first.length + second.length === segs.length);
++ assert.ok(Number(first[first.length - 1].seq) < Number(second[0].seq), 'the periods follow each other');
++ // Every frame is on its period's canvas, never the other's: the portrait part is 180x320 (its 90x160 seconds
++ // scaled up into it), and no frame of either shape lands in the other's period.
++ const initFile = (r) => (r.init_name ? init2 : rows.find((x) => Number(x.seq) === 0)).local_path;
++ const sizes = (list) => {
++ const joined = path.join(tmp, 'joined.mp4');
++ const out = [];
++ for (const r of list) {
++ fs.writeFileSync(joined, Buffer.concat([fs.readFileSync(initFile(r)), fs.readFileSync(r.local_path)]));
++ out.push(...frames(joined));
++ }
++ return out;
++ };
++ const landscape = sizes(first);
++ const portrait = sizes(second);
++ assert.ok(landscape.every((s) => s === '320,180'), `landscape period: ${[...new Set(landscape)]}`);
++ assert.ok(portrait.every((s) => s === '180,320'), `portrait period: ${[...new Set(portrait)]}`);
++ assert.ok(Math.abs(landscape.length - 60) <= 1, `${landscape.length} landscape frames, 60 sent`);
++ assert.ok(Math.abs(landscape.length + portrait.length - sourceFrames) <= 1, `${landscape.length + portrait.length} frames of ${sourceFrames}`);
++ // The timeline tiles the whole video.
++ assert.ok(Math.abs(Number(segs[segs.length - 1].end_ms) - 16000) < 400, `ends at ${segs[segs.length - 1].end_ms} ms`);
++
++ // The playlist switches init with a discontinuity between the periods, exactly once.
++ const playlist = timeline.mediaPlaylist(rows);
++ assert.strictEqual((playlist.match(/#EXT-X-DISCONTINUITY/g) || []).length, 1);
++ assert.match(playlist, /#EXT-X-MAP:URI="init\.mp4"[\s\S]*#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);
++
++ // A virtual clip across the turn carries init-2 and each segment's init; one inside the portrait part opens on it.
++ const turnMs = Number(second[0].start_ms);
++ const across = timeline.clipRows(rows, turnMs - 2500, turnMs + 2500);
++ assert.ok(across.some((r) => r.name === 'init-2.mp4' && Number(r.seq) === -102));
++ assert.match(timeline.mediaPlaylist(across), /#EXT-X-DISCONTINUITY\n#EXT-X-MAP:URI="init-2\.mp4"/);
++ const inside = timeline.clipRows(rows, turnMs + 1000, turnMs + 5000);
++ assert.match(timeline.mediaPlaylist(inside), /^#EXTM3U[\s\S]*?#EXT-X-MAP:URI="init-2\.mp4"/);
++ assert.ok(!/DISCONTINUITY/.test(timeline.mediaPlaylist(inside)));
++ // A materialized clip's own rows over the source (the edges keep refusing a virtual clip, as before).
++ assert.deepStrictEqual(timeline.clipRows([...rows, { seq: -1, name: timeline.HEAD_INIT }], 0, 4000), []);
++ const materialize = require('../server/vod/clip-materialize');
++ const plan = materialize.planWindow(rows, Number(second[0].start_ms), Number(second[second.length - 1].end_ms));
++ assert.ok(plan.sourceRows.some((r) => r.name === 'init-2.mp4'), 'the pin names the period init it copies');
++ assert.ok(!plan.sourceRows.some((r) => Number(r.seq) === 0), 'and not the first period\'s');
++ assert.strictEqual(plan.initOf(second[0]).name, 'init-2.mp4');
++ const spanning = materialize.planWindow(rows, Number(first[0].start_ms), Number(second[second.length - 1].end_ms));
++ assert.deepStrictEqual(spanning.sourceRows.filter((r) => Number(r.seq) <= 0).map((r) => r.name).sort(), ['init-2.mp4', 'init.mp4']);
++
++ // One size throughout keeps the cut as it was: a stream copy of an H.264 MP4.
++ const plain = path.join(tmp, 'plain.mp4');
++ ffmpeg('-f', 'lavfi', '-i', 'testsrc=duration=5:size=160x90:rate=10', '-f', 'lavfi', '-i', 'sine=duration=5',
++ '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', '-shortest', plain);
++ const copied = await run(await object(plain, 'video/mp4'), 'plain');
++ assert.deepStrictEqual([copied.video, copied.audio, copied.shapes], ['copy', 'copy', 1]);
++ console.log('✅ shape periods: planning, one init per shape, frames on their canvas, playlists and clips');
++ } finally { fs.rmSync(tmp, { recursive: true, force: true }); }
++})().catch((err) => { console.error(err); process.exitCode = 1; });