openvibers/openvibe.media Commits

Clip trim fixes: a held clip is not trimmed (409 media.object.held) and a held re-cut keeps its old file, a trim that fails for good gives the clip its previous window and file back, a trimmed file clip is processing until its cut is in, a trim never joins a running cut, and a clip's object.cmaf is keyed on its file and commits only while that file is current (#154)

OP OpenVibers committed 22 days ago · adf7291be5 · 11 files, +283 −41

Changes

ddiff --git a/README.md b/README.md
iindex 14fb326..7339136 100644
---- a/README.md
++++ b/README.md
@@@ -285,7 +285,7 @@ lossless `.master.mkv` recovery archive. A `.seekable` sidecar is remuxed every
| POST | `/clips` | `{ vod_id, start_s, end_s, title?, description?, user_id?, visibility?, auto_generated?, materialize? }` → over a VOD with a CMAF timeline (`MEDIA_HLS_ENABLED`): a **virtual** clip, **201** ready at once (the clip, `storage_provider: 'timeline'`), played as HLS over the source's segments (`/c/:id` redirects to its playlist); with `materialize: true` or no timeline → **202** `{ id, status: 'processing', job_id }`; cut (`clip.cut`) runs in background — with `MEDIA_HLS_ENABLED` + `MEDIA_MATERIALIZED_CLIPS=1` and a source timeline it is **materialized** over the source's bytes (the clip's own media_timeline rows copy the interior segments; only the two window edges are re-encoded, docs/materialized-clips.md), otherwise the full re-encode (from the local file or a presigned B2/R2 URL); duplicate windows are deduplicated; live recordings are clamped to flushed footage. Multipart `video` = direct upload of an already-cut blob → **201** ready |
| GET | `/clips/:id` | status: `processing | ready | failed`; with `MEDIA_HLS_ENABLED` also `hls_url` (and `hls_expires_at` when signed) — the clip's playlist over its own timeline (a materialized clip's persisted rows) or, virtual, its source's window (docs/media-fabric.md §4) |
| POST | `/clips/:id/recut` | `{ materialize? }` → a file-less clip over a timeline turns virtual (200, the clip); otherwise (or `materialize: true`) **202** `{ id, status, job_id }` (`clip.cut`; a virtual clip stays ready while it is cut) |
--| POST | `/clips/:id/trim` | `{ start_s, end_s }`, seconds into the clip (at least 1 s apart) → the clip narrows in place: same id and title, its window of the source shrinks and it is cut again from the source (**202** `{ id, status: 'processing', job_id }`, `clip.cut`, `clip.ready` again; the superseded file is removed), or a virtual clip plays the narrower window at once (200, the clip); the whole clip → 200 `unchanged: true`; still being cut → 409; no source VOD (a browser upload) or a file moved to cloud storage → 422 |
++| POST | `/clips/:id/trim` | `{ start_s, end_s }`, seconds into the clip (at least 1 s apart) → the clip narrows in place: same id and title, its window of the source shrinks and it is cut again from the source (**202** `{ id, status: 'processing', job_id }`, `clip.cut`, `clip.ready` again; the superseded file is removed), or a virtual clip plays the narrower window at once (200, the clip); the whole clip → 200 `unchanged: true`; a file clip is `processing` from the answer until its cut is in; still being cut (or a queued/running `clip.cut`) → 409; under a retention hold (its own or its VOD's) → 409 `media.object.held`; no source VOD (a browser upload), a file moved to cloud storage or an unreadable source recording → 422; a cut that fails for good gives the clip its previous window and file back (`clip.ready` again) |
| GET | `/clips?limit&offset&vod_id&stream_id&user_id&channel_user_id&hide_self&include_private&order&auto_generated&status&since` | list; `channel_user_id` = clipped-channel owner; `include_private` app-key only; `auto_generated=1\|0` = cut by the app's automation (AI auto-clips) or made by a person; `status=ready` = playable clips only; `since` as for VODs |
| PUT | `/clips/:id` | `{ title?, visibility?, auto_generated? }` (`auto_generated` app key only, 403 when acting for a user) |
| DELETE | `/clips/:id` | local + offloaded objects + row; a timeline clip's own rows and edge bytes go with it, while every location the source still names — a materialized clip's interior — stays (docs/materialized-clips.md) |
ddiff --git a/docs/media-fabric.md b/docs/media-fabric.md
iindex 4bcb39a..eb071f8 100644
---- a/docs/media-fabric.md
++++ b/docs/media-fabric.md
@@@ -159,7 +159,9 @@ med_xyz/{captions/*.vtt, storyboard.webp+.vtt, waveform, metadata}
rule. No schema besides the two indexes; additive, an older release never reads it.
- **F3.c, shipped: the timeline queued by itself, signed playlists, the MP4 fallback.** A recording's finalize queues
`object.cmaf` for its object once it is ready (`server/objects/timeline-queue.js`; `dedupeActive` and the idempotency
-- key `object.cmaf:<id>`, so a second finalize joins the same job; a queue error is logged, never fails the finalize),
++ key `object.cmaf:<id>`, so a second finalize joins the same job; a clip's key also names its file,
++ `object.cmaf:<id>:<file version>`, because a trim or re-cut replaces the file in place and drops the rows, and a
++ clip's cut commits nothing once its file changed under it; a queue error is logged, never fails the finalize),
and `GET …/download?format=json` queues it lazily for a media object without a timeline (a VOD older than F3.1):
`hls_url` appears once the job ran. Nothing is queued with `MEDIA_HLS_ENABLED` off, for a non-media object or once
the source timeline exists. For a private or sandbox object `hls_url` is a **playlist token**: its own MAC purpose
ddiff --git a/server/jobs/cmaf.js b/server/jobs/cmaf.js
iindex ce644ae..021bd1b 100644
---- a/server/jobs/cmaf.js
++++ b/server/jobs/cmaf.js
@@@ -20,7 +20,8 @@
* Idempotent: a rerun makes the same bytes (bitexact), reuses a row whose sha256 matches and its location as it is, and
* uploads only what is not durable yet. A run that aborts or fails before the commit removes its staged files and keys.
* Refused: MEDIA_HLS_ENABLED off, a source that is not ready or not media (permanent), less free disk than the source
-- * size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent).
++ * size plus MEDIA_UPLOAD_MIN_FREE_MB (retried later), a codec fMP4 cannot carry (permanent), a clip whose file was
++ * replaced (a trim or re-cut) while it ran (permanent, media.source.changed: nothing is committed).
*/
'use strict';
@@@ -58,6 +59,10 @@ function validate({ obj, params }) {
async function run(job, ctx) {
if (!config.hls.enabled) throw disabled();
const src = await derive.loadSource(job);
++ // A clip can be cut again in place (a trim, a re-cut): its object keeps its id while its file changes. The rows are
++ // committed only while the file is still the one this run cut (read FOR SHARE in the commit's transaction, so a
++ // re-cut's row update waits for it); otherwise nothing is published and the next request queues the new file's job.
++ const version = await derive.fileVersion(src);
const source = await derive.resolveSource(src);
if (!source) throw new JobError('media_unavailable', 'The source bytes are unavailable (no local file and no cloud copy)', { permanent: true });
let video = 'copy';
@@@ -111,7 +116,12 @@ async function run(job, ctx) {
const list = derive.existing(path.join(out, 'index.m3u8')) ? segments.parsePlaylist(fs.readFileSync(path.join(out, 'index.m3u8'), 'utf8')) : [];
if (!list.length || !derive.existing(path.join(out, timeline.INIT_NAME))) throw new JobError('ffmpeg_failed', `ffmpeg wrote no segments: ${tail}`);
-- const published = await segments.publishRendition({ job, ctx, src, rendition, out, list, label: 'Cmaf' });
++ const check = version == null ? null : async () => {
++ if (await derive.fileVersion(src, { lock: true }) !== version) {
++ throw new JobError('media.source.changed', 'The clip was cut again while this ran; its new file gets a timeline of its own', { permanent: true });
++ }
++ };
++ const published = await segments.publishRendition({ job, ctx, src, rendition, out, list, label: 'Cmaf', check });
if (video === 'h264') require('../placement/metrics-binding').inc('media_cmaf_transcodes_total', { stream: 'video' });
if (audio === 'aac') require('../placement/metrics-binding').inc('media_cmaf_transcodes_total', { stream: 'audio' });
return {
ddiff --git a/server/jobs/derive.js b/server/jobs/derive.js
iindex 4455c6c..8c6aa70 100644
---- a/server/jobs/derive.js
++++ b/server/jobs/derive.js
@@@ -94,6 +94,20 @@ async function resolveSource(obj) {
return null;
}
++/**
++ * Which file a clip object is cut from now: the first 12 hex of sha256 over its row's file_path ('none' without one), or
++ * null for any other object (a VOD or native object keeps its bytes). A trim or re-cut writes a new file in place under
++ * the same object, so object.cmaf is keyed on this (objects/timeline-queue.js) and commits only while it is unchanged
++ * (cmaf.js). `lock` reads the row FOR SHARE, inside the caller's transaction: a re-cut's row update waits for it.
++ */
++async function fileVersion(obj, { lock = false } = {}) {
++ const ref = obj && model.parseLegacyRef(obj.legacy_ref);
++ if (!ref || ref.kind !== 'clip') return null;
++ const row = await db.get(`SELECT file_path FROM clips WHERE id = ?${lock ? ' FOR SHARE' : ''}`, [Number(ref.id)]);
++ const file = row && row.file_path ? String(row.file_path) : '';
++ return file ? crypto.createHash('sha256').update(file).digest('hex').slice(0, 12) : 'none';
++}
++
function freeBytes(dir) {
try { const st = fs.statfsSync(dir); return Number(st.bavail) * Number(st.bsize); } catch { return Infinity; }
}
@@@ -386,7 +400,7 @@ async function runRemux(job, ctx) {
module.exports = {
split: { lane: 'heavy', maxAttempts: 3, timeoutMs: 6 * 3600 * 1000, needsObject: true, validate: validateSplit, run: runSplit },
remux: { lane: 'heavy', maxAttempts: 3, timeoutMs: 3 * 3600 * 1000, needsObject: true, validate: validateRemux, run: runRemux },
-- resolveSource, isMediaObject, extFor,
++ resolveSource, fileVersion, isMediaObject, extFor,
// for the preview jobs (previews.js)
loadSource, adopt, ffmpeg, inputArgs, probeDuration, workDir, cleanupWork, sweepWork, existing, checkRoom,
// and object.cmaf (cmaf.js)
ddiff --git a/server/jobs/segments.js b/server/jobs/segments.js
iindex faacd59..a92f910 100644
---- a/server/jobs/segments.js
++++ b/server/jobs/segments.js
@@@ -84,9 +84,10 @@ async function storeRenditionMeta(objectId, rendition, meta) {
/**
* Publish `files` (producedFiles) of `rendition` for `src` from the work directory `out`. Returns the row counts and
* the totals the job result is built from. Throws JobError: the timeline changed under a pack, or a segment that is not
-- * durable yet. `label` names the job in warnings (e.g. 'Cmaf').
++ * durable yet. `label` names the job in warnings (e.g. 'Cmaf'). `check` is passed to timeline.replace: it runs in the
++ * commit's transaction and throws to publish nothing (the staged bytes are removed as for any failure before the commit).
*/
--async function publishRendition({ job, ctx, src, rendition, out, list, label = 'Segments' }) {
++async function publishRendition({ job, ctx, src, rendition, out, list, label = 'Segments', check = null }) {
const files = producedFiles(list);
// Read the init segment while it is still in the work directory (the loop below moves it to its content-addressed
// location): its codecs/resolution go into the object's metadata for the master playlist.
@@@ -168,7 +169,7 @@ async function publishRendition({ job, ctx, src, rendition, out, list, label = '
try {
// A source re-cut (object.cmaf) drops every other rendition's rows here, in this same transaction: the
// rendition was cut against the old source and would otherwise play beside the new one.
-- written = await timeline.replace(src.id, rendition, rows, { jobId: job.id, expect: prev, dropOtherRenditions: rendition === timeline.SOURCE });
++ written = await timeline.replace(src.id, rendition, rows, { jobId: job.id, expect: prev, dropOtherRenditions: rendition === timeline.SOURCE, check });
} catch (err) {
// An object.pack committed after `prev` was read: these rows would name the segment keys it deleted.
if (err instanceof timeline.TimelineChanged) throw new JobError('media.timeline.changed', 'The timeline changed while cutting (a pack); a retry cuts against the new rows', { retryAfterS: 60 });
ddiff --git a/server/objects/timeline-queue.js b/server/objects/timeline-queue.js
iindex a13d0dc..680de57 100644
---- a/server/objects/timeline-queue.js
++++ b/server/objects/timeline-queue.js
@@@ -6,7 +6,9 @@
* gets one on its first request. Nothing happens (null) with MEDIA_HLS_ENABLED off, for a missing, unready or non-media
* object, or once the timeline exists. One job per object: dedupeActive joins a queued or running cut, and the
* idempotency key `object.cmaf:<id>` (or `:transcode` when first queued with transcoding on) answers a later call with
-- * the same job. → the job id, or null.
++ * the same job. A clip's key also names its file (`object.cmaf:<id>:<file version>`, derive.fileVersion): a trim or
++ * re-cut gives the clip a new file and drops its rows, and its next request queues a job for the new file instead of
++ * replaying the old one. → the job id, or null.
*
* queuePack(appId, objectId, { rendition }) enqueues `object.pack` once a cut's rows are committed (server/jobs/cmaf.js,
* server/jobs/rendition.js), so packing follows the cut by itself instead of an operator's hand. Nothing happens (null)
@@@ -57,11 +59,14 @@ async function queueCmaf(appId, objectId, { segment_seconds = null, lazy = false
const params = require('../jobs/cmaf').spec.validate({ appId, obj, params: segment_seconds != null ? { segment_seconds } : {} });
// Mark jobs first queued with transcoding on. An older copy-only codec failure gets this one new key on a lazy
// request. A failed transcode job already has the key and is replayed, so lazy reads cannot start a retry loop.
-- let idempotencyKey = `object.cmaf:${obj.id}`;
++ // A clip is keyed on its file as well (derive.fileVersion): a trim or re-cut replaces the file under the same object and
++ // drops its rows, and the old key would only replay the job that cut the superseded file.
++ const version = await require('../jobs/derive').fileVersion(obj);
++ let idempotencyKey = version ? `object.cmaf:${obj.id}:${version}` : `object.cmaf:${obj.id}`;
if (config.hls.transcode) {
const last = await require('../db/database').get(`SELECT status, error_code, idempotency_key FROM media_jobs
-- WHERE app_id = ? AND object_id = ? AND job_type = 'object.cmaf'
-- ORDER BY created_at DESC, id DESC LIMIT 1`, [appId, obj.id]);
++ WHERE app_id = ? AND object_id = ? AND job_type = 'object.cmaf'${version ? ' AND idempotency_key IN (?, ?)' : ''}
++ ORDER BY created_at DESC, id DESC LIMIT 1`, version ? [appId, obj.id, idempotencyKey, `${idempotencyKey}:transcode`] : [appId, obj.id]);
if (!last || last.idempotency_key === `${idempotencyKey}:transcode` ||
(lazy && last.status === 'failed' && last.error_code === 'codec_unsupported')) idempotencyKey += ':transcode';
}
ddiff --git a/server/objects/timeline.js b/server/objects/timeline.js
iindex 4259082..e411a7b 100644
---- a/server/objects/timeline.js
++++ b/server/objects/timeline.js
@@@ -102,12 +102,15 @@ class TimelineChanged extends Error {}
* `dropOtherRenditions` (a source re-cut) also removes every other rendition's rows of the object IN THE SAME
* TRANSACTION: a rendition cut against the old source can never play against the new one. The removed rows are returned
* as `dropped`, for the caller to delete their bytes after the commit (deleteBytes, guarded by isNamed).
++ * `check` (async, optional) runs inside the transaction before anything is written; it throws to write nothing (cmaf.js:
++ * the clip's file is still the one that was cut).
*/
--async function replace(objectId, rendition, rows, { jobId = null, expect = null, dropOtherRenditions = false } = {}) {
++async function replace(objectId, rendition, rows, { jobId = null, expect = null, dropOtherRenditions = false, check = null } = {}) {
const out = { inserted: 0, updated: 0, unchanged: 0, removed: 0 };
let dropped = null;
await db.getDb().tx(async () => {
await lockObject(objectId);
++ if (check) await check();
const before = new Map((await list(objectId, rendition)).map((r) => [Number(r.seq), r]));
if (expect) {
const at = (r) => (r ? ['key', 'local_path', 'durable_provider', 'packed_object_id', 'sha256'].map((f) => (r[f] == null ? '' : String(r[f]))).join('\n') : null);
ddiff --git a/server/vod/clip-jobs.js b/server/vod/clip-jobs.js
iindex 911c9cc..85b33c0 100644
---- a/server/vod/clip-jobs.js
++++ b/server/vod/clip-jobs.js
@@@ -133,10 +133,17 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false, job = null
const tl = await require('../objects/timeline').removeObject(clip.object_id)
.catch(err => { console.warn(`[Clips] Clip ${clipId} timeline cleanup failed: ${err.message}`); return null; });
if (tl && tl.pending) console.warn(`[Clips] Clip ${clipId}: ${tl.pending} timeline segment row(s) kept — their durable copy could not be deleted; the object sweep retries`);
++ // A clip that played as HLS (its rows just went with the old cut) gets the new file's timeline now, not on a later read.
++ if (tl && tl.removed > 0 && config.hls.enabled) {
++ await require('../objects/timeline-queue').queueCmaf(clip.app_id, clip.object_id)
++ .catch(err => console.warn(`[Clips] Clip ${clipId}: queueing the new cut's timeline failed: ${err.message}`));
++ }
// A clip that had a file already (a trim, a re-cut of a ready clip) leaves its previous cut behind: the row names the new one now.
++ // Under a retention hold (its own or its source VOD's) the previous cut is kept: a hold keeps bytes, whatever the row says.
const old = clip.file_path ? path.resolve(clip.file_path) : null;
if (old && old !== path.resolve(cut.filePath) && path.dirname(old) === cutter.CLIPS_DIR) {
-- try { fs.unlinkSync(old); } catch { /* already gone */ }
++ if (await objects().isHeldRow(clip)) console.warn(`[Clips] Clip ${clipId} is under a retention hold: its previous cut ${path.basename(old)} is kept`);
++ else { try { fs.unlinkSync(old); } catch { /* already gone */ } }
}
console.log(`[Clips] Clip ${clipId} ${reason} OK from vod ${clip.vod_id} (${startTime.toFixed(1)}-${(startTime + cut.duration).toFixed(1)}s, attempt ${attempt})`);
// A public, person-made clip that just became ready is indexable: announce its watch page and the sitemap.
@@@ -281,14 +288,52 @@ async function clipObjectId(appId, clipId) {
return row ? row.id : null;
}
--/** Queue the cut of a clip row → { job, created }. A repeated request for the same clip reuses its active job. */
--async function enqueueCut(appId, clipId, { reason = 'cut', createdBy = null, ownerUserId = null } = {}) {
++/**
++ * Queue the cut of a clip row → { job, created }. A repeated request for the same clip reuses its active job. `prev` (a
++ * trim) is the window the clip had before it was narrowed, { start_time, end_time, duration_seconds }: a trim whose cut
++ * fails for good puts it back (restoreTrim), so the clip plays its previous cut again instead of turning failed.
++ */
++async function enqueueCut(appId, clipId, { reason = 'cut', createdBy = null, ownerUserId = null, prev = null } = {}) {
const queue = require('../jobs/queue');
-- const out = await queue.enqueue({ appId, type: 'clip.cut', objectId: await clipObjectId(appId, clipId), params: { clip_id: Number(clipId), reason }, dedupeActive: true, createdBy, ownerUserId });
++ const params = { clip_id: Number(clipId), reason, ...(prev ? { prev } : {}) };
++ const out = await queue.enqueue({ appId, type: 'clip.cut', objectId: await clipObjectId(appId, clipId), params, dedupeActive: true, createdBy, ownerUserId });
try { require('../jobs/worker').kick(); } catch { /* the poll picks it up */ }
return out;
}
++/** The queued or running clip.cut job of a clip, or null. */
++async function activeCut(appId, clipId) {
++ return await db.get(`SELECT id, status FROM media_jobs WHERE app_id = ? AND job_type = 'clip.cut' AND status IN ('queued', 'running')
++ AND CAST(json_extract(params, '$.clip_id') AS INTEGER) = ? ORDER BY id DESC LIMIT 1`, [appId, Number(clipId)]) || null;
++}
++
++/**
++ * A trim whose cut failed for good: the clip gets the window it had before (`prev`) and, a file clip, is ready again on
++ * the file it already had (the superseded file is only removed once a new cut is in, so it is still there), with its
++ * clip.ready event. A virtual or materialized clip never left ready; it only gets its window back. Nothing changes when
++ * the previous file is gone or the row no longer looks like this trim's failure. → true when the clip was restored.
++ */
++async function restoreTrim(clipId, prev, error) {
++ const clip = await db.getClipById(clipId);
++ if (!clip || !prev) return false;
++ const w = [Number(prev.start_time) || 0, Number(prev.end_time) || 0, Number(prev.duration_seconds) || 0];
++ if (!(w[1] > w[0])) return false;
++ if (!clip.file_path) {
++ if (!isVirtual(clip)) return false;
++ await objects().withObject('clip', clipId, async () => await db.run('UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = ? WHERE id = ?', [...w, clipId]));
++ console.warn(`[Clips] Clip ${clipId}: the trim could not be cut (${error}) — it keeps its previous window`);
++ return true;
++ }
++ if (clip.status !== 'failed' || !fs.existsSync(clip.file_path)) return false;
++ await announce(clip.app_id, 'clip.ready', {
++ change: async () => await objects().withObject('clip', clipId, async () => await db.run(`UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = ?,
++ status = 'ready', cut_error = NULL, cut_next_at = NULL WHERE id = ? AND status = 'failed'`, [...w, clipId])),
++ payload: async () => await _clipPublic(await db.getClipById(clipId)),
++ });
++ console.warn(`[Clips] Clip ${clipId}: the trim could not be cut (${error}) — it is ready again on its previous cut`);
++ return true;
++}
++
const spec = {
lane: 'clips',
maxAttempts: MAX_ATTEMPTS,
@@@ -305,9 +350,18 @@ const spec = {
const clip = await db.getClipById(job.params.clip_id);
if (!clip || clip.app_id !== job.app_id) throw new JobError('not_found', 'The clip no longer exists', { permanent: true });
const r = await recutClip(clip.id, { reason: job.params.reason || 'cut', viaJob: true, job, ctx });
-- if (!r.ok) throw new JobError('clip_cut_failed', r.error || 'cut failed', { permanent: !!r.hopeless });
++ if (!r.ok) {
++ // The last attempt of a trim (the worker retries only a non-permanent failure with attempts left): the
++ // clip gets its previous window back instead of staying failed.
++ const attempts = Number((ctx && ctx.attempt) ?? job.attempts) || 0;
++ const last = !!r.hopeless || attempts >= (Number(job.max_attempts) || MAX_ATTEMPTS);
++ if (last && job.params.reason === 'trim' && job.params.prev) {
++ try { await restoreTrim(clip.id, job.params.prev, r.error); } catch (err) { console.warn(`[Clips] Clip ${clip.id}: restoring the window before the trim failed: ${err.message}`); }
++ }
++ throw new JobError('clip_cut_failed', r.error || 'cut failed', { permanent: !!r.hopeless });
++ }
return { clip_id: clip.id, status: 'ready', duration_seconds: r.duration_seconds ?? null };
},
};
--module.exports = { recutClip, sweep, start, enqueueCut, clipObjectId, timelineWindow, makeVirtual, isVirtual, spec, MAX_ATTEMPTS };
++module.exports = { recutClip, sweep, start, enqueueCut, activeCut, restoreTrim, clipObjectId, timelineWindow, makeVirtual, isVirtual, spec, MAX_ATTEMPTS };
ddiff --git a/server/vod/clips-routes.js b/server/vod/clips-routes.js
iindex 8bb0e41..74970f6 100644
---- a/server/vod/clips-routes.js
++++ b/server/vod/clips-routes.js
@@@ -416,9 +416,13 @@ router.post('/:id/recut', tenantAuth({ verb: 'write' }), async (req, res) => {
* keeps its id, title and links; its window of the source VOD shrinks to that part and it is cut again from the source
* (202 { id, status: 'processing', job_id }, clip.cut; the superseded file is removed once the new one is in), or, a
* virtual clip, it plays the narrower window of the source's timeline at once (200, the clip). Asking for the whole
-- * clip changes nothing (200, the clip, unchanged: true). A clip with no source VOD (a browser upload, already cut) or
-- * whose file has moved to cloud storage is not trimmed (422). An app-key call acting for a user trims only that user's
-- * own clips, as for every write.
++ * clip changes nothing (200, the clip, unchanged: true). A clip with no source VOD (a browser upload, already cut),
++ * whose file has moved to cloud storage, or whose source recording cannot be read is not trimmed (422). A clip under a
++ * retention hold (its own or its source VOD's, as DELETE decides) is not trimmed (409 media.object.held), nor one that
++ * is being cut (409: a cut job that already read the old window would finish it with the old end). A file clip is
++ * `processing` from the answer until its cut is in (clip.ready); a cut that fails for good gives the clip its previous
++ * window and file back (clip.ready again). An app-key call acting for a user trims only that user's own clips, as for
++ * every write.
*/
router.post('/:id/trim', tenantAuth({ verb: 'write' }), async (req, res) => {
try {
@@@ -433,30 +437,51 @@ router.post('/:id/trim', tenantAuth({ verb: 'write' }), async (req, res) => {
to = Math.min(to, len);
if (to - from < 1) return res.status(400).json({ error: 'Clip must be at least 1 second' });
if (from < 0.05 && to > len - 0.05) return res.json({ ...await clipPublic(clip, { readiness: true }), unchanged: true });
++ if (await objects.isHeldRow(clip)) return res.status(409).json({ error: 'Clip is under a retention hold', code: 'media.object.held' });
const clipJobs = require('./clip-jobs');
++ if (await clipJobs.activeCut(req.appId, clip.id)) return res.status(409).json({ error: 'Clip is still being cut' });
const vod = clip.vod_id ? await db.getVodById(clip.vod_id, req.appId) : null;
if (!vod) return res.status(422).json({ error: 'Clip has no source VOD to trim from' });
const virtual = clipJobs.isVirtual(clip);
-- if (!virtual && require('./vod-storage').isRemote(clip)) return res.status(422).json({ error: 'Clip has moved to cloud storage and can no longer be trimmed' });
++ const vodStorage = require('./vod-storage');
++ if (!virtual && vodStorage.isRemote(clip)) return res.status(422).json({ error: 'Clip has moved to cloud storage and can no longer be trimmed' });
-- const before = [clip.start_time, clip.end_time, clip.duration_seconds];
const start = (Number(clip.start_time) || 0) + from, end = (Number(clip.start_time) || 0) + to;
-- const setWindow = async (w) => await objects.withObject('clip', clip.id, async () => await db.run('UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = ?, cut_attempts = 0 WHERE id = ?', [...w, clip.id]));
-- await setWindow([start, end, end - start]);
// A plain virtual clip plays the narrower window at once; a materialized one has rows of its own, which the cut rebuilds.
const obj = virtual ? await objects.getObject(clip.object_id) : null;
const materialized = !!(obj && /"materialized":\s*true/.test(String(obj.metadata || '')));
const win = virtual && !materialized && await clipJobs.timelineWindow(vod, start, end);
++ // Anything else is cut again: refuse now, before the window changes, when there is nothing to cut it from (no
++ // readable source file, and no source timeline for a materialized cut either).
++ if (!win && !await vodStorage.resolveMediaSource(vod)
++ && !(config.hls.enabled && config.hls.materialized && await clipJobs.timelineWindow(vod, start, end))) {
++ return res.status(422).json({ error: 'The source recording is unavailable, so the clip cannot be cut again' });
++ }
++
++ // A file clip is processing until the new cut is in (a second trim waits, and so does whoever waits for clip.ready);
++ // a virtual one keeps playing. The status condition makes a concurrent trim of the same clip find nothing to update.
++ const file = !virtual;
++ const prev = { start_time: clip.start_time, end_time: clip.end_time, duration_seconds: clip.duration_seconds };
++ const setWindow = async (w, { forward }) => await objects.withObject('clip', clip.id, async () => await db.run(
++ `UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = ?, cut_attempts = 0${file ? `, status = '${forward ? 'processing' : 'ready'}'` : ''}
++ WHERE id = ?${file && forward ? " AND status = 'ready'" : ''}`, [...w, clip.id]));
++ const set = await setWindow([start, end, end - start], { forward: true });
++ if (file && !(set && set.changes)) return res.status(409).json({ error: 'Clip is still being cut' });
if (win) {
await clipJobs.makeVirtual(clip.id, win);
return res.json({ ...await clipPublic(await db.getClipById(clip.id, req.appId), { readiness: true }) });
}
let job;
-- try { ({ job } = await clipJobs.enqueueCut(req.appId, clip.id, { reason: 'trim', createdBy: `app:${req.appId}` })); }
-- catch (err) { await setWindow(before); throw err; }
++ try { ({ job } = await clipJobs.enqueueCut(req.appId, clip.id, { reason: 'trim', createdBy: `app:${req.appId}`, prev })); }
++ catch (err) { await setWindow([prev.start_time, prev.end_time, prev.duration_seconds], { forward: false }); throw err; }
console.log(`[Clips] Clip ${clip.id} trimmed to ${start.toFixed(1)}-${end.toFixed(1)}s of vod ${clip.vod_id}`);
res.status(202).json({ id: clip.id, status: 'processing', job_id: job.id });
} catch (err) {
++ // A queue refusal (too many cuts queued: 429 with Retry-After) is answered as the queue says; the window is back.
++ if (err && err.code === 'media.job.too_many') {
++ if (err.retryAfterS) res.set('Retry-After', String(err.retryAfterS));
++ return res.status(429).json({ error: err.message, code: err.code });
++ }
console.error('[Clips] Trim error:', err.message);
res.status(500).json({ error: 'Failed to trim clip' });
}
ddiff --git a/test/clip-trim.test.js b/test/clip-trim.test.js
iindex cc28a7e..6f4f120 100644
---- a/test/clip-trim.test.js
++++ b/test/clip-trim.test.js
@@@ -2,7 +2,12 @@
// POST /clips/:id/trim: a ready clip narrows in place. Its window of the source VOD shrinks and it is cut again from
// the source (202 with the clip.cut job); the id and title stay, the superseded file is removed, clip.ready is told
// again. The whole clip is no change; a window under 1 s or outside the clip is refused; a clip still being cut is
--// 409; a clip with no source VOD (a browser upload) is 422; acting for a user, only that user's own clips.
++// 409 (from the answer on: a file clip is `processing` at once, and a queued or running cut also refuses); a clip with
++// no source VOD (a browser upload) or whose source recording cannot be read is 422; acting for a user, only that
++// user's own clips. A clip under a retention hold (its own or its VOD's) is 409 media.object.held, and a re-cut of a
++// held clip keeps its previous file. A trim whose cut fails for good gives the clip its previous window and file back
++// (ready, clip.ready again); an earlier failed attempt leaves that to the retry. object.cmaf of a clip is keyed on its
++// file, and commits nothing when the clip was cut again while it ran.
const assert = require('assert');
const fs = require('fs');
const os = require('os');
@@@ -21,6 +26,7 @@ const { spawnSync } = require('child_process');
process.env.MEDIA_JOBS_POLL_MS = '50';
process.env.MEDIA_INVARIANT_SCAN_HOURS = '0';
process.env.MEDIA_UPLOAD_MIN_FREE_MB = '0';
++ for (const p of ['B2', 'R2']) for (const k of ['ENDPOINT', 'BUCKET', 'KEY_ID', 'APP_KEY', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']) process.env[`MEDIA_${p}_${k}`] = '';
const stub = http.createServer((req, res) => {
req.resume();
req.on('end', () => {
@@@ -30,7 +36,7 @@ const { spawnSync } = require('child_process');
});
});
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
-- const waitFor = async (fn, ms = 30000) => { const end = Date.now() + ms; while (Date.now() < end) { if (await fn()) return true; await sleep(25); } return false; };
++ const waitFor = async (fn, ms = 240000) => { const end = Date.now() + ms; while (Date.now() < end) { if (await fn()) return true; await sleep(25); } return false; };
if (spawnSync('ffmpeg', ['-version']).status !== 0) { console.log('clip-trim: skipped (ffmpeg not found)'); process.exit(0); }
await new Promise((r) => stub.listen(0, '127.0.0.1', r));
@@@ -42,6 +48,9 @@ const { spawnSync } = require('child_process');
events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
const queue = require('../server/jobs/queue');
const worker = require('../server/jobs/worker');
++ const model = require('../server/objects/model');
++ const clipJobs = require('../server/vod/clip-jobs');
++ const cutter = require('../server/vod/clip-cutter');
const express = require('express');
const app = express();
app.use(express.json());
@@@ -53,10 +62,12 @@ const { spawnSync } = require('child_process');
const res = await fetch(api + p, { method, headers: { authorization: 'Bearer live-key', 'content-type': 'application/json', ...headers }, body: body === undefined ? undefined : JSON.stringify(body) });
return { status: res.status, body: await res.json().catch(() => null) };
};
++ const trim = async (id, body) => await call('POST', `/api/v1/live/clips/${id}/trim`, body);
const readyEvents = async (id) => (await conn.prepare('SELECT envelope FROM event_outbox ORDER BY id').all())
.map((r) => (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope))
.filter((e) => e.event_type === 'media.clip.ready' && String(e.subject.id) === String(id)).length;
const settled = async (jobId) => await waitFor(async () => ['succeeded', 'failed'].includes((await queue.get(jobId)).status));
++ const windowOf = (c) => [Number(c.start_time), Number(c.end_time), Number(c.duration_seconds)];
await worker.start();
const src = path.join(process.env.VOD_PATH, 'rec-30.webm');
@@@ -64,6 +75,7 @@ const { spawnSync } = require('child_process');
'-c:v', 'libvpx', '-g', '10', '-b:v', '200k', '-c:a', 'libopus', '-shortest', src]);
assert.strictEqual(mk.status, 0, String(mk.stderr));
await conn.prepare("INSERT INTO vods (id, app_id, user_id, title, file_path, file_size, is_public, duration_seconds) OVERRIDING SYSTEM VALUE VALUES (30, 'live', 5, 'Thirty seconds', ?, ?, 1, 30) RETURNING id").run(src, fs.statSync(src).size);
++ await model.syncVod(30);
// A 15 s clip of the VOD (5-20 s), cut.
let r = await call('POST', '/api/v1/live/clips', { vod_id: 30, start_s: 5, end_s: 20, title: 'Best food', user_id: 5 });
@@@ -76,30 +88,88 @@ const { spawnSync } = require('child_process');
assert.ok(Math.abs(len - 15) < 1.5, `a 15 s clip (${len})`);
// ── The whole clip: nothing changes ──
-- r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 0, end_s: len });
++ r = await trim(clipId, { start_s: 0, end_s: len });
assert.strictEqual(r.status, 200, JSON.stringify(r.body));
assert.strictEqual(r.body.unchanged, true);
assert.strictEqual((await db.getClipById(clipId)).file_path, first.file_path, 'the file is untouched');
// ── Refused windows ──
for (const [body, why] of [[{}, 'no window'], [{ start_s: 4, end_s: 4.5 }, 'under 1 s'], [{ start_s: -1, end_s: 5 }, 'before the clip'], [{ start_s: 2, end_s: len + 5 }, 'past its end']]) {
-- assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, body)).status, 400, why);
++ assert.strictEqual((await trim(clipId, body)).status, 400, why);
}
// ── Acting for another user: not theirs ──
r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 3, end_s: 9 }, { 'x-ov-user-id': '6' });
assert.strictEqual(r.status, 403, JSON.stringify(r.body));
++ // ── Under a retention hold, its own or its source VOD's: not trimmed (the whole clip still answers unchanged) ──
++ const vodObject = (await db.get('SELECT object_id FROM vods WHERE id = 30')).object_id;
++ assert.ok(vodObject, 'the VOD has an object');
++ for (const [objectId, whose] of [[first.object_id, 'its own'], [vodObject, "its VOD's"]]) {
++ const hold = await model.placeHold({ object_id: objectId, kind: 'evidence', reason: 'clip-trim test' });
++ r = await trim(clipId, { start_s: 3, end_s: 9 });
++ assert.deepStrictEqual([r.status, r.body && r.body.code], [409, 'media.object.held'], `${whose} hold: ${JSON.stringify(r.body)}`);
++ assert.strictEqual((await trim(clipId, { start_s: 0, end_s: len })).body.unchanged, true, `${whose} hold: the whole clip is still no change`);
++ const still = await db.getClipById(clipId);
++ assert.deepStrictEqual([still.status, windowOf(still), still.file_path], ['ready', windowOf(first), first.file_path], `${whose} hold: the clip is untouched`);
++ await model.releaseHold(hold.id);
++ }
++
++ // ── A queued or running cut of the clip: a trim would join it (it read the old window), so it is refused ──
++ const pending = await queue.enqueue({ appId: 'live', type: 'clip.cut', objectId: first.object_id, params: { clip_id: clipId, reason: 're-cut' }, runAfterS: 3600 });
++ r = await trim(clipId, { start_s: 3, end_s: 9 });
++ assert.strictEqual(r.status, 409, JSON.stringify(r.body));
++ assert.deepStrictEqual(windowOf(await db.getClipById(clipId)), windowOf(first), 'the window is untouched');
++ await queue.cancel(pending.job.id, { reason: 'test' });
++
++ // ── The source recording cannot be read: refused before the window changes ──
++ fs.renameSync(src, `${src}.away`);
++ r = await trim(clipId, { start_s: 3, end_s: 9 });
++ assert.strictEqual(r.status, 422, JSON.stringify(r.body));
++ const unread = await db.getClipById(clipId);
++ assert.deepStrictEqual([unread.status, windowOf(unread)], ['ready', windowOf(first)], 'the clip is untouched');
++ fs.renameSync(`${src}.away`, src);
++
++ // ── A cut that fails for good: the clip gets its previous window and file back ──
++ const realCut = cutter.cutClipFile;
++ cutter.cutClipFile = async () => ({ ok: false, error: 'No decodable footage in that window' }); // hopeless: no retry
++ let readyBefore = await readyEvents(clipId);
++ r = await trim(clipId, { start_s: 2, end_s: 6 });
++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
++ assert.ok(await settled(r.body.job_id));
++ assert.strictEqual((await queue.get(r.body.job_id)).status, 'failed');
++ const restored = await db.getClipById(clipId);
++ assert.deepStrictEqual([restored.status, restored.file_path, restored.cut_error], ['ready', first.file_path, null], 'ready again on its previous cut');
++ assert.deepStrictEqual(windowOf(restored), windowOf(first), 'its previous window');
++ assert.ok(fs.existsSync(first.file_path), 'the previous file is still there');
++ assert.strictEqual(await readyEvents(clipId), readyBefore + 1, 'clip.ready is told again');
++ // An attempt with retries left leaves the clip to its retry; the last one restores (the job's own run, directly).
++ cutter.cutClipFile = async () => ({ ok: false, error: 'ffmpeg exited 1' });
++ const prev = { start_time: first.start_time, end_time: first.end_time, duration_seconds: first.duration_seconds };
++ const narrow = async () => await model.withObject('clip', clipId, async () => await db.run("UPDATE clips SET start_time = ?, end_time = ?, duration_seconds = 4, status = 'processing' WHERE id = ?", [Number(first.start_time) + 2, Number(first.start_time) + 6, clipId]));
++ const runJob = async (attempt) => await clipJobs.spec.run({ id: `mjob_test${attempt}`, app_id: 'live', params: { clip_id: clipId, reason: 'trim', prev }, attempts: attempt, max_attempts: 4 }, { attempt })
++ .then(() => null, (err) => err);
++ await narrow();
++ let err = await runJob(1);
++ assert.strictEqual(err && err.code, 'clip_cut_failed');
++ let row = await db.getClipById(clipId);
++ assert.deepStrictEqual([row.status, Number(row.duration_seconds)], ['failed', 4], 'attempt 1 of 4: failed, the trimmed window waits for the retry');
++ err = await runJob(4);
++ assert.strictEqual(err && err.code, 'clip_cut_failed');
++ row = await db.getClipById(clipId);
++ assert.deepStrictEqual([row.status, windowOf(row), row.file_path], ['ready', windowOf(first), first.file_path], 'attempt 4 of 4: restored');
++ cutter.cutClipFile = realCut;
++
// ── Seconds 3-9 of the clip: cut again from the source, in place ──
const before = await readyEvents(clipId);
-- r = await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 3, end_s: 9 });
++ r = await trim(clipId, { start_s: 3, end_s: 9 });
assert.strictEqual(r.status, 202, JSON.stringify(r.body));
assert.strictEqual(r.body.id, clipId);
assert.ok(/^mjob_/.test(r.body.job_id), 'the answer names the cut job');
-- // Still being cut: a second trim waits.
-- if ((await db.getClipById(clipId)).status === 'processing') {
-- assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 1, end_s: 3 })).status, 409);
-- }
++ // Processing from the answer on: a second trim (a double submit) is refused, it does not narrow the window again.
++ assert.strictEqual((await db.getClipById(clipId)).status, 'processing', 'processing at once');
++ const again = await trim(clipId, { start_s: 1, end_s: 3 });
++ assert.strictEqual(again.status, 409, JSON.stringify(again.body));
assert.ok(await settled(r.body.job_id));
assert.strictEqual((await queue.get(r.body.job_id)).status, 'succeeded');
const trimmed = await db.getClipById(clipId);
@@@ -112,17 +182,74 @@ const { spawnSync } = require('child_process');
assert.ok(!fs.existsSync(first.file_path), 'the superseded file is removed');
assert.strictEqual(await readyEvents(clipId), before + 1, 'clip.ready is told again');
++ // ── A re-cut of a held clip keeps its previous file (a hold keeps bytes) ──
++ const hold = await model.placeHold({ object_id: vodObject, kind: 'evidence', reason: 'clip-trim test' });
++ r = await call('POST', `/api/v1/live/clips/${clipId}/recut`, {});
++ assert.strictEqual(r.status, 202, JSON.stringify(r.body));
++ assert.ok(await settled(r.body.job_id));
++ const recut = await db.getClipById(clipId);
++ assert.notStrictEqual(recut.file_path, trimmed.file_path, 'a new cut');
++ assert.ok(fs.existsSync(trimmed.file_path), 'the held clip\'s previous file is kept');
++ await model.releaseHold(hold.id);
++
// ── A clip still being cut ──
await conn.prepare("UPDATE clips SET status = 'processing' WHERE id = ?").run(clipId);
-- assert.strictEqual((await call('POST', `/api/v1/live/clips/${clipId}/trim`, { start_s: 1, end_s: 3 })).status, 409);
++ assert.strictEqual((await trim(clipId, { start_s: 1, end_s: 3 })).status, 409);
await conn.prepare("UPDATE clips SET status = 'ready' WHERE id = ?").run(clipId);
// ── A browser upload has no source to trim from ──
-- const up = await db.createClip({ app_id: 'live', vod_id: null, user_id: 5, title: 'Uploaded', file_path: trimmed.file_path, start_time: 0, end_time: 6, duration_seconds: 6, status: 'ready' });
-- r = await call('POST', `/api/v1/live/clips/${up.lastInsertRowid}/trim`, { start_s: 1, end_s: 4 });
++ const up = await db.createClip({ app_id: 'live', vod_id: null, user_id: 5, title: 'Uploaded', file_path: recut.file_path, start_time: 0, end_time: 6, duration_seconds: 6, status: 'ready' });
++ r = await trim(up.lastInsertRowid, { start_s: 1, end_s: 4 });
assert.strictEqual(r.status, 422, JSON.stringify(r.body));
worker.stop();
++ await sleep(100);
++
++ // ── object.cmaf of a clip: keyed on its file; a cut that lands while it runs makes it commit nothing ──
++ const config = require('../server/config');
++ const timelineQueue = require('../server/objects/timeline-queue');
++ const timeline = require('../server/objects/timeline');
++ const derive = require('../server/jobs/derive');
++ const cmaf = require('../server/jobs/cmaf');
++ config.hls.enabled = true;
++ const objectId = recut.object_id;
++ const keyOf = async (jobId) => (await queue.get(jobId)).idempotency_key;
++ const j1 = await timelineQueue.queueCmaf('live', objectId);
++ const k1 = await keyOf(j1);
++ assert.ok(new RegExp(`^object\\.cmaf:${objectId}:[0-9a-f]{12}$`).test(k1), `the key names the file (${k1})`);
++ await queue.cancel(j1, { reason: 'test' });
++ assert.strictEqual(await timelineQueue.queueCmaf('live', objectId), j1, 'the same file: the same job (replayed)');
++ const copy = path.join(process.env.CLIPS_PATH, 'clip-copy.webm');
++ fs.copyFileSync(recut.file_path, copy);
++ await conn.prepare('UPDATE clips SET file_path = ? WHERE id = ?').run(copy, clipId);
++ const j2 = await timelineQueue.queueCmaf('live', objectId);
++ assert.ok(j2 && j2 !== j1 && await keyOf(j2) !== k1, 'a new file: a new job, not the old one replayed');
++ await queue.cancel(j2, { reason: 'test' });
++ await conn.prepare('UPDATE clips SET file_path = ? WHERE id = ?').run(recut.file_path, clipId);
++
++ const x264 = /libx264/.test(String(spawnSync('ffmpeg', ['-hide_banner', '-encoders']).stdout));
++ if (x264) {
++ config.hls.transcode = true; // the clip is VP8: fMP4 needs it transcoded
++ const ctx = () => ({ signal: new AbortController().signal, checkpoint: null, attempt: 1, saveCheckpoint: async () => {}, cancelled: () => false });
++ const job = (n) => ({ id: `mjob_cmaftest${n}`, app_id: 'live', object_id: objectId, type: 'object.cmaf', params: {} });
++ // The clip is cut again (its row names a new file) between the cmaf run's read of its source and its commit.
++ const realResolve = derive.resolveSource;
++ derive.resolveSource = async (obj) => {
++ const s = await realResolve(obj);
++ await conn.prepare('UPDATE clips SET file_path = ? WHERE id = ?').run(copy, clipId);
++ return s;
++ };
++ err = await cmaf.spec.run(job(1), ctx()).then(() => null, (e) => e);
++ derive.resolveSource = realResolve;
++ assert.deepStrictEqual([err && err.code, err && err.permanent], ['media.source.changed', true], String(err && err.stack));
++ assert.deepStrictEqual(await timeline.list(objectId, timeline.SOURCE), [], 'nothing committed');
++ // The file it reads stays the clip's: committed (the check passes a normal run).
++ const out = await cmaf.spec.run(job(2), ctx());
++ assert.ok(out.segments > 0 && (await timeline.list(objectId, timeline.SOURCE)).length > 0, 'a normal run commits its rows');
++ config.hls.transcode = false;
++ } else console.log('clip-trim: cmaf commit check skipped (no libx264)');
++ config.hls.enabled = false;
++
events._reset();
server.close();
stub.close();
ddiff --git a/test/media-timeline.test.js b/test/media-timeline.test.js
iindex b46172f..62f7d32 100644
---- a/test/media-timeline.test.js
++++ b/test/media-timeline.test.js
@@@ -126,7 +126,8 @@ const { spawn, spawnSync } = require('child_process');
const pub = await native('public');
const priv = await native('private');
// The cut adds only derived metadata (renditions.<name> = width/height/codecs): everything the object IS stays.
-- const untouched = async () => { const o = await model.getObject(pub); const md = model.parseJson(o.metadata, {}); delete md.renditions; return [{ ...o, metadata: JSON.stringify(md) }, await model.listLocations(pub)]; };
++ // The derived rendition metadata (and the updated_at its write sets, segments.storeRenditionMeta) is left out.
++ const untouched = async () => { const o = await model.getObject(pub); const md = model.parseJson(o.metadata, {}); delete md.renditions; return [{ ...o, metadata: JSON.stringify(md), updated_at: null }, await model.listLocations(pub)]; };
const before = JSON.stringify(await untouched());
const runJob = async (objectId, params, type = 'object.cmaf') => {