openvibers/openvibe.live Commits

VOD clips are scored before they are cut, and may be none (item 151, C08) (#580)

OP OpenVibers committed 10 days ago · 0286df847f · 10 files, +253 −31

The VOD cutter cut whatever the moment picker returned, public: with the AI off a long scene note was enough, the
AI's own clips counted as 'people clipped here', and the picker could not say nothing was worth it. Now the picker
hands back the moment's evidence (the chat bucket against the stream's zero-filled average, the people in it, the
sounds and words near it, who picked it) and the cutter scores it with clip-score.js like the live cutter: thin is
not cut, kept is unlisted, a reacted room can be public. A clip never re-cuts a second a person clipped, the AI may
abstain, and the log and Bench name the policy (vod-moment@2026-11-03).

Changes

ddiff --git a/docs/vods-and-clips.md b/docs/vods-and-clips.md
iindex af2eaaff..e7b7a50a 100644
---- a/docs/vods-and-clips.md
++++ b/docs/vods-and-clips.md
@@@ -214,6 +214,7 @@ Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image
- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.
- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.
- **Only good live clips, cut on sentences** — a live chat spike is not enough on its own. `server/ai/clip-score.js` scores the moment from 0 to 1: the spike (how far above the window's average and how big), how many people reacted, what was heard (laughter, cheering, a bang; music and speech alone count for nothing), how much was said, and the AI's yes or no (a no is a veto; with the AI off its part is neutral and both bars rise by 0.1). Below **0.40** no clip is cut, the log says why ("a weak chat spike (10 messages, 3.1× the average)"). From 0.40 the clip is made **unlisted**: the streamer can find it and make it public. Calibrated by the production audit (C01, `scripts/diagnostics/clip-audit-*.sql`: the old rule made 99.6 % of automatic clips public and 29 % were never watched in 30 days): a moment needs **evidence** to be kept at all (a spike of at least 3× the average and 10 messages, a strong sound, or 20 spoken words; the AI's yes alone is not enough), and it is cut **public** only at **0.70** or more **and** in the reacted bucket: at least 12 messages in the peak from at least 3 people (`chatReads.recentChatters`; unknown means unlisted). Every cut logs its bucket (A no chat, B 1–5, C 6–11, D 12+ with 3+ people), peak and chatters for the next audit. `server/ai/clip-trim.js` then moves the fixed 16 s before / 9 s after window onto the speech rows: the start goes back to where the sentence being spoken began (so the setup is in), at most 25 s before the moment; the end goes on to where the sentence ends, at most 20 s after, never past the recording's flushed edge. A snap that would make the clip shorter than 12 s or longer than 45 s is not taken. Both are pure, with their cases in `test/clip-score-trim.test.js`.
++- **VOD clips are scored too, and may be none** (item 151, C08). The VOD cutter (`clipVodMoment`, from the backfill and from the paste job's second beat) used to cut whatever the picker returned, public. Now the picker hands back the moment's **evidence**, gathered at that second from everything Live has (not the sample the prompt reads): the chat bucket there against the stream's own average per 30 s with the quiet stretches counted, how many people were in it (when Chat counts them), the sounds within 15 s, the words said within 20 s, and who picked it (`ai`, `chat`, `sound`, `people`, `scene`). The cutter runs it through the same `clip-score.js` as the live cutter: below the keep bar nothing is cut (the log says the score and why), a kept moment is **unlisted**, and only a reacted room (12+ messages from 3+ people, a score of 0.70 or more) is cut **public**, never above the recording's own visibility. A moment with no evidence is not cut. A clip never re-cuts a second a person already clipped (that footage is their clip already; it goes in the picker's avoid list), and with the AI off a clip needs a chat spike or a strong sound: a long scene note alone only ever makes a paste. The AI may answer that nothing is worth it (`{ abstain: true }` or `t: null`), and then no weaker signal is tried. The AI's own clips are no longer counted as "people clipped here" (`lookups.clipStartTimes`). Each cut logs its score, bucket, picker and policy (`vod-moment@2026-11-03`), and the daily Bench evaluation labels the AI's entry with that policy (`clip-evaluation.policyOf`), so Bench's accept rate tells this policy from the one before. Cases: `test/vod-moment-evidence.test.js`.
- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).
## AI-inferred category
ddiff --git a/server/ai/ai-moments-job.js b/server/ai/ai-moments-job.js
iindex 4b1649ad..65fbbd06 100644
---- a/server/ai/ai-moments-job.js
++++ b/server/ai/ai-moments-job.js
@@@ -24,6 +24,7 @@ const registry = require('./moment-registry');
const clipPolicy = require('./clip-policy');
const dedupe = require('./clip-dedupe');
const explicit = require('./explicit');
++const clipScore = require('./clip-score');
let cfg = null; try { cfg = require('../config'); } catch { /* */ }
let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }
const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');
@@@ -148,7 +149,9 @@ function _sample(arr, max) {
}
async function _momentContext(streamId, vodId) {
const memories = _sample((await db.getStreamMemories(streamId) || []).filter(m => m.description), 45);
-- const transcript = _sample(await db.getStreamTranscriptSegments(streamId) || [], 60);
++ // The picker reads a sample spread over the stream; the evidence at the chosen second reads everything near it.
++ const said = await db.getStreamTranscriptSegments(streamId) || [];
++ const transcript = _sample(said, 60);
// Where viewers clipped: the clips live in OpenVibe.Media.
const clipTimes = await require('../media-proxy/lookups').clipStartTimes(streamId, vodId);
// Chat's own buckets over the stream (offsets are relative to started_at, as before; it ends at ended_at, or now).
@@@ -158,13 +161,21 @@ async function _momentContext(streamId, vodId) {
const spikes = started > 0 ? (await chatReads.spikeOffsets(streamId, 30, 8, started, ms(span.ended_at) || undefined) || []) : [];
// Non-speech sounds are strong moment candidates — an explosion or a burst of
// laughter marks a highlight as reliably as anything said out loud.
-- let sounds = [];
++ let heard = [];
try {
-- sounds = _sample((await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])
++ heard = (await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])
// Background music (audio-events.js spans) is the stream's bed, not a moment.
-- .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music'), 30);
++ .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music');
} catch { /* timeline optional */ }
-- return _inPart({ memories, transcript, clipTimes, spikes, sounds }, vodId);
++ const sounds = _sample(heard, 30);
++ // The stream's average chat per 30 s, empty stretches counted (C07/C08: averaging only the busy buckets made every
++ // spike look small on a quiet channel and every lull look big on a busy one).
++ let chatAvg = 0;
++ try {
++ const secs = started > 0 ? ((ms(span.ended_at) || Date.now()) - started) / 1000 : 0;
++ if (secs >= 60) chatAvg = (Number((await chatReads.streamStats(streamId) || {}).messages) || 0) / (secs / 30);
++ } catch { /* chat optional */ }
++ return _inPart({ memories, transcript, said, clipTimes, spikes, sounds, heard, chatAvg }, vodId);
}
/**
@@@ -183,8 +194,10 @@ async function _inPart(ctx, vodId) {
...ctx,
memories: ctx.memories.filter(m => inside(Number(m.offset_seconds) || 0)).map(m => ({ ...m, offset_seconds: move(Number(m.offset_seconds) || 0) })),
transcript: ctx.transcript.filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),
++ said: (ctx.said || []).filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),
spikes: ctx.spikes.filter(sp => inside(Number(sp.offset))).map(sp => ({ ...sp, offset: move(Number(sp.offset)) })),
sounds: ctx.sounds.filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),
++ heard: (ctx.heard || []).filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),
};
}
function _nearestMemory(memories, offset) {
@@@ -192,6 +205,25 @@ function _nearestMemory(memories, offset) {
for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }
return best;
}
++/**
++ * What there is to go on at one second (item 151, C08), in the shape clip-score.js reads: the chat bucket at it against
++ * the stream's average, how many people were in it (when Chat says), the sounds within 15 s, the words said within 20 s,
++ * and who picked it (`picked_by`: ai, chat, sound, people, scene). The cutter scores it before anything is cut.
++ */
++function _evidenceAt(ctx, t, pickedBy) {
++ const at = Number(t) || 0;
++ const bucket = (ctx.spikes || []).filter(sp => Math.abs(Number(sp.offset) - at) <= 30).sort((a, b) => b.count - a.count)[0] || null;
++ const sounds = (ctx.heard || ctx.sounds || []).filter(e => Math.abs(Number(e.start_sec) - at) <= 15)
++ .map(e => ({ label: String(e.label || ''), confidence: Number(e.confidence) || 0 }));
++ const words = (ctx.said || ctx.transcript || []).filter(x => Math.abs(Number(x.start || 0) - at) <= 20)
++ .reduce((n, x) => n + String(x.text || '').trim().split(/\s+/).filter(Boolean).length, 0);
++ return {
++ picked_by: pickedBy,
++ spike: { peak: bucket ? Number(bucket.count) || 0 : 0, avg: Math.round((Number(ctx.chatAvg) || 0) * 100) / 100 },
++ chatters: bucket && bucket.people != null ? Number(bucket.people) : null,
++ sounds, speechWords: words,
++ };
++}
/** Transcript text around a moment, so dedupe can tell a new conversation from a repeated scene. */
function _speechNear(transcript, offset, windowSec = 45) {
try {
@@@ -202,10 +234,14 @@ function _speechNear(transcript, offset, windowSec = 45) {
}
// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.
// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.
++// The result carries `evidence` (_evidenceAt): the clip cutter scores it and abstains when it is thin (C08).
async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
const dur = Math.floor(vod.duration || 0);
const ctx = await _momentContext(vod.stream_id, vod.vod_id);
-- const avoidList = [...new Set([...(avoid || []), ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);
++ // A clip never re-cuts what a person already clipped: that footage is a clip already, theirs (C08). A paste (one
++ // frame) may still use the second people chose.
++ const peopleClipped = flavor === 'clip' ? ctx.clipTimes : [];
++ const avoidList = [...new Set([...(avoid || []), ...peopleClipped, ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);
const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);
if (!ctx.memories.length && !ctx.transcript.length) return null;
const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));
@@@ -219,18 +255,24 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
timeline: ctx.memories.map(m => ({ t: secs(m.offset_seconds), text: _cleanText(_deJson(m.description), 180) })).filter(x => x.text).slice(0, 400),
transcript: ctx.transcript.map(x => ({ t: secs(x.start), text: _cleanText(x.text, 160) })).filter(x => x.text).slice(0, 1500),
sounds: (ctx.sounds || []).map(e => ({ t: secs(e.start_sec), label: _cleanText(e.label, 120) })).filter(x => x.label).slice(0, 500),
-- clipped: ctx.clipTimes.slice(0, 12).map(secs),
++ clipped: (flavor === 'clip' ? [] : ctx.clipTimes).slice(0, 12).map(secs),
spikes: ctx.spikes.slice(0, 6).map(sp => secs(sp.offset)),
avoid: avoidList.map(secs).slice(0, 500),
};
try {
const j = await aiService.structured('live.moments.pick', input, { target: { service: 'live', type: 'vod', id: String(vod.vod_id) }, meter: { kind: 'moment_pick', role: 'legacy' } });
++ // The AI may say nothing here is worth it (C08/C09): that is an answer, not a failure, so no weaker signal
++ // is tried in its place.
++ if (j && (j.abstain === true || (Object.prototype.hasOwnProperty.call(j, 't') && j.t === null))) {
++ console.log(`[AI-Moments] VOD ${vod.vod_id}: the AI found nothing worth a ${flavor === 'clip' ? 'clip' : 'paste'}${j.why ? ` (${_cleanText(j.why, 120)})` : ''}`);
++ return null;
++ }
let t = j ? Number(j.t) : NaN;
if (!isNaN(t)) {
t = clamp(t);
if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }
const near = _nearestMemory(ctx.memories, t);
-- const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.transcript, t), tags: _memTags(near) };
++ const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.said || ctx.transcript, t), tags: _memTags(near), evidence: _evidenceAt(ctx, t, 'ai') };
if (explicit.isExplicit(j.desc, j.title, near && near.description, result.desc)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: the pick at ${_mmss(t)} shows sexual content — no moment from it`); return null; }
return _isEmptyScene(result.desc) ? null : result;
}
@@@ -238,18 +280,22 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
}
// No-AI fallback (also used when the AI pick collided with a used moment): the strongest
-- // objective signal that is not next to something already made.
++ // objective signal that is not next to something already made. A clip needs something people or the room did — a
++ // chat spike or a strong sound; a long scene note alone is a description, not a moment (C08), so only a paste may
++ // fall back to one.
++ const strongSounds = (ctx.heard || ctx.sounds || []).filter(e => Number(e.confidence) >= clipScore.EVIDENCE.soundConfidence && clipScore.STRONG_SOUND.test(String(e.label || '')));
const candidates = [
-- ...ctx.clipTimes.map(t => ({ t, w: 3 })), // where viewers clipped
-- ...ctx.spikes.map(s => ({ t: s.offset, w: 2 })), // busiest chat moments
-- ...ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1 })), // richest scene notes
++ ...(flavor === 'clip' ? [] : ctx.clipTimes.map(t => ({ t, w: 3, by: 'people' }))), // where people clipped
++ ...ctx.spikes.map(s => ({ t: s.offset, w: 2, by: 'chat' })), // busiest chat moments
++ ...strongSounds.map(e => ({ t: Number(e.start_sec) || 0, w: 2, by: 'sound' })), // laughter, a crowd, a bang
++ ...(flavor === 'clip' ? [] : ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1, by: 'scene' }))), // richest scene notes
].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);
if (!candidates.length) return null;
let offset = clamp(candidates[0].t);
const near = _nearestMemory(ctx.memories, offset);
const desc = near ? _deJson(near.description) : '';
if (near && explicit.isExplicit(near.description)) return null; // never a moment of sexual content (server/ai/explicit.js)
-- return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.transcript, offset), tags: _memTags(near) };
++ return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.said || ctx.transcript, offset), tags: _memTags(near), evidence: _evidenceAt(ctx, offset, candidates[0].by) };
}
// True when a description says the frame is essentially empty — black/dark/loading/blank — so we
// never turn "nothing" into a clip or paste.
@@@ -537,7 +583,7 @@ async function tick(opts = {}) {
try {
const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });
if (second && !await registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title, speech: second.speech })) {
-- const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, archival: v.archival, source: momentSource });
++ const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, evidence: second.evidence, archival: v.archival, source: momentSource });
if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);
} else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);
} catch { /* clipping is best-effort */ }
@@@ -566,7 +612,7 @@ function start() {
setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);
}
--module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle } };
++module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle, evidenceAt: _evidenceAt } };
// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:
// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2
ddiff --git a/server/ai/auto-clip-job.js b/server/ai/auto-clip-job.js
iindex 484d66c0..16f3df22 100644
---- a/server/ai/auto-clip-job.js
++++ b/server/ai/auto-clip-job.js
@@@ -311,7 +311,7 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
if (dark) { console.log(`[AutoClip] backfill skip VOD ${v.vod_id} — dark/empty frame`); newSkip.push(v.vod_id); continue; }
}
} catch { /* probe optional */ }
-- const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, archival: v.archival, source });
++ const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, evidence: moment.evidence, archival: v.archival, source });
if (clip) { made++; if (v.archival) archiveMade++; console.log(`[AutoClip] Backfilled VOD ${v.vod_id}${v.archival ? ' (archive)' : ''} ("${String(moment.title || '').slice(0, 60)}")`); }
else if (v.archival && !clipPolicy.archiveAllowed(await _clipLog(), Date.now())) continue; // another cutter took today's archive slot: not this VOD's fault, don't skip it forever
else newSkip.push(v.vod_id); // cut failed (e.g. unseekable/short) — don't retry forever
@@@ -325,8 +325,28 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
/**
* Cut a clip around a moment the AI-moments pipeline already chose for a finished VOD.
-- * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, archival, source }
++ * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, evidence, archival, source }
++ *
++ * The moment is scored before anything is cut (item 151, C08), by the same clip-score.js as the live cutter: `evidence`
++ * (ai-moments-job's evidenceAt) is the chat bucket at the moment against the stream's average, the people in it, the
++ * sounds and the words near it, and who picked it. A thin moment is not cut ("no clip beats a bad clip"), one that
++ * clears the keep bar is unlisted, and only a reacted moment (12+ messages from 3+ people) can be public. A moment with
++ * no evidence at all is not cut. The log keeps the score, bucket, picker and policy, so Bench and the audits can tell
++ * this policy from the one before.
*/
++const VOD_POLICY = Object.freeze({ selector: 'vod-moment', version: '2026-11-03' });
++function rateVodMoment(evidence) {
++ if (!evidence || typeof evidence !== 'object') return { verdict: 'skip', score: 0, bucket: 'A', reasons: ['no evidence was gathered for the moment'] };
++ const sp = evidence.spike || {};
++ return clipScore.score({
++ spike: { peak: Number(sp.peak) || 0, avg: Number(sp.avg) || 0 },
++ chatters: evidence.chatters == null ? null : Number(evidence.chatters),
++ sounds: Array.isArray(evidence.sounds) ? evidence.sounds : [],
++ speechWords: Number(evidence.speechWords) || 0,
++ // The AI's pick counts as its yes; a pick from the signals alone has no AI opinion (both bars rise).
++ ai: { clip: evidence.picked_by === 'ai' ? true : null },
++ });
++}
function _sceneSig(text) {
return dedupe.shortSignature(text, 5);
}
@@@ -345,9 +365,14 @@ async function _isDuplicateAutoClip(vod, offset, desc, title, speech) {
async function clipVodMoment(o) {
let archiveHeld = false;
try {
-- const { vod, offset, title, desc, speech, archival } = o || {};
++ const { vod, offset, title, desc, speech, archival, evidence } = o || {};
if (!vod || !(offset >= 0)) return null;
if (!await _derivationOn(vod.user_id)) return null; // the channel turned AI Moments off
++ const rated = rateVodMoment(evidence);
++ if (rated.verdict === 'skip') {
++ console.log(`[AutoClip] VOD ${vod.vod_id || vod.id} @${Math.round(offset)}s not cut — score ${rated.score}: ${(rated.reasons || []).slice(-2).join('; ') || 'below the bar'}`);
++ return null;
++ }
if (require('./explicit').isExplicit(desc, title)) { // never a clip of sexual content (server/ai/explicit.js)
console.log(`[AutoClip] skip VOD ${vod.vod_id || vod.id} — the AI describes the moment as sexual content`);
return null;
@@@ -361,7 +386,7 @@ async function clipVodMoment(o) {
// the clip public whatever the recording is.
const publication = require('./clip-publication');
if (!publication.mayCut(vodId)) { console.log(`[AutoClip] VOD ${vodId}: protected, not cut`); return null; }
-- const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: 'public' });
++ const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: rated.verdict === 'public' ? 'public' : 'unlisted' });
const start = Math.max(0, Math.floor(offset) - CLIP_PRE);
// One archive clip per UTC day is shared with the AI moments job. Both may pass the pool
// gate before either writes auto_clip_log, so claim the slot atomically before cutting and
@@@ -381,7 +406,11 @@ async function clipVodMoment(o) {
});
if (clip) {
archiveHeld = false; // the clip exists: keep the day reserved even if the log write fails
-- await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title) });
++ const sp = (evidence && evidence.spike) || {};
++ await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title),
++ score: rated.score, visibility: rated.verdict, bucket: rated.bucket, peak: Number(sp.peak) || 0, chatters: evidence.chatters == null ? undefined : evidence.chatters,
++ picked_by: evidence.picked_by || undefined, policy: VOD_POLICY });
++ console.log(`[AutoClip] VOD ${vodId} @${Math.round(offset)}s cut (${evidence.picked_by || 'signals'}) — score ${rated.score} → ${rated.verdict}, bucket ${rated.bucket}`);
}
return clip || null;
} catch { return null; }
@@@ -452,7 +481,7 @@ function stop() {
require('./clip-evaluation').stop();
}
--module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool } };
++module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool, rateVodMoment, VOD_POLICY } };
// CLI: force a historical backfill batch, e.g. `node server/ai/auto-clip-job.js --backfill --limit=4`
if (require.main === module) {
ddiff --git a/server/ai/clip-evaluation.js b/server/ai/clip-evaluation.js
iindex d7bb362f..b61ee831 100644
---- a/server/ai/clip-evaluation.js
++++ b/server/ai/clip-evaluation.js
@@@ -3,8 +3,8 @@
* clip-evaluation.js — the AI's clip picks against people's, blind, on OpenVibe.Bench (item 151, C20).
*
* Once a day, for up to MAX_PER_RUN recent recordings that have both a public clip the AI cut and a public clip a
-- * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY)
-- * and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at
++ * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY, or
++ * the one the auto-clip log names for it: policyOf) and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at
* https://bench.openvibe.services/battle?kind=live.clip (Bench #106: two players, blind, no titles), and
* results() reads each side's accept rate back from Bench (both_bad counts against both).
*
@@@ -66,6 +66,17 @@ async function bench(method, path, body) {
return json;
}
++/**
++ * Which picker cut an AI clip, for Bench's per-policy accept rate: the auto-clip log names it on clips cut since C08
++ * (2026-11-03: VOD moments are `vod-moment@2026-11-03`); older clips and the live cutter are `auto-clip@AUTO_CLIP_POLICY`.
++ */
++function policyOf(clip, clipLog = []) {
++ const e = (Array.isArray(clipLog) ? clipLog : []).find((x) => x && x.clip_id != null && String(x.clip_id) === String(clip.id));
++ const p = e && e.policy;
++ if (p && typeof p.selector === 'string' && typeof p.version === 'string') return { selector: p.selector.slice(0, 80), version: p.version.slice(0, 80), models: [] };
++ return { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] };
++}
++
function entryOf(clip, policy, signed) {
const start = Math.round(Number(clip.start_time) * 1000);
const len = Math.round(Number(clip.duration_seconds ?? clip.duration ?? (Number(clip.end_time) - Number(clip.start_time))) * 1000);
@@@ -85,6 +96,8 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
if (!ai || !Array.isArray(ai.clips)) return { submitted: [], skipped: 0, why: 'Media did not answer' };
let done;
try { done = new Set((await _log()).map((x) => String(x.vod_id))); } catch (e) { return { submitted: [], skipped: 0, why: `state unreadable: ${e.message}` }; }
++ let clipLog = [];
++ try { clipLog = JSON.parse(await db.getState('auto_clip_log') || '[]'); } catch { clipLog = []; }
const byVod = new Map();
for (const c of ai.clips) if (c.auto_generated === true && eligible(c) && !done.has(String(c.vod_id)) && !byVod.has(String(c.vod_id))) byVod.set(String(c.vod_id), c);
const submitted = [];
@@@ -106,7 +119,7 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
const task = await bench('POST', '/api/v1/kinds/live.clip/tasks', { prompt: 'Two clips of one stream', public: false });
const round = await bench('POST', `/api/v1/tasks/${task.task.id}/rounds`, {});
const roundId = round.round.id;
-- await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] }, sa) });
++ await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, policyOf(aiClip, clipLog), sa) });
await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(person, { selector: 'people', version: '1', models: [] }, sp) });
const log = await _log();
const mine = log.find((x) => String(x.vod_id) === vodId && x.round_id == null);
@@@ -136,4 +149,4 @@ function start() {
}
function stop() { if (_stop) { _stop(); _stop = null; } }
--module.exports = { run, results, start, stop, eligible, entryOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };
++module.exports = { run, results, start, stop, eligible, entryOf, policyOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };
ddiff --git a/server/chat/chat-reads.js b/server/chat/chat-reads.js
iindex f332d11e..a17fd7c6 100644
---- a/server/chat/chat-reads.js
++++ b/server/chat/chat-reads.js
@@@ -500,7 +500,7 @@ function recentChatters(streamId, sinceSec = 30, limit = 200) {
}
/** The busiest `bucketSec` time-buckets of a stream between `sinceMs` and `untilMs` (default now), as offsets from
-- * `sinceMs`: [{ offset, count }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to
++ * `sinceMs`: [{ offset, count, people }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to
* now, a stream from two months ago asked for 184,000 and got a 400 (2026-10-23). */
function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
const id = Number(streamId) || 0;
@@@ -512,7 +512,8 @@ function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
if (!out || !Array.isArray(out.buckets)) throw unavailable('timeline');
const base = sinceMs || 0;
return out.buckets
-- .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0 }))
++ // people: how many different chatters were in the bucket, when Chat says (null when it does not).
++ .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0, people: b.people == null ? null : Number(b.people) || 0 }))
.sort((a, b) => b.count - a.count || a.offset - b.offset)
.slice(0, topN);
}, () => []);
ddiff --git a/server/media-proxy/lookups.js b/server/media-proxy/lookups.js
iindex c536ba83..fe80b824 100644
---- a/server/media-proxy/lookups.js
++++ b/server/media-proxy/lookups.js
@@@ -218,9 +218,13 @@ async function streamClips(streamId, vodId) {
return [...byId.values()];
}
--/** Seconds into the VOD where viewers clipped (the AI moment picker's strongest signal). */
++/**
++ * Seconds into the VOD where people clipped (the AI moment picker's strongest signal). The AI's own clips are left out
++ * (item 151, C08): they are not people choosing a moment, and counting them made every AI clip a reason for the next.
++ */
async function clipStartTimes(streamId, vodId) {
return (await streamClips(streamId, vodId))
++ .filter((c) => c.auto_generated !== true)
.map((c) => Number(c.start_time)).filter((t) => t > 0).map(Math.floor).sort((a, b) => a - b);
}
ddiff --git a/test/ai-derivation-optout.test.js b/test/ai-derivation-optout.test.js
iindex 848e854c..db875e40 100644
---- a/test/ai-derivation-optout.test.js
++++ b/test/ai-derivation-optout.test.js
@@@ -148,10 +148,10 @@ async function check(name, fn) {
const pool = await autoClip._internals.backfillPool(5);
assert.deepStrictEqual(pool.map((v) => v.vod_id), [10], 'the VOD backfill pool leaves the opted-out channel out');
created.length = 0;
-- const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment' });
++ const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
assert.strictEqual(none, null);
assert.strictEqual(created.length, 0);
-- const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment' });
++ const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
assert.ok(clip && clip.id, 'Alice still gets auto-clips');
assert.strictEqual(created.length, 1);
assert.strictEqual(created[0].auto_generated, true);
ddiff --git a/test/chat-spike-window.test.js b/test/chat-spike-window.test.js
iindex 356f59fa..90a69267 100644
---- a/test/chat-spike-window.test.js
++++ b/test/chat-spike-window.test.js
@@@ -15,7 +15,8 @@ const chatReads = require('../server/chat/chat-reads');
const started = Date.parse('2026-08-20T12:00:00Z');
const ended = started + 2 * 3600 * 1000;
const spikes = await chatReads.spikeOffsets(2125, 30, 8, started, ended);
-- assert.deepStrictEqual(spikes, [{ offset: 30, count: 9 }, { offset: 60, count: 3 }]);
++ // people is null when Chat does not count chatters per bucket (C08 reads it as "unknown").
++ assert.deepStrictEqual(spikes, [{ offset: 30, count: 9, people: null }, { offset: 60, count: 3, people: null }]);
assert.deepStrictEqual(calls.pop(), { stream_id: 2125, since: started, until: ended, bucket_ms: 30000 }, 'an ended stream is read up to its end');
// A stream still live (no end): up to now, as before; the client leaves a null `until` out of the query.
@@@ -30,6 +31,9 @@ const chatReads = require('../server/chat/chat-reads');
// An end before the start is ignored (asked up to now).
await chatReads.spikeOffsets(2128, 30, 8, started, started - 1000);
assert.strictEqual(calls.pop().until, null);
++ // When Chat counts the people in each bucket, it rides along.
++ client.readTimeline = async (q2) => ({ buckets: [{ t: q2.since + 30000, count: 14, people: 5 }] });
++ assert.deepStrictEqual(await chatReads.spikeOffsets(2129, 30, 8, started, ended), [{ offset: 30, count: 14, people: 5 }]);
console.log('chat-spike-window: ok');
process.exit(0);
})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/clip-freshness.test.js b/test/clip-freshness.test.js
iindex 078ebd28..e8abbb25 100644
---- a/test/clip-freshness.test.js
++++ b/test/clip-freshness.test.js
@@@ -248,6 +248,8 @@ const archiveCut = (id) => autoClip.clipVodMoment({
offset: 60,
title: `Archive moment ${id}`,
desc: `A distinct archival scene number ${id} with enough words`,
++ // A moment worth cutting (C08: the cutter scores the evidence first).
++ evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 },
archival: true,
});
const firstCut = archiveCut(70);
ddiff --git a/test/vod-moment-evidence.test.js b/test/vod-moment-evidence.test.js
nnew file mode 100644
iindex 00000000..0515c00a
---- /dev/null
++++ b/test/vod-moment-evidence.test.js
@@@ -0,0 +1,122 @@
++'use strict';
++// The VOD-moment clip cutter scores what it has before it cuts, and may abstain (item 151, C08):
++// - the AI's own clips are not "people clipped here" (lookups.clipStartTimes);
++// - a clip never re-cuts footage a person already clipped, and never comes from a long scene note alone;
++// - the AI may say nothing is worth it, and no weaker signal is tried in its place;
++// - the moment's evidence (chat bucket against the stream's zero-filled average, people, sounds, words) goes through
++// clip-score.js: thin → not cut, kept → unlisted, a reacted room → public; no evidence → not cut;
++// - the clip log and Bench name the policy that cut it.
++const assert = require('assert');
++const db = require('../server/db/database');
++console.warn = () => {};
++
++(async () => {
++ await db.initDb();
++ const media = require('../server/media-client');
++ const lookups = require('../server/media-proxy/lookups');
++ const chatReads = require('../server/chat/chat-reads');
++ const registry = require('../server/ai/moment-registry');
++ const ai = require('../server/ai/ai-analysis');
++ const aiService = require('../server/ai/ai-service');
++ const autoClip = require('../server/ai/auto-clip-job');
++ const moments = require('../server/ai/ai-moments-job');
++ const evaluation = require('../server/ai/clip-evaluation');
++ const log = []; const logs = console.log; console.log = (...a) => log.push(a.join(' '));
++
++ // ── People's clips only ───────────────────────────────────────────────
++ media.listClips = async () => ({ clips: [
++ { id: 1, start_time: 600, auto_generated: false },
++ { id: 2, start_time: 1200, auto_generated: true },
++ { id: 3, start_time: 1800 },
++ ] });
++ assert.deepStrictEqual(await lookups.clipStartTimes(91, 92), [600, 1800], 'the AI\'s clip at 1200 s is not a person choosing a moment');
++
++ // ── The score at the cutter ───────────────────────────────────────────
++ const rate = autoClip._internals.rateVodMoment;
++ assert.strictEqual(rate(undefined).verdict, 'skip', 'no evidence, no clip');
++ const quiet = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 });
++ assert.strictEqual(quiet.verdict, 'unlisted', 'a quiet channel\'s laugh the AI picked is kept: a small audience is not a poor moment');
++ assert.strictEqual(quiet.bucket, 'A');
++ const talk = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 });
++ assert.strictEqual(talk.verdict, 'skip', 'talk the AI liked, with nothing heard or typed, is not cut: no quota to fill');
++ const room = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 });
++ assert.strictEqual(room.verdict, 'public', 'a reacted room (22 messages from 7 people) can be public');
++ const unknown = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: null, sounds: [], speechWords: 25 });
++ assert.strictEqual(unknown.verdict, 'unlisted', 'the same spike with nobody counted is kept unlisted');
++ const signals = rate({ picked_by: 'chat', spike: { peak: 9, avg: 3 }, chatters: 2, sounds: [], speechWords: 5 });
++ assert.strictEqual(signals.verdict, 'skip', 'a small spike with no AI opinion is not cut (both bars rise without the AI)');
++
++ // ── clipVodMoment: the verdict is the visibility, and the log keeps the provenance ──
++ const created = [];
++ media.createClip = async (body) => { created.push(body); return { id: 9100 + created.length }; };
++ media.getVod = async () => ({ id: 1, visibility: 'public' });
++ await db.setState('auto_clip_log', '[]');
++ await registry._clear();
++ const vod = (id) => ({ vod_id: id, id, user_id: 3, stream_id: id, visibility: 'public' });
++ assert.strictEqual(await autoClip.clipVodMoment({ vod: vod(501), offset: 300, title: 'Just talking', desc: 'A person talks to the camera about lunch', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 } }), null);
++ assert.strictEqual(created.length, 0, 'the thin moment never reached Media');
++ assert.ok(log.some((l) => /VOD 501 @300s not cut — score/.test(l)), 'the abstention is logged with its score');
++ const c1 = await autoClip.clipVodMoment({ vod: vod(502), offset: 300, title: 'The room lost it', desc: 'The streamer trips over the cable mid-sentence', evidence: { picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 } });
++ assert.ok(c1 && c1.id);
++ const c2 = await autoClip.clipVodMoment({ vod: vod(503), offset: 300, title: 'A laugh', desc: 'A dog barks and the streamer laughs at the window', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
++ assert.ok(c2 && c2.id);
++ assert.deepStrictEqual(created.map((b) => b.visibility), ['public', 'unlisted'], 'public only for the reacted room');
++ const entries = JSON.parse(await db.getState('auto_clip_log'));
++ const e1 = entries.find((e) => e.clip_id === c1.id);
++ assert.deepStrictEqual(e1.policy, { selector: 'vod-moment', version: '2026-11-03' });
++ assert.strictEqual(e1.picked_by, 'ai');
++ assert.strictEqual(e1.bucket, 'D');
++ assert.ok(e1.score >= 0.7);
++
++ // ── Bench names the policy that cut the clip ──
++ assert.deepStrictEqual(evaluation.policyOf({ id: c1.id }, entries), { selector: 'vod-moment', version: '2026-11-03', models: [] });
++ assert.deepStrictEqual(evaluation.policyOf({ id: 4242 }, entries), { selector: 'auto-clip', version: evaluation.AUTO_CLIP_POLICY, models: [] }, 'an older clip keeps the old label');
++
++ // ── The picker, AI off: a clip needs something people or the room did ──
++ ai.isEnabled = async () => false;
++ await db.run(`INSERT INTO users (id, username, display_name, email, password_hash, role, created_at) OVERRIDING SYSTEM VALUE
++ VALUES (3, 'eve', 'eve', 'eve@x', 'x', 'streamer', '2025-01-01 00:00:00') ON CONFLICT DO NOTHING`);
++ const sid = (await db.get(`INSERT INTO streams (user_id, title, started_at, ended_at) VALUES (3, 'evidence', '2026-11-01 10:00:00', '2026-11-01 12:00:00') RETURNING id`)).id;
++ db.getStreamMemories = async () => [
++ { offset_seconds: 400, description: 'A very long and detailed scene note about the streamer sitting at a desk with a lamp and a plant and a poster' },
++ { offset_seconds: 1500, description: 'The streamer laughs at a clip on the second monitor' },
++ { offset_seconds: 3000, description: 'The streamer reads chat' },
++ ];
++ db.getStreamTranscriptSegments = async () => [{ start: 1495, end: 1505, text: 'oh no that is the funniest thing I have seen all week honestly' }];
++ db.getTimeline = async () => [];
++ chatReads.spikeOffsets = async () => [];
++ chatReads.streamStats = async () => ({ messages: 240, chatters: 9, sounds: 0 });
++ media.listClips = async () => ({ clips: [{ id: 11, start_time: 3000, auto_generated: false }] });
++ const v = { vod_id: 600, stream_id: sid, duration: 7200, title: 'Evidence' };
++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null,
++ 'scene notes alone, and the second a person already clipped, give no clip');
++ const paste = await moments.findBestMoment(v, { flavor: 'paste' });
++ assert.ok(paste && paste.offset === 3000 && paste.evidence.picked_by === 'people', 'a paste may still use the second people chose');
++
++ db.getTimeline = async () => [{ start_sec: 1500, end_sec: 1503, label: 'Laughter', confidence: 0.85 }];
++ chatReads.spikeOffsets = async () => [{ offset: 1500, count: 14, people: 5 }];
++ const clip = await moments.findBestMoment(v, { flavor: 'clip' });
++ assert.ok(clip, 'a laugh and a chat spike make a clip moment');
++ assert.strictEqual(clip.offset, 1500);
++ assert.strictEqual(clip.evidence.spike.peak, 14);
++ assert.strictEqual(clip.evidence.chatters, 5);
++ assert.ok(clip.evidence.spike.avg > 0.9 && clip.evidence.spike.avg < 1.1, 'the average counts the quiet stretches: 240 messages over 240 buckets');
++ assert.ok(clip.evidence.speechWords >= 10, 'the words come from the whole transcript near the moment, not a sample');
++ assert.strictEqual(clip.evidence.sounds[0].label, 'Laughter');
++
++ // ── The AI may abstain: no weaker signal is tried in its place ──
++ ai.isEnabled = async () => true; ai.withinBudget = async () => true;
++ aiService.structured = async () => ({ abstain: true, why: 'nothing stands out' });
++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 'the AI said nothing is worth it');
++ aiService.structured = async () => ({ t: null });
++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 't: null is an abstention too');
++ let asked = null;
++ aiService.structured = async (_name, input) => { asked = input; return { t: 1500, title: 'Lost it', desc: 'The streamer laughs at a clip on the second monitor' }; };
++ const picked = await moments.findBestMoment(v, { flavor: 'clip' });
++ assert.ok(picked && picked.evidence.picked_by === 'ai');
++ assert.deepStrictEqual(asked.clipped, [], 'a clip pick is not steered to the second people already clipped');
++ assert.ok(asked.avoid.includes(3000), '…it is told to stay away from it');
++
++ console.log = logs;
++ console.log('vod moment evidence: all checks passed');
++})().catch((e) => { console.error(e); process.exit(1); });