openvibers/openvibe.live Commits
VOD clips are scored before they are cut, and may be none (item 151, C08) (#580)
Changes
@@@ -214,6 +214,7 @@ Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.- **Only good live clips, cut on sentences** — a live chat spike is not enough on its own. `server/ai/clip-score.js` scores the moment from 0 to 1: the spike (how far above the window's average and how big), how many people reacted, what was heard (laughter, cheering, a bang; music and speech alone count for nothing), how much was said, and the AI's yes or no (a no is a veto; with the AI off its part is neutral and both bars rise by 0.1). Below **0.40** no clip is cut, the log says why ("a weak chat spike (10 messages, 3.1× the average)"). From 0.40 the clip is made **unlisted**: the streamer can find it and make it public. Calibrated by the production audit (C01, `scripts/diagnostics/clip-audit-*.sql`: the old rule made 99.6 % of automatic clips public and 29 % were never watched in 30 days): a moment needs **evidence** to be kept at all (a spike of at least 3× the average and 10 messages, a strong sound, or 20 spoken words; the AI's yes alone is not enough), and it is cut **public** only at **0.70** or more **and** in the reacted bucket: at least 12 messages in the peak from at least 3 people (`chatReads.recentChatters`; unknown means unlisted). Every cut logs its bucket (A no chat, B 1–5, C 6–11, D 12+ with 3+ people), peak and chatters for the next audit. `server/ai/clip-trim.js` then moves the fixed 16 s before / 9 s after window onto the speech rows: the start goes back to where the sentence being spoken began (so the setup is in), at most 25 s before the moment; the end goes on to where the sentence ends, at most 20 s after, never past the recording's flushed edge. A snap that would make the clip shorter than 12 s or longer than 45 s is not taken. Both are pure, with their cases in `test/clip-score-trim.test.js`.++- **VOD clips are scored too, and may be none** (item 151, C08). The VOD cutter (`clipVodMoment`, from the backfill and from the paste job's second beat) used to cut whatever the picker returned, public. Now the picker hands back the moment's **evidence**, gathered at that second from everything Live has (not the sample the prompt reads): the chat bucket there against the stream's own average per 30 s with the quiet stretches counted, how many people were in it (when Chat counts them), the sounds within 15 s, the words said within 20 s, and who picked it (`ai`, `chat`, `sound`, `people`, `scene`). The cutter runs it through the same `clip-score.js` as the live cutter: below the keep bar nothing is cut (the log says the score and why), a kept moment is **unlisted**, and only a reacted room (12+ messages from 3+ people, a score of 0.70 or more) is cut **public**, never above the recording's own visibility. A moment with no evidence is not cut. A clip never re-cuts a second a person already clipped (that footage is their clip already; it goes in the picker's avoid list), and with the AI off a clip needs a chat spike or a strong sound: a long scene note alone only ever makes a paste. The AI may answer that nothing is worth it (`{ abstain: true }` or `t: null`), and then no weaker signal is tried. The AI's own clips are no longer counted as "people clipped here" (`lookups.clipStartTimes`). Each cut logs its score, bucket, picker and policy (`vod-moment@2026-11-03`), and the daily Bench evaluation labels the AI's entry with that policy (`clip-evaluation.policyOf`), so Bench's accept rate tells this policy from the one before. Cases: `test/vod-moment-evidence.test.js`.- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).## AI-inferred category@@@ -24,6 +24,7 @@ const registry = require('./moment-registry');const clipPolicy = require('./clip-policy');const dedupe = require('./clip-dedupe');const explicit = require('./explicit');++const clipScore = require('./clip-score');let cfg = null; try { cfg = require('../config'); } catch { /* */ }let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');@@@ -148,7 +149,9 @@ function _sample(arr, max) {}async function _momentContext(streamId, vodId) {const memories = _sample((await db.getStreamMemories(streamId) || []).filter(m => m.description), 45);-- const transcript = _sample(await db.getStreamTranscriptSegments(streamId) || [], 60);++ // The picker reads a sample spread over the stream; the evidence at the chosen second reads everything near it.++ const said = await db.getStreamTranscriptSegments(streamId) || [];++ const transcript = _sample(said, 60);// Where viewers clipped: the clips live in OpenVibe.Media.const clipTimes = await require('../media-proxy/lookups').clipStartTimes(streamId, vodId);// Chat's own buckets over the stream (offsets are relative to started_at, as before; it ends at ended_at, or now).@@@ -158,13 +161,21 @@ async function _momentContext(streamId, vodId) {const spikes = started > 0 ? (await chatReads.spikeOffsets(streamId, 30, 8, started, ms(span.ended_at) || undefined) || []) : [];// Non-speech sounds are strong moment candidates — an explosion or a burst of// laughter marks a highlight as reliably as anything said out loud.-- let sounds = [];++ let heard = [];try {-- sounds = _sample((await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])++ heard = (await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])// Background music (audio-events.js spans) is the stream's bed, not a moment.-- .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music'), 30);++ .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music');} catch { /* timeline optional */ }-- return _inPart({ memories, transcript, clipTimes, spikes, sounds }, vodId);++ const sounds = _sample(heard, 30);++ // The stream's average chat per 30 s, empty stretches counted (C07/C08: averaging only the busy buckets made every++ // spike look small on a quiet channel and every lull look big on a busy one).++ let chatAvg = 0;++ try {++ const secs = started > 0 ? ((ms(span.ended_at) || Date.now()) - started) / 1000 : 0;++ if (secs >= 60) chatAvg = (Number((await chatReads.streamStats(streamId) || {}).messages) || 0) / (secs / 30);++ } catch { /* chat optional */ }++ return _inPart({ memories, transcript, said, clipTimes, spikes, sounds, heard, chatAvg }, vodId);}/**@@@ -183,8 +194,10 @@ async function _inPart(ctx, vodId) {...ctx,memories: ctx.memories.filter(m => inside(Number(m.offset_seconds) || 0)).map(m => ({ ...m, offset_seconds: move(Number(m.offset_seconds) || 0) })),transcript: ctx.transcript.filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),++ said: (ctx.said || []).filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),spikes: ctx.spikes.filter(sp => inside(Number(sp.offset))).map(sp => ({ ...sp, offset: move(Number(sp.offset)) })),sounds: ctx.sounds.filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),++ heard: (ctx.heard || []).filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),};}function _nearestMemory(memories, offset) {@@@ -192,6 +205,25 @@ function _nearestMemory(memories, offset) {for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }return best;}++/**++ * What there is to go on at one second (item 151, C08), in the shape clip-score.js reads: the chat bucket at it against++ * the stream's average, how many people were in it (when Chat says), the sounds within 15 s, the words said within 20 s,++ * and who picked it (`picked_by`: ai, chat, sound, people, scene). The cutter scores it before anything is cut.++ */++function _evidenceAt(ctx, t, pickedBy) {++ const at = Number(t) || 0;++ const bucket = (ctx.spikes || []).filter(sp => Math.abs(Number(sp.offset) - at) <= 30).sort((a, b) => b.count - a.count)[0] || null;++ const sounds = (ctx.heard || ctx.sounds || []).filter(e => Math.abs(Number(e.start_sec) - at) <= 15)++ .map(e => ({ label: String(e.label || ''), confidence: Number(e.confidence) || 0 }));++ const words = (ctx.said || ctx.transcript || []).filter(x => Math.abs(Number(x.start || 0) - at) <= 20)++ .reduce((n, x) => n + String(x.text || '').trim().split(/\s+/).filter(Boolean).length, 0);++ return {++ picked_by: pickedBy,++ spike: { peak: bucket ? Number(bucket.count) || 0 : 0, avg: Math.round((Number(ctx.chatAvg) || 0) * 100) / 100 },++ chatters: bucket && bucket.people != null ? Number(bucket.people) : null,++ sounds, speechWords: words,++ };++}/** Transcript text around a moment, so dedupe can tell a new conversation from a repeated scene. */function _speechNear(transcript, offset, windowSec = 45) {try {@@@ -202,10 +234,14 @@ function _speechNear(transcript, offset, windowSec = 45) {}// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.++// The result carries `evidence` (_evidenceAt): the clip cutter scores it and abstains when it is thin (C08).async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {const dur = Math.floor(vod.duration || 0);const ctx = await _momentContext(vod.stream_id, vod.vod_id);-- const avoidList = [...new Set([...(avoid || []), ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);++ // A clip never re-cuts what a person already clipped: that footage is a clip already, theirs (C08). A paste (one++ // frame) may still use the second people chose.++ const peopleClipped = flavor === 'clip' ? ctx.clipTimes : [];++ const avoidList = [...new Set([...(avoid || []), ...peopleClipped, ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);if (!ctx.memories.length && !ctx.transcript.length) return null;const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));@@@ -219,18 +255,24 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {timeline: ctx.memories.map(m => ({ t: secs(m.offset_seconds), text: _cleanText(_deJson(m.description), 180) })).filter(x => x.text).slice(0, 400),transcript: ctx.transcript.map(x => ({ t: secs(x.start), text: _cleanText(x.text, 160) })).filter(x => x.text).slice(0, 1500),sounds: (ctx.sounds || []).map(e => ({ t: secs(e.start_sec), label: _cleanText(e.label, 120) })).filter(x => x.label).slice(0, 500),-- clipped: ctx.clipTimes.slice(0, 12).map(secs),++ clipped: (flavor === 'clip' ? [] : ctx.clipTimes).slice(0, 12).map(secs),spikes: ctx.spikes.slice(0, 6).map(sp => secs(sp.offset)),avoid: avoidList.map(secs).slice(0, 500),};try {const j = await aiService.structured('live.moments.pick', input, { target: { service: 'live', type: 'vod', id: String(vod.vod_id) }, meter: { kind: 'moment_pick', role: 'legacy' } });++ // The AI may say nothing here is worth it (C08/C09): that is an answer, not a failure, so no weaker signal++ // is tried in its place.++ if (j && (j.abstain === true || (Object.prototype.hasOwnProperty.call(j, 't') && j.t === null))) {++ console.log(`[AI-Moments] VOD ${vod.vod_id}: the AI found nothing worth a ${flavor === 'clip' ? 'clip' : 'paste'}${j.why ? ` (${_cleanText(j.why, 120)})` : ''}`);++ return null;++ }let t = j ? Number(j.t) : NaN;if (!isNaN(t)) {t = clamp(t);if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }const near = _nearestMemory(ctx.memories, t);-- const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.transcript, t), tags: _memTags(near) };++ const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.said || ctx.transcript, t), tags: _memTags(near), evidence: _evidenceAt(ctx, t, 'ai') };if (explicit.isExplicit(j.desc, j.title, near && near.description, result.desc)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: the pick at ${_mmss(t)} shows sexual content — no moment from it`); return null; }return _isEmptyScene(result.desc) ? null : result;}@@@ -238,18 +280,22 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {}// No-AI fallback (also used when the AI pick collided with a used moment): the strongest-- // objective signal that is not next to something already made.++ // objective signal that is not next to something already made. A clip needs something people or the room did — a++ // chat spike or a strong sound; a long scene note alone is a description, not a moment (C08), so only a paste may++ // fall back to one.++ const strongSounds = (ctx.heard || ctx.sounds || []).filter(e => Number(e.confidence) >= clipScore.EVIDENCE.soundConfidence && clipScore.STRONG_SOUND.test(String(e.label || '')));const candidates = [-- ...ctx.clipTimes.map(t => ({ t, w: 3 })), // where viewers clipped-- ...ctx.spikes.map(s => ({ t: s.offset, w: 2 })), // busiest chat moments-- ...ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1 })), // richest scene notes++ ...(flavor === 'clip' ? [] : ctx.clipTimes.map(t => ({ t, w: 3, by: 'people' }))), // where people clipped++ ...ctx.spikes.map(s => ({ t: s.offset, w: 2, by: 'chat' })), // busiest chat moments++ ...strongSounds.map(e => ({ t: Number(e.start_sec) || 0, w: 2, by: 'sound' })), // laughter, a crowd, a bang++ ...(flavor === 'clip' ? [] : ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1, by: 'scene' }))), // richest scene notes].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);if (!candidates.length) return null;let offset = clamp(candidates[0].t);const near = _nearestMemory(ctx.memories, offset);const desc = near ? _deJson(near.description) : '';if (near && explicit.isExplicit(near.description)) return null; // never a moment of sexual content (server/ai/explicit.js)-- return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.transcript, offset), tags: _memTags(near) };++ return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.said || ctx.transcript, offset), tags: _memTags(near), evidence: _evidenceAt(ctx, offset, candidates[0].by) };}// True when a description says the frame is essentially empty — black/dark/loading/blank — so we// never turn "nothing" into a clip or paste.@@@ -537,7 +583,7 @@ async function tick(opts = {}) {try {const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });if (second && !await registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title, speech: second.speech })) {-- const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, archival: v.archival, source: momentSource });++ const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, evidence: second.evidence, archival: v.archival, source: momentSource });if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);} else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);} catch { /* clipping is best-effort */ }@@@ -566,7 +612,7 @@ function start() {setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);}--module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle } };++module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle, evidenceAt: _evidenceAt } };// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2@@@ -311,7 +311,7 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}if (dark) { console.log(`[AutoClip] backfill skip VOD ${v.vod_id} — dark/empty frame`); newSkip.push(v.vod_id); continue; }}} catch { /* probe optional */ }-- const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, archival: v.archival, source });++ const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, evidence: moment.evidence, archival: v.archival, source });if (clip) { made++; if (v.archival) archiveMade++; console.log(`[AutoClip] Backfilled VOD ${v.vod_id}${v.archival ? ' (archive)' : ''} ("${String(moment.title || '').slice(0, 60)}")`); }else if (v.archival && !clipPolicy.archiveAllowed(await _clipLog(), Date.now())) continue; // another cutter took today's archive slot: not this VOD's fault, don't skip it foreverelse newSkip.push(v.vod_id); // cut failed (e.g. unseekable/short) — don't retry forever@@@ -325,8 +325,28 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}/*** Cut a clip around a moment the AI-moments pipeline already chose for a finished VOD.-- * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, archival, source }++ * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, evidence, archival, source }++ *++ * The moment is scored before anything is cut (item 151, C08), by the same clip-score.js as the live cutter: `evidence`++ * (ai-moments-job's evidenceAt) is the chat bucket at the moment against the stream's average, the people in it, the++ * sounds and the words near it, and who picked it. A thin moment is not cut ("no clip beats a bad clip"), one that++ * clears the keep bar is unlisted, and only a reacted moment (12+ messages from 3+ people) can be public. A moment with++ * no evidence at all is not cut. The log keeps the score, bucket, picker and policy, so Bench and the audits can tell++ * this policy from the one before.*/++const VOD_POLICY = Object.freeze({ selector: 'vod-moment', version: '2026-11-03' });++function rateVodMoment(evidence) {++ if (!evidence || typeof evidence !== 'object') return { verdict: 'skip', score: 0, bucket: 'A', reasons: ['no evidence was gathered for the moment'] };++ const sp = evidence.spike || {};++ return clipScore.score({++ spike: { peak: Number(sp.peak) || 0, avg: Number(sp.avg) || 0 },++ chatters: evidence.chatters == null ? null : Number(evidence.chatters),++ sounds: Array.isArray(evidence.sounds) ? evidence.sounds : [],++ speechWords: Number(evidence.speechWords) || 0,++ // The AI's pick counts as its yes; a pick from the signals alone has no AI opinion (both bars rise).++ ai: { clip: evidence.picked_by === 'ai' ? true : null },++ });++}function _sceneSig(text) {return dedupe.shortSignature(text, 5);}@@@ -345,9 +365,14 @@ async function _isDuplicateAutoClip(vod, offset, desc, title, speech) {async function clipVodMoment(o) {let archiveHeld = false;try {-- const { vod, offset, title, desc, speech, archival } = o || {};++ const { vod, offset, title, desc, speech, archival, evidence } = o || {};if (!vod || !(offset >= 0)) return null;if (!await _derivationOn(vod.user_id)) return null; // the channel turned AI Moments off++ const rated = rateVodMoment(evidence);++ if (rated.verdict === 'skip') {++ console.log(`[AutoClip] VOD ${vod.vod_id || vod.id} @${Math.round(offset)}s not cut — score ${rated.score}: ${(rated.reasons || []).slice(-2).join('; ') || 'below the bar'}`);++ return null;++ }if (require('./explicit').isExplicit(desc, title)) { // never a clip of sexual content (server/ai/explicit.js)console.log(`[AutoClip] skip VOD ${vod.vod_id || vod.id} — the AI describes the moment as sexual content`);return null;@@@ -361,7 +386,7 @@ async function clipVodMoment(o) {// the clip public whatever the recording is.const publication = require('./clip-publication');if (!publication.mayCut(vodId)) { console.log(`[AutoClip] VOD ${vodId}: protected, not cut`); return null; }-- const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: 'public' });++ const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: rated.verdict === 'public' ? 'public' : 'unlisted' });const start = Math.max(0, Math.floor(offset) - CLIP_PRE);// One archive clip per UTC day is shared with the AI moments job. Both may pass the pool// gate before either writes auto_clip_log, so claim the slot atomically before cutting and@@@ -381,7 +406,11 @@ async function clipVodMoment(o) {});if (clip) {archiveHeld = false; // the clip exists: keep the day reserved even if the log write fails-- await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title) });++ const sp = (evidence && evidence.spike) || {};++ await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title),++ score: rated.score, visibility: rated.verdict, bucket: rated.bucket, peak: Number(sp.peak) || 0, chatters: evidence.chatters == null ? undefined : evidence.chatters,++ picked_by: evidence.picked_by || undefined, policy: VOD_POLICY });++ console.log(`[AutoClip] VOD ${vodId} @${Math.round(offset)}s cut (${evidence.picked_by || 'signals'}) — score ${rated.score} → ${rated.verdict}, bucket ${rated.bucket}`);}return clip || null;} catch { return null; }@@@ -452,7 +481,7 @@ function stop() {require('./clip-evaluation').stop();}--module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool } };++module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool, rateVodMoment, VOD_POLICY } };// CLI: force a historical backfill batch, e.g. `node server/ai/auto-clip-job.js --backfill --limit=4`if (require.main === module) {@@@ -3,8 +3,8 @@* clip-evaluation.js — the AI's clip picks against people's, blind, on OpenVibe.Bench (item 151, C20).** Once a day, for up to MAX_PER_RUN recent recordings that have both a public clip the AI cut and a public clip a-- * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY)-- * and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at++ * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY, or++ * the one the auto-clip log names for it: policyOf) and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at* https://bench.openvibe.services/battle?kind=live.clip (Bench #106: two players, blind, no titles), and* results() reads each side's accept rate back from Bench (both_bad counts against both).*@@@ -66,6 +66,17 @@ async function bench(method, path, body) {return json;}++/**++ * Which picker cut an AI clip, for Bench's per-policy accept rate: the auto-clip log names it on clips cut since C08++ * (2026-11-03: VOD moments are `vod-moment@2026-11-03`); older clips and the live cutter are `auto-clip@AUTO_CLIP_POLICY`.++ */++function policyOf(clip, clipLog = []) {++ const e = (Array.isArray(clipLog) ? clipLog : []).find((x) => x && x.clip_id != null && String(x.clip_id) === String(clip.id));++ const p = e && e.policy;++ if (p && typeof p.selector === 'string' && typeof p.version === 'string') return { selector: p.selector.slice(0, 80), version: p.version.slice(0, 80), models: [] };++ return { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] };++}++function entryOf(clip, policy, signed) {const start = Math.round(Number(clip.start_time) * 1000);const len = Math.round(Number(clip.duration_seconds ?? clip.duration ?? (Number(clip.end_time) - Number(clip.start_time))) * 1000);@@@ -85,6 +96,8 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {if (!ai || !Array.isArray(ai.clips)) return { submitted: [], skipped: 0, why: 'Media did not answer' };let done;try { done = new Set((await _log()).map((x) => String(x.vod_id))); } catch (e) { return { submitted: [], skipped: 0, why: `state unreadable: ${e.message}` }; }++ let clipLog = [];++ try { clipLog = JSON.parse(await db.getState('auto_clip_log') || '[]'); } catch { clipLog = []; }const byVod = new Map();for (const c of ai.clips) if (c.auto_generated === true && eligible(c) && !done.has(String(c.vod_id)) && !byVod.has(String(c.vod_id))) byVod.set(String(c.vod_id), c);const submitted = [];@@@ -106,7 +119,7 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {const task = await bench('POST', '/api/v1/kinds/live.clip/tasks', { prompt: 'Two clips of one stream', public: false });const round = await bench('POST', `/api/v1/tasks/${task.task.id}/rounds`, {});const roundId = round.round.id;-- await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] }, sa) });++ await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, policyOf(aiClip, clipLog), sa) });await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(person, { selector: 'people', version: '1', models: [] }, sp) });const log = await _log();const mine = log.find((x) => String(x.vod_id) === vodId && x.round_id == null);@@@ -136,4 +149,4 @@ function start() {}function stop() { if (_stop) { _stop(); _stop = null; } }--module.exports = { run, results, start, stop, eligible, entryOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };++module.exports = { run, results, start, stop, eligible, entryOf, policyOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };@@@ -500,7 +500,7 @@ function recentChatters(streamId, sinceSec = 30, limit = 200) {}/** The busiest `bucketSec` time-buckets of a stream between `sinceMs` and `untilMs` (default now), as offsets from-- * `sinceMs`: [{ offset, count }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to++ * `sinceMs`: [{ offset, count, people }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to* now, a stream from two months ago asked for 184,000 and got a 400 (2026-10-23). */function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {const id = Number(streamId) || 0;@@@ -512,7 +512,8 @@ function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {if (!out || !Array.isArray(out.buckets)) throw unavailable('timeline');const base = sinceMs || 0;return out.buckets-- .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0 }))++ // people: how many different chatters were in the bucket, when Chat says (null when it does not).++ .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0, people: b.people == null ? null : Number(b.people) || 0 })).sort((a, b) => b.count - a.count || a.offset - b.offset).slice(0, topN);}, () => []);@@@ -218,9 +218,13 @@ async function streamClips(streamId, vodId) {return [...byId.values()];}--/** Seconds into the VOD where viewers clipped (the AI moment picker's strongest signal). */++/**++ * Seconds into the VOD where people clipped (the AI moment picker's strongest signal). The AI's own clips are left out++ * (item 151, C08): they are not people choosing a moment, and counting them made every AI clip a reason for the next.++ */async function clipStartTimes(streamId, vodId) {return (await streamClips(streamId, vodId))++ .filter((c) => c.auto_generated !== true).map((c) => Number(c.start_time)).filter((t) => t > 0).map(Math.floor).sort((a, b) => a - b);}@@@ -148,10 +148,10 @@ async function check(name, fn) {const pool = await autoClip._internals.backfillPool(5);assert.deepStrictEqual(pool.map((v) => v.vod_id), [10], 'the VOD backfill pool leaves the opted-out channel out');created.length = 0;-- const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment' });++ const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });assert.strictEqual(none, null);assert.strictEqual(created.length, 0);-- const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment' });++ const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });assert.ok(clip && clip.id, 'Alice still gets auto-clips');assert.strictEqual(created.length, 1);assert.strictEqual(created[0].auto_generated, true);@@@ -15,7 +15,8 @@ const chatReads = require('../server/chat/chat-reads');const started = Date.parse('2026-08-20T12:00:00Z');const ended = started + 2 * 3600 * 1000;const spikes = await chatReads.spikeOffsets(2125, 30, 8, started, ended);-- assert.deepStrictEqual(spikes, [{ offset: 30, count: 9 }, { offset: 60, count: 3 }]);++ // people is null when Chat does not count chatters per bucket (C08 reads it as "unknown").++ assert.deepStrictEqual(spikes, [{ offset: 30, count: 9, people: null }, { offset: 60, count: 3, people: null }]);assert.deepStrictEqual(calls.pop(), { stream_id: 2125, since: started, until: ended, bucket_ms: 30000 }, 'an ended stream is read up to its end');// A stream still live (no end): up to now, as before; the client leaves a null `until` out of the query.@@@ -30,6 +31,9 @@ const chatReads = require('../server/chat/chat-reads');// An end before the start is ignored (asked up to now).await chatReads.spikeOffsets(2128, 30, 8, started, started - 1000);assert.strictEqual(calls.pop().until, null);++ // When Chat counts the people in each bucket, it rides along.++ client.readTimeline = async (q2) => ({ buckets: [{ t: q2.since + 30000, count: 14, people: 5 }] });++ assert.deepStrictEqual(await chatReads.spikeOffsets(2129, 30, 8, started, ended), [{ offset: 30, count: 14, people: 5 }]);console.log('chat-spike-window: ok');process.exit(0);})().catch((e) => { console.error(e); process.exit(1); });@@@ -248,6 +248,8 @@ const archiveCut = (id) => autoClip.clipVodMoment({offset: 60,title: `Archive moment ${id}`,desc: `A distinct archival scene number ${id} with enough words`,++ // A moment worth cutting (C08: the cutter scores the evidence first).++ evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 },archival: true,});const firstCut = archiveCut(70);nnew file mode 100644@@@ -0,0 +1,122 @@++'use strict';++// The VOD-moment clip cutter scores what it has before it cuts, and may abstain (item 151, C08):++// - the AI's own clips are not "people clipped here" (lookups.clipStartTimes);++// - a clip never re-cuts footage a person already clipped, and never comes from a long scene note alone;++// - the AI may say nothing is worth it, and no weaker signal is tried in its place;++// - the moment's evidence (chat bucket against the stream's zero-filled average, people, sounds, words) goes through++// clip-score.js: thin → not cut, kept → unlisted, a reacted room → public; no evidence → not cut;++// - the clip log and Bench name the policy that cut it.++const assert = require('assert');++const db = require('../server/db/database');++console.warn = () => {};++++(async () => {++ await db.initDb();++ const media = require('../server/media-client');++ const lookups = require('../server/media-proxy/lookups');++ const chatReads = require('../server/chat/chat-reads');++ const registry = require('../server/ai/moment-registry');++ const ai = require('../server/ai/ai-analysis');++ const aiService = require('../server/ai/ai-service');++ const autoClip = require('../server/ai/auto-clip-job');++ const moments = require('../server/ai/ai-moments-job');++ const evaluation = require('../server/ai/clip-evaluation');++ const log = []; const logs = console.log; console.log = (...a) => log.push(a.join(' '));++++ // ── People's clips only ───────────────────────────────────────────────++ media.listClips = async () => ({ clips: [++ { id: 1, start_time: 600, auto_generated: false },++ { id: 2, start_time: 1200, auto_generated: true },++ { id: 3, start_time: 1800 },++ ] });++ assert.deepStrictEqual(await lookups.clipStartTimes(91, 92), [600, 1800], 'the AI\'s clip at 1200 s is not a person choosing a moment');++++ // ── The score at the cutter ───────────────────────────────────────────++ const rate = autoClip._internals.rateVodMoment;++ assert.strictEqual(rate(undefined).verdict, 'skip', 'no evidence, no clip');++ const quiet = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 });++ assert.strictEqual(quiet.verdict, 'unlisted', 'a quiet channel\'s laugh the AI picked is kept: a small audience is not a poor moment');++ assert.strictEqual(quiet.bucket, 'A');++ const talk = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 });++ assert.strictEqual(talk.verdict, 'skip', 'talk the AI liked, with nothing heard or typed, is not cut: no quota to fill');++ const room = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 });++ assert.strictEqual(room.verdict, 'public', 'a reacted room (22 messages from 7 people) can be public');++ const unknown = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: null, sounds: [], speechWords: 25 });++ assert.strictEqual(unknown.verdict, 'unlisted', 'the same spike with nobody counted is kept unlisted');++ const signals = rate({ picked_by: 'chat', spike: { peak: 9, avg: 3 }, chatters: 2, sounds: [], speechWords: 5 });++ assert.strictEqual(signals.verdict, 'skip', 'a small spike with no AI opinion is not cut (both bars rise without the AI)');++++ // ── clipVodMoment: the verdict is the visibility, and the log keeps the provenance ──++ const created = [];++ media.createClip = async (body) => { created.push(body); return { id: 9100 + created.length }; };++ media.getVod = async () => ({ id: 1, visibility: 'public' });++ await db.setState('auto_clip_log', '[]');++ await registry._clear();++ const vod = (id) => ({ vod_id: id, id, user_id: 3, stream_id: id, visibility: 'public' });++ assert.strictEqual(await autoClip.clipVodMoment({ vod: vod(501), offset: 300, title: 'Just talking', desc: 'A person talks to the camera about lunch', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 } }), null);++ assert.strictEqual(created.length, 0, 'the thin moment never reached Media');++ assert.ok(log.some((l) => /VOD 501 @300s not cut — score/.test(l)), 'the abstention is logged with its score');++ const c1 = await autoClip.clipVodMoment({ vod: vod(502), offset: 300, title: 'The room lost it', desc: 'The streamer trips over the cable mid-sentence', evidence: { picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 } });++ assert.ok(c1 && c1.id);++ const c2 = await autoClip.clipVodMoment({ vod: vod(503), offset: 300, title: 'A laugh', desc: 'A dog barks and the streamer laughs at the window', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });++ assert.ok(c2 && c2.id);++ assert.deepStrictEqual(created.map((b) => b.visibility), ['public', 'unlisted'], 'public only for the reacted room');++ const entries = JSON.parse(await db.getState('auto_clip_log'));++ const e1 = entries.find((e) => e.clip_id === c1.id);++ assert.deepStrictEqual(e1.policy, { selector: 'vod-moment', version: '2026-11-03' });++ assert.strictEqual(e1.picked_by, 'ai');++ assert.strictEqual(e1.bucket, 'D');++ assert.ok(e1.score >= 0.7);++++ // ── Bench names the policy that cut the clip ──++ assert.deepStrictEqual(evaluation.policyOf({ id: c1.id }, entries), { selector: 'vod-moment', version: '2026-11-03', models: [] });++ assert.deepStrictEqual(evaluation.policyOf({ id: 4242 }, entries), { selector: 'auto-clip', version: evaluation.AUTO_CLIP_POLICY, models: [] }, 'an older clip keeps the old label');++++ // ── The picker, AI off: a clip needs something people or the room did ──++ ai.isEnabled = async () => false;++ await db.run(`INSERT INTO users (id, username, display_name, email, password_hash, role, created_at) OVERRIDING SYSTEM VALUE++ VALUES (3, 'eve', 'eve', 'eve@x', 'x', 'streamer', '2025-01-01 00:00:00') ON CONFLICT DO NOTHING`);++ const sid = (await db.get(`INSERT INTO streams (user_id, title, started_at, ended_at) VALUES (3, 'evidence', '2026-11-01 10:00:00', '2026-11-01 12:00:00') RETURNING id`)).id;++ db.getStreamMemories = async () => [++ { offset_seconds: 400, description: 'A very long and detailed scene note about the streamer sitting at a desk with a lamp and a plant and a poster' },++ { offset_seconds: 1500, description: 'The streamer laughs at a clip on the second monitor' },++ { offset_seconds: 3000, description: 'The streamer reads chat' },++ ];++ db.getStreamTranscriptSegments = async () => [{ start: 1495, end: 1505, text: 'oh no that is the funniest thing I have seen all week honestly' }];++ db.getTimeline = async () => [];++ chatReads.spikeOffsets = async () => [];++ chatReads.streamStats = async () => ({ messages: 240, chatters: 9, sounds: 0 });++ media.listClips = async () => ({ clips: [{ id: 11, start_time: 3000, auto_generated: false }] });++ const v = { vod_id: 600, stream_id: sid, duration: 7200, title: 'Evidence' };++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null,++ 'scene notes alone, and the second a person already clipped, give no clip');++ const paste = await moments.findBestMoment(v, { flavor: 'paste' });++ assert.ok(paste && paste.offset === 3000 && paste.evidence.picked_by === 'people', 'a paste may still use the second people chose');++++ db.getTimeline = async () => [{ start_sec: 1500, end_sec: 1503, label: 'Laughter', confidence: 0.85 }];++ chatReads.spikeOffsets = async () => [{ offset: 1500, count: 14, people: 5 }];++ const clip = await moments.findBestMoment(v, { flavor: 'clip' });++ assert.ok(clip, 'a laugh and a chat spike make a clip moment');++ assert.strictEqual(clip.offset, 1500);++ assert.strictEqual(clip.evidence.spike.peak, 14);++ assert.strictEqual(clip.evidence.chatters, 5);++ assert.ok(clip.evidence.spike.avg > 0.9 && clip.evidence.spike.avg < 1.1, 'the average counts the quiet stretches: 240 messages over 240 buckets');++ assert.ok(clip.evidence.speechWords >= 10, 'the words come from the whole transcript near the moment, not a sample');++ assert.strictEqual(clip.evidence.sounds[0].label, 'Laughter');++++ // ── The AI may abstain: no weaker signal is tried in its place ──++ ai.isEnabled = async () => true; ai.withinBudget = async () => true;++ aiService.structured = async () => ({ abstain: true, why: 'nothing stands out' });++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 'the AI said nothing is worth it');++ aiService.structured = async () => ({ t: null });++ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 't: null is an abstention too');++ let asked = null;++ aiService.structured = async (_name, input) => { asked = input; return { t: 1500, title: 'Lost it', desc: 'The streamer laughs at a clip on the second monitor' }; };++ const picked = await moments.findBestMoment(v, { flavor: 'clip' });++ assert.ok(picked && picked.evidence.picked_by === 'ai');++ assert.deepStrictEqual(asked.clipped, [], 'a clip pick is not steered to the second people already clipped');++ assert.ok(asked.avoid.includes(3000), '…it is told to stay away from it');++++ console.log = logs;++ console.log('vod moment evidence: all checks passed');++})().catch((e) => { console.error(e); process.exit(1); });
@@ -214,6 +214,7 @@ Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image
@@ -214,6 +214,7 @@ Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image
- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.
- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.
- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.
- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.
- **Only good live clips, cut on sentences** — a live chat spike is not enough on its own. `server/ai/clip-score.js` scores the moment from 0 to 1: the spike (how far above the window's average and how big), how many people reacted, what was heard (laughter, cheering, a bang; music and speech alone count for nothing), how much was said, and the AI's yes or no (a no is a veto; with the AI off its part is neutral and both bars rise by 0.1). Below **0.40** no clip is cut, the log says why ("a weak chat spike (10 messages, 3.1× the average)"). From 0.40 the clip is made **unlisted**: the streamer can find it and make it public. Calibrated by the production audit (C01, `scripts/diagnostics/clip-audit-*.sql`: the old rule made 99.6 % of automatic clips public and 29 % were never watched in 30 days): a moment needs **evidence** to be kept at all (a spike of at least 3× the average and 10 messages, a strong sound, or 20 spoken words; the AI's yes alone is not enough), and it is cut **public** only at **0.70** or more **and** in the reacted bucket: at least 12 messages in the peak from at least 3 people (`chatReads.recentChatters`; unknown means unlisted). Every cut logs its bucket (A no chat, B 1–5, C 6–11, D 12+ with 3+ people), peak and chatters for the next audit. `server/ai/clip-trim.js` then moves the fixed 16 s before / 9 s after window onto the speech rows: the start goes back to where the sentence being spoken began (so the setup is in), at most 25 s before the moment; the end goes on to where the sentence ends, at most 20 s after, never past the recording's flushed edge. A snap that would make the clip shorter than 12 s or longer than 45 s is not taken. Both are pure, with their cases in `test/clip-score-trim.test.js`.
- **Only good live clips, cut on sentences** — a live chat spike is not enough on its own. `server/ai/clip-score.js` scores the moment from 0 to 1: the spike (how far above the window's average and how big), how many people reacted, what was heard (laughter, cheering, a bang; music and speech alone count for nothing), how much was said, and the AI's yes or no (a no is a veto; with the AI off its part is neutral and both bars rise by 0.1). Below **0.40** no clip is cut, the log says why ("a weak chat spike (10 messages, 3.1× the average)"). From 0.40 the clip is made **unlisted**: the streamer can find it and make it public. Calibrated by the production audit (C01, `scripts/diagnostics/clip-audit-*.sql`: the old rule made 99.6 % of automatic clips public and 29 % were never watched in 30 days): a moment needs **evidence** to be kept at all (a spike of at least 3× the average and 10 messages, a strong sound, or 20 spoken words; the AI's yes alone is not enough), and it is cut **public** only at **0.70** or more **and** in the reacted bucket: at least 12 messages in the peak from at least 3 people (`chatReads.recentChatters`; unknown means unlisted). Every cut logs its bucket (A no chat, B 1–5, C 6–11, D 12+ with 3+ people), peak and chatters for the next audit. `server/ai/clip-trim.js` then moves the fixed 16 s before / 9 s after window onto the speech rows: the start goes back to where the sentence being spoken began (so the setup is in), at most 25 s before the moment; the end goes on to where the sentence ends, at most 20 s after, never past the recording's flushed edge. A snap that would make the clip shorter than 12 s or longer than 45 s is not taken. Both are pure, with their cases in `test/clip-score-trim.test.js`.
+- **VOD clips are scored too, and may be none** (item 151, C08). The VOD cutter (`clipVodMoment`, from the backfill and from the paste job's second beat) used to cut whatever the picker returned, public. Now the picker hands back the moment's **evidence**, gathered at that second from everything Live has (not the sample the prompt reads): the chat bucket there against the stream's own average per 30 s with the quiet stretches counted, how many people were in it (when Chat counts them), the sounds within 15 s, the words said within 20 s, and who picked it (`ai`, `chat`, `sound`, `people`, `scene`). The cutter runs it through the same `clip-score.js` as the live cutter: below the keep bar nothing is cut (the log says the score and why), a kept moment is **unlisted**, and only a reacted room (12+ messages from 3+ people, a score of 0.70 or more) is cut **public**, never above the recording's own visibility. A moment with no evidence is not cut. A clip never re-cuts a second a person already clipped (that footage is their clip already; it goes in the picker's avoid list), and with the AI off a clip needs a chat spike or a strong sound: a long scene note alone only ever makes a paste. The AI may answer that nothing is worth it (`{ abstain: true }` or `t: null`), and then no weaker signal is tried. The AI's own clips are no longer counted as "people clipped here" (`lookups.clipStartTimes`). Each cut logs its score, bucket, picker and policy (`vod-moment@2026-11-03`), and the daily Bench evaluation labels the AI's entry with that policy (`clip-evaluation.policyOf`), so Bench's accept rate tells this policy from the one before. Cases: `test/vod-moment-evidence.test.js`.
- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).
- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).
## AI-inferred category
## AI-inferred category
@@ -24,6 +24,7 @@ const registry = require('./moment-registry');
@@ -24,6 +24,7 @@ const registry = require('./moment-registry');
const clipPolicy = require('./clip-policy');
const clipPolicy = require('./clip-policy');
const dedupe = require('./clip-dedupe');
const dedupe = require('./clip-dedupe');
const explicit = require('./explicit');
const explicit = require('./explicit');
+const clipScore = require('./clip-score');
let cfg = null; try { cfg = require('../config'); } catch { /* */ }
let cfg = null; try { cfg = require('../config'); } catch { /* */ }
let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }
let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }
const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');
const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');
@@ -148,7 +149,9 @@ function _sample(arr, max) {
@@ -148,7 +149,9 @@ function _sample(arr, max) {
}
}
async function _momentContext(streamId, vodId) {
async function _momentContext(streamId, vodId) {
const memories = _sample((await db.getStreamMemories(streamId) || []).filter(m => m.description), 45);
const memories = _sample((await db.getStreamMemories(streamId) || []).filter(m => m.description), 45);
- const transcript = _sample(await db.getStreamTranscriptSegments(streamId) || [], 60);
+ // The picker reads a sample spread over the stream; the evidence at the chosen second reads everything near it.
+ const said = await db.getStreamTranscriptSegments(streamId) || [];
+ const transcript = _sample(said, 60);
// Where viewers clipped: the clips live in OpenVibe.Media.
// Where viewers clipped: the clips live in OpenVibe.Media.
const clipTimes = await require('../media-proxy/lookups').clipStartTimes(streamId, vodId);
const clipTimes = await require('../media-proxy/lookups').clipStartTimes(streamId, vodId);
// Chat's own buckets over the stream (offsets are relative to started_at, as before; it ends at ended_at, or now).
// Chat's own buckets over the stream (offsets are relative to started_at, as before; it ends at ended_at, or now).
@@ -158,13 +161,21 @@ async function _momentContext(streamId, vodId) {
@@ -158,13 +161,21 @@ async function _momentContext(streamId, vodId) {
const spikes = started > 0 ? (await chatReads.spikeOffsets(streamId, 30, 8, started, ms(span.ended_at) || undefined) || []) : [];
const spikes = started > 0 ? (await chatReads.spikeOffsets(streamId, 30, 8, started, ms(span.ended_at) || undefined) || []) : [];
// Non-speech sounds are strong moment candidates — an explosion or a burst of
// Non-speech sounds are strong moment candidates — an explosion or a burst of
// laughter marks a highlight as reliably as anything said out loud.
// laughter marks a highlight as reliably as anything said out loud.
- let sounds = [];
+ let heard = [];
try {
try {
- sounds = _sample((await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])
+ heard = (await db.getTimeline(streamId, { kind: 'sound', limit: 400 }) || [])
// Background music (audio-events.js spans) is the stream's bed, not a moment.
// Background music (audio-events.js spans) is the stream's bed, not a moment.
- .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music'), 30);
+ .filter(e => Number(e.confidence || 0) >= 0.45 && e.label !== 'Music');
} catch { /* timeline optional */ }
} catch { /* timeline optional */ }
- return _inPart({ memories, transcript, clipTimes, spikes, sounds }, vodId);
+ const sounds = _sample(heard, 30);
+ // The stream's average chat per 30 s, empty stretches counted (C07/C08: averaging only the busy buckets made every
+ // spike look small on a quiet channel and every lull look big on a busy one).
+ let chatAvg = 0;
+ try {
+ const secs = started > 0 ? ((ms(span.ended_at) || Date.now()) - started) / 1000 : 0;
+ if (secs >= 60) chatAvg = (Number((await chatReads.streamStats(streamId) || {}).messages) || 0) / (secs / 30);
+ } catch { /* chat optional */ }
+ return _inPart({ memories, transcript, said, clipTimes, spikes, sounds, heard, chatAvg }, vodId);
}
}
/**
/**
@@ -183,8 +194,10 @@ async function _inPart(ctx, vodId) {
@@ -183,8 +194,10 @@ async function _inPart(ctx, vodId) {
...ctx,
...ctx,
memories: ctx.memories.filter(m => inside(Number(m.offset_seconds) || 0)).map(m => ({ ...m, offset_seconds: move(Number(m.offset_seconds) || 0) })),
memories: ctx.memories.filter(m => inside(Number(m.offset_seconds) || 0)).map(m => ({ ...m, offset_seconds: move(Number(m.offset_seconds) || 0) })),
transcript: ctx.transcript.filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),
transcript: ctx.transcript.filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),
+ said: (ctx.said || []).filter(t => inside(Number(t.start) || 0)).map(t => ({ ...t, start: move(Number(t.start) || 0), end: t.end == null ? null : move(Number(t.end)) })),
spikes: ctx.spikes.filter(sp => inside(Number(sp.offset))).map(sp => ({ ...sp, offset: move(Number(sp.offset)) })),
spikes: ctx.spikes.filter(sp => inside(Number(sp.offset))).map(sp => ({ ...sp, offset: move(Number(sp.offset)) })),
sounds: ctx.sounds.filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),
sounds: ctx.sounds.filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),
+ heard: (ctx.heard || []).filter(e => inside(Number(e.start_sec))).map(e => ({ ...e, start_sec: move(Number(e.start_sec)), end_sec: e.end_sec == null ? null : move(Number(e.end_sec)) })),
};
};
}
}
function _nearestMemory(memories, offset) {
function _nearestMemory(memories, offset) {
@@ -192,6 +205,25 @@ function _nearestMemory(memories, offset) {
@@ -192,6 +205,25 @@ function _nearestMemory(memories, offset) {
for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }
for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }
return best;
return best;
}
}
+/**
+ * What there is to go on at one second (item 151, C08), in the shape clip-score.js reads: the chat bucket at it against
+ * the stream's average, how many people were in it (when Chat says), the sounds within 15 s, the words said within 20 s,
+ * and who picked it (`picked_by`: ai, chat, sound, people, scene). The cutter scores it before anything is cut.
+ */
+function _evidenceAt(ctx, t, pickedBy) {
+ const at = Number(t) || 0;
+ const bucket = (ctx.spikes || []).filter(sp => Math.abs(Number(sp.offset) - at) <= 30).sort((a, b) => b.count - a.count)[0] || null;
+ const sounds = (ctx.heard || ctx.sounds || []).filter(e => Math.abs(Number(e.start_sec) - at) <= 15)
+ .map(e => ({ label: String(e.label || ''), confidence: Number(e.confidence) || 0 }));
+ const words = (ctx.said || ctx.transcript || []).filter(x => Math.abs(Number(x.start || 0) - at) <= 20)
+ .reduce((n, x) => n + String(x.text || '').trim().split(/\s+/).filter(Boolean).length, 0);
+ return {
+ picked_by: pickedBy,
+ spike: { peak: bucket ? Number(bucket.count) || 0 : 0, avg: Math.round((Number(ctx.chatAvg) || 0) * 100) / 100 },
+ chatters: bucket && bucket.people != null ? Number(bucket.people) : null,
+ sounds, speechWords: words,
+ };
+}
/** Transcript text around a moment, so dedupe can tell a new conversation from a repeated scene. */
/** Transcript text around a moment, so dedupe can tell a new conversation from a repeated scene. */
function _speechNear(transcript, offset, windowSec = 45) {
function _speechNear(transcript, offset, windowSec = 45) {
try {
try {
@@ -202,10 +234,14 @@ function _speechNear(transcript, offset, windowSec = 45) {
@@ -202,10 +234,14 @@ function _speechNear(transcript, offset, windowSec = 45) {
}
}
// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.
// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.
// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.
// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.
+// The result carries `evidence` (_evidenceAt): the clip cutter scores it and abstains when it is thin (C08).
async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
const dur = Math.floor(vod.duration || 0);
const dur = Math.floor(vod.duration || 0);
const ctx = await _momentContext(vod.stream_id, vod.vod_id);
const ctx = await _momentContext(vod.stream_id, vod.vod_id);
- const avoidList = [...new Set([...(avoid || []), ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);
+ // A clip never re-cuts what a person already clipped: that footage is a clip already, theirs (C08). A paste (one
+ // frame) may still use the second people chose.
+ const peopleClipped = flavor === 'clip' ? ctx.clipTimes : [];
+ const avoidList = [...new Set([...(avoid || []), ...peopleClipped, ...await registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);
const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);
const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);
if (!ctx.memories.length && !ctx.transcript.length) return null;
if (!ctx.memories.length && !ctx.transcript.length) return null;
const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));
const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));
@@ -219,18 +255,24 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
@@ -219,18 +255,24 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
timeline: ctx.memories.map(m => ({ t: secs(m.offset_seconds), text: _cleanText(_deJson(m.description), 180) })).filter(x => x.text).slice(0, 400),
timeline: ctx.memories.map(m => ({ t: secs(m.offset_seconds), text: _cleanText(_deJson(m.description), 180) })).filter(x => x.text).slice(0, 400),
transcript: ctx.transcript.map(x => ({ t: secs(x.start), text: _cleanText(x.text, 160) })).filter(x => x.text).slice(0, 1500),
transcript: ctx.transcript.map(x => ({ t: secs(x.start), text: _cleanText(x.text, 160) })).filter(x => x.text).slice(0, 1500),
sounds: (ctx.sounds || []).map(e => ({ t: secs(e.start_sec), label: _cleanText(e.label, 120) })).filter(x => x.label).slice(0, 500),
sounds: (ctx.sounds || []).map(e => ({ t: secs(e.start_sec), label: _cleanText(e.label, 120) })).filter(x => x.label).slice(0, 500),
- clipped: ctx.clipTimes.slice(0, 12).map(secs),
+ clipped: (flavor === 'clip' ? [] : ctx.clipTimes).slice(0, 12).map(secs),
spikes: ctx.spikes.slice(0, 6).map(sp => secs(sp.offset)),
spikes: ctx.spikes.slice(0, 6).map(sp => secs(sp.offset)),
avoid: avoidList.map(secs).slice(0, 500),
avoid: avoidList.map(secs).slice(0, 500),
};
};
try {
try {
const j = await aiService.structured('live.moments.pick', input, { target: { service: 'live', type: 'vod', id: String(vod.vod_id) }, meter: { kind: 'moment_pick', role: 'legacy' } });
const j = await aiService.structured('live.moments.pick', input, { target: { service: 'live', type: 'vod', id: String(vod.vod_id) }, meter: { kind: 'moment_pick', role: 'legacy' } });
+ // The AI may say nothing here is worth it (C08/C09): that is an answer, not a failure, so no weaker signal
+ // is tried in its place.
+ if (j && (j.abstain === true || (Object.prototype.hasOwnProperty.call(j, 't') && j.t === null))) {
+ console.log(`[AI-Moments] VOD ${vod.vod_id}: the AI found nothing worth a ${flavor === 'clip' ? 'clip' : 'paste'}${j.why ? ` (${_cleanText(j.why, 120)})` : ''}`);
+ return null;
+ }
let t = j ? Number(j.t) : NaN;
let t = j ? Number(j.t) : NaN;
if (!isNaN(t)) {
if (!isNaN(t)) {
t = clamp(t);
t = clamp(t);
if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }
if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }
const near = _nearestMemory(ctx.memories, t);
const near = _nearestMemory(ctx.memories, t);
- const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.transcript, t), tags: _memTags(near) };
+ const result = { offset: t, title: _publicTitle(j.title, vod, t, j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', speech: _speechNear(ctx.said || ctx.transcript, t), tags: _memTags(near), evidence: _evidenceAt(ctx, t, 'ai') };
if (explicit.isExplicit(j.desc, j.title, near && near.description, result.desc)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: the pick at ${_mmss(t)} shows sexual content — no moment from it`); return null; }
if (explicit.isExplicit(j.desc, j.title, near && near.description, result.desc)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: the pick at ${_mmss(t)} shows sexual content — no moment from it`); return null; }
return _isEmptyScene(result.desc) ? null : result;
return _isEmptyScene(result.desc) ? null : result;
}
}
@@ -238,18 +280,22 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
@@ -238,18 +280,22 @@ async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
}
}
// No-AI fallback (also used when the AI pick collided with a used moment): the strongest
// No-AI fallback (also used when the AI pick collided with a used moment): the strongest
- // objective signal that is not next to something already made.
+ // objective signal that is not next to something already made. A clip needs something people or the room did — a
+ // chat spike or a strong sound; a long scene note alone is a description, not a moment (C08), so only a paste may
+ // fall back to one.
+ const strongSounds = (ctx.heard || ctx.sounds || []).filter(e => Number(e.confidence) >= clipScore.EVIDENCE.soundConfidence && clipScore.STRONG_SOUND.test(String(e.label || '')));
const candidates = [
const candidates = [
- ...ctx.clipTimes.map(t => ({ t, w: 3 })), // where viewers clipped
- ...ctx.spikes.map(s => ({ t: s.offset, w: 2 })), // busiest chat moments
- ...ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1 })), // richest scene notes
+ ...(flavor === 'clip' ? [] : ctx.clipTimes.map(t => ({ t, w: 3, by: 'people' }))), // where people clipped
+ ...ctx.spikes.map(s => ({ t: s.offset, w: 2, by: 'chat' })), // busiest chat moments
+ ...strongSounds.map(e => ({ t: Number(e.start_sec) || 0, w: 2, by: 'sound' })), // laughter, a crowd, a bang
+ ...(flavor === 'clip' ? [] : ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1, by: 'scene' }))), // richest scene notes
].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);
].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);
if (!candidates.length) return null;
if (!candidates.length) return null;
let offset = clamp(candidates[0].t);
let offset = clamp(candidates[0].t);
const near = _nearestMemory(ctx.memories, offset);
const near = _nearestMemory(ctx.memories, offset);
const desc = near ? _deJson(near.description) : '';
const desc = near ? _deJson(near.description) : '';
if (near && explicit.isExplicit(near.description)) return null; // never a moment of sexual content (server/ai/explicit.js)
if (near && explicit.isExplicit(near.description)) return null; // never a moment of sexual content (server/ai/explicit.js)
- return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.transcript, offset), tags: _memTags(near) };
+ return _isEmptyScene(desc) ? null : { offset, title: _fallbackTitle(vod, offset, desc), desc, speech: _speechNear(ctx.said || ctx.transcript, offset), tags: _memTags(near), evidence: _evidenceAt(ctx, offset, candidates[0].by) };
}
}
// True when a description says the frame is essentially empty — black/dark/loading/blank — so we
// True when a description says the frame is essentially empty — black/dark/loading/blank — so we
// never turn "nothing" into a clip or paste.
// never turn "nothing" into a clip or paste.
@@ -537,7 +583,7 @@ async function tick(opts = {}) {
@@ -537,7 +583,7 @@ async function tick(opts = {}) {
try {
try {
const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });
const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });
if (second && !await registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title, speech: second.speech })) {
if (second && !await registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title, speech: second.speech })) {
- const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, archival: v.archival, source: momentSource });
+ const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, speech: second.speech, evidence: second.evidence, archival: v.archival, source: momentSource });
if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);
if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);
} else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);
} else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);
} catch { /* clipping is best-effort */ }
} catch { /* clipping is best-effort */ }
@@ -566,7 +612,7 @@ function start() {
@@ -566,7 +612,7 @@ function start() {
setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);
setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);
}
}
-module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle } };
+module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug, publicTitle: _publicTitle, _internals: { momentPool: _momentPool, fallbackTitle: _fallbackTitle, captionTitle: _captionTitle, evidenceAt: _evidenceAt } };
// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:
// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:
// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2
// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2
@@ -311,7 +311,7 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
@@ -311,7 +311,7 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
if (dark) { console.log(`[AutoClip] backfill skip VOD ${v.vod_id} — dark/empty frame`); newSkip.push(v.vod_id); continue; }
if (dark) { console.log(`[AutoClip] backfill skip VOD ${v.vod_id} — dark/empty frame`); newSkip.push(v.vod_id); continue; }
}
}
} catch { /* probe optional */ }
} catch { /* probe optional */ }
- const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, archival: v.archival, source });
+ const clip = await clipVodMoment({ vod: v, offset: moment.offset, title: moment.title, desc: moment.desc, speech: moment.speech, evidence: moment.evidence, archival: v.archival, source });
if (clip) { made++; if (v.archival) archiveMade++; console.log(`[AutoClip] Backfilled VOD ${v.vod_id}${v.archival ? ' (archive)' : ''} ("${String(moment.title || '').slice(0, 60)}")`); }
if (clip) { made++; if (v.archival) archiveMade++; console.log(`[AutoClip] Backfilled VOD ${v.vod_id}${v.archival ? ' (archive)' : ''} ("${String(moment.title || '').slice(0, 60)}")`); }
else if (v.archival && !clipPolicy.archiveAllowed(await _clipLog(), Date.now())) continue; // another cutter took today's archive slot: not this VOD's fault, don't skip it forever
else if (v.archival && !clipPolicy.archiveAllowed(await _clipLog(), Date.now())) continue; // another cutter took today's archive slot: not this VOD's fault, don't skip it forever
else newSkip.push(v.vod_id); // cut failed (e.g. unseekable/short) — don't retry forever
else newSkip.push(v.vod_id); // cut failed (e.g. unseekable/short) — don't retry forever
@@ -325,8 +325,28 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
@@ -325,8 +325,28 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
/**
/**
* Cut a clip around a moment the AI-moments pipeline already chose for a finished VOD.
* Cut a clip around a moment the AI-moments pipeline already chose for a finished VOD.
- * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, archival, source }
+ * @param {object} o { vod (row w/ user_id, stream_id, vod_id/id), offset, title, desc, speech, evidence, archival, source }
+ *
+ * The moment is scored before anything is cut (item 151, C08), by the same clip-score.js as the live cutter: `evidence`
+ * (ai-moments-job's evidenceAt) is the chat bucket at the moment against the stream's average, the people in it, the
+ * sounds and the words near it, and who picked it. A thin moment is not cut ("no clip beats a bad clip"), one that
+ * clears the keep bar is unlisted, and only a reacted moment (12+ messages from 3+ people) can be public. A moment with
+ * no evidence at all is not cut. The log keeps the score, bucket, picker and policy, so Bench and the audits can tell
+ * this policy from the one before.
*/
*/
+const VOD_POLICY = Object.freeze({ selector: 'vod-moment', version: '2026-11-03' });
+function rateVodMoment(evidence) {
+ if (!evidence || typeof evidence !== 'object') return { verdict: 'skip', score: 0, bucket: 'A', reasons: ['no evidence was gathered for the moment'] };
+ const sp = evidence.spike || {};
+ return clipScore.score({
+ spike: { peak: Number(sp.peak) || 0, avg: Number(sp.avg) || 0 },
+ chatters: evidence.chatters == null ? null : Number(evidence.chatters),
+ sounds: Array.isArray(evidence.sounds) ? evidence.sounds : [],
+ speechWords: Number(evidence.speechWords) || 0,
+ // The AI's pick counts as its yes; a pick from the signals alone has no AI opinion (both bars rise).
+ ai: { clip: evidence.picked_by === 'ai' ? true : null },
+ });
+}
function _sceneSig(text) {
function _sceneSig(text) {
return dedupe.shortSignature(text, 5);
return dedupe.shortSignature(text, 5);
}
}
@@ -345,9 +365,14 @@ async function _isDuplicateAutoClip(vod, offset, desc, title, speech) {
@@ -345,9 +365,14 @@ async function _isDuplicateAutoClip(vod, offset, desc, title, speech) {
async function clipVodMoment(o) {
async function clipVodMoment(o) {
let archiveHeld = false;
let archiveHeld = false;
try {
try {
- const { vod, offset, title, desc, speech, archival } = o || {};
+ const { vod, offset, title, desc, speech, archival, evidence } = o || {};
if (!vod || !(offset >= 0)) return null;
if (!vod || !(offset >= 0)) return null;
if (!await _derivationOn(vod.user_id)) return null; // the channel turned AI Moments off
if (!await _derivationOn(vod.user_id)) return null; // the channel turned AI Moments off
+ const rated = rateVodMoment(evidence);
+ if (rated.verdict === 'skip') {
+ console.log(`[AutoClip] VOD ${vod.vod_id || vod.id} @${Math.round(offset)}s not cut — score ${rated.score}: ${(rated.reasons || []).slice(-2).join('; ') || 'below the bar'}`);
+ return null;
+ }
if (require('./explicit').isExplicit(desc, title)) { // never a clip of sexual content (server/ai/explicit.js)
if (require('./explicit').isExplicit(desc, title)) { // never a clip of sexual content (server/ai/explicit.js)
console.log(`[AutoClip] skip VOD ${vod.vod_id || vod.id} — the AI describes the moment as sexual content`);
console.log(`[AutoClip] skip VOD ${vod.vod_id || vod.id} — the AI describes the moment as sexual content`);
return null;
return null;
@@ -361,7 +386,7 @@ async function clipVodMoment(o) {
@@ -361,7 +386,7 @@ async function clipVodMoment(o) {
// the clip public whatever the recording is.
// the clip public whatever the recording is.
const publication = require('./clip-publication');
const publication = require('./clip-publication');
if (!publication.mayCut(vodId)) { console.log(`[AutoClip] VOD ${vodId}: protected, not cut`); return null; }
if (!publication.mayCut(vodId)) { console.log(`[AutoClip] VOD ${vodId}: protected, not cut`); return null; }
- const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: 'public' });
+ const visibility = publication.visibilityFor({ vodVisibility: await publication.vodVisibility(vod), wanted: rated.verdict === 'public' ? 'public' : 'unlisted' });
const start = Math.max(0, Math.floor(offset) - CLIP_PRE);
const start = Math.max(0, Math.floor(offset) - CLIP_PRE);
// One archive clip per UTC day is shared with the AI moments job. Both may pass the pool
// One archive clip per UTC day is shared with the AI moments job. Both may pass the pool
// gate before either writes auto_clip_log, so claim the slot atomically before cutting and
// gate before either writes auto_clip_log, so claim the slot atomically before cutting and
@@ -381,7 +406,11 @@ async function clipVodMoment(o) {
@@ -381,7 +406,11 @@ async function clipVodMoment(o) {
});
});
if (clip) {
if (clip) {
archiveHeld = false; // the clip exists: keep the day reserved even if the log write fails
archiveHeld = false; // the clip exists: keep the day reserved even if the log write fails
- await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title) });
+ const sp = (evidence && evidence.spike) || {};
+ await _logClip({ stream_id: vod.stream_id || null, vod_id: vodId, start_time: start, title: title || 'Standout Moment', clip_id: clip.id, dedup: !!clip.deduplicated || undefined, desc: desc || '', speech, archival: archival || undefined, sig: _sceneSig(desc || title),
+ score: rated.score, visibility: rated.verdict, bucket: rated.bucket, peak: Number(sp.peak) || 0, chatters: evidence.chatters == null ? undefined : evidence.chatters,
+ picked_by: evidence.picked_by || undefined, policy: VOD_POLICY });
+ console.log(`[AutoClip] VOD ${vodId} @${Math.round(offset)}s cut (${evidence.picked_by || 'signals'}) — score ${rated.score} → ${rated.verdict}, bucket ${rated.bucket}`);
}
}
return clip || null;
return clip || null;
} catch { return null; }
} catch { return null; }
@@ -452,7 +481,7 @@ function stop() {
@@ -452,7 +481,7 @@ function stop() {
require('./clip-evaluation').stop();
require('./clip-evaluation').stop();
}
}
-module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool } };
+module.exports = { start, stop, clipVodMoment, backfillVodClips, syncAutoClipFlags, _tick, _internals: { checkLiveStream: _checkLiveStream, backfillPool: _backfillPool, rateVodMoment, VOD_POLICY } };
// CLI: force a historical backfill batch, e.g. `node server/ai/auto-clip-job.js --backfill --limit=4`
// CLI: force a historical backfill batch, e.g. `node server/ai/auto-clip-job.js --backfill --limit=4`
if (require.main === module) {
if (require.main === module) {
@@ -3,8 +3,8 @@
@@ -3,8 +3,8 @@
* clip-evaluation.js — the AI's clip picks against people's, blind, on OpenVibe.Bench (item 151, C20).
* clip-evaluation.js — the AI's clip picks against people's, blind, on OpenVibe.Bench (item 151, C20).
*
*
* Once a day, for up to MAX_PER_RUN recent recordings that have both a public clip the AI cut and a public clip a
* Once a day, for up to MAX_PER_RUN recent recordings that have both a public clip the AI cut and a public clip a
- * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY)
- * and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at
+ * person made, Live submits one live.clip round to Bench (#105): the AI's pick (policy auto-clip@AUTO_CLIP_POLICY, or
+ * the one the auto-clip log names for it: policyOf) and the person's (policy people@1), each with a 6-hour signed playback link from Media. Signed-in people vote at
* https://bench.openvibe.services/battle?kind=live.clip (Bench #106: two players, blind, no titles), and
* https://bench.openvibe.services/battle?kind=live.clip (Bench #106: two players, blind, no titles), and
* results() reads each side's accept rate back from Bench (both_bad counts against both).
* results() reads each side's accept rate back from Bench (both_bad counts against both).
*
*
@@ -66,6 +66,17 @@ async function bench(method, path, body) {
@@ -66,6 +66,17 @@ async function bench(method, path, body) {
return json;
return json;
}
}
+/**
+ * Which picker cut an AI clip, for Bench's per-policy accept rate: the auto-clip log names it on clips cut since C08
+ * (2026-11-03: VOD moments are `vod-moment@2026-11-03`); older clips and the live cutter are `auto-clip@AUTO_CLIP_POLICY`.
+ */
+function policyOf(clip, clipLog = []) {
+ const e = (Array.isArray(clipLog) ? clipLog : []).find((x) => x && x.clip_id != null && String(x.clip_id) === String(clip.id));
+ const p = e && e.policy;
+ if (p && typeof p.selector === 'string' && typeof p.version === 'string') return { selector: p.selector.slice(0, 80), version: p.version.slice(0, 80), models: [] };
+ return { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] };
+}
+
function entryOf(clip, policy, signed) {
function entryOf(clip, policy, signed) {
const start = Math.round(Number(clip.start_time) * 1000);
const start = Math.round(Number(clip.start_time) * 1000);
const len = Math.round(Number(clip.duration_seconds ?? clip.duration ?? (Number(clip.end_time) - Number(clip.start_time))) * 1000);
const len = Math.round(Number(clip.duration_seconds ?? clip.duration ?? (Number(clip.end_time) - Number(clip.start_time))) * 1000);
@@ -85,6 +96,8 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
@@ -85,6 +96,8 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
if (!ai || !Array.isArray(ai.clips)) return { submitted: [], skipped: 0, why: 'Media did not answer' };
if (!ai || !Array.isArray(ai.clips)) return { submitted: [], skipped: 0, why: 'Media did not answer' };
let done;
let done;
try { done = new Set((await _log()).map((x) => String(x.vod_id))); } catch (e) { return { submitted: [], skipped: 0, why: `state unreadable: ${e.message}` }; }
try { done = new Set((await _log()).map((x) => String(x.vod_id))); } catch (e) { return { submitted: [], skipped: 0, why: `state unreadable: ${e.message}` }; }
+ let clipLog = [];
+ try { clipLog = JSON.parse(await db.getState('auto_clip_log') || '[]'); } catch { clipLog = []; }
const byVod = new Map();
const byVod = new Map();
for (const c of ai.clips) if (c.auto_generated === true && eligible(c) && !done.has(String(c.vod_id)) && !byVod.has(String(c.vod_id))) byVod.set(String(c.vod_id), c);
for (const c of ai.clips) if (c.auto_generated === true && eligible(c) && !done.has(String(c.vod_id)) && !byVod.has(String(c.vod_id))) byVod.set(String(c.vod_id), c);
const submitted = [];
const submitted = [];
@@ -106,7 +119,7 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
@@ -106,7 +119,7 @@ async function run({ now = Date.now(), max = MAX_PER_RUN } = {}) {
const task = await bench('POST', '/api/v1/kinds/live.clip/tasks', { prompt: 'Two clips of one stream', public: false });
const task = await bench('POST', '/api/v1/kinds/live.clip/tasks', { prompt: 'Two clips of one stream', public: false });
const round = await bench('POST', `/api/v1/tasks/${task.task.id}/rounds`, {});
const round = await bench('POST', `/api/v1/tasks/${task.task.id}/rounds`, {});
const roundId = round.round.id;
const roundId = round.round.id;
- await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, { selector: 'auto-clip', version: AUTO_CLIP_POLICY, models: [] }, sa) });
+ await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(aiClip, policyOf(aiClip, clipLog), sa) });
await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(person, { selector: 'people', version: '1', models: [] }, sp) });
await bench('POST', `/api/v1/rounds/${roundId}/entries`, { payload: entryOf(person, { selector: 'people', version: '1', models: [] }, sp) });
const log = await _log();
const log = await _log();
const mine = log.find((x) => String(x.vod_id) === vodId && x.round_id == null);
const mine = log.find((x) => String(x.vod_id) === vodId && x.round_id == null);
@@ -136,4 +149,4 @@ function start() {
@@ -136,4 +149,4 @@ function start() {
}
}
function stop() { if (_stop) { _stop(); _stop = null; } }
function stop() { if (_stop) { _stop(); _stop = null; } }
-module.exports = { run, results, start, stop, eligible, entryOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };
+module.exports = { run, results, start, stop, eligible, entryOf, policyOf, AUTO_CLIP_POLICY, LOG, MAX_PER_RUN };
@@ -500,7 +500,7 @@ function recentChatters(streamId, sinceSec = 30, limit = 200) {
@@ -500,7 +500,7 @@ function recentChatters(streamId, sinceSec = 30, limit = 200) {
}
}
/** The busiest `bucketSec` time-buckets of a stream between `sinceMs` and `untilMs` (default now), as offsets from
/** The busiest `bucketSec` time-buckets of a stream between `sinceMs` and `untilMs` (default now), as offsets from
- * `sinceMs`: [{ offset, count }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to
+ * `sinceMs`: [{ offset, count, people }]. Chat answers at most 10,000 buckets, so a stream that ended gives its end: up to
* now, a stream from two months ago asked for 184,000 and got a 400 (2026-10-23). */
* now, a stream from two months ago asked for 184,000 and got a 400 (2026-10-23). */
function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
const id = Number(streamId) || 0;
const id = Number(streamId) || 0;
@@ -512,7 +512,8 @@ function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
@@ -512,7 +512,8 @@ function spikeOffsets(streamId, bucketSec = 30, topN = 8, sinceMs, untilMs) {
if (!out || !Array.isArray(out.buckets)) throw unavailable('timeline');
if (!out || !Array.isArray(out.buckets)) throw unavailable('timeline');
const base = sinceMs || 0;
const base = sinceMs || 0;
return out.buckets
return out.buckets
- .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0 }))
+ // people: how many different chatters were in the bucket, when Chat says (null when it does not).
+ .map((b) => ({ offset: Math.max(0, Math.round((Number(b.t) - base) / 1000 / bucketSec) * bucketSec), count: Number(b.count) || 0, people: b.people == null ? null : Number(b.people) || 0 }))
.sort((a, b) => b.count - a.count || a.offset - b.offset)
.sort((a, b) => b.count - a.count || a.offset - b.offset)
.slice(0, topN);
.slice(0, topN);
}, () => []);
}, () => []);
@@ -218,9 +218,13 @@ async function streamClips(streamId, vodId) {
@@ -218,9 +218,13 @@ async function streamClips(streamId, vodId) {
return [...byId.values()];
return [...byId.values()];
}
}
-/** Seconds into the VOD where viewers clipped (the AI moment picker's strongest signal). */
+/**
+ * Seconds into the VOD where people clipped (the AI moment picker's strongest signal). The AI's own clips are left out
+ * (item 151, C08): they are not people choosing a moment, and counting them made every AI clip a reason for the next.
+ */
async function clipStartTimes(streamId, vodId) {
async function clipStartTimes(streamId, vodId) {
return (await streamClips(streamId, vodId))
return (await streamClips(streamId, vodId))
+ .filter((c) => c.auto_generated !== true)
.map((c) => Number(c.start_time)).filter((t) => t > 0).map(Math.floor).sort((a, b) => a - b);
.map((c) => Number(c.start_time)).filter((t) => t > 0).map(Math.floor).sort((a, b) => a - b);
}
}
@@ -148,10 +148,10 @@ async function check(name, fn) {
@@ -148,10 +148,10 @@ async function check(name, fn) {
const pool = await autoClip._internals.backfillPool(5);
const pool = await autoClip._internals.backfillPool(5);
assert.deepStrictEqual(pool.map((v) => v.vod_id), [10], 'the VOD backfill pool leaves the opted-out channel out');
assert.deepStrictEqual(pool.map((v) => v.vod_id), [10], 'the VOD backfill pool leaves the opted-out channel out');
created.length = 0;
created.length = 0;
- const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment' });
+ const none = await autoClip.clipVodMoment({ vod: { vod_id: 11, user_id: 4, stream_id: sOlive }, offset: 300, title: 'Olive moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
assert.strictEqual(none, null);
assert.strictEqual(none, null);
assert.strictEqual(created.length, 0);
assert.strictEqual(created.length, 0);
- const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment' });
+ const clip = await autoClip.clipVodMoment({ vod: { vod_id: 10, user_id: 3, stream_id: sAlice }, offset: 300, title: 'Alice moment', evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
assert.ok(clip && clip.id, 'Alice still gets auto-clips');
assert.ok(clip && clip.id, 'Alice still gets auto-clips');
assert.strictEqual(created.length, 1);
assert.strictEqual(created.length, 1);
assert.strictEqual(created[0].auto_generated, true);
assert.strictEqual(created[0].auto_generated, true);
@@ -15,7 +15,8 @@ const chatReads = require('../server/chat/chat-reads');
@@ -15,7 +15,8 @@ const chatReads = require('../server/chat/chat-reads');
const started = Date.parse('2026-08-20T12:00:00Z');
const started = Date.parse('2026-08-20T12:00:00Z');
const ended = started + 2 * 3600 * 1000;
const ended = started + 2 * 3600 * 1000;
const spikes = await chatReads.spikeOffsets(2125, 30, 8, started, ended);
const spikes = await chatReads.spikeOffsets(2125, 30, 8, started, ended);
- assert.deepStrictEqual(spikes, [{ offset: 30, count: 9 }, { offset: 60, count: 3 }]);
+ // people is null when Chat does not count chatters per bucket (C08 reads it as "unknown").
+ assert.deepStrictEqual(spikes, [{ offset: 30, count: 9, people: null }, { offset: 60, count: 3, people: null }]);
assert.deepStrictEqual(calls.pop(), { stream_id: 2125, since: started, until: ended, bucket_ms: 30000 }, 'an ended stream is read up to its end');
assert.deepStrictEqual(calls.pop(), { stream_id: 2125, since: started, until: ended, bucket_ms: 30000 }, 'an ended stream is read up to its end');
// A stream still live (no end): up to now, as before; the client leaves a null `until` out of the query.
// A stream still live (no end): up to now, as before; the client leaves a null `until` out of the query.
@@ -30,6 +31,9 @@ const chatReads = require('../server/chat/chat-reads');
@@ -30,6 +31,9 @@ const chatReads = require('../server/chat/chat-reads');
// An end before the start is ignored (asked up to now).
// An end before the start is ignored (asked up to now).
await chatReads.spikeOffsets(2128, 30, 8, started, started - 1000);
await chatReads.spikeOffsets(2128, 30, 8, started, started - 1000);
assert.strictEqual(calls.pop().until, null);
assert.strictEqual(calls.pop().until, null);
+ // When Chat counts the people in each bucket, it rides along.
+ client.readTimeline = async (q2) => ({ buckets: [{ t: q2.since + 30000, count: 14, people: 5 }] });
+ assert.deepStrictEqual(await chatReads.spikeOffsets(2129, 30, 8, started, ended), [{ offset: 30, count: 14, people: 5 }]);
console.log('chat-spike-window: ok');
console.log('chat-spike-window: ok');
process.exit(0);
process.exit(0);
})().catch((e) => { console.error(e); process.exit(1); });
})().catch((e) => { console.error(e); process.exit(1); });
@@ -248,6 +248,8 @@ const archiveCut = (id) => autoClip.clipVodMoment({
@@ -248,6 +248,8 @@ const archiveCut = (id) => autoClip.clipVodMoment({
offset: 60,
offset: 60,
title: `Archive moment ${id}`,
title: `Archive moment ${id}`,
desc: `A distinct archival scene number ${id} with enough words`,
desc: `A distinct archival scene number ${id} with enough words`,
+ // A moment worth cutting (C08: the cutter scores the evidence first).
+ evidence: { picked_by: 'ai', spike: { peak: 20, avg: 2 }, chatters: 6, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 },
archival: true,
archival: true,
});
});
const firstCut = archiveCut(70);
const firstCut = archiveCut(70);
new file mode 100644
new file mode 100644
@@ -0,0 +1,122 @@
@@ -0,0 +1,122 @@
+'use strict';
+// The VOD-moment clip cutter scores what it has before it cuts, and may abstain (item 151, C08):
+// - the AI's own clips are not "people clipped here" (lookups.clipStartTimes);
+// - a clip never re-cuts footage a person already clipped, and never comes from a long scene note alone;
+// - the AI may say nothing is worth it, and no weaker signal is tried in its place;
+// - the moment's evidence (chat bucket against the stream's zero-filled average, people, sounds, words) goes through
+// clip-score.js: thin → not cut, kept → unlisted, a reacted room → public; no evidence → not cut;
+// - the clip log and Bench name the policy that cut it.
+const assert = require('assert');
+const db = require('../server/db/database');
+console.warn = () => {};
+
+(async () => {
+ await db.initDb();
+ const media = require('../server/media-client');
+ const lookups = require('../server/media-proxy/lookups');
+ const chatReads = require('../server/chat/chat-reads');
+ const registry = require('../server/ai/moment-registry');
+ const ai = require('../server/ai/ai-analysis');
+ const aiService = require('../server/ai/ai-service');
+ const autoClip = require('../server/ai/auto-clip-job');
+ const moments = require('../server/ai/ai-moments-job');
+ const evaluation = require('../server/ai/clip-evaluation');
+ const log = []; const logs = console.log; console.log = (...a) => log.push(a.join(' '));
+
+ // ── People's clips only ───────────────────────────────────────────────
+ media.listClips = async () => ({ clips: [
+ { id: 1, start_time: 600, auto_generated: false },
+ { id: 2, start_time: 1200, auto_generated: true },
+ { id: 3, start_time: 1800 },
+ ] });
+ assert.deepStrictEqual(await lookups.clipStartTimes(91, 92), [600, 1800], 'the AI\'s clip at 1200 s is not a person choosing a moment');
+
+ // ── The score at the cutter ───────────────────────────────────────────
+ const rate = autoClip._internals.rateVodMoment;
+ assert.strictEqual(rate(undefined).verdict, 'skip', 'no evidence, no clip');
+ const quiet = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 });
+ assert.strictEqual(quiet.verdict, 'unlisted', 'a quiet channel\'s laugh the AI picked is kept: a small audience is not a poor moment');
+ assert.strictEqual(quiet.bucket, 'A');
+ const talk = rate({ picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 });
+ assert.strictEqual(talk.verdict, 'skip', 'talk the AI liked, with nothing heard or typed, is not cut: no quota to fill');
+ const room = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 });
+ assert.strictEqual(room.verdict, 'public', 'a reacted room (22 messages from 7 people) can be public');
+ const unknown = rate({ picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: null, sounds: [], speechWords: 25 });
+ assert.strictEqual(unknown.verdict, 'unlisted', 'the same spike with nobody counted is kept unlisted');
+ const signals = rate({ picked_by: 'chat', spike: { peak: 9, avg: 3 }, chatters: 2, sounds: [], speechWords: 5 });
+ assert.strictEqual(signals.verdict, 'skip', 'a small spike with no AI opinion is not cut (both bars rise without the AI)');
+
+ // ── clipVodMoment: the verdict is the visibility, and the log keeps the provenance ──
+ const created = [];
+ media.createClip = async (body) => { created.push(body); return { id: 9100 + created.length }; };
+ media.getVod = async () => ({ id: 1, visibility: 'public' });
+ await db.setState('auto_clip_log', '[]');
+ await registry._clear();
+ const vod = (id) => ({ vod_id: id, id, user_id: 3, stream_id: id, visibility: 'public' });
+ assert.strictEqual(await autoClip.clipVodMoment({ vod: vod(501), offset: 300, title: 'Just talking', desc: 'A person talks to the camera about lunch', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [], speechWords: 30 } }), null);
+ assert.strictEqual(created.length, 0, 'the thin moment never reached Media');
+ assert.ok(log.some((l) => /VOD 501 @300s not cut — score/.test(l)), 'the abstention is logged with its score');
+ const c1 = await autoClip.clipVodMoment({ vod: vod(502), offset: 300, title: 'The room lost it', desc: 'The streamer trips over the cable mid-sentence', evidence: { picked_by: 'ai', spike: { peak: 22, avg: 1.5 }, chatters: 7, sounds: [], speechWords: 25 } });
+ assert.ok(c1 && c1.id);
+ const c2 = await autoClip.clipVodMoment({ vod: vod(503), offset: 300, title: 'A laugh', desc: 'A dog barks and the streamer laughs at the window', evidence: { picked_by: 'ai', spike: { peak: 0, avg: 0 }, chatters: null, sounds: [{ label: 'Laughter', confidence: 0.8 }], speechWords: 30 } });
+ assert.ok(c2 && c2.id);
+ assert.deepStrictEqual(created.map((b) => b.visibility), ['public', 'unlisted'], 'public only for the reacted room');
+ const entries = JSON.parse(await db.getState('auto_clip_log'));
+ const e1 = entries.find((e) => e.clip_id === c1.id);
+ assert.deepStrictEqual(e1.policy, { selector: 'vod-moment', version: '2026-11-03' });
+ assert.strictEqual(e1.picked_by, 'ai');
+ assert.strictEqual(e1.bucket, 'D');
+ assert.ok(e1.score >= 0.7);
+
+ // ── Bench names the policy that cut the clip ──
+ assert.deepStrictEqual(evaluation.policyOf({ id: c1.id }, entries), { selector: 'vod-moment', version: '2026-11-03', models: [] });
+ assert.deepStrictEqual(evaluation.policyOf({ id: 4242 }, entries), { selector: 'auto-clip', version: evaluation.AUTO_CLIP_POLICY, models: [] }, 'an older clip keeps the old label');
+
+ // ── The picker, AI off: a clip needs something people or the room did ──
+ ai.isEnabled = async () => false;
+ await db.run(`INSERT INTO users (id, username, display_name, email, password_hash, role, created_at) OVERRIDING SYSTEM VALUE
+ VALUES (3, 'eve', 'eve', 'eve@x', 'x', 'streamer', '2025-01-01 00:00:00') ON CONFLICT DO NOTHING`);
+ const sid = (await db.get(`INSERT INTO streams (user_id, title, started_at, ended_at) VALUES (3, 'evidence', '2026-11-01 10:00:00', '2026-11-01 12:00:00') RETURNING id`)).id;
+ db.getStreamMemories = async () => [
+ { offset_seconds: 400, description: 'A very long and detailed scene note about the streamer sitting at a desk with a lamp and a plant and a poster' },
+ { offset_seconds: 1500, description: 'The streamer laughs at a clip on the second monitor' },
+ { offset_seconds: 3000, description: 'The streamer reads chat' },
+ ];
+ db.getStreamTranscriptSegments = async () => [{ start: 1495, end: 1505, text: 'oh no that is the funniest thing I have seen all week honestly' }];
+ db.getTimeline = async () => [];
+ chatReads.spikeOffsets = async () => [];
+ chatReads.streamStats = async () => ({ messages: 240, chatters: 9, sounds: 0 });
+ media.listClips = async () => ({ clips: [{ id: 11, start_time: 3000, auto_generated: false }] });
+ const v = { vod_id: 600, stream_id: sid, duration: 7200, title: 'Evidence' };
+ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null,
+ 'scene notes alone, and the second a person already clipped, give no clip');
+ const paste = await moments.findBestMoment(v, { flavor: 'paste' });
+ assert.ok(paste && paste.offset === 3000 && paste.evidence.picked_by === 'people', 'a paste may still use the second people chose');
+
+ db.getTimeline = async () => [{ start_sec: 1500, end_sec: 1503, label: 'Laughter', confidence: 0.85 }];
+ chatReads.spikeOffsets = async () => [{ offset: 1500, count: 14, people: 5 }];
+ const clip = await moments.findBestMoment(v, { flavor: 'clip' });
+ assert.ok(clip, 'a laugh and a chat spike make a clip moment');
+ assert.strictEqual(clip.offset, 1500);
+ assert.strictEqual(clip.evidence.spike.peak, 14);
+ assert.strictEqual(clip.evidence.chatters, 5);
+ assert.ok(clip.evidence.spike.avg > 0.9 && clip.evidence.spike.avg < 1.1, 'the average counts the quiet stretches: 240 messages over 240 buckets');
+ assert.ok(clip.evidence.speechWords >= 10, 'the words come from the whole transcript near the moment, not a sample');
+ assert.strictEqual(clip.evidence.sounds[0].label, 'Laughter');
+
+ // ── The AI may abstain: no weaker signal is tried in its place ──
+ ai.isEnabled = async () => true; ai.withinBudget = async () => true;
+ aiService.structured = async () => ({ abstain: true, why: 'nothing stands out' });
+ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 'the AI said nothing is worth it');
+ aiService.structured = async () => ({ t: null });
+ assert.strictEqual(await moments.findBestMoment(v, { flavor: 'clip' }), null, 't: null is an abstention too');
+ let asked = null;
+ aiService.structured = async (_name, input) => { asked = input; return { t: 1500, title: 'Lost it', desc: 'The streamer laughs at a clip on the second monitor' }; };
+ const picked = await moments.findBestMoment(v, { flavor: 'clip' });
+ assert.ok(picked && picked.evidence.picked_by === 'ai');
+ assert.deepStrictEqual(asked.clipped, [], 'a clip pick is not steered to the second people already clipped');
+ assert.ok(asked.avoid.includes(3000), '…it is told to stay away from it');
+
+ console.log = logs;
+ console.log('vod moment evidence: all checks passed');
+})().catch((e) => { console.error(e); process.exit(1); });