openvibers/openvibe.live Commits
AI content: shared moment registry, distinct clips vs pastes, live pastes, AI-inferred category
Changes
@@@ -52,3 +52,19 @@ From the dashboard:## Chat ReplayVODs include synchronized chat replay. Messages are stored in the database with timestamps relative to the stream start. Deleted messages are automatically excluded from replay (soft-delete with `is_deleted` flag).++++## AI moments: pastes and clips never overlap++++Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image **pastes** for the home hero + pastes tab, every 6 h) and `server/ai/auto-clip-job.js` (live chat-spike **clips** + a VOD backfill). They used to pick the same second of the same VOD with the same title. Now:++++- **One shared registry** — `server/ai/moment-registry.js` (`ai_used_moments` state) records every paste/clip (`vod_id`, `stream_id`, `offset`, scene signature, title). Both jobs ask it before creating anything: a moment is refused if it is within **2 min** of a used moment on the same VOD/stream, or if its scene signature (first five words of the description) was used in the last two weeks. Legacy logs are imported once.++- **Flavoured picks** — `findBestMoment(vod, { flavor, avoid })`: `paste` asks for a frame that is striking on its own (a face, a gag, something odd on screen); `clip` asks for a beat that plays out over 25 s (a line, a reaction, a sound, chat exploding). The prompt lists the already-used timestamps and the model is told to stay away from them; if it still lands next to one, the objective signals (viewer clips, chat spikes, richest scene notes) pick a free spot.++- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.++- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.++- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).++++## AI-inferred category++++Go-live used to default every stream to `irl`, so every AI prompt (AI viewers, streamer overviews, VOD overviews, the Arena, sound detection) assumed everyone is an IRL streamer. Now the category selector defaults to **Auto** (stored as `NULL`), and the stream-memory rollup — the same summarise call that already runs — returns `{ overview, category, tags }` with `category` from the fixed taxonomy (`outdoors, travel, building, music, gaming, robot, desktop, irl, other`). It is stored on `streams.ai_category` / `streams.ai_tags` and the channel inherits the latest read (`channels.ai_category`).++++Everything reads the effective value: listing queries return `COALESCE(ai_category, category) AS category` (the self-selected value is still there as `chosen_category`), `stream_category` on VODs/clips, the AI viewers' session block ("judged from the stream itself"), streamer overviews, sound detection and the Arena all prefer `ai_category`. A streamer can still pick a category manually; the AI read wins once it exists.@@@ -804,7 +804,7 @@<h2 id="ch-display-name">Channel</h2><span class="ch-staff-badge" id="ch-staff-badge"></span><span class="ch-username" id="ch-username">username</span>-- <span class="ch-category-badge badge" id="ch-category-badge">irl</span>++ <span class="ch-category-badge badge" id="ch-category-badge">Live</span><span id="ch-protocol-badge"></span></div><p class="ch-stream-title" id="ch-stream-title">Stream Title</p>@@@ -934,7 +934,7 @@<p class="ch-description" id="ch-description-offline"></p><div class="ch-meta"><span class="ch-follower-count" id="ch-follower-count-offline">0 followers</span>-- <span class="ch-category-badge badge" id="ch-category-badge-offline">irl</span>++ <span class="ch-category-badge badge" id="ch-category-badge-offline">Offline</span></div><div class="ch-weather-widget" id="ch-weather-widget-offline" style="display:none"></div></div>@@@ -3467,7 +3467,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionI_activeChannelIsOwnerRank = !!ch.is_owner;const _chUser = document.getElementById('ch-username');if (_chUser) _chUser.style.display = 'none';-- document.getElementById('ch-category-badge').textContent = _capTag((liveStreams[0] && liveStreams[0].category) || ch.category || 'irl');++ document.getElementById('ch-category-badge').textContent = _capTag((liveStreams[0] && liveStreams[0].category) || ch.ai_category || ch.category || 'Live');document.getElementById('ch-follower-count').textContent = `${ch.follower_count || 0} followers`;setupFollowBtn(document.getElementById('ch-btn-follow'));setupBanBtn(document.getElementById('ch-btn-ban'));@@@ -3528,7 +3528,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionIdocument.getElementById('ch-username-offline').textContent = '@' + ch.username;document.getElementById('ch-description-offline').textContent = ch.description || '';document.getElementById('ch-follower-count-offline').textContent = `${ch.follower_count || 0} followers`;-- document.getElementById('ch-category-badge-offline').textContent = _capTag(ch.category || 'irl');++ document.getElementById('ch-category-badge-offline').textContent = _capTag(ch.ai_category || ch.category || 'Offline');setupFollowBtn(document.getElementById('ch-btn-follow-offline'));setupBanBtn(document.getElementById('ch-btn-ban-offline'));@@@ -7936,7 +7936,8 @@ function createManagedStreamModal() {<div class="form-group"><label>Category</label><select id="cms-category" class="form-input">-- <option value="irl" selected>IRL</option>++ <option value="" selected>Auto — the AI decides from the stream</option>++ <option value="irl">IRL</option><option value="outdoors">Outdoors</option><option value="travel">Travel</option><option value="building">Building/Craft</option>@@@ -7984,7 +7985,7 @@ async function _cmsCreate() {const btn = document.getElementById('cms-create-btn');const errEl = document.getElementById('cms-error');const title = (document.getElementById('cms-title')?.value || '').trim() || 'Untitled Stream';-- const category = document.getElementById('cms-category')?.value || 'irl';++ const category = document.getElementById('cms-category')?.value || '';const protocol = document.getElementById('cms-protocol')?.value || 'webrtc';const streamingMethod = document.getElementById('cms-method')?.value || 'browser';const slug = (document.getElementById('cms-slug')?.value || '').trim().toLowerCase() || undefined;@@@ -421,7 +421,7 @@ function _wsRenderPanel() {<div class="form-group" style="flex:2"><label>Category</label><select id="bc-category" class="form-input" onchange="_wsMarkDirty()">-- ${_wsRenderCategoryOptions(ms.category || 'irl')}++ ${_wsRenderCategoryOptions(ms.category || '')}</select></div><div class="form-group" style="flex:1">@@@ -1391,6 +1391,7 @@ function _wsRenderBrowserModeCards(selected) {function _wsRenderCategoryOptions(selected) {const cats = [++ ['', 'Auto — the AI decides from the stream'],['outdoors', 'Outdoors'], ['travel', 'Travel'], ['building', 'Building/Craft'],['music', 'Music'], ['gaming', 'Gaming'], ['robot', 'Robot'],['desktop', 'Desktop'], ['irl', 'IRL'], ['other', 'Other'],@@@ -1959,7 +1960,7 @@ async function _wsSaveAll() {const streamFields = {title: document.getElementById('bc-title')?.value.trim() || ms.title,description: document.getElementById('bc-description')?.value.trim() || '',-- category: document.getElementById('bc-category')?.value || 'irl',++ category: document.getElementById('bc-category')?.value || '',is_nsfw: document.getElementById('bc-nsfw')?.checked ? 1 : 0,protocol: ms.protocol,streaming_method: ms.streaming_method || 'browser',@@@ -1440,7 +1440,7 @@ async function loadBroadcastControlConfigs() {async function createNewStream() {const title = document.getElementById('bc-title')?.value.trim();const description = document.getElementById('bc-description')?.value.trim() || '';-- const category = document.getElementById('bc-category')?.value || 'irl';++ const category = document.getElementById('bc-category')?.value || '';const method = broadcastState.selectedMethod;if (!title) return toast('Stream title is required', 'error');@@@ -139,13 +139,21 @@ async function analyzeTextPaste(content, title) {/** Analyze a live-stream frame → { description, tags }. */async function analyzeStreamFrame(image) {-- const prompt = `This is a frame from a live stream. Reply ONLY with compact JSON: {"description":"one concise sentence describing what is happening on screen right now","tags":["2-5","short","tags"]}.`;-- const text = await _complete({ prompt, image, maxTokens: 200, kind: 'stream_memory', role: 'vision', imageMaxWidth: 768 });++ // One vision call does three jobs: the memory description, the tags, and a screenshot-worthiness++ // verdict + caption (so live pastes need no extra call).++ const prompt = `This is a frame from a live stream. Reply ONLY with compact JSON: {"description":"one concise sentence describing what is happening on screen right now","tags":["2-5","short","tags"],"worthy":<true only if this exact frame is genuinely screenshot-worthy on its own: a face/reaction, a visual gag, something unusual or funny on screen — false for ordinary gameplay/desktop/chat/talking-head frames>,"title":"<if worthy: a punchy, funny 3-7 word caption for it, else empty>"}.`;++ const text = await _complete({ prompt, image, maxTokens: 240, kind: 'stream_memory', role: 'vision', imageMaxWidth: 768 });if (!text) return null;const description = _extractDescription(text, 400);-- return description ? { description, tags: _extractTags(text, 6) } : null;++ if (!description) return null;++ const j = _parseJson(text) || {};++ return { description, tags: _extractTags(text, 6), worthy: j.worthy === true, title: j.worthy === true ? String(j.title || '').replace(/^["'\s]+|["'\s]+$/g, '').slice(0, 80) : '' };}++/** Fixed category taxonomy the AI picks from (matches the go-live selector). */++const CATEGORIES = ['outdoors', 'travel', 'building', 'music', 'gaming', 'robot', 'desktop', 'irl', 'other'];++function normalizeCategory(c) { const t = String(c || '').toLowerCase().trim(); if (!t) return null; if (CATEGORIES.includes(t)) return t; const map = { 'just chatting': 'irl', chatting: 'irl', talk: 'irl', coding: 'desktop', programming: 'desktop', software: 'desktop', tech: 'desktop', art: 'building', craft: 'building', diy: 'building', cooking: 'irl', hiking: 'outdoors', driving: 'travel', 'road trip': 'travel', robotics: 'robot', game: 'gaming', games: 'gaming', dj: 'music' }; return map[t] || null; }++/** Condense a stream's memories into a one-line "AI Overview" for the home card. */async function summarizeStreamMemories(memories, streamId = null) {// Use observations from across the whole session (capped for token budget) so the@@@ -179,9 +187,15 @@ async function summarizeStreamMemories(memories, streamId = null) {} catch { /* timeline optional */ }}-- const prompt = `These are timestamped observations from a live stream, in order since it started. Give a thorough overview (2-6 sentences) of what this stream has been about overall — the main activities, topics, and vibe across the whole session (not just the latest moment):\n${lines}${audioBlock}`;-- const text = await _complete({ prompt, maxTokens: 500, kind: 'stream_memory' });-- return text ? text.slice(0, 2000) : null;++ // The same call also classifies the stream — category is inferred from what is actually on++ // screen / said, never from the streamer's go-live selector (which used to default to "irl"++ // and taught every downstream AI that everyone is an IRL streamer).++ const prompt = `These are timestamped observations from a live stream, in order since it started. Reply ONLY with compact JSON: {"overview":"a thorough overview (2-6 sentences) of what this stream has been about overall — the main activities, topics, and vibe across the whole session (not just the latest moment)","category":"<exactly one of: ${CATEGORIES.join(', ')} — what this stream mostly IS, judged from the observations; desktop = screen/software/coding, gaming = playing games, irl = camera on a person talking/doing things indoors, outdoors/travel = out in the world>","tags":["3-6","short","lowercase","tags"]}\n${lines}${audioBlock}`;++ const text = await _complete({ prompt, maxTokens: 560, kind: 'stream_memory' });++ if (!text) return null;++ const j = _parseJson(text);++ const overview = j && j.overview ? String(j.overview).slice(0, 2000) : (_extractDescription(text, 2000) || text.slice(0, 2000));++ return { overview, category: normalizeCategory(j && j.category), tags: j && Array.isArray(j.tags) ? j.tags.map(String).slice(0, 6) : [] };}/**@@@ -198,7 +212,7 @@ async function generateStreamerOverview(userId) {const memLines = memories.slice(0, 40).map(m => `- ${m.description}`).filter(l => l.length > 2);const vods = (db.getVodsByUser ? db.getVodsByUser(userId, false, 20, 0) : []) || [];-- const vodLines = vods.map(v => `- ${v.title || 'Untitled VOD'}${v.category ? ` [${v.category}]` : ''}`);++ const vodLines = vods.map(v => `- ${v.title || 'Untitled VOD'}${(v.ai_category || v.category) ? ` [${v.ai_category || v.category}]` : ''}`);const pastes = (db.getUserPastesForAi ? db.getUserPastesForAi(userId, 25) : []) || [];const pasteLines = pastes.filter(p => p.ai_summary).map(p => `- "${p.title || 'paste'}": ${p.ai_summary}`);@@@ -208,7 +222,7 @@ async function generateStreamerOverview(userId) {const ctx = [`Streamer: ${user.display_name || user.username} (@${user.username})`,(channel?.bio || user.bio) ? `Bio: ${(channel?.bio || user.bio).slice(0, 400)}` : '',-- channel?.category ? `Usual category: ${channel.category}` : '',++ (channel?.ai_category || channel?.category) ? `Usual category (${channel?.ai_category ? 'inferred by AI from their streams' : 'self-selected'}): ${channel.ai_category || channel.category}` : '',memLines.length ? `\nLive-stream observations (across sessions):\n${memLines.join('\n')}` : '',vodLines.length ? `\nRecent VODs:\n${vodLines.join('\n')}` : '',pasteLines.length ? `\nPaste summaries:\n${pasteLines.join('\n')}` : '',@@@ -466,6 +480,7 @@ async function testStatus({ probe = true } = {}) {}module.exports = {++ CATEGORIES, normalizeCategory,isEnabled, withinBudget, pasteAnalysisEnabled, streamMemoryEnabled, transcriptionEnabled, captureIntervalSec,analyzeImagePaste, analyzeTextPaste, analyzeStreamFrame, summarizeStreamMemories,generateStreamerOverview, generateVodOverview, generateClipOverview, ensureVodTimeline,@@@ -18,6 +18,7 @@ const fs = require('node:fs');const db = require('../db/database');const ai = require('./ai-analysis');const media = require('../media-client');++const registry = require('./moment-registry');let cfg = null; try { cfg = require('../config'); } catch { /* */ }let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');@@@ -86,7 +87,27 @@ Return STRICT JSON only, nothing else: [{"index": <n>, "score": <1-100>, "why":}// Rank the whole VOD set (batched tournament so "every VOD ever" scales). Falls back to the// objective prior order (already applied by the DB) when AI is unavailable.++// Ranking cache: a VOD's showcase score barely changes, so score each VOD ONCE (7 days) and only++// spend calls on VODs that have never been scored. Persisted in the job state as `rankCache`.++const RANK_TTL_MS = 7 * 24 * 3600_000;++let _rankCacheOut = {};async function _rankVods(vods, want) {++ if (!_aiOn() || vods.length <= 1) return vods.map(v => ({ vod: v }));++ const CH = 25;++ const cache = _load().rankCache || {};++ const now = Date.now();++ const fresh = [], stale = [];++ for (const v of vods) { const c = cache[v.vod_id]; if (c && now - (c.at || 0) < RANK_TTL_MS) fresh.push({ vod: v, score: c.score, why: c.why }); else stale.push(v); }++ let scored = [];++ for (let i = 0; i < stale.length; i += CH) scored.push(...(await _scoreVodChunk(stale.slice(i, i + CH), Math.max(want, 6))));++ for (const s of scored) cache[s.vod.vod_id] = { score: s.score, why: s.why, at: now };++ for (const v of stale) if (!cache[v.vod_id]) cache[v.vod_id] = { score: 0, why: 'unscored', at: now }; // never re-spend on a VOD the model left out++ _rankCacheOut = Object.fromEntries(Object.entries(cache).filter(([, c]) => now - (c.at || 0) < RANK_TTL_MS).slice(-600));++ const all = [...fresh, ...scored].sort((a, b) => b.score - a.score);++ if (all.length) { console.log(`[AI-Moments] ranked ${vods.length} VOD(s): ${fresh.length} from cache, ${stale.length} scored now`); return all; }++ return vods.map(v => ({ vod: v }));++}++async function _rankVodsUncached(vods, want) {if (!_aiOn() || vods.length <= 1) return vods.map(v => ({ vod: v }));const CH = 25;if (vods.length <= CH) {@@@ -134,9 +155,17 @@ function _nearestMemory(memories, offset) {for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }return best;}--async function _findBestMoment(vod) {++// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.++// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.++const FLAVOR_HINT = {++ paste: 'This moment becomes a SCREENSHOT paste: pick something visually striking in a single frame — a face or reaction, a visual gag, something odd on screen, a scene change. What was said matters less than what is SEEN.',++ clip: 'This moment becomes a 25-SECOND VIDEO CLIP: pick a beat that plays out over time — a line that lands, a reaction, a sound, chat exploding, something HAPPENING — not a static pretty frame.',++};++async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {const dur = Math.floor(vod.duration || 0);const ctx = _momentContext(vod.stream_id, vod.vod_id);++ const avoidList = [...new Set([...(avoid || []), ...registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);++ const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);if (!ctx.memories.length && !ctx.transcript.length) return null;const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));@@@ -160,7 +189,10 @@ ${soundLine || '(none)'}Viewers CLIPPED these timestamps (very strong "this was a highlight" signal): ${clipHint}Chat activity SPIKED around: ${spikeHint}--Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in this VOD with something clearly VISIBLE happening. Prefer moments backed by the clip/chat signals when they line up with something notable. NEVER pick a black/dark/loading/blank screen, an intro/BRB card, or a moment with no visible content or activity. Return STRICT JSON only, nothing else: {"t": <seconds into the vod>, "title": "<specific punchy 3-8 word title, not the stream name>", "desc": "<one vivid sentence describing the moment>"}`;++${FLAVOR_HINT[flavor] || FLAVOR_HINT.paste}++${avoidList.length ? `ALREADY USED (a paste or clip exists there) — do NOT pick these or anything within 2 minutes of them: ${avoidList.map(_mmss).join(', ')}. Find a DIFFERENT moment.` : ''}++++Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in this VOD with something clearly VISIBLE happening. Prefer moments backed by the clip/chat signals when they line up with something notable. NEVER pick a black/dark/loading/blank screen, an intro/BRB card, or a moment with no visible content or activity. Return STRICT JSON only, nothing else: {"t": <seconds into the vod>, "title": "<specific punchy 3-8 word title, not the stream name, funny, never generic>", "desc": "<one vivid sentence describing the moment>"}`;try {const text = await ai.summarizeText(prompt, 300, 'moment_pick');const m = text && text.match(/\{[\s\S]*\}/);@@@ -169,6 +201,7 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in thlet t = Number(j.t);if (!isNaN(t)) {t = clamp(t);++ if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }const near = _nearestMemory(ctx.memories, t);const result = { offset: t, title: _cleanTitle(j.title) || _titleFromDesc(j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', tags: _memTags(near) };return _isEmptyScene(result.desc) ? null : result;@@@ -177,15 +210,15 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in th} catch { /* fall through to signal-based pick */ }}-- // No-AI fallback: pick from the strongest objective signal.-- let offset = null;-- if (ctx.clipTimes.length) offset = ctx.clipTimes[Math.floor(ctx.clipTimes.length / 2)]; // where viewers clipped-- else if (ctx.spikes.length) offset = ctx.spikes[0].offset; // busiest chat moment-- else { // richest scene note-- const rich = ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length)[0];-- offset = rich ? (rich.offset_seconds || 0) : 0;-- }-- offset = clamp(offset);++ // No-AI fallback (also used when the AI pick collided with a used moment): the strongest++ // objective signal that is not next to something already made.++ const candidates = [++ ...ctx.clipTimes.map(t => ({ t, w: 3 })), // where viewers clipped++ ...ctx.spikes.map(s => ({ t: s.offset, w: 2 })), // busiest chat moments++ ...ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1 })), // richest scene notes++ ].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);++ if (!candidates.length) return null;++ let offset = clamp(candidates[0].t);const near = _nearestMemory(ctx.memories, offset);const desc = near ? _deJson(near.description) : '';return _isEmptyScene(desc) ? null : { offset, title: _titleFromDesc(desc), desc, tags: _memTags(near) };@@@ -274,8 +307,7 @@ async function tick(opts = {}) {// Content-signature dedup: even different VODs from the same streamer (same setup) yield// near-identical scenes → we skip a moment whose description signature was used recently,// so the hero/pastes never show visual duplicates. Respected even on --fresh.-- const usedSigs = new Set(prev.usedSigs || []);-- const thisSigs = new Set();++ const thisSigs = new Set(); // scenes picked in THIS run (cross-run dedupe lives in moment-registry)// Stage 1: consider every eligible VOD, ranked by its AI overview + popularity prior.// The VOD pool comes from OpenVibe.Media (most-viewed public VODs), overlaid with@@@ -316,10 +348,11 @@ async function tick(opts = {}) {// Stage 2: find the best moment within this VOD.const moment = await _findBestMoment(v);if (!moment) continue;-- // Skip near-duplicate scenes (same signature as a recent or already-picked moment).++ // Skip near-duplicate scenes / spots — the registry is shared with the clip job.const sig = _sig(moment.desc || v.ai_overview_short || v.title);-- if (sig && (usedSigs.has(sig) || thisSigs.has(sig))) {-- console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — duplicate scene ("${sig}")`);++ const why0 = registry.usedReason({ vod_id: v.vod_id, stream_id: v.stream_id, offset: moment.offset, sig });++ if (why0 || (sig && thisSigs.has(sig))) {++ console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — ${why0 || `duplicate scene ("${sig}")`}`);continue;}if (sig) thisSigs.add(sig);@@@ -390,7 +423,7 @@ async function tick(opts = {}) {// Re-check the signature against the FINAL (vision-verified) description — this is// what actually catches same-scene duplicates (e.g. a split-image close-up re-picked).const finalSig = _sig(desc);-- if (finalSig && finalSig !== sig && (usedSigs.has(finalSig) || thisSigs.has(finalSig))) {++ if (finalSig && finalSig !== sig && (registry.isUsed({ sig: finalSig }) || thisSigs.has(finalSig))) {console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — duplicate scene after vision ("${finalSig}")`);continue;}@@@ -423,19 +456,23 @@ async function tick(opts = {}) {try { fs.unlinkSync(screenshotPath); } catch { /* tmp frame */ }moments.push({ vodId: v.vod_id, offset, title: title.slice(0, 80), thumbnail: img, username: v.username, pasteSlug });++ registry.record({ kind: 'paste', vod_id: v.vod_id, stream_id: v.stream_id, offset, sig: finalSig || sig, title });-- // Also cut a real VOD clip around this same moment (AI auto-clip), unless disabled.++ // Also cut a VOD clip from the SAME VOD — but a DIFFERENT beat (clip flavor, ≥ 2 min away),++ // so the clip and the paste never show the identical second with the identical title.if (opts.clip !== false && momentSource) {try {-- const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset, title, desc, source: momentSource });-- if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} ("${title.slice(0, 60)}")`);++ const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });++ if (second && !registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title })) {++ const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, source: momentSource });++ if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);++ } else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);} catch { /* clipping is best-effort */ }}}const usedLog = [...newUsedVods, ...(prev.usedVods || [])].slice(0, 300);-- const sigLog = [...thisSigs, ...(prev.usedSigs || [])].slice(0, 120);-- db.setState(SETTING, JSON.stringify({ moments, usedVods: usedLog, usedSigs: sigLog, updated_at: Date.now() }));++ db.setState(SETTING, JSON.stringify({ moments, usedVods: usedLog, rankCache: _rankCacheOut, updated_at: Date.now() }));console.log(`[AI-Moments] ${moments.length} moment(s) from ${chosen.length}/${totalEligible} ranked VODs + pastes created`);} catch (e) {console.warn('[AI-Moments] tick error:', e.message);@@@ -456,7 +493,7 @@ function start() {setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);}--module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark };++module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug };// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2@@@ -19,6 +19,7 @@ const db = require('../db/database');const ai = require('./ai-analysis');const media = require('../media-client');const recorder = require('../streaming/recorder');++const registry = require('./moment-registry');// ── Tunables ────────────────────────────────────────────────const CHECK_INTERVAL_MS = 90 * 1000;@@@ -49,6 +50,8 @@ function _logClip(entry) {const log = _clipLog();log.unshift({ ...entry, ts: Date.now() });db.setState(CLIP_LOG_SETTING, JSON.stringify(log.slice(0, 300)));++ // Shared with the paste job, so a paste never lands on a clipped second (and vice versa).++ registry.record({ kind: 'clip', vod_id: entry.vod_id || null, stream_id: entry.stream_id || null, offset: (entry.start_time || 0) + CLIP_PRE, sig: entry.sig || null, title: entry.title || null });} catch { /* */ }}function _countClipsSince(streamId, minutes) {@@@ -241,7 +244,8 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}try {const source = await _resolveVodSource(v.vod_id);if (!source) { newSkip.push(v.vod_id); continue; } // media gone / unreadable-- const moment = await moments.findBestMoment(v);++ // Clip flavor (a beat, not a frame) and never within 2 min of a moment a paste already used.++ const moment = await moments.findBestMoment(v, { flavor: 'clip', avoid: registry.usedOffsets(v.vod_id, v.stream_id) });if (!moment) { newSkip.push(v.vod_id); continue; }// Pixel backstop: don't clip a black/empty frame.try {@@@ -276,14 +280,8 @@ function _sceneSig(text) {function _isDuplicateAutoClip(vod, offset, desc, title) {try {const vId = vod.vod_id || vod.id;-- const sig = _sceneSig(desc || title);-- const start = Math.max(0, Math.floor(offset) - CLIP_PRE);-- const cutoff = Date.now() - 48 * 3600_000;-- for (const c of _clipLog()) {-- if (c.ts < cutoff) continue;-- if ((c.vod_id === vId || c.stream_id === vod.stream_id) && Math.abs((c.start_time || 0) - start) < 45) return true;-- if (sig && c.sig === sig) return true;-- }++ const why = registry.usedReason({ vod_id: vId, stream_id: vod.stream_id, offset: Math.floor(offset), sig: _sceneSig(desc || title) });++ if (why) { console.log(`[AutoClip] VOD ${vId}: ${why}`); return true; }} catch { /* */ }return false;}@@@ -65,7 +65,8 @@ function sessionBlock(stream) {if (!stream) return '';const parts = [];const title = stream.ai_title || stream.title;-- if (title) parts.push(`Stream title: "${clip(title, 140)}"${stream.category ? ` (${clip(stream.category, 40)})` : ''}`);++ const cat = stream.ai_category || stream.category;++ if (title) parts.push(`Stream title: "${clip(title, 140)}"${cat ? ` (category: ${clip(cat, 40)}${stream.ai_category ? ', judged from the stream itself' : ', self-selected'})` : ''}`);if (stream.ai_overview) parts.push(`What this session has been about so far: ${clip(stream.ai_overview, 500)}`);return parts.join('\n');}nnew file mode 100644@@@ -0,0 +1,79 @@++/**++ * moment-registry.js — ONE shared record of every stream moment the AI has already turned into++ * something (a paste, a clip), so the paste job and the clip job never pick the same spot or the++ * same scene twice. Replaces the two separate logs (`home_hero_moments.usedSigs` and++ * `auto_clip_log[].sig`) that could not see each other — which is how a paste and a clip ended++ * up with the identical title from the identical second of the same VOD.++ *++ * record({ kind, vod_id, stream_id, offset, sig, title }) after a paste/clip is created++ * isUsed({ vod_id, stream_id, offset, sig }) same VOD within GAP_SEC, or same scene++ * usedOffsets(vod_id, stream_id) → [sec…] to tell the picker to avoid++ *++ * State key `ai_used_moments` (capped). Legacy logs are imported once on first use.++ */++'use strict';++++const db = require('../db/database');++++const KEY = 'ai_used_moments';++const MAX = 800;++const GAP_SEC = 120; // two moments on the same VOD must be at least this far apart++const SIG_TTL_MS = 14 * 24 * 3600_000; // a scene signature blocks re-use for two weeks++const OFFSET_TTL_MS = 90 * 24 * 3600_000;++++let _cache = null;++function _load() {++ if (_cache) return _cache;++ let list = [];++ try { const l = JSON.parse(db.getState(KEY) || '[]'); if (Array.isArray(l)) list = l; } catch { /* */ }++ if (!list.length) list = _importLegacy();++ _cache = list;++ return list;++}++function _save(list) { _cache = list.slice(0, MAX); try { db.setState(KEY, JSON.stringify(_cache)); } catch { /* */ } }++++function _importLegacy() {++ const out = [];++ try { for (const c of JSON.parse(db.getState('auto_clip_log') || '[]') || []) out.push({ kind: 'clip', vod_id: c.vod_id || null, stream_id: c.stream_id || null, offset: (c.start_time || 0) + 16, sig: c.sig || null, title: c.title || null, ts: c.ts || Date.now() }); } catch { /* */ }++ try { const h = JSON.parse(db.getState('home_hero_moments') || '{}') || {}; for (const m of h.moments || []) out.push({ kind: 'paste', vod_id: m.vodId || null, stream_id: null, offset: m.offset || 0, sig: sig(m.title), title: m.title || null, ts: h.updated_at || Date.now() }); for (const s of h.usedSigs || []) out.push({ kind: 'paste', vod_id: null, stream_id: null, offset: null, sig: s, title: null, ts: h.updated_at || Date.now() }); } catch { /* */ }++ return out.slice(0, MAX);++}++++/** Coarse scene signature — the first five significant words of a description/title. */++function sig(text) {++ return String(text || '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').replace(/\s+/g, ' ').trim().split(' ').filter(Boolean).slice(0, 5).join(' ');++}++++function record({ kind, vod_id = null, stream_id = null, offset = null, sig: s = null, title = null, desc = null }) {++ const list = _load();++ list.unshift({ kind: kind || 'moment', vod_id: vod_id || null, stream_id: stream_id || null, offset: offset == null ? null : Math.floor(offset), sig: s || sig(desc || title), title: title ? String(title).slice(0, 120) : null, ts: Date.now() });++ _save(list);++}++++/** Why a moment is blocked, or null when it is free. */++function usedReason({ vod_id = null, stream_id = null, offset = null, sig: s = null, desc = null, title = null } = {}, { gapSec = GAP_SEC } = {}) {++ const now = Date.now();++ const scene = s || sig(desc || title);++ for (const e of _load()) {++ const age = now - (e.ts || 0);++ if (age > OFFSET_TTL_MS) continue;++ if (offset != null && e.offset != null && ((vod_id && e.vod_id === vod_id) || (stream_id && e.stream_id === stream_id)) && Math.abs(e.offset - offset) < gapSec) return `${e.kind} already made at ${Math.floor(e.offset)}s of this ${e.vod_id === vod_id ? 'VOD' : 'stream'}`;++ if (scene && e.sig && e.sig === scene && age <= SIG_TTL_MS) return `same scene as a recent ${e.kind} ("${scene}")`;++ }++ return null;++}++function isUsed(m, o) { return !!usedReason(m, o); }++++/** Offsets already used on this VOD/stream (for the picker's "avoid these" list). */++function usedOffsets(vod_id = null, stream_id = null) {++ const now = Date.now();++ return [...new Set(_load().filter(e => e.offset != null && now - (e.ts || 0) <= OFFSET_TTL_MS && ((vod_id && e.vod_id === vod_id) || (stream_id && e.stream_id === stream_id))).map(e => Math.floor(e.offset)))].sort((a, b) => a - b);++}++++function lastOfKind(kind, { stream_id = null, vod_id = null } = {}) {++ return _load().find(e => e.kind === kind && (!stream_id || e.stream_id === stream_id) && (!vod_id || e.vod_id === vod_id)) || null;++}++++function recent(hours = 48) { const cut = Date.now() - hours * 3600_000; return _load().filter(e => (e.ts || 0) >= cut); }++++module.exports = { record, isUsed, usedReason, usedOffsets, lastOfKind, recent, sig, GAP_SEC, _reset: () => { _cache = null; } };@@@ -155,8 +155,9 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =const stale = (now - last.at) >= SUMMARY_MAX_AGE_MS;if (memories.length > 1 && (!last.at || grew || stale)) {const summary = await ai.summarizeStreamMemories(memories, stream.id);-- if (summary) {-- db.updateStreamAiOverview(stream.id, summary);++ if (summary && summary.overview) {++ db.updateStreamAiOverview(stream.id, summary.overview);++ if (summary.category) db.setStreamAiCategory(stream.id, summary.category, summary.tags);_lastSummary.set(stream.id, { count: memories.length, at: now });}} else if (!stream.ai_overview) {@@@ -165,6 +166,41 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =}} catch { /* keep existing overview */ }} catch (e) { console.warn('[AI] memory store failed:', e.message); }++++ // Live paste: when the vision model flags THIS frame as screenshot-worthy, post it as an image++ // paste right now (no extra AI call — the caption came with the memory). Rate-limited per stream++ // and deduped through the shared moment registry so the paste job / clip job never re-use it.++ try { await _maybeLivePaste(stream, image, r, offset); } catch (e) { console.warn('[AI] live paste:', e.message); }++}++++const LIVE_PASTE_MIN_GAP_MS = 90 * 60 * 1000;++async function _maybeLivePaste(stream, image, r, offset) {++ if (!r || !r.worthy || !r.title || !image) return;++ try { if (String(db.getSetting('ai_live_pastes_enabled') ?? 'true') === 'false') return; } catch { /* default on */ }++ const registry = require('./moment-registry');++ const last = registry.lastOfKind('paste', { stream_id: stream.id });++ if (last && Date.now() - (last.ts || 0) < LIVE_PASTE_MIN_GAP_MS) return;++ const why = registry.usedReason({ stream_id: stream.id, offset, desc: r.description, title: r.title });++ if (why) return;++ const moments = require('./ai-moments-job');++ if (typeof image === 'string' && !image.startsWith('data:') && moments.frameTooDark && await moments.frameTooDark(image)) return;++ let buffer = null;++ try { buffer = Buffer.isBuffer(image) ? image : (typeof image === 'string' && image.startsWith('data:') ? Buffer.from(image.split(',')[1] || '', 'base64') : require('fs').readFileSync(image)); } catch { return; }++ if (!buffer || buffer.length < 2000) return;++ const username = stream.username || (db.getUserById(stream.user_id) || {}).username || 'someone';++ let base = 'https://openvibe.live'; try { const c = require('../config'); base = String(c.baseUrl || c.publicUrl || base).replace(/\/$/, ''); } catch { /* */ }++ const media = require('../media-client');++ const paste = await media.createPaste({++ slug: moments.makeSlug ? moments.makeSlug() : undefined, user_id: stream.user_id,++ title: r.title.slice(0, 80), content: `${r.description}\n\n🔴 Caught live on @${username}'s stream — ${base}/${username}`, language: 'text', visibility: 'public',++ stream_id: stream.id, metadata: JSON.stringify({ ai_moment: true, live: true, stream_id: stream.id, offset, username, vod_link: null }),++ ai_summary: r.description, ai_tags: JSON.stringify(Array.isArray(r.tags) ? r.tags.slice(0, 8) : []),++ screenshot: { buffer, filename: `live-${stream.id}-${offset}.jpg`, contentType: 'image/jpeg' },++ });++ if (paste) {++ registry.record({ kind: 'paste', stream_id: stream.id, offset, desc: r.description, title: r.title });++ console.log(`[AI-Moments] live paste for stream ${stream.id} at ${offset}s: "${r.title}"`);++ }}async function tick() {@@@ -82,7 +82,7 @@ async function _processSegment(stream, seg) {try {const sounds = require('./audio-events');if (sounds.available()) {-- const cat = stream.category || null;++ const cat = stream.ai_category || stream.category || null;const detected = await sounds.detect(seg.path, { offsetSec: seg.offsetSec, category: cat });for (const d of (detected || [])) {events.push({@@@ -175,7 +175,7 @@ function rawStatsFor(userId) {const allTime = db.get(`SELECT COUNT(*) AS streams, COALESCE(SUM(duration_seconds), 0) / 3600.0 AS hours, COALESCE(MAX(peak_viewers), 0) AS peak_viewers FROM streams WHERE user_id = ? AND duration_seconds > 0`, [userId]) || {};const followers = db.get('SELECT COUNT(*) AS n FROM follows WHERE streamer_id = ?', [userId])?.n || 0;const tips = db.get(`SELECT COALESCE(SUM(amount), 0) AS n FROM transactions WHERE to_user_id = ? AND type = 'donation' AND created_at >= datetime('now', ?)`, [userId, win])?.n || 0;-- const category = db.get(`SELECT category, COUNT(*) AS n FROM streams WHERE user_id = ? AND duration_seconds > 0 AND category IS NOT NULL AND category != '' GROUP BY category ORDER BY n DESC LIMIT 1`, [userId])?.category || null;++ const category = db.get(`SELECT COALESCE(NULLIF(ai_category, ''), category) AS category, COUNT(*) AS n FROM streams WHERE user_id = ? AND duration_seconds > 0 AND COALESCE(NULLIF(ai_category, ''), category) IS NOT NULL AND COALESCE(NULLIF(ai_category, ''), category) != '' GROUP BY 1 ORDER BY n DESC LIMIT 1`, [userId])?.category || null;const hours = Math.max(Number(agg.hours) || 0, 0.1);return {window_days: STATS_WINDOW_DAYS, streams: agg.streams || 0, hours: Number((agg.hours || 0).toFixed(1)), peak_viewers: agg.peak_viewers || 0,@@@ -1345,6 +1345,12 @@ function initDb() {if (!scols.includes('ai_overview')) database.exec('ALTER TABLE streams ADD COLUMN ai_overview TEXT');// Short AI-generated session title for the AI Timeline (streamers reuse literal titles).if (!scols.includes('ai_title')) database.exec('ALTER TABLE streams ADD COLUMN ai_title TEXT');++ // Category/tags inferred by the AI from what the stream actually shows (the go-live selector++ // used to default to 'irl', which made every AI think everyone is an IRL streamer).++ if (!scols.includes('ai_category')) database.exec('ALTER TABLE streams ADD COLUMN ai_category TEXT');++ if (!scols.includes('ai_tags')) database.exec('ALTER TABLE streams ADD COLUMN ai_tags TEXT');++ const chcols = database.prepare('PRAGMA table_info(channels)').all().map(c => c.name);++ if (!chcols.includes('ai_category')) database.exec('ALTER TABLE channels ADD COLUMN ai_category TEXT');// AI overview + transcript on VODs and clips.const vcols = database.prepare('PRAGMA table_info(vods)').all().map(c => c.name);@@@ -1395,6 +1401,16 @@ function initDb() {}} catch (e) { console.warn('[DB] visibility migration:', e.message); }++ // AI-inferred category/tags. The go-live selector used to default to 'irl', which taught++ // every downstream AI (viewers, overviews, arena) that everyone is an IRL streamer; the++ // stream-memory rollup now classifies each stream from what it actually shows.++ try {++ const sc = database.prepare('PRAGMA table_info(streams)').all().map(c => c.name);++ for (const col of ['ai_overview TEXT', 'ai_title TEXT', 'ai_category TEXT', 'ai_tags TEXT']) { if (!sc.includes(col.split(' ')[0])) database.exec(`ALTER TABLE streams ADD COLUMN ${col}`); }++ const cc = database.prepare('PRAGMA table_info(channels)').all().map(c => c.name);++ if (!cc.includes('ai_category')) database.exec('ALTER TABLE channels ADD COLUMN ai_category TEXT');++ } catch (e) { console.warn('[DB] ai-category migration:', e.message); }++// Cached SHORT overview (a concise, locally-derived version of the long AI// overview) shown on listing cards — computed once at write time, no extra API.try {@@@ -2676,7 +2692,7 @@ function getOrCreateAnonGameUser(anonId) {function getLiveStreams() {return all(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,ms.slug AS managed_stream_slug, ms.id AS managed_stream_id,ms.stream_key AS managed_stream_key,ms.browser_mode, ms.streaming_method@@@ -2690,7 +2706,7 @@ function getLiveStreams() {function getRecentStreams(limit = 20) {return all(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,v.id AS vod_id, v.is_public AS vod_is_public, v.thumbnail_url AS vod_thumbnail_url,v.duration_seconds AS vod_durationFROM streams s@@@ -2710,7 +2726,7 @@ function getRecentStreams(limit = 20) {function getStreamById(id) {return get(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,ms.title AS managed_stream_title, ms.protocol AS managed_stream_protocolFROM streams s@@@ -2729,7 +2745,7 @@ function getStreamByUserId(userId) {function getLiveStreamsByUserId(userId) {return all(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,ms.title AS managed_stream_titleFROM streams s@@@ -2742,7 +2758,7 @@ function getLiveStreamsByUserId(userId) {function getLiveStreamsByControlConfigId(controlConfigId) {return all(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,ms.title AS managed_stream_titleFROM streams s@@@ -2755,7 +2771,7 @@ function getLiveStreamsByControlConfigId(controlConfigId) {function getStreamsByUserId(userId, limit = 50) {return all(`-- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,++ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,ms.title AS managed_stream_title, ms.id AS managed_stream_ref_idFROM streams s@@@ -2785,7 +2801,7 @@ function createStream({ user_id, channel_id, managed_stream_id, control_config_ireturn run(`INSERT INTO streams (user_id, channel_id, managed_stream_id, control_config_id, title, description, category, protocol, is_nsfw, thumbnail_url, is_live, started_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)`,-- [user_id, channel_id || null, managed_stream_id || null, control_config_id || null, title || 'Untitled Stream', description || '', category || 'irl', protocol || 'webrtc', is_nsfw ? 1 : 0, thumbnail_url || null]++ [user_id, channel_id || null, managed_stream_id || null, control_config_id || null, title || 'Untitled Stream', description || '', category || null, protocol || 'webrtc', is_nsfw ? 1 : 0, thumbnail_url || null]);}@@@ -2856,6 +2872,15 @@ function _shortOverview(text) {function updateStreamAiOverview(streamId, text) {return run('UPDATE streams SET ai_overview = ?, ai_overview_short = ? WHERE id = ?', [text || null, _shortOverview(text), streamId]);}++/** AI-inferred category/tags for a stream; the channel inherits the latest inferred category. */++function setStreamAiCategory(streamId, category, tags) {++ const cat = category ? String(category).toLowerCase().slice(0, 40) : null;++ const r = run('UPDATE streams SET ai_category = ?, ai_tags = ? WHERE id = ?', [cat, Array.isArray(tags) && tags.length ? JSON.stringify(tags.slice(0, 8)) : null, streamId]);++ if (cat) { const s = get('SELECT user_id FROM streams WHERE id = ?', [streamId]); if (s) run('UPDATE channels SET ai_category = ? WHERE user_id = ?', [cat, s.user_id]); }++ return r;++}++/** What to call a stream's category: the AI's read of the stream beats the self-selected default. */++function effectiveCategory(row) { if (!row) return null; return row.ai_category || row.category || null; }// AI overview/transcript state lives in vod_ai_state / clip_ai_state (Live-owned,// keyed by the Media vod/clip id) — the moved vods/clips tables are never written.function _ensureVodAiState(vodId) {@@@ -3528,7 +3553,7 @@ function createManagedStream({ user_id, channel_id, slug, title, description, careturn run(`INSERT INTO managed_streams (user_id, channel_id, slug, title, description, category, protocol, streaming_method, stream_key, is_nsfw, control_config_id)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,-- [user_id, channel_id || null, slug || null, title || 'Untitled Stream', description || '', category || 'irl', protocol || 'webrtc', streaming_method || null, stream_key, is_nsfw ? 1 : 0, control_config_id || null]++ [user_id, channel_id || null, slug || null, title || 'Untitled Stream', description || '', category || null, protocol || 'webrtc', streaming_method || null, stream_key, is_nsfw ? 1 : 0, control_config_id || null]);}@@@ -4224,7 +4249,7 @@ function createChannel({ user_id, title, description, category, protocol }) {return run(`INSERT OR IGNORE INTO channels (user_id, title, description, category, protocol)VALUES (?, ?, ?, ?, ?)`,-- [user_id, title || 'Untitled Channel', description || '', category || 'irl', protocol || 'webrtc']++ [user_id, title || 'Untitled Channel', description || '', category || null, protocol || 'webrtc']);}@@@ -7999,7 +8024,7 @@ module.exports = {isValidManagedStreamSlug, isManagedStreamSlugTaken,ensureStreamerRoleOnFeed,// Streams (sessions)-- getLiveStreams, getRecentStreams, getStreamById, getStreamByUserId, getLiveStreamsByUserId, getLiveStreamsByControlConfigId, getStreamsByUserId, getStreamHistoryByManagedStream,++ getLiveStreams, getRecentStreams, getStreamById, setStreamAiCategory, effectiveCategory, getStreamByUserId, getLiveStreamsByUserId, getLiveStreamsByControlConfigId, getStreamsByUserId, getStreamHistoryByManagedStream,createStream, endStream, endOtherLiveStreamsForSlot, updateViewerCount,addStreamMemory, getStreamMemories, getLatestStreamMemory, updateStreamAiOverview,setVodAiOverview, setClipAiOverview, setVodTranscript, setClipTranscript, getStreamMemoriesInRange,@@@ -204,7 +204,7 @@ router.get('/:id', optionalAuth, async (req, res) => {clip.stream_started_at = stream.started_at;clip.stream_ended_at = stream.ended_at;clip.stream_title = stream.title;-- clip.stream_category = stream.category;++ clip.stream_category = stream.ai_category || stream.category;clip.stream_peak_viewers = stream.peak_viewers;clip.stream_protocol = stream.protocol;}@@@ -351,7 +351,7 @@ router.get('/:id', optionalAuth, async (req, res) => {if (stream) {vod.stream_started_at = stream.started_at;vod.stream_ended_at = stream.ended_at;-- vod.stream_category = stream.category;++ vod.stream_category = stream.ai_category || stream.category;vod.stream_peak_viewers = stream.peak_viewers;}}@@@ -1446,7 +1446,7 @@ router.post('/managed', requireAuth, (req, res) => {const title = cleanText(req.body.title, { maxLength: MAX_TITLE_LENGTH }) || 'Untitled Stream';const description = cleanText(req.body.description, { maxLength: MAX_DESCRIPTION_LENGTH, allowEmpty: true }) || '';-- const category = cleanText(req.body.category, { maxLength: MAX_CATEGORY_LENGTH }) || 'irl';++ const category = cleanText(req.body.category, { maxLength: MAX_CATEGORY_LENGTH }) || null; // empty = let the AI infer it from the streamconst protocol = cleanProtocol(req.body.protocol) || 'webrtc';const is_nsfw = cleanBooleanFlag(req.body.is_nsfw);let slug = req.body.slug ? req.body.slug.trim().toLowerCase() : null;@@@ -1783,7 +1783,7 @@ router.post('/', requireAuth, (req, res) => {// Use managed stream's settings as defaults, allow per-session overridesconst streamProtocol = protocol || cleanProtocol(managedStream.protocol) || cleanProtocol(channel.protocol) || 'webrtc';-- const streamCategory = category || cleanText(managedStream.category, { maxLength: MAX_CATEGORY_LENGTH }) || cleanText(channel.category, { maxLength: MAX_CATEGORY_LENGTH }) || 'irl';++ const streamCategory = category || cleanText(managedStream.category, { maxLength: MAX_CATEGORY_LENGTH }) || cleanText(channel.category, { maxLength: MAX_CATEGORY_LENGTH }) || null; // null → the AI classifies the stream from what it showsconst requestedControlConfigId = req.body.control_config_id !== undefined ? (req.body.control_config_id === null ? null : parseInt(req.body.control_config_id)) : undefined;if (requestedControlConfigId !== undefined && requestedControlConfigId !== null) {@@@ -578,7 +578,7 @@ function autoCreateWhipSession(managedStream, user) {managed_stream_id: managedStream.id,title: managedStream.title || `${user.display_name || user.username}'s Stream`,description: managedStream.description || '',-- category: managedStream.category || 'irl',++ category: managedStream.category || null, // null → the AI classifies the stream from what it showsprotocol: 'webrtc',is_nsfw: managedStream.is_nsfw || 0,});nnew file mode 100644@@@ -0,0 +1,40 @@++'use strict';++// The paste job and the clip job share ONE registry of used moments, so they can never pick the++// same second or the same scene of a VOD (the "same title on the clip and the paste" bug).++const assert = require('assert');++const fs = require('fs'); const os = require('os'); const path = require('path');++const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-registry-'));++process.env.DB_PATH = path.join(tmp, 'live.db');++const db = require('../server/db/database'); db.initDb();++const reg = require('../server/ai/moment-registry');++++// Legacy logs are imported once so history is not lost on upgrade.++db.setState('auto_clip_log', JSON.stringify([{ stream_id: 5, vod_id: 900, start_time: 100, title: 'Old clip', sig: 'old clip scene', ts: Date.now() - 3600_000 }]));++db.setState('home_hero_moments', JSON.stringify({ moments: [{ vodId: 901, offset: 300, title: 'Orange Shirt Shocked in aisle' }], usedSigs: ['a person wearing an orange'], updated_at: Date.now() - 7200_000 }));++reg._reset();++assert.ok(reg.isUsed({ vod_id: 900, offset: 116 }), 'legacy clip (start + pre-roll) blocks the spot');++assert.ok(reg.isUsed({ vod_id: 901, offset: 330 }), 'legacy paste blocks within the gap');++assert.ok(reg.isUsed({ desc: 'A person wearing an orange shirt is shown leaning' }), 'legacy scene signature blocks the same scene');++assert.ok(!reg.isUsed({ vod_id: 901, offset: 900, desc: 'A dog steals the mic' }), 'far away + new scene is free');++assert.deepStrictEqual(reg.usedOffsets(901), [300]);++console.log('✅ legacy logs imported');++++reg.record({ kind: 'paste', vod_id: 42, stream_id: 7, offset: 1000, title: 'Depth Estimation tease', desc: 'The video appears to be a live streaming session with a chat overlay' });++assert.strictEqual(reg.usedReason({ vod_id: 42, offset: 1050 }), 'paste already made at 1000s of this VOD', 'same VOD within 2 min is blocked');++assert.ok(/same scene as a recent paste/.test(reg.usedReason({ vod_id: 43, offset: 5, desc: 'the video appears to be a live streaming session with music' })), 'same opening words = same scene, even on another VOD');++assert.strictEqual(reg.usedReason({ vod_id: 42, offset: 1300, desc: 'A robot arm knocks over the coffee' }), null, '≥ 2 min away with a different scene is fine');++assert.ok(reg.isUsed({ stream_id: 7, offset: 1010 }), 'live streams are matched by stream id too');++reg.record({ kind: 'clip', vod_id: 42, stream_id: 7, offset: 1300, title: 'Robot arm vs coffee' });++assert.deepStrictEqual(reg.usedOffsets(42), [1000, 1300]);++assert.strictEqual(reg.lastOfKind('paste', { stream_id: 7 }).offset, 1000);++assert.strictEqual(reg.lastOfKind('clip', { vod_id: 42 }).title, 'Robot arm vs coffee');++assert.ok(reg.recent(1).length >= 2);++reg._reset();++assert.ok(reg.isUsed({ vod_id: 42, offset: 1290 }), 'persisted across a cache reset');++console.log('✅ shared paste/clip registry: spots, scenes, persistence');++++const ai = require('../server/ai/ai-analysis');++assert.deepStrictEqual(['gaming', 'Just Chatting', 'coding', 'nonsense', ''].map(ai.normalizeCategory), ['gaming', 'irl', 'desktop', null, null]);++console.log('✅ category taxonomy normalisation');++console.log('\n✅ All moment-registry tests passed');++process.exit(0);
@@ -52,3 +52,19 @@ From the dashboard:
@@ -52,3 +52,19 @@ From the dashboard:
## Chat Replay
## Chat Replay
VODs include synchronized chat replay. Messages are stored in the database with timestamps relative to the stream start. Deleted messages are automatically excluded from replay (soft-delete with `is_deleted` flag).
VODs include synchronized chat replay. Messages are stored in the database with timestamps relative to the stream start. Deleted messages are automatically excluded from replay (soft-delete with `is_deleted` flag).
+
+## AI moments: pastes and clips never overlap
+
+Two jobs turn stream moments into content: `server/ai/ai-moments-job.js` (image **pastes** for the home hero + pastes tab, every 6 h) and `server/ai/auto-clip-job.js` (live chat-spike **clips** + a VOD backfill). They used to pick the same second of the same VOD with the same title. Now:
+
+- **One shared registry** — `server/ai/moment-registry.js` (`ai_used_moments` state) records every paste/clip (`vod_id`, `stream_id`, `offset`, scene signature, title). Both jobs ask it before creating anything: a moment is refused if it is within **2 min** of a used moment on the same VOD/stream, or if its scene signature (first five words of the description) was used in the last two weeks. Legacy logs are imported once.
+- **Flavoured picks** — `findBestMoment(vod, { flavor, avoid })`: `paste` asks for a frame that is striking on its own (a face, a gag, something odd on screen); `clip` asks for a beat that plays out over 25 s (a line, a reaction, a sound, chat exploding). The prompt lists the already-used timestamps and the model is told to stay away from them; if it still lands next to one, the objective signals (viewer clips, chat spikes, richest scene notes) pick a free spot.
+- The paste job no longer clips the paste's own moment; it asks for a **second, different beat** from the same VOD for the clip. The clip backfill avoids every offset a paste used.
+- **Live pastes** — the stream-memory vision call now also answers "is this frame screenshot-worthy?" with a caption (no extra call). When it says yes, `stream-memory-job` posts the frame as an image paste right away (≤ 1 per stream per 90 min, dark-frame and registry checks, setting `ai_live_pastes_enabled`, default on). These pastes carry `metadata.live = true` and no VOD link.
+- **Fewer tokens** — VOD showcase scores are cached for 7 days (`home_hero_moments.rankCache`); a run only scores VODs it has never scored. Every `llm.complete` call keeps its `kind` so `ai_usage` shows exactly where the budget goes (`moment_vod_rank`, `moment_pick`, `moment_frame`, `stream_memory`, `auto_clip_confirm`).
+
+## AI-inferred category
+
+Go-live used to default every stream to `irl`, so every AI prompt (AI viewers, streamer overviews, VOD overviews, the Arena, sound detection) assumed everyone is an IRL streamer. Now the category selector defaults to **Auto** (stored as `NULL`), and the stream-memory rollup — the same summarise call that already runs — returns `{ overview, category, tags }` with `category` from the fixed taxonomy (`outdoors, travel, building, music, gaming, robot, desktop, irl, other`). It is stored on `streams.ai_category` / `streams.ai_tags` and the channel inherits the latest read (`channels.ai_category`).
+
+Everything reads the effective value: listing queries return `COALESCE(ai_category, category) AS category` (the self-selected value is still there as `chosen_category`), `stream_category` on VODs/clips, the AI viewers' session block ("judged from the stream itself"), streamer overviews, sound detection and the Arena all prefer `ai_category`. A streamer can still pick a category manually; the AI read wins once it exists.
@@ -804,7 +804,7 @@
@@ -804,7 +804,7 @@
<h2 id="ch-display-name">Channel</h2>
<h2 id="ch-display-name">Channel</h2>
<span class="ch-staff-badge" id="ch-staff-badge"></span>
<span class="ch-staff-badge" id="ch-staff-badge"></span>
<span class="ch-username" id="ch-username">username</span>
<span class="ch-username" id="ch-username">username</span>
- <span class="ch-category-badge badge" id="ch-category-badge">irl</span>
+ <span class="ch-category-badge badge" id="ch-category-badge">Live</span>
<span id="ch-protocol-badge"></span>
<span id="ch-protocol-badge"></span>
</div>
</div>
<p class="ch-stream-title" id="ch-stream-title">Stream Title</p>
<p class="ch-stream-title" id="ch-stream-title">Stream Title</p>
@@ -934,7 +934,7 @@
@@ -934,7 +934,7 @@
<p class="ch-description" id="ch-description-offline"></p>
<p class="ch-description" id="ch-description-offline"></p>
<div class="ch-meta">
<div class="ch-meta">
<span class="ch-follower-count" id="ch-follower-count-offline">0 followers</span>
<span class="ch-follower-count" id="ch-follower-count-offline">0 followers</span>
- <span class="ch-category-badge badge" id="ch-category-badge-offline">irl</span>
+ <span class="ch-category-badge badge" id="ch-category-badge-offline">Offline</span>
</div>
</div>
<div class="ch-weather-widget" id="ch-weather-widget-offline" style="display:none"></div>
<div class="ch-weather-widget" id="ch-weather-widget-offline" style="display:none"></div>
</div>
</div>
@@ -3467,7 +3467,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionI
@@ -3467,7 +3467,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionI
_activeChannelIsOwnerRank = !!ch.is_owner;
_activeChannelIsOwnerRank = !!ch.is_owner;
const _chUser = document.getElementById('ch-username');
const _chUser = document.getElementById('ch-username');
if (_chUser) _chUser.style.display = 'none';
if (_chUser) _chUser.style.display = 'none';
- document.getElementById('ch-category-badge').textContent = _capTag((liveStreams[0] && liveStreams[0].category) || ch.category || 'irl');
+ document.getElementById('ch-category-badge').textContent = _capTag((liveStreams[0] && liveStreams[0].category) || ch.ai_category || ch.category || 'Live');
document.getElementById('ch-follower-count').textContent = `${ch.follower_count || 0} followers`;
document.getElementById('ch-follower-count').textContent = `${ch.follower_count || 0} followers`;
setupFollowBtn(document.getElementById('ch-btn-follow'));
setupFollowBtn(document.getElementById('ch-btn-follow'));
setupBanBtn(document.getElementById('ch-btn-ban'));
setupBanBtn(document.getElementById('ch-btn-ban'));
@@ -3528,7 +3528,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionI
@@ -3528,7 +3528,7 @@ async function loadChannelPage(username, managedStreamRef = null, legacySessionI
document.getElementById('ch-username-offline').textContent = '@' + ch.username;
document.getElementById('ch-username-offline').textContent = '@' + ch.username;
document.getElementById('ch-description-offline').textContent = ch.description || '';
document.getElementById('ch-description-offline').textContent = ch.description || '';
document.getElementById('ch-follower-count-offline').textContent = `${ch.follower_count || 0} followers`;
document.getElementById('ch-follower-count-offline').textContent = `${ch.follower_count || 0} followers`;
- document.getElementById('ch-category-badge-offline').textContent = _capTag(ch.category || 'irl');
+ document.getElementById('ch-category-badge-offline').textContent = _capTag(ch.ai_category || ch.category || 'Offline');
setupFollowBtn(document.getElementById('ch-btn-follow-offline'));
setupFollowBtn(document.getElementById('ch-btn-follow-offline'));
setupBanBtn(document.getElementById('ch-btn-ban-offline'));
setupBanBtn(document.getElementById('ch-btn-ban-offline'));
@@ -7936,7 +7936,8 @@ function createManagedStreamModal() {
@@ -7936,7 +7936,8 @@ function createManagedStreamModal() {
<div class="form-group">
<div class="form-group">
<label>Category</label>
<label>Category</label>
<select id="cms-category" class="form-input">
<select id="cms-category" class="form-input">
- <option value="irl" selected>IRL</option>
+ <option value="" selected>Auto — the AI decides from the stream</option>
+ <option value="irl">IRL</option>
<option value="outdoors">Outdoors</option>
<option value="outdoors">Outdoors</option>
<option value="travel">Travel</option>
<option value="travel">Travel</option>
<option value="building">Building/Craft</option>
<option value="building">Building/Craft</option>
@@ -7984,7 +7985,7 @@ async function _cmsCreate() {
@@ -7984,7 +7985,7 @@ async function _cmsCreate() {
const btn = document.getElementById('cms-create-btn');
const btn = document.getElementById('cms-create-btn');
const errEl = document.getElementById('cms-error');
const errEl = document.getElementById('cms-error');
const title = (document.getElementById('cms-title')?.value || '').trim() || 'Untitled Stream';
const title = (document.getElementById('cms-title')?.value || '').trim() || 'Untitled Stream';
- const category = document.getElementById('cms-category')?.value || 'irl';
+ const category = document.getElementById('cms-category')?.value || '';
const protocol = document.getElementById('cms-protocol')?.value || 'webrtc';
const protocol = document.getElementById('cms-protocol')?.value || 'webrtc';
const streamingMethod = document.getElementById('cms-method')?.value || 'browser';
const streamingMethod = document.getElementById('cms-method')?.value || 'browser';
const slug = (document.getElementById('cms-slug')?.value || '').trim().toLowerCase() || undefined;
const slug = (document.getElementById('cms-slug')?.value || '').trim().toLowerCase() || undefined;
@@ -421,7 +421,7 @@ function _wsRenderPanel() {
@@ -421,7 +421,7 @@ function _wsRenderPanel() {
<div class="form-group" style="flex:2">
<div class="form-group" style="flex:2">
<label>Category</label>
<label>Category</label>
<select id="bc-category" class="form-input" onchange="_wsMarkDirty()">
<select id="bc-category" class="form-input" onchange="_wsMarkDirty()">
- ${_wsRenderCategoryOptions(ms.category || 'irl')}
+ ${_wsRenderCategoryOptions(ms.category || '')}
</select>
</select>
</div>
</div>
<div class="form-group" style="flex:1">
<div class="form-group" style="flex:1">
@@ -1391,6 +1391,7 @@ function _wsRenderBrowserModeCards(selected) {
@@ -1391,6 +1391,7 @@ function _wsRenderBrowserModeCards(selected) {
function _wsRenderCategoryOptions(selected) {
function _wsRenderCategoryOptions(selected) {
const cats = [
const cats = [
+ ['', 'Auto — the AI decides from the stream'],
['outdoors', 'Outdoors'], ['travel', 'Travel'], ['building', 'Building/Craft'],
['outdoors', 'Outdoors'], ['travel', 'Travel'], ['building', 'Building/Craft'],
['music', 'Music'], ['gaming', 'Gaming'], ['robot', 'Robot'],
['music', 'Music'], ['gaming', 'Gaming'], ['robot', 'Robot'],
['desktop', 'Desktop'], ['irl', 'IRL'], ['other', 'Other'],
['desktop', 'Desktop'], ['irl', 'IRL'], ['other', 'Other'],
@@ -1959,7 +1960,7 @@ async function _wsSaveAll() {
@@ -1959,7 +1960,7 @@ async function _wsSaveAll() {
const streamFields = {
const streamFields = {
title: document.getElementById('bc-title')?.value.trim() || ms.title,
title: document.getElementById('bc-title')?.value.trim() || ms.title,
description: document.getElementById('bc-description')?.value.trim() || '',
description: document.getElementById('bc-description')?.value.trim() || '',
- category: document.getElementById('bc-category')?.value || 'irl',
+ category: document.getElementById('bc-category')?.value || '',
is_nsfw: document.getElementById('bc-nsfw')?.checked ? 1 : 0,
is_nsfw: document.getElementById('bc-nsfw')?.checked ? 1 : 0,
protocol: ms.protocol,
protocol: ms.protocol,
streaming_method: ms.streaming_method || 'browser',
streaming_method: ms.streaming_method || 'browser',
@@ -1440,7 +1440,7 @@ async function loadBroadcastControlConfigs() {
@@ -1440,7 +1440,7 @@ async function loadBroadcastControlConfigs() {
async function createNewStream() {
async function createNewStream() {
const title = document.getElementById('bc-title')?.value.trim();
const title = document.getElementById('bc-title')?.value.trim();
const description = document.getElementById('bc-description')?.value.trim() || '';
const description = document.getElementById('bc-description')?.value.trim() || '';
- const category = document.getElementById('bc-category')?.value || 'irl';
+ const category = document.getElementById('bc-category')?.value || '';
const method = broadcastState.selectedMethod;
const method = broadcastState.selectedMethod;
if (!title) return toast('Stream title is required', 'error');
if (!title) return toast('Stream title is required', 'error');
@@ -139,13 +139,21 @@ async function analyzeTextPaste(content, title) {
@@ -139,13 +139,21 @@ async function analyzeTextPaste(content, title) {
/** Analyze a live-stream frame → { description, tags }. */
/** Analyze a live-stream frame → { description, tags }. */
async function analyzeStreamFrame(image) {
async function analyzeStreamFrame(image) {
- const prompt = `This is a frame from a live stream. Reply ONLY with compact JSON: {"description":"one concise sentence describing what is happening on screen right now","tags":["2-5","short","tags"]}.`;
- const text = await _complete({ prompt, image, maxTokens: 200, kind: 'stream_memory', role: 'vision', imageMaxWidth: 768 });
+ // One vision call does three jobs: the memory description, the tags, and a screenshot-worthiness
+ // verdict + caption (so live pastes need no extra call).
+ const prompt = `This is a frame from a live stream. Reply ONLY with compact JSON: {"description":"one concise sentence describing what is happening on screen right now","tags":["2-5","short","tags"],"worthy":<true only if this exact frame is genuinely screenshot-worthy on its own: a face/reaction, a visual gag, something unusual or funny on screen — false for ordinary gameplay/desktop/chat/talking-head frames>,"title":"<if worthy: a punchy, funny 3-7 word caption for it, else empty>"}.`;
+ const text = await _complete({ prompt, image, maxTokens: 240, kind: 'stream_memory', role: 'vision', imageMaxWidth: 768 });
if (!text) return null;
if (!text) return null;
const description = _extractDescription(text, 400);
const description = _extractDescription(text, 400);
- return description ? { description, tags: _extractTags(text, 6) } : null;
+ if (!description) return null;
+ const j = _parseJson(text) || {};
+ return { description, tags: _extractTags(text, 6), worthy: j.worthy === true, title: j.worthy === true ? String(j.title || '').replace(/^["'\s]+|["'\s]+$/g, '').slice(0, 80) : '' };
}
}
+/** Fixed category taxonomy the AI picks from (matches the go-live selector). */
+const CATEGORIES = ['outdoors', 'travel', 'building', 'music', 'gaming', 'robot', 'desktop', 'irl', 'other'];
+function normalizeCategory(c) { const t = String(c || '').toLowerCase().trim(); if (!t) return null; if (CATEGORIES.includes(t)) return t; const map = { 'just chatting': 'irl', chatting: 'irl', talk: 'irl', coding: 'desktop', programming: 'desktop', software: 'desktop', tech: 'desktop', art: 'building', craft: 'building', diy: 'building', cooking: 'irl', hiking: 'outdoors', driving: 'travel', 'road trip': 'travel', robotics: 'robot', game: 'gaming', games: 'gaming', dj: 'music' }; return map[t] || null; }
+
/** Condense a stream's memories into a one-line "AI Overview" for the home card. */
/** Condense a stream's memories into a one-line "AI Overview" for the home card. */
async function summarizeStreamMemories(memories, streamId = null) {
async function summarizeStreamMemories(memories, streamId = null) {
// Use observations from across the whole session (capped for token budget) so the
// Use observations from across the whole session (capped for token budget) so the
@@ -179,9 +187,15 @@ async function summarizeStreamMemories(memories, streamId = null) {
@@ -179,9 +187,15 @@ async function summarizeStreamMemories(memories, streamId = null) {
} catch { /* timeline optional */ }
} catch { /* timeline optional */ }
}
}
- const prompt = `These are timestamped observations from a live stream, in order since it started. Give a thorough overview (2-6 sentences) of what this stream has been about overall — the main activities, topics, and vibe across the whole session (not just the latest moment):\n${lines}${audioBlock}`;
- const text = await _complete({ prompt, maxTokens: 500, kind: 'stream_memory' });
- return text ? text.slice(0, 2000) : null;
+ // The same call also classifies the stream — category is inferred from what is actually on
+ // screen / said, never from the streamer's go-live selector (which used to default to "irl"
+ // and taught every downstream AI that everyone is an IRL streamer).
+ const prompt = `These are timestamped observations from a live stream, in order since it started. Reply ONLY with compact JSON: {"overview":"a thorough overview (2-6 sentences) of what this stream has been about overall — the main activities, topics, and vibe across the whole session (not just the latest moment)","category":"<exactly one of: ${CATEGORIES.join(', ')} — what this stream mostly IS, judged from the observations; desktop = screen/software/coding, gaming = playing games, irl = camera on a person talking/doing things indoors, outdoors/travel = out in the world>","tags":["3-6","short","lowercase","tags"]}\n${lines}${audioBlock}`;
+ const text = await _complete({ prompt, maxTokens: 560, kind: 'stream_memory' });
+ if (!text) return null;
+ const j = _parseJson(text);
+ const overview = j && j.overview ? String(j.overview).slice(0, 2000) : (_extractDescription(text, 2000) || text.slice(0, 2000));
+ return { overview, category: normalizeCategory(j && j.category), tags: j && Array.isArray(j.tags) ? j.tags.map(String).slice(0, 6) : [] };
}
}
/**
/**
@@ -198,7 +212,7 @@ async function generateStreamerOverview(userId) {
@@ -198,7 +212,7 @@ async function generateStreamerOverview(userId) {
const memLines = memories.slice(0, 40).map(m => `- ${m.description}`).filter(l => l.length > 2);
const memLines = memories.slice(0, 40).map(m => `- ${m.description}`).filter(l => l.length > 2);
const vods = (db.getVodsByUser ? db.getVodsByUser(userId, false, 20, 0) : []) || [];
const vods = (db.getVodsByUser ? db.getVodsByUser(userId, false, 20, 0) : []) || [];
- const vodLines = vods.map(v => `- ${v.title || 'Untitled VOD'}${v.category ? ` [${v.category}]` : ''}`);
+ const vodLines = vods.map(v => `- ${v.title || 'Untitled VOD'}${(v.ai_category || v.category) ? ` [${v.ai_category || v.category}]` : ''}`);
const pastes = (db.getUserPastesForAi ? db.getUserPastesForAi(userId, 25) : []) || [];
const pastes = (db.getUserPastesForAi ? db.getUserPastesForAi(userId, 25) : []) || [];
const pasteLines = pastes.filter(p => p.ai_summary).map(p => `- "${p.title || 'paste'}": ${p.ai_summary}`);
const pasteLines = pastes.filter(p => p.ai_summary).map(p => `- "${p.title || 'paste'}": ${p.ai_summary}`);
@@ -208,7 +222,7 @@ async function generateStreamerOverview(userId) {
@@ -208,7 +222,7 @@ async function generateStreamerOverview(userId) {
const ctx = [
const ctx = [
`Streamer: ${user.display_name || user.username} (@${user.username})`,
`Streamer: ${user.display_name || user.username} (@${user.username})`,
(channel?.bio || user.bio) ? `Bio: ${(channel?.bio || user.bio).slice(0, 400)}` : '',
(channel?.bio || user.bio) ? `Bio: ${(channel?.bio || user.bio).slice(0, 400)}` : '',
- channel?.category ? `Usual category: ${channel.category}` : '',
+ (channel?.ai_category || channel?.category) ? `Usual category (${channel?.ai_category ? 'inferred by AI from their streams' : 'self-selected'}): ${channel.ai_category || channel.category}` : '',
memLines.length ? `\nLive-stream observations (across sessions):\n${memLines.join('\n')}` : '',
memLines.length ? `\nLive-stream observations (across sessions):\n${memLines.join('\n')}` : '',
vodLines.length ? `\nRecent VODs:\n${vodLines.join('\n')}` : '',
vodLines.length ? `\nRecent VODs:\n${vodLines.join('\n')}` : '',
pasteLines.length ? `\nPaste summaries:\n${pasteLines.join('\n')}` : '',
pasteLines.length ? `\nPaste summaries:\n${pasteLines.join('\n')}` : '',
@@ -466,6 +480,7 @@ async function testStatus({ probe = true } = {}) {
@@ -466,6 +480,7 @@ async function testStatus({ probe = true } = {}) {
}
}
module.exports = {
module.exports = {
+ CATEGORIES, normalizeCategory,
isEnabled, withinBudget, pasteAnalysisEnabled, streamMemoryEnabled, transcriptionEnabled, captureIntervalSec,
isEnabled, withinBudget, pasteAnalysisEnabled, streamMemoryEnabled, transcriptionEnabled, captureIntervalSec,
analyzeImagePaste, analyzeTextPaste, analyzeStreamFrame, summarizeStreamMemories,
analyzeImagePaste, analyzeTextPaste, analyzeStreamFrame, summarizeStreamMemories,
generateStreamerOverview, generateVodOverview, generateClipOverview, ensureVodTimeline,
generateStreamerOverview, generateVodOverview, generateClipOverview, ensureVodTimeline,
@@ -18,6 +18,7 @@ const fs = require('node:fs');
@@ -18,6 +18,7 @@ const fs = require('node:fs');
const db = require('../db/database');
const db = require('../db/database');
const ai = require('./ai-analysis');
const ai = require('./ai-analysis');
const media = require('../media-client');
const media = require('../media-client');
+const registry = require('./moment-registry');
let cfg = null; try { cfg = require('../config'); } catch { /* */ }
let cfg = null; try { cfg = require('../config'); } catch { /* */ }
let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }
let thumb = null; try { thumb = require('../media-proxy/live-thumbs'); } catch { /* optional */ }
const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');
const FRAMES_TMP_DIR = path.join(os.tmpdir(), 'openvibe-live-moments');
@@ -86,7 +87,27 @@ Return STRICT JSON only, nothing else: [{"index": <n>, "score": <1-100>, "why":
@@ -86,7 +87,27 @@ Return STRICT JSON only, nothing else: [{"index": <n>, "score": <1-100>, "why":
}
}
// Rank the whole VOD set (batched tournament so "every VOD ever" scales). Falls back to the
// Rank the whole VOD set (batched tournament so "every VOD ever" scales). Falls back to the
// objective prior order (already applied by the DB) when AI is unavailable.
// objective prior order (already applied by the DB) when AI is unavailable.
+// Ranking cache: a VOD's showcase score barely changes, so score each VOD ONCE (7 days) and only
+// spend calls on VODs that have never been scored. Persisted in the job state as `rankCache`.
+const RANK_TTL_MS = 7 * 24 * 3600_000;
+let _rankCacheOut = {};
async function _rankVods(vods, want) {
async function _rankVods(vods, want) {
+ if (!_aiOn() || vods.length <= 1) return vods.map(v => ({ vod: v }));
+ const CH = 25;
+ const cache = _load().rankCache || {};
+ const now = Date.now();
+ const fresh = [], stale = [];
+ for (const v of vods) { const c = cache[v.vod_id]; if (c && now - (c.at || 0) < RANK_TTL_MS) fresh.push({ vod: v, score: c.score, why: c.why }); else stale.push(v); }
+ let scored = [];
+ for (let i = 0; i < stale.length; i += CH) scored.push(...(await _scoreVodChunk(stale.slice(i, i + CH), Math.max(want, 6))));
+ for (const s of scored) cache[s.vod.vod_id] = { score: s.score, why: s.why, at: now };
+ for (const v of stale) if (!cache[v.vod_id]) cache[v.vod_id] = { score: 0, why: 'unscored', at: now }; // never re-spend on a VOD the model left out
+ _rankCacheOut = Object.fromEntries(Object.entries(cache).filter(([, c]) => now - (c.at || 0) < RANK_TTL_MS).slice(-600));
+ const all = [...fresh, ...scored].sort((a, b) => b.score - a.score);
+ if (all.length) { console.log(`[AI-Moments] ranked ${vods.length} VOD(s): ${fresh.length} from cache, ${stale.length} scored now`); return all; }
+ return vods.map(v => ({ vod: v }));
+}
+async function _rankVodsUncached(vods, want) {
if (!_aiOn() || vods.length <= 1) return vods.map(v => ({ vod: v }));
if (!_aiOn() || vods.length <= 1) return vods.map(v => ({ vod: v }));
const CH = 25;
const CH = 25;
if (vods.length <= CH) {
if (vods.length <= CH) {
@@ -134,9 +155,17 @@ function _nearestMemory(memories, offset) {
@@ -134,9 +155,17 @@ function _nearestMemory(memories, offset) {
for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }
for (const m of memories) { const d = Math.abs((m.offset_seconds || 0) - offset); if (d < bestD) { bestD = d; best = m; } }
return best;
return best;
}
}
-async function _findBestMoment(vod) {
+// flavor: 'paste' → a frame that is striking on its own; 'clip' → a beat that plays out over ~25 s.
+// avoid: offsets (seconds) already turned into something — the picker must stay ≥ 2 min away.
+const FLAVOR_HINT = {
+ paste: 'This moment becomes a SCREENSHOT paste: pick something visually striking in a single frame — a face or reaction, a visual gag, something odd on screen, a scene change. What was said matters less than what is SEEN.',
+ clip: 'This moment becomes a 25-SECOND VIDEO CLIP: pick a beat that plays out over time — a line that lands, a reaction, a sound, chat exploding, something HAPPENING — not a static pretty frame.',
+};
+async function _findBestMoment(vod, { flavor = 'paste', avoid = [] } = {}) {
const dur = Math.floor(vod.duration || 0);
const dur = Math.floor(vod.duration || 0);
const ctx = _momentContext(vod.stream_id, vod.vod_id);
const ctx = _momentContext(vod.stream_id, vod.vod_id);
+ const avoidList = [...new Set([...(avoid || []), ...registry.usedOffsets(vod.vod_id, vod.stream_id)])].sort((a, b) => a - b);
+ const farFromUsed = (t) => avoidList.every(a => Math.abs(a - t) >= registry.GAP_SEC);
if (!ctx.memories.length && !ctx.transcript.length) return null;
if (!ctx.memories.length && !ctx.transcript.length) return null;
const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));
const clamp = (t) => Math.max(1, Math.min(Math.floor(t || 0), dur > 3 ? dur - 2 : (t || 0)));
@@ -160,7 +189,10 @@ ${soundLine || '(none)'}
@@ -160,7 +189,10 @@ ${soundLine || '(none)'}
Viewers CLIPPED these timestamps (very strong "this was a highlight" signal): ${clipHint}
Viewers CLIPPED these timestamps (very strong "this was a highlight" signal): ${clipHint}
Chat activity SPIKED around: ${spikeHint}
Chat activity SPIKED around: ${spikeHint}
-Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in this VOD with something clearly VISIBLE happening. Prefer moments backed by the clip/chat signals when they line up with something notable. NEVER pick a black/dark/loading/blank screen, an intro/BRB card, or a moment with no visible content or activity. Return STRICT JSON only, nothing else: {"t": <seconds into the vod>, "title": "<specific punchy 3-8 word title, not the stream name>", "desc": "<one vivid sentence describing the moment>"}`;
+${FLAVOR_HINT[flavor] || FLAVOR_HINT.paste}
+${avoidList.length ? `ALREADY USED (a paste or clip exists there) — do NOT pick these or anything within 2 minutes of them: ${avoidList.map(_mmss).join(', ')}. Find a DIFFERENT moment.` : ''}
+
+Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in this VOD with something clearly VISIBLE happening. Prefer moments backed by the clip/chat signals when they line up with something notable. NEVER pick a black/dark/loading/blank screen, an intro/BRB card, or a moment with no visible content or activity. Return STRICT JSON only, nothing else: {"t": <seconds into the vod>, "title": "<specific punchy 3-8 word title, not the stream name, funny, never generic>", "desc": "<one vivid sentence describing the moment>"}`;
try {
try {
const text = await ai.summarizeText(prompt, 300, 'moment_pick');
const text = await ai.summarizeText(prompt, 300, 'moment_pick');
const m = text && text.match(/\{[\s\S]*\}/);
const m = text && text.match(/\{[\s\S]*\}/);
@@ -169,6 +201,7 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in th
@@ -169,6 +201,7 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in th
let t = Number(j.t);
let t = Number(j.t);
if (!isNaN(t)) {
if (!isNaN(t)) {
t = clamp(t);
t = clamp(t);
+ if (!farFromUsed(t)) { console.log(`[AI-Moments] VOD ${vod.vod_id}: pick at ${_mmss(t)} is next to a used moment — trying the objective signals instead`); throw new Error('near used'); }
const near = _nearestMemory(ctx.memories, t);
const near = _nearestMemory(ctx.memories, t);
const result = { offset: t, title: _cleanTitle(j.title) || _titleFromDesc(j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', tags: _memTags(near) };
const result = { offset: t, title: _cleanTitle(j.title) || _titleFromDesc(j.desc), desc: _cleanText(_deJson(j.desc), 400) || (near && _deJson(near.description)) || '', tags: _memTags(near) };
return _isEmptyScene(result.desc) ? null : result;
return _isEmptyScene(result.desc) ? null : result;
@@ -177,15 +210,15 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in th
@@ -177,15 +210,15 @@ Find the SINGLE most interesting/funny/dramatic/surprising/striking moment in th
} catch { /* fall through to signal-based pick */ }
} catch { /* fall through to signal-based pick */ }
}
}
- // No-AI fallback: pick from the strongest objective signal.
- let offset = null;
- if (ctx.clipTimes.length) offset = ctx.clipTimes[Math.floor(ctx.clipTimes.length / 2)]; // where viewers clipped
- else if (ctx.spikes.length) offset = ctx.spikes[0].offset; // busiest chat moment
- else { // richest scene note
- const rich = ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length)[0];
- offset = rich ? (rich.offset_seconds || 0) : 0;
- }
- offset = clamp(offset);
+ // No-AI fallback (also used when the AI pick collided with a used moment): the strongest
+ // objective signal that is not next to something already made.
+ const candidates = [
+ ...ctx.clipTimes.map(t => ({ t, w: 3 })), // where viewers clipped
+ ...ctx.spikes.map(s => ({ t: s.offset, w: 2 })), // busiest chat moments
+ ...ctx.memories.slice().sort((a, b) => String(b.description).length - String(a.description).length).slice(0, 8).map(m => ({ t: m.offset_seconds || 0, w: 1 })), // richest scene notes
+ ].filter(c => farFromUsed(clamp(c.t))).sort((a, b) => b.w - a.w);
+ if (!candidates.length) return null;
+ let offset = clamp(candidates[0].t);
const near = _nearestMemory(ctx.memories, offset);
const near = _nearestMemory(ctx.memories, offset);
const desc = near ? _deJson(near.description) : '';
const desc = near ? _deJson(near.description) : '';
return _isEmptyScene(desc) ? null : { offset, title: _titleFromDesc(desc), desc, tags: _memTags(near) };
return _isEmptyScene(desc) ? null : { offset, title: _titleFromDesc(desc), desc, tags: _memTags(near) };
@@ -274,8 +307,7 @@ async function tick(opts = {}) {
@@ -274,8 +307,7 @@ async function tick(opts = {}) {
// Content-signature dedup: even different VODs from the same streamer (same setup) yield
// Content-signature dedup: even different VODs from the same streamer (same setup) yield
// near-identical scenes → we skip a moment whose description signature was used recently,
// near-identical scenes → we skip a moment whose description signature was used recently,
// so the hero/pastes never show visual duplicates. Respected even on --fresh.
// so the hero/pastes never show visual duplicates. Respected even on --fresh.
- const usedSigs = new Set(prev.usedSigs || []);
- const thisSigs = new Set();
+ const thisSigs = new Set(); // scenes picked in THIS run (cross-run dedupe lives in moment-registry)
// Stage 1: consider every eligible VOD, ranked by its AI overview + popularity prior.
// Stage 1: consider every eligible VOD, ranked by its AI overview + popularity prior.
// The VOD pool comes from OpenVibe.Media (most-viewed public VODs), overlaid with
// The VOD pool comes from OpenVibe.Media (most-viewed public VODs), overlaid with
@@ -316,10 +348,11 @@ async function tick(opts = {}) {
@@ -316,10 +348,11 @@ async function tick(opts = {}) {
// Stage 2: find the best moment within this VOD.
// Stage 2: find the best moment within this VOD.
const moment = await _findBestMoment(v);
const moment = await _findBestMoment(v);
if (!moment) continue;
if (!moment) continue;
- // Skip near-duplicate scenes (same signature as a recent or already-picked moment).
+ // Skip near-duplicate scenes / spots — the registry is shared with the clip job.
const sig = _sig(moment.desc || v.ai_overview_short || v.title);
const sig = _sig(moment.desc || v.ai_overview_short || v.title);
- if (sig && (usedSigs.has(sig) || thisSigs.has(sig))) {
- console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — duplicate scene ("${sig}")`);
+ const why0 = registry.usedReason({ vod_id: v.vod_id, stream_id: v.stream_id, offset: moment.offset, sig });
+ if (why0 || (sig && thisSigs.has(sig))) {
+ console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — ${why0 || `duplicate scene ("${sig}")`}`);
continue;
continue;
}
}
if (sig) thisSigs.add(sig);
if (sig) thisSigs.add(sig);
@@ -390,7 +423,7 @@ async function tick(opts = {}) {
@@ -390,7 +423,7 @@ async function tick(opts = {}) {
// Re-check the signature against the FINAL (vision-verified) description — this is
// Re-check the signature against the FINAL (vision-verified) description — this is
// what actually catches same-scene duplicates (e.g. a split-image close-up re-picked).
// what actually catches same-scene duplicates (e.g. a split-image close-up re-picked).
const finalSig = _sig(desc);
const finalSig = _sig(desc);
- if (finalSig && finalSig !== sig && (usedSigs.has(finalSig) || thisSigs.has(finalSig))) {
+ if (finalSig && finalSig !== sig && (registry.isUsed({ sig: finalSig }) || thisSigs.has(finalSig))) {
console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — duplicate scene after vision ("${finalSig}")`);
console.log(`[AI-Moments] Skipped VOD ${v.vod_id} — duplicate scene after vision ("${finalSig}")`);
continue;
continue;
}
}
@@ -423,19 +456,23 @@ async function tick(opts = {}) {
@@ -423,19 +456,23 @@ async function tick(opts = {}) {
try { fs.unlinkSync(screenshotPath); } catch { /* tmp frame */ }
try { fs.unlinkSync(screenshotPath); } catch { /* tmp frame */ }
moments.push({ vodId: v.vod_id, offset, title: title.slice(0, 80), thumbnail: img, username: v.username, pasteSlug });
moments.push({ vodId: v.vod_id, offset, title: title.slice(0, 80), thumbnail: img, username: v.username, pasteSlug });
+ registry.record({ kind: 'paste', vod_id: v.vod_id, stream_id: v.stream_id, offset, sig: finalSig || sig, title });
- // Also cut a real VOD clip around this same moment (AI auto-clip), unless disabled.
+ // Also cut a VOD clip from the SAME VOD — but a DIFFERENT beat (clip flavor, ≥ 2 min away),
+ // so the clip and the paste never show the identical second with the identical title.
if (opts.clip !== false && momentSource) {
if (opts.clip !== false && momentSource) {
try {
try {
- const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset, title, desc, source: momentSource });
- if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} ("${title.slice(0, 60)}")`);
+ const second = await _findBestMoment(v, { flavor: 'clip', avoid: [offset] });
+ if (second && !registry.isUsed({ vod_id: v.vod_id, stream_id: v.stream_id, offset: second.offset, desc: second.desc, title: second.title })) {
+ const clip = await require('./auto-clip-job').clipVodMoment({ vod: v, offset: second.offset, title: second.title, desc: second.desc, source: momentSource });
+ if (clip) console.log(`[AI-Moments] Auto-clip cut for VOD ${v.vod_id} at ${_mmss(second.offset)} ("${String(second.title || '').slice(0, 60)}") — paste was at ${_mmss(offset)}`);
+ } else console.log(`[AI-Moments] VOD ${v.vod_id}: no second, distinct beat for a clip — skipped`);
} catch { /* clipping is best-effort */ }
} catch { /* clipping is best-effort */ }
}
}
}
}
const usedLog = [...newUsedVods, ...(prev.usedVods || [])].slice(0, 300);
const usedLog = [...newUsedVods, ...(prev.usedVods || [])].slice(0, 300);
- const sigLog = [...thisSigs, ...(prev.usedSigs || [])].slice(0, 120);
- db.setState(SETTING, JSON.stringify({ moments, usedVods: usedLog, usedSigs: sigLog, updated_at: Date.now() }));
+ db.setState(SETTING, JSON.stringify({ moments, usedVods: usedLog, rankCache: _rankCacheOut, updated_at: Date.now() }));
console.log(`[AI-Moments] ${moments.length} moment(s) from ${chosen.length}/${totalEligible} ranked VODs + pastes created`);
console.log(`[AI-Moments] ${moments.length} moment(s) from ${chosen.length}/${totalEligible} ranked VODs + pastes created`);
} catch (e) {
} catch (e) {
console.warn('[AI-Moments] tick error:', e.message);
console.warn('[AI-Moments] tick error:', e.message);
@@ -456,7 +493,7 @@ function start() {
@@ -456,7 +493,7 @@ function start() {
setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);
setInterval(() => { tick().catch(() => {}); }, 5 * 60 * 1000);
}
}
-module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark };
+module.exports = { start, tick, findBestMoment: _findBestMoment, frameTooDark: _frameTooDark, makeSlug: _slug };
// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:
// CLI: force a one-off regeneration, e.g. a whole-dataset "best of all-time" test run:
// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2
// node server/ai/ai-moments-job.js --fresh --target=6 --perUser=2
@@ -19,6 +19,7 @@ const db = require('../db/database');
@@ -19,6 +19,7 @@ const db = require('../db/database');
const ai = require('./ai-analysis');
const ai = require('./ai-analysis');
const media = require('../media-client');
const media = require('../media-client');
const recorder = require('../streaming/recorder');
const recorder = require('../streaming/recorder');
+const registry = require('./moment-registry');
// ── Tunables ────────────────────────────────────────────────
// ── Tunables ────────────────────────────────────────────────
const CHECK_INTERVAL_MS = 90 * 1000;
const CHECK_INTERVAL_MS = 90 * 1000;
@@ -49,6 +50,8 @@ function _logClip(entry) {
@@ -49,6 +50,8 @@ function _logClip(entry) {
const log = _clipLog();
const log = _clipLog();
log.unshift({ ...entry, ts: Date.now() });
log.unshift({ ...entry, ts: Date.now() });
db.setState(CLIP_LOG_SETTING, JSON.stringify(log.slice(0, 300)));
db.setState(CLIP_LOG_SETTING, JSON.stringify(log.slice(0, 300)));
+ // Shared with the paste job, so a paste never lands on a clipped second (and vice versa).
+ registry.record({ kind: 'clip', vod_id: entry.vod_id || null, stream_id: entry.stream_id || null, offset: (entry.start_time || 0) + CLIP_PRE, sig: entry.sig || null, title: entry.title || null });
} catch { /* */ }
} catch { /* */ }
}
}
function _countClipsSince(streamId, minutes) {
function _countClipsSince(streamId, minutes) {
@@ -241,7 +244,8 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
@@ -241,7 +244,8 @@ async function backfillVodClips({ limit = BACKFILL_PER_RUN, force = false } = {}
try {
try {
const source = await _resolveVodSource(v.vod_id);
const source = await _resolveVodSource(v.vod_id);
if (!source) { newSkip.push(v.vod_id); continue; } // media gone / unreadable
if (!source) { newSkip.push(v.vod_id); continue; } // media gone / unreadable
- const moment = await moments.findBestMoment(v);
+ // Clip flavor (a beat, not a frame) and never within 2 min of a moment a paste already used.
+ const moment = await moments.findBestMoment(v, { flavor: 'clip', avoid: registry.usedOffsets(v.vod_id, v.stream_id) });
if (!moment) { newSkip.push(v.vod_id); continue; }
if (!moment) { newSkip.push(v.vod_id); continue; }
// Pixel backstop: don't clip a black/empty frame.
// Pixel backstop: don't clip a black/empty frame.
try {
try {
@@ -276,14 +280,8 @@ function _sceneSig(text) {
@@ -276,14 +280,8 @@ function _sceneSig(text) {
function _isDuplicateAutoClip(vod, offset, desc, title) {
function _isDuplicateAutoClip(vod, offset, desc, title) {
try {
try {
const vId = vod.vod_id || vod.id;
const vId = vod.vod_id || vod.id;
- const sig = _sceneSig(desc || title);
- const start = Math.max(0, Math.floor(offset) - CLIP_PRE);
- const cutoff = Date.now() - 48 * 3600_000;
- for (const c of _clipLog()) {
- if (c.ts < cutoff) continue;
- if ((c.vod_id === vId || c.stream_id === vod.stream_id) && Math.abs((c.start_time || 0) - start) < 45) return true;
- if (sig && c.sig === sig) return true;
- }
+ const why = registry.usedReason({ vod_id: vId, stream_id: vod.stream_id, offset: Math.floor(offset), sig: _sceneSig(desc || title) });
+ if (why) { console.log(`[AutoClip] VOD ${vId}: ${why}`); return true; }
} catch { /* */ }
} catch { /* */ }
return false;
return false;
}
}
@@ -65,7 +65,8 @@ function sessionBlock(stream) {
@@ -65,7 +65,8 @@ function sessionBlock(stream) {
if (!stream) return '';
if (!stream) return '';
const parts = [];
const parts = [];
const title = stream.ai_title || stream.title;
const title = stream.ai_title || stream.title;
- if (title) parts.push(`Stream title: "${clip(title, 140)}"${stream.category ? ` (${clip(stream.category, 40)})` : ''}`);
+ const cat = stream.ai_category || stream.category;
+ if (title) parts.push(`Stream title: "${clip(title, 140)}"${cat ? ` (category: ${clip(cat, 40)}${stream.ai_category ? ', judged from the stream itself' : ', self-selected'})` : ''}`);
if (stream.ai_overview) parts.push(`What this session has been about so far: ${clip(stream.ai_overview, 500)}`);
if (stream.ai_overview) parts.push(`What this session has been about so far: ${clip(stream.ai_overview, 500)}`);
return parts.join('\n');
return parts.join('\n');
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,79 @@
@@ -0,0 +1,79 @@
+/**
+ * moment-registry.js — ONE shared record of every stream moment the AI has already turned into
+ * something (a paste, a clip), so the paste job and the clip job never pick the same spot or the
+ * same scene twice. Replaces the two separate logs (`home_hero_moments.usedSigs` and
+ * `auto_clip_log[].sig`) that could not see each other — which is how a paste and a clip ended
+ * up with the identical title from the identical second of the same VOD.
+ *
+ * record({ kind, vod_id, stream_id, offset, sig, title }) after a paste/clip is created
+ * isUsed({ vod_id, stream_id, offset, sig }) same VOD within GAP_SEC, or same scene
+ * usedOffsets(vod_id, stream_id) → [sec…] to tell the picker to avoid
+ *
+ * State key `ai_used_moments` (capped). Legacy logs are imported once on first use.
+ */
+'use strict';
+
+const db = require('../db/database');
+
+const KEY = 'ai_used_moments';
+const MAX = 800;
+const GAP_SEC = 120; // two moments on the same VOD must be at least this far apart
+const SIG_TTL_MS = 14 * 24 * 3600_000; // a scene signature blocks re-use for two weeks
+const OFFSET_TTL_MS = 90 * 24 * 3600_000;
+
+let _cache = null;
+function _load() {
+ if (_cache) return _cache;
+ let list = [];
+ try { const l = JSON.parse(db.getState(KEY) || '[]'); if (Array.isArray(l)) list = l; } catch { /* */ }
+ if (!list.length) list = _importLegacy();
+ _cache = list;
+ return list;
+}
+function _save(list) { _cache = list.slice(0, MAX); try { db.setState(KEY, JSON.stringify(_cache)); } catch { /* */ } }
+
+function _importLegacy() {
+ const out = [];
+ try { for (const c of JSON.parse(db.getState('auto_clip_log') || '[]') || []) out.push({ kind: 'clip', vod_id: c.vod_id || null, stream_id: c.stream_id || null, offset: (c.start_time || 0) + 16, sig: c.sig || null, title: c.title || null, ts: c.ts || Date.now() }); } catch { /* */ }
+ try { const h = JSON.parse(db.getState('home_hero_moments') || '{}') || {}; for (const m of h.moments || []) out.push({ kind: 'paste', vod_id: m.vodId || null, stream_id: null, offset: m.offset || 0, sig: sig(m.title), title: m.title || null, ts: h.updated_at || Date.now() }); for (const s of h.usedSigs || []) out.push({ kind: 'paste', vod_id: null, stream_id: null, offset: null, sig: s, title: null, ts: h.updated_at || Date.now() }); } catch { /* */ }
+ return out.slice(0, MAX);
+}
+
+/** Coarse scene signature — the first five significant words of a description/title. */
+function sig(text) {
+ return String(text || '').toLowerCase().replace(/[^a-z0-9 ]+/g, ' ').replace(/\s+/g, ' ').trim().split(' ').filter(Boolean).slice(0, 5).join(' ');
+}
+
+function record({ kind, vod_id = null, stream_id = null, offset = null, sig: s = null, title = null, desc = null }) {
+ const list = _load();
+ list.unshift({ kind: kind || 'moment', vod_id: vod_id || null, stream_id: stream_id || null, offset: offset == null ? null : Math.floor(offset), sig: s || sig(desc || title), title: title ? String(title).slice(0, 120) : null, ts: Date.now() });
+ _save(list);
+}
+
+/** Why a moment is blocked, or null when it is free. */
+function usedReason({ vod_id = null, stream_id = null, offset = null, sig: s = null, desc = null, title = null } = {}, { gapSec = GAP_SEC } = {}) {
+ const now = Date.now();
+ const scene = s || sig(desc || title);
+ for (const e of _load()) {
+ const age = now - (e.ts || 0);
+ if (age > OFFSET_TTL_MS) continue;
+ if (offset != null && e.offset != null && ((vod_id && e.vod_id === vod_id) || (stream_id && e.stream_id === stream_id)) && Math.abs(e.offset - offset) < gapSec) return `${e.kind} already made at ${Math.floor(e.offset)}s of this ${e.vod_id === vod_id ? 'VOD' : 'stream'}`;
+ if (scene && e.sig && e.sig === scene && age <= SIG_TTL_MS) return `same scene as a recent ${e.kind} ("${scene}")`;
+ }
+ return null;
+}
+function isUsed(m, o) { return !!usedReason(m, o); }
+
+/** Offsets already used on this VOD/stream (for the picker's "avoid these" list). */
+function usedOffsets(vod_id = null, stream_id = null) {
+ const now = Date.now();
+ return [...new Set(_load().filter(e => e.offset != null && now - (e.ts || 0) <= OFFSET_TTL_MS && ((vod_id && e.vod_id === vod_id) || (stream_id && e.stream_id === stream_id))).map(e => Math.floor(e.offset)))].sort((a, b) => a - b);
+}
+
+function lastOfKind(kind, { stream_id = null, vod_id = null } = {}) {
+ return _load().find(e => e.kind === kind && (!stream_id || e.stream_id === stream_id) && (!vod_id || e.vod_id === vod_id)) || null;
+}
+
+function recent(hours = 48) { const cut = Date.now() - hours * 3600_000; return _load().filter(e => (e.ts || 0) >= cut); }
+
+module.exports = { record, isUsed, usedReason, usedOffsets, lastOfKind, recent, sig, GAP_SEC, _reset: () => { _cache = null; } };
@@ -155,8 +155,9 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =
@@ -155,8 +155,9 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =
const stale = (now - last.at) >= SUMMARY_MAX_AGE_MS;
const stale = (now - last.at) >= SUMMARY_MAX_AGE_MS;
if (memories.length > 1 && (!last.at || grew || stale)) {
if (memories.length > 1 && (!last.at || grew || stale)) {
const summary = await ai.summarizeStreamMemories(memories, stream.id);
const summary = await ai.summarizeStreamMemories(memories, stream.id);
- if (summary) {
- db.updateStreamAiOverview(stream.id, summary);
+ if (summary && summary.overview) {
+ db.updateStreamAiOverview(stream.id, summary.overview);
+ if (summary.category) db.setStreamAiCategory(stream.id, summary.category, summary.tags);
_lastSummary.set(stream.id, { count: memories.length, at: now });
_lastSummary.set(stream.id, { count: memories.length, at: now });
}
}
} else if (!stream.ai_overview) {
} else if (!stream.ai_overview) {
@@ -165,6 +166,41 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =
@@ -165,6 +166,41 @@ async function _analyzeOne(stream, { allowFfmpeg = true, reason = 'periodic' } =
}
}
} catch { /* keep existing overview */ }
} catch { /* keep existing overview */ }
} catch (e) { console.warn('[AI] memory store failed:', e.message); }
} catch (e) { console.warn('[AI] memory store failed:', e.message); }
+
+ // Live paste: when the vision model flags THIS frame as screenshot-worthy, post it as an image
+ // paste right now (no extra AI call — the caption came with the memory). Rate-limited per stream
+ // and deduped through the shared moment registry so the paste job / clip job never re-use it.
+ try { await _maybeLivePaste(stream, image, r, offset); } catch (e) { console.warn('[AI] live paste:', e.message); }
+}
+
+const LIVE_PASTE_MIN_GAP_MS = 90 * 60 * 1000;
+async function _maybeLivePaste(stream, image, r, offset) {
+ if (!r || !r.worthy || !r.title || !image) return;
+ try { if (String(db.getSetting('ai_live_pastes_enabled') ?? 'true') === 'false') return; } catch { /* default on */ }
+ const registry = require('./moment-registry');
+ const last = registry.lastOfKind('paste', { stream_id: stream.id });
+ if (last && Date.now() - (last.ts || 0) < LIVE_PASTE_MIN_GAP_MS) return;
+ const why = registry.usedReason({ stream_id: stream.id, offset, desc: r.description, title: r.title });
+ if (why) return;
+ const moments = require('./ai-moments-job');
+ if (typeof image === 'string' && !image.startsWith('data:') && moments.frameTooDark && await moments.frameTooDark(image)) return;
+ let buffer = null;
+ try { buffer = Buffer.isBuffer(image) ? image : (typeof image === 'string' && image.startsWith('data:') ? Buffer.from(image.split(',')[1] || '', 'base64') : require('fs').readFileSync(image)); } catch { return; }
+ if (!buffer || buffer.length < 2000) return;
+ const username = stream.username || (db.getUserById(stream.user_id) || {}).username || 'someone';
+ let base = 'https://openvibe.live'; try { const c = require('../config'); base = String(c.baseUrl || c.publicUrl || base).replace(/\/$/, ''); } catch { /* */ }
+ const media = require('../media-client');
+ const paste = await media.createPaste({
+ slug: moments.makeSlug ? moments.makeSlug() : undefined, user_id: stream.user_id,
+ title: r.title.slice(0, 80), content: `${r.description}\n\n🔴 Caught live on @${username}'s stream — ${base}/${username}`, language: 'text', visibility: 'public',
+ stream_id: stream.id, metadata: JSON.stringify({ ai_moment: true, live: true, stream_id: stream.id, offset, username, vod_link: null }),
+ ai_summary: r.description, ai_tags: JSON.stringify(Array.isArray(r.tags) ? r.tags.slice(0, 8) : []),
+ screenshot: { buffer, filename: `live-${stream.id}-${offset}.jpg`, contentType: 'image/jpeg' },
+ });
+ if (paste) {
+ registry.record({ kind: 'paste', stream_id: stream.id, offset, desc: r.description, title: r.title });
+ console.log(`[AI-Moments] live paste for stream ${stream.id} at ${offset}s: "${r.title}"`);
+ }
}
}
async function tick() {
async function tick() {
@@ -82,7 +82,7 @@ async function _processSegment(stream, seg) {
@@ -82,7 +82,7 @@ async function _processSegment(stream, seg) {
try {
try {
const sounds = require('./audio-events');
const sounds = require('./audio-events');
if (sounds.available()) {
if (sounds.available()) {
- const cat = stream.category || null;
+ const cat = stream.ai_category || stream.category || null;
const detected = await sounds.detect(seg.path, { offsetSec: seg.offsetSec, category: cat });
const detected = await sounds.detect(seg.path, { offsetSec: seg.offsetSec, category: cat });
for (const d of (detected || [])) {
for (const d of (detected || [])) {
events.push({
events.push({
@@ -175,7 +175,7 @@ function rawStatsFor(userId) {
@@ -175,7 +175,7 @@ function rawStatsFor(userId) {
const allTime = db.get(`SELECT COUNT(*) AS streams, COALESCE(SUM(duration_seconds), 0) / 3600.0 AS hours, COALESCE(MAX(peak_viewers), 0) AS peak_viewers FROM streams WHERE user_id = ? AND duration_seconds > 0`, [userId]) || {};
const allTime = db.get(`SELECT COUNT(*) AS streams, COALESCE(SUM(duration_seconds), 0) / 3600.0 AS hours, COALESCE(MAX(peak_viewers), 0) AS peak_viewers FROM streams WHERE user_id = ? AND duration_seconds > 0`, [userId]) || {};
const followers = db.get('SELECT COUNT(*) AS n FROM follows WHERE streamer_id = ?', [userId])?.n || 0;
const followers = db.get('SELECT COUNT(*) AS n FROM follows WHERE streamer_id = ?', [userId])?.n || 0;
const tips = db.get(`SELECT COALESCE(SUM(amount), 0) AS n FROM transactions WHERE to_user_id = ? AND type = 'donation' AND created_at >= datetime('now', ?)`, [userId, win])?.n || 0;
const tips = db.get(`SELECT COALESCE(SUM(amount), 0) AS n FROM transactions WHERE to_user_id = ? AND type = 'donation' AND created_at >= datetime('now', ?)`, [userId, win])?.n || 0;
- const category = db.get(`SELECT category, COUNT(*) AS n FROM streams WHERE user_id = ? AND duration_seconds > 0 AND category IS NOT NULL AND category != '' GROUP BY category ORDER BY n DESC LIMIT 1`, [userId])?.category || null;
+ const category = db.get(`SELECT COALESCE(NULLIF(ai_category, ''), category) AS category, COUNT(*) AS n FROM streams WHERE user_id = ? AND duration_seconds > 0 AND COALESCE(NULLIF(ai_category, ''), category) IS NOT NULL AND COALESCE(NULLIF(ai_category, ''), category) != '' GROUP BY 1 ORDER BY n DESC LIMIT 1`, [userId])?.category || null;
const hours = Math.max(Number(agg.hours) || 0, 0.1);
const hours = Math.max(Number(agg.hours) || 0, 0.1);
return {
return {
window_days: STATS_WINDOW_DAYS, streams: agg.streams || 0, hours: Number((agg.hours || 0).toFixed(1)), peak_viewers: agg.peak_viewers || 0,
window_days: STATS_WINDOW_DAYS, streams: agg.streams || 0, hours: Number((agg.hours || 0).toFixed(1)), peak_viewers: agg.peak_viewers || 0,
@@ -1345,6 +1345,12 @@ function initDb() {
@@ -1345,6 +1345,12 @@ function initDb() {
if (!scols.includes('ai_overview')) database.exec('ALTER TABLE streams ADD COLUMN ai_overview TEXT');
if (!scols.includes('ai_overview')) database.exec('ALTER TABLE streams ADD COLUMN ai_overview TEXT');
// Short AI-generated session title for the AI Timeline (streamers reuse literal titles).
// Short AI-generated session title for the AI Timeline (streamers reuse literal titles).
if (!scols.includes('ai_title')) database.exec('ALTER TABLE streams ADD COLUMN ai_title TEXT');
if (!scols.includes('ai_title')) database.exec('ALTER TABLE streams ADD COLUMN ai_title TEXT');
+ // Category/tags inferred by the AI from what the stream actually shows (the go-live selector
+ // used to default to 'irl', which made every AI think everyone is an IRL streamer).
+ if (!scols.includes('ai_category')) database.exec('ALTER TABLE streams ADD COLUMN ai_category TEXT');
+ if (!scols.includes('ai_tags')) database.exec('ALTER TABLE streams ADD COLUMN ai_tags TEXT');
+ const chcols = database.prepare('PRAGMA table_info(channels)').all().map(c => c.name);
+ if (!chcols.includes('ai_category')) database.exec('ALTER TABLE channels ADD COLUMN ai_category TEXT');
// AI overview + transcript on VODs and clips.
// AI overview + transcript on VODs and clips.
const vcols = database.prepare('PRAGMA table_info(vods)').all().map(c => c.name);
const vcols = database.prepare('PRAGMA table_info(vods)').all().map(c => c.name);
@@ -1395,6 +1401,16 @@ function initDb() {
@@ -1395,6 +1401,16 @@ function initDb() {
}
}
} catch (e) { console.warn('[DB] visibility migration:', e.message); }
} catch (e) { console.warn('[DB] visibility migration:', e.message); }
+ // AI-inferred category/tags. The go-live selector used to default to 'irl', which taught
+ // every downstream AI (viewers, overviews, arena) that everyone is an IRL streamer; the
+ // stream-memory rollup now classifies each stream from what it actually shows.
+ try {
+ const sc = database.prepare('PRAGMA table_info(streams)').all().map(c => c.name);
+ for (const col of ['ai_overview TEXT', 'ai_title TEXT', 'ai_category TEXT', 'ai_tags TEXT']) { if (!sc.includes(col.split(' ')[0])) database.exec(`ALTER TABLE streams ADD COLUMN ${col}`); }
+ const cc = database.prepare('PRAGMA table_info(channels)').all().map(c => c.name);
+ if (!cc.includes('ai_category')) database.exec('ALTER TABLE channels ADD COLUMN ai_category TEXT');
+ } catch (e) { console.warn('[DB] ai-category migration:', e.message); }
+
// Cached SHORT overview (a concise, locally-derived version of the long AI
// Cached SHORT overview (a concise, locally-derived version of the long AI
// overview) shown on listing cards — computed once at write time, no extra API.
// overview) shown on listing cards — computed once at write time, no extra API.
try {
try {
@@ -2676,7 +2692,7 @@ function getOrCreateAnonGameUser(anonId) {
@@ -2676,7 +2692,7 @@ function getOrCreateAnonGameUser(anonId) {
function getLiveStreams() {
function getLiveStreams() {
return all(`
return all(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
ms.slug AS managed_stream_slug, ms.id AS managed_stream_id,
ms.slug AS managed_stream_slug, ms.id AS managed_stream_id,
ms.stream_key AS managed_stream_key,
ms.stream_key AS managed_stream_key,
ms.browser_mode, ms.streaming_method
ms.browser_mode, ms.streaming_method
@@ -2690,7 +2706,7 @@ function getLiveStreams() {
@@ -2690,7 +2706,7 @@ function getLiveStreams() {
function getRecentStreams(limit = 20) {
function getRecentStreams(limit = 20) {
return all(`
return all(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
v.id AS vod_id, v.is_public AS vod_is_public, v.thumbnail_url AS vod_thumbnail_url,
v.id AS vod_id, v.is_public AS vod_is_public, v.thumbnail_url AS vod_thumbnail_url,
v.duration_seconds AS vod_duration
v.duration_seconds AS vod_duration
FROM streams s
FROM streams s
@@ -2710,7 +2726,7 @@ function getRecentStreams(limit = 20) {
@@ -2710,7 +2726,7 @@ function getRecentStreams(limit = 20) {
function getStreamById(id) {
function getStreamById(id) {
return get(`
return get(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.title AS managed_stream_title, ms.protocol AS managed_stream_protocol
ms.title AS managed_stream_title, ms.protocol AS managed_stream_protocol
FROM streams s
FROM streams s
@@ -2729,7 +2745,7 @@ function getStreamByUserId(userId) {
@@ -2729,7 +2745,7 @@ function getStreamByUserId(userId) {
function getLiveStreamsByUserId(userId) {
function getLiveStreamsByUserId(userId) {
return all(`
return all(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.title AS managed_stream_title
ms.title AS managed_stream_title
FROM streams s
FROM streams s
@@ -2742,7 +2758,7 @@ function getLiveStreamsByUserId(userId) {
@@ -2742,7 +2758,7 @@ function getLiveStreamsByUserId(userId) {
function getLiveStreamsByControlConfigId(controlConfigId) {
function getLiveStreamsByControlConfigId(controlConfigId) {
return all(`
return all(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.title AS managed_stream_title
ms.title AS managed_stream_title
FROM streams s
FROM streams s
@@ -2755,7 +2771,7 @@ function getLiveStreamsByControlConfigId(controlConfigId) {
@@ -2755,7 +2771,7 @@ function getLiveStreamsByControlConfigId(controlConfigId) {
function getStreamsByUserId(userId, limit = 50) {
function getStreamsByUserId(userId, limit = 50) {
return all(`
return all(`
- SELECT s.*, u.username, u.display_name, u.avatar_url, u.profile_color,
+ SELECT s.*, COALESCE(NULLIF(s.ai_category, ''), s.category) AS category, s.category AS chosen_category, u.username, u.display_name, u.avatar_url, u.profile_color,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.slug AS managed_stream_slug, ms.stream_key AS managed_stream_key,
ms.title AS managed_stream_title, ms.id AS managed_stream_ref_id
ms.title AS managed_stream_title, ms.id AS managed_stream_ref_id
FROM streams s
FROM streams s
@@ -2785,7 +2801,7 @@ function createStream({ user_id, channel_id, managed_stream_id, control_config_i
@@ -2785,7 +2801,7 @@ function createStream({ user_id, channel_id, managed_stream_id, control_config_i
return run(
return run(
`INSERT INTO streams (user_id, channel_id, managed_stream_id, control_config_id, title, description, category, protocol, is_nsfw, thumbnail_url, is_live, started_at)
`INSERT INTO streams (user_id, channel_id, managed_stream_id, control_config_id, title, description, category, protocol, is_nsfw, thumbnail_url, is_live, started_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)`,
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)`,
- [user_id, channel_id || null, managed_stream_id || null, control_config_id || null, title || 'Untitled Stream', description || '', category || 'irl', protocol || 'webrtc', is_nsfw ? 1 : 0, thumbnail_url || null]
+ [user_id, channel_id || null, managed_stream_id || null, control_config_id || null, title || 'Untitled Stream', description || '', category || null, protocol || 'webrtc', is_nsfw ? 1 : 0, thumbnail_url || null]
);
);
}
}
@@ -2856,6 +2872,15 @@ function _shortOverview(text) {
@@ -2856,6 +2872,15 @@ function _shortOverview(text) {
function updateStreamAiOverview(streamId, text) {
function updateStreamAiOverview(streamId, text) {
return run('UPDATE streams SET ai_overview = ?, ai_overview_short = ? WHERE id = ?', [text || null, _shortOverview(text), streamId]);
return run('UPDATE streams SET ai_overview = ?, ai_overview_short = ? WHERE id = ?', [text || null, _shortOverview(text), streamId]);
}
}
+/** AI-inferred category/tags for a stream; the channel inherits the latest inferred category. */
+function setStreamAiCategory(streamId, category, tags) {
+ const cat = category ? String(category).toLowerCase().slice(0, 40) : null;
+ const r = run('UPDATE streams SET ai_category = ?, ai_tags = ? WHERE id = ?', [cat, Array.isArray(tags) && tags.length ? JSON.stringify(tags.slice(0, 8)) : null, streamId]);
+ if (cat) { const s = get('SELECT user_id FROM streams WHERE id = ?', [streamId]); if (s) run('UPDATE channels SET ai_category = ? WHERE user_id = ?', [cat, s.user_id]); }
+ return r;
+}
+/** What to call a stream's category: the AI's read of the stream beats the self-selected default. */
+function effectiveCategory(row) { if (!row) return null; return row.ai_category || row.category || null; }
// AI overview/transcript state lives in vod_ai_state / clip_ai_state (Live-owned,
// AI overview/transcript state lives in vod_ai_state / clip_ai_state (Live-owned,
// keyed by the Media vod/clip id) — the moved vods/clips tables are never written.
// keyed by the Media vod/clip id) — the moved vods/clips tables are never written.
function _ensureVodAiState(vodId) {
function _ensureVodAiState(vodId) {
@@ -3528,7 +3553,7 @@ function createManagedStream({ user_id, channel_id, slug, title, description, ca
@@ -3528,7 +3553,7 @@ function createManagedStream({ user_id, channel_id, slug, title, description, ca
return run(
return run(
`INSERT INTO managed_streams (user_id, channel_id, slug, title, description, category, protocol, streaming_method, stream_key, is_nsfw, control_config_id)
`INSERT INTO managed_streams (user_id, channel_id, slug, title, description, category, protocol, streaming_method, stream_key, is_nsfw, control_config_id)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
- [user_id, channel_id || null, slug || null, title || 'Untitled Stream', description || '', category || 'irl', protocol || 'webrtc', streaming_method || null, stream_key, is_nsfw ? 1 : 0, control_config_id || null]
+ [user_id, channel_id || null, slug || null, title || 'Untitled Stream', description || '', category || null, protocol || 'webrtc', streaming_method || null, stream_key, is_nsfw ? 1 : 0, control_config_id || null]
);
);
}
}
@@ -4224,7 +4249,7 @@ function createChannel({ user_id, title, description, category, protocol }) {
@@ -4224,7 +4249,7 @@ function createChannel({ user_id, title, description, category, protocol }) {
return run(
return run(
`INSERT OR IGNORE INTO channels (user_id, title, description, category, protocol)
`INSERT OR IGNORE INTO channels (user_id, title, description, category, protocol)
VALUES (?, ?, ?, ?, ?)`,
VALUES (?, ?, ?, ?, ?)`,
- [user_id, title || 'Untitled Channel', description || '', category || 'irl', protocol || 'webrtc']
+ [user_id, title || 'Untitled Channel', description || '', category || null, protocol || 'webrtc']
);
);
}
}
@@ -7999,7 +8024,7 @@ module.exports = {
@@ -7999,7 +8024,7 @@ module.exports = {
isValidManagedStreamSlug, isManagedStreamSlugTaken,
isValidManagedStreamSlug, isManagedStreamSlugTaken,
ensureStreamerRoleOnFeed,
ensureStreamerRoleOnFeed,
// Streams (sessions)
// Streams (sessions)
- getLiveStreams, getRecentStreams, getStreamById, getStreamByUserId, getLiveStreamsByUserId, getLiveStreamsByControlConfigId, getStreamsByUserId, getStreamHistoryByManagedStream,
+ getLiveStreams, getRecentStreams, getStreamById, setStreamAiCategory, effectiveCategory, getStreamByUserId, getLiveStreamsByUserId, getLiveStreamsByControlConfigId, getStreamsByUserId, getStreamHistoryByManagedStream,
createStream, endStream, endOtherLiveStreamsForSlot, updateViewerCount,
createStream, endStream, endOtherLiveStreamsForSlot, updateViewerCount,
addStreamMemory, getStreamMemories, getLatestStreamMemory, updateStreamAiOverview,
addStreamMemory, getStreamMemories, getLatestStreamMemory, updateStreamAiOverview,
setVodAiOverview, setClipAiOverview, setVodTranscript, setClipTranscript, getStreamMemoriesInRange,
setVodAiOverview, setClipAiOverview, setVodTranscript, setClipTranscript, getStreamMemoriesInRange,
@@ -204,7 +204,7 @@ router.get('/:id', optionalAuth, async (req, res) => {
@@ -204,7 +204,7 @@ router.get('/:id', optionalAuth, async (req, res) => {
clip.stream_started_at = stream.started_at;
clip.stream_started_at = stream.started_at;
clip.stream_ended_at = stream.ended_at;
clip.stream_ended_at = stream.ended_at;
clip.stream_title = stream.title;
clip.stream_title = stream.title;
- clip.stream_category = stream.category;
+ clip.stream_category = stream.ai_category || stream.category;
clip.stream_peak_viewers = stream.peak_viewers;
clip.stream_peak_viewers = stream.peak_viewers;
clip.stream_protocol = stream.protocol;
clip.stream_protocol = stream.protocol;
}
}
@@ -351,7 +351,7 @@ router.get('/:id', optionalAuth, async (req, res) => {
@@ -351,7 +351,7 @@ router.get('/:id', optionalAuth, async (req, res) => {
if (stream) {
if (stream) {
vod.stream_started_at = stream.started_at;
vod.stream_started_at = stream.started_at;
vod.stream_ended_at = stream.ended_at;
vod.stream_ended_at = stream.ended_at;
- vod.stream_category = stream.category;
+ vod.stream_category = stream.ai_category || stream.category;
vod.stream_peak_viewers = stream.peak_viewers;
vod.stream_peak_viewers = stream.peak_viewers;
}
}
}
}
@@ -1446,7 +1446,7 @@ router.post('/managed', requireAuth, (req, res) => {
@@ -1446,7 +1446,7 @@ router.post('/managed', requireAuth, (req, res) => {
const title = cleanText(req.body.title, { maxLength: MAX_TITLE_LENGTH }) || 'Untitled Stream';
const title = cleanText(req.body.title, { maxLength: MAX_TITLE_LENGTH }) || 'Untitled Stream';
const description = cleanText(req.body.description, { maxLength: MAX_DESCRIPTION_LENGTH, allowEmpty: true }) || '';
const description = cleanText(req.body.description, { maxLength: MAX_DESCRIPTION_LENGTH, allowEmpty: true }) || '';
- const category = cleanText(req.body.category, { maxLength: MAX_CATEGORY_LENGTH }) || 'irl';
+ const category = cleanText(req.body.category, { maxLength: MAX_CATEGORY_LENGTH }) || null; // empty = let the AI infer it from the stream
const protocol = cleanProtocol(req.body.protocol) || 'webrtc';
const protocol = cleanProtocol(req.body.protocol) || 'webrtc';
const is_nsfw = cleanBooleanFlag(req.body.is_nsfw);
const is_nsfw = cleanBooleanFlag(req.body.is_nsfw);
let slug = req.body.slug ? req.body.slug.trim().toLowerCase() : null;
let slug = req.body.slug ? req.body.slug.trim().toLowerCase() : null;
@@ -1783,7 +1783,7 @@ router.post('/', requireAuth, (req, res) => {
@@ -1783,7 +1783,7 @@ router.post('/', requireAuth, (req, res) => {
// Use managed stream's settings as defaults, allow per-session overrides
// Use managed stream's settings as defaults, allow per-session overrides
const streamProtocol = protocol || cleanProtocol(managedStream.protocol) || cleanProtocol(channel.protocol) || 'webrtc';
const streamProtocol = protocol || cleanProtocol(managedStream.protocol) || cleanProtocol(channel.protocol) || 'webrtc';
- const streamCategory = category || cleanText(managedStream.category, { maxLength: MAX_CATEGORY_LENGTH }) || cleanText(channel.category, { maxLength: MAX_CATEGORY_LENGTH }) || 'irl';
+ const streamCategory = category || cleanText(managedStream.category, { maxLength: MAX_CATEGORY_LENGTH }) || cleanText(channel.category, { maxLength: MAX_CATEGORY_LENGTH }) || null; // null → the AI classifies the stream from what it shows
const requestedControlConfigId = req.body.control_config_id !== undefined ? (req.body.control_config_id === null ? null : parseInt(req.body.control_config_id)) : undefined;
const requestedControlConfigId = req.body.control_config_id !== undefined ? (req.body.control_config_id === null ? null : parseInt(req.body.control_config_id)) : undefined;
if (requestedControlConfigId !== undefined && requestedControlConfigId !== null) {
if (requestedControlConfigId !== undefined && requestedControlConfigId !== null) {
@@ -578,7 +578,7 @@ function autoCreateWhipSession(managedStream, user) {
@@ -578,7 +578,7 @@ function autoCreateWhipSession(managedStream, user) {
managed_stream_id: managedStream.id,
managed_stream_id: managedStream.id,
title: managedStream.title || `${user.display_name || user.username}'s Stream`,
title: managedStream.title || `${user.display_name || user.username}'s Stream`,
description: managedStream.description || '',
description: managedStream.description || '',
- category: managedStream.category || 'irl',
+ category: managedStream.category || null, // null → the AI classifies the stream from what it shows
protocol: 'webrtc',
protocol: 'webrtc',
is_nsfw: managedStream.is_nsfw || 0,
is_nsfw: managedStream.is_nsfw || 0,
});
});
new file mode 100644
new file mode 100644
@@ -0,0 +1,40 @@
@@ -0,0 +1,40 @@
+'use strict';
+// The paste job and the clip job share ONE registry of used moments, so they can never pick the
+// same second or the same scene of a VOD (the "same title on the clip and the paste" bug).
+const assert = require('assert');
+const fs = require('fs'); const os = require('os'); const path = require('path');
+const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-registry-'));
+process.env.DB_PATH = path.join(tmp, 'live.db');
+const db = require('../server/db/database'); db.initDb();
+const reg = require('../server/ai/moment-registry');
+
+// Legacy logs are imported once so history is not lost on upgrade.
+db.setState('auto_clip_log', JSON.stringify([{ stream_id: 5, vod_id: 900, start_time: 100, title: 'Old clip', sig: 'old clip scene', ts: Date.now() - 3600_000 }]));
+db.setState('home_hero_moments', JSON.stringify({ moments: [{ vodId: 901, offset: 300, title: 'Orange Shirt Shocked in aisle' }], usedSigs: ['a person wearing an orange'], updated_at: Date.now() - 7200_000 }));
+reg._reset();
+assert.ok(reg.isUsed({ vod_id: 900, offset: 116 }), 'legacy clip (start + pre-roll) blocks the spot');
+assert.ok(reg.isUsed({ vod_id: 901, offset: 330 }), 'legacy paste blocks within the gap');
+assert.ok(reg.isUsed({ desc: 'A person wearing an orange shirt is shown leaning' }), 'legacy scene signature blocks the same scene');
+assert.ok(!reg.isUsed({ vod_id: 901, offset: 900, desc: 'A dog steals the mic' }), 'far away + new scene is free');
+assert.deepStrictEqual(reg.usedOffsets(901), [300]);
+console.log('✅ legacy logs imported');
+
+reg.record({ kind: 'paste', vod_id: 42, stream_id: 7, offset: 1000, title: 'Depth Estimation tease', desc: 'The video appears to be a live streaming session with a chat overlay' });
+assert.strictEqual(reg.usedReason({ vod_id: 42, offset: 1050 }), 'paste already made at 1000s of this VOD', 'same VOD within 2 min is blocked');
+assert.ok(/same scene as a recent paste/.test(reg.usedReason({ vod_id: 43, offset: 5, desc: 'the video appears to be a live streaming session with music' })), 'same opening words = same scene, even on another VOD');
+assert.strictEqual(reg.usedReason({ vod_id: 42, offset: 1300, desc: 'A robot arm knocks over the coffee' }), null, '≥ 2 min away with a different scene is fine');
+assert.ok(reg.isUsed({ stream_id: 7, offset: 1010 }), 'live streams are matched by stream id too');
+reg.record({ kind: 'clip', vod_id: 42, stream_id: 7, offset: 1300, title: 'Robot arm vs coffee' });
+assert.deepStrictEqual(reg.usedOffsets(42), [1000, 1300]);
+assert.strictEqual(reg.lastOfKind('paste', { stream_id: 7 }).offset, 1000);
+assert.strictEqual(reg.lastOfKind('clip', { vod_id: 42 }).title, 'Robot arm vs coffee');
+assert.ok(reg.recent(1).length >= 2);
+reg._reset();
+assert.ok(reg.isUsed({ vod_id: 42, offset: 1290 }), 'persisted across a cache reset');
+console.log('✅ shared paste/clip registry: spots, scenes, persistence');
+
+const ai = require('../server/ai/ai-analysis');
+assert.deepStrictEqual(['gaming', 'Just Chatting', 'coding', 'nonsense', ''].map(ai.normalizeCategory), ['gaming', 'irl', 'desktop', null, null]);
+console.log('✅ category taxonomy normalisation');
+console.log('\n✅ All moment-registry tests passed');
+process.exit(0);