openvibers/openvibe.live Commits

The Rundown: lines in a language the streamer does not speak never reach the AI (a Portuguese clip on iZoR_One's Greek and English stream was written up as theirs), and the AI's speaker labels (others) drop quotes from lines that are not the streamer and segments built mostly on them (League champion lines read as 'queen roleplay') (#256)

OP OpenVibers committed 25 days ago · bcfd3fbb60 · 4 files, +106 −10

Changes

ddiff --git a/docs/rundown.md b/docs/rundown.md
iindex df86b9e5..beb59372 100644
---- a/docs/rundown.md
++++ b/docs/rundown.md
@@@ -32,7 +32,9 @@ The first deploy reads back the last 7 days, newest streams first. At most 4 AI
Speech-to-text hears a stream's whole audio track. On 2026-10-18 the Arena's "quote of the day" was League of Legends' announcer ("You have slain an enemy"), credited to a streamer. The desk's rules (`server/rundown/filters.js` and the prompts in OpenVibe.AI):
- **Game lines** (League, Counter-Strike and Valorant announcer and system lines) never reach the AI and are never a quote. Neither does a line heard word for word on 3 or more different streams in 30 days: a game's voice lines ("Wanna see a trick?"), speech-to-text's hallucinations on silence ("Thanks for watching, and I'll see you in the next video.") and stock filler. The prompt tells the AI that announcers, characters, lyrics, ads, video narration and articles read aloud are never the streamer; a reaction to them is the take.
--- **A quote** is kept only when it is in the transcript the AI read, word for word.
++- **Other languages.** A line in a language the streamer doesn't speak never reaches the AI. Their languages are those in 10% or more of their lines over 30 days, the top one always: iZoR_One speaks Greek and English, so a Portuguese clip playing in the background isn't theirs.
++- **Who speaks each line.** The AI first returns `others`, the times of the lines that aren't the streamer (game characters, videos, songs, other people). A quote from one of those lines is dropped, and so is a segment whose lines are mostly theirs. A game character's boasts ("Kneel before me or bleed out.") are the game even in English.
++- **A quote** is kept only when it is in the streamer's own lines the AI read, word for word.
- **No invented accusations.** A headline, account or take that uses an accusing word (threat, attack, scam, hacked…) the transcript never used drops the whole segment; in a quip, only the quip goes. Episodes and topic pages follow the same rule.
- **No looks, ever.** Every text passes `db.noLooks()`, and the prompts forbid jokes about looks, body, clothing, age, race, religion, sexuality, gender or disability, sexual content about anyone, threats and self-harm jokes. Swearing is fine.
- **Names and pronouns.** Names are written exactly as given, and people are called by their name or "they".
ddiff --git a/server/rundown/desk.js b/server/rundown/desk.js
iindex 9eb8ccb7..c081daea 100644
---- a/server/rundown/desk.js
++++ b/server/rundown/desk.js
@@@ -40,6 +40,8 @@ const ANNOUNCE_WITHIN_H = 6;
let _calls = [];
let _fails = new Map(); // stream id → { at: read_to_sec, n }
let _repeated = { at: 0, set: new Set() };
++let _langs = new Map(); // user id → { at, set } of the languages they speak
++const OWN_LANGUAGE_SHARE = 0.1; // a language is theirs at 10% of their lines or more
let _announced = [];
let _busy = false;
let _job = null;
@@@ -149,17 +151,40 @@ async function repeatedLines() {
return _repeated.set;
}
--/** The next stretch: speech lines after read_to_sec, game and repeated lines out, at most MAX_WORDS. */
++/**
++ * The languages a streamer speaks: those in 10% or more of their lines over 30 days (their top one always). A line in
++ * any other language is something playing on their stream (iZoR_One speaks Greek and English; a Portuguese clip in the
++ * background was written up as their story, 2026-10-19). null when there are too few lines to tell.
++ */
++async function ownLanguages(userId) {
++ const c = _langs.get(userId);
++ if (c && Date.now() - c.at < 6 * 3600_000) return c.set;
++ const rows = await db.all("SELECT text FROM stream_timeline_events WHERE user_id = ? AND kind = 'speech' AND created_at >= ? ORDER BY id DESC LIMIT 4000", [userId, daysAgo(30)]);
++ const counts = new Map();
++ let total = 0;
++ for (const r of rows) { const l = filters.lineLang(r.text); if (l) { counts.set(l, (counts.get(l) || 0) + 1); total++; } }
++ let set = null;
++ if (total >= 50) {
++ const ranked = [...counts.entries()].sort((a, b) => b[1] - a[1]);
++ set = new Set(ranked.filter(([, n], i) => i === 0 || n / total >= OWN_LANGUAGE_SHARE).map(([l]) => l));
++ }
++ _langs.set(userId, { at: Date.now(), set });
++ return set;
++}
++
++/** The next stretch: speech lines after read_to_sec; game lines, repeated lines and lines in a language the streamer does not speak out; at most MAX_WORDS. */
async function nextStretch(stream) {
const rows = await db.all(`SELECT start_sec, text, vod_id FROM stream_timeline_events
WHERE stream_id = ? AND kind = 'speech' AND start_sec > ? ORDER BY start_sec ASC LIMIT 800`, [stream.id, stream.read_to_sec]);
const repeated = await repeatedLines();
++ const langs = await ownLanguages(stream.user_id);
++ const theirs = (text) => { if (!langs) return true; const l = filters.lineLang(text); return !l || langs.has(l); };
const lines = [];
let n = 0, lastSec = Number(stream.read_to_sec);
for (const r of rows) {
const text = String(r.text || '').replace(/\s+/g, ' ').trim();
const w = words(text);
-- if (text && !filters.gameLine(text) && !repeated.has(filters.norm(text))) {
++ if (text && !filters.gameLine(text) && !repeated.has(filters.norm(text)) && theirs(text)) {
if ((n + w > MAX_WORDS || lines.length >= MAX_LINES) && lines.length) break;
lines.push({ t: Math.round(Number(r.start_sec) * 10) / 10, text: text.slice(0, 1000), vod_id: r.vod_id });
n += w;
@@@ -177,9 +202,16 @@ function ready(stream, stretch) {
return stretch.words >= MAX_WORDS * 0.9 || stretch.words >= READ_WORDS || (span >= READ_SPAN_SEC && stretch.words >= MIN_WORDS);
}
--/** Store the AI's segments for one stretch. Returns the number kept. */
--async function keepSegments(stream, stretch, segments, user) {
-- const source = stretch.lines.map((l) => l.text).join('\n');
++/**
++ * Store the AI's segments for one stretch. Returns the number kept. `others` are the times of the lines the AI decided
++ * were not the streamer (a game character, a video, other people): a quote from one goes, and a segment whose lines are
++ * mostly theirs goes too.
++ */
++async function keepSegments(stream, stretch, segments, user, others = []) {
++ const notTheirs = new Set((others || []).map((t) => Math.round(Number(t) * 10)));
++ const isOther = (l) => notTheirs.has(Math.round(Number(l.t) * 10));
++ const own = stretch.lines.filter((l) => !isOther(l));
++ const source = own.map((l) => l.text).join('\n');
const startedMs = toMs(stream.started_at) || Date.now();
let kept = 0;
for (const s of segments || []) {
@@@ -190,7 +222,8 @@ async function keepSegments(stream, stretch, segments, user) {
if (filters.addsAccusation(`${title} ${what} ${take || ''}`, source)) continue;
if (quip && filters.addsAccusation(quip, source)) quip = null;
const span = stretch.lines.filter((l) => l.t >= Number(s.start) - 1 && l.t <= Number(s.end) + 1);
-- const spanText = (span.length ? span : stretch.lines).map((l) => l.text).join('\n');
++ if (span.length && span.filter(isOther).length / span.length > 0.6) continue;
++ const spanText = (span.length ? span.filter((l) => !isOther(l)) : own).map((l) => l.text).join('\n');
const quotes = (s.quotes || []).map((q) => {
const text = filters.verifiedQuote(q.text, source);
return text ? { text, sec: Number(q.t) || Number(s.start) || 0, en: q.en ? clean(q.en, 400) : null } : null;
@@@ -230,7 +263,7 @@ async function readStretch(stream, user) {
return { called: true, failed: true };
}
_fails.delete(stream.id);
-- const kept = await keepSegments(stream, stretch, out.segments, user);
++ const kept = await keepSegments(stream, stretch, out.segments, user, out.others);
await setProgress(stream.id, stretch.lastSec);
return { called: true, kept };
}
@@@ -341,4 +374,4 @@ function start() {
}
function stop() { if (typeof _job === 'function') _job(); _job = null; }
--module.exports = { start, stop, tick, readStretch, writeEpisode, refreshTopic, keepSegments, topicFor, nextStretch, ready, optedOut, MAX_LINES, _test: { reset() { _calls = []; _announced = []; _fails = new Map(); _repeated = { at: 0, set: new Set() }; _people = { at: 0, entries: [] }; } } };
++module.exports = { start, stop, tick, readStretch, writeEpisode, refreshTopic, keepSegments, topicFor, nextStretch, ready, optedOut, MAX_LINES, _test: { reset() { _calls = []; _announced = []; _fails = new Map(); _repeated = { at: 0, set: new Set() }; _langs = new Map(); _people = { at: 0, entries: [] }; } } };
ddiff --git a/server/rundown/filters.js b/server/rundown/filters.js
iindex 0b7362bc..3db5f89b 100644
---- a/server/rundown/filters.js
++++ b/server/rundown/filters.js
@@@ -9,6 +9,8 @@
* verifiedQuote(q, text) a quote is kept only when it is in the transcript the AI read, word for word, and is no game line
* addsAccusation(t, src) an AI line may use an accusing word (threat, attack, scam…) only when the transcript did
* slugify / sameTopic topic identity
++ * lineLang(text) a line's language, roughly (desk.js keeps a streamer's own languages, so a clip in Portuguese
++ * playing on a Greek streamer's stream is never theirs)
*/
// Announcer and system lines of the games streamed here (League of Legends, Counter-Strike, Valorant), as speech-to-text
@@@ -33,6 +35,31 @@ function gameLine(text) {
return !!t && GAME_LINES.some((re) => re.test(t));
}
++// A line's language, roughly: by script (Greek, Cyrillic, Japanese, Chinese, Korean, Arabic), else by common words among
++// English, Portuguese, Spanish, French, German and Italian; null when a line is too short to tell.
++const STOPWORDS = {
++ en: ['the', 'and', 'you', 'that', 'is', 'it', 'what', 'i', 'to', 'this', 'are', 'my', 'so', 'bro', 'yeah', 'just', 'like', 'gonna', 'chat', 'guys', 'okay', 'oh', 'no'],
++ pt: ['eu', 'você', 'voce', 'que', 'não', 'nao', 'essa', 'esse', 'isso', 'muito', 'está', 'esta', 'meu', 'minha', 'então', 'também', 'nenhum', 'foi', 'uma', 'com', 'para', 'tudo'],
++ es: ['que', 'el', 'la', 'los', 'las', 'pero', 'muy', 'está', 'esto', 'eso', 'yo', 'tú', 'qué', 'por', 'con', 'una', 'para', 'como', 'bueno', 'vamos'],
++ fr: ['le', 'la', 'les', 'est', 'je', 'tu', 'pas', 'que', 'une', 'des', 'et', 'ça', "c'est", 'mais', 'avec', 'pour', 'oui', 'non'],
++ de: ['der', 'die', 'das', 'und', 'ist', 'ich', 'nicht', 'du', 'ein', 'eine', 'zu', 'mit', 'auch', 'was', 'ja', 'nein'],
++ it: ['che', 'il', 'non', 'è', 'per', 'una', 'sono', 'questo', 'ma', 'come', 'anche', 'perché', 'molto', 'io'],
++};
++function lineLang(text) {
++ const t = String(text || '');
++ if (/[Ͱ-Ͽἀ-῿]/.test(t)) return 'el';
++ if (/[Ѐ-ӿ]/.test(t)) return 'ru';
++ if (/[぀-ヿ]/.test(t)) return 'ja';
++ if (/[一-鿿]/.test(t)) return 'zh';
++ if (/[가-힯]/.test(t)) return 'ko';
++ if (/[؀-ۿ]/.test(t)) return 'ar';
++ const ws = t.toLowerCase().normalize('NFC').split(/[^\p{L}']+/u).filter(Boolean);
++ if (ws.length < 3) return null;
++ let best = null, bestN = 0;
++ for (const [k, list] of Object.entries(STOPWORDS)) { const n = ws.filter((w) => list.includes(w)).length; if (n > bestN) { best = k; bestN = n; } }
++ return best;
++}
++
const norm = (s) => String(s || '').toLowerCase().normalize('NFKD').replace(/[̀-ͯ]/g, '').replace(/[^\p{L}\p{N}]+/gu, ' ').trim();
/** The quote, trimmed, when it is in `source` (the lines the AI read) word for word and no game line; else null. */
@@@ -68,4 +95,4 @@ function sameTopic(a, b) {
return both / (x.size + y.size - both) >= 0.6;
}
--module.exports = { gameLine, verifiedQuote, addsAccusation, slugify, sameTopic, norm, GAME_LINES };
++module.exports = { gameLine, verifiedQuote, addsAccusation, slugify, sameTopic, norm, lineLang, GAME_LINES };
ddiff --git a/test/rundown.test.js b/test/rundown.test.js
iindex ec0292b2..9c7fa40a 100644
---- a/test/rundown.test.js
++++ b/test/rundown.test.js
@@@ -211,6 +211,40 @@ console.log('✅ API: front page, topic, reactions as a signed-out reader; old /
console.log('✅ lines heard on three streams never reach the AI');
}
++// ── A streamer's own languages; lines the AI says are not the streamer ──
++{
++ const gr = await mk('greekgamer');
++ const old = await stream(gr, { startedAgoMin: 20 * 1440, endedAgoMin: 20 * 1440 - 60 });
++ await speak(old, gr, [...Array.from({ length: 60 }, (_, i) => `Λοιπόν παιδιά πάμε ${i} τώρα`), ...Array.from({ length: 30 }, (_, i) => `bro push mid lane what are you doing ${i}`)]);
++ const sg = await stream(gr, { startedAgoMin: 300, endedAgoMin: 200, title: 'Ranked' });
++ await speak(sg, gr, [
++ 'Hoje eu acordei e essa bichinha estava assim no meu bode.',
++ 'Kneel before me or bleed out.',
++ 'No one can find the queen!',
++ 'bro this team is so bad I want to cry right now',
++ ...Array.from({ length: 30 }, (_, i) => `Λοιπόν παιδιά πάμε για καφέ ${i} και μετά παίζουμε`),
++ ]);
++ const prev = answers['live.rundown.segment'];
++ answers['live.rundown.segment'] = (input) => input.streamer !== 'greekgamer' ? prev(input) : {
++ others: input.lines.filter((l) => /queen|Kneel/.test(l.text)).map((l) => l.t),
++ segments: [
++ { start: 10, end: 20, topic: 'Queen roleplay', kind: 'games', title: 'greekgamer plays a queen', what: 'greekgamer did a queen bit.', take: null, quip: 'Long live.', score: 4, quotes: [{ t: 20, text: 'No one can find the queen!', en: null }] },
++ { start: 30, end: 400, topic: 'Bad teammates', kind: 'games', title: 'greekgamer and the team', what: 'greekgamer complained about the team.', take: 'The team is bad.', quip: 'Classic.', score: 4, quotes: [{ t: 20, text: 'No one can find the queen!', en: null }, { t: 30, text: 'this team is so bad I want to cry', en: null }] },
++ ],
++ };
++ desk._test.reset();
++ await desk.tick();
++ const c = calls.filter((x) => x.workflow === 'live.rundown.segment' && x.input.streamer === 'greekgamer');
++ assert.strictEqual(c.length, 1);
++ assert.ok(!c[0].input.lines.some((l) => /Hoje eu acordei/.test(l.text)), 'a Portuguese clip on a Greek and English stream never reaches the AI');
++ assert.ok(c[0].input.lines.some((l) => /Λοιπόν/.test(l.text)) && c[0].input.lines.some((l) => /team is so bad/.test(l.text)), 'their Greek and English stay');
++ const kept = await db.all('SELECT title, quotes_json FROM rundown_segments WHERE stream_id = ? ORDER BY start_sec', [sg]);
++ assert.deepStrictEqual(kept.map((k) => k.title), ['greekgamer and the team'], 'a segment built on the game\'s lines is dropped');
++ assert.deepStrictEqual(JSON.parse(kept[0].quotes_json).map((q) => q.text), ['this team is so bad I want to cry'], 'a quote from a line that is not theirs is dropped');
++ answers['live.rundown.segment'] = prev;
++ console.log('✅ own languages and the AI\'s speaker labels');
++}
++
// ── Limits: a stretch is at most MAX_LINES lines; a stretch the AI keeps refusing is skipped after three tries ──
{
const greek = await mk('greekline');