openvibers/openvibe.ai Commits
A local model for media.analyze (WS-O task 5): deploy/systemd/openvibe-llm.service runs llama.cpp's llama-server on 127.0.0.1:8090 (OpenAI API, no key), held back for live streams (2 threads, CPUQuota 150%, nice 19, MemoryMax 3G). Installed on production 2026-09-27: llama.cpp b11222 CPU build and Qwen2.5-1.5B-Instruct Q4_K_M (Apache-2.0), both checked against their published sha256; ai.env gains AI_LOCAL_LLM_URL/AI_LOCAL_LLM_MODEL, so the provider local and route media.local exist. First production run: the overview came from the local model (65 s for 3 min of a Japanese VOD, about 20 s of it the model), but the 1.5B model retold the transcript past the limit. The template now asks for English, at most 500 characters and one short quote; answers over 600 characters are cut after the last whole sentence (sentences()); max_tokens is 250. README: Local model.
Changes
@@@ -203,6 +203,17 @@ stub joins every route as a last resort outside production only (`AI_STUB_FALLBA`AI_NS_FALLBACK`.- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.++## Local model++++`deploy/systemd/openvibe-llm.service` runs llama.cpp's `llama-server` on `127.0.0.1:8090` (OpenAI API, no key),++which AI reaches as the provider `local` and the route `media.local` (`AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1`,++`AI_LOCAL_LLM_MODEL`). On the production host since 2026-09-27: llama.cpp b11222 (the CPU build, checked against++the release's sha256) in `/opt/llama.cpp/b11222` with `/opt/llama.cpp/current`, and Qwen2.5-1.5B-Instruct Q4_K_M++(Apache-2.0, checked against Hugging Face's sha256) in `/var/lib/openvibe-llm/models`. It is held back so live++streams always win: 2 threads, at most 1.5 CPUs, nice 19, 3 GB. It uses about 0.9 GB and generates about++28 tokens/s. Without it, `media.analyze` writes an extractive overview and says so. To swap the model, put the++file beside it, change `--model`/`--alias` and `AI_LOCAL_LLM_MODEL`, and restart both units.++## Guarantees and where they are tested| Guarantee | Test |nnew file mode 100644@@@ -0,0 +1,33 @@++[Unit]++Description=OpenVibe local model server — llama.cpp llama-server, OpenAI API on 127.0.0.1:8090, for OpenVibe.AI's media.local route (roadmap WS-O task 5)++# OpenVibe.AI reaches it as the provider `local` (AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1). Loopback only,++# no key. It shares the host with live streams, so it is held back: 2 threads, at most 1.5 CPUs, nice 19 and++# 3 GB of memory. A stream's RTMP ingest and the web services always win.++# Install (README "Local model"): llama.cpp's CPU build under /opt/llama.cpp/<build> with /opt/llama.cpp/current,++# the model in /var/lib/openvibe-llm/models, both checked against their published sha256.++After=network.target++++[Service]++Type=simple++User=ubuntu++Group=ubuntu++ExecStart=/opt/llama.cpp/current/llama-server \++ --model /var/lib/openvibe-llm/models/qwen2.5-1.5b-instruct-q4_k_m.gguf \++ --alias qwen2.5-1.5b-instruct \++ --host 127.0.0.1 --port 8090 \++ --threads 2 --ctx-size 8192 --parallel 1 \++ --no-webui++Nice=19++CPUQuota=150%++MemoryMax=3G++Restart=on-failure++RestartSec=10++TimeoutStopSec=15++NoNewPrivileges=true++PrivateTmp=true++ProtectSystem=strict++ProtectHome=true++ReadOnlyPaths=/var/lib/openvibe-llm /opt/llama.cpp++++[Install]++WantedBy=multi-user.target@@@ -167,6 +167,14 @@ function factsOf({ duration, streams, signals, scenes, highlights, ratio, transcreturn lines;}++/** At most `max` characters, cut after the last whole sentence that fits (a small model may ramble past the prompt's limit). */++function sentences(text, max) {++ if (text.length <= max) return text;++ const head = text.slice(0, max);++ const end = Math.max(...['. ', '! ', '? ', '。', '!', '?'].map((p) => head.lastIndexOf(p)));++ return end > max / 3 ? head.slice(0, end + 1).trim() : `${head.slice(0, max - 1).trim()}…`;++}++function extractive(facts, highlights) {const parts = [facts.join(' ')];const quoted = highlights.filter((h) => h.excerpt).slice(0, 3);@@@ -272,11 +280,11 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spaconst route = registry.resolveRoute(routeKey);if (!route || route.disabled) return { missing: true };const system = render(template.system_prompt, vars).trim();-- const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 350, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };++ const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };const exec = await pool.execute(route, 'summarize', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });-- const text = String(exec.result.text || '').trim();++ const text = sentences(String(exec.result.text || '').trim(), 600);if (!text) throw new AiError(502, 'output.empty', 'the model answered with nothing');-- return { text: text.slice(0, 1500), exec, route };++ return { text, exec, route };};if (template) {try {@@@ -360,4 +368,4 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spareturn { step };}--module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, clock };++module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, sentences, clock };@@@ -19,7 +19,7 @@ const templates = [key: 'media.analyze.overview', name: 'Media: overview from signals and a transcript',description: 'The overview step of media.analyze, for a local model (media.local) or, with a budget, a paid one (media.paid).',input_schema: { type: 'object' }, output_schema: { type: 'object' },-- system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences (at most 600 characters): what the recording is about and what happens in it, mentioning the highlight times that matter. No markup, no emoji, no preamble.',++ system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences in English (at most 500 characters): what the recording is about and what happens in it, with the times of the highlights that matter. Do not retell or translate the transcript; quote at most one short phrase (under 60 characters). No markup, no emoji, no preamble.',user_prompt: 'Measured facts:\n{{facts}}\n\nHighlights (time, why, what was said):\n{{highlights}}\n\nTranscript:\n{{transcript}}',default_route: 'media.local', owner: 'media', metadata: {},},@@@ -58,6 +58,9 @@ t.test('the parser reads every filter\'s lines; loud means well above the typicaassert.strictEqual(ma.scenesOf(many, 4010).length, 300, 'at most 300 scenes; the strongest changes are the boundaries');assert.strictEqual(ma.speechRatio([{ start: 0, end: 10, text: 'a' }, { start: 5, end: 12, text: 'b' }], 24), 0.5, 'overlapping speech counts once');assert.strictEqual(ma.clock(3725), '1:02:05');++ const long = 'First point here. '.repeat(50);++ assert.ok(ma.sentences(long, 600).length <= 600 && ma.sentences(long, 600).endsWith('here.'), 'a rambling answer is cut after a whole sentence');++ assert.strictEqual(ma.sentences('短い。', 600), '短い。');// A quiet stream: someone talking over a still picture is a highlight, not dead air; a silent black stretch is.const talk = { scene_changes: [], black: [{ start: 60, end: 90 }], frozen: [{ start: 0, end: 60 }], silence: [{ start: 60, end: 90 }], loudness: { typical_lufs: -41, integrated_lufs: -39, peaks: [] } };const segs = Array.from({ length: 12 }, (_, i) => ({ start: i * 5, end: i * 5 + 5, text: 'and then we go over there and look at this one more time okay' }));
@@ -203,6 +203,17 @@ stub joins every route as a last resort outside production only (`AI_STUB_FALLBA
@@ -203,6 +203,17 @@ stub joins every route as a last resort outside production only (`AI_STUB_FALLBA
`AI_NS_FALLBACK`.
`AI_NS_FALLBACK`.
- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.
- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.
+## Local model
+
+`deploy/systemd/openvibe-llm.service` runs llama.cpp's `llama-server` on `127.0.0.1:8090` (OpenAI API, no key),
+which AI reaches as the provider `local` and the route `media.local` (`AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1`,
+`AI_LOCAL_LLM_MODEL`). On the production host since 2026-09-27: llama.cpp b11222 (the CPU build, checked against
+the release's sha256) in `/opt/llama.cpp/b11222` with `/opt/llama.cpp/current`, and Qwen2.5-1.5B-Instruct Q4_K_M
+(Apache-2.0, checked against Hugging Face's sha256) in `/var/lib/openvibe-llm/models`. It is held back so live
+streams always win: 2 threads, at most 1.5 CPUs, nice 19, 3 GB. It uses about 0.9 GB and generates about
+28 tokens/s. Without it, `media.analyze` writes an extractive overview and says so. To swap the model, put the
+file beside it, change `--model`/`--alias` and `AI_LOCAL_LLM_MODEL`, and restart both units.
+
## Guarantees and where they are tested
## Guarantees and where they are tested
| Guarantee | Test |
| Guarantee | Test |
new file mode 100644
new file mode 100644
@@ -0,0 +1,33 @@
@@ -0,0 +1,33 @@
+[Unit]
+Description=OpenVibe local model server — llama.cpp llama-server, OpenAI API on 127.0.0.1:8090, for OpenVibe.AI's media.local route (roadmap WS-O task 5)
+# OpenVibe.AI reaches it as the provider `local` (AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1). Loopback only,
+# no key. It shares the host with live streams, so it is held back: 2 threads, at most 1.5 CPUs, nice 19 and
+# 3 GB of memory. A stream's RTMP ingest and the web services always win.
+# Install (README "Local model"): llama.cpp's CPU build under /opt/llama.cpp/<build> with /opt/llama.cpp/current,
+# the model in /var/lib/openvibe-llm/models, both checked against their published sha256.
+After=network.target
+
+[Service]
+Type=simple
+User=ubuntu
+Group=ubuntu
+ExecStart=/opt/llama.cpp/current/llama-server \
+ --model /var/lib/openvibe-llm/models/qwen2.5-1.5b-instruct-q4_k_m.gguf \
+ --alias qwen2.5-1.5b-instruct \
+ --host 127.0.0.1 --port 8090 \
+ --threads 2 --ctx-size 8192 --parallel 1 \
+ --no-webui
+Nice=19
+CPUQuota=150%
+MemoryMax=3G
+Restart=on-failure
+RestartSec=10
+TimeoutStopSec=15
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=true
+ReadOnlyPaths=/var/lib/openvibe-llm /opt/llama.cpp
+
+[Install]
+WantedBy=multi-user.target
@@ -167,6 +167,14 @@ function factsOf({ duration, streams, signals, scenes, highlights, ratio, transc
@@ -167,6 +167,14 @@ function factsOf({ duration, streams, signals, scenes, highlights, ratio, transc
return lines;
return lines;
}
}
+/** At most `max` characters, cut after the last whole sentence that fits (a small model may ramble past the prompt's limit). */
+function sentences(text, max) {
+ if (text.length <= max) return text;
+ const head = text.slice(0, max);
+ const end = Math.max(...['. ', '! ', '? ', '。', '!', '?'].map((p) => head.lastIndexOf(p)));
+ return end > max / 3 ? head.slice(0, end + 1).trim() : `${head.slice(0, max - 1).trim()}…`;
+}
+
function extractive(facts, highlights) {
function extractive(facts, highlights) {
const parts = [facts.join(' ')];
const parts = [facts.join(' ')];
const quoted = highlights.filter((h) => h.excerpt).slice(0, 3);
const quoted = highlights.filter((h) => h.excerpt).slice(0, 3);
@@ -272,11 +280,11 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -272,11 +280,11 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
const route = registry.resolveRoute(routeKey);
const route = registry.resolveRoute(routeKey);
if (!route || route.disabled) return { missing: true };
if (!route || route.disabled) return { missing: true };
const system = render(template.system_prompt, vars).trim();
const system = render(template.system_prompt, vars).trim();
- const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 350, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
+ const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
const exec = await pool.execute(route, 'summarize', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });
const exec = await pool.execute(route, 'summarize', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });
- const text = String(exec.result.text || '').trim();
+ const text = sentences(String(exec.result.text || '').trim(), 600);
if (!text) throw new AiError(502, 'output.empty', 'the model answered with nothing');
if (!text) throw new AiError(502, 'output.empty', 'the model answered with nothing');
- return { text: text.slice(0, 1500), exec, route };
+ return { text, exec, route };
};
};
if (template) {
if (template) {
try {
try {
@@ -360,4 +368,4 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -360,4 +368,4 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
return { step };
return { step };
}
}
-module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, clock };
+module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, sentences, clock };
@@ -19,7 +19,7 @@ const templates = [
@@ -19,7 +19,7 @@ const templates = [
key: 'media.analyze.overview', name: 'Media: overview from signals and a transcript',
key: 'media.analyze.overview', name: 'Media: overview from signals and a transcript',
description: 'The overview step of media.analyze, for a local model (media.local) or, with a budget, a paid one (media.paid).',
description: 'The overview step of media.analyze, for a local model (media.local) or, with a budget, a paid one (media.paid).',
input_schema: { type: 'object' }, output_schema: { type: 'object' },
input_schema: { type: 'object' }, output_schema: { type: 'object' },
- system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences (at most 600 characters): what the recording is about and what happens in it, mentioning the highlight times that matter. No markup, no emoji, no preamble.',
+ system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences in English (at most 500 characters): what the recording is about and what happens in it, with the times of the highlights that matter. Do not retell or translate the transcript; quote at most one short phrase (under 60 characters). No markup, no emoji, no preamble.',
user_prompt: 'Measured facts:\n{{facts}}\n\nHighlights (time, why, what was said):\n{{highlights}}\n\nTranscript:\n{{transcript}}',
user_prompt: 'Measured facts:\n{{facts}}\n\nHighlights (time, why, what was said):\n{{highlights}}\n\nTranscript:\n{{transcript}}',
default_route: 'media.local', owner: 'media', metadata: {},
default_route: 'media.local', owner: 'media', metadata: {},
},
},
@@ -58,6 +58,9 @@ t.test('the parser reads every filter\'s lines; loud means well above the typica
@@ -58,6 +58,9 @@ t.test('the parser reads every filter\'s lines; loud means well above the typica
assert.strictEqual(ma.scenesOf(many, 4010).length, 300, 'at most 300 scenes; the strongest changes are the boundaries');
assert.strictEqual(ma.scenesOf(many, 4010).length, 300, 'at most 300 scenes; the strongest changes are the boundaries');
assert.strictEqual(ma.speechRatio([{ start: 0, end: 10, text: 'a' }, { start: 5, end: 12, text: 'b' }], 24), 0.5, 'overlapping speech counts once');
assert.strictEqual(ma.speechRatio([{ start: 0, end: 10, text: 'a' }, { start: 5, end: 12, text: 'b' }], 24), 0.5, 'overlapping speech counts once');
assert.strictEqual(ma.clock(3725), '1:02:05');
assert.strictEqual(ma.clock(3725), '1:02:05');
+ const long = 'First point here. '.repeat(50);
+ assert.ok(ma.sentences(long, 600).length <= 600 && ma.sentences(long, 600).endsWith('here.'), 'a rambling answer is cut after a whole sentence');
+ assert.strictEqual(ma.sentences('短い。', 600), '短い。');
// A quiet stream: someone talking over a still picture is a highlight, not dead air; a silent black stretch is.
// A quiet stream: someone talking over a still picture is a highlight, not dead air; a silent black stretch is.
const talk = { scene_changes: [], black: [{ start: 60, end: 90 }], frozen: [{ start: 0, end: 60 }], silence: [{ start: 60, end: 90 }], loudness: { typical_lufs: -41, integrated_lufs: -39, peaks: [] } };
const talk = { scene_changes: [], black: [{ start: 60, end: 90 }], frozen: [{ start: 0, end: 60 }], silence: [{ start: 60, end: 90 }], loudness: { typical_lufs: -41, integrated_lufs: -39, peaks: [] } };
const segs = Array.from({ length: 12 }, (_, i) => ({ start: i * 5, end: i * 5 + 5, text: 'and then we go over there and look at this one more time okay' }));
const segs = Array.from({ length: 12 }, (_, i) => ({ start: i * 5, end: i * 5 + 5, text: 'and then we go over there and look at this one more time okay' }));