openvibers/openvibe.ai Commits

A local model for media.analyze (WS-O task 5): deploy/systemd/openvibe-llm.service runs llama.cpp's llama-server on 127.0.0.1:8090 (OpenAI API, no key), held back for live streams (2 threads, CPUQuota 150%, nice 19, MemoryMax 3G). Installed on production 2026-09-27: llama.cpp b11222 CPU build and Qwen2.5-1.5B-Instruct Q4_K_M (Apache-2.0), both checked against their published sha256; ai.env gains AI_LOCAL_LLM_URL/AI_LOCAL_LLM_MODEL, so the provider local and route media.local exist. First production run: the overview came from the local model (65 s for 3 min of a Japanese VOD, about 20 s of it the model), but the 1.5B model retold the transcript past the limit. The template now asks for English, at most 500 characters and one short quote; answers over 600 characters are cut after the last whole sentence (sentences()); max_tokens is 250. README: Local model.

OP OpenVibers committed 27 Sept 2026 · 566822e12d · 5 files, +60 −5


Changes

ddiff --git a/README.md b/README.md
iindex e2ef503..cdadfe3 100644
---- a/README.md
++++ b/README.md
@@@ -203,6 +203,17 @@ stub joins every route as a last resort outside production only (`AI_STUB_FALLBA
`AI_NS_FALLBACK`.
- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.
++## Local model
++
++`deploy/systemd/openvibe-llm.service` runs llama.cpp's `llama-server` on `127.0.0.1:8090` (OpenAI API, no key),
++which AI reaches as the provider `local` and the route `media.local` (`AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1`,
++`AI_LOCAL_LLM_MODEL`). On the production host since 2026-09-27: llama.cpp b11222 (the CPU build, checked against
++the release's sha256) in `/opt/llama.cpp/b11222` with `/opt/llama.cpp/current`, and Qwen2.5-1.5B-Instruct Q4_K_M
++(Apache-2.0, checked against Hugging Face's sha256) in `/var/lib/openvibe-llm/models`. It is held back so live
++streams always win: 2 threads, at most 1.5 CPUs, nice 19, 3 GB. It uses about 0.9 GB and generates about
++28 tokens/s. Without it, `media.analyze` writes an extractive overview and says so. To swap the model, put the
++file beside it, change `--model`/`--alias` and `AI_LOCAL_LLM_MODEL`, and restart both units.
++
## Guarantees and where they are tested
| Guarantee | Test |
ddiff --git a/deploy/systemd/openvibe-llm.service b/deploy/systemd/openvibe-llm.service
nnew file mode 100644
iindex 0000000..f49d568
---- /dev/null
++++ b/deploy/systemd/openvibe-llm.service
@@@ -0,0 +1,33 @@
++[Unit]
++Description=OpenVibe local model server — llama.cpp llama-server, OpenAI API on 127.0.0.1:8090, for OpenVibe.AI's media.local route (roadmap WS-O task 5)
++# OpenVibe.AI reaches it as the provider `local` (AI_LOCAL_LLM_URL=http://127.0.0.1:8090/v1). Loopback only,
++# no key. It shares the host with live streams, so it is held back: 2 threads, at most 1.5 CPUs, nice 19 and
++# 3 GB of memory. A stream's RTMP ingest and the web services always win.
++# Install (README "Local model"): llama.cpp's CPU build under /opt/llama.cpp/<build> with /opt/llama.cpp/current,
++# the model in /var/lib/openvibe-llm/models, both checked against their published sha256.
++After=network.target
++
++[Service]
++Type=simple
++User=ubuntu
++Group=ubuntu
++ExecStart=/opt/llama.cpp/current/llama-server \
++ --model /var/lib/openvibe-llm/models/qwen2.5-1.5b-instruct-q4_k_m.gguf \
++ --alias qwen2.5-1.5b-instruct \
++ --host 127.0.0.1 --port 8090 \
++ --threads 2 --ctx-size 8192 --parallel 1 \
++ --no-webui
++Nice=19
++CPUQuota=150%
++MemoryMax=3G
++Restart=on-failure
++RestartSec=10
++TimeoutStopSec=15
++NoNewPrivileges=true
++PrivateTmp=true
++ProtectSystem=strict
++ProtectHome=true
++ReadOnlyPaths=/var/lib/openvibe-llm /opt/llama.cpp
++
++[Install]
++WantedBy=multi-user.target
ddiff --git a/server/workflows/media-analysis.js b/server/workflows/media-analysis.js
iindex 6ee87d3..232b45f 100644
---- a/server/workflows/media-analysis.js
++++ b/server/workflows/media-analysis.js
@@@ -167,6 +167,14 @@ function factsOf({ duration, streams, signals, scenes, highlights, ratio, transc
return lines;
}
++/** At most `max` characters, cut after the last whole sentence that fits (a small model may ramble past the prompt's limit). */
++function sentences(text, max) {
++ if (text.length <= max) return text;
++ const head = text.slice(0, max);
++ const end = Math.max(...['. ', '! ', '? ', '。', '!', '?'].map((p) => head.lastIndexOf(p)));
++ return end > max / 3 ? head.slice(0, end + 1).trim() : `${head.slice(0, max - 1).trim()}…`;
++}
++
function extractive(facts, highlights) {
const parts = [facts.join(' ')];
const quoted = highlights.filter((h) => h.excerpt).slice(0, 3);
@@@ -272,11 +280,11 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
const route = registry.resolveRoute(routeKey);
if (!route || route.disabled) return { missing: true };
const system = render(template.system_prompt, vars).trim();
-- const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 350, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
++ const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
const exec = await pool.execute(route, 'summarize', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });
-- const text = String(exec.result.text || '').trim();
++ const text = sentences(String(exec.result.text || '').trim(), 600);
if (!text) throw new AiError(502, 'output.empty', 'the model answered with nothing');
-- return { text: text.slice(0, 1500), exec, route };
++ return { text, exec, route };
};
if (template) {
try {
@@@ -360,4 +368,4 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
return { step };
}
--module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, clock };
++module.exports = { createMediaAnalysis, createSignalParser, parseSignals, scenesOf, highlightsOf, speechRatio, extractive, sentences, clock };
ddiff --git a/server/workflows/media.js b/server/workflows/media.js
iindex 6d2f533..4a9f0b7 100644
---- a/server/workflows/media.js
++++ b/server/workflows/media.js
@@@ -19,7 +19,7 @@ const templates = [
key: 'media.analyze.overview', name: 'Media: overview from signals and a transcript',
description: 'The overview step of media.analyze, for a local model (media.local) or, with a budget, a paid one (media.paid).',
input_schema: { type: 'object' }, output_schema: { type: 'object' },
-- system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences (at most 600 characters): what the recording is about and what happens in it, mentioning the highlight times that matter. No markup, no emoji, no preamble.',
++ system_prompt: 'You describe a video or audio recording for its creator, from facts a machine measured and from a speech transcript. Use only what is given. Do not guess what is on screen beyond what the transcript says, and never invent names, numbers or events. Write 2 to 4 plain sentences in English (at most 500 characters): what the recording is about and what happens in it, with the times of the highlights that matter. Do not retell or translate the transcript; quote at most one short phrase (under 60 characters). No markup, no emoji, no preamble.',
user_prompt: 'Measured facts:\n{{facts}}\n\nHighlights (time, why, what was said):\n{{highlights}}\n\nTranscript:\n{{transcript}}',
default_route: 'media.local', owner: 'media', metadata: {},
},
ddiff --git a/test/media-analysis.test.js b/test/media-analysis.test.js
iindex 16a5b01..6d84074 100644
---- a/test/media-analysis.test.js
++++ b/test/media-analysis.test.js
@@@ -58,6 +58,9 @@ t.test('the parser reads every filter\'s lines; loud means well above the typica
assert.strictEqual(ma.scenesOf(many, 4010).length, 300, 'at most 300 scenes; the strongest changes are the boundaries');
assert.strictEqual(ma.speechRatio([{ start: 0, end: 10, text: 'a' }, { start: 5, end: 12, text: 'b' }], 24), 0.5, 'overlapping speech counts once');
assert.strictEqual(ma.clock(3725), '1:02:05');
++ const long = 'First point here. '.repeat(50);
++ assert.ok(ma.sentences(long, 600).length <= 600 && ma.sentences(long, 600).endsWith('here.'), 'a rambling answer is cut after a whole sentence');
++ assert.strictEqual(ma.sentences('短い。', 600), '短い。');
// A quiet stream: someone talking over a still picture is a highlight, not dead air; a silent black stretch is.
const talk = { scene_changes: [], black: [{ start: 60, end: 90 }], frozen: [{ start: 0, end: 60 }], silence: [{ start: 60, end: 90 }], loudness: { typical_lufs: -41, integrated_lufs: -39, peaks: [] } };
const segs = Array.from({ length: 12 }, (_, i) => ({ start: i * 5, end: i * 5 + 5, text: 'and then we go over there and look at this one more time okay' }));