AI speech-to-text: NVIDIA Parakeet TDT 0.6B v3 for English and 24 European languages when PARAKEET_MODEL is set (VOD and clip transcripts: more words, punctuated, about 4x faster than large-v3-turbo); whisper keeps the rest and takes any Parakeet failure (#107)

ddiff --git a/.env.example b/.env.example
iindex 10b0475..422ff2e 100644
---- a/.env.example
++++ b/.env.example
@@@ -132,6 +132,11 @@ AI_MODEL=
  #WHISPER_THREADS=4
  #WHISPER_BEAM=1
  #WHISPER_MAX_CONCURRENT=1
++# NVIDIA Parakeet TDT 0.6B v3 for English and 24 European languages (whisper keeps the rest and takes any failure).
++# Set the model to turn it on; the binary defaults to parakeet-cli next to WHISPER_BIN. ASR_ENGINE=whisper forces whisper.
++#PARAKEET_MODEL=/home/ubuntu/whisper.cpp/models/ggml-parakeet-tdt-0.6b-v3-q8_0.bin
++#PARAKEET_BIN=
++#ASR_ENGINE=
  
  # ── Quotas seeded at boot (admin API edits them afterwards; 0 = none) ──
  # Live's ai_max_cost_usd_per_day: one daily spend cap across every caller
ddiff --git a/README.md b/README.md
iindex b985ac5..8efd075 100644
---- a/README.md
++++ b/README.md
@@@ -313,7 +313,7 @@ definition, but never over an admin's version.
  | `responses` | The OpenAI Responses API (`POST {base}/responses`) for GPT-family models that only speak it, at any OpenAI-compatible `base_url` (default `https://api.openai.com/v1`): chat and every text operation, structured output through `text.format` with the same step-down, vision; no embeddings or transcription. | the admin API (`kind: "responses"`, `base_url`, `secret_ref`) |
  | `anthropic` | Messages API with cached system blocks and forced-tool JSON. Ported from Live's `llm.js`. | same, with `AI_PROVIDER=anthropic` |
  | `http` | The local HTTP seam: `POST {operation, …}` → `{text, json, usage}` | `AI_HTTP_SEAM_URL` or the admin API |
--| `whisper` | whisper.cpp on this host, with Live's VAD, hallucination filter, multilingual model and live/batch lanes | `WHISPER_*` — Live's names |
++| `whisper` | whisper.cpp on this host, with Live's VAD, hallucination filter, multilingual model and live/batch lanes; NVIDIA Parakeet TDT 0.6B v3 (`parakeet-cli`) for English and 24 European languages when `PARAKEET_MODEL` is set, falling back to whisper | `WHISPER_*`, `PARAKEET_*`, `ASR_ENGINE` — Live's names |
  | `openai`, key `local` | A model server on this host speaking the OpenAI API (llama.cpp's `llama-server`, Ollama's `/v1`): no key, never paid; route `media.local` | `AI_LOCAL_LLM_URL`, `AI_LOCAL_LLM_MODEL`, `AI_LOCAL_LLM_TIMEOUT_MS` |
  
  Routing: a route either names a `capability` and the router builds its pool from every provider that
@@@ -490,6 +490,7 @@ mix vectors from different providers (re-embed when the provider changes).
  | Public price/latency page: 200 without auth, rows from seeded stats and rate cards, the empty state, no caller ids, own-key providers, URLs or secrets | `test/stats-page.test.js` |
  | The OpenAI Responses adapter (`kind: responses`) with a stubbed fetch: request shape, output and usage parsing, `text.format`, error mapping, the key never in an error | `test/provider-responses.test.js` |
  | Ported adapters against fake OpenAI/Anthropic servers, stub determinism, whisper filter, templates | `test/providers.test.js` |
++| Parakeet: engine by language, 30 s pieces past 45 s, fallback to whisper, `ASR_ENGINE=whisper` | `test/parakeet.test.js` |
  | `media.analyze` on a real 28 s fixture: signals, scenes and loud moments measured, highlights with evidence, the local model's overview from the facts only; the local model down → extractive and said; paid only with `allow_paid` and a `media.paid` budget, never past it; too little disk refused before downloading | `test/media-analysis.test.js` |
  | Import from a Live snapshot: dry run, holds, idempotent re-run | `test/import.test.js` |
  | Capability proposals are valid contracts documents matching what is enforced | `test/proposals.test.js` |
ddiff --git a/server/config.js b/server/config.js
iindex 176cc45..acfbb12 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -146,6 +146,10 @@ function load(env = process.env) {
              threads: Math.max(2, Math.min(8, int(env.WHISPER_THREADS, 4))),
              beam: Math.max(1, Math.min(8, int(env.WHISPER_BEAM, 1))),
              maxConcurrent: Math.max(1, int(env.WHISPER_MAX_CONCURRENT, 1)),
++            // NVIDIA Parakeet TDT (parakeet-cli, same whisper.cpp build): on when PARAKEET_MODEL names the ggml model.
++            parakeetBin: env.PARAKEET_BIN || null,
++            parakeetModel: env.PARAKEET_MODEL || null,
++            engine: env.ASR_ENGINE === 'whisper' ? 'whisper' : 'auto',
          },
  
          // ── Quotas seeded at boot (admin API edits them afterwards) ──
ddiff --git a/server/providers/whisper.js b/server/providers/whisper.js
iindex 7f3e408..d3601d4 100644
---- a/server/providers/whisper.js
++++ b/server/providers/whisper.js
@@@ -8,6 +8,11 @@
   *     filler words when there is no VAD) and collapsing of looped repeats
   *   - a multilingual model (WHISPER_MODEL_MULTI) for non-English audio, a faster live model
   *     (WHISPER_MODEL_LIVE), two independent lanes (live never waits on batch), nice -n 15
++ *   - NVIDIA Parakeet TDT 0.6B v3 (parakeet-cli from the same build) for English and 24 European
++ *     languages when PARAKEET_MODEL is set, as Live does since 10-16: on a 2-minute stream sample it
++ *     heard more words than large-v3-turbo (394 vs 369, punctuated) in a quarter of the time (31 s vs
++ *     121 s). Audio over 45 s goes in 30 s pieces (a 300 s window needed 3.1 GB in one run). Any
++ *     Parakeet failure falls back to whisper on the same audio; ASR_ENGINE=whisper forces whisper.
   * Same environment names as Live, so the same install works on either host.
   *
   * Input is a local media file the engine already fetched through the SSRF-safe fetcher; ffmpeg
@@@ -58,6 +63,42 @@ function cleanSegments(segsRaw, hasVad) {
      return out;
  }
  
++// ── Parakeet output → timed sentences (Live's parseParakeet) ──
++// `parakeet-cli -ps` prints every token on stderr with its start and end in centiseconds and whether it starts a word
++// ("▁For"). A sentence ends at . ? or !, at a pause of 1.5 s, or after 20 s. Fillers ("uh", "um") are dropped.
++const PARAKEET_LANGS = new Set(['bg', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'de', 'el', 'hu', 'it', 'lv', 'lt', 'mt', 'pl', 'pt', 'ro', 'sk', 'sl', 'es', 'sv', 'ru', 'uk']);
++const TOKEN_LINE = /t0=\s*(-?\d+)\s+t1=\s*(-?\d+)\s+word_start=(true|false)\s+"(.*)"\s*$/;
++const FILLER_RE = /(^|[\s,])(?:uh+|um+|uhm|erm|mm+)(?=[\s,.!?]|$)[,]?/gi;
++function parseParakeet(stderr, offsetSec = 0) {
++    const out = [];
++    let cur = null;
++    const close = () => {
++        if (!cur) return;
++        const text = cur.text.replace(FILLER_RE, '$1').replace(/\s+([,.!?])/g, '$1').replace(/\s+/g, ' ').replace(/^[,\s]+/, '').trim();
++        if (text) out.push({ start: offsetSec + cur.t0 / 100, end: offsetSec + cur.t1 / 100, text });
++        cur = null;
++    };
++    for (const line of String(stderr || '').split('\n')) {
++        const m = TOKEN_LINE.exec(line);
++        if (!m) continue;
++        const t0 = Number(m[1]), t1 = Number(m[2]);
++        const wordStart = m[3] === 'true';
++        const raw = m[4];
++        if (/^<.*>$/.test(raw)) continue;                      // <unk>, <|…|> control tokens
++        const piece = raw.replace(/\u2581/g, ' ');
++        if (cur && wordStart && (t0 - cur.t1 > 150 || t0 - cur.t0 > 2000)) close();
++        if (!cur) cur = { t0, t1, text: '' };
++        cur.text += wordStart && !piece.startsWith(' ') ? ` ${piece}` : piece;
++        cur.t1 = Math.max(cur.t1, t1);
++        if (/[.!?]["')\]]?$/.test(piece.trim())) close();
++    }
++    close();
++    return out;
++}
++const CHUNK_SEC = 30;
++const CHUNK_MAX_SEC = 45;
++const wavSeconds = (p) => { try { return Math.max(0, (fs.statSync(p).size - 44) / 32000); } catch { return 0; } };   // 16 kHz mono s16le
++
  function createWhisperProvider(record, { config }) {
      const HOME = os.homedir();
      const w = config.whisper;
@@@ -71,6 +112,9 @@ function createWhisperProvider(record, { config }) {
      const vadModel = () => (w.vad && exists(VAD_MODEL) ? VAD_MODEL : null);
      const multi = () => (exists(MODEL_MULTI) ? MODEL_MULTI : null);
      const available = () => Boolean(bin()) && exists(MODEL);
++    const parakeetBin = () => [w.parakeetBin, bin() && path.join(path.dirname(bin()), 'parakeet-cli')].find(exists) || null;
++    const parakeetOn = () => w.engine !== 'whisper' && exists(w.parakeetModel) && Boolean(parakeetBin());
++    const engineFor = (language) => (parakeetOn() && PARAKEET_LANGS.has(String(language || 'en').slice(0, 2).toLowerCase()) ? 'parakeet' : 'whisper');
  
      const lanes = { live: { max: 1, running: 0, waiters: [] }, batch: { max: w.maxConcurrent, running: 0, waiters: [] } };
      const acquire = (lane) => { const L = lanes[lane]; if (L.running < L.max) { L.running++; return Promise.resolve(); } return new Promise(r => L.waiters.push(r)); };
@@@ -155,6 +199,53 @@ function createWhisperProvider(record, { config }) {
          });
      }
  
++    // One parakeet-cli run on a short WAV: the tokens on stderr become timed sentences.
++    function parakeetOnce(wavPath, { threads, offsetSec, signal, deadline }) {
++        return new Promise((resolve, reject) => {
++            let ch;
++            try { ch = spawnTracked('nice', ['-n', '15', parakeetBin(), '-t', String(threads), '-ps', '-np', '-m', w.parakeetModel, '-f', wavPath], { stderr: true }); }
++            catch (e) { return reject(new ProviderError(e.message, { code: 'provider.unavailable' })); }
++            const chunks = []; let size = 0;
++            if (ch.stderr) ch.stderr.on('data', (d) => { if (size < 64 * 1024 * 1024) { chunks.push(d); size += d.length; } });
++            const kill = () => { try { ch.kill('SIGKILL'); } catch { /* */ } };
++            const timer = setTimeout(kill, Math.max(1000, deadline - Date.now()));
++            if (signal) signal.addEventListener('abort', kill, { once: true });
++            ch.on('close', (code) => {
++                clearTimeout(timer);
++                if (signal && signal.aborted) return reject(new ProviderError('cancelled', { code: 'provider.cancelled' }));
++                if (code !== 0) return reject(new ProviderError(`parakeet exited ${code}`, { code: 'provider.error' }));
++                resolve(parseParakeet(Buffer.concat(chunks).toString('utf8'), offsetSec));
++            });
++            ch.on('error', (e) => { clearTimeout(timer); reject(new ProviderError(e.message, { code: 'provider.unavailable' })); });
++        });
++    }
++    // Parakeet on a whole WAV: in 30 s pieces past 45 s (PCM cut exactly, so a piece's times shift only by its start).
++    async function decodeParakeet(wavPath, { language = 'en', live = false, lowPower = false, offsetSec = 0, signal, timeoutMs }) {
++        const threads = live || lowPower ? Math.max(1, Math.min(2, w.threads)) : w.threads;
++        const deadline = Date.now() + timeoutMs;
++        let pieces = [wavPath], dir = null;
++        if (wavSeconds(wavPath) > CHUNK_MAX_SEC) {
++            dir = fs.mkdtempSync(path.join(os.tmpdir(), 'openvibe-ai-pk-'));
++            await new Promise((resolve, reject) => {
++                const ff = spawnTracked('nice', ['-n', '15', 'ffmpeg', '-nostdin', '-loglevel', 'error', '-y', '-i', wavPath, '-f', 'segment', '-segment_time', String(CHUNK_SEC), '-c', 'copy', path.join(dir, 'p%04d.wav')]);
++                ff.on('close', (code) => (code === 0 ? resolve() : reject(new ProviderError(`ffmpeg split exited ${code}`, { code: 'provider.error' }))));
++                ff.on('error', (e) => reject(new ProviderError(e.message, { code: 'provider.unavailable' })));
++            });
++            pieces = fs.readdirSync(dir).filter((n) => /^p\d{4}\.wav$/.test(n)).sort().map((n) => path.join(dir, n));
++        }
++        try {
++            const raw = [];
++            for (let i = 0; i < pieces.length; i++) {
++                if (Date.now() >= deadline) throw new ProviderError('parakeet timed out', { code: 'provider.error' });
++                raw.push(...await parakeetOnce(pieces[i], { threads, offsetSec: offsetSec + i * CHUNK_SEC, signal, deadline }));
++            }
++            const segments = cleanSegments(raw, true);
++            return { text: segments.map(s => s.text).join(' ').replace(/\s+/g, ' ').trim(), segments, language: String(language || 'en'), model: path.basename(w.parakeetModel) };
++        } finally {
++            if (dir) { try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* */ } }
++        }
++    }
++
      async function transcribe(req) {
          if (!available()) throw new ProviderError('whisper.cpp is not installed on this host', { code: 'provider.unavailable' });
          const lane = req.live ? 'live' : 'batch';
@@@ -163,7 +254,17 @@ function createWhisperProvider(record, { config }) {
          try {
              await acquire(lane);
              try {
--                const r = await decode(wav, { language: req.language, live: req.live, lowPower: req.lowPower === true, offsetSec: req.offsetSec || 0, signal: req.signal, timeoutMs });
++                const opts = { language: req.language, live: req.live, lowPower: req.lowPower === true, offsetSec: req.offsetSec || 0, signal: req.signal, timeoutMs };
++                let r = null;
++                if (engineFor(req.language) === 'parakeet') {
++                    try { r = await decodeParakeet(wav, opts); }
++                    catch (e) {
++                        if (e && e.code === 'provider.cancelled') throw e;
++                        // A Parakeet failure (a crash, a bad model file) must not leave a hole: whisper takes the same audio.
++                        console.warn(`[whisper] parakeet failed (${e.message}); whisper takes this audio`);
++                    }
++                }
++                if (!r) r = await decode(wav, opts);
                  return { ...r, usage: { input: 0, output: 0, cached: 0 } };
              } finally { release(lane); }
          } finally { try { fs.unlinkSync(wav); } catch { /* */ } }
@@@ -174,9 +275,9 @@ function createWhisperProvider(record, { config }) {
          kind: 'whisper',
          supports: (f) => f === 'transcribe' && available(),
          transcribe,
--        describe: () => ({ available: available(), model: path.basename(MODEL), multilingual: Boolean(multi()), vad: Boolean(vadModel()), lanes: Object.fromEntries(Object.entries(lanes).map(([k, L]) => [k, { running: L.running, queued: L.waiters.length }])) }),
++        describe: () => ({ available: available(), model: path.basename(MODEL), parakeet: parakeetOn() ? path.basename(w.parakeetModel) : null, multilingual: Boolean(multi()), vad: Boolean(vadModel()), lanes: Object.fromEntries(Object.entries(lanes).map(([k, L]) => [k, { running: L.running, queued: L.waiters.length }])) }),
          killActive() { for (const ch of active) { try { ch.kill('SIGKILL'); } catch { /* */ } } active.clear(); },
      };
  }
  
--module.exports = { createWhisperProvider, cleanSegments, isNoise };
++module.exports = { createWhisperProvider, cleanSegments, isNoise, parseParakeet };
ddiff --git a/test/parakeet.test.js b/test/parakeet.test.js
nnew file mode 100644
iindex 0000000..d474b27
---- /dev/null
++++ b/test/parakeet.test.js
@@@ -0,0 +1,90 @@
++'use strict';
++// Parakeet in the speech-to-text provider (as Live has had since 10-16): with PARAKEET_MODEL set, English and the 24
++// European languages go to parakeet-cli, audio over 45 s in 30 s pieces whose times shift by the piece's start, fillers
++// dropped; Japanese stays on whisper; a Parakeet failure falls back to whisper on the same audio; ASR_ENGINE=whisper
++// forces whisper. Fake parakeet-cli and whisper-cli stand in for the real ones. Needs ffmpeg; skipped without it.
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const { spawnSync } = require('child_process');
++const { createWhisperProvider, parseParakeet } = require('../server/providers/whisper');
++
++if (spawnSync('ffmpeg', ['-version']).status !== 0) {
++    console.log('parakeet: skipped (no ffmpeg)');
++    process.exit(0);
++}
++
++(async () => {
++    console.warn = () => {};
++    const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-ai-parakeet-'));
++    const whisperBin = path.join(tmp, 'whisper-cli');
++    fs.writeFileSync(whisperBin, [
++        '#!/bin/sh',
++        'of=""',
++        'while [ $# -gt 0 ]; do if [ "$1" = "-of" ]; then of="$2"; fi; shift; done',
++        'printf \'{"transcription":[{"offsets":{"from":0,"to":1000},"text":" from whisper"}],"result":{"language":"en"}}\' > "$of.json"',
++    ].join('\n') + '\n', { mode: 0o755 });
++    // The fake parakeet-cli sits next to whisper-cli (where the provider looks without PARAKEET_BIN) and names the piece
++    // it decoded in its last word, so every piece's sentence is different.
++    const log = path.join(tmp, 'pk.log');
++    fs.writeFileSync(path.join(tmp, 'parakeet-cli'), [
++        '#!/bin/sh',
++        'f=""',
++        'while [ $# -gt 0 ]; do if [ "$1" = "-f" ]; then f="$2"; fi; shift; done',
++        `echo "$f" >> ${log}`,
++        'if [ -n "$PK_FAIL" ]; then exit 1; fi',
++        'b=$(basename "$f" .wav)',
++        'echo \'tok t0=  10 t1=  40 word_start=true "\u2581Hello"\' >&2',
++        'echo \'tok t0=  45 t1=  60 word_start=true "\u2581um"\' >&2',
++        'echo "tok t0=  62 t1=  80 word_start=true \\"\u2581$b.\\"" >&2',
++    ].join('\n') + '\n', { mode: 0o755 });
++    const model = path.join(tmp, 'model.bin'), pk = path.join(tmp, 'parakeet.bin');
++    fs.writeFileSync(model, 'x'); fs.writeFileSync(pk, 'x');
++    const media = path.join(tmp, 'talk.wav');
++    assert.strictEqual(spawnSync('ffmpeg', ['-y', '-loglevel', 'error', '-f', 'lavfi', '-i', 'anullsrc=r=16000:cl=mono', '-t', '70', media]).status, 0);
++
++    const make = (extra = {}) => createWhisperProvider({ key: 'whisper' }, { config: { whisper: { bin: whisperBin, model, threads: 2, beam: 1, maxConcurrent: 1, vad: false, parakeetModel: pk, ...extra } } });
++    const lines = () => (fs.existsSync(log) ? fs.readFileSync(log, 'utf8').trim().split('\n').filter(Boolean) : []);
++    try {
++        let w = make();
++        let r = await w.transcribe({ filePath: media, language: 'en', timeoutMs: 60000 });
++        assert.strictEqual(r.model, 'parakeet.bin');
++        assert.strictEqual(lines().length, 3, '70 s goes to Parakeet in three pieces');
++        assert.deepStrictEqual(r.segments.map((s) => [s.start, s.text]), [[0.1, 'Hello p0000.'], [30.1, 'Hello p0001.'], [60.1, 'Hello p0002.']]);
++        assert.strictEqual(r.text, 'Hello p0000. Hello p0001. Hello p0002.');
++        assert.strictEqual(w.describe().parakeet, 'parakeet.bin');
++        console.log('✓ English: Parakeet in 30 s pieces, times shifted by each piece, fillers dropped');
++
++        r = await w.transcribe({ filePath: media, language: 'ja', timeoutMs: 60000 });
++        assert.deepStrictEqual([r.model, r.text, lines().length], ['model.bin', 'from whisper', 3], 'Japanese stays on whisper');
++        r = await w.transcribe({ filePath: media, language: 'el', seconds: 20, timeoutMs: 60000 });
++        assert.deepStrictEqual([r.model, lines().length], ['parakeet.bin', 4], 'Greek is one of Parakeet\'s languages; 20 s is one piece');
++        console.log('✓ Japanese on whisper; Greek on Parakeet');
++
++        process.env.PK_FAIL = '1';
++        r = await w.transcribe({ filePath: media, language: 'en', seconds: 20, timeoutMs: 60000 });
++        delete process.env.PK_FAIL;
++        assert.deepStrictEqual([r.model, r.text], ['model.bin', 'from whisper'], 'a Parakeet failure: whisper takes the audio');
++        console.log('✓ a Parakeet failure falls back to whisper');
++
++        w = make({ engine: 'whisper' });
++        r = await w.transcribe({ filePath: media, language: 'en', seconds: 20, timeoutMs: 60000 });
++        assert.strictEqual(r.model, 'model.bin');
++        w = make({ parakeetModel: null });
++        assert.strictEqual(w.describe().parakeet, null, 'no PARAKEET_MODEL: whisper only');
++        console.log('✓ ASR_ENGINE=whisper and no PARAKEET_MODEL: whisper');
++
++        // A pause of 1.5 s or 20 s of talk ends a sentence; control tokens are skipped.
++        const out = parseParakeet([
++            'x t0=   0 t1=  20 word_start=true "\u2581so"', 'x t0=  20 t1=  30 word_start=false "<unk>"',
++            'x t0= 300 t1= 320 word_start=true "\u2581anyway"', 'x t0= 320 t1= 340 word_start=false "!"',
++        ].join('\n'), 100);
++        assert.deepStrictEqual(out, [{ start: 100, end: 100.2, text: 'so' }, { start: 103, end: 103.4, text: 'anyway!' }]);
++        console.log('✓ parseParakeet: pauses split sentences, control tokens skipped');
++    } finally {
++        fs.rmSync(tmp, { recursive: true, force: true });
++    }
++    console.log('parakeet: all checks passed');
++    process.exit(0);
++})().catch((e) => { console.error(e); process.exit(1); });