openvibers/openvibe.live Commits

Backend: cut boot from ~18s, index the home-page hot paths, and bound upstream calls

OP OpenVibers committed 16 Sept 2026 · 647a9fec05 · 3 files, +102 −11

Measured against a synthetic production-scale database (2M chat_messages, 400k timeline events,
8k streams) built from the app's own schema and migrations.

BOOT — this is the deploy downtime. _adoptOrphanedTimelineRows() ran unconditionally before
listen() and is a full scan of stream_timeline_events that SQLite will not index. With a backlog
it measured 10.6 seconds; with nothing at all to do it still cost 60-110ms. Speech rows are
written with a null vod_id while a stream is live, so there is a fresh backlog after most
restarts — which is why production took ~18 seconds to start while a local run took under one.
It now probes with a partial index first (0.129ms to answer "is there anything to adopt?") and
runs after listen() rather than in front of it. It is a backlog repair; no request depends on it.

INDEXES, each replacing a measured scan:
  - streams(managed_stream_id, ended_at). getRecentlyOnlineStreamers correlates streams by
    managed_stream_id inside a json_group_array and there was no index on that column at all, so
    the subquery scanned the whole streams table once per managed stream per row. 463ms -> 18ms at
    limit=20; 1533ms -> 24ms at limit=100.
  - chat_messages(timestamp, is_deleted). The hero stat board's heaviest counts filter on
    COALESCE(is_deleted,0)=0, which forced a row fetch for every row in the window even with the
    timestamp index in use.
  - ai_viewer_log(created_at), for the six-hourly prune that had none.

SEO RENDER — six regexes, five of them global, ran across the whole 378KB document to rewrite
tags that only ever live in the first 8.5KB. 60ms of blocking CPU per uncached render, against
0.73ms for the head alone. They now run on the head and the rest is concatenated back. Output
verified byte-identical in shape: one title, one canonical, no duplicates.

UPSTREAM DEADLINES — media-client's request() cleared its timeout in a finally attached to the
fetch() await, so the deadline expired when the response *headers* arrived and reading the body
had no limit at all. An upstream that answers and then stalls mid-body pinned the handler
indefinitely. The body is now read inside the guarded region so one deadline covers the exchange.
proxy() had no timeout whatsoever and backs pastes, likes, comments and admin storage: it gets 20s
for reads and 120s for uploads, aborts the upstream when the client disconnects, and returns 504
rather than hanging.

FIRST BOOT — a PRAGMA table_info(pastes) sat several hundred lines above the CREATE TABLE for
pastes and shared a try block with unrelated migrations, so on a brand-new database it threw and
took them with it: three ai_overview columns and channels.ai_category were never added, which
aborted the next block and left streamer_overviews.overview_short missing. Since
getRecentlyOnlineStreamers selects that column, a first boot served 500s until the process was
restarted a second time. Isolated, and verified on a fresh database that overview_short now
exists and the query runs on boot one.

Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01CMxJm1dB8rdsaMSgNjVa82

Changes

ddiff --git a/server/db/database.js b/server/db/database.js
iindex 491ce0e3..a0726d07 100644
---- a/server/db/database.js
++++ b/server/db/database.js
@@@ -226,6 +226,19 @@ function _dedupeStreamMemories(database) {
   */
  function _adoptOrphanedTimelineRows(database) {
      try {
++        // Cheap probe first. The UPDATE below is a full scan of stream_timeline_events — SQLite
++        // will not use a partial index for it — and it ran unconditionally on every boot, before
++        // listen(). Measured against a production-scale table it was 10.6 seconds with a backlog
++        // and still 60-110ms with nothing at all to do, and it was the single largest component
++        // of an 18-second production start. Speech rows are written with vod_id NULL while a
++        // stream is live, so there is a fresh backlog after most restarts.
++        //
++        // The partial index makes "is there anything to adopt?" a 0.1ms question. When the answer
++        // is no — the overwhelmingly common case — boot skips the scan entirely.
++        try { database.exec('CREATE INDEX IF NOT EXISTS idx_timeline_null_vod ON stream_timeline_events(stream_id) WHERE vod_id IS NULL'); } catch { /* */ }
++        const pending = database.prepare('SELECT 1 AS x FROM stream_timeline_events WHERE vod_id IS NULL LIMIT 1').get();
++        if (!pending) return;
++
          const res = database.prepare(`UPDATE stream_timeline_events AS t
              SET vod_id = (SELECT s.vod_id FROM stream_timeline_events s
                            WHERE s.stream_id = t.stream_id AND s.vod_id IS NOT NULL LIMIT 1)
@@@ -1218,7 +1231,9 @@ function initDb() {
          _dedupeKeyedTable(database, 'vod_ai_state', 'vod_id');
          _dedupeKeyedTable(database, 'clip_ai_state', 'clip_id');
          _dedupeStreamMemories(database);
--        _adoptOrphanedTimelineRows(database);
++        // Deferred: this repairs a historical backlog and no request depends on it, so it must
++        // not sit between process start and the first served request. Runs shortly after boot.
++        setTimeout(() => { try { _adoptOrphanedTimelineRows(database); } catch { /* */ } }, 4000).unref?.();
  
          // ── Unified audio timeline ───────────────────────────────────────────────
          // One time-indexed row per thing heard on a stream: a phrase that was spoken
@@@ -1360,10 +1375,25 @@ function initDb() {
              updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
          )`);
  
--        const pcols = database.prepare('PRAGMA table_info(pastes)').all().map(c => c.name);
--        if (!pcols.includes('ai_summary')) database.exec('ALTER TABLE pastes ADD COLUMN ai_summary TEXT');
--        if (!pcols.includes('ai_tags')) database.exec('ALTER TABLE pastes ADD COLUMN ai_tags TEXT');
--        if (!pcols.includes('ai_analyzed_at')) database.exec('ALTER TABLE pastes ADD COLUMN ai_analyzed_at DATETIME');
++        // `pastes` is created several hundred lines below this point, so on a brand-new database
++        // this PRAGMA throws — and because it shares a try block with the migrations that follow,
++        // it took them down with it: streams.ai_overview, vods.ai_overview, clips.ai_overview and
++        // channels.ai_category were never added, which in turn aborted the short-overview block
++        // and left streamer_overviews.overview_short missing. getRecentlyOnlineStreamers() selects
++        // that column, so a first boot served 500s on /api/streams/recently-online until the
++        // process was restarted a second time and the table finally existed.
++        //
++        // Isolated in its own try so a missing table on first boot costs nothing but a skip; the
++        // columns are added on the next start, and the paste AI columns are also declared at the
++        // CREATE TABLE below for fresh databases.
++        try {
++            const pcols = database.prepare('PRAGMA table_info(pastes)').all().map(c => c.name);
++            if (pcols.length) {
++                if (!pcols.includes('ai_summary')) database.exec('ALTER TABLE pastes ADD COLUMN ai_summary TEXT');
++                if (!pcols.includes('ai_tags')) database.exec('ALTER TABLE pastes ADD COLUMN ai_tags TEXT');
++                if (!pcols.includes('ai_analyzed_at')) database.exec('ALTER TABLE pastes ADD COLUMN ai_analyzed_at DATETIME');
++            }
++        } catch { /* table not created yet on a first boot — added on the next start */ }
  
          const scols = database.prepare('PRAGMA table_info(streams)').all().map(c => c.name);
          if (!scols.includes('ai_overview')) database.exec('ALTER TABLE streams ADD COLUMN ai_overview TEXT');
@@@ -2208,6 +2238,16 @@ function initDb() {
          'CREATE INDEX IF NOT EXISTS idx_vods_transcript ON vods(transcript_status)',
          'CREATE INDEX IF NOT EXISTS idx_clips_transcript ON clips(transcript_status)',
          'CREATE INDEX IF NOT EXISTS idx_viewer_samples_at ON viewer_samples(sampled_at)',
++        // getRecentlyOnlineStreamers() correlates streams by managed_stream_id inside a
++        // json_group_array, and there was no index on that column at all — so the subquery scanned
++        // the whole streams table once per managed stream, per row. Measured on a production-scale
++        // database: 463ms -> 18ms at limit=20, and 1533ms -> 24ms at limit=100.
++        'CREATE INDEX IF NOT EXISTS idx_streams_managed_ended ON streams(managed_stream_id, ended_at)',
++        // The hero stat board's heaviest queries count chat_messages over a time window with
++        // COALESCE(is_deleted,0)=0, which forced a row fetch for every row in the window even
++        // though the timestamp index was being used. These make those counts covering.
++        'CREATE INDEX IF NOT EXISTS idx_chat_ts_deleted ON chat_messages(timestamp, is_deleted)',
++        'CREATE INDEX IF NOT EXISTS idx_ai_viewer_log_created ON ai_viewer_log(created_at)',
      ]) {
          try { database.exec(ix); } catch { /* table not present in this deployment */ }
      }
ddiff --git a/server/media-client.js b/server/media-client.js
iindex 5b929764..6beb6ea2 100644
---- a/server/media-client.js
++++ b/server/media-client.js
@@@ -84,15 +84,28 @@ async function request(method, apiPath, { body, query, actingUser, headers = {},
      const timer = setTimeout(() => ctrl.abort(), timeoutMs);
      opts.signal = ctrl.signal;
      let res;
++    let text = '';
      try {
          res = await fetch(url, opts);
++        // The deadline has to cover reading the body, not just receiving the headers.
++        //
++        // clearTimeout used to sit in a `finally` attached to the fetch() await, so the timer was
++        // cancelled and the abort signal disarmed the moment headers arrived. Reading the body
++        // then had no deadline at all: an upstream that responds and then stalls mid-body pinned
++        // this handler — and whatever request was waiting on it — indefinitely. Consuming the body
++        // inside the same guarded region means one deadline covers the whole exchange.
++        text = await res.text().catch(() => '');
      } catch (err) {
--        throw new MediaApiError(`Media unreachable (${method} ${apiPath}): ${err.message}`, 0, null);
++        const aborted = err && (err.name === 'AbortError' || ctrl.signal.aborted);
++        throw new MediaApiError(
++            aborted
++                ? `Media timed out after ${timeoutMs}ms (${method} ${apiPath})`
++                : `Media unreachable (${method} ${apiPath}): ${err.message}`,
++            0, null);
      } finally {
          clearTimeout(timer);
      }
      let json = null;
--    const text = await res.text().catch(() => '');
      if (text) { try { json = JSON.parse(text); } catch { json = null; } }
      if (!res.ok) {
          const msg = (json && (json.error || json.message)) || `Media API ${res.status} on ${method} ${apiPath}`;
@@@ -326,13 +339,35 @@ async function proxy(req, res, apiPath, { actingUser, method, query, body } = {}
              opts.body = req.rawBody;
          }
      }
++    // proxy() had no deadline at all — an unresponsive upstream held the connection, the handler
++    // and its socket open indefinitely. This backs pastes, likes, comments and the admin storage
++    // routes, so a stalled Media could accumulate stuck handlers until the process ran out.
++    // Uploads legitimately take longer than reads, hence the two budgets.
++    const isUpload = !['GET', 'HEAD'].includes(m);
++    const ctrl = new AbortController();
++    const timer = setTimeout(() => ctrl.abort(), isUpload ? 120000 : 20000);
++    opts.signal = ctrl.signal;
++    // If the client goes away mid-proxy, stop waiting on the upstream too.
++    const onClientGone = () => { try { ctrl.abort(); } catch { /* */ } };
++    req.on('aborted', onClientGone);
++    res.on('close', onClientGone);
++    const cleanup = () => {
++        clearTimeout(timer);
++        req.off?.('aborted', onClientGone);
++        res.off?.('close', onClientGone);
++    };
++
      let upstream;
      try {
          upstream = await fetch(url, opts);
      } catch (err) {
--        console.warn(`[MediaClient] proxy failed (${m} ${apiPath}):`, err.message);
--        return res.status(502).json({ error: 'Media service unavailable' });
++        cleanup();
++        const aborted = err && (err.name === 'AbortError' || ctrl.signal.aborted);
++        console.warn(`[MediaClient] proxy ${aborted ? 'timed out' : 'failed'} (${m} ${apiPath}):`, err.message);
++        if (res.headersSent) return;
++        return res.status(504).json({ error: aborted ? 'Media service timed out' : 'Media service unavailable' });
      }
++    cleanup();
      res.status(upstream.status);
      const passHeaders = ['content-type', 'cache-control', 'content-disposition', 'etag', 'x-robots-tag'];
      for (const h of passHeaders) {
ddiff --git a/server/seo/seo.js b/server/seo/seo.js
iindex 9dea3059..3cce1ee1 100644
---- a/server/seo/seo.js
++++ b/server/seo/seo.js
@@@ -439,17 +439,33 @@ function _headBlock(meta) {
  function render(meta) {
      let html = _base();
      if (!html) return null;
++
++    // Only the <head> is rewritten, so only the <head> is scanned.
++    //
++    // These six patterns used to run over the whole document. index.html is ~378KB and the head
++    // is the first ~8.5KB of it, so five /ig regexes were scanning 370KB of page markup that
++    // cannot contain a <title> or an og: tag — measured at 60ms of blocking CPU per uncached
++    // render, against 0.73ms for the head alone. That cost lands on every cache miss, and the
++    // sitemap advertises far more URLs than the 500-entry cache holds, so a crawler walking it
++    // misses every time.
++    const headEnd = html.search(/<\/head>/i);
++    let head = headEnd === -1 ? html : html.slice(0, headEnd);
++    const rest = headEnd === -1 ? '' : html.slice(headEnd);
++
      // Strip the hardcoded homepage tags we're replacing (title, description, canonical, all
      // og:/twitter:, robots) so there are no duplicates.
--    html = html
++    head = head
          .replace(/\s*<title>[\s\S]*?<\/title>/i, '')
          .replace(/\s*<meta\s+name=["']description["'][^>]*>/ig, '')
          .replace(/\s*<meta\s+name=["']robots["'][^>]*>/ig, '')
          .replace(/\s*<link\s+rel=["']canonical["'][^>]*>/ig, '')
          .replace(/\s*<meta\s+property=["']og:[^"']*["'][^>]*>/ig, '')
          .replace(/\s*<meta\s+name=["']twitter:[^"']*["'][^>]*>/ig, '');
++
      // Insert the fresh head block just before </head>.
--    html = html.replace(/<\/head>/i, _headBlock(meta) + '</head>');
++    html = headEnd === -1
++        ? head.replace(/<\/head>/i, _headBlock(meta) + '</head>')
++        : head + _headBlock(meta) + rest;
      // Server-rendered crawlable content, right after <body>. It's REAL content (so AI text
      // extractors read it — unlike <noscript>, which many strip), but visually-hidden so users
      // never see a flash, and the SPA removes #seo-prerender on boot (see app.js). Not cloaking: