openvibers/openvibe.ai Commits

AI: telemetry flushes each minute and keeps 14 days (#86)

OP OpenVibers committed 29 days ago · c403eae56a · 2 files, +6 −6

The request telemetry added earlier today flushed every 15 s and kept 30 days. The idle gauges alone write three rows per flush. Measured in production, that is about 650 rows an hour at about 270 B each including the indexes. At 30 days that's about 470k rows and 125 MB per service, roughly 4.4 GB across the estate on a disk with 16 GB free.

This changes it to one flush a minute and 14 days of retention: about 55k rows and 15 MB per service.

- The collector's interval (the tick, or `intervalMs` for `createHttpTelemetry` and `telemetry.init`) is 60 s.
- `startTelemetryRetention` passes `olderThanDays: 14`.
- The comments and `test/telemetry.test.js` follow: the retention check inserts rows 20 days and 1 day old and expects one deleted with `olderThanDays: 14`.

Applied migrations are unchanged (their comment still says 30 days; a migration that has run is never edited).

Changes

ddiff --git a/server/index.js b/server/index.js
iindex 9b0781b..497759a 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -120,9 +120,9 @@ if (require.main === module) {
          // then handles.close() (runs drained, the relay stopped, the database closed; a rejection exits 1);
          // past 10 s the process exits 1.
          // Request telemetry (openvibe-sdk/service, plan T1): one aggregated sample per route template, method and status
--        // class each 15 s, stored in this service's telemetry_samples table; rows older than 30 days go daily.
--        telemetry.init({ service: 'ai', sink: createPgTelemetrySink({ db: handles.db, log: console }), skipPrefixes: ['/shared'] });
--        const telemetryRetention = startTelemetryRetention({ db: handles.db, log: console });
++        // class each minute, stored in this service's telemetry_samples table; rows older than 14 days go daily.
++        telemetry.init({ service: 'ai', sink: createPgTelemetrySink({ db: handles.db, log: console }), skipPrefixes: ['/shared'], intervalMs: 60_000 });
++        const telemetryRetention = startTelemetryRetention({ db: handles.db, log: console, olderThanDays: 14 });
          gracefulStop({ name: 'ai', server: handles.server, handles, drainMs: 8000, deadlineMs: 10000, stop: [async () => { if (telemetryRetention) await telemetryRetention.stop(); await telemetry.stop(); }] });
      }).catch((err) => {
          console.error(`[ai] failed to start: ${err.stack || err}`);
ddiff --git a/test/telemetry.test.js b/test/telemetry.test.js
iindex 5b702e0..42b22c7 100644
---- a/test/telemetry.test.js
++++ b/test/telemetry.test.js
@@@ -3,7 +3,7 @@
   * Request telemetry (openvibe-sdk/service, plan T1 step 10): app.js mounts the process-wide telemetryMiddleware (a no-op
   * until init), server/index.js calls telemetry.init() for service 'ai' with a PostgreSQL sink, starts the 30-day
   * retention and flushes on stop. Five hits on one route flush as ONE aggregated http.request sample; /metrics and
-- * /shared make none; the retention deletes only rows older than 30 days.
++ * /shared make none; the retention deletes only rows older than 14 days.
   */
  const assert = require('assert');
  const fs = require('fs');
@@@ -38,9 +38,9 @@ const { boot, request } = require('./helpers');
          const db = t.db;
          await db.query('DELETE FROM telemetry_samples');
          const insert = (days) => db.query("INSERT INTO telemetry_samples (at, service, operation) VALUES (now() - make_interval(days => $1), 'ai', 'http.request')", [days]);
--        await insert(45);
++        await insert(20);
          await insert(1);
--        assert.strictEqual(await pruneTelemetry({ db }), 1);
++        assert.strictEqual(await pruneTelemetry({ db, olderThanDays: 14 }), 1);
          assert.strictEqual((await db.query('SELECT count(*)::int AS n FROM telemetry_samples')).rows[0].n, 1);
          console.log('telemetry ok');
      } finally {