openvibers/openvibe.host Commits

The deploy controller, phase 1 (roadmap WS-X11, D72): ovhost reconcile. Main at its last green CI commit is the desired state, and a 5-minute timer makes each service whose inventory says deploy.policy auto run it without a person. Per pass and service: plan with a fetch; up to date or only documentation and tests changed (the baseline's rule) means nothing; otherwise GitHub's check runs for that exact commit must all be completed and passed (none, running, red or unreachable wait or block, never deploy on hope); then the same lib.deploy as ovhost deploy --wait-idle (freezes, protected sessions, readiness, automatic rollback) and the release notification. gated and manual services are only reported. Each pass goes to /var/lib/openvibe-host/reconcile.json and the textfile collector; OpenVibeReconcileFailed (page) and OpenVibeReconcileStalled (ticket) read it. The inventory validates deploy.policy (default manual). openvibe-reconcile.service and .timer; an optional read-only token in /etc/openvibe/reconcile.env. test/reconcile.test.js on the fake host: docs-only deploys nothing; no CI, pending and red never deploy; green deploys, and the next pass finds it current; manual waits; a dry run writes nothing; GitHub unreachable is unknown. npm test 40/40.

OP OpenVibers committed 28 Sept 2026 · 1bfffabb4c · 9 files, +281 −0





Changes
ddiff --git a/deploy/prometheus/openvibe-rules.yml b/deploy/prometheus/openvibe-rules.yml
iindex f444ead..55442bd 100644
---- a/deploy/prometheus/openvibe-rules.yml
++++ b/deploy/prometheus/openvibe-rules.yml
@@@ -146,3 +146,18 @@ groups:
          labels: { severity: ticket }
          annotations:
            summary: "Valkey above 90% of maxmemory: only TTL keys can be evicted"
++  # The deploy controller (ovhost reconcile, WS-X11).
++  - name: openvibe-deploy-controller
++    rules:
++      - alert: OpenVibeReconcileFailed
++        expr: openvibe_reconcile_failed_total_last > 0
++        for: 1m
++        labels: { severity: page }
++        annotations:
++          summary: "ovhost reconcile: an automatic deploy failed or rolled back (see /var/lib/openvibe-host/reconcile.json)"
++      - alert: OpenVibeReconcileStalled
++        expr: time() - openvibe_reconcile_last_run_timestamp_seconds > 3600
++        for: 15m
++        labels: { severity: ticket }
++        annotations:
++          summary: "ovhost reconcile has not completed a pass for an hour: auto services are not following main"
ddiff --git a/deploy/systemd/openvibe-reconcile.service b/deploy/systemd/openvibe-reconcile.service
nnew file mode 100644
iindex 0000000..740ccc6
---- /dev/null
++++ b/deploy/systemd/openvibe-reconcile.service
@@@ -0,0 +1,18 @@
++[Unit]
++Description=OpenVibe deploy controller: auto services follow main at green CI (ovhost reconcile, WS-X11)
++Documentation=https://github.com/OpenVibers/OpenVibe.Host/blob/main/docs/deploy-strategies.md
++After=network-online.target
++Wants=network-online.target
++
++[Service]
++Type=oneshot
++# Root, as `ovhost deploy`: it fetches and installs as each checkout's owner, writes units and vhosts, and restarts.
++User=root
++ExecStart=/usr/local/bin/ovhost reconcile
++Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++Environment=OVHOST_INVENTORY=/etc/openvibe/host.json
++# OVHOST_GITHUB_TOKEN (read-only, optional: raises GitHub's 60-an-hour anonymous limit; O19)
++EnvironmentFile=-/etc/openvibe/reconcile.env
++UMask=0022
++TimeoutStartSec=30min
++Nice=5
ddiff --git a/deploy/systemd/openvibe-reconcile.timer b/deploy/systemd/openvibe-reconcile.timer
nnew file mode 100644
iindex 0000000..873a2d3
---- /dev/null
++++ b/deploy/systemd/openvibe-reconcile.timer
@@@ -0,0 +1,10 @@
++[Unit]
++Description=Run the OpenVibe deploy controller every 5 minutes (WS-X11)
++
++[Timer]
++OnBootSec=5min
++OnUnitActiveSec=5min
++AccuracySec=30s
++
++[Install]
++WantedBy=timers.target
ddiff --git a/docs/deploy-strategies.md b/docs/deploy-strategies.md
iindex 8c3064e..8241b3b 100644
---- a/docs/deploy-strategies.md
++++ b/docs/deploy-strategies.md
@@@ -313,3 +313,39 @@ comments: the `//` notes below are for reading only.
    "installUnits": true                           // the unit file is installed whenever it differs
  }
  ```
++
++## The deploy controller: `ovhost reconcile` (roadmap WS-X11 phase 1, D72)
++
++Main at its last green CI commit is the desired state. `openvibe-reconcile.timer` runs `ovhost reconcile` every
++5 minutes. For each service whose inventory entry has `"deploy": { "policy": "auto" }` (default `manual`):
++
++1. It plans with a fetch. Up to date: nothing happens.
++2. If the new commits change only documentation and tests (`README.md`, `STATUS.json`, `docs/*.md`, `test/`,
++   `.github/`, `*.test.*`), it does nothing; the change rides the next deploy.
++3. It reads GitHub's check runs for that exact commit. It deploys only when every one completed and passed.
++   None yet, one still running, a failure, or GitHub unreachable all wait for the next pass (or block, on red).
++4. It deploys with the same code as `ovhost deploy --wait-idle` (freezes, protected sessions, readiness and
++   automatic rollback all apply), then sends the release notification.
++
++`gated` and `manual` services are listed with what waits for a person. They are never deployed by the
++controller. Each pass is written to `/var/lib/openvibe-host/reconcile.json` and to the textfile collector.
++Alerts: `OpenVibeReconcileFailed` (page: an automatic deploy failed or rolled back) and `OpenVibeReconcileStalled`
++(ticket: no pass for an hour).
++
++```bash
++sudo ovhost reconcile --dry-run          # what it would do now
++sudo ovhost reconcile wiki blog          # one pass for some services
++sudo systemctl list-timers openvibe-reconcile.timer
++```
++
++Install (once):
++
++```bash
++sudo cp /opt/openvibe.host/deploy/systemd/openvibe-reconcile.{service,timer} /etc/systemd/system/
++sudo systemctl daemon-reload && sudo systemctl enable --now openvibe-reconcile.timer
++```
++
++An optional read-only GitHub token (`OVHOST_GITHUB_TOKEN` in `/etc/openvibe/reconcile.env`, O19) lifts the
++60-requests-an-hour anonymous limit. Without one, only a commit that changed code costs API calls.
++Later phases (WS-X11): CI-built signed packages instead of `git` and `npm` on the host, contract-ordered waves,
++canaries, and the GitHub App webhook for immediate passes.
ddiff --git a/lib/cli.js b/lib/cli.js
iindex d6544d6..4c78c95 100644
---- a/lib/cli.js
++++ b/lib/cli.js
@@@ -318,6 +318,19 @@ async function main(argv, { exec = lib.createSystemExecutor(), out = (s) => proc
              }
              return 0;
          }
++        case 'reconcile': {
++            // The deploy controller, phase 1 (WS-X11): auto services follow main at green CI (lib/reconcile.js).
++            const only = pos.slice(1).length ? pos.slice(1) : null;
++            const pass = await lib.reconcile(ctx, { only, dryRun: !!opts['dry-run'], token: env.OVHOST_GITHUB_TOKEN || env.GITHUB_TOKEN || null, env });
++            if (json) { emit(pass); return pass.services.some((x) => x.action === 'failed' || x.action === 'error') ? 1 : 0; }
++            for (const x of pass.services) {
++                if (x.action === 'current' && !opts.verbose) continue;
++                out(`${x.id.padEnd(10)} ${String(x.policy).padEnd(7)} ${x.action}${x.from && x.to && x.from !== x.to ? ` ${short(x.from)} → ${short(x.to)}` : ''}${x.detail ? `: ${x.detail}` : ''}`);
++            }
++            const n = (a) => pass.services.filter((x) => x.action === a).length;
++            out(`[ovhost] reconcile${pass.dryRun ? ' (dry run)' : ''}: ${n('deployed')} deployed, ${n('current')} current, ${n('docs-only')} docs-only, ${n('waiting')} waiting, ${n('blocked')} blocked, ${n('failed') + n('error')} failed`);
++            return n('failed') + n('error') ? 1 : 0;
++        }
          case 'releases': {
              const id = lib.inventory.service(inv, needService()).id;
              const list = await lib.releases.list(exec, inv, id);
ddiff --git a/lib/index.js b/lib/index.js
iindex edc64b9..568bcae 100644
---- a/lib/index.js
++++ b/lib/index.js
@@@ -23,6 +23,7 @@ module.exports = {
      validate,
      envNames,
      plan: ops.plan,
++    reconcile: (ctx, opts) => require('./reconcile').reconcile(ctx, module.exports, opts),
      deploy: ops.deploy,
      rollback: ops.rollback,
      status,
ddiff --git a/lib/inventory.js b/lib/inventory.js
iindex 11bf392..cd6974e 100644
---- a/lib/inventory.js
++++ b/lib/inventory.js
@@@ -170,6 +170,12 @@ function normaliseService(id, raw, defaults) {
          // A release-layout entry without strategy "release-layout" is deployed by its own script.
          layout: releaseLayout ? 'release' : 'git',
          managed: strategy === null ? false : s.managed !== false,
++        // The deploy controller (ovhost reconcile, WS-X11): auto deploys main at green CI; gated and manual wait for a person.
++        deploy: (() => {
++            const d = s.deploy || {};
++            if (d.policy != null && !['auto', 'gated', 'manual'].includes(d.policy)) throw new InventoryError(`${where}.deploy.policy must be auto, gated or manual`);
++            return { policy: d.policy || 'manual' };
++        })(),
          unmanagedReason: s.unmanagedReason || (strategy === null ? 'release layout without strategy "release-layout": deployed by its own deploy script' : null),
          skipPackages: s.skipPackages || [],
          removeUntrackedLockfiles: s.removeUntrackedLockfiles === true,
ddiff --git a/lib/reconcile.js b/lib/reconcile.js
nnew file mode 100644
iindex 0000000..f14551e
---- /dev/null
++++ b/lib/reconcile.js
@@@ -0,0 +1,108 @@
++'use strict';
++/**
++ * `ovhost reconcile` — the deploy controller, phase 1 (roadmap WS-X11, D72): main at its last green CI commit
++ * is the desired state; this makes each `auto` service run it, without a person.
++ *
++ * For every service whose inventory entry says `"deploy": { "policy": "auto" }`:
++ *   1. plan with a fetch (lib/release-ops plan): what main has that the host does not;
++ *   2. up to date → nothing; only documentation and tests changed → nothing (the change rides the next deploy);
++ *   3. CI: every check run on that exact commit completed and passed (GitHub API; a commit with no check runs,
++ *      or one still running, waits for the next pass — never deployed on hope);
++ *   4. deploy it (the same lib.deploy as `ovhost deploy --wait-idle`: freezes, protected sessions, readiness and
++ *      automatic rollback all apply), then announce the release.
++ * `gated` and `manual` services are reported, never deployed here. One service at a time, in inventory order.
++ * The outcome of every pass goes to <stateDir>/reconcile.json and the node-exporter textfile collector
++ * (openvibe_reconcile_*), which the OpenVibeReconcileFailed alert reads.
++ */
++const path = require('path');
++const { git } = require('./git');
++
++// Files that document or test a service and are not part of what it runs (the roadmap baseline's rule).
++const DOCS_OR_TESTS = /^(README\.md|STATUS\.json|CHANGELOG\.md|SECURITY\.md|docs\/.*\.md|tests?\/|\.github\/)|\.test\.[cm]?[jt]s$/;
++const TEXTFILE_DIR = '/var/lib/prometheus/node-exporter';
++
++/** owner/repo from a GitHub remote URL (https or ssh), or null. */
++function githubSlug(url) {
++    const m = /github\.com[:/]([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+?)(?:\.git)?\/?$/.exec(String(url || ''));
++    return m ? `${m[1]}/${m[2]}` : null;
++}
++
++/** The CI verdict for one commit: 'green' | 'pending' | 'red' | 'none' | 'unknown'. */
++async function ciStatus(exec, slug, sha, { token = null } = {}) {
++    const headers = { accept: 'application/vnd.github+json', 'user-agent': 'openvibe-host-reconcile', 'x-github-api-version': '2022-11-28' };
++    if (token) headers.authorization = `Bearer ${token}`;
++    const r = await exec.request(`https://api.github.com/repos/${slug}/commits/${sha}/check-runs?per_page=100`, { method: 'GET', headers, timeoutMs: 15000 });
++    if (r.status !== 200) return { verdict: 'unknown', detail: `GitHub answered ${r.status || r.error}` };
++    let body;
++    try { body = JSON.parse(r.body); } catch { return { verdict: 'unknown', detail: 'unreadable GitHub answer' }; }
++    const runs = body.check_runs || [];
++    if (!runs.length) return { verdict: 'none', detail: 'no check runs on this commit' };
++    if (runs.some((c) => c.status !== 'completed')) return { verdict: 'pending', detail: `${runs.filter((c) => c.status !== 'completed').length} check(s) still running` };
++    const bad = runs.filter((c) => !['success', 'skipped', 'neutral'].includes(c.conclusion));
++    if (bad.length) return { verdict: 'red', detail: bad.map((c) => `${c.name}: ${c.conclusion}`).join(', ').slice(0, 300) };
++    return { verdict: 'green', detail: `${runs.length} check(s) passed` };
++}
++
++/**
++ * One pass. ctx is the ovhost context ({ exec, inv, log, … }); lib is lib/index.js (plan, deploy, announce).
++ * opts: { only: [ids], dryRun, token, stateDir, textfileDir, env }
++ * → { at, services: [{ id, policy, action, from, to, detail, result? }] }
++ */
++async function reconcile(ctx, lib, { only = null, dryRun = false, token = null, stateDir = '/var/lib/openvibe-host', textfileDir = TEXTFILE_DIR, env = process.env } = {}) {
++    const { exec, inv } = ctx;
++    const log = ctx.log || (() => {});
++    const pass = { at: new Date(exec.now()).toISOString(), dryRun, services: [] };
++    for (const [id, svc] of Object.entries(inv.services)) {
++        if (only && !only.includes(id)) continue;
++        const policy = (svc.deploy && svc.deploy.policy) || 'manual';
++        const entry = { id, policy };
++        pass.services.push(entry);
++        if (!svc.managed || !svc.repo) { entry.action = 'skip'; entry.detail = 'not managed by ovhost'; continue; }
++        let p;
++        try { p = await lib.plan(ctx, id, { fetch: true }); } catch (err) { entry.action = 'error'; entry.detail = `plan failed: ${err.message}`; continue; }
++        Object.assign(entry, { from: p.from, to: p.to });
++        if (p.upToDate) { entry.action = 'current'; continue; }
++        const runtime = (p.changed || []).filter((f) => !DOCS_OR_TESTS.test(f));
++        if (!runtime.length) { entry.action = 'docs-only'; entry.detail = `${(p.changed || []).length} file(s), documentation and tests only`; continue; }
++        if (policy !== 'auto') { entry.action = 'waiting'; entry.detail = `policy ${policy}: ${runtime.length} runtime file(s) wait for a person`; continue; }
++        const slug = githubSlug(await git(exec, svc).remoteUrl().catch(() => ''));
++        if (!slug) { entry.action = 'error'; entry.detail = 'the remote is not a GitHub repository: no CI to read'; continue; }
++        const ci = await ciStatus(exec, slug, p.to, { token });
++        entry.ci = ci.verdict;
++        if (ci.verdict !== 'green') { entry.action = ci.verdict === 'red' ? 'blocked' : 'waiting'; entry.detail = `CI ${ci.verdict}: ${ci.detail}`; continue; }
++        if (dryRun) { entry.action = 'would-deploy'; entry.detail = `${runtime.length} runtime file(s), CI green`; continue; }
++        log(`[reconcile] ${id}: deploying ${p.from.slice(0, 12)} → ${p.to.slice(0, 12)} (${runtime.length} runtime file(s), CI green)`);
++        let r;
++        try { r = await lib.deploy(ctx, id, { waitIdle: true }); } catch (err) { entry.action = 'error'; entry.detail = `deploy threw: ${err.message}`; continue; }
++        entry.action = 'deployed';
++        entry.result = r.record && r.record.result;
++        entry.exitCode = r.exitCode;
++        if (r.exitCode !== 0) { entry.action = 'failed'; entry.detail = `deploy exit ${r.exitCode} (${entry.result})`; continue; }
++        if (['deployed'].includes(entry.result) && lib.announce) {
++            try { const a = await lib.announce(ctx, id, { head: r.record.to, env }); entry.announced = !!(a && a.published); } catch { entry.announced = false; }
++        }
++    }
++    if (!dryRun) await record(exec, pass, { stateDir, textfileDir, log });
++    return pass;
++}
++
++async function record(exec, pass, { stateDir, textfileDir, log }) {
++    const count = (a) => pass.services.filter((s) => s.action === a).length;
++    const lines = [
++        '# HELP openvibe_reconcile_last_run_timestamp_seconds When ovhost reconcile last ran.',
++        '# TYPE openvibe_reconcile_last_run_timestamp_seconds gauge',
++        `openvibe_reconcile_last_run_timestamp_seconds ${Math.floor(Date.parse(pass.at) / 1000)}`,
++        '# HELP openvibe_reconcile_services Services by the last pass\'s action.',
++        '# TYPE openvibe_reconcile_services gauge',
++        ...['current', 'docs-only', 'waiting', 'blocked', 'deployed', 'failed', 'error', 'skip'].map((a) => `openvibe_reconcile_services{action="${a}"} ${count(a)}`),
++        '# HELP openvibe_reconcile_failed_total_last Deploys the last pass started that failed or rolled back.',
++        '# TYPE openvibe_reconcile_failed_total_last gauge',
++        `openvibe_reconcile_failed_total_last ${count('failed') + count('error')}`,
++    ];
++    try {
++        await exec.writeFile(path.join(textfileDir, 'openvibe_reconcile.prom'), `${lines.join('\n')}\n`, { mode: 0o644 });
++        await exec.writeFile(path.join(stateDir, 'reconcile.json'), `${JSON.stringify(pass, null, 1)}\n`, { mode: 0o644 });
++    } catch (err) { log(`[reconcile] could not record the pass: ${err.message}`); }
++}
++
++module.exports = { reconcile, ciStatus, githubSlug, DOCS_OR_TESTS };
ddiff --git a/test/reconcile.test.js b/test/reconcile.test.js
nnew file mode 100644
iindex 0000000..4775d79
---- /dev/null
++++ b/test/reconcile.test.js
@@@ -0,0 +1,74 @@
++'use strict';
++// ovhost reconcile (WS-X11 phase 1, D72): an `auto` service follows main at green CI without a person; a
++// documentation-only change deploys nothing; missing, pending or failed CI never deploys; `manual` waits; a dry run
++// changes nothing; every pass is recorded for the alert.
++const assert = require('assert');
++const { scenario, push, test, runTests } = require('./helpers');
++const lib = require('../lib');
++const { githubSlug, ciStatus } = require('../lib/reconcile');
++
++function setup(policy = 'auto') {
++    const host = scenario();
++    host.inv.services.live.deploy = { policy };
++    const ctx = { exec: host.exec, inv: host.inv, log: () => {} };
++    const ci = new Map();   // sha -> check runs
++    const hook = (sha) => `https://api.github.com/repos/OpenVibers/openvibe.live/commits/${sha}/check-runs?per_page=100`;
++    const setCi = (sha, runs) => { ci.set(sha, runs); host.http.set(hook(sha), () => ({ status: 200, body: { total_count: runs.length, check_runs: runs } })); };
++    return { host, ctx, setCi };
++}
++const only = (pass, id = 'live') => pass.services.find((s) => s.id === id);
++
++runTests([
++    test('githubSlug reads https and ssh remotes', () => {
++        assert.strictEqual(githubSlug('https://github.com/OpenVibers/OpenVibe.Wiki.git'), 'OpenVibers/OpenVibe.Wiki');
++        assert.strictEqual(githubSlug('[email protected]:OpenVibers/OpenVibe.Live.git'), 'OpenVibers/OpenVibe.Live');
++        assert.strictEqual(githubSlug('https://gitlab.com/x/y.git'), null);
++    }),
++    test('up to date: nothing; documentation and tests only: nothing deployed', async () => {
++        const { host, ctx } = setup();
++        let pass = await lib.reconcile(ctx, { only: ['live'], textfileDir: '/var/lib/prometheus/node-exporter' });
++        assert.strictEqual(only(pass).action, 'current');
++        push(host, 'live', { 'README.md': '# new docs', 'test/a.test.js': 'x' });
++        pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.strictEqual(only(pass).action, 'docs-only');
++        assert.ok(!host.calls.some((c) => /systemctl/.test(c.cmd) && (c.args || []).includes('restart')), 'nothing restarted');
++    }),
++    test('a runtime change waits for CI, is blocked by red CI, and deploys on green', async () => {
++        const { host, ctx, setCi } = setup();
++        const sha = push(host, 'live', { 'server/index.js': 'console.log(2);' });
++        let pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.strictEqual(only(pass).action, 'waiting', 'GitHub has no answer: never deployed on hope');
++        setCi(sha, []);
++        pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.deepStrictEqual([only(pass).action, only(pass).ci], ['waiting', 'none']);
++        setCi(sha, [{ name: 'test', status: 'in_progress', conclusion: null }]);
++        assert.strictEqual(only(await lib.reconcile(ctx, { only: ['live'] })).ci, 'pending');
++        setCi(sha, [{ name: 'test', status: 'completed', conclusion: 'failure' }, { name: 'security', status: 'completed', conclusion: 'success' }]);
++        pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.deepStrictEqual([only(pass).action, only(pass).ci], ['blocked', 'red']);
++        assert.match(only(pass).detail, /test: failure/);
++        setCi(sha, [{ name: 'test', status: 'completed', conclusion: 'success' }, { name: 'lint', status: 'completed', conclusion: 'skipped' }]);
++        const dry = await lib.reconcile(ctx, { only: ['live'], dryRun: true });
++        assert.strictEqual(only(dry).action, 'would-deploy');
++        pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.strictEqual(only(pass).action, 'deployed', JSON.stringify(only(pass)));
++        assert.strictEqual(only(pass).result, 'deployed');
++        assert.strictEqual(only(await lib.reconcile(ctx, { only: ['live'] })).action, 'current', 'the next pass finds it current');
++        const prom = host.read('/var/lib/prometheus/node-exporter/openvibe_reconcile.prom');
++        assert.match(prom, /openvibe_reconcile_services\{action="current"\} 1/);
++        assert.match(prom, /openvibe_reconcile_failed_total_last 0/);
++        assert.ok(JSON.parse(host.read('/var/lib/openvibe-host/reconcile.json')).services.length >= 1);
++    }),
++    test('a manual (or gated) service waits for a person, whatever CI says', async () => {
++        const { host, ctx, setCi } = setup('manual');
++        const sha = push(host, 'live', { 'server/index.js': 'console.log(3);' });
++        setCi(sha, [{ name: 'test', status: 'completed', conclusion: 'success' }]);
++        const pass = await lib.reconcile(ctx, { only: ['live'] });
++        assert.deepStrictEqual([only(pass).policy, only(pass).action], ['manual', 'waiting']);
++        assert.throws(() => lib.inventory.normalise({ services: { x: { repo: '/opt/x', deploy: { policy: 'yolo' } } } }), /auto, gated or manual/);
++    }),
++    test('ciStatus: GitHub unreachable is unknown, never green', async () => {
++        const exec = { request: async () => ({ status: 0, error: 'ECONNREFUSED' }) };
++        assert.strictEqual((await ciStatus(exec, 'OpenVibers/x', 'abc')).verdict, 'unknown');
++    }),
++]);