openvibers/openvibe.host Commits

Sign-in canary: every public site's /auth/health every 15 minutes (#206)

OP OpenVibers committed 14 days ago · 51d3711bc8 · 9 files, +446 −4

* Sign-in canary: every public site's /auth/health every 15 minutes

On 2026-10-29 openvibe.cam's OAuth client secret was the .env.example placeholder for two
days and nobody could sign in; nothing noticed. Sites on openvibe-sdk 0.73.0+ answer GET
/auth/health (200 the Network accepts the client, 503 it refuses it, 404 not on the SDK's
sign-in yet).

deploy/scripts/auth-health.sh reads the public origins from Network's registry (the list
scripts/browser-check.js uses), probes each with a 10 s timeout and writes node_exporter's
textfile collector in one rename: openvibe_auth_health{site="<id>"} 1 for 200, 0 for 503
or no answer, no line for 404. A registry it cannot read keeps the previous file.

openvibe-authhealth.timer runs it every 15 minutes; OpenVibeSignInBroken fires on
openvibe_auth_health == 0 for 30 minutes at severity ticket, through the usual relay.

* Sign-in canary: a 401 is a site's own gate, not a broken client

The first run over the fleet found openvibe.ai: its staff-only operator console answers
/auth/health itself with 401 and an HTML sign-in page, so the SDK's handler never sees the
request. Judged as 0 it would have opened a permanent ticket. 401 now writes no line, like
404, and the doc says why.

Changes

ddiff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
iindex 871bc46..8263689 100644
---- a/.github/workflows/ci.yml
++++ b/.github/workflows/ci.yml
@@@ -15,7 +15,7 @@ jobs:
        test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'
        contracts-src: lib server
        docs-status: true
--      extra: node --check bin/ovhost && for f in scripts/*.sh; do bash -n "$f"; done
++      extra: node --check bin/ovhost && for f in scripts/*.sh deploy/scripts/*.sh; do bash -n "$f"; done
    # Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
    security:
      uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@aa570e4815063dc65624a6a486038d83ff6f6ec0
ddiff --git a/README.md b/README.md
iindex 3db024a..7bfb73f 100644
---- a/README.md
++++ b/README.md
@@@ -293,7 +293,7 @@ Two things deploy from this repository:
    `cd /usr/local/lib/openvibe-host && sudo git pull --ff-only && sudo npm ci --omit=dev --no-audit --no-fund`,
    then `ovhost --version` and `sudo ovhost capabilities` (ovhost does not deploy itself; an update is
    undone by checking out the previous commit there). Its timers (`openvibe-backup`, `openvibe-alerts`, `openvibe-browsercheck`,
--  `openvibe-devpath`, `openvibe-toolsjob`) are unit files in [deploy/systemd/](deploy/systemd/).
++  `openvibe-authhealth`, `openvibe-devpath`, `openvibe-toolsjob`) are unit files in [deploy/systemd/](deploy/systemd/).
  - **Boot order.** Three drop-ins in [deploy/systemd/](deploy/systemd/) order every `openvibe-*` and `openre-*`
    service after PostgreSQL, PgBouncer and Valkey (systemd prefix drop-ins, `openvibe-.service.d/`), and PgBouncer
    after PostgreSQL. Without them a reboot starts services against a PgBouncer that caches PostgreSQL's refused
ddiff --git a/deploy/prometheus/openvibe-rules.yml b/deploy/prometheus/openvibe-rules.yml
iindex 3a26f4f..7536bf8 100644
---- a/deploy/prometheus/openvibe-rules.yml
++++ b/deploy/prometheus/openvibe-rules.yml
@@@ -95,6 +95,16 @@ groups:
          labels: { severity: ticket }
          annotations:
            summary: "{{ $labels.service }}: no browser check has run to the end for more than 26 hours"
++      # The sign-in canary (deploy/scripts/auth-health.sh, openvibe-authhealth.timer; docs/auth-health.md):
++      # a public site answers GET /auth/health 200 while the Network accepts its OAuth client and 503 when
++      # it refuses it. A site not on the SDK's sign-in yet writes no line, so it never fires this. A ticket,
++      # not a page: sign-in is broken, the site still serves.
++      - alert: OpenVibeSignInBroken
++        expr: openvibe_auth_health == 0
++        for: 30m
++        labels: { severity: ticket }
++        annotations:
++          summary: "Sign-in is broken on {{ $labels.site }}: the Network refuses its OAuth client (check OV_OAUTH_CLIENT_SECRET in its env)"
        # The relay itself (openvibe-alerts.timer; WS-H task 11). It cannot page while it is broken; it pages
        # this as soon as a run gets through again, so an outage of the pager is never silent afterwards.
        - alert: OpenVibeAlertRelayFailing
ddiff --git a/deploy/scripts/auth-health.sh b/deploy/scripts/auth-health.sh
nnew file mode 100755
iindex 0000000..948100e
---- /dev/null
++++ b/deploy/scripts/auth-health.sh
@@@ -0,0 +1,149 @@
++#!/usr/bin/env bash
++# Sign-in canary for every OpenVibe site (docs/auth-health.md, openvibe-authhealth.timer).
++#
++#   deploy/scripts/auth-health.sh                  every public origin in Network's registry
++#   deploy/scripts/auth-health.sh --sites live,cam  only these (a post-deploy run)
++#   AUTH_HEALTH_REGISTRY=<url|file>                where the list comes from (default Network's registry)
++#   TEXTFILE_DIR=<dir>                             node_exporter's textfile collector (tests)
++#
++# For each site: GET <origin>/auth/health with a 10 s timeout (openvibe-sdk ≥ 0.73.0, src/sso/index.js
++# handlers.health). One textfile line per site, written to <TEXTFILE_DIR>/openvibe_auth_health.prom:
++#
++#   openvibe_auth_health{site="<id>"} 1   200: the Network accepts the site's OAuth client
++#   openvibe_auth_health{site="<id>"} 0   503 (the Network refuses it), any other answer (a 500, a 502,
++#                                         a redirect) or no answer at all
++#   (no line)                            404: the site is not on the SDK's sign-in yet; 401: its own
++#                                         sign-in gate answered (openvibe.ai's console is staff-only).
++#                                         Neither is the SDK's health handler, so neither is judged.
++#
++# The file is written atomically (a temp file beside it, then rename), so Prometheus never reads half of
++# it. Exit: 0 when every probed site is 1 or 404 · 1 when one is 0 · 2 when the registry could not be
++# read or held no public origin (the previous file is kept: an unreadable list is not a green canary).
++set -uo pipefail
++
++REGISTRY=${AUTH_HEALTH_REGISTRY:-https://openvibe.network/api/v1/registry/services}
++DIR=${TEXTFILE_DIR:-/var/lib/prometheus/node-exporter}
++OUT=$DIR/openvibe_auth_health.prom
++SITES=
++TRIES=${AUTH_HEALTH_REGISTRY_TRIES:-3}
++WAIT=${AUTH_HEALTH_REGISTRY_WAIT:-10}
++
++usage() { sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; }
++
++while [ $# -gt 0 ]; do
++    case "$1" in
++        --sites) SITES=${2:-}; shift 2 ;;
++        --sites=*) SITES=${1#*=}; shift ;;
++        --registry) REGISTRY=${2:-}; shift 2 ;;
++        -h|--help) usage; exit 0 ;;
++        *) echo "auth-health: unknown argument $1" >&2; exit 2 ;;
++    esac
++done
++
++command -v curl >/dev/null || { echo "auth-health: curl is needed" >&2; exit 2; }
++command -v node >/dev/null || { echo "auth-health: node is needed" >&2; exit 2; }
++
++TMP=$(mktemp -d) || exit 2
++trap 'rm -rf "$TMP"' EXIT
++
++# The list of sites: Network's registry, asked again a couple of times because a release of Network
++# restarts it (scripts/browser-check.js does the same). A file is read as is.
++if [[ $REGISTRY =~ ^https?:// ]]; then
++    attempt=1
++    while :; do
++        if curl -fsS --max-time 20 -H 'accept: application/json' -o "$TMP/registry.json" "$REGISTRY" 2>/dev/null; then break; fi
++        if [ "$attempt" -ge "$TRIES" ]; then
++            echo "auth-health: the registry did not answer after $TRIES tries ($REGISTRY); keeping $OUT" >&2
++            exit 2
++        fi
++        echo "auth-health: the registry did not answer (Network restarting?); asking again in ${WAIT}s" >&2
++        sleep "$WAIT"
++        attempt=$((attempt + 1))
++    done
++elif ! cp -- "$REGISTRY" "$TMP/registry.json" 2>/dev/null; then
++    echo "auth-health: cannot read the registry file $REGISTRY; keeping $OUT" >&2
++    exit 2
++fi
++
++# id<TAB>origin, one line per service with a public origin. Both shapes Network has served are read: an
++# array, or { "services": [...] }, with publicOrigin or public_origin. A trailing slash goes, and an id
++# that is not a service id is skipped (it must be a Prometheus label value).
++if ! node - "$TMP/registry.json" > "$TMP/sites" <<'JS'
++const fs = require('fs');
++let body;
++try { body = JSON.parse(fs.readFileSync(process.argv[2], 'utf8')); } catch { process.stderr.write('auth-health: the registry is not JSON\n'); process.exit(1); }
++const list = Array.isArray(body) ? body : (body && Array.isArray(body.services) ? body.services : []);
++const seen = new Set();
++for (const s of list) {
++    if (!s || typeof s !== 'object') continue;
++    const id = String(s.id || s.name || '').trim();
++    const origin = String(s.publicOrigin || s.public_origin || '').trim().replace(/\/+$/, '');
++    if (!/^[a-z][a-z0-9-]{1,39}$/.test(id)) continue;
++    if (!/^https?:\/\/[^/?#]+$/.test(origin)) continue;
++    if (seen.has(id)) continue;
++    seen.add(id);
++    process.stdout.write(`${id}\t${origin}\n`);
++}
++JS
++then
++    echo "auth-health: the registry is not JSON; keeping $OUT" >&2
++    exit 2
++fi
++
++if [ -n "$SITES" ]; then
++    : > "$TMP/wanted"
++    for s in ${SITES//,/ }; do echo "$s" >> "$TMP/wanted"; done
++    awk -F'\t' 'NR==FNR { want[$1]=1; next } ($1 in want)' "$TMP/wanted" "$TMP/sites" > "$TMP/sites.wanted"
++    mv "$TMP/sites.wanted" "$TMP/sites"
++fi
++
++if [ ! -s "$TMP/sites" ]; then
++    echo "auth-health: the registry named no site with a public origin; keeping $OUT" >&2
++    exit 2
++fi
++
++LINES=()
++FAILED=0
++PROBED=0
++while IFS=$'\t' read -r id origin; do
++    [ -n "$id" ] || continue
++    code=$(curl -sS -o /dev/null --max-time 10 -w '%{http_code}' "$origin/auth/health" 2>/dev/null) || code=000
++    case "$code" in
++        200) value=1 ;;
++        401|404) value= ;;   # not the SDK's health handler: a staff gate, or a site not on the sign-in yet
++        *)   value=0 ;;
++    esac
++    [ -n "${value-}" ] || continue
++    PROBED=$((PROBED + 1))
++    [ "$value" = 1 ] || FAILED=$((FAILED + 1))
++    LINES+=("openvibe_auth_health{site=\"$id\"} $value")
++done < "$TMP/sites"
++
++if [ "$PROBED" -eq 0 ]; then
++    echo "auth-health: no site answered 200, 503 or an error (all 404: none on the SDK's sign-in yet); keeping $OUT" >&2
++    exit 0
++fi
++
++if [ -d "$DIR" ]; then
++    TMPFILE=$OUT.tmp.$$
++    if ! {
++        echo '# HELP openvibe_auth_health 1 when GET /auth/health answered 200 (the Network accepts the site OAuth client), 0 when it answered 503 or did not answer.'
++        echo '# TYPE openvibe_auth_health gauge'
++        printf '%s\n' "${LINES[@]}"
++    } > "$TMPFILE"; then
++        rm -f "$TMPFILE"
++        echo "auth-health: could not write $TMPFILE" >&2
++        exit 2
++    fi
++    chmod 0644 "$TMPFILE" || { rm -f "$TMPFILE"; echo "auth-health: could not chmod $TMPFILE" >&2; exit 2; }
++    if ! mv -f "$TMPFILE" "$OUT"; then
++        rm -f "$TMPFILE"
++        echo "auth-health: could not rename $TMPFILE to $OUT" >&2
++        exit 2
++    fi
++fi
++
++printf '%s\n' "${LINES[@]}"
++
++[ "$FAILED" -eq 0 ] || exit 1
++exit 0
ddiff --git a/deploy/systemd/openvibe-authhealth.service b/deploy/systemd/openvibe-authhealth.service
nnew file mode 100644
iindex 0000000..52a180d
---- /dev/null
++++ b/deploy/systemd/openvibe-authhealth.service
@@@ -0,0 +1,17 @@
++[Unit]
++Description=OpenVibe sign-in canary: GET /auth/health on every public site (docs/auth-health.md)
++Documentation=https://github.com/OpenVibers/OpenVibe.Host/blob/main/docs/auth-health.md
++After=network-online.target
++Wants=network-online.target
++
++[Service]
++Type=oneshot
++# Root writes node_exporter's textfile collector; nothing else is touched.
++User=root
++ExecStart=/usr/local/lib/openvibe-host/deploy/scripts/auth-health.sh
++Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++Environment=OVHOST_INVENTORY=/etc/openvibe/host.json
++Nice=10
++TimeoutStartSec=10min
++PrivateTmp=true
++ProtectSystem=full
ddiff --git a/deploy/systemd/openvibe-authhealth.timer b/deploy/systemd/openvibe-authhealth.timer
nnew file mode 100644
iindex 0000000..cea1488
---- /dev/null
++++ b/deploy/systemd/openvibe-authhealth.timer
@@@ -0,0 +1,10 @@
++[Unit]
++Description=Check that every OpenVibe site's sign-in still works, every 15 minutes (docs/auth-health.md)
++
++[Timer]
++OnBootSec=5min
++OnUnitActiveSec=15min
++AccuracySec=30s
++
++[Install]
++WantedBy=timers.target
ddiff --git a/docs/auth-health.md b/docs/auth-health.md
nnew file mode 100644
iindex 0000000..b8acf48
---- /dev/null
++++ b/docs/auth-health.md
@@@ -0,0 +1,71 @@
++# Sign-in canary
++
++A canary for every OpenVibe site's sign-in, on the production host. On 2026-10-29 openvibe.cam's OAuth
++client secret was the `.env.example` placeholder for two days: nobody could sign in, and nothing noticed.
++This notices.
++
++Sites on openvibe-sdk 0.73.0 or later answer `GET /auth/health` (`src/sso/index.js`, `handlers.health`):
++they make one token request with a code no sign-in ever has, and read which way the Network refuses it.
++`invalid_grant` means the client is fine, `invalid_client` that its id or secret is wrong. The answer is
++cached for ten minutes in the site's process.
++
++| Answer | Means | Metric |
++|---|---|---|
++| 200 | the Network accepts the site's OAuth client | `openvibe_auth_health{site="<id>"} 1` |
++| 503 | the Network refuses it (a wrong `OV_OAUTH_CLIENT_SECRET` or `OV_OAUTH_CLIENT_ID`) | `openvibe_auth_health{site="<id>"} 0` |
++| any other status (a 500, a 502, a redirect), or no answer at all | a broken site or a broken path, not a healthy one | `openvibe_auth_health{site="<id>"} 0` |
++| 404 | the site is not on the SDK's sign-in yet: nothing to judge | no line |
++| 401 | the site's own sign-in gate answered: openvibe.ai's operator console is staff-only, so `/auth/health` is never the SDK's handler there | no line |
++
++## The run
++
++`deploy/scripts/auth-health.sh` (openvibe-authhealth.timer, every 15 minutes) asks OpenVibe.Network's
++registry (`/api/v1/registry/services`, the same list `scripts/browser-check.js` uses) for every service
++with a public origin, and gets `<origin>/auth/health` with a 10 s timeout, one site at a time. It writes
++node_exporter's textfile collector in one rename (`/var/lib/prometheus/node-exporter/openvibe_auth_health.prom`),
++so Prometheus never reads half a file, and prints the lines it wrote.
++
++- **The registry does not answer**: it asks up to three times, ten seconds apart (a Network release
++  restarts it), then exits 2 and leaves the previous file alone. An unreadable list is not a green canary.
++- **No collector directory**: it writes nothing and exits 0, so it is harmless off the host.
++- **Exit code**: 0 every probed site is 1 or 404 · 1 one site is 0 · 2 the list could not be read.
++- **Secrets**: it reads no env file and prints nothing from one. The metric is a site id and 0 or 1.
++
++After a deploy, or when checking one site now:
++
++```bash
++sudo systemctl start openvibe-authhealth.service        # every site, now
++sudo ovhost deploy <service>                            # a deploy is followed by the timer within 15 minutes
++deploy/scripts/auth-health.sh --sites live,cam          # these sites, printing the lines
++```
++
++## The alert
++
++`OpenVibeSignInBroken` in [deploy/prometheus/openvibe-rules.yml](../deploy/prometheus/openvibe-rules.yml)
++fires on `openvibe_auth_health == 0` for 30 minutes, severity `ticket`: the owner's bell, not a push,
++because the site still serves; only its sign-in is broken. It travels the usual path
++([operator-alerts.md](operator-alerts.md)):
++
++> Sign-in is broken on cam: the Network refuses its OAuth client (check OV_OAUTH_CLIENT_SECRET in its env)
++
++The fix is the site's env file, never a change made by this check.
++
++## Install (once)
++
++```bash
++sudo cp /usr/local/lib/openvibe-host/deploy/systemd/openvibe-authhealth.{service,timer} /etc/systemd/system/
++sudo systemctl daemon-reload && sudo systemctl enable --now openvibe-authhealth.timer
++sudo cp /usr/local/lib/openvibe-host/deploy/prometheus/openvibe-rules.yml /etc/prometheus/rules/openvibe.yml
++sudo promtool check rules /etc/prometheus/rules/openvibe.yml && sudo systemctl reload prometheus
++```
++
++The script lives in the `ovhost` CLI's own checkout (`/usr/local/lib/openvibe-host`), which
++`sudo ovhost self-update` brings up to `main`; the unit runs it from there.
++
++## What it cannot see
++
++- A site on openvibe-sdk before 0.73.0 answers 404, and is silently outside the canary until its pin moves.
++  So is a site whose own sign-in gate answers first, as openvibe.ai's staff console does (401).
++- `/auth/health` tells a broken client secret from a working one. It does not finish a sign-in, so a
++  broken callback URL, a Network that is down, or a site whose sessions cannot be verified still look fine.
++- The check is one request per site per 15 minutes, from the host. It never signs anybody in.
ddiff --git a/docs/operator-alerts.md b/docs/operator-alerts.md
iindex 1cae4f1..07ce4ec 100644
---- a/docs/operator-alerts.md
++++ b/docs/operator-alerts.md
@@@ -3,7 +3,8 @@
  Roadmap WS-H task 11. The host's Prometheus evaluates the rules in
  [deploy/prometheus/openvibe-rules.yml](../deploy/prometheus/openvibe-rules.yml): targets down, error
  rates, disk and memory, backups, deploy drift, the developer path, the Tools job proof, the browser check
--of each release and the relay itself. Until 2026-09-26 nothing delivered them.
++of each release, the sign-in canary of every site ([auth-health.md](auth-health.md)) and the relay itself.
++Until 2026-09-26 nothing delivered them.
  
  ## The path
  
@@@ -76,7 +77,8 @@ systemctl list-timers openvibe-alerts.timer
  ```bash
  sudo cp /usr/local/lib/openvibe-host/deploy/systemd/openvibe-alerts.{service,timer} /etc/systemd/system/
  sudo cp /usr/local/lib/openvibe-host/deploy/systemd/openvibe-browsercheck.{service,timer} /etc/systemd/system/
--sudo systemctl daemon-reload && sudo systemctl enable --now openvibe-alerts.timer openvibe-browsercheck.timer
++sudo cp /usr/local/lib/openvibe-host/deploy/systemd/openvibe-authhealth.{service,timer} /etc/systemd/system/
++sudo systemctl daemon-reload && sudo systemctl enable --now openvibe-alerts.timer openvibe-browsercheck.timer openvibe-authhealth.timer
  # Chrome and the check account (once):
  curl -fsSL https://dl.google.com/linux/linux_signing_key.pub | sudo gpg --dearmor -o /usr/share/keyrings/google-chrome.gpg
  echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome.gpg] https://dl.google.com/linux/chrome/deb/ stable main" | sudo tee /etc/apt/sources.list.d/google-chrome.list
ddiff --git a/test/auth-health.test.js b/test/auth-health.test.js
nnew file mode 100644
iindex 0000000..d3938a6
---- /dev/null
++++ b/test/auth-health.test.js
@@@ -0,0 +1,183 @@
++'use strict';
++/**
++ * The sign-in canary (deploy/scripts/auth-health.sh, docs/auth-health.md): every public origin in Network's
++ * registry gets one GET /auth/health — 1 for 200, 0 for 503, another answer or no answer at all, and no
++ * line for 404 (a site not on the SDK's sign-in yet) — and the textfile collector's file is replaced in
++ * one rename. A registry it cannot read leaves the previous file alone, so a broken list is never green.
++ * The script runs for real here; the sites and the registry are local HTTP servers.
++ */
++const assert = require('assert');
++const fs = require('fs');
++const http = require('http');
++const os = require('os');
++const path = require('path');
++const { execFile } = require('child_process');
++const { test, runTests } = require('./helpers');
++
++const SCRIPT = path.join(__dirname, '..', 'deploy', 'scripts', 'auth-health.sh');
++const PROM = 'openvibe_auth_health.prom';
++
++function site(status) {
++    const s = http.createServer((req, res) => {
++        s.hits.push(req.url);
++        res.statusCode = status;
++        res.setHeader('content-type', 'application/json');
++        res.end(JSON.stringify({ ok: status === 200, client: status === 200 ? 'ok' : 'invalid' }));
++    });
++    s.hits = [];
++    return new Promise((resolve) => s.listen(0, '127.0.0.1', () => resolve(s)));
++}
++
++const originOf = (s) => `http://127.0.0.1:${s.address().port}`;
++
++/** A port nothing listens on: the canary is asked to meet a site that does not answer. */
++function closedPort() {
++    return new Promise((resolve) => {
++        const s = http.createServer();
++        s.listen(0, '127.0.0.1', () => { const port = s.address().port; s.close(() => resolve(port)); });
++    });
++}
++
++function dir() {
++    return fs.mkdtempSync(path.join(os.tmpdir(), 'auth-health-'));
++}
++
++function registryFile(dirPath, body) {
++    const file = path.join(dirPath, 'registry.json');
++    fs.writeFileSync(file, typeof body === 'string' ? body : JSON.stringify(body));
++    return file;
++}
++
++// execFile, not spawnSync: the sites and the registry are servers in this process, and a blocked event
++// loop could not answer them (the canary would meet four timeouts instead).
++function run(textfileDir, registry, args = []) {
++    return new Promise((resolve) => {
++        execFile('bash', [SCRIPT, '--registry', registry, ...args], {
++            timeout: 120000,
++            env: { ...process.env, TEXTFILE_DIR: textfileDir, AUTH_HEALTH_REGISTRY_TRIES: '1', AUTH_HEALTH_REGISTRY_WAIT: '0' },
++        }, (err, stdout, stderr) => resolve({ status: err ? (err.code == null ? null : err.code) : 0, stdout, stderr }));
++    });
++}
++
++const prom = (textfileDir) => {
++    try { return fs.readFileSync(path.join(textfileDir, PROM), 'utf8'); } catch { return null; }
++};
++
++runTests([
++    test('every public origin is probed once; 200 is 1, 503 and no answer are 0, 404 is no line', async () => {
++        const ok = await site(200);
++        const refused = await site(503);
++        const missing = await site(404);
++        const broken = await site(500);
++        const gate = await site(401);
++        const dead = await closedPort();
++        const textfile = dir();
++        const registry = registryFile(textfile, {
++            services: [
++                { id: 'live', publicOrigin: originOf(ok) },
++                { id: 'cam', public_origin: `${originOf(refused)}/` },
++                { id: 'blog', publicOrigin: originOf(missing) },
++                { id: 'chat', publicOrigin: originOf(broken) },
++                { id: 'tools', publicOrigin: `http://127.0.0.1:${dead}` },
++                { id: 'ai', publicOrigin: originOf(gate) },
++                { id: 'contracts' },
++                { id: 'Bad Id', publicOrigin: 'https://openvibe.example' },
++                { id: 'noscheme', publicOrigin: 'openvibe.example' },
++                { id: 'live', publicOrigin: originOf(ok) },
++            ],
++        });
++        const r = await run(textfile, registry);
++        for (const s of [ok, refused, missing, broken, gate]) s.close();
++        assert.strictEqual(r.status, 1, r.stderr);
++        assert.deepStrictEqual(ok.hits, ['/auth/health'], 'exactly one probe per site');
++        const file = prom(textfile);
++        assert.match(file, /^# TYPE openvibe_auth_health gauge$/m);
++        assert.match(file, /^openvibe_auth_health\{site="live"\} 1$/m);
++        assert.match(file, /^openvibe_auth_health\{site="cam"\} 0$/m, '503: the Network refuses its client');
++        assert.ok(!/site="blog"/.test(file), '404: not on the SDK sign-in, no line');
++        assert.ok(!/site="ai"/.test(file), '401: the site\'s own sign-in gate answered, no line');
++        assert.match(file, /^openvibe_auth_health\{site="chat"\} 0$/m, 'another status is not a pass');
++        assert.match(file, /^openvibe_auth_health\{site="tools"\} 0$/m, 'no answer is 0');
++        assert.ok(!/site="Bad Id"/.test(file) && !/site="noscheme"/.test(file) && !/site="contracts"/.test(file));
++        assert.strictEqual(file.match(/site="live"/g).length, 1, 'a repeated id keeps one line');
++        assert.deepStrictEqual(r.stdout.trim().split('\n'), [
++            'openvibe_auth_health{site="live"} 1',
++            'openvibe_auth_health{site="cam"} 0',
++            'openvibe_auth_health{site="chat"} 0',
++            'openvibe_auth_health{site="tools"} 0',
++        ]);
++        assert.deepStrictEqual(fs.readdirSync(textfile).filter((f) => f !== 'registry.json'), [PROM], 'no temp file is left behind');
++    }),
++    test('a bare array is read too, --sites narrows a run, and an all-good list exits 0', async () => {
++        const ok = await site(200);
++        const other = await site(200);
++        const textfile = dir();
++        const registry = registryFile(textfile, [{ id: 'live', publicOrigin: originOf(ok) }, { id: 'blog', publicOrigin: originOf(other) }]);
++        let r = await run(textfile, registry);
++        assert.strictEqual(r.status, 0, r.stderr);
++        assert.match(prom(textfile), /^openvibe_auth_health\{site="live"\} 1$/m);
++        assert.match(prom(textfile), /^openvibe_auth_health\{site="blog"\} 1$/m);
++        r = await run(textfile, registry, ['--sites', 'live']);
++        assert.strictEqual(r.status, 0, r.stderr);
++        assert.deepStrictEqual([ok.hits.length, other.hits.length], [2, 1], 'the filtered run leaves the other site alone');
++        assert.ok(!/site="blog"/.test(prom(textfile)));
++        ok.close();
++        other.close();
++    }),
++    test('the registry is taken from a URL, and one that does not answer keeps the previous file', async () => {
++        const ok = await site(200);
++        const registryServer = await site(200);
++        const registryBody = JSON.stringify({ services: [{ id: 'live', publicOrigin: originOf(ok) }] });
++        registryServer.removeAllListeners('request');
++        registryServer.on('request', (req, res) => { res.statusCode = 200; res.end(registryBody); });
++        const textfile = dir();
++        let r = await run(textfile, originOf(registryServer));
++        assert.strictEqual(r.status, 0, r.stderr);
++        assert.match(prom(textfile), /^openvibe_auth_health\{site="live"\} 1$/m);
++        const before = prom(textfile);
++        r = await run(textfile, 'http://127.0.0.1:1/api/v1/registry/services');
++        assert.strictEqual(r.status, 2);
++        assert.match(r.stderr, /the registry did not answer/);
++        assert.strictEqual(prom(textfile), before, 'a registry that cannot be read is not a green canary');
++        const deadServer = http.createServer((req, res) => { res.statusCode = 500; res.end('no'); });
++        await new Promise((resolve) => deadServer.listen(0, '127.0.0.1', resolve));
++        r = await run(textfile, `http://127.0.0.1:${deadServer.address().port}/api/v1/registry/services`);
++        assert.strictEqual(r.status, 2);
++        assert.strictEqual(prom(textfile), before, 'a 500 from the registry changes nothing');
++        ok.close();
++        registryServer.close();
++        deadServer.close();
++    }),
++    test('a registry that is not JSON or names no site keeps the previous file', async () => {
++        const ok = await site(200);
++        const textfile = dir();
++        fs.writeFileSync(path.join(textfile, PROM), 'openvibe_auth_health{site="live"} 1\n');
++        let r = await run(textfile, path.join(textfile, 'absent.json'));
++        assert.strictEqual(r.status, 2);
++        assert.match(r.stderr, /cannot read the registry file/);
++        r = await run(textfile, registryFile(textfile, '<html>502</html>'));
++        assert.strictEqual(r.status, 2);
++        assert.match(r.stderr, /not JSON/);
++        r = await run(textfile, registryFile(textfile, { services: [{ id: 'live' }] }));
++        assert.strictEqual(r.status, 2);
++        assert.match(r.stderr, /no site with a public origin/);
++        assert.strictEqual(prom(textfile), 'openvibe_auth_health{site="live"} 1\n');
++        ok.close();
++    }),
++    test('the file is replaced in one rename; without the collector directory nothing is written', async () => {
++        const ok = await site(200);
++        const textfile = dir();
++        const registry = registryFile(textfile, [{ id: 'live', publicOrigin: originOf(ok) }]);
++        assert.strictEqual((await run(textfile, registry)).status, 0);
++        const first = fs.statSync(path.join(textfile, PROM)).ino;
++        assert.strictEqual((await run(textfile, registry)).status, 0);
++        const second = fs.statSync(path.join(textfile, PROM)).ino;
++        assert.notStrictEqual(second, first, 'a rename, not a rewrite in place');
++        assert.ok(!fs.readdirSync(textfile).some((f) => f.endsWith('.tmp.' + process.pid) || f.startsWith(PROM + '.tmp')), 'no temp file');
++        const absent = path.join(textfile, 'no-collector-here');
++        const r = await run(absent, registry);
++        assert.strictEqual(r.status, 0);
++        assert.ok(!fs.existsSync(path.join(absent, PROM)), 'nothing to write to, nothing written');
++        ok.close();
++    }),
++]);