openvibers/openvibe.bot Commits

OpenVibe.Bot: serves its own legal pages, sitemap, manifest and 404 (OpenVibe.Sites is being deleted) (#51)

OP OpenVibers committed 10 Oct 2026 · 2ceb58a715 · 6 files, +127 −21

OpenVibe.Sites is being deleted (plan T11). Bot's vhost served the legal pages, sitemap, manifest and 404 page from Sites' checkout; the app now serves them itself.

- **Legal pages:** `/terms`, `/privacy` and `/dmca` come from `openvibe-shared/legal`, with the front page's CSP so the shared theme and navbar load.
- **Discovery files:** `/sitemap.xml` and `/manifest.webmanifest` are served by the app. Unknown pages get an HTML 404.
- **vhost:** no static root any more. `location /` proxies to the app with Bot's API rate limit.
- **Tests:** the new `test/legal.test.js`; `test/home.test.js` asserts the vhost no longer mentions `/opt/openvibe.sites`.

Changes

ddiff --git a/README.md b/README.md
iindex 489783b..d62d0ba 100644
---- a/README.md
++++ b/README.md
@@@ -15,6 +15,7 @@ OpenVibe.Bot pairs a robot's machines, keeps their state and gates every operato
- The `/device` and `/control` WebSockets, the command gate and the latched e-stop.
- The front page, `GET /` (`server/web/home.js`): the OpenVibe Frame (`openvibe-shared/shell`) around `openvibe-shared/showcase` sections that say only what works today (pairing, the panel's controls, the gate and the e-stop, people and the queue, the read-only embed, the simulator and the kits with drivers), with its own CSP for the Frame's calls to the Network; the rest of the site keeps `default-src 'self'`. `/shared/` is openvibe-shared's serve handler.
- The signed-in pages (`server/web/routes.js`): `/robots` to list and add robots, `/pair/:id` for a pairing code and the installer command, and `/panel/:id`, the panel rendered from the profile. A `sim.rover` robot is driven by an in-process simulator (`server/sim`), so the panel works before any hardware exists. It gets no jobs, and it is not counted as a device or reported online in the outbox. The panel (`public/panel.js`, no build step) drives with a touch joystick, the keyboard or a gamepad, shows a latency meter and one camera tile per camera the profile lists; a tile plays the robot's OpenRestream WebRTC session when one is live (the mediasoup-client viewer flow OpenVibe.Live runs, against the session's playback descriptor), and stays the placeholder when there is none.
++- The legal pages (`/terms`, `/privacy`, `/dmca`, `openvibe-shared/legal`), `/sitemap.xml`, `/manifest.webmanifest`, `/robots.txt` and the browser-facing 404 (`server/app.js`, `server/web/routes.js`): Bot serves them all itself, and the nginx vhost has no static root.
- Server-side ONVIF cameras (`camera.onvif`, `mapping.driver: onvif`): an in-process connector beside the simulator (`server/onvif`) attaches a camera configured with `BOT_ONVIF_CAMERAS`, writes a real `kind: server` device row, translates a `ptz` command into ONVIF `ContinuousMove`/`Stop` (with the profile's deadline as a deadman) and carries the camera's `GetStatus` readout in telemetry. Its credentials are secret references only (`username_ref`/`password_ref`).
- The command audit (`command_audit`, kept 30 days) and the `bot.*` outbox events.
- Operators and viewers from the panel: the owner adds a person to a robot by `@username` (OpenVibe.Network resolves the name to a subject) and removes them again, each listed on the panel's People card. On an open-queue robot a signed-in visitor sees their turn — or their place in line — and can leave the queue, which promotes the next person at once (`POST /robots/:id/operators`, `POST /robots/:id/operators/:subject/remove`, `POST /robots/:id/queue/leave`, all plain forms that work without JavaScript).
@@@ -26,7 +27,6 @@ OpenVibe.Bot pairs a robot's machines, keeps their state and gates every operato
- Identity, subjects and tokens (OpenVibe.Network) and the robot's record on Network.
- Video: a device publishes over WHIP to OpenRestream; Bot only hands out the URL at pairing.
- The agent that runs on the robot; it ships with the T15 device-agent job.
--- The legal pages (`/terms`, `/privacy`, `/dmca`) and `sitemap.xml`: the frozen OpenVibe.Sites files.
## Devices
@@@ -115,9 +115,9 @@ Set `BOT_ONVIF_CAMERAS` (JSON, unset by default) to the server-side cameras Bot
## Deploy files
- `deploy/systemd/openvibe-bot.service` — systemd unit (`EnvironmentFile=/etc/openvibe/bot.env`, `PORT=4630`, hardening and `Restart=always`).
--- `deploy/nginx/openvibe.bot.conf` — nginx reference site for openvibe.bot: TLS, www→apex, rate limits, and WebSocket upgrade headers for the device (`/device`) and operator (`/control`) sockets, passing the `Authorization` header through. It proxies the front page (`location = /`) and `/shared/` to the app, and serves the frozen OpenVibe.Sites legal pages, `robots.txt`, `sitemap.xml` and 404 page from `/opt/openvibe.sites/dist/openvibe.bot`.
++- `deploy/nginx/openvibe.bot.conf` — nginx reference site for openvibe.bot: TLS, www→apex, rate limits, and WebSocket upgrade headers for the device (`/device`) and operator (`/control`) sockets, passing the `Authorization` header through. It proxies the front page (`location = /`), `/shared/`, `/robots.txt`, `/release.json` and everything else to the app, which serves the legal pages, `sitemap.xml`, `manifest.webmanifest` and the 404 page itself; there is no static root.
--OpenVibe.Sites keeps the frozen legal pages, `robots.txt` and `sitemap.xml` under `/opt/openvibe.sites/dist/openvibe.bot` (its front page there is no longer served), but no longer generates `deploy/nginx/openvibe.bot.conf` (sites.json marks the vhost as owned by OpenVibe.Bot). A Sites deploy leaves an installed vhost it no longer generates in place, so the current Sites vhost keeps serving openvibe.bot until Bot's first deploy installs this file over it; nothing has to be removed by hand, and later Sites deploys never overwrite Bot's `/device` and `/control` routes.
++OpenVibe.Sites no longer generates `deploy/nginx/openvibe.bot.conf` (sites.json marks the vhost as owned by OpenVibe.Bot), and Bot serves nothing from a Sites checkout: the legal pages, `robots.txt`, `sitemap.xml`, `manifest.webmanifest` and the 404 page all come from the app. A Sites deploy leaves an installed vhost it no longer generates in place, so the current Sites vhost keeps serving openvibe.bot until Bot's first deploy installs this file over it; later Sites deploys never overwrite Bot's `/device` and `/control` routes.
<!-- versions:start -->
- openvibe-contracts: v0.122.1
ddiff --git a/deploy/nginx/openvibe.bot.conf b/deploy/nginx/openvibe.bot.conf
iindex 0a582bb..f5c5e54 100644
---- a/deploy/nginx/openvibe.bot.conf
++++ b/deploy/nginx/openvibe.bot.conf
@@@ -5,8 +5,8 @@
## after changing it. The Node app (port 4630, systemd unit openvibe-bot) serves the REST API and
## terminates the device (`/device`), operator (`/control`) and watch (`/watch`) WebSockets; nginx terminates TLS,
## redirects www → apex, upgrades the three WebSocket routes, rate-limits sign-in, the API and the pages,
--## proxies the front page, /shared/, /robots.txt and /release.json to the app, and serves the frozen
--## OpenVibe.Sites legal pages, sitemap.xml and 404 page.
++## proxies the front page, /shared/, /robots.txt and /release.json to the app, and the app itself serves
++## the legal pages, sitemap.xml, manifest.webmanifest and the 404 page.
##
## No proxy_cache: the app decides cacheability per response (no-store on the API), and device and
## operator sockets must never be buffered or cached. Never log request bodies or the Authorization
@@@ -55,8 +55,6 @@ server {
ssl_certificate /etc/letsencrypt/live/openvibe.bot/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/openvibe.bot/privkey.pem;
-- root /opt/openvibe.sites/dist/openvibe.bot;
-- index index.html;
ssl_protocols TLSv1.2 TLSv1.3;
access_log /var/log/nginx/openvibe.bot.access.log;
@@@ -193,14 +191,12 @@ server {
# Events posts to http://127.0.0.1:4630/internal/events directly; never proxied from outside (ADR-033).
location /internal/ { return 404; }
-- # ── The legal pages, robots.txt, sitemap.xml and 404 page: still the frozen OpenVibe.Sites files ──
--
-- # proxy_intercept_errors stays off, so this 404 page does not affect proxied responses.
-- error_page 404 /404.html;
-- location = /404.html { internal; }
--
++ # ── Everything else: the app. There is no static root here any more — the app serves the legal pages,
++ # robots.txt, sitemap.xml, manifest.webmanifest and its own 404 page (server/app.js, server/web/routes.js). ──
location / {
-- try_files $uri $uri.html $uri/ =404;
++ limit_req zone=ovbot_api burst=20 nodelay;
++ limit_req_status 429;
++ proxy_pass http://127.0.0.1:4630;
}
location ~ /\.(?!well-known) {
ddiff --git a/server/app.js b/server/app.js
iindex 3ddbc58..073bcfa 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -7,7 +7,8 @@
* GET /api/health, /api/ready, /release.json, /metrics (direct loopback callers only)
* /api/v1/* the API (service tokens + user tokens, see api/v1.js)
* /auth/* Network SSO session for people (the control WS accepts the ov_token cookie)
-- * GET / the front page (web/home.js)
++ * GET / the front page (web/home.js); /sitemap.xml, /manifest.webmanifest and /robots.txt (web/routes.js)
++ * GET /terms, /privacy, /dmca the legal pages (openvibe-shared/legal)
* /robots, /pair/:id, /panel/:id the signed-in pages (web/routes.js); a `sim` robot is driven by the
* in-process simulator (sim/index.js), which server/index.js starts for existing ones at boot
*
@@@ -38,11 +39,24 @@ const { createJobs } = require('./jobs');
const { createActorLimits } = require('./api/actor-limits');
const { createSessionRoutes, viewerMiddleware } = require('./web/session');
const { createWebRoutes } = require('./web/routes');
++const { HOME_CSP } = require('./web/home');
const { inputError } = require('./util');
const { createBotReadiness, registerBotGauges } = require('./observability');
const VERSION = require('../package.json').version;
++/**
++ * The browser-facing 404 (the API's is problem+json): a plain, JavaScript-free page with a way back.
++ * The app's CSP applies, so it carries no inline styles or scripts.
++ */
++const NOT_FOUND_HTML = `<!doctype html>
++<html lang="en">
++<head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
++<meta name="robots" content="noindex"><title>Not found — OpenVibe.Bot</title></head>
++<body><main><h1>Not found</h1><p>There is nothing at this address. Go to the <a href="/">OpenVibe.Bot home page</a>.</p></main></body>
++</html>
++`;
++
function createApp(opts = {}) {
const config = opts.config || loadConfig();
const db = opts.db || openDb(config);
@@@ -112,12 +126,18 @@ function createApp(opts = {}) {
const limits = createActorLimits({ config, valkey, now: opts.limitsNow || now, registry: metrics.registry, log });
app.use('/api/v1', express.json({ limit: '64kb' }), (req, res, next) => { res.setHeader('Cache-Control', 'no-store'); next(); }, apiAuth.middleware, v1Router({ domain, apiAuth, limits, hub, jobs, config }));
app.use('/auth', createSessionRoutes(config, userAuth, { fetchImpl }));
++ // ── The legal pages: Terms, Privacy and DMCA on openvibe.bot itself (openvibe-shared/legal). ──
++ // The document carries its own inline theme and OpenVibe Frame boot, which the strict site-wide CSP
++ // refuses, so these three public pages get the front page's allowances (HOME_CSP). The `profile` is the
++ // legal profile from openvibe-contracts manifests/services/bot.json (site.legalProfile).
++ { const legal = require('openvibe-shared/legal');
++ app.get(legal.PATHS, (req, res, next) => { res.setHeader('Content-Security-Policy', HOME_CSP); next(); }, legal.handler({ id: 'bot', service: 'bot', host: 'openvibe.bot', name: 'OpenVibe.Bot', profile: 'games' })); }
app.use(viewerMiddleware(userAuth));
app.use(createWebRoutes(config, { domain, sim, onvif, limits, identity, hub, log }));
app.use((req, res) => {
if (req.path.startsWith('/api/') || req.path.startsWith('/internal/')) return http.sendProblem(res, 404, 'not_found', { ctx: req.ov });
-- return res.status(404).type('text/plain').send('Not found\n');
++ return res.status(404).type('html').send(NOT_FOUND_HTML);
});
// eslint-disable-next-line no-unused-vars
app.use((err, req, res, next) => {
ddiff --git a/server/web/routes.js b/server/web/routes.js
iindex e214281..f69c30b 100644
---- a/server/web/routes.js
++++ b/server/web/routes.js
@@@ -1,9 +1,9 @@
'use strict';
/**
-- * The non-API surface: the front page at `/` (web/home.js), `/robots.txt`, `/install`, a 302 to
-- * OpenVibe.Node's installer script (the one-paste command, ADR-043), and the signed-in pages (the five-minute
-- * path and the profile-rendered panel, plan T15 step 3):
++ * The non-API surface: the front page at `/` (web/home.js), `/robots.txt`, `/sitemap.xml`,
++ * `/manifest.webmanifest`, `/install`, a 302 to OpenVibe.Node's installer script (the one-paste command,
++ * ADR-043), and the signed-in pages (the five-minute path and the profile-rendered panel, plan T15 step 3):
*
* GET /robots the signed-in person's robots and the "add a robot" form
* POST /robots add one (form post) → 303 to its panel (a `sim` robot), else 201 with its pairing page
@@@ -30,6 +30,8 @@
*/
const path = require('path');
const express = require('express');
++const seo = require('openvibe-shared/seo');
++const appIcon = require('openvibe-shared/app-icon');
const { getProfile, listProfiles } = require('../profiles');
const { frameAncestors } = require('../config');
const { BotError } = require('../util');
@@@ -39,6 +41,11 @@ const { renderHome, HOME_CSP } = require('./home');
const PUBLIC = path.join(__dirname, '..', '..', 'public');
const HOLD_RESEND_MS = 150;
++// The tagline stated for openvibe.bot (openvibe-contracts manifests/services/bot.json site.tagline).
++const SITE_TAGLINE = 'An open control panel for robots';
++// Every public GET page, for the sitemap. The interactive pages (/robots, /pair/, /panel/) need a session and
++// are Disallowed in robots.txt; the API and sign-in are not pages.
++const PUBLIC_PAGES = ['/', '/install', '/terms', '/privacy', '/dmca'];
// The panel plays the robot's live video with mediasoup-client, whose npm package ships no browser
// bundle: the page imports the pinned ESM build OpenVibe.Live uses from esm.sh, and its viewer
// signaling endpoint is OpenRestream's wss host (OpenRestream README "playback descriptor"). Exactly these two
@@@ -58,6 +65,16 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
'Disallow: /robots', 'Disallow: /pair/', 'Disallow: /panel/', 'Disallow: /install', `Sitemap: ${config.baseUrl}/sitemap.xml`, ''].join('\n')));
// `curl -fsSL` follows the redirect. The target is config only (checked at boot): no query parameter steers it.
r.get('/install', (req, res) => res.redirect(302, config.installer.sourceUrl));
++ // The sitemap lists every public GET page (openvibe-shared/seo); robots.txt already names it.
++ r.get('/sitemap.xml', (req, res) => {
++ res.type('application/xml').send(seo.sitemapXml(PUBLIC_PAGES.map((p) => ({ loc: `${config.baseUrl}${p}` }))));
++ });
++ // The web app manifest (openvibe-shared/app-icon): the same icon design and start URL every OpenVibe site ships.
++ r.get('/manifest.webmanifest', (req, res) => {
++ res.type('application/manifest+json').send(JSON.stringify(appIcon.manifest({
++ site: 'bot', name: 'OpenVibe.Bot', shortName: 'Bot', description: SITE_TAGLINE, startUrl: '/',
++ })));
++ });
if (!domain) return r;
const asset = (file, type) => (req, res) => { res.setHeader('Cache-Control', 'no-cache'); res.type(type).sendFile(path.join(PUBLIC, file)); };
ddiff --git a/test/home.test.js b/test/home.test.js
iindex 009d6ea..b06da83 100644
---- a/test/home.test.js
++++ b/test/home.test.js
@@@ -51,16 +51,19 @@ const { boot, check, done } = require('./helpers/app');
assert.strictEqual(r.headers.get('content-security-policy'), "default-src 'self'; frame-ancestors 'self'; object-src 'none'; base-uri 'self'");
});
-- await check('the vhost proxies / and /shared/ to the app and keeps the legal pages on the Sites files', async () => {
++ await check('the vhost proxies / and /shared/ to the app; the legal pages are the app\'s, not the Sites files', async () => {
const conf = fs.readFileSync(path.join(__dirname, '..', 'deploy', 'nginx', 'openvibe.bot.conf'), 'utf8');
assert.match(conf, /location \^~ \/shared\/ \{\s*proxy_pass http:\/\/127\.0\.0\.1:4630;\s*\}/);
assert.ok(!conf.includes('alias /opt/openvibe.sites/dist/_shared/'), 'no second copy of the shared assets');
++ assert.ok(!conf.includes('/opt/openvibe.sites'), 'nothing is served from a Sites checkout');
assert.match(conf, /location = \/index\.html \{ return 301 \/; \}/, 'the replaced Sites page is never served by name');
assert.match(conf, /location = \/robots\.txt \{\s*proxy_pass http:\/\/127\.0\.0\.1:4630;\s*\}/);
assert.match(conf, /location = \/release\.json \{\s*proxy_pass http:\/\/127\.0\.0\.1:4630;\s*\}/);
assert.match(conf, /location = \/status\.json \{ return 404; \}/);
assert.match(conf, /location = \/ \{\s*limit_req zone=ovbot_api/);
-- assert.match(conf, /root \/opt\/openvibe\.sites\/dist\/openvibe\.bot;/);
++ assert.match(conf, /location \/ \{\s*limit_req zone=ovbot_api burst=20 nodelay;\s*limit_req_status 429;\s*proxy_pass http:\/\/127\.0\.0\.1:4630;\s*\}/, 'location / proxies everything else to the app');
++ assert.ok(!conf.includes('try_files'), 'no static file fallback');
++ assert.ok(!/error_page 404/.test(conf), 'the app owns the 404 page');
});
await t.close();
ddiff --git a/test/legal.test.js b/test/legal.test.js
nnew file mode 100644
iindex 0000000..47e4a10
---- /dev/null
++++ b/test/legal.test.js
@@@ -0,0 +1,70 @@
++'use strict';
++// The pages openvibe.bot used to get from the frozen OpenVibe.Sites checkout: the legal pages, the sitemap,
++// the web app manifest and the browser-facing 404. Bot serves them itself now, and the nginx vhost no longer
++// points at a Sites root.
++const assert = require('assert');
++const fs = require('fs');
++const path = require('path');
++const { boot, check, done } = require('./helpers/app');
++
++(async () => {
++ const t = await boot({ openre: false });
++ const get = (p) => fetch(t.base + p, { redirect: 'manual' });
++
++ await check('GET /terms, /privacy, /dmca: OpenVibe.Bot legal pages served by the app', async () => {
++ for (const p of ['/terms', '/privacy', '/dmca']) {
++ const r = await get(p);
++ assert.strictEqual(r.status, 200, p);
++ assert.match(r.headers.get('content-type'), /^text\/html/, p);
++ // The document ships its own inline theme and Frame boot: its CSP must allow them, not the site-wide one.
++ assert.match(r.headers.get('content-security-policy'), /script-src 'self' 'unsafe-inline' https:\/\/openvibe\.network/, `${p} lets its Frame boot run`);
++ const html = await r.text();
++ assert.match(html, /OpenVibe\.Bot/, `${p} names OpenVibe.Bot`);
++ assert.match(html, /<h1>/, `${p} has a heading`);
++ }
++ });
++
++ await check('GET /sitemap.xml: every public page, absolute, named by robots.txt', async () => {
++ const robots = await (await get('/robots.txt')).text();
++ assert.match(robots, /^Sitemap: http:\/\/bot\.test\/sitemap\.xml$/m, 'robots.txt names the sitemap');
++ const r = await get('/sitemap.xml');
++ assert.strictEqual(r.status, 200);
++ assert.match(r.headers.get('content-type'), /xml/);
++ const xml = await r.text();
++ for (const p of ['/', '/install', '/terms', '/privacy', '/dmca']) {
++ assert.ok(xml.includes(`<loc>http://bot.test${p}</loc>`), `sitemap lists ${p}`);
++ }
++ });
++
++ await check('GET /manifest.webmanifest: JSON, OpenVibe.Bot / Bot', async () => {
++ const r = await get('/manifest.webmanifest');
++ assert.strictEqual(r.status, 200);
++ assert.match(r.headers.get('content-type'), /application\/(manifest\+)?json/);
++ const m = JSON.parse(await r.text());
++ assert.strictEqual(m.name, 'OpenVibe.Bot');
++ assert.strictEqual(m.short_name, 'Bot');
++ assert.strictEqual(m.start_url, '/');
++ assert.ok(Array.isArray(m.icons) && m.icons.length > 0, 'has icons');
++ });
++
++ await check('an unknown page is the app\'s 404 HTML page; the API keeps its problem+json 404', async () => {
++ const page = await get('/no-such-page');
++ assert.strictEqual(page.status, 404);
++ assert.match(page.headers.get('content-type'), /^text\/html/);
++ assert.match(await page.text(), /Not found/);
++ const api = await fetch(t.base + '/api/no-such-route');
++ assert.strictEqual(api.status, 404);
++ assert.match(api.headers.get('content-type'), /application\/problem\+json/);
++ });
++
++ await check('the vhost no longer serves anything from a Sites checkout', async () => {
++ const conf = fs.readFileSync(path.join(__dirname, '..', 'deploy', 'nginx', 'openvibe.bot.conf'), 'utf8');
++ assert.ok(!conf.includes('/opt/openvibe.sites'), 'no Sites path in the vhost');
++ assert.ok(!conf.includes('try_files'), 'no static file fallback');
++ assert.ok(!/error_page\s+404/.test(conf), 'the app owns the 404 page');
++ assert.match(conf, /location \/ \{\s*limit_req zone=ovbot_api burst=20 nodelay;\s*limit_req_status 429;\s*proxy_pass http:\/\/127\.0\.0\.1:4630;\s*\}/, 'location / proxies everything else to the app');
++ });
++
++ await t.close();
++ done();
++})();