openvibers/openvibe.bot Commits
Bot: a phone is the robot: scan the pairing QR code, and its browser streams its camera, speaks, shows a face and reports battery and tilt (#91)
Changes
@@@ -16,6 +16,7 @@ OpenVibe.Bot pairs a robot's machines, keeps their state and gates every operato- The front page, `GET /` (`server/web/home.js`): the OpenVibe Frame (`openvibe-shared/shell`) around `openvibe-shared/showcase` sections that say only what works today (pairing, the panel's controls, the gate and the e-stop, people and the queue, the read-only embed, the simulator and the kits with drivers), with its own CSP for the Frame's calls to the Network; the rest of the site keeps `default-src 'self'`. `/shared/` is openvibe-shared's serve handler.- The signed-in pages (`server/web/routes.js`): `/robots` to list and add robots, `/pair/:id` for a pairing code and the installer command, and `/panel/:id`, the panel rendered from the profile. A `sim.rover` robot is driven by an in-process simulator (`server/sim`), so the panel works before any hardware exists. It gets no jobs, and it is not counted as a device or reported online in the outbox. The rover lives in a small arena (`public/sim-world.js`, shared by the server and the browser): walls and crates stop it, its telemetry carries a range finder (`distance_cm`), three line sensors over a strip of tape and its pose, and its panel draws the camera from that pose (`public/sim-view.js`) with a map of the arena in the corner.- The robot directory (`server/web/directory.js`, inside the OpenVibe Frame): `/robots` is public and lists the robots their owners list (online state, model, owner, who may drive), every model Bot drives and, signed in, your own robots; `/models` and `/models/:id` say how each model connects, its setup steps, its guide and the controls its panel gets; `/r/:id` is a listed robot's public page, watching it live through the embeddable panel; `/robots/new` adds one (name, what it is, who may drive). The owner lists a robot from its panel (`POST /robots/:id/listing`, migration 0009): listing turns on watching (`embed_public`), and turning watching off unlists it.++- A phone as the robot (`phone.camera`, driver `phone`): the pairing page shows a QR code (`server/web/qr.js`) that opens `/device/phone` with the robot and code in the URL fragment; the phone pairs over `POST /api/v1/pair` and its browser is the device (`public/phone-device.js`): `/device` with the credential offered as the subprotocol `ov.device.<credential>` (a browser cannot set `Authorization`), its camera published over WHIP to OpenRestream, `say` through speech synthesis, `display` faces and text full-screen, a torch and a chime, battery and tilt telemetry, and a Stop button that latches a local stop. Nothing to install.- `/try`, public and without an account: the same panel on a rover that runs in the visitor's own browser (`public/try-device.js` answers the panel's `/control` frames with the gate's shaping, deadman and latched e-stop), so anyone can drive before owning hardware. Nothing after the page reaches the server. The panel (`public/panel.js`, no build step) drives with a touch joystick, the keyboard or a gamepad, shows a latency meter and one camera tile per camera the profile lists; a tile plays the robot's OpenRestream WebRTC session when one is live (the mediasoup-client viewer flow OpenVibe.Live runs, against the session's playback descriptor), and stays the placeholder when there is none.- The legal pages (`/terms`, `/privacy`, `/dmca`, `openvibe-shared/legal`), `/sitemap.xml`, `/manifest.webmanifest`, `/robots.txt` and the browser-facing 404 (`server/app.js`, `server/web/routes.js`): Bot serves them all itself, and the nginx vhost has no static root.- The public build pages (`server/web/docs.js`): `/docs` (an index), `/docs/drivers` (write a driver for the OpenVibe Node — the plugin protocol, every message, the safety rules and how to test), and `/docs/profiles` (the `bot.robot-profile@1` fields, a full example and a validator). The validator answers `POST /docs/profiles/validate` (a form) and `POST /api/v1/profiles/validate` (JSON in, `{ valid, problems[] }` out) by running the same checks a shipped profile gets (`server/profiles` `checkProfile`); the pages carry no script and run under the site-wide CSP.@@@ -128,7 +129,7 @@ Set `BOT_ONVIF_CAMERAS` (JSON, unset by default) to the server-side cameras BotOpenVibe.Sites no longer generates `deploy/nginx/openvibe.bot.conf` (sites.json marks the vhost as owned by OpenVibe.Bot), and Bot serves nothing from a Sites checkout: the legal pages, `robots.txt`, `sitemap.xml`, `manifest.webmanifest` and the 404 page all come from the app. A Sites deploy leaves an installed vhost it no longer generates in place, so the current Sites vhost keeps serving openvibe.bot until Bot's first deploy installs this file over it; later Sites deploys never overwrite Bot's `/device` and `/control` routes.<!-- versions:start -->--- openvibe-contracts: v0.143.0++- openvibe-contracts: v0.157.0- openvibe-sdk: v0.44.0- openvibe-shared: v4.1.3<!-- versions:end -->@@@ -5,11 +5,11 @@"runtime": "node 22, express 4, ws; PostgreSQL 18 via openvibe-sdk/db (pg, or embedded PGlite in tests); Valkey optional; no build step","port": 4630,"domain": "openvibe.bot (www -> apex; /device and /control WebSockets; /install)",-- "contracts": "openvibe-contracts v0.143.0",++ "contracts": "openvibe-contracts v0.157.0","sdk": "openvibe-sdk v0.44.0","shared": "openvibe-shared v4.1.3","packages": [-- "openvibe-contracts v0.143.0",++ "openvibe-contracts v0.157.0","openvibe-sdk v0.44.0","openvibe-shared v4.1.3"],@@@ -44,6 +44,7 @@"signed-in pages: /robots/new (add), /pair/:id (the code and the one-paste installer command) and /panel/:id, rendered from the robot's profile and driven over /control; a sim.rover robot is driven by an in-process simulated device in a small arena shared with the browser (public/sim-world.js: walls and crates stop it; telemetry carries distance_cm, line sensors and its pose, from which the panel draws its camera and a map)","/try (public, no account): the real panel on a sim.rover that runs in the visitor's browser (public/try-device.js answers /control locally with the gate's value shaping, the deadman and the latched e-stop); listed in the sitemap and llms.txt","the robot directory (Frame pages, server/web/directory.js): public /robots (listed robots with online state, model and owner; every model; a signed-in person's own robots), /models and /models/:id (how it connects, setup steps, guide, panel controls), /r/:id (a listed robot's public page with its live embeddable panel), /robots/new (name, model choice grouped by connection kind, who may drive); owners list from the panel (POST /robots/:id/listing, robots.listed, migration 0009), listing turns on embed_public and unwatching unlists; listed robots and every model are in the sitemap",++ "a phone as the robot (profile phone.camera, driver phone, Contracts 0.157.0): the pairing page's QR code (server/web/qr.js, qrcode-generator) opens /device/phone with the robot and code in the fragment; the phone pairs over POST /api/v1/pair and its browser is the device (public/phone-device.js): /device with the credential as the subprotocol ov.device.<credential>, the camera over WHIP to OpenRestream, say (speech synthesis), display (faces, text), torch, chime, battery and tilt telemetry, a local Stop (estop_state local_stop); /device/phone has its own CSP (the WHIP ingest origin) and Permissions-Policy camera=(self)","the panel's People card: the owner adds an operator or viewer by @username (OpenVibe.Network resolves the name to a subject) and removes them again, shown with Network's names; a signed-in visitor on an open-queue robot sees their turn or their place in line and can leave the queue, promoting the next person (POST /robots/:id/operators and …/operators/:subject/remove, POST /robots/:id/queue/leave)","panel input: a touch joystick, the keyboard and a gamepad (Gamepad API), each held control re-sent while held and zeroed at once on release; a latency meter (round-trip median and telemetry age); one camera tile per camera the profile lists, playing the robot's live OpenRestream WebRTC session (the Adeept and Cozmo profiles carry a camera widget since 2026-10-13); a readout shows the sensor keys the Node's drivers report (distance_cm, line, cliff and picked_up); a live online indicator on /pair/:id read from /control","platform.job@1 jobs dispatched to a paired Node over /device (platform.job-frame@1: resent unacked on reconnect, only to an advertised runtime class) and metered per second into run_usage_outbox for billing.usage.record",@@@ -13,10 +13,11 @@"dotenv": "^16.4.7","express": "^4.21.2","iovalkey": "^0.4.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3","pg": "^8.23.0",++ "qrcode-generator": "^2.0.4","ws": "^8.18.0"},"devDependencies": {@@@ -844,9 +845,9 @@}},"node_modules/openvibe-contracts": {-- "version": "0.143.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",-- "integrity": "sha512-Cdbf+9RoPAUBibpM2zXQvtsiKMugb55dKubTt8pFZvAmxsLtGvX+zaC++yL1vOSub8FX3+GgqcGtoJS6tklrUg==",++ "version": "0.157.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0",++ "integrity": "sha512-GhcDBEdSZOxuKjO3ivN7lZlm685ULRABNkOIxSpJIRwv9qTuATcJe/PwjHfUlcr+VmZeJkd2eJZJl07yIuomdw==","license": "AGPL-3.0-only","dependencies": {"ajv": "^8.17.1",@@@ -1056,6 +1057,12 @@"url": "https://opencollective.com/express"}},++ "node_modules/qrcode-generator": {++ "version": "2.0.4",++ "resolved": "https://registry.npmjs.org/qrcode-generator/-/qrcode-generator-2.0.4.tgz",++ "integrity": "sha512-mZSiP6RnbHl4xL2Ap5HfkjLnmxfKcPWpWe/c+5XxCuetEenqmNFf1FH/ftXPCtFG5/TDobjsjz6sSNL0Sr8Z9g==",++ "license": "MIT"++ },"node_modules/qs": {"version": "6.16.0","resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",@@@ -25,10 +25,11 @@"dotenv": "^16.4.7","express": "^4.21.2","iovalkey": "^0.4.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3","pg": "^8.23.0",++ "qrcode-generator": "^2.0.4","ws": "^8.18.0"},"engines": {@@@ -40,7 +40,7 @@.dir-chips li { font-size: 12.5px; padding: 2px 9px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); color: var(--text-secondary, #96a7c2); margin: 0; }.dir-badge { flex: none; font-size: 11.5px; font-weight: 700; letter-spacing: .03em; padding: 2px 8px; border-radius: 999px; border: 1px solid currentColor; }--.dir-browser { color: #a78bfa; } .dir-onboard { color: #22c55e; } .dir-bridge { color: #f59e0b; } .dir-server { color: #38bdf8; }++.dir-browser { color: #a78bfa; } .dir-phone { color: #f472b6; } .dir-onboard { color: #22c55e; } .dir-bridge { color: #f59e0b; } .dir-server { color: #38bdf8; }.dir-pill { display: inline-block; font-size: 12.5px; padding: 2px 10px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); text-decoration: none; color: var(--text-primary, #e6edf7) !important; }.dir-pill.dir-quiet { color: var(--text-secondary, #96a7c2) !important; }.dir-state { display: inline-flex; align-items: center; gap: 6px; font-size: 13px; font-weight: 650; color: var(--text-secondary, #96a7c2); }@@@ -295,3 +295,40 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf.try-next h2 { margin: 0 0 0.35rem; font-size: 1.1rem; }.try-next p { color: var(--muted); line-height: 1.55; }.try-actions { display: flex; flex-wrap: wrap; gap: 0.5rem; margin-bottom: 0; }++++/* The pairing page's QR code for a phone (server/web/qr.js): white on its own quiet zone, sized for a phone camera. */++.qr-box { width: min(15rem, 70vw); margin: 0.5rem 0 0.75rem; padding: 0.5rem; background: #fff; border-radius: 12px; }++.qr-box .qr { display: block; width: 100%; height: auto; }++++/* /device/phone (public/phone-device.js): setup, then the phone is the robot's face. */++.phone-page { min-height: 100dvh; }++.phone { min-height: 100dvh; display: grid; }++.phone-setup { max-width: 30rem; margin: 0 auto; padding: 1.5rem 1rem 2rem; align-self: center; }++.phone-brand { display: inline-flex; align-items: center; gap: 0.4rem; margin: 0 0 1rem; font-weight: 700; }++.phone-brand b { color: var(--accent); }++.phone-setup h1 { margin: 0 0 0.5rem; font-size: 1.6rem; }++.phone-setup .muted { color: var(--muted); line-height: 1.55; }++.phone-setup .small { font-size: 0.85rem; }++.phone-form { display: grid; gap: 0.85rem; margin: 1.25rem 0; }++.phone-form .field { display: grid; gap: 0.35rem; }++.phone-form .field-label { font-size: 0.8rem; font-weight: 700; letter-spacing: 0.05em; text-transform: uppercase; color: var(--muted); }++.phone-form input:not([type="checkbox"]) { padding: 0 0.75rem; border: 1px solid var(--line); background: var(--bg); color: var(--text); font-size: 1.05rem; }++.phone-form .check { display: flex; align-items: center; gap: 0.5rem; color: var(--muted); }++.phone-form button.primary { font-size: 1.1rem; min-height: 52px; }++.phone-paired { margin: 0; }++.phone-run { display: grid; grid-template-rows: auto 1fr auto auto; min-height: 100dvh; }++.phone-bar { display: flex; flex-wrap: wrap; gap: 0.4rem; align-items: center; padding: 0.6rem 0.75rem; padding-top: max(0.6rem, env(safe-area-inset-top)); }++.phone-pill { font-size: 0.8rem; padding: 0.15rem 0.6rem; border-radius: 999px; border: 1px solid var(--line); color: var(--muted); }++.phone-pill:empty { display: none; }++.phone-pill[data-state="online"] { color: var(--ok); border-color: color-mix(in srgb, var(--ok) 45%, transparent); }++.phone-pill[data-state="connecting"] { color: var(--warn); }++.phone-rtt { margin-left: auto; font: 0.8rem ui-monospace, monospace; color: var(--muted); }++.phone-face { display: grid; place-items: center; color: #e6edf3; padding: 1rem; }++.phone-face svg { width: min(92vw, 70vh); height: auto; }++.phone-text { margin: 0; max-width: 92vw; text-align: center; font-weight: 700; line-height: 1.15; overflow-wrap: anywhere; }++.phone[data-stopped="true"] .phone-face { color: var(--warn); }++.phone-safety { margin: 0 0.75rem; padding: 0.6rem 0.75rem; border-radius: 10px; background: color-mix(in srgb, var(--warn) 14%, transparent); color: var(--text); }++.phone-preview { position: fixed; right: 0.75rem; top: 3rem; width: 26vw; max-width: 9rem; aspect-ratio: 9 / 16; object-fit: cover; border-radius: 10px; border: 1px solid var(--line); background: #000; opacity: 0.85; }++.phone-foot { display: flex; gap: 0.5rem; padding: 0.75rem; padding-bottom: max(0.75rem, env(safe-area-inset-bottom)); }++.phone-stop { flex: 1; min-height: 56px; font-size: 1.15rem; font-weight: 800; letter-spacing: 0.04em; text-transform: uppercase; background: var(--danger); border-color: var(--danger); color: #fff; }++.phone-stop.resume { background: var(--ok); border-color: var(--ok); color: #04130a; }nnew file mode 100644@@@ -0,0 +1,404 @@++'use strict';++++/**++ * /device/phone: this phone is the robot's device, from its browser, with nothing to install (plan T15 "an Android++ * phone as the robot's brain", profile driver `phone`). Plain ES, no build step, no framework.++ *++ * pair POST /api/v1/pair with the one-time code (from the pairing page's QR code, or typed) → the device id,++ * its credential and its WHIP URL, kept in this browser's localStorage (shown once by Bot, so kept here)++ * link wss://<host>/device, the credential offered as the subprotocol `ov.device.<credential>` (a browser++ * cannot set the Authorization header); hello → config → status, then a heartbeat every++ * config.heartbeat_ms and telemetry (battery, tilt) at most once a second++ * camera getUserMedia → WHIP (RFC 9725) to OpenRestream at the whip_url Bot gave at pairing; the robot's panel++ * plays it. Restarted with backoff when it drops++ * commands say (speechSynthesis), display (a face or text, full screen), actuator torch (the camera's torch) and++ * chime (a tone while it is held: its deadline ends it, as a motor's does), halt; each acked or nacked++ * safety Bot's e-stop silences everything and refuses all but halt; the phone's own Stop button latches a local++ * stop (estop_state local_stop) that only this phone's Resume releases++ *++ * The page needs one tap to start (browsers allow sound, the camera and the wake lock only after a gesture).++ */++(() => {++ const main = document.getElementById('phone');++ if (!main) return;++ const $ = (sel) => main.querySelector(sel);++ const KEY = 'ovbot.phone.v1';++ const AGENT = 'openvibe-phone/1';++ const load = () => { try { return JSON.parse(localStorage.getItem(KEY) || 'null'); } catch { return null; } };++ const save = (d) => { try { localStorage.setItem(KEY, JSON.stringify(d)); } catch { /* private mode: this visit only */ } };++ const forget = () => { try { localStorage.removeItem(KEY); } catch { /* nothing kept */ } };++++ // ── Setup: the QR code's fragment (#robot=…&code=…) fills the form, then leaves the address bar ──────────++ const form = $('[data-pair-form]');++ const errorEl = $('[data-error]');++ const fragment = new URLSearchParams(location.hash.slice(1));++ if (fragment.get('robot')) form.elements.robot.value = fragment.get('robot');++ if (fragment.get('code')) form.elements.code.value = fragment.get('code');++ if (location.hash) history.replaceState(null, '', location.pathname);++ if (form.elements.robot.value) $('[data-robot-field]').hidden = true;++++ let dev = load();++ function showError(text) { errorEl.textContent = text || ''; errorEl.hidden = !text; }++ function showSetup() {++ main.dataset.state = 'setup';++ $('[data-setup]').hidden = false; $('[data-run]').hidden = true;++ $('[data-paired]').hidden = !dev; $('[data-pair-fields]').hidden = !!dev;++ if (dev) $('[data-paired-name]').textContent = dev.robot_name || dev.robot_id;++ }++ showSetup();++++ const deviceName = () => {++ const ua = navigator.userAgent;++ const model = (/Android [^;]+; ([^;)]+)/.exec(ua) || [])[1] || (/iPhone/.test(ua) ? 'iPhone' : /iPad/.test(ua) ? 'iPad' : 'Phone');++ return `${model.trim()} (browser)`.slice(0, 80);++ };++++ async function pair(robot, code) {++ const r = await fetch('/api/v1/pair', {++ method: 'POST', headers: { 'Content-Type': 'application/json' },++ body: JSON.stringify({++ robot, code, agent_version: AGENT, device_kind: 'onboard', drivers: ['phone'], name: deviceName(),++ capabilities: { camera: !!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia), speech: 'speechSynthesis' in window, tilt: 'DeviceOrientationEvent' in window },++ }),++ });++ let j = {};++ try { j = await r.json(); } catch { /* no body */ }++ if (!r.ok) throw new Error(j.detail || j.title || `pairing failed (${r.status})`);++ return { device_id: j.device_id, credential: j.credential, whip_url: j.whip_url || null, robot_id: j.robot_id, robot_name: null, profile_id: j.profile ? j.profile.id : null };++ }++++ form.addEventListener('submit', async (e) => {++ e.preventDefault();++ showError('');++ const button = form.querySelector('button[type="submit"]');++ try {++ if (!dev) {++ const robot = form.elements.robot.value.trim();++ const code = form.elements.code.value.trim();++ if (!robot || !code) return showError('Enter the robot and the pairing code from its pairing page.');++ button.disabled = true;++ dev = await pair(robot, code);++ save(dev);++ }++ front = !!form.elements.front.checked;++ await start();++ } catch (err) {++ showError(err.message || String(err));++ } finally { button.disabled = false; }++ });++ $('[data-forget]').addEventListener('click', () => {++ if (!confirm('Unpair this phone? The robot keeps its other devices; pairing again needs a new code.')) return;++ stopAll(); forget(); dev = null; showSetup();++ });++ $('[data-unpair-setup]').addEventListener('click', () => { forget(); dev = null; showSetup(); });++++ // ── Running ───────────────────────────────────────────────────────────────────────────────────────────++ let ws = null; let seq = 0; let retry = 0; let running = false;++ let heartbeatMs = 1000; let heartbeatTimer = null; let telemetryTimer = null;++ let rtt = null; let remoteLatched = false; let localStop = false;++ let front = false;++ const linkEl = $('[data-link-state]');++ const videoEl = $('[data-video-state]');++ const stopBtn = $('[data-local-stop]');++++ function out(type, fields = {}) {++ if (!ws || ws.readyState !== 1) return false;++ ws.send(JSON.stringify({ v: 1, seq: ++seq, ts: Date.now(), type, ...fields }));++ return true;++ }++ function setLink(text, state) { linkEl.textContent = text; linkEl.dataset.state = state; }++ function setVideo(text, state) { videoEl.textContent = text; videoEl.dataset.state = state; }++++ async function start() {++ running = true;++ main.dataset.state = 'run';++ $('[data-setup]').hidden = true; $('[data-run]').hidden = false;++ unlockAudio();++ askTilt();++ keepAwake();++ battery();++ showFace('neutral');++ connect();++ startVideo();++ }++ function stopAll() {++ running = false;++ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);++ if (ws) { ws.onclose = null; try { ws.close(); } catch { /* gone */ } ws = null; }++ stopVideo();++ stopEffects();++ }++++ function connect() {++ if (!running || !dev) return;++ setLink('Connecting…', 'connecting');++ ws = new WebSocket(`${location.protocol === 'https:' ? 'wss' : 'ws'}://${location.host}/device`, ['ov.device', `ov.device.${dev.credential}`]);++ ws.onmessage = (e) => { let m; try { m = JSON.parse(e.data); } catch { return; } onFrame(m); };++ ws.onclose = (e) => {++ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);++ stopEffects();++ if (!running) return;++ if (e.code === 4002 || e.code === 4003) {++ // The credential was revoked or rotated, or the robot was removed: this phone must pair again.++ running = false; stopVideo(); forget(); dev = null; showSetup();++ return showError('This phone is no longer paired with the robot. Ask its owner for a new pairing code.');++ }++ setLink('Reconnecting…', 'connecting');++ setTimeout(connect, Math.min(15000, 500 * 2 ** retry++) + Math.random() * 400);++ };++ }++++ function onFrame(m) {++ if (m.type === 'hello') { retry = 0; return; }++ if (m.type === 'config') {++ heartbeatMs = Math.max(250, Math.min(5000, Number(m.heartbeat_ms) || 1000));++ remoteLatched = !!m.estop_latched;++ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);++ heartbeatTimer = setInterval(() => out('heartbeat', { t: Date.now(), rtt_ms: rtt }), heartbeatMs);++ telemetryTimer = setInterval(sendTelemetry, 1000);++ out('status', { firmware: AGENT, capabilities: { torch: torchSupported(), speech: 'speechSynthesis' in window }, faults: faults(), estop_latched: remoteLatched || localStop });++ if (localStop) out('estop_state', { latched: true, local_stop: true, reason: 'stopped on the phone' });++ paintSafety();++ setLink('Connected', 'online');++ return;++ }++ if (m.type === 'heartbeat_ack') { if (Number.isFinite(m.echo)) rtt = Math.max(0, Date.now() - m.echo); return; }++ if (m.type === 'estop') { remoteLatched = !!m.latched; if (remoteLatched) stopEffects(); paintSafety(); return; }++ if (m.type === 'command') return onCommand(m);++ if (m.type === 'rotate') return undefined; // a Network-paired machine's frame; a phone holds a Bot credential++ return undefined;++ }++++ function onCommand(m) {++ const ack = () => out('ack', { id: m.id });++ const nack = (code) => out('nack', { id: m.id, fault_code: code });++ const v = m.value && typeof m.value === 'object' ? m.value : {};++ if (m.kind === 'halt') { stopEffects(); return ack(); }++ if (localStop) return nack('local_stop');++ if (remoteLatched) return nack('estopped');++ if (m.kind === 'say') return speak(String(v.text || '')) ? ack() : nack('unsupported');++ if (m.kind === 'display') {++ if (v.face) showFace(String(v.face)); else if (v.text) showText(String(v.text)); else return nack('invalid');++ return ack();++ }++ if (m.kind === 'actuator' && v.name === 'torch') {++ setTorch(!!v.value).then((ok) => (ok ? ack() : nack('unsupported')));++ return undefined;++ }++ if (m.kind === 'actuator' && v.name === 'chime') {++ tone(v.value && Number.isFinite(v.value.hz) ? v.value.hz : null, m.deadline_ms);++ return ack();++ }++ return nack('unsupported');++ }++++ // ── The phone's own Stop ──────────────────────────────────────────────────────────────────────────────++ stopBtn.addEventListener('click', () => {++ localStop = !localStop;++ if (localStop) { stopEffects(); out('estop_state', { latched: true, local_stop: true, reason: 'stopped on the phone' }); }++ else out('estop_state', { latched: false });++ paintSafety();++ });++ function paintSafety() {++ main.dataset.stopped = String(localStop || remoteLatched);++ stopBtn.textContent = localStop ? 'Resume' : 'Stop';++ stopBtn.classList.toggle('resume', localStop);++ const note = $('[data-safety]');++ note.textContent = localStop ? 'Stopped on this phone: press Resume, then the owner clears the e-stop.' : remoteLatched ? 'The robot\'s e-stop is latched: the owner clears it on the panel.' : '';++ note.hidden = !note.textContent;++ }++++ // ── Telemetry ─────────────────────────────────────────────────────────────────────────────────────────++ let tilt = null; let batt = null;++ function askTilt() {++ const listen = () => window.addEventListener('deviceorientation', (e) => {++ if (e.beta == null && e.gamma == null) return;++ tilt = { pitch: Math.round(e.beta || 0), roll: Math.round(e.gamma || 0) };++ });++ // iOS asks once, from a tap.++ if (window.DeviceOrientationEvent && typeof DeviceOrientationEvent.requestPermission === 'function') {++ DeviceOrientationEvent.requestPermission().then((s) => { if (s === 'granted') listen(); }, () => {});++ } else listen();++ }++ function battery() {++ if (!navigator.getBattery) return;++ navigator.getBattery().then((b) => {++ const read = () => { batt = { percent: Math.round(b.level * 100), charging: !!b.charging }; };++ read(); b.addEventListener('levelchange', read); b.addEventListener('chargingchange', read);++ }, () => {});++ }++ function faults() {++ const list = [];++ if (videoFault) list.push({ code: 'camera', driver: 'phone', message: videoFault });++ return list;++ }++ function sendTelemetry() {++ const frame = { sensors: tilt ? { tilt } : {} };++ if (batt) frame.battery = { percent: batt.percent };++ out('telemetry', frame);++ $('[data-rtt]').textContent = rtt == null ? '' : `${rtt} ms`;++ }++++ // ── Camera over WHIP ──────────────────────────────────────────────────────────────────────────────────++ const preview = $('[data-preview]');++ let stream = null; let pc = null; let resource = null; let videoRetry = 0; let videoTimer = null; let videoFault = null;++ async function startVideo() {++ clearTimeout(videoTimer);++ if (!running) return;++ if (!dev.whip_url) { setVideo('No video on this server', 'off'); return; }++ try {++ setVideo('Starting the camera…', 'connecting');++ if (!stream) {++ stream = await navigator.mediaDevices.getUserMedia({++ video: { facingMode: front ? 'user' : 'environment', width: { ideal: 1280 }, height: { ideal: 720 }, frameRate: { ideal: 30, max: 30 } },++ audio: false,++ });++ preview.srcObject = stream;++ preview.hidden = false;++ }++ pc = new RTCPeerConnection({ bundlePolicy: 'max-bundle' });++ for (const track of stream.getVideoTracks()) pc.addTransceiver(track, { direction: 'sendonly', streams: [stream] });++ await pc.setLocalDescription(await pc.createOffer());++ await new Promise((done) => {++ if (pc.iceGatheringState === 'complete') return done();++ const t = setTimeout(done, 2500);++ pc.addEventListener('icegatheringstatechange', () => { if (pc.iceGatheringState === 'complete') { clearTimeout(t); done(); } });++ });++ const r = await fetch(dev.whip_url, { method: 'POST', headers: { 'Content-Type': 'application/sdp' }, body: pc.localDescription.sdp });++ if (r.status !== 201) throw new Error(`the video server answered ${r.status}${r.headers.get('X-WHIP-ERROR') ? ` (${r.headers.get('X-WHIP-ERROR')})` : ''}`);++ resource = r.headers.get('Location') ? new URL(r.headers.get('Location'), dev.whip_url).href : null;++ await pc.setRemoteDescription({ type: 'answer', sdp: await r.text() });++ pc.addEventListener('connectionstatechange', () => {++ if (pc && pc.connectionState === 'connected') { videoRetry = 0; videoFault = null; setVideo('Streaming', 'online'); }++ if (pc && (pc.connectionState === 'failed' || pc.connectionState === 'disconnected')) restartVideo('the video connection dropped');++ });++ } catch (err) {++ const denied = err && (err.name === 'NotAllowedError' || err.name === 'SecurityError');++ videoFault = denied ? 'camera permission was refused' : String(err.message || err).slice(0, 160);++ setVideo(denied ? 'Camera blocked: allow it in the browser\'s site settings' : 'Camera trouble, retrying…', 'off');++ if (!denied) restartVideo(videoFault);++ }++ }++ function restartVideo(why) {++ videoFault = why;++ closePeer();++ setVideo('Reconnecting the camera…', 'connecting');++ clearTimeout(videoTimer);++ videoTimer = setTimeout(startVideo, Math.min(30000, 1000 * 2 ** videoRetry++));++ }++ function closePeer() {++ if (resource) { fetch(resource, { method: 'DELETE', keepalive: true }).catch(() => {}); resource = null; }++ if (pc) { try { pc.close(); } catch { /* gone */ } pc = null; }++ }++ function stopVideo() {++ clearTimeout(videoTimer);++ closePeer();++ if (stream) { for (const t of stream.getTracks()) t.stop(); stream = null; }++ preview.srcObject = null;++ preview.hidden = true;++ setVideo('', 'off');++ }++ const torchTrack = () => (stream ? stream.getVideoTracks()[0] : null);++ function torchSupported() {++ const t = torchTrack();++ const caps = t && t.getCapabilities ? t.getCapabilities() : {};++ return !!caps.torch;++ }++ async function setTorch(on) {++ const t = torchTrack();++ if (!t || !torchSupported()) return false;++ try { await t.applyConstraints({ advanced: [{ torch: on }] }); return true; } catch { return false; }++ }++++ // ── Voice, chime and face ─────────────────────────────────────────────────────────────────────────────++ let audio = null; let osc = null; let gain = null; let toneTimer = null;++ function unlockAudio() {++ const Ctx = window.AudioContext || window.webkitAudioContext;++ if (!audio && Ctx) { try { audio = new Ctx(); } catch { audio = null; } }++ if (audio && audio.state === 'suspended') audio.resume().catch(() => {});++ if ('speechSynthesis' in window) speechSynthesis.getVoices();++ }++ function speak(text) {++ if (!('speechSynthesis' in window) || !text) return false;++ speechSynthesis.cancel();++ const u = new SpeechSynthesisUtterance(text.slice(0, 200));++ speechSynthesis.speak(u);++ showText(text, 4000);++ return true;++ }++ function tone(hz, deadline) {++ clearTimeout(toneTimer);++ if (!audio || hz == null) { silence(); return; }++ if (!osc) {++ osc = audio.createOscillator(); gain = audio.createGain();++ osc.type = 'triangle'; gain.gain.value = 0;++ osc.connect(gain).connect(audio.destination); osc.start();++ }++ osc.frequency.setValueAtTime(hz, audio.currentTime);++ gain.gain.setTargetAtTime(0.25, audio.currentTime, 0.01);++ const until = Number.isFinite(deadline) ? Math.max(50, deadline - Date.now()) : 400;++ toneTimer = setTimeout(silence, until); // the chime's deadman: held means re-sent++ }++ function silence() {++ clearTimeout(toneTimer);++ if (osc && audio) { gain.gain.setTargetAtTime(0, audio.currentTime, 0.02); const o = osc; osc = null; setTimeout(() => { try { o.stop(); } catch { /* stopped */ } }, 120); }++ }++ function stopEffects() {++ silence();++ if ('speechSynthesis' in window) speechSynthesis.cancel();++ setTorch(false);++ }++++ const face = $('[data-face]');++ const SVG = 'http://www.w3.org/2000/svg';++ const el = (name, attrs) => { const n = document.createElementNS(SVG, name); for (const [k, v] of Object.entries(attrs)) n.setAttribute(k, v); return n; };++ // Each face: two eyes and a mouth on a 200×120 box.++ const EYE = { cx: [62, 138], cy: 46 };++ const FACES = {++ neutral: { eyes: 'round', mouth: 'M70 92 L130 92' },++ happy: { eyes: 'round', mouth: 'M64 84 Q100 116 136 84' },++ sad: { eyes: 'round', mouth: 'M66 100 Q100 74 134 100' },++ surprised: { eyes: 'wide', mouth: 'circle' },++ sleepy: { eyes: 'closed', mouth: 'M84 94 L116 94' },++ wink: { eyes: 'wink', mouth: 'M64 84 Q100 112 136 84' },++ love: { eyes: 'heart', mouth: 'M64 84 Q100 116 136 84' },++ angry: { eyes: 'angry', mouth: 'M66 100 Q100 80 134 100' },++ };++ let textTimer = null;++ function showFace(name) {++ clearTimeout(textTimer);++ const f = FACES[name] || FACES.neutral;++ const svg = el('svg', { viewBox: '0 0 200 120', role: 'img', 'aria-label': `${name} face` });++ const stroke = { stroke: 'currentColor', 'stroke-width': 8, 'stroke-linecap': 'round', fill: 'none' };++ EYE.cx.forEach((cx, i) => {++ if (f.eyes === 'closed' || (f.eyes === 'wink' && i === 1)) svg.append(el('path', { d: `M${cx - 16} ${EYE.cy} Q${cx} ${EYE.cy + 12} ${cx + 16} ${EYE.cy}`, ...stroke }));++ else if (f.eyes === 'heart') svg.append(el('path', { d: `M${cx} ${EYE.cy + 14} L${cx - 16} ${EYE.cy - 2} A8 8 0 0 1 ${cx} ${EYE.cy - 10} A8 8 0 0 1 ${cx + 16} ${EYE.cy - 2} Z`, fill: '#ff5c8a' }));++ else {++ svg.append(el('circle', { cx, cy: EYE.cy, r: f.eyes === 'wide' ? 17 : 13, fill: 'currentColor' }));++ if (f.eyes === 'angry') svg.append(el('path', { d: i === 0 ? `M${cx - 20} ${EYE.cy - 26} L${cx + 14} ${EYE.cy - 14}` : `M${cx + 20} ${EYE.cy - 26} L${cx - 14} ${EYE.cy - 14}`, ...stroke }));++ }++ });++ svg.append(f.mouth === 'circle' ? el('circle', { cx: 100, cy: 94, r: 11, ...stroke }) : el('path', { d: f.mouth, ...stroke }));++ face.replaceChildren(svg);++ face.dataset.mode = 'face';++ }++ function showText(text, ms = 0) {++ clearTimeout(textTimer);++ const p = document.createElement('p');++ p.className = 'phone-text';++ p.textContent = text.slice(0, 200);++ p.style.fontSize = `${Math.max(5, Math.min(18, 160 / Math.max(6, text.length)))}vmin`;++ face.replaceChildren(p);++ face.dataset.mode = 'text';++ if (ms) textTimer = setTimeout(() => showFace('neutral'), ms);++ }++++ // ── Keep the screen on while running ──────────────────────────────────────────────────────────────────++ let wake = null;++ async function keepAwake() {++ if (!('wakeLock' in navigator) || !running) return;++ try { wake = await navigator.wakeLock.request('screen'); } catch { wake = null; }++ }++ document.addEventListener('visibilitychange', () => { if (!document.hidden && running) keepAwake(); });++ window.addEventListener('pagehide', closePeer); // ends the WHIP session (keepalive) so the panel's video stops at once++})();@@@ -34,10 +34,10 @@ const DIR = path.join(__dirname);const CAPABILITIES = new Set(['drive.differential', 'drive.mecanum','servo.pan_tilt', 'head', 'lift',-- 'lights.rgb', 'lights.backpack', 'lights.cube',++ 'lights.rgb', 'lights.backpack', 'lights.cube', 'lights.torch','speaker.horn', 'speaker.say','display.text',-- 'sensor.ultrasonic', 'sensor.line', 'sensor.cliff', 'sensor.pickup',++ 'sensor.ultrasonic', 'sensor.line', 'sensor.cliff', 'sensor.pickup', 'sensor.tilt','battery', 'ptz', 'camera',// A robot that carries no hardware of its own: the owner's Node forwards the panel's commands to a// local script with the relay plugin (relay.generic, plan T15 R9).@@@ -49,7 +49,8 @@ const WIDGETS = new Set(['drive', 'pan-tilt', 'servo', 'lights', 'horn', 'speake/** Device drivers a mapping may name. `relay` is a Node plugin with no hardware map of its own. */// `looi`: the community's Generic LOOI driver, a Bluetooth bridge plugin (OpenVibe.Node docs/hardware-looi.md).--const DRIVERS = new Set(['pca9685', 'ads7830', 'cozmo', 'looi', 'onvif', 'sim', 'relay']);++// `phone`: a phone that is the device itself, from its browser (public/phone-device.js), no Node and nothing installed.++const DRIVERS = new Set(['pca9685', 'ads7830', 'cozmo', 'looi', 'onvif', 'sim', 'relay', 'phone']);const ID_RE = /^[a-z][a-z0-9._-]{1,63}$/;const KINDS = new Set(['onboard', 'bridge', 'server']);nnew file mode 100644@@@ -0,0 +1,27 @@++{++ "id": "phone.camera",++ "version": 1,++ "name": "A phone",++ "vendor": "Any",++ "kind": "onboard",++ "description": "Any phone as a robot's eyes, voice and face, from its browser with nothing to install: its camera streams to the panel, it says what you type, shows a face or text full-screen, turns its torch on, chimes, and reports its battery and how it is tilted.",++ "capabilities": ["camera", "speaker.say", "speaker.horn", "display.text", "lights.torch", "battery", "sensor.tilt"],++ "mapping": { "driver": "phone" },++ "commands": {++ "say": { "max_chars": 200 },++ "display": { "modes": ["face", "text"], "faces": ["happy", "neutral", "sad", "surprised", "sleepy", "wink", "love", "angry"], "max_chars": 80 },++ "actuator": { "names": { "torch": { "type": "bool" }, "chime": { "type": "tone", "hz": [220, 1760] } } }++ },++ "widgets": [++ { "type": "camera", "capability": "camera", "label": "Camera", "order": 1 },++ { "type": "speaker", "capability": "speaker.say", "label": "Say", "order": 2, "command": { "kind": "say" } },++ { "type": "display", "capability": "display.text", "label": "Face", "order": 3, "command": { "kind": "display" } },++ { "type": "lights", "capability": "lights.torch", "label": "Torch", "order": 4, "command": { "kind": "actuator", "names": ["torch"] } },++ { "type": "horn", "capability": "speaker.horn", "label": "Chime", "order": 5, "command": { "kind": "actuator", "names": ["chime"] } },++ { "type": "telemetry", "capability": "sensor.tilt", "label": "Tilt", "order": 6 },++ { "type": "battery", "capability": "battery", "label": "Battery", "order": 7 },++ { "type": "latency", "label": "Latency", "order": 8 }++ ],++ "camera": { "transport": "whip", "resolution": "1280x720" },++ "limits": { "max_speed": 1, "max_turn": 1, "max_command_ms": 300, "heartbeat_ms": 1000 }++}@@@ -45,6 +45,16 @@ function parseCookies(header) {}const bearer = (req) => { const h = String(req.headers.authorization || ''); return h.startsWith('Bearer ') ? h.slice(7).trim() : null; };const isSubject = (v) => typeof v === 'string' && ids.isSubjectId('user', v);++const DEVICE_PROTOCOL = 'ov.device';++const CREDENTIAL_RE = /^[A-Za-z0-9_-]{43}$/; // a device credential (32 random bytes, base64url); never a node token++/** A browser device's credential, offered as `ov.device.<credential>` beside `ov.device`; anything else → null. */++function deviceCredentialFromProtocols(header) {++ const offered = String(header || '').split(',').map((s) => s.trim()).filter(Boolean);++ if (!offered.includes(DEVICE_PROTOCOL)) return null;++ const named = offered.find((p) => p.startsWith(`${DEVICE_PROTOCOL}.`));++ const value = named ? named.slice(DEVICE_PROTOCOL.length + 1) : '';++ return CREDENTIAL_RE.test(value) ? value : null;++}// The keys the Node's drivers report a sensor capability under (the Adeept and Cozmo plugins: distance_cm, line).const SENSOR_KEYS = { ultrasonic: ['distance_cm'], line: ['line'], cliff: ['cliff', 'picked_up'], pickup: ['picked_up', 'pickup'] };const finite = (v) => (typeof v === 'number' && Number.isFinite(v) ? v : 0);@@@ -71,7 +81,10 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Datconst timers = [];let closed = false;-- const deviceWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false });++ // A browser device (a phone, public/phone-device.js) cannot set the upgrade's Authorization header: it offers its++ // credential as a subprotocol (`ov.device`, `ov.device.<credential>`) and gets `ov.device` back.++ const deviceWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false,++ handleProtocols: (protocols) => (protocols.has(DEVICE_PROTOCOL) ? DEVICE_PROTOCOL : false) });// A socket that offers a control ticket gets the ticket protocol back (a browser drops a socket whose offered// subprotocol is not answered); a plain one, none.const controlWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false,@@@ -148,7 +161,7 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Datws.on('message', (raw) => onDeviceFrame(conn, raw));ws.on('close', () => { conn.backlog = null; dropDevice(conn); });ws.on('error', () => { /* the close event does the work */ });-- const cred = bearer(req); // the query string is never read++ const cred = bearer(req) || deviceCredentialFromProtocols(req.headers['sec-websocket-protocol']); // the query string is never readif (cred) {conn.backlog = [];try {@@@ -25,6 +25,7 @@ const CONNECT = {onboard: { label: 'On-board', text: 'The OpenVibe Node runs on the robot\'s own computer and drives its hardware.' },bridge: { label: 'Bridge', text: 'The OpenVibe Node runs on a computer or phone next to the robot and joins the robot\'s own link. Nothing on the robot changes.' },server: { label: 'Server-side', text: 'Nothing to install: Bot connects to the device itself.' },++ phone: { label: 'Phone', text: 'The phone\'s browser is the device: scan a code, allow the camera, keep the page open. Nothing to install.' },};const MODELS = {'sim.rover': {@@@ -54,6 +55,12 @@ const MODELS = {setup: ['The community\'s Generic LOOI driver bridges to the desk robot over Bluetooth from a Windows PC.', 'Install the Node with --driver looi; the phone in LOOI keeps doing what it does.'],guide: { href: `${NODE_BLOB}/docs/hardware-looi.md`, label: 'LOOI driver contract' },},++ 'phone.camera': {++ connect: 'phone', order: 1,++ setup: ['Add the robot here and choose A phone; its pairing page shows a QR code.', 'Scan it with the phone, tap Start and allow the camera. The phone streams, speaks, shows a face and reports its battery and tilt.',++ 'Keep the page open (the screen stays on). Its Stop button holds the robot until it is released on the phone.'],++ guide: { href: '/device/phone', label: 'The phone\'s page' },++ },'camera.onvif': {connect: 'server', order: 5,setup: ['Any ONVIF pan-tilt-zoom camera: Bot moves it, with the same gate and limits as a robot.', 'Today an operator names the camera in Bot\'s configuration (its address and the names of its secrets).'],@@@ -70,6 +77,7 @@ const CAPABILITY_WORDS = {'lights.rgb': 'RGB lights', 'lights.backpack': 'Backpack lights', 'lights.cube': 'Cube lights', 'speaker.horn': 'Horn', 'speaker.say': 'Speaks','display.text': 'Face and text', 'sensor.ultrasonic': 'Range finder', 'sensor.line': 'Line tracking', 'sensor.cliff': 'Cliff sensors','sensor.pickup': 'Pick-up sensor', battery: 'Battery', ptz: 'Pan, tilt, zoom', camera: 'Live camera', relay: 'Your own buttons',++ 'lights.torch': 'Torch', 'sensor.tilt': 'Tilt sensor',};const POLICY_WORDS = { private: 'Only its owner and the people they add drive it.', invite: 'People the owner invites drive it.', queue: 'Anyone signed in can take a timed turn.' };@@@ -138,7 +146,7 @@ ${listed.length ? `<ul class="dir-grid">${listed.map(robotCard).join('')}</ul>`// ── GET /models ─────────────────────────────────────────────────────────────────────────────────────function renderModels({ config, models = [] }) {-- const groups = ['browser', 'onboard', 'bridge', 'server'].map((k) => [k, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);++ const groups = ['browser', 'phone', 'onboard', 'bridge', 'server'].map((k) => [k, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);const body = `<header class="dir-hero"><h1>Models</h1><p class="muted">Every robot OpenVibe.Bot has a profile for. A profile lays out the panel, so each of these drives with the controls it really has.</p></header>${groups.map(([k, ms]) => `<section class="dir-sec" aria-labelledby="g-${esc(k)}"><h2 id="g-${esc(k)}">${esc(CONNECT[k].label)}</h2><p class="muted">${esc(CONNECT[k].text)}</p>@@@ -190,7 +198,7 @@ function renderAddRobot({ config, models = [], values = {}, error = null }) {const chosen = values.profile_id || 'sim.rover';const option = (m) => `<label class="dir-choice"><input type="radio" name="profile_id" value="${esc(m.id)}"${m.id === chosen ? ' checked' : ''} required>`+ `<span class="dir-choice-body"><span class="dir-card-head"><b>${esc(m.name)}</b>${badge(m.connect)}</span><span class="dir-card-text">${esc(CONNECT[m.connect].text)}</span></span></label>`;-- const groups = [['browser', 'Try it with no hardware'], ['onboard', 'A robot with its own computer'], ['bridge', 'A robot you connect through a computer next to it'], ['server', 'A camera Bot connects to']]++ const groups = [['browser', 'Try it with no hardware'], ['phone', 'A phone, from its browser'], ['onboard', 'A robot with its own computer'], ['bridge', 'A robot you connect through a computer next to it'], ['server', 'A camera Bot connects to']].map(([k, title]) => [title, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);const body = `<p class="dir-crumb">${link('/robots', 'Robots')}</p><header class="dir-hero"><h1>Add a robot</h1><p class="muted">Name it, say what it is, and choose who may drive. A simulated rover drives the moment it is added; anything else gets a one-time pairing code and one command to paste on its computer.</p></header>nnew file mode 100644@@@ -0,0 +1,29 @@++'use strict';++++/**++ * A QR code as inline SVG markup (no script, no image request, so the strict CSP holds): one path of dark modules++ * on a light quiet zone, scaling with the box it is put in. qrcode-generator (MIT) does the encoding.++ *++ * qrSvg(text, { label }) → '<svg …>' error correction M; `label` is the image's accessible name++ */++const qrcode = require('qrcode-generator');++++const ESC = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };++const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => ESC[c]);++++function qrSvg(text, { label = 'QR code' } = {}) {++ const qr = qrcode(0, 'M');++ qr.addData(String(text));++ qr.make();++ const n = qr.getModuleCount();++ const quiet = 4;++ let d = '';++ for (let r = 0; r < n; r++) {++ for (let c = 0; c < n; c++) if (qr.isDark(r, c)) d += `M${c + quiet} ${r + quiet}h1v1h-1z`;++ }++ const size = n + quiet * 2;++ return `<svg class="qr" viewBox="0 0 ${size} ${size}" role="img" aria-label="${esc(label)}" shape-rendering="crispEdges" xmlns="http://www.w3.org/2000/svg">`++ + `<rect width="${size}" height="${size}" fill="#fff"/><path d="${d}" fill="#000"/></svg>`;++}++++module.exports = { qrSvg };@@@ -18,6 +18,7 @@* camerasOf(profile)* esc(value)*/++const { qrSvg } = require('./qr');const ESC = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => ESC[c]);/** A data-* attribute's JSON, escaped for a double-quoted attribute. */@@@ -493,15 +494,23 @@ ${error ? `<p class="error" role="alert">${esc(error)}</p>` : ''}return page('Your robots', `${topbar()}\n<main class="page">\n<h1>Your robots</h1>\n${list}\n${form}\n${kitsSection(kits, profiles)}\n</main>`);}--function renderPairingPage({ robot, pairing, profile = null }) {++function renderPairingPage({ robot, pairing, profile = null, phoneUrl = null }) {++ // A phone is the device itself: it pairs from its browser (the QR code opens /device/phone with the code).++ const phone = !!(profile && profile.mapping && profile.mapping.driver === 'phone' && phoneUrl);++ const first = phone++ ? `<li class="card phone-pair"><h2>Scan this with the phone <span class="pill">${esc(profile.name)}</span></h2>++<div class="qr-box">${qrSvg(phoneUrl, { label: `Opens the pairing page for ${robot.name} on the phone` })}</div>++<p class="muted">It opens the robot's page on the phone with this code filled in. Tap Pair, allow the camera, and keep the page open: the phone is the robot now. No app to install.</p>++<p class="muted">No camera app handy? Open <b>openvibe.bot/device/phone</b> on the phone and enter the robot and code by hand: <code>${esc(robot.id)}</code></p></li>`++ : `<li class="card"><h2>Run this on the machine that serves the robot${profile ? ` <span class="pill">${esc(profile.name)}</span>` : ''}</h2>++<pre class="installer"><code data-copy-source>${esc(pairing.installer)}</code></pre>++<button type="button" class="copy" data-copy>${icon('copy')}<span data-copy-label>Copy the command</span></button></li>`;const body = `${topbar(`<span class="crumb">${esc(robot.name)}</span>`)}<main class="page pairing" data-pair-robot="${esc(robot.id)}"><h1>Pair ${esc(robot.name)}</h1><p class="pair-status" data-pair-status data-state="waiting" role="status" aria-live="polite"><span class="dot"></span><span data-pair-text>Waiting for the device.</span> <a class="button primary" href="/panel/${esc(robot.id)}" data-pair-open hidden>Open the panel</a></p><ol class="steps">--<li class="card"><h2>Run this on the machine that serves the robot${profile ? ` <span class="pill">${esc(profile.name)}</span>` : ''}</h2>--<pre class="installer"><code data-copy-source>${esc(pairing.installer)}</code></pre>--<button type="button" class="copy" data-copy>${icon('copy')}<span data-copy-label>Copy the command</span></button></li>++${first}<li class="card"><h2>Or enter the pairing code</h2><p><strong class="code">${esc(pairing.code)}</strong></p><p class="muted">Valid until <time datetime="${esc(pairing.expires_at)}">${esc(pairing.expires_at)}</time>. It works once.</p></li>@@@ -511,4 +520,38 @@ function renderPairingPage({ robot, pairing, profile = null }) {return page(`Pair ${robot.name}`, body, { scripts: ['/panel/panel.js'] });}--module.exports = { page, renderPanel, panelScripts, isSim, TRY_DESCRIPTION, renderEmbedRefused, renderDrivePage, renderWidget, renderRobotsPage, renderPairingPage, camerasOf, esc };++/**++ * /device/phone: the page that makes a phone the robot's device (public/phone-device.js). Its setup form pairs it (the++ * QR code fills in the robot and code); once running, the screen is the robot's face, with the link and camera state++ * above it and the phone's own Stop below.++ */++function renderPhonePage() {++ const body = `<main id="phone" class="phone" data-state="setup">++<section class="phone-setup" data-setup>++<p class="phone-brand">${icon('robot')}<span>OpenVibe<b>.Bot</b></span></p>++<h1>Make this phone a robot</h1>++<p class="muted">Pair it with a robot you added on openvibe.bot. Its camera streams to the robot's panel, it says what the driver types, shows a face, and reports its battery and tilt. Nothing to install: keep this page open.</p>++<form class="phone-form" data-pair-form>++<div data-pair-fields>++<label class="field" data-robot-field><span class="field-label">Robot</span><input name="robot" autocomplete="off" spellcheck="false" placeholder="rob_…"></label>++<label class="field"><span class="field-label">Pairing code</span><input name="code" autocomplete="one-time-code" autocapitalize="characters" spellcheck="false" placeholder="ABCD-EFGH"></label>++</div>++<p class="phone-paired" data-paired hidden>Paired as <b data-paired-name></b>. <button type="button" class="quiet" data-unpair-setup>Pair with another robot</button></p>++<label class="check"><input type="checkbox" name="front"> Use the front camera</label>++<button type="submit" class="primary">Start</button>++<p class="error" data-error role="alert" hidden></p>++</form>++<p class="muted small">Get a code: on openvibe.bot, add a robot, choose <b>A phone</b>, and scan the code its pairing page shows.</p>++</section>++<section class="phone-run" data-run hidden>++<header class="phone-bar"><span class="phone-pill" data-link-state data-state="connecting">Connecting…</span><span class="phone-pill" data-video-state data-state="off"></span><span class="phone-rtt" data-rtt></span></header>++<div class="phone-face" data-face aria-live="polite"></div>++<p class="phone-safety" data-safety role="status" hidden></p>++<video class="phone-preview" data-preview muted playsinline autoplay hidden></video>++<footer class="phone-foot"><button type="button" class="phone-stop" data-local-stop>Stop</button><button type="button" class="quiet" data-forget>Unpair</button></footer>++</section>++</main>`;++ return page('This phone is a robot', body, { scripts: ['/panel/phone-device.js'], bodyClass: 'phone-page', description: 'Turn a phone into a robot on OpenVibe.Bot from its browser: its camera, voice and face, with nothing to install.', canonical: '/device/phone' });++}++++module.exports = { page, renderPanel, renderPhonePage, panelScripts, isSim, TRY_DESCRIPTION, renderEmbedRefused, renderDrivePage, renderWidget, renderRobotsPage, renderPairingPage, camerasOf, esc };@@@ -36,7 +36,7 @@ const { getProfile, listProfiles, loadProfiles } = require('../profiles');const { frameAncestors } = require('../config');const { BotError } = require('../util');const kits = require('../kits');--const { renderPanel, renderEmbedRefused, renderDrivePage, renderPairingPage } = require('./render');++const { renderPanel, renderEmbedRefused, renderDrivePage, renderPairingPage, renderPhonePage } = require('./render');const { renderDocsIndex, renderDriversPage, renderProfilesPage } = require('./docs');const { checkProfile } = require('../profiles');const { renderHome, HOME_CSP } = require('./home');@@@ -171,6 +171,21 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limir.get('/panel/sim-world.js', asset('sim-world.js', 'application/javascript'));r.get('/panel/sim-view.js', asset('sim-view.js', 'application/javascript'));r.get('/panel/try-device.js', asset('try-device.js', 'application/javascript'));++ r.get('/panel/phone-device.js', asset('phone-device.js', 'application/javascript'));++++ // /device/phone: a phone becomes the robot's device from its browser (plan T15). Public: the one-time pairing code++ // is the credential, and a paired phone keeps its device credential in its own storage. Its CSP adds only the++ // WHIP ingest the camera publishes to (BOT_WHIP_BASE's origin) and Cloudflare's beacon.++ const whipOrigin = (() => { try { return config.media.whipBase ? new URL(config.media.whipBase).origin : ''; } catch { return ''; } })();++ const PHONE_CSP = ["default-src 'self'", "script-src 'self' https://static.cloudflareinsights.com",++ `connect-src 'self'${whipOrigin ? ` ${whipOrigin}` : ''} https://cloudflareinsights.com`, "media-src 'self' blob:",++ "frame-ancestors 'none'", "object-src 'none'", "base-uri 'self'"].join('; ');++ r.get('/device/phone', (req, res) => {++ res.setHeader('Content-Security-Policy', PHONE_CSP);++ res.setHeader('Permissions-Policy', 'camera=(self), microphone=(), geolocation=()');++ res.setHeader('Cache-Control', 'public, max-age=300');++ res.type('html').send(renderPhonePage());++ });r.get('/directory.css', asset('directory.css', 'text/css'));// /try: the real panel driving a rover simulated in the visitor's own browser (public/try-device.js), so anyone@@@ -249,6 +264,9 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi} catch (e) { return next(e); }});++ // The phone's pairing link (the QR code): the robot and code ride in the fragment, which never reaches a server.++ const phoneUrl = (robotId, pairing) => `${config.baseUrl}/device/phone#robot=${encodeURIComponent(robotId)}&code=${encodeURIComponent(pairing.code)}`;++// The embed flag is web-only: domain.present.robot (and so /api/v1) does not carry it.const presentForPage = (row) => ({ ...domain.present.robot(row), embed_public: !!row.embed_public, listed: !!row.listed });@@@ -332,7 +350,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi// A simulated robot needs no machine: straight to its panel. Otherwise the code create minted is shown here.const profile = await profileOf(robot);if (await attachInProcess(robot, profile)) return res.redirect(303, `/panel/${robot.id}`);-- return res.status(201).type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile }));++ return res.status(201).type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile, phoneUrl: phoneUrl(robot.id, pairing) }));}));// A load replaces the robot's unused code, so only a navigation from this site (or a typed address) mints one.@@@ -348,7 +366,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limiif (!robot) throw new BotError(404, 'bot.robot_not_found', 'no such robot');if (robot.owner_subject !== me.subject) throw new BotError(403, 'bot.forbidden', 'only the owner may pair a device');const pairing = await domain.pairing.create(robot.id, me.subject, undefined, robot);-- res.type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile: await profileOf(robot) }));++ res.type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile: await profileOf(robot), phoneUrl: phoneUrl(robot.id, pairing) }));}));r.post('/robots/:id/listing', ...manage, express.urlencoded({ extended: false, limit: '1kb' }), page(async (req, res) => {@@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');await check('profiles are public reads; an unknown profile is a problem+json 404', async () => {const list = await t.call('GET', '/api/v1/profiles', { token: null });assert.strictEqual(list.status, 200);-- assert.deepStrictEqual(list.json.profiles.map((p) => p.id).sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);++ assert.deepStrictEqual(list.json.profiles.map((p) => p.id).sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);const one = await t.call('GET', '/api/v1/profiles/adeept.adr036', { token: null });assert.strictEqual(one.json.profile.limits.max_command_ms, 300);const missing = await t.call('GET', '/api/v1/profiles/nope', { token: null });@@@ -200,8 +200,33 @@ const looi = {};/** The contract a profile's commands must meet: its Node plugin, or the server driver. */++/**++ * The phone driver = public/phone-device.js onCommand: say {text}; display {face} (its FACES) or {text}; actuator++ * torch (true/false) and chime ({hz} or null); halt. Anything else is nacked unsupported.++ */++const PHONE_FACES = ['happy', 'neutral', 'sad', 'surprised', 'sleepy', 'wink', 'love', 'angry'];++const phone = {++ name: 'phone (browser)',++ accepts(kind, v) {++ if (kind === 'halt') return halt(v);++ if (!isObj(v)) return `${kind} takes an object`;++ if (kind === 'say') return typeof v.text === 'string' && v.text && only(v, ['text']) ? null : 'say is {text}';++ if (kind === 'display') {++ if (only(v, ['face']) && PHONE_FACES.includes(v.face)) return null;++ return only(v, ['text']) && typeof v.text === 'string' && v.text ? null : 'display is {face} or {text}';++ }++ if (kind === 'actuator') {++ if (v.name === 'torch') return typeof v.value === 'boolean' ? null : 'torch is true or false';++ if (v.name === 'chime') return v.value === null || (isObj(v.value) && only(v.value, ['hz']) && num(v.value.hz, 220, 1760)) ? null : 'chime is {hz} or null';++ return `no actuator ${v.name}`;++ }++ return `kind ${kind} not supported`;++ },++};++function contractFor(profile) {const m = profile.mapping || {};++ if (m.driver === 'phone') return phone;if (m.plugin === 'adeept_adr036') return adeept(m.wheels);if (m.plugin === 'cozmo') return cozmo();if (m.plugin === 'relay') return relay;@@@ -209,4 +234,4 @@ function contractFor(profile) {return server[profile.id] || null;}--module.exports = { adeept, cozmo, relay, looi, server, contractFor, COZMO_FACES };++module.exports = { adeept, cozmo, relay, looi, phone, server, contractFor, COZMO_FACES, PHONE_FACES };@@@ -26,7 +26,7 @@ const { renderPanel } = require('../server/web/render');try {await check('every existing catalogue profile still loads under the contract', () => {-- assert.deepStrictEqual([...loadProfiles().keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);++ assert.deepStrictEqual([...loadProfiles().keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);});await check('an operator cannot edit or switch the robot-owned profile', async () => {nnew file mode 100644@@@ -0,0 +1,94 @@++'use strict';++// A phone as the robot's device (plan T15 "an Android phone as the robot's brain", profile driver `phone`): the++// pairing page shows a QR code that opens /device/phone with the code, the phone pairs over POST /api/v1/pair, and its++// browser offers the credential as a subprotocol on /device (a browser cannot set Authorization).++const assert = require('assert');++const { boot, check, done } = require('./helpers/app');++const { loadProfiles } = require('../server/profiles');++++(async () => {++ const t = await boot();++ const alex = t.network.newUser('alex');++ const cookie = (user) => ({ Cookie: `ov_token=${t.network.signUser(user)}` });++ const get = (p, user) => fetch(t.base + p, { redirect: 'manual', headers: user ? cookie(user) : {} });++++ await check('phone.camera ships: camera, voice, a face, torch, chime, tilt and battery, driver phone', () => {++ const p = loadProfiles().get('phone.camera');++ assert.strictEqual(p.mapping.driver, 'phone');++ assert.deepStrictEqual(Object.keys(p.commands).sort(), ['actuator', 'display', 'halt', 'say']);++ assert.deepStrictEqual(p.widgets.map((w) => w.type), ['camera', 'speaker', 'display', 'lights', 'horn', 'telemetry', 'battery', 'latency']);++ });++++ await check('GET /device/phone: public, its own CSP (the WHIP ingest only), camera allowed, the client script', async () => {++ const r = await get('/device/phone');++ assert.strictEqual(r.status, 200);++ const csp = r.headers.get('content-security-policy');++ assert.match(csp, /connect-src 'self'[^;]*https:\/\/cloudflareinsights\.com/);++ assert.doesNotMatch(csp, /esm\.sh|unsafe-inline/);++ assert.match(r.headers.get('permissions-policy'), /camera=\(self\)/);++ const html = await r.text();++ assert.ok(html.includes('<script src="/panel/phone-device.js" defer></script>') && html.includes('data-pair-form') && html.includes('data-local-stop'));++ assert.doesNotMatch(html, /<script>|style="/, 'nothing inline');++ assert.strictEqual((await get('/panel/phone-device.js')).status, 200);++ });++++ let robot; let pairing;++ await check('a phone robot\'s pairing page is a QR code to /device/phone with the code in the fragment, no installer', async () => {++ ({ robot, pairing } = await t.robot(alex, { name: 'Pocket bot', profile_id: 'phone.camera' }));++ const html = await (await fetch(`${t.base}/pair/${robot.id}`, { headers: { ...cookie(alex), 'Sec-Fetch-Site': 'same-origin' } })).text();++ assert.ok(html.includes('<svg class="qr"') && html.includes('Scan this with the phone'));++ assert.ok(!html.includes('data-copy-source'), 'no command to paste');++ const fresh = /class="code">([^<]+)</.exec(html)[1];++ pairing = { code: fresh };++ assert.ok(html.includes(robot.id));++ });++++ let credential;++ await check('the phone pairs over HTTP and connects with its credential as a subprotocol: hello, config, online', async () => {++ const r = await t.call('POST', '/api/v1/pair', { token: null, body: { robot: robot.id, code: pairing.code, agent_version: 'openvibe-phone/1', drivers: ['phone'], name: 'Pixel (browser)' } });++ assert.strictEqual(r.status, 201, r.text);++ assert.strictEqual(r.json.profile.id, 'phone.camera');++ credential = r.json.credential;++ assert.match(credential, /^[A-Za-z0-9_-]{43}$/);++ const d = await t.ws('/device', { protocols: ['ov.device', `ov.device.${credential}`] });++ assert.strictEqual(d.ws.protocol, 'ov.device', 'the server answers the offered protocol (a browser drops the socket otherwise)');++ assert.ok(await d.waitFor((m) => m.type === 'hello'));++ const config = await d.waitFor((m) => m.type === 'config');++ assert.ok(config.allowed_commands.includes('say') && config.allowed_commands.includes('display'));++ d.send({ v: 1, type: 'status', firmware: 'openvibe-phone/1', capabilities: {}, faults: [], estop_latched: false });++ d.send({ v: 1, type: 'telemetry', battery: { percent: 81 }, sensors: { tilt: { pitch: 4, roll: -2 } } });++ const c = await t.ws('/control', { headers: cookie(alex) });++ c.send({ type: 'join', robot_id: robot.id });++ const joined = await c.waitFor((m) => m.type === 'joined');++ assert.ok(joined.state.online, 'the phone is the robot\'s device');++ c.send({ type: 'command', id: 's1', kind: 'say', value: { text: 'Hello from the panel' } });++ const cmd = await d.waitFor((m) => m.type === 'command' && m.kind === 'say');++ assert.deepStrictEqual(cmd.value, { text: 'Hello from the panel' });++ d.send({ v: 1, type: 'ack', id: cmd.id });++ assert.strictEqual((await c.waitFor((m) => m.type === 'command_result' && m.id === 's1')).result, 'ack');++ c.send({ type: 'command', id: 'f1', kind: 'display', value: { face: 'happy' } });++ assert.deepStrictEqual((await d.waitFor((m) => m.type === 'command' && m.kind === 'display')).value, { face: 'happy' });++ c.close(); d.close();++ });++++ await check('a wrong credential as a subprotocol closes 4002; a malformed one is never read as a credential', async () => {++ const bad = await t.ws('/device', { protocols: ['ov.device', `ov.device.${'A'.repeat(43)}`] });++ assert.strictEqual(await bad.waitForClose(), 4002);++ // Not credential-shaped: the socket stays unpaired (it may only pair), it is not closed as a bad credential.++ const odd = await t.ws('/device', { protocols: ['ov.device', 'ov.device.short'] });++ odd.send({ v: 1, type: 'heartbeat', t: Date.now() });++ assert.strictEqual((await odd.waitFor((m) => m.type === 'error')).code, 'bot.not_paired');++ odd.close();++ });++++ await check('the phone model is in the directory, grouped as a phone, and offered when adding a robot', async () => {++ const models = await (await get('/models')).text();++ assert.ok(models.includes('>Phone</h2>') && models.includes('href="/models/phone.camera"'));++ const add = await (await get('/robots/new?profile=phone.camera', alex)).text();++ assert.match(add, /value="phone\.camera" checked/);++ assert.ok(add.includes('A phone, from its browser'));++ });++++ await t.close();++ done();++})();@@@ -58,8 +58,8 @@ function samples(profile, kind, names) {(async () => {const profiles = loadProfiles();-- await check('the seven shipped profiles load and validate', () => {-- assert.deepStrictEqual([...profiles.keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);++ await check('the eight shipped profiles load and validate', () => {++ assert.deepStrictEqual([...profiles.keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);for (const p of profiles.values()) {assert.ok(p.capabilities.length > 0, `${p.id} capabilities`);assert.ok(p.widgets.length > 0, `${p.id} widgets`);
@@ -16,6 +16,7 @@ OpenVibe.Bot pairs a robot's machines, keeps their state and gates every operato
@@ -16,6 +16,7 @@ OpenVibe.Bot pairs a robot's machines, keeps their state and gates every operato
- The front page, `GET /` (`server/web/home.js`): the OpenVibe Frame (`openvibe-shared/shell`) around `openvibe-shared/showcase` sections that say only what works today (pairing, the panel's controls, the gate and the e-stop, people and the queue, the read-only embed, the simulator and the kits with drivers), with its own CSP for the Frame's calls to the Network; the rest of the site keeps `default-src 'self'`. `/shared/` is openvibe-shared's serve handler.
- The front page, `GET /` (`server/web/home.js`): the OpenVibe Frame (`openvibe-shared/shell`) around `openvibe-shared/showcase` sections that say only what works today (pairing, the panel's controls, the gate and the e-stop, people and the queue, the read-only embed, the simulator and the kits with drivers), with its own CSP for the Frame's calls to the Network; the rest of the site keeps `default-src 'self'`. `/shared/` is openvibe-shared's serve handler.
- The signed-in pages (`server/web/routes.js`): `/robots` to list and add robots, `/pair/:id` for a pairing code and the installer command, and `/panel/:id`, the panel rendered from the profile. A `sim.rover` robot is driven by an in-process simulator (`server/sim`), so the panel works before any hardware exists. It gets no jobs, and it is not counted as a device or reported online in the outbox. The rover lives in a small arena (`public/sim-world.js`, shared by the server and the browser): walls and crates stop it, its telemetry carries a range finder (`distance_cm`), three line sensors over a strip of tape and its pose, and its panel draws the camera from that pose (`public/sim-view.js`) with a map of the arena in the corner.
- The signed-in pages (`server/web/routes.js`): `/robots` to list and add robots, `/pair/:id` for a pairing code and the installer command, and `/panel/:id`, the panel rendered from the profile. A `sim.rover` robot is driven by an in-process simulator (`server/sim`), so the panel works before any hardware exists. It gets no jobs, and it is not counted as a device or reported online in the outbox. The rover lives in a small arena (`public/sim-world.js`, shared by the server and the browser): walls and crates stop it, its telemetry carries a range finder (`distance_cm`), three line sensors over a strip of tape and its pose, and its panel draws the camera from that pose (`public/sim-view.js`) with a map of the arena in the corner.
- The robot directory (`server/web/directory.js`, inside the OpenVibe Frame): `/robots` is public and lists the robots their owners list (online state, model, owner, who may drive), every model Bot drives and, signed in, your own robots; `/models` and `/models/:id` say how each model connects, its setup steps, its guide and the controls its panel gets; `/r/:id` is a listed robot's public page, watching it live through the embeddable panel; `/robots/new` adds one (name, what it is, who may drive). The owner lists a robot from its panel (`POST /robots/:id/listing`, migration 0009): listing turns on watching (`embed_public`), and turning watching off unlists it.
- The robot directory (`server/web/directory.js`, inside the OpenVibe Frame): `/robots` is public and lists the robots their owners list (online state, model, owner, who may drive), every model Bot drives and, signed in, your own robots; `/models` and `/models/:id` say how each model connects, its setup steps, its guide and the controls its panel gets; `/r/:id` is a listed robot's public page, watching it live through the embeddable panel; `/robots/new` adds one (name, what it is, who may drive). The owner lists a robot from its panel (`POST /robots/:id/listing`, migration 0009): listing turns on watching (`embed_public`), and turning watching off unlists it.
+- A phone as the robot (`phone.camera`, driver `phone`): the pairing page shows a QR code (`server/web/qr.js`) that opens `/device/phone` with the robot and code in the URL fragment; the phone pairs over `POST /api/v1/pair` and its browser is the device (`public/phone-device.js`): `/device` with the credential offered as the subprotocol `ov.device.<credential>` (a browser cannot set `Authorization`), its camera published over WHIP to OpenRestream, `say` through speech synthesis, `display` faces and text full-screen, a torch and a chime, battery and tilt telemetry, and a Stop button that latches a local stop. Nothing to install.
- `/try`, public and without an account: the same panel on a rover that runs in the visitor's own browser (`public/try-device.js` answers the panel's `/control` frames with the gate's shaping, deadman and latched e-stop), so anyone can drive before owning hardware. Nothing after the page reaches the server. The panel (`public/panel.js`, no build step) drives with a touch joystick, the keyboard or a gamepad, shows a latency meter and one camera tile per camera the profile lists; a tile plays the robot's OpenRestream WebRTC session when one is live (the mediasoup-client viewer flow OpenVibe.Live runs, against the session's playback descriptor), and stays the placeholder when there is none.
- `/try`, public and without an account: the same panel on a rover that runs in the visitor's own browser (`public/try-device.js` answers the panel's `/control` frames with the gate's shaping, deadman and latched e-stop), so anyone can drive before owning hardware. Nothing after the page reaches the server. The panel (`public/panel.js`, no build step) drives with a touch joystick, the keyboard or a gamepad, shows a latency meter and one camera tile per camera the profile lists; a tile plays the robot's OpenRestream WebRTC session when one is live (the mediasoup-client viewer flow OpenVibe.Live runs, against the session's playback descriptor), and stays the placeholder when there is none.
- The legal pages (`/terms`, `/privacy`, `/dmca`, `openvibe-shared/legal`), `/sitemap.xml`, `/manifest.webmanifest`, `/robots.txt` and the browser-facing 404 (`server/app.js`, `server/web/routes.js`): Bot serves them all itself, and the nginx vhost has no static root.
- The legal pages (`/terms`, `/privacy`, `/dmca`, `openvibe-shared/legal`), `/sitemap.xml`, `/manifest.webmanifest`, `/robots.txt` and the browser-facing 404 (`server/app.js`, `server/web/routes.js`): Bot serves them all itself, and the nginx vhost has no static root.
- The public build pages (`server/web/docs.js`): `/docs` (an index), `/docs/drivers` (write a driver for the OpenVibe Node — the plugin protocol, every message, the safety rules and how to test), and `/docs/profiles` (the `bot.robot-profile@1` fields, a full example and a validator). The validator answers `POST /docs/profiles/validate` (a form) and `POST /api/v1/profiles/validate` (JSON in, `{ valid, problems[] }` out) by running the same checks a shipped profile gets (`server/profiles` `checkProfile`); the pages carry no script and run under the site-wide CSP.
- The public build pages (`server/web/docs.js`): `/docs` (an index), `/docs/drivers` (write a driver for the OpenVibe Node — the plugin protocol, every message, the safety rules and how to test), and `/docs/profiles` (the `bot.robot-profile@1` fields, a full example and a validator). The validator answers `POST /docs/profiles/validate` (a form) and `POST /api/v1/profiles/validate` (JSON in, `{ valid, problems[] }` out) by running the same checks a shipped profile gets (`server/profiles` `checkProfile`); the pages carry no script and run under the site-wide CSP.
@@ -128,7 +129,7 @@ Set `BOT_ONVIF_CAMERAS` (JSON, unset by default) to the server-side cameras Bot
@@ -128,7 +129,7 @@ Set `BOT_ONVIF_CAMERAS` (JSON, unset by default) to the server-side cameras Bot
OpenVibe.Sites no longer generates `deploy/nginx/openvibe.bot.conf` (sites.json marks the vhost as owned by OpenVibe.Bot), and Bot serves nothing from a Sites checkout: the legal pages, `robots.txt`, `sitemap.xml`, `manifest.webmanifest` and the 404 page all come from the app. A Sites deploy leaves an installed vhost it no longer generates in place, so the current Sites vhost keeps serving openvibe.bot until Bot's first deploy installs this file over it; later Sites deploys never overwrite Bot's `/device` and `/control` routes.
OpenVibe.Sites no longer generates `deploy/nginx/openvibe.bot.conf` (sites.json marks the vhost as owned by OpenVibe.Bot), and Bot serves nothing from a Sites checkout: the legal pages, `robots.txt`, `sitemap.xml`, `manifest.webmanifest` and the 404 page all come from the app. A Sites deploy leaves an installed vhost it no longer generates in place, so the current Sites vhost keeps serving openvibe.bot until Bot's first deploy installs this file over it; later Sites deploys never overwrite Bot's `/device` and `/control` routes.
<!-- versions:start -->
<!-- versions:start -->
-- openvibe-contracts: v0.143.0
+- openvibe-contracts: v0.157.0
- openvibe-sdk: v0.44.0
- openvibe-sdk: v0.44.0
- openvibe-shared: v4.1.3
- openvibe-shared: v4.1.3
<!-- versions:end -->
<!-- versions:end -->
@@ -5,11 +5,11 @@
@@ -5,11 +5,11 @@
"runtime": "node 22, express 4, ws; PostgreSQL 18 via openvibe-sdk/db (pg, or embedded PGlite in tests); Valkey optional; no build step",
"runtime": "node 22, express 4, ws; PostgreSQL 18 via openvibe-sdk/db (pg, or embedded PGlite in tests); Valkey optional; no build step",
"port": 4630,
"port": 4630,
"domain": "openvibe.bot (www -> apex; /device and /control WebSockets; /install)",
"domain": "openvibe.bot (www -> apex; /device and /control WebSockets; /install)",
- "contracts": "openvibe-contracts v0.143.0",
+ "contracts": "openvibe-contracts v0.157.0",
"sdk": "openvibe-sdk v0.44.0",
"sdk": "openvibe-sdk v0.44.0",
"shared": "openvibe-shared v4.1.3",
"shared": "openvibe-shared v4.1.3",
"packages": [
"packages": [
- "openvibe-contracts v0.143.0",
+ "openvibe-contracts v0.157.0",
"openvibe-sdk v0.44.0",
"openvibe-sdk v0.44.0",
"openvibe-shared v4.1.3"
"openvibe-shared v4.1.3"
],
],
@@ -44,6 +44,7 @@
@@ -44,6 +44,7 @@
"signed-in pages: /robots/new (add), /pair/:id (the code and the one-paste installer command) and /panel/:id, rendered from the robot's profile and driven over /control; a sim.rover robot is driven by an in-process simulated device in a small arena shared with the browser (public/sim-world.js: walls and crates stop it; telemetry carries distance_cm, line sensors and its pose, from which the panel draws its camera and a map)",
"signed-in pages: /robots/new (add), /pair/:id (the code and the one-paste installer command) and /panel/:id, rendered from the robot's profile and driven over /control; a sim.rover robot is driven by an in-process simulated device in a small arena shared with the browser (public/sim-world.js: walls and crates stop it; telemetry carries distance_cm, line sensors and its pose, from which the panel draws its camera and a map)",
"/try (public, no account): the real panel on a sim.rover that runs in the visitor's browser (public/try-device.js answers /control locally with the gate's value shaping, the deadman and the latched e-stop); listed in the sitemap and llms.txt",
"/try (public, no account): the real panel on a sim.rover that runs in the visitor's browser (public/try-device.js answers /control locally with the gate's value shaping, the deadman and the latched e-stop); listed in the sitemap and llms.txt",
"the robot directory (Frame pages, server/web/directory.js): public /robots (listed robots with online state, model and owner; every model; a signed-in person's own robots), /models and /models/:id (how it connects, setup steps, guide, panel controls), /r/:id (a listed robot's public page with its live embeddable panel), /robots/new (name, model choice grouped by connection kind, who may drive); owners list from the panel (POST /robots/:id/listing, robots.listed, migration 0009), listing turns on embed_public and unwatching unlists; listed robots and every model are in the sitemap",
"the robot directory (Frame pages, server/web/directory.js): public /robots (listed robots with online state, model and owner; every model; a signed-in person's own robots), /models and /models/:id (how it connects, setup steps, guide, panel controls), /r/:id (a listed robot's public page with its live embeddable panel), /robots/new (name, model choice grouped by connection kind, who may drive); owners list from the panel (POST /robots/:id/listing, robots.listed, migration 0009), listing turns on embed_public and unwatching unlists; listed robots and every model are in the sitemap",
+ "a phone as the robot (profile phone.camera, driver phone, Contracts 0.157.0): the pairing page's QR code (server/web/qr.js, qrcode-generator) opens /device/phone with the robot and code in the fragment; the phone pairs over POST /api/v1/pair and its browser is the device (public/phone-device.js): /device with the credential as the subprotocol ov.device.<credential>, the camera over WHIP to OpenRestream, say (speech synthesis), display (faces, text), torch, chime, battery and tilt telemetry, a local Stop (estop_state local_stop); /device/phone has its own CSP (the WHIP ingest origin) and Permissions-Policy camera=(self)",
"the panel's People card: the owner adds an operator or viewer by @username (OpenVibe.Network resolves the name to a subject) and removes them again, shown with Network's names; a signed-in visitor on an open-queue robot sees their turn or their place in line and can leave the queue, promoting the next person (POST /robots/:id/operators and …/operators/:subject/remove, POST /robots/:id/queue/leave)",
"the panel's People card: the owner adds an operator or viewer by @username (OpenVibe.Network resolves the name to a subject) and removes them again, shown with Network's names; a signed-in visitor on an open-queue robot sees their turn or their place in line and can leave the queue, promoting the next person (POST /robots/:id/operators and …/operators/:subject/remove, POST /robots/:id/queue/leave)",
"panel input: a touch joystick, the keyboard and a gamepad (Gamepad API), each held control re-sent while held and zeroed at once on release; a latency meter (round-trip median and telemetry age); one camera tile per camera the profile lists, playing the robot's live OpenRestream WebRTC session (the Adeept and Cozmo profiles carry a camera widget since 2026-10-13); a readout shows the sensor keys the Node's drivers report (distance_cm, line, cliff and picked_up); a live online indicator on /pair/:id read from /control",
"panel input: a touch joystick, the keyboard and a gamepad (Gamepad API), each held control re-sent while held and zeroed at once on release; a latency meter (round-trip median and telemetry age); one camera tile per camera the profile lists, playing the robot's live OpenRestream WebRTC session (the Adeept and Cozmo profiles carry a camera widget since 2026-10-13); a readout shows the sensor keys the Node's drivers report (distance_cm, line, cliff and picked_up); a live online indicator on /pair/:id read from /control",
"platform.job@1 jobs dispatched to a paired Node over /device (platform.job-frame@1: resent unacked on reconnect, only to an advertised runtime class) and metered per second into run_usage_outbox for billing.usage.record",
"platform.job@1 jobs dispatched to a paired Node over /device (platform.job-frame@1: resent unacked on reconnect, only to an advertised runtime class) and metered per second into run_usage_outbox for billing.usage.record",
@@ -13,10 +13,11 @@
@@ -13,10 +13,11 @@
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3",
"pg": "^8.23.0",
"pg": "^8.23.0",
+ "qrcode-generator": "^2.0.4",
"ws": "^8.18.0"
"ws": "^8.18.0"
},
},
"devDependencies": {
"devDependencies": {
@@ -844,9 +845,9 @@
@@ -844,9 +845,9 @@
}
}
},
},
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
- "version": "0.143.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",
- "integrity": "sha512-Cdbf+9RoPAUBibpM2zXQvtsiKMugb55dKubTt8pFZvAmxsLtGvX+zaC++yL1vOSub8FX3+GgqcGtoJS6tklrUg==",
+ "version": "0.157.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0",
+ "integrity": "sha512-GhcDBEdSZOxuKjO3ivN7lZlm685ULRABNkOIxSpJIRwv9qTuATcJe/PwjHfUlcr+VmZeJkd2eJZJl07yIuomdw==",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
"ajv": "^8.17.1",
"ajv": "^8.17.1",
@@ -1056,6 +1057,12 @@
@@ -1056,6 +1057,12 @@
"url": "https://opencollective.com/express"
"url": "https://opencollective.com/express"
}
}
},
},
+ "node_modules/qrcode-generator": {
+ "version": "2.0.4",
+ "resolved": "https://registry.npmjs.org/qrcode-generator/-/qrcode-generator-2.0.4.tgz",
+ "integrity": "sha512-mZSiP6RnbHl4xL2Ap5HfkjLnmxfKcPWpWe/c+5XxCuetEenqmNFf1FH/ftXPCtFG5/TDobjsjz6sSNL0Sr8Z9g==",
+ "license": "MIT"
+ },
"node_modules/qs": {
"node_modules/qs": {
"version": "6.16.0",
"version": "6.16.0",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",
"resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz",
@@ -25,10 +25,11 @@
@@ -25,10 +25,11 @@
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.143.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.157.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.44.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.1.3",
"pg": "^8.23.0",
"pg": "^8.23.0",
+ "qrcode-generator": "^2.0.4",
"ws": "^8.18.0"
"ws": "^8.18.0"
},
},
"engines": {
"engines": {
@@ -40,7 +40,7 @@
@@ -40,7 +40,7 @@
.dir-chips li { font-size: 12.5px; padding: 2px 9px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); color: var(--text-secondary, #96a7c2); margin: 0; }
.dir-chips li { font-size: 12.5px; padding: 2px 9px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); color: var(--text-secondary, #96a7c2); margin: 0; }
.dir-badge { flex: none; font-size: 11.5px; font-weight: 700; letter-spacing: .03em; padding: 2px 8px; border-radius: 999px; border: 1px solid currentColor; }
.dir-badge { flex: none; font-size: 11.5px; font-weight: 700; letter-spacing: .03em; padding: 2px 8px; border-radius: 999px; border: 1px solid currentColor; }
-.dir-browser { color: #a78bfa; } .dir-onboard { color: #22c55e; } .dir-bridge { color: #f59e0b; } .dir-server { color: #38bdf8; }
+.dir-browser { color: #a78bfa; } .dir-phone { color: #f472b6; } .dir-onboard { color: #22c55e; } .dir-bridge { color: #f59e0b; } .dir-server { color: #38bdf8; }
.dir-pill { display: inline-block; font-size: 12.5px; padding: 2px 10px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); text-decoration: none; color: var(--text-primary, #e6edf7) !important; }
.dir-pill { display: inline-block; font-size: 12.5px; padding: 2px 10px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); text-decoration: none; color: var(--text-primary, #e6edf7) !important; }
.dir-pill.dir-quiet { color: var(--text-secondary, #96a7c2) !important; }
.dir-pill.dir-quiet { color: var(--text-secondary, #96a7c2) !important; }
.dir-state { display: inline-flex; align-items: center; gap: 6px; font-size: 13px; font-weight: 650; color: var(--text-secondary, #96a7c2); }
.dir-state { display: inline-flex; align-items: center; gap: 6px; font-size: 13px; font-weight: 650; color: var(--text-secondary, #96a7c2); }
@@ -295,3 +295,40 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf
@@ -295,3 +295,40 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf
.try-next h2 { margin: 0 0 0.35rem; font-size: 1.1rem; }
.try-next h2 { margin: 0 0 0.35rem; font-size: 1.1rem; }
.try-next p { color: var(--muted); line-height: 1.55; }
.try-next p { color: var(--muted); line-height: 1.55; }
.try-actions { display: flex; flex-wrap: wrap; gap: 0.5rem; margin-bottom: 0; }
.try-actions { display: flex; flex-wrap: wrap; gap: 0.5rem; margin-bottom: 0; }
+
+/* The pairing page's QR code for a phone (server/web/qr.js): white on its own quiet zone, sized for a phone camera. */
+.qr-box { width: min(15rem, 70vw); margin: 0.5rem 0 0.75rem; padding: 0.5rem; background: #fff; border-radius: 12px; }
+.qr-box .qr { display: block; width: 100%; height: auto; }
+
+/* /device/phone (public/phone-device.js): setup, then the phone is the robot's face. */
+.phone-page { min-height: 100dvh; }
+.phone { min-height: 100dvh; display: grid; }
+.phone-setup { max-width: 30rem; margin: 0 auto; padding: 1.5rem 1rem 2rem; align-self: center; }
+.phone-brand { display: inline-flex; align-items: center; gap: 0.4rem; margin: 0 0 1rem; font-weight: 700; }
+.phone-brand b { color: var(--accent); }
+.phone-setup h1 { margin: 0 0 0.5rem; font-size: 1.6rem; }
+.phone-setup .muted { color: var(--muted); line-height: 1.55; }
+.phone-setup .small { font-size: 0.85rem; }
+.phone-form { display: grid; gap: 0.85rem; margin: 1.25rem 0; }
+.phone-form .field { display: grid; gap: 0.35rem; }
+.phone-form .field-label { font-size: 0.8rem; font-weight: 700; letter-spacing: 0.05em; text-transform: uppercase; color: var(--muted); }
+.phone-form input:not([type="checkbox"]) { padding: 0 0.75rem; border: 1px solid var(--line); background: var(--bg); color: var(--text); font-size: 1.05rem; }
+.phone-form .check { display: flex; align-items: center; gap: 0.5rem; color: var(--muted); }
+.phone-form button.primary { font-size: 1.1rem; min-height: 52px; }
+.phone-paired { margin: 0; }
+.phone-run { display: grid; grid-template-rows: auto 1fr auto auto; min-height: 100dvh; }
+.phone-bar { display: flex; flex-wrap: wrap; gap: 0.4rem; align-items: center; padding: 0.6rem 0.75rem; padding-top: max(0.6rem, env(safe-area-inset-top)); }
+.phone-pill { font-size: 0.8rem; padding: 0.15rem 0.6rem; border-radius: 999px; border: 1px solid var(--line); color: var(--muted); }
+.phone-pill:empty { display: none; }
+.phone-pill[data-state="online"] { color: var(--ok); border-color: color-mix(in srgb, var(--ok) 45%, transparent); }
+.phone-pill[data-state="connecting"] { color: var(--warn); }
+.phone-rtt { margin-left: auto; font: 0.8rem ui-monospace, monospace; color: var(--muted); }
+.phone-face { display: grid; place-items: center; color: #e6edf3; padding: 1rem; }
+.phone-face svg { width: min(92vw, 70vh); height: auto; }
+.phone-text { margin: 0; max-width: 92vw; text-align: center; font-weight: 700; line-height: 1.15; overflow-wrap: anywhere; }
+.phone[data-stopped="true"] .phone-face { color: var(--warn); }
+.phone-safety { margin: 0 0.75rem; padding: 0.6rem 0.75rem; border-radius: 10px; background: color-mix(in srgb, var(--warn) 14%, transparent); color: var(--text); }
+.phone-preview { position: fixed; right: 0.75rem; top: 3rem; width: 26vw; max-width: 9rem; aspect-ratio: 9 / 16; object-fit: cover; border-radius: 10px; border: 1px solid var(--line); background: #000; opacity: 0.85; }
+.phone-foot { display: flex; gap: 0.5rem; padding: 0.75rem; padding-bottom: max(0.75rem, env(safe-area-inset-bottom)); }
+.phone-stop { flex: 1; min-height: 56px; font-size: 1.15rem; font-weight: 800; letter-spacing: 0.04em; text-transform: uppercase; background: var(--danger); border-color: var(--danger); color: #fff; }
+.phone-stop.resume { background: var(--ok); border-color: var(--ok); color: #04130a; }
new file mode 100644
new file mode 100644
@@ -0,0 +1,404 @@
@@ -0,0 +1,404 @@
+'use strict';
+
+/**
+ * /device/phone: this phone is the robot's device, from its browser, with nothing to install (plan T15 "an Android
+ * phone as the robot's brain", profile driver `phone`). Plain ES, no build step, no framework.
+ *
+ * pair POST /api/v1/pair with the one-time code (from the pairing page's QR code, or typed) → the device id,
+ * its credential and its WHIP URL, kept in this browser's localStorage (shown once by Bot, so kept here)
+ * link wss://<host>/device, the credential offered as the subprotocol `ov.device.<credential>` (a browser
+ * cannot set the Authorization header); hello → config → status, then a heartbeat every
+ * config.heartbeat_ms and telemetry (battery, tilt) at most once a second
+ * camera getUserMedia → WHIP (RFC 9725) to OpenRestream at the whip_url Bot gave at pairing; the robot's panel
+ * plays it. Restarted with backoff when it drops
+ * commands say (speechSynthesis), display (a face or text, full screen), actuator torch (the camera's torch) and
+ * chime (a tone while it is held: its deadline ends it, as a motor's does), halt; each acked or nacked
+ * safety Bot's e-stop silences everything and refuses all but halt; the phone's own Stop button latches a local
+ * stop (estop_state local_stop) that only this phone's Resume releases
+ *
+ * The page needs one tap to start (browsers allow sound, the camera and the wake lock only after a gesture).
+ */
+(() => {
+ const main = document.getElementById('phone');
+ if (!main) return;
+ const $ = (sel) => main.querySelector(sel);
+ const KEY = 'ovbot.phone.v1';
+ const AGENT = 'openvibe-phone/1';
+ const load = () => { try { return JSON.parse(localStorage.getItem(KEY) || 'null'); } catch { return null; } };
+ const save = (d) => { try { localStorage.setItem(KEY, JSON.stringify(d)); } catch { /* private mode: this visit only */ } };
+ const forget = () => { try { localStorage.removeItem(KEY); } catch { /* nothing kept */ } };
+
+ // ── Setup: the QR code's fragment (#robot=…&code=…) fills the form, then leaves the address bar ──────────
+ const form = $('[data-pair-form]');
+ const errorEl = $('[data-error]');
+ const fragment = new URLSearchParams(location.hash.slice(1));
+ if (fragment.get('robot')) form.elements.robot.value = fragment.get('robot');
+ if (fragment.get('code')) form.elements.code.value = fragment.get('code');
+ if (location.hash) history.replaceState(null, '', location.pathname);
+ if (form.elements.robot.value) $('[data-robot-field]').hidden = true;
+
+ let dev = load();
+ function showError(text) { errorEl.textContent = text || ''; errorEl.hidden = !text; }
+ function showSetup() {
+ main.dataset.state = 'setup';
+ $('[data-setup]').hidden = false; $('[data-run]').hidden = true;
+ $('[data-paired]').hidden = !dev; $('[data-pair-fields]').hidden = !!dev;
+ if (dev) $('[data-paired-name]').textContent = dev.robot_name || dev.robot_id;
+ }
+ showSetup();
+
+ const deviceName = () => {
+ const ua = navigator.userAgent;
+ const model = (/Android [^;]+; ([^;)]+)/.exec(ua) || [])[1] || (/iPhone/.test(ua) ? 'iPhone' : /iPad/.test(ua) ? 'iPad' : 'Phone');
+ return `${model.trim()} (browser)`.slice(0, 80);
+ };
+
+ async function pair(robot, code) {
+ const r = await fetch('/api/v1/pair', {
+ method: 'POST', headers: { 'Content-Type': 'application/json' },
+ body: JSON.stringify({
+ robot, code, agent_version: AGENT, device_kind: 'onboard', drivers: ['phone'], name: deviceName(),
+ capabilities: { camera: !!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia), speech: 'speechSynthesis' in window, tilt: 'DeviceOrientationEvent' in window },
+ }),
+ });
+ let j = {};
+ try { j = await r.json(); } catch { /* no body */ }
+ if (!r.ok) throw new Error(j.detail || j.title || `pairing failed (${r.status})`);
+ return { device_id: j.device_id, credential: j.credential, whip_url: j.whip_url || null, robot_id: j.robot_id, robot_name: null, profile_id: j.profile ? j.profile.id : null };
+ }
+
+ form.addEventListener('submit', async (e) => {
+ e.preventDefault();
+ showError('');
+ const button = form.querySelector('button[type="submit"]');
+ try {
+ if (!dev) {
+ const robot = form.elements.robot.value.trim();
+ const code = form.elements.code.value.trim();
+ if (!robot || !code) return showError('Enter the robot and the pairing code from its pairing page.');
+ button.disabled = true;
+ dev = await pair(robot, code);
+ save(dev);
+ }
+ front = !!form.elements.front.checked;
+ await start();
+ } catch (err) {
+ showError(err.message || String(err));
+ } finally { button.disabled = false; }
+ });
+ $('[data-forget]').addEventListener('click', () => {
+ if (!confirm('Unpair this phone? The robot keeps its other devices; pairing again needs a new code.')) return;
+ stopAll(); forget(); dev = null; showSetup();
+ });
+ $('[data-unpair-setup]').addEventListener('click', () => { forget(); dev = null; showSetup(); });
+
+ // ── Running ───────────────────────────────────────────────────────────────────────────────────────────
+ let ws = null; let seq = 0; let retry = 0; let running = false;
+ let heartbeatMs = 1000; let heartbeatTimer = null; let telemetryTimer = null;
+ let rtt = null; let remoteLatched = false; let localStop = false;
+ let front = false;
+ const linkEl = $('[data-link-state]');
+ const videoEl = $('[data-video-state]');
+ const stopBtn = $('[data-local-stop]');
+
+ function out(type, fields = {}) {
+ if (!ws || ws.readyState !== 1) return false;
+ ws.send(JSON.stringify({ v: 1, seq: ++seq, ts: Date.now(), type, ...fields }));
+ return true;
+ }
+ function setLink(text, state) { linkEl.textContent = text; linkEl.dataset.state = state; }
+ function setVideo(text, state) { videoEl.textContent = text; videoEl.dataset.state = state; }
+
+ async function start() {
+ running = true;
+ main.dataset.state = 'run';
+ $('[data-setup]').hidden = true; $('[data-run]').hidden = false;
+ unlockAudio();
+ askTilt();
+ keepAwake();
+ battery();
+ showFace('neutral');
+ connect();
+ startVideo();
+ }
+ function stopAll() {
+ running = false;
+ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);
+ if (ws) { ws.onclose = null; try { ws.close(); } catch { /* gone */ } ws = null; }
+ stopVideo();
+ stopEffects();
+ }
+
+ function connect() {
+ if (!running || !dev) return;
+ setLink('Connecting…', 'connecting');
+ ws = new WebSocket(`${location.protocol === 'https:' ? 'wss' : 'ws'}://${location.host}/device`, ['ov.device', `ov.device.${dev.credential}`]);
+ ws.onmessage = (e) => { let m; try { m = JSON.parse(e.data); } catch { return; } onFrame(m); };
+ ws.onclose = (e) => {
+ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);
+ stopEffects();
+ if (!running) return;
+ if (e.code === 4002 || e.code === 4003) {
+ // The credential was revoked or rotated, or the robot was removed: this phone must pair again.
+ running = false; stopVideo(); forget(); dev = null; showSetup();
+ return showError('This phone is no longer paired with the robot. Ask its owner for a new pairing code.');
+ }
+ setLink('Reconnecting…', 'connecting');
+ setTimeout(connect, Math.min(15000, 500 * 2 ** retry++) + Math.random() * 400);
+ };
+ }
+
+ function onFrame(m) {
+ if (m.type === 'hello') { retry = 0; return; }
+ if (m.type === 'config') {
+ heartbeatMs = Math.max(250, Math.min(5000, Number(m.heartbeat_ms) || 1000));
+ remoteLatched = !!m.estop_latched;
+ clearInterval(heartbeatTimer); clearInterval(telemetryTimer);
+ heartbeatTimer = setInterval(() => out('heartbeat', { t: Date.now(), rtt_ms: rtt }), heartbeatMs);
+ telemetryTimer = setInterval(sendTelemetry, 1000);
+ out('status', { firmware: AGENT, capabilities: { torch: torchSupported(), speech: 'speechSynthesis' in window }, faults: faults(), estop_latched: remoteLatched || localStop });
+ if (localStop) out('estop_state', { latched: true, local_stop: true, reason: 'stopped on the phone' });
+ paintSafety();
+ setLink('Connected', 'online');
+ return;
+ }
+ if (m.type === 'heartbeat_ack') { if (Number.isFinite(m.echo)) rtt = Math.max(0, Date.now() - m.echo); return; }
+ if (m.type === 'estop') { remoteLatched = !!m.latched; if (remoteLatched) stopEffects(); paintSafety(); return; }
+ if (m.type === 'command') return onCommand(m);
+ if (m.type === 'rotate') return undefined; // a Network-paired machine's frame; a phone holds a Bot credential
+ return undefined;
+ }
+
+ function onCommand(m) {
+ const ack = () => out('ack', { id: m.id });
+ const nack = (code) => out('nack', { id: m.id, fault_code: code });
+ const v = m.value && typeof m.value === 'object' ? m.value : {};
+ if (m.kind === 'halt') { stopEffects(); return ack(); }
+ if (localStop) return nack('local_stop');
+ if (remoteLatched) return nack('estopped');
+ if (m.kind === 'say') return speak(String(v.text || '')) ? ack() : nack('unsupported');
+ if (m.kind === 'display') {
+ if (v.face) showFace(String(v.face)); else if (v.text) showText(String(v.text)); else return nack('invalid');
+ return ack();
+ }
+ if (m.kind === 'actuator' && v.name === 'torch') {
+ setTorch(!!v.value).then((ok) => (ok ? ack() : nack('unsupported')));
+ return undefined;
+ }
+ if (m.kind === 'actuator' && v.name === 'chime') {
+ tone(v.value && Number.isFinite(v.value.hz) ? v.value.hz : null, m.deadline_ms);
+ return ack();
+ }
+ return nack('unsupported');
+ }
+
+ // ── The phone's own Stop ──────────────────────────────────────────────────────────────────────────────
+ stopBtn.addEventListener('click', () => {
+ localStop = !localStop;
+ if (localStop) { stopEffects(); out('estop_state', { latched: true, local_stop: true, reason: 'stopped on the phone' }); }
+ else out('estop_state', { latched: false });
+ paintSafety();
+ });
+ function paintSafety() {
+ main.dataset.stopped = String(localStop || remoteLatched);
+ stopBtn.textContent = localStop ? 'Resume' : 'Stop';
+ stopBtn.classList.toggle('resume', localStop);
+ const note = $('[data-safety]');
+ note.textContent = localStop ? 'Stopped on this phone: press Resume, then the owner clears the e-stop.' : remoteLatched ? 'The robot\'s e-stop is latched: the owner clears it on the panel.' : '';
+ note.hidden = !note.textContent;
+ }
+
+ // ── Telemetry ─────────────────────────────────────────────────────────────────────────────────────────
+ let tilt = null; let batt = null;
+ function askTilt() {
+ const listen = () => window.addEventListener('deviceorientation', (e) => {
+ if (e.beta == null && e.gamma == null) return;
+ tilt = { pitch: Math.round(e.beta || 0), roll: Math.round(e.gamma || 0) };
+ });
+ // iOS asks once, from a tap.
+ if (window.DeviceOrientationEvent && typeof DeviceOrientationEvent.requestPermission === 'function') {
+ DeviceOrientationEvent.requestPermission().then((s) => { if (s === 'granted') listen(); }, () => {});
+ } else listen();
+ }
+ function battery() {
+ if (!navigator.getBattery) return;
+ navigator.getBattery().then((b) => {
+ const read = () => { batt = { percent: Math.round(b.level * 100), charging: !!b.charging }; };
+ read(); b.addEventListener('levelchange', read); b.addEventListener('chargingchange', read);
+ }, () => {});
+ }
+ function faults() {
+ const list = [];
+ if (videoFault) list.push({ code: 'camera', driver: 'phone', message: videoFault });
+ return list;
+ }
+ function sendTelemetry() {
+ const frame = { sensors: tilt ? { tilt } : {} };
+ if (batt) frame.battery = { percent: batt.percent };
+ out('telemetry', frame);
+ $('[data-rtt]').textContent = rtt == null ? '' : `${rtt} ms`;
+ }
+
+ // ── Camera over WHIP ──────────────────────────────────────────────────────────────────────────────────
+ const preview = $('[data-preview]');
+ let stream = null; let pc = null; let resource = null; let videoRetry = 0; let videoTimer = null; let videoFault = null;
+ async function startVideo() {
+ clearTimeout(videoTimer);
+ if (!running) return;
+ if (!dev.whip_url) { setVideo('No video on this server', 'off'); return; }
+ try {
+ setVideo('Starting the camera…', 'connecting');
+ if (!stream) {
+ stream = await navigator.mediaDevices.getUserMedia({
+ video: { facingMode: front ? 'user' : 'environment', width: { ideal: 1280 }, height: { ideal: 720 }, frameRate: { ideal: 30, max: 30 } },
+ audio: false,
+ });
+ preview.srcObject = stream;
+ preview.hidden = false;
+ }
+ pc = new RTCPeerConnection({ bundlePolicy: 'max-bundle' });
+ for (const track of stream.getVideoTracks()) pc.addTransceiver(track, { direction: 'sendonly', streams: [stream] });
+ await pc.setLocalDescription(await pc.createOffer());
+ await new Promise((done) => {
+ if (pc.iceGatheringState === 'complete') return done();
+ const t = setTimeout(done, 2500);
+ pc.addEventListener('icegatheringstatechange', () => { if (pc.iceGatheringState === 'complete') { clearTimeout(t); done(); } });
+ });
+ const r = await fetch(dev.whip_url, { method: 'POST', headers: { 'Content-Type': 'application/sdp' }, body: pc.localDescription.sdp });
+ if (r.status !== 201) throw new Error(`the video server answered ${r.status}${r.headers.get('X-WHIP-ERROR') ? ` (${r.headers.get('X-WHIP-ERROR')})` : ''}`);
+ resource = r.headers.get('Location') ? new URL(r.headers.get('Location'), dev.whip_url).href : null;
+ await pc.setRemoteDescription({ type: 'answer', sdp: await r.text() });
+ pc.addEventListener('connectionstatechange', () => {
+ if (pc && pc.connectionState === 'connected') { videoRetry = 0; videoFault = null; setVideo('Streaming', 'online'); }
+ if (pc && (pc.connectionState === 'failed' || pc.connectionState === 'disconnected')) restartVideo('the video connection dropped');
+ });
+ } catch (err) {
+ const denied = err && (err.name === 'NotAllowedError' || err.name === 'SecurityError');
+ videoFault = denied ? 'camera permission was refused' : String(err.message || err).slice(0, 160);
+ setVideo(denied ? 'Camera blocked: allow it in the browser\'s site settings' : 'Camera trouble, retrying…', 'off');
+ if (!denied) restartVideo(videoFault);
+ }
+ }
+ function restartVideo(why) {
+ videoFault = why;
+ closePeer();
+ setVideo('Reconnecting the camera…', 'connecting');
+ clearTimeout(videoTimer);
+ videoTimer = setTimeout(startVideo, Math.min(30000, 1000 * 2 ** videoRetry++));
+ }
+ function closePeer() {
+ if (resource) { fetch(resource, { method: 'DELETE', keepalive: true }).catch(() => {}); resource = null; }
+ if (pc) { try { pc.close(); } catch { /* gone */ } pc = null; }
+ }
+ function stopVideo() {
+ clearTimeout(videoTimer);
+ closePeer();
+ if (stream) { for (const t of stream.getTracks()) t.stop(); stream = null; }
+ preview.srcObject = null;
+ preview.hidden = true;
+ setVideo('', 'off');
+ }
+ const torchTrack = () => (stream ? stream.getVideoTracks()[0] : null);
+ function torchSupported() {
+ const t = torchTrack();
+ const caps = t && t.getCapabilities ? t.getCapabilities() : {};
+ return !!caps.torch;
+ }
+ async function setTorch(on) {
+ const t = torchTrack();
+ if (!t || !torchSupported()) return false;
+ try { await t.applyConstraints({ advanced: [{ torch: on }] }); return true; } catch { return false; }
+ }
+
+ // ── Voice, chime and face ─────────────────────────────────────────────────────────────────────────────
+ let audio = null; let osc = null; let gain = null; let toneTimer = null;
+ function unlockAudio() {
+ const Ctx = window.AudioContext || window.webkitAudioContext;
+ if (!audio && Ctx) { try { audio = new Ctx(); } catch { audio = null; } }
+ if (audio && audio.state === 'suspended') audio.resume().catch(() => {});
+ if ('speechSynthesis' in window) speechSynthesis.getVoices();
+ }
+ function speak(text) {
+ if (!('speechSynthesis' in window) || !text) return false;
+ speechSynthesis.cancel();
+ const u = new SpeechSynthesisUtterance(text.slice(0, 200));
+ speechSynthesis.speak(u);
+ showText(text, 4000);
+ return true;
+ }
+ function tone(hz, deadline) {
+ clearTimeout(toneTimer);
+ if (!audio || hz == null) { silence(); return; }
+ if (!osc) {
+ osc = audio.createOscillator(); gain = audio.createGain();
+ osc.type = 'triangle'; gain.gain.value = 0;
+ osc.connect(gain).connect(audio.destination); osc.start();
+ }
+ osc.frequency.setValueAtTime(hz, audio.currentTime);
+ gain.gain.setTargetAtTime(0.25, audio.currentTime, 0.01);
+ const until = Number.isFinite(deadline) ? Math.max(50, deadline - Date.now()) : 400;
+ toneTimer = setTimeout(silence, until); // the chime's deadman: held means re-sent
+ }
+ function silence() {
+ clearTimeout(toneTimer);
+ if (osc && audio) { gain.gain.setTargetAtTime(0, audio.currentTime, 0.02); const o = osc; osc = null; setTimeout(() => { try { o.stop(); } catch { /* stopped */ } }, 120); }
+ }
+ function stopEffects() {
+ silence();
+ if ('speechSynthesis' in window) speechSynthesis.cancel();
+ setTorch(false);
+ }
+
+ const face = $('[data-face]');
+ const SVG = 'http://www.w3.org/2000/svg';
+ const el = (name, attrs) => { const n = document.createElementNS(SVG, name); for (const [k, v] of Object.entries(attrs)) n.setAttribute(k, v); return n; };
+ // Each face: two eyes and a mouth on a 200×120 box.
+ const EYE = { cx: [62, 138], cy: 46 };
+ const FACES = {
+ neutral: { eyes: 'round', mouth: 'M70 92 L130 92' },
+ happy: { eyes: 'round', mouth: 'M64 84 Q100 116 136 84' },
+ sad: { eyes: 'round', mouth: 'M66 100 Q100 74 134 100' },
+ surprised: { eyes: 'wide', mouth: 'circle' },
+ sleepy: { eyes: 'closed', mouth: 'M84 94 L116 94' },
+ wink: { eyes: 'wink', mouth: 'M64 84 Q100 112 136 84' },
+ love: { eyes: 'heart', mouth: 'M64 84 Q100 116 136 84' },
+ angry: { eyes: 'angry', mouth: 'M66 100 Q100 80 134 100' },
+ };
+ let textTimer = null;
+ function showFace(name) {
+ clearTimeout(textTimer);
+ const f = FACES[name] || FACES.neutral;
+ const svg = el('svg', { viewBox: '0 0 200 120', role: 'img', 'aria-label': `${name} face` });
+ const stroke = { stroke: 'currentColor', 'stroke-width': 8, 'stroke-linecap': 'round', fill: 'none' };
+ EYE.cx.forEach((cx, i) => {
+ if (f.eyes === 'closed' || (f.eyes === 'wink' && i === 1)) svg.append(el('path', { d: `M${cx - 16} ${EYE.cy} Q${cx} ${EYE.cy + 12} ${cx + 16} ${EYE.cy}`, ...stroke }));
+ else if (f.eyes === 'heart') svg.append(el('path', { d: `M${cx} ${EYE.cy + 14} L${cx - 16} ${EYE.cy - 2} A8 8 0 0 1 ${cx} ${EYE.cy - 10} A8 8 0 0 1 ${cx + 16} ${EYE.cy - 2} Z`, fill: '#ff5c8a' }));
+ else {
+ svg.append(el('circle', { cx, cy: EYE.cy, r: f.eyes === 'wide' ? 17 : 13, fill: 'currentColor' }));
+ if (f.eyes === 'angry') svg.append(el('path', { d: i === 0 ? `M${cx - 20} ${EYE.cy - 26} L${cx + 14} ${EYE.cy - 14}` : `M${cx + 20} ${EYE.cy - 26} L${cx - 14} ${EYE.cy - 14}`, ...stroke }));
+ }
+ });
+ svg.append(f.mouth === 'circle' ? el('circle', { cx: 100, cy: 94, r: 11, ...stroke }) : el('path', { d: f.mouth, ...stroke }));
+ face.replaceChildren(svg);
+ face.dataset.mode = 'face';
+ }
+ function showText(text, ms = 0) {
+ clearTimeout(textTimer);
+ const p = document.createElement('p');
+ p.className = 'phone-text';
+ p.textContent = text.slice(0, 200);
+ p.style.fontSize = `${Math.max(5, Math.min(18, 160 / Math.max(6, text.length)))}vmin`;
+ face.replaceChildren(p);
+ face.dataset.mode = 'text';
+ if (ms) textTimer = setTimeout(() => showFace('neutral'), ms);
+ }
+
+ // ── Keep the screen on while running ──────────────────────────────────────────────────────────────────
+ let wake = null;
+ async function keepAwake() {
+ if (!('wakeLock' in navigator) || !running) return;
+ try { wake = await navigator.wakeLock.request('screen'); } catch { wake = null; }
+ }
+ document.addEventListener('visibilitychange', () => { if (!document.hidden && running) keepAwake(); });
+ window.addEventListener('pagehide', closePeer); // ends the WHIP session (keepalive) so the panel's video stops at once
+})();
@@ -34,10 +34,10 @@ const DIR = path.join(__dirname);
@@ -34,10 +34,10 @@ const DIR = path.join(__dirname);
const CAPABILITIES = new Set([
const CAPABILITIES = new Set([
'drive.differential', 'drive.mecanum',
'drive.differential', 'drive.mecanum',
'servo.pan_tilt', 'head', 'lift',
'servo.pan_tilt', 'head', 'lift',
- 'lights.rgb', 'lights.backpack', 'lights.cube',
+ 'lights.rgb', 'lights.backpack', 'lights.cube', 'lights.torch',
'speaker.horn', 'speaker.say',
'speaker.horn', 'speaker.say',
'display.text',
'display.text',
- 'sensor.ultrasonic', 'sensor.line', 'sensor.cliff', 'sensor.pickup',
+ 'sensor.ultrasonic', 'sensor.line', 'sensor.cliff', 'sensor.pickup', 'sensor.tilt',
'battery', 'ptz', 'camera',
'battery', 'ptz', 'camera',
// A robot that carries no hardware of its own: the owner's Node forwards the panel's commands to a
// A robot that carries no hardware of its own: the owner's Node forwards the panel's commands to a
// local script with the relay plugin (relay.generic, plan T15 R9).
// local script with the relay plugin (relay.generic, plan T15 R9).
@@ -49,7 +49,8 @@ const WIDGETS = new Set(['drive', 'pan-tilt', 'servo', 'lights', 'horn', 'speake
@@ -49,7 +49,8 @@ const WIDGETS = new Set(['drive', 'pan-tilt', 'servo', 'lights', 'horn', 'speake
/** Device drivers a mapping may name. `relay` is a Node plugin with no hardware map of its own. */
/** Device drivers a mapping may name. `relay` is a Node plugin with no hardware map of its own. */
// `looi`: the community's Generic LOOI driver, a Bluetooth bridge plugin (OpenVibe.Node docs/hardware-looi.md).
// `looi`: the community's Generic LOOI driver, a Bluetooth bridge plugin (OpenVibe.Node docs/hardware-looi.md).
-const DRIVERS = new Set(['pca9685', 'ads7830', 'cozmo', 'looi', 'onvif', 'sim', 'relay']);
+// `phone`: a phone that is the device itself, from its browser (public/phone-device.js), no Node and nothing installed.
+const DRIVERS = new Set(['pca9685', 'ads7830', 'cozmo', 'looi', 'onvif', 'sim', 'relay', 'phone']);
const ID_RE = /^[a-z][a-z0-9._-]{1,63}$/;
const ID_RE = /^[a-z][a-z0-9._-]{1,63}$/;
const KINDS = new Set(['onboard', 'bridge', 'server']);
const KINDS = new Set(['onboard', 'bridge', 'server']);
new file mode 100644
new file mode 100644
@@ -0,0 +1,27 @@
@@ -0,0 +1,27 @@
+{
+ "id": "phone.camera",
+ "version": 1,
+ "name": "A phone",
+ "vendor": "Any",
+ "kind": "onboard",
+ "description": "Any phone as a robot's eyes, voice and face, from its browser with nothing to install: its camera streams to the panel, it says what you type, shows a face or text full-screen, turns its torch on, chimes, and reports its battery and how it is tilted.",
+ "capabilities": ["camera", "speaker.say", "speaker.horn", "display.text", "lights.torch", "battery", "sensor.tilt"],
+ "mapping": { "driver": "phone" },
+ "commands": {
+ "say": { "max_chars": 200 },
+ "display": { "modes": ["face", "text"], "faces": ["happy", "neutral", "sad", "surprised", "sleepy", "wink", "love", "angry"], "max_chars": 80 },
+ "actuator": { "names": { "torch": { "type": "bool" }, "chime": { "type": "tone", "hz": [220, 1760] } } }
+ },
+ "widgets": [
+ { "type": "camera", "capability": "camera", "label": "Camera", "order": 1 },
+ { "type": "speaker", "capability": "speaker.say", "label": "Say", "order": 2, "command": { "kind": "say" } },
+ { "type": "display", "capability": "display.text", "label": "Face", "order": 3, "command": { "kind": "display" } },
+ { "type": "lights", "capability": "lights.torch", "label": "Torch", "order": 4, "command": { "kind": "actuator", "names": ["torch"] } },
+ { "type": "horn", "capability": "speaker.horn", "label": "Chime", "order": 5, "command": { "kind": "actuator", "names": ["chime"] } },
+ { "type": "telemetry", "capability": "sensor.tilt", "label": "Tilt", "order": 6 },
+ { "type": "battery", "capability": "battery", "label": "Battery", "order": 7 },
+ { "type": "latency", "label": "Latency", "order": 8 }
+ ],
+ "camera": { "transport": "whip", "resolution": "1280x720" },
+ "limits": { "max_speed": 1, "max_turn": 1, "max_command_ms": 300, "heartbeat_ms": 1000 }
+}
@@ -45,6 +45,16 @@ function parseCookies(header) {
@@ -45,6 +45,16 @@ function parseCookies(header) {
}
}
const bearer = (req) => { const h = String(req.headers.authorization || ''); return h.startsWith('Bearer ') ? h.slice(7).trim() : null; };
const bearer = (req) => { const h = String(req.headers.authorization || ''); return h.startsWith('Bearer ') ? h.slice(7).trim() : null; };
const isSubject = (v) => typeof v === 'string' && ids.isSubjectId('user', v);
const isSubject = (v) => typeof v === 'string' && ids.isSubjectId('user', v);
+const DEVICE_PROTOCOL = 'ov.device';
+const CREDENTIAL_RE = /^[A-Za-z0-9_-]{43}$/; // a device credential (32 random bytes, base64url); never a node token
+/** A browser device's credential, offered as `ov.device.<credential>` beside `ov.device`; anything else → null. */
+function deviceCredentialFromProtocols(header) {
+ const offered = String(header || '').split(',').map((s) => s.trim()).filter(Boolean);
+ if (!offered.includes(DEVICE_PROTOCOL)) return null;
+ const named = offered.find((p) => p.startsWith(`${DEVICE_PROTOCOL}.`));
+ const value = named ? named.slice(DEVICE_PROTOCOL.length + 1) : '';
+ return CREDENTIAL_RE.test(value) ? value : null;
+}
// The keys the Node's drivers report a sensor capability under (the Adeept and Cozmo plugins: distance_cm, line).
// The keys the Node's drivers report a sensor capability under (the Adeept and Cozmo plugins: distance_cm, line).
const SENSOR_KEYS = { ultrasonic: ['distance_cm'], line: ['line'], cliff: ['cliff', 'picked_up'], pickup: ['picked_up', 'pickup'] };
const SENSOR_KEYS = { ultrasonic: ['distance_cm'], line: ['line'], cliff: ['cliff', 'picked_up'], pickup: ['picked_up', 'pickup'] };
const finite = (v) => (typeof v === 'number' && Number.isFinite(v) ? v : 0);
const finite = (v) => (typeof v === 'number' && Number.isFinite(v) ? v : 0);
@@ -71,7 +81,10 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Dat
@@ -71,7 +81,10 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Dat
const timers = [];
const timers = [];
let closed = false;
let closed = false;
- const deviceWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false });
+ // A browser device (a phone, public/phone-device.js) cannot set the upgrade's Authorization header: it offers its
+ // credential as a subprotocol (`ov.device`, `ov.device.<credential>`) and gets `ov.device` back.
+ const deviceWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false,
+ handleProtocols: (protocols) => (protocols.has(DEVICE_PROTOCOL) ? DEVICE_PROTOCOL : false) });
// A socket that offers a control ticket gets the ticket protocol back (a browser drops a socket whose offered
// A socket that offers a control ticket gets the ticket protocol back (a browser drops a socket whose offered
// subprotocol is not answered); a plain one, none.
// subprotocol is not answered); a plain one, none.
const controlWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false,
const controlWss = new WebSocketServer({ noServer: true, maxPayload: 64 * 1024, perMessageDeflate: false,
@@ -148,7 +161,7 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Dat
@@ -148,7 +161,7 @@ function createRealtime({ config, keys, userAuth, log = console, now = () => Dat
ws.on('message', (raw) => onDeviceFrame(conn, raw));
ws.on('message', (raw) => onDeviceFrame(conn, raw));
ws.on('close', () => { conn.backlog = null; dropDevice(conn); });
ws.on('close', () => { conn.backlog = null; dropDevice(conn); });
ws.on('error', () => { /* the close event does the work */ });
ws.on('error', () => { /* the close event does the work */ });
- const cred = bearer(req); // the query string is never read
+ const cred = bearer(req) || deviceCredentialFromProtocols(req.headers['sec-websocket-protocol']); // the query string is never read
if (cred) {
if (cred) {
conn.backlog = [];
conn.backlog = [];
try {
try {
@@ -25,6 +25,7 @@ const CONNECT = {
@@ -25,6 +25,7 @@ const CONNECT = {
onboard: { label: 'On-board', text: 'The OpenVibe Node runs on the robot\'s own computer and drives its hardware.' },
onboard: { label: 'On-board', text: 'The OpenVibe Node runs on the robot\'s own computer and drives its hardware.' },
bridge: { label: 'Bridge', text: 'The OpenVibe Node runs on a computer or phone next to the robot and joins the robot\'s own link. Nothing on the robot changes.' },
bridge: { label: 'Bridge', text: 'The OpenVibe Node runs on a computer or phone next to the robot and joins the robot\'s own link. Nothing on the robot changes.' },
server: { label: 'Server-side', text: 'Nothing to install: Bot connects to the device itself.' },
server: { label: 'Server-side', text: 'Nothing to install: Bot connects to the device itself.' },
+ phone: { label: 'Phone', text: 'The phone\'s browser is the device: scan a code, allow the camera, keep the page open. Nothing to install.' },
};
};
const MODELS = {
const MODELS = {
'sim.rover': {
'sim.rover': {
@@ -54,6 +55,12 @@ const MODELS = {
@@ -54,6 +55,12 @@ const MODELS = {
setup: ['The community\'s Generic LOOI driver bridges to the desk robot over Bluetooth from a Windows PC.', 'Install the Node with --driver looi; the phone in LOOI keeps doing what it does.'],
setup: ['The community\'s Generic LOOI driver bridges to the desk robot over Bluetooth from a Windows PC.', 'Install the Node with --driver looi; the phone in LOOI keeps doing what it does.'],
guide: { href: `${NODE_BLOB}/docs/hardware-looi.md`, label: 'LOOI driver contract' },
guide: { href: `${NODE_BLOB}/docs/hardware-looi.md`, label: 'LOOI driver contract' },
},
},
+ 'phone.camera': {
+ connect: 'phone', order: 1,
+ setup: ['Add the robot here and choose A phone; its pairing page shows a QR code.', 'Scan it with the phone, tap Start and allow the camera. The phone streams, speaks, shows a face and reports its battery and tilt.',
+ 'Keep the page open (the screen stays on). Its Stop button holds the robot until it is released on the phone.'],
+ guide: { href: '/device/phone', label: 'The phone\'s page' },
+ },
'camera.onvif': {
'camera.onvif': {
connect: 'server', order: 5,
connect: 'server', order: 5,
setup: ['Any ONVIF pan-tilt-zoom camera: Bot moves it, with the same gate and limits as a robot.', 'Today an operator names the camera in Bot\'s configuration (its address and the names of its secrets).'],
setup: ['Any ONVIF pan-tilt-zoom camera: Bot moves it, with the same gate and limits as a robot.', 'Today an operator names the camera in Bot\'s configuration (its address and the names of its secrets).'],
@@ -70,6 +77,7 @@ const CAPABILITY_WORDS = {
@@ -70,6 +77,7 @@ const CAPABILITY_WORDS = {
'lights.rgb': 'RGB lights', 'lights.backpack': 'Backpack lights', 'lights.cube': 'Cube lights', 'speaker.horn': 'Horn', 'speaker.say': 'Speaks',
'lights.rgb': 'RGB lights', 'lights.backpack': 'Backpack lights', 'lights.cube': 'Cube lights', 'speaker.horn': 'Horn', 'speaker.say': 'Speaks',
'display.text': 'Face and text', 'sensor.ultrasonic': 'Range finder', 'sensor.line': 'Line tracking', 'sensor.cliff': 'Cliff sensors',
'display.text': 'Face and text', 'sensor.ultrasonic': 'Range finder', 'sensor.line': 'Line tracking', 'sensor.cliff': 'Cliff sensors',
'sensor.pickup': 'Pick-up sensor', battery: 'Battery', ptz: 'Pan, tilt, zoom', camera: 'Live camera', relay: 'Your own buttons',
'sensor.pickup': 'Pick-up sensor', battery: 'Battery', ptz: 'Pan, tilt, zoom', camera: 'Live camera', relay: 'Your own buttons',
+ 'lights.torch': 'Torch', 'sensor.tilt': 'Tilt sensor',
};
};
const POLICY_WORDS = { private: 'Only its owner and the people they add drive it.', invite: 'People the owner invites drive it.', queue: 'Anyone signed in can take a timed turn.' };
const POLICY_WORDS = { private: 'Only its owner and the people they add drive it.', invite: 'People the owner invites drive it.', queue: 'Anyone signed in can take a timed turn.' };
@@ -138,7 +146,7 @@ ${listed.length ? `<ul class="dir-grid">${listed.map(robotCard).join('')}</ul>`
@@ -138,7 +146,7 @@ ${listed.length ? `<ul class="dir-grid">${listed.map(robotCard).join('')}</ul>`
// ── GET /models ─────────────────────────────────────────────────────────────────────────────────────
// ── GET /models ─────────────────────────────────────────────────────────────────────────────────────
function renderModels({ config, models = [] }) {
function renderModels({ config, models = [] }) {
- const groups = ['browser', 'onboard', 'bridge', 'server'].map((k) => [k, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);
+ const groups = ['browser', 'phone', 'onboard', 'bridge', 'server'].map((k) => [k, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);
const body = `<header class="dir-hero"><h1>Models</h1>
const body = `<header class="dir-hero"><h1>Models</h1>
<p class="muted">Every robot OpenVibe.Bot has a profile for. A profile lays out the panel, so each of these drives with the controls it really has.</p></header>
<p class="muted">Every robot OpenVibe.Bot has a profile for. A profile lays out the panel, so each of these drives with the controls it really has.</p></header>
${groups.map(([k, ms]) => `<section class="dir-sec" aria-labelledby="g-${esc(k)}"><h2 id="g-${esc(k)}">${esc(CONNECT[k].label)}</h2><p class="muted">${esc(CONNECT[k].text)}</p>
${groups.map(([k, ms]) => `<section class="dir-sec" aria-labelledby="g-${esc(k)}"><h2 id="g-${esc(k)}">${esc(CONNECT[k].label)}</h2><p class="muted">${esc(CONNECT[k].text)}</p>
@@ -190,7 +198,7 @@ function renderAddRobot({ config, models = [], values = {}, error = null }) {
@@ -190,7 +198,7 @@ function renderAddRobot({ config, models = [], values = {}, error = null }) {
const chosen = values.profile_id || 'sim.rover';
const chosen = values.profile_id || 'sim.rover';
const option = (m) => `<label class="dir-choice"><input type="radio" name="profile_id" value="${esc(m.id)}"${m.id === chosen ? ' checked' : ''} required>`
const option = (m) => `<label class="dir-choice"><input type="radio" name="profile_id" value="${esc(m.id)}"${m.id === chosen ? ' checked' : ''} required>`
+ `<span class="dir-choice-body"><span class="dir-card-head"><b>${esc(m.name)}</b>${badge(m.connect)}</span><span class="dir-card-text">${esc(CONNECT[m.connect].text)}</span></span></label>`;
+ `<span class="dir-choice-body"><span class="dir-card-head"><b>${esc(m.name)}</b>${badge(m.connect)}</span><span class="dir-card-text">${esc(CONNECT[m.connect].text)}</span></span></label>`;
- const groups = [['browser', 'Try it with no hardware'], ['onboard', 'A robot with its own computer'], ['bridge', 'A robot you connect through a computer next to it'], ['server', 'A camera Bot connects to']]
+ const groups = [['browser', 'Try it with no hardware'], ['phone', 'A phone, from its browser'], ['onboard', 'A robot with its own computer'], ['bridge', 'A robot you connect through a computer next to it'], ['server', 'A camera Bot connects to']]
.map(([k, title]) => [title, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);
.map(([k, title]) => [title, sortModels(models).filter((m) => m.connect === k)]).filter(([, ms]) => ms.length);
const body = `<p class="dir-crumb">${link('/robots', 'Robots')}</p>
const body = `<p class="dir-crumb">${link('/robots', 'Robots')}</p>
<header class="dir-hero"><h1>Add a robot</h1><p class="muted">Name it, say what it is, and choose who may drive. A simulated rover drives the moment it is added; anything else gets a one-time pairing code and one command to paste on its computer.</p></header>
<header class="dir-hero"><h1>Add a robot</h1><p class="muted">Name it, say what it is, and choose who may drive. A simulated rover drives the moment it is added; anything else gets a one-time pairing code and one command to paste on its computer.</p></header>
new file mode 100644
new file mode 100644
@@ -0,0 +1,29 @@
@@ -0,0 +1,29 @@
+'use strict';
+
+/**
+ * A QR code as inline SVG markup (no script, no image request, so the strict CSP holds): one path of dark modules
+ * on a light quiet zone, scaling with the box it is put in. qrcode-generator (MIT) does the encoding.
+ *
+ * qrSvg(text, { label }) → '<svg …>' error correction M; `label` is the image's accessible name
+ */
+const qrcode = require('qrcode-generator');
+
+const ESC = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };
+const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => ESC[c]);
+
+function qrSvg(text, { label = 'QR code' } = {}) {
+ const qr = qrcode(0, 'M');
+ qr.addData(String(text));
+ qr.make();
+ const n = qr.getModuleCount();
+ const quiet = 4;
+ let d = '';
+ for (let r = 0; r < n; r++) {
+ for (let c = 0; c < n; c++) if (qr.isDark(r, c)) d += `M${c + quiet} ${r + quiet}h1v1h-1z`;
+ }
+ const size = n + quiet * 2;
+ return `<svg class="qr" viewBox="0 0 ${size} ${size}" role="img" aria-label="${esc(label)}" shape-rendering="crispEdges" xmlns="http://www.w3.org/2000/svg">`
+ + `<rect width="${size}" height="${size}" fill="#fff"/><path d="${d}" fill="#000"/></svg>`;
+}
+
+module.exports = { qrSvg };
@@ -18,6 +18,7 @@
@@ -18,6 +18,7 @@
* camerasOf(profile)
* camerasOf(profile)
* esc(value)
* esc(value)
*/
*/
+const { qrSvg } = require('./qr');
const ESC = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };
const ESC = { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' };
const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => ESC[c]);
const esc = (s) => String(s == null ? '' : s).replace(/[&<>"']/g, (c) => ESC[c]);
/** A data-* attribute's JSON, escaped for a double-quoted attribute. */
/** A data-* attribute's JSON, escaped for a double-quoted attribute. */
@@ -493,15 +494,23 @@ ${error ? `<p class="error" role="alert">${esc(error)}</p>` : ''}
@@ -493,15 +494,23 @@ ${error ? `<p class="error" role="alert">${esc(error)}</p>` : ''}
return page('Your robots', `${topbar()}\n<main class="page">\n<h1>Your robots</h1>\n${list}\n${form}\n${kitsSection(kits, profiles)}\n</main>`);
return page('Your robots', `${topbar()}\n<main class="page">\n<h1>Your robots</h1>\n${list}\n${form}\n${kitsSection(kits, profiles)}\n</main>`);
}
}
-function renderPairingPage({ robot, pairing, profile = null }) {
+function renderPairingPage({ robot, pairing, profile = null, phoneUrl = null }) {
+ // A phone is the device itself: it pairs from its browser (the QR code opens /device/phone with the code).
+ const phone = !!(profile && profile.mapping && profile.mapping.driver === 'phone' && phoneUrl);
+ const first = phone
+ ? `<li class="card phone-pair"><h2>Scan this with the phone <span class="pill">${esc(profile.name)}</span></h2>
+<div class="qr-box">${qrSvg(phoneUrl, { label: `Opens the pairing page for ${robot.name} on the phone` })}</div>
+<p class="muted">It opens the robot's page on the phone with this code filled in. Tap Pair, allow the camera, and keep the page open: the phone is the robot now. No app to install.</p>
+<p class="muted">No camera app handy? Open <b>openvibe.bot/device/phone</b> on the phone and enter the robot and code by hand: <code>${esc(robot.id)}</code></p></li>`
+ : `<li class="card"><h2>Run this on the machine that serves the robot${profile ? ` <span class="pill">${esc(profile.name)}</span>` : ''}</h2>
+<pre class="installer"><code data-copy-source>${esc(pairing.installer)}</code></pre>
+<button type="button" class="copy" data-copy>${icon('copy')}<span data-copy-label>Copy the command</span></button></li>`;
const body = `${topbar(`<span class="crumb">${esc(robot.name)}</span>`)}
const body = `${topbar(`<span class="crumb">${esc(robot.name)}</span>`)}
<main class="page pairing" data-pair-robot="${esc(robot.id)}">
<main class="page pairing" data-pair-robot="${esc(robot.id)}">
<h1>Pair ${esc(robot.name)}</h1>
<h1>Pair ${esc(robot.name)}</h1>
<p class="pair-status" data-pair-status data-state="waiting" role="status" aria-live="polite"><span class="dot"></span><span data-pair-text>Waiting for the device.</span> <a class="button primary" href="/panel/${esc(robot.id)}" data-pair-open hidden>Open the panel</a></p>
<p class="pair-status" data-pair-status data-state="waiting" role="status" aria-live="polite"><span class="dot"></span><span data-pair-text>Waiting for the device.</span> <a class="button primary" href="/panel/${esc(robot.id)}" data-pair-open hidden>Open the panel</a></p>
<ol class="steps">
<ol class="steps">
-<li class="card"><h2>Run this on the machine that serves the robot${profile ? ` <span class="pill">${esc(profile.name)}</span>` : ''}</h2>
-<pre class="installer"><code data-copy-source>${esc(pairing.installer)}</code></pre>
-<button type="button" class="copy" data-copy>${icon('copy')}<span data-copy-label>Copy the command</span></button></li>
+${first}
<li class="card"><h2>Or enter the pairing code</h2>
<li class="card"><h2>Or enter the pairing code</h2>
<p><strong class="code">${esc(pairing.code)}</strong></p>
<p><strong class="code">${esc(pairing.code)}</strong></p>
<p class="muted">Valid until <time datetime="${esc(pairing.expires_at)}">${esc(pairing.expires_at)}</time>. It works once.</p></li>
<p class="muted">Valid until <time datetime="${esc(pairing.expires_at)}">${esc(pairing.expires_at)}</time>. It works once.</p></li>
@@ -511,4 +520,38 @@ function renderPairingPage({ robot, pairing, profile = null }) {
@@ -511,4 +520,38 @@ function renderPairingPage({ robot, pairing, profile = null }) {
return page(`Pair ${robot.name}`, body, { scripts: ['/panel/panel.js'] });
return page(`Pair ${robot.name}`, body, { scripts: ['/panel/panel.js'] });
}
}
-module.exports = { page, renderPanel, panelScripts, isSim, TRY_DESCRIPTION, renderEmbedRefused, renderDrivePage, renderWidget, renderRobotsPage, renderPairingPage, camerasOf, esc };
+/**
+ * /device/phone: the page that makes a phone the robot's device (public/phone-device.js). Its setup form pairs it (the
+ * QR code fills in the robot and code); once running, the screen is the robot's face, with the link and camera state
+ * above it and the phone's own Stop below.
+ */
+function renderPhonePage() {
+ const body = `<main id="phone" class="phone" data-state="setup">
+<section class="phone-setup" data-setup>
+<p class="phone-brand">${icon('robot')}<span>OpenVibe<b>.Bot</b></span></p>
+<h1>Make this phone a robot</h1>
+<p class="muted">Pair it with a robot you added on openvibe.bot. Its camera streams to the robot's panel, it says what the driver types, shows a face, and reports its battery and tilt. Nothing to install: keep this page open.</p>
+<form class="phone-form" data-pair-form>
+<div data-pair-fields>
+<label class="field" data-robot-field><span class="field-label">Robot</span><input name="robot" autocomplete="off" spellcheck="false" placeholder="rob_…"></label>
+<label class="field"><span class="field-label">Pairing code</span><input name="code" autocomplete="one-time-code" autocapitalize="characters" spellcheck="false" placeholder="ABCD-EFGH"></label>
+</div>
+<p class="phone-paired" data-paired hidden>Paired as <b data-paired-name></b>. <button type="button" class="quiet" data-unpair-setup>Pair with another robot</button></p>
+<label class="check"><input type="checkbox" name="front"> Use the front camera</label>
+<button type="submit" class="primary">Start</button>
+<p class="error" data-error role="alert" hidden></p>
+</form>
+<p class="muted small">Get a code: on openvibe.bot, add a robot, choose <b>A phone</b>, and scan the code its pairing page shows.</p>
+</section>
+<section class="phone-run" data-run hidden>
+<header class="phone-bar"><span class="phone-pill" data-link-state data-state="connecting">Connecting…</span><span class="phone-pill" data-video-state data-state="off"></span><span class="phone-rtt" data-rtt></span></header>
+<div class="phone-face" data-face aria-live="polite"></div>
+<p class="phone-safety" data-safety role="status" hidden></p>
+<video class="phone-preview" data-preview muted playsinline autoplay hidden></video>
+<footer class="phone-foot"><button type="button" class="phone-stop" data-local-stop>Stop</button><button type="button" class="quiet" data-forget>Unpair</button></footer>
+</section>
+</main>`;
+ return page('This phone is a robot', body, { scripts: ['/panel/phone-device.js'], bodyClass: 'phone-page', description: 'Turn a phone into a robot on OpenVibe.Bot from its browser: its camera, voice and face, with nothing to install.', canonical: '/device/phone' });
+}
+
+module.exports = { page, renderPanel, renderPhonePage, panelScripts, isSim, TRY_DESCRIPTION, renderEmbedRefused, renderDrivePage, renderWidget, renderRobotsPage, renderPairingPage, camerasOf, esc };
@@ -36,7 +36,7 @@ const { getProfile, listProfiles, loadProfiles } = require('../profiles');
@@ -36,7 +36,7 @@ const { getProfile, listProfiles, loadProfiles } = require('../profiles');
const { frameAncestors } = require('../config');
const { frameAncestors } = require('../config');
const { BotError } = require('../util');
const { BotError } = require('../util');
const kits = require('../kits');
const kits = require('../kits');
-const { renderPanel, renderEmbedRefused, renderDrivePage, renderPairingPage } = require('./render');
+const { renderPanel, renderEmbedRefused, renderDrivePage, renderPairingPage, renderPhonePage } = require('./render');
const { renderDocsIndex, renderDriversPage, renderProfilesPage } = require('./docs');
const { renderDocsIndex, renderDriversPage, renderProfilesPage } = require('./docs');
const { checkProfile } = require('../profiles');
const { checkProfile } = require('../profiles');
const { renderHome, HOME_CSP } = require('./home');
const { renderHome, HOME_CSP } = require('./home');
@@ -171,6 +171,21 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
@@ -171,6 +171,21 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
r.get('/panel/sim-world.js', asset('sim-world.js', 'application/javascript'));
r.get('/panel/sim-world.js', asset('sim-world.js', 'application/javascript'));
r.get('/panel/sim-view.js', asset('sim-view.js', 'application/javascript'));
r.get('/panel/sim-view.js', asset('sim-view.js', 'application/javascript'));
r.get('/panel/try-device.js', asset('try-device.js', 'application/javascript'));
r.get('/panel/try-device.js', asset('try-device.js', 'application/javascript'));
+ r.get('/panel/phone-device.js', asset('phone-device.js', 'application/javascript'));
+
+ // /device/phone: a phone becomes the robot's device from its browser (plan T15). Public: the one-time pairing code
+ // is the credential, and a paired phone keeps its device credential in its own storage. Its CSP adds only the
+ // WHIP ingest the camera publishes to (BOT_WHIP_BASE's origin) and Cloudflare's beacon.
+ const whipOrigin = (() => { try { return config.media.whipBase ? new URL(config.media.whipBase).origin : ''; } catch { return ''; } })();
+ const PHONE_CSP = ["default-src 'self'", "script-src 'self' https://static.cloudflareinsights.com",
+ `connect-src 'self'${whipOrigin ? ` ${whipOrigin}` : ''} https://cloudflareinsights.com`, "media-src 'self' blob:",
+ "frame-ancestors 'none'", "object-src 'none'", "base-uri 'self'"].join('; ');
+ r.get('/device/phone', (req, res) => {
+ res.setHeader('Content-Security-Policy', PHONE_CSP);
+ res.setHeader('Permissions-Policy', 'camera=(self), microphone=(), geolocation=()');
+ res.setHeader('Cache-Control', 'public, max-age=300');
+ res.type('html').send(renderPhonePage());
+ });
r.get('/directory.css', asset('directory.css', 'text/css'));
r.get('/directory.css', asset('directory.css', 'text/css'));
// /try: the real panel driving a rover simulated in the visitor's own browser (public/try-device.js), so anyone
// /try: the real panel driving a rover simulated in the visitor's own browser (public/try-device.js), so anyone
@@ -249,6 +264,9 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
@@ -249,6 +264,9 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
} catch (e) { return next(e); }
} catch (e) { return next(e); }
});
});
+ // The phone's pairing link (the QR code): the robot and code ride in the fragment, which never reaches a server.
+ const phoneUrl = (robotId, pairing) => `${config.baseUrl}/device/phone#robot=${encodeURIComponent(robotId)}&code=${encodeURIComponent(pairing.code)}`;
+
// The embed flag is web-only: domain.present.robot (and so /api/v1) does not carry it.
// The embed flag is web-only: domain.present.robot (and so /api/v1) does not carry it.
const presentForPage = (row) => ({ ...domain.present.robot(row), embed_public: !!row.embed_public, listed: !!row.listed });
const presentForPage = (row) => ({ ...domain.present.robot(row), embed_public: !!row.embed_public, listed: !!row.listed });
@@ -332,7 +350,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
@@ -332,7 +350,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
// A simulated robot needs no machine: straight to its panel. Otherwise the code create minted is shown here.
// A simulated robot needs no machine: straight to its panel. Otherwise the code create minted is shown here.
const profile = await profileOf(robot);
const profile = await profileOf(robot);
if (await attachInProcess(robot, profile)) return res.redirect(303, `/panel/${robot.id}`);
if (await attachInProcess(robot, profile)) return res.redirect(303, `/panel/${robot.id}`);
- return res.status(201).type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile }));
+ return res.status(201).type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile, phoneUrl: phoneUrl(robot.id, pairing) }));
}));
}));
// A load replaces the robot's unused code, so only a navigation from this site (or a typed address) mints one.
// A load replaces the robot's unused code, so only a navigation from this site (or a typed address) mints one.
@@ -348,7 +366,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
@@ -348,7 +366,7 @@ function createWebRoutes(config, { domain = null, sim = null, onvif = null, limi
if (!robot) throw new BotError(404, 'bot.robot_not_found', 'no such robot');
if (!robot) throw new BotError(404, 'bot.robot_not_found', 'no such robot');
if (robot.owner_subject !== me.subject) throw new BotError(403, 'bot.forbidden', 'only the owner may pair a device');
if (robot.owner_subject !== me.subject) throw new BotError(403, 'bot.forbidden', 'only the owner may pair a device');
const pairing = await domain.pairing.create(robot.id, me.subject, undefined, robot);
const pairing = await domain.pairing.create(robot.id, me.subject, undefined, robot);
- res.type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile: await profileOf(robot) }));
+ res.type('html').send(renderPairingPage({ robot: domain.present.robot(robot), pairing, profile: await profileOf(robot), phoneUrl: phoneUrl(robot.id, pairing) }));
}));
}));
r.post('/robots/:id/listing', ...manage, express.urlencoded({ extended: false, limit: '1kb' }), page(async (req, res) => {
r.post('/robots/:id/listing', ...manage, express.urlencoded({ extended: false, limit: '1kb' }), page(async (req, res) => {
@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');
@@ -27,7 +27,7 @@ const { boot, check, done } = require('./helpers/app');
await check('profiles are public reads; an unknown profile is a problem+json 404', async () => {
await check('profiles are public reads; an unknown profile is a problem+json 404', async () => {
const list = await t.call('GET', '/api/v1/profiles', { token: null });
const list = await t.call('GET', '/api/v1/profiles', { token: null });
assert.strictEqual(list.status, 200);
assert.strictEqual(list.status, 200);
- assert.deepStrictEqual(list.json.profiles.map((p) => p.id).sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);
+ assert.deepStrictEqual(list.json.profiles.map((p) => p.id).sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);
const one = await t.call('GET', '/api/v1/profiles/adeept.adr036', { token: null });
const one = await t.call('GET', '/api/v1/profiles/adeept.adr036', { token: null });
assert.strictEqual(one.json.profile.limits.max_command_ms, 300);
assert.strictEqual(one.json.profile.limits.max_command_ms, 300);
const missing = await t.call('GET', '/api/v1/profiles/nope', { token: null });
const missing = await t.call('GET', '/api/v1/profiles/nope', { token: null });
@@ -200,8 +200,33 @@ const looi = {
@@ -200,8 +200,33 @@ const looi = {
};
};
/** The contract a profile's commands must meet: its Node plugin, or the server driver. */
/** The contract a profile's commands must meet: its Node plugin, or the server driver. */
+/**
+ * The phone driver = public/phone-device.js onCommand: say {text}; display {face} (its FACES) or {text}; actuator
+ * torch (true/false) and chime ({hz} or null); halt. Anything else is nacked unsupported.
+ */
+const PHONE_FACES = ['happy', 'neutral', 'sad', 'surprised', 'sleepy', 'wink', 'love', 'angry'];
+const phone = {
+ name: 'phone (browser)',
+ accepts(kind, v) {
+ if (kind === 'halt') return halt(v);
+ if (!isObj(v)) return `${kind} takes an object`;
+ if (kind === 'say') return typeof v.text === 'string' && v.text && only(v, ['text']) ? null : 'say is {text}';
+ if (kind === 'display') {
+ if (only(v, ['face']) && PHONE_FACES.includes(v.face)) return null;
+ return only(v, ['text']) && typeof v.text === 'string' && v.text ? null : 'display is {face} or {text}';
+ }
+ if (kind === 'actuator') {
+ if (v.name === 'torch') return typeof v.value === 'boolean' ? null : 'torch is true or false';
+ if (v.name === 'chime') return v.value === null || (isObj(v.value) && only(v.value, ['hz']) && num(v.value.hz, 220, 1760)) ? null : 'chime is {hz} or null';
+ return `no actuator ${v.name}`;
+ }
+ return `kind ${kind} not supported`;
+ },
+};
+
function contractFor(profile) {
function contractFor(profile) {
const m = profile.mapping || {};
const m = profile.mapping || {};
+ if (m.driver === 'phone') return phone;
if (m.plugin === 'adeept_adr036') return adeept(m.wheels);
if (m.plugin === 'adeept_adr036') return adeept(m.wheels);
if (m.plugin === 'cozmo') return cozmo();
if (m.plugin === 'cozmo') return cozmo();
if (m.plugin === 'relay') return relay;
if (m.plugin === 'relay') return relay;
@@ -209,4 +234,4 @@ function contractFor(profile) {
@@ -209,4 +234,4 @@ function contractFor(profile) {
return server[profile.id] || null;
return server[profile.id] || null;
}
}
-module.exports = { adeept, cozmo, relay, looi, server, contractFor, COZMO_FACES };
+module.exports = { adeept, cozmo, relay, looi, phone, server, contractFor, COZMO_FACES, PHONE_FACES };
@@ -26,7 +26,7 @@ const { renderPanel } = require('../server/web/render');
@@ -26,7 +26,7 @@ const { renderPanel } = require('../server/web/render');
try {
try {
await check('every existing catalogue profile still loads under the contract', () => {
await check('every existing catalogue profile still loads under the contract', () => {
- assert.deepStrictEqual([...loadProfiles().keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);
+ assert.deepStrictEqual([...loadProfiles().keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);
});
});
await check('an operator cannot edit or switch the robot-owned profile', async () => {
await check('an operator cannot edit or switch the robot-owned profile', async () => {
new file mode 100644
new file mode 100644
@@ -0,0 +1,94 @@
@@ -0,0 +1,94 @@
+'use strict';
+// A phone as the robot's device (plan T15 "an Android phone as the robot's brain", profile driver `phone`): the
+// pairing page shows a QR code that opens /device/phone with the code, the phone pairs over POST /api/v1/pair, and its
+// browser offers the credential as a subprotocol on /device (a browser cannot set Authorization).
+const assert = require('assert');
+const { boot, check, done } = require('./helpers/app');
+const { loadProfiles } = require('../server/profiles');
+
+(async () => {
+ const t = await boot();
+ const alex = t.network.newUser('alex');
+ const cookie = (user) => ({ Cookie: `ov_token=${t.network.signUser(user)}` });
+ const get = (p, user) => fetch(t.base + p, { redirect: 'manual', headers: user ? cookie(user) : {} });
+
+ await check('phone.camera ships: camera, voice, a face, torch, chime, tilt and battery, driver phone', () => {
+ const p = loadProfiles().get('phone.camera');
+ assert.strictEqual(p.mapping.driver, 'phone');
+ assert.deepStrictEqual(Object.keys(p.commands).sort(), ['actuator', 'display', 'halt', 'say']);
+ assert.deepStrictEqual(p.widgets.map((w) => w.type), ['camera', 'speaker', 'display', 'lights', 'horn', 'telemetry', 'battery', 'latency']);
+ });
+
+ await check('GET /device/phone: public, its own CSP (the WHIP ingest only), camera allowed, the client script', async () => {
+ const r = await get('/device/phone');
+ assert.strictEqual(r.status, 200);
+ const csp = r.headers.get('content-security-policy');
+ assert.match(csp, /connect-src 'self'[^;]*https:\/\/cloudflareinsights\.com/);
+ assert.doesNotMatch(csp, /esm\.sh|unsafe-inline/);
+ assert.match(r.headers.get('permissions-policy'), /camera=\(self\)/);
+ const html = await r.text();
+ assert.ok(html.includes('<script src="/panel/phone-device.js" defer></script>') && html.includes('data-pair-form') && html.includes('data-local-stop'));
+ assert.doesNotMatch(html, /<script>|style="/, 'nothing inline');
+ assert.strictEqual((await get('/panel/phone-device.js')).status, 200);
+ });
+
+ let robot; let pairing;
+ await check('a phone robot\'s pairing page is a QR code to /device/phone with the code in the fragment, no installer', async () => {
+ ({ robot, pairing } = await t.robot(alex, { name: 'Pocket bot', profile_id: 'phone.camera' }));
+ const html = await (await fetch(`${t.base}/pair/${robot.id}`, { headers: { ...cookie(alex), 'Sec-Fetch-Site': 'same-origin' } })).text();
+ assert.ok(html.includes('<svg class="qr"') && html.includes('Scan this with the phone'));
+ assert.ok(!html.includes('data-copy-source'), 'no command to paste');
+ const fresh = /class="code">([^<]+)</.exec(html)[1];
+ pairing = { code: fresh };
+ assert.ok(html.includes(robot.id));
+ });
+
+ let credential;
+ await check('the phone pairs over HTTP and connects with its credential as a subprotocol: hello, config, online', async () => {
+ const r = await t.call('POST', '/api/v1/pair', { token: null, body: { robot: robot.id, code: pairing.code, agent_version: 'openvibe-phone/1', drivers: ['phone'], name: 'Pixel (browser)' } });
+ assert.strictEqual(r.status, 201, r.text);
+ assert.strictEqual(r.json.profile.id, 'phone.camera');
+ credential = r.json.credential;
+ assert.match(credential, /^[A-Za-z0-9_-]{43}$/);
+ const d = await t.ws('/device', { protocols: ['ov.device', `ov.device.${credential}`] });
+ assert.strictEqual(d.ws.protocol, 'ov.device', 'the server answers the offered protocol (a browser drops the socket otherwise)');
+ assert.ok(await d.waitFor((m) => m.type === 'hello'));
+ const config = await d.waitFor((m) => m.type === 'config');
+ assert.ok(config.allowed_commands.includes('say') && config.allowed_commands.includes('display'));
+ d.send({ v: 1, type: 'status', firmware: 'openvibe-phone/1', capabilities: {}, faults: [], estop_latched: false });
+ d.send({ v: 1, type: 'telemetry', battery: { percent: 81 }, sensors: { tilt: { pitch: 4, roll: -2 } } });
+ const c = await t.ws('/control', { headers: cookie(alex) });
+ c.send({ type: 'join', robot_id: robot.id });
+ const joined = await c.waitFor((m) => m.type === 'joined');
+ assert.ok(joined.state.online, 'the phone is the robot\'s device');
+ c.send({ type: 'command', id: 's1', kind: 'say', value: { text: 'Hello from the panel' } });
+ const cmd = await d.waitFor((m) => m.type === 'command' && m.kind === 'say');
+ assert.deepStrictEqual(cmd.value, { text: 'Hello from the panel' });
+ d.send({ v: 1, type: 'ack', id: cmd.id });
+ assert.strictEqual((await c.waitFor((m) => m.type === 'command_result' && m.id === 's1')).result, 'ack');
+ c.send({ type: 'command', id: 'f1', kind: 'display', value: { face: 'happy' } });
+ assert.deepStrictEqual((await d.waitFor((m) => m.type === 'command' && m.kind === 'display')).value, { face: 'happy' });
+ c.close(); d.close();
+ });
+
+ await check('a wrong credential as a subprotocol closes 4002; a malformed one is never read as a credential', async () => {
+ const bad = await t.ws('/device', { protocols: ['ov.device', `ov.device.${'A'.repeat(43)}`] });
+ assert.strictEqual(await bad.waitForClose(), 4002);
+ // Not credential-shaped: the socket stays unpaired (it may only pair), it is not closed as a bad credential.
+ const odd = await t.ws('/device', { protocols: ['ov.device', 'ov.device.short'] });
+ odd.send({ v: 1, type: 'heartbeat', t: Date.now() });
+ assert.strictEqual((await odd.waitFor((m) => m.type === 'error')).code, 'bot.not_paired');
+ odd.close();
+ });
+
+ await check('the phone model is in the directory, grouped as a phone, and offered when adding a robot', async () => {
+ const models = await (await get('/models')).text();
+ assert.ok(models.includes('>Phone</h2>') && models.includes('href="/models/phone.camera"'));
+ const add = await (await get('/robots/new?profile=phone.camera', alex)).text();
+ assert.match(add, /value="phone\.camera" checked/);
+ assert.ok(add.includes('A phone, from its browser'));
+ });
+
+ await t.close();
+ done();
+})();
@@ -58,8 +58,8 @@ function samples(profile, kind, names) {
@@ -58,8 +58,8 @@ function samples(profile, kind, names) {
(async () => {
(async () => {
const profiles = loadProfiles();
const profiles = loadProfiles();
- await check('the seven shipped profiles load and validate', () => {
- assert.deepStrictEqual([...profiles.keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'relay.generic', 'sim.rover']);
+ await check('the eight shipped profiles load and validate', () => {
+ assert.deepStrictEqual([...profiles.keys()].sort(), ['adeept.adr036', 'adeept.adr036.mecanum', 'camera.onvif', 'cozmo', 'looi', 'phone.camera', 'relay.generic', 'sim.rover']);
for (const p of profiles.values()) {
for (const p of profiles.values()) {
assert.ok(p.capabilities.length > 0, `${p.id} capabilities`);
assert.ok(p.capabilities.length > 0, `${p.id} capabilities`);
assert.ok(p.widgets.length > 0, `${p.id} widgets`);
assert.ok(p.widgets.length > 0, `${p.id} widgets`);