openvibers/openvibe.bot Commits
Panel inputs: WASD, a gamepad and the joystick drive a robot made of buttons; the joystick on pan-tilt cameras; a framed panel takes its page's keys and gamepad (#171)
Changes
@@@ -1,7 +1,24 @@# Moving OpenVibe.Live's stream controls into Bot (plan T15, row R9)--Status: steps 1–3 built in Contracts 0.109.0 and Bot, and step 5 (the conversion) built in Bot, 2026-10-07.--Live still serves its controls; step 4 (the Node relay plugin) and step 6 (Live's deletion) remain.++Status: **done** (updated 2026-10-27). Every step below is built and in production:++- steps 1–3 and 5: Contracts 0.109.0 and Bot, 2026-10-07;++- step 4: OpenVibe.Node's `relay` plugin (`plugins/relay`);++- step 6: Live deleted its controls (`0011_drop_live_controls`). Its old hardware socket `/ws/control` answers++ **410 Gone**, naming OpenVibe.Bot, so a client that still connects there learns where to go++ (Live `server/net/ws-paths.js`).++++What changed since, from the first owner to move (Maticus, 2026-10-26: "Buttons on the stream. That's like my number++one priority"):++- **One press puts the buttons under a stream.** The robot page's **On your stream** card ("Put the buttons under my++ OpenVibe.Live stream") turns the embed on and opens the owner's channel to confirm the binding. Who may drive is++ chosen beside it (Bot#169).++- **Drive under the stream needs no second sign-in.** The panel asks the Live page around it for a ticket++ (`POST /api/v1/robots/:id/control-ticket`).++- **Chat commands reach a button robot.** `!forward`, `!left` … press the robot's buttons of the same name when it++ has no drive, and any other `!word` that is not a channel sound presses the button of that name (Live#383,++ Chat#194).++- **The Node's lifecycle:** `openvibe-node quit` ends a running Node cleanly with nothing latched (Node 0.16.0), for++ launchers that start and stop it with the rest of a stream setup. `stop` stays the kill switch.## What Live has today@@@ -45,7 +62,7 @@ lines) let viewers drive a streamer's hardware from the channel page:3. **Built — Bot gate and panel**: the gate accepts `button`/`point` only for names and shapes the profiledeclares; the panel renders `buttons` (keyboard bindings, hold behaviour) and `video_click` (a click layer on thecamera tile, normalised coordinates).--4. **Node — the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live's++4. **Built — Node, the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live'shardware clients already understand — one JSON line per command on stdout of a supervised child process, or aloopback WebSocket that speaks Live's `{type:'command'|'key_held'|'key_released'|'video_click'}` messages — so anexisting Pi script keeps working with a one-line change of where it connects. Local policy and the kill switch@@@ -63,7 +80,7 @@ lines) let viewers drive a streamer's hardware from the channel page:and skipped (they must sign in to Network once). Idempotent on the Live config id (`live_conversions`,migration 0007): a second `--apply` changes nothing and says "already converted". The owner then pairs aNode (the five-minute path) and moves the script onto the relay plugin.--6. **Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has a++6. **Done — Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has arobot; then delete `server/controls/*`, the hardware WebSocket path and its stream-key auth, the controls tablesand `public/js/controls.js` / `onvif-dashboard.js` (the embed stays).@@@ -106,10 +123,11 @@ if anything, needs redoing by hand.is one line.3. Point the Pi script at the paired Node instead of Live's stream-key socket. The stream key is no longer arobot credential.++4. Robot page → **On your stream** → **Put the buttons under my OpenVibe.Live stream**, and choose who may drive.## Order and cutoverContracts → robot-local profiles → gate/panel → relay plugin → conversion run (owners notified) → Live embed on →owners re-pair → Live deletion. The conversion (step 5) is built and runs once; the deletion waits for the threeconverted owners' robots to be paired, because the hardware clients are physical devices only their owners can--move; it does not wait on a date.++move; it does not wait on a date. (Done: Live's controls are gone, and its old socket answers 410 naming Bot.)@@@ -369,6 +369,21 @@ a spent or expired one closes **4002**. A panel on another site gets one in twotakes an answer only from `window.parent` on one of them: `{ type: "ov-bot-ticket", robot_id, ticket }`, or`{ type: "ov-bot-ticket-none", robot_id, message }` (shown as is: sign in, or the owner decides who drives).With no answer in 2.5 s the next Drive opens the popup.++++While a framed panel drives, the page around it can drive with its own keyboard and gamepad. This is for when the++person is on the page and not in the frame (a Live channel's player).++- The panel tells each `BOT_EMBED_ORIGINS` origin when its controls go live, change or end:++ `{ type: "ov-bot-controls", robot_id, active, keys, pad }`.++ - `keys` lists the `KeyboardEvent.code`s it takes: the drive keys, WASD and the arrows on a robot made of buttons++ (they press its forward/back/left/right buttons), and the profile's button keys.++ - `active: false`, with no keys, ends it.++- The page sends those keys as `{ type: "ov-bot-key", robot_id, code, key, down, repeat }`, never while the person++ types.++- It sends its gamepad at most about 30 times a second as `{ type: "ov-bot-pad", robot_id, axes, buttons }`: axes++ -1…1 and buttons true while pressed. Pad input that has not arrived for 500 ms counts as let go.++- The panel takes these messages only from `window.parent` on an allowed origin, for its own robot, and a press only++ while it drives. They go through the same handlers and the same gate as its own input. A key's release always++ goes through.The gate then applies the **acted-for subject's** role — a service acting for a viewer is a viewer.The same gate is reachable over HTTP as `POST /robots/:id/commands` (§3), so a bound channel's chat canforward commands without holding a socket. The HTTP gate is the same gate but not the same join: a caller@@@ -125,6 +125,11 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf.stick.held .stick-knob { transition: none; }.stick.active .stick-knob { background: radial-gradient(circle at 40% 35%, #7fb2ff, var(--accent) 70%); border-color: #8fbaff; }.stick[aria-disabled="true"] { opacity: 0.4; cursor: not-allowed; }++/* The same joystick on a robot made of buttons (it presses forward, back, left and right) and on a pan-tilt camera. */++.stick--buttons, .stick--ptz { width: min(12rem, 60vw); }++.stick--buttons { margin: 0 auto; }++.stick--buttons + .hint { margin-bottom: 0.9rem; }++.ptz-pad { display: grid; justify-items: center; gap: 0.9rem; }.pad { display: grid; gap: 0.5rem; width: 100%; max-width: 22rem; }.axis { display: grid; grid-template-columns: 1fr auto 1fr; gap: 0.5rem; align-items: center; }.axis .hold { min-height: 56px; user-select: none; -webkit-user-select: none; touch-action: none; }@@@ -434,6 +434,7 @@for (const el of [...tones.keys()]) releaseTone(el);for (const [el, h] of [...buttonHolds]) for (const source of [...h.sources]) releaseProfileButton(el, source);for (const s of sticks) s.release();++ for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }pad.reset();for (const el of $$('.held')) el.classList.remove('held');}@@@ -512,6 +513,102 @@sticks.push({ release });}++ // ── A robot made of buttons drives too (Maticus, 2026-10-27: "WASD keyboard AND game controller … that cool virtual++ // joystick from the sim") ──────────────────────────────────────────────────────────────────────────────────++ // A relay robot's direction buttons (forward, backward, turn_left, turn_right …; render.js DIRECTION_BUTTONS) are++ // pressed from WASD and the arrows, its joystick, and a gamepad's stick and D-pad, when the robot has no drive of its++ // own. A hold button is held while the input is; a plain one is pressed again every REPEAT_MS while it is.++ const DIR_NAMES = { up: ['forward', 'drive_forward', 'up'], down: ['backward', 'back', 'reverse', 'down'], left: ['turn_left', 'left', 'rotate_left'], right: ['turn_right', 'right', 'rotate_right'] };++ const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };++ const REPEAT_MS = 300;++ const dirButton = (dir) => (DIR_NAMES[dir] || []).map((n) => $(`[data-button-name="${n}"]`)).find(Boolean) || null;++ const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;++ const dirRepeats = new Map(); // `${dir}|${source}` → the timer re-pressing a plain button++ function dirPress(dir, source) {++ const el = dirButton(dir);++ if (!el || el.disabled || !joined || !allowed.includes('button')) return false;++ if (el.dataset.buttonHold === 'true') { pressProfileButton(el, source); return true; }++ const k = `${dir}|${source}`;++ if (dirRepeats.has(k)) return true;++ el.classList.add('held');++ pressProfileButton(el, source);++ dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));++ return true;++ }++ function dirRelease(dir, source) {++ const el = dirButton(dir);++ if (!el) return;++ if (el.dataset.buttonHold === 'true') { releaseProfileButton(el, source); return; }++ const k = `${dir}|${source}`;++ if (!dirRepeats.has(k)) return;++ clearInterval(dirRepeats.get(k));++ dirRepeats.delete(k);++ if (![...dirRepeats.keys()].some((x) => x.startsWith(`${dir}|`))) el.classList.remove('held');++ }++ /** A stick position (x right, y up) → the direction it points in once past half way, or null. */++ const stickDir = (x, y) => (Math.hypot(x, y) < 0.5 ? null : Math.abs(x) > Math.abs(y) ? (x > 0 ? 'right' : 'left') : (y > 0 ? 'up' : 'down'));++ /** Where a pointer is on a stick: x right, y up, both -1…1, the knob drawn there. */++ function stickAt(el, e) {++ const r = el.getBoundingClientRect();++ const radius = r.width / 2;++ let x = (e.clientX - (r.left + radius)) / radius;++ let y = -(e.clientY - (r.top + r.height / 2)) / radius;++ const m = Math.hypot(x, y);++ if (m > 1) { x /= m; y /= m; }++ const knob = $('[data-knob]', el);++ if (knob) knob.style.transform = `translate(${x * 75}%, ${-y * 75}%)`;++ return { x, y };++ }++ /** A joystick other than the drive's: onMove({ x, y }) while dragged, onEnd() when let go of (or released). */++ function extraStick(el, onMove, onEnd) {++ let pointer = null;++ const end = () => {++ if (pointer == null) return;++ pointer = null;++ el.classList.remove('held', 'active');++ const knob = $('[data-knob]', el);++ if (knob) knob.style.transform = '';++ onEnd();++ };++ el.addEventListener('pointerdown', (e) => {++ if (el.getAttribute('aria-disabled') === 'true' || pointer != null) return;++ e.preventDefault();++ pointer = e.pointerId;++ try { el.setPointerCapture(e.pointerId); } catch { /* not capturable */ }++ el.classList.add('held', 'active');++ onMove(stickAt(el, e));++ });++ el.addEventListener('pointermove', (e) => { if (e.pointerId === pointer) onMove(stickAt(el, e)); });++ for (const ev of ['pointerup', 'pointercancel', 'lostpointercapture']) el.addEventListener(ev, (e) => { if (e.pointerId === pointer) end(); });++ el.addEventListener('contextmenu', (e) => e.preventDefault());++ sticks.push({ release: end });++ }++ const buttonStick = $('[data-button-stick]');++ if (buttonStick) {++ let dir = null;++ extraStick(buttonStick, ({ x, y }) => {++ const d = stickDir(x, y);++ if (d === dir) return;++ if (dir) dirRelease(dir, 'bstick');++ dir = d;++ if (d) dirPress(d, 'bstick');++ }, () => { if (dir) dirRelease(dir, 'bstick'); dir = null; });++ }++ // A camera's pan and tilt from a joystick (the PTZ widget): the stick's lean is the speed, let go stops it.++ const ptzStick = $('[data-ptz-stick]');++ if (ptzStick) {++ const kind = kindOf(ptzStick) || 'ptz';++ const axes = JSON.parse(ptzStick.dataset.axes || '{}');++ const map = JSON.parse(ptzStick.dataset.ptzStick);++ const at = (axis, u) => { const r = axes[axis]; return r ? round(u >= 0 ? u * r[1] : -u * r[0]) : 0; };++ extraStick(ptzStick, ({ x, y }) => {++ const r = Math.hypot(x, y);++ const k = r <= STICK_DEADZONE ? 0 : Math.min(1, (r - STICK_DEADZONE) / (1 - STICK_DEADZONE)) / r;++ if (!joined || !allowed.includes(kind)) return;++ holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });++ }, () => holdRelease(kind, 'ptzstick'));++ }++// ── Keyboard: arrows (and WASD) drive; Q/E rotate; Space stops ────────────────────────────────────const KEYS = {ArrowUp: [['throttle', 'y'], 1], KeyW: [['throttle', 'y'], 1], ArrowDown: [['throttle', 'y'], 0], KeyS: [['throttle', 'y'], 0],@@@ -541,9 +638,10 @@holdRelease(DRIVE, source);}const typing = (e) => /^(INPUT|SELECT|TEXTAREA)$/.test(e.target.tagName);-- const profileKey = (e) => main.contains(document.activeElement) &&++ // A key from the page around the frame (e.remote) was already checked there: not while typing, only while driving.++ const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&$$('[data-button-key]').find((el) => !el.disabled && (el.dataset.buttonKey === e.code || el.dataset.buttonKey.toLowerCase() === e.key.toLowerCase()));-- document.addEventListener('keydown', (e) => {++ function onKeyDown(e) {if (typing(e)) return;const focusedButton = e.target.closest && e.target.closest('[data-button-name]');if (focusedButton && (e.code === 'Enter' || e.code === 'Space')) {@@@ -552,13 +650,18 @@// Stop and the drive keys first: a button's key never shadows them.if (e.code === 'Space') { e.preventDefault(); stopDrive(); return; }if (keyValue(e.code)) { e.preventDefault(); if (!e.repeat) keyPress(e.code, `key:${e.code}`); return; }++ // A robot made of buttons: the drive keys press its direction buttons.++ if (buttonDrive && KEY_DIR[e.code] && dirButton(KEY_DIR[e.code])) { e.preventDefault(); if (!e.repeat) dirPress(KEY_DIR[e.code], `key:${e.code}`); return; }const button = profileKey(e);if (button) { e.preventDefault(); if (!e.repeat) pressProfileButton(button, `key:${e.code}`); }-- });-- document.addEventListener('keyup', (e) => {++ }++ function onKeyUp(e) {for (const [el, h] of buttonHolds) if (h.sources.has(`key:${e.code}`)) releaseProfileButton(el, `key:${e.code}`);if (KEYS[e.code]) keyRelease(e.code, `key:${e.code}`);-- });++ if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);++ }++ document.addEventListener('keydown', onKeyDown);++ document.addEventListener('keyup', onKeyUp);function stopDrive() {const h = holds.get(DRIVE);@@@ -581,23 +684,61 @@// After a reset (blur, a hidden page, the e-stop) nothing is pressed again until the pad is let go of// entirely: a stick still pushed or a button still held does not undo a release.latched: false,++ dir: null, // the direction button the pad holds (a robot made of buttons)reset() {holdRelease(DRIVE, 'pad');++ if (this.dir) { dirRelease(this.dir, 'pad'); this.dir = null; }for (const [b, code] of Object.entries(PAD_DPAD)) if (this.down.has(Number(b))) keyRelease(code, `pad:${b}`);if (this.tone) { releaseTone(this.tone); this.tone = null; }this.latched = true;},};++ // The page around the frame (a Live channel) sends its gamepad while this panel drives and the page, not the frame,++ // has the focus (Live public/js/bot-embed.js): { axes, buttons: [pressed] }, at most about 30 times a second.++ let remotePad = null;++ const REMOTE_PAD_MS = 500;++ function gamepad() {++ const local = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);++ if (local && document.hasFocus()) return { pad: local, focused: true };++ if (remotePad && performance.now() - remotePad.at < REMOTE_PAD_MS) {++ return { pad: { axes: remotePad.axes, buttons: remotePad.buttons.map((b) => ({ pressed: !!b })), connected: true }, focused: true };++ }++ return local ? { pad: local, focused: document.hasFocus() } : null;++ }function padPoll() {pad.loop = 0;-- const p = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);++ const g = gamepad();++ const p = g && g.pad;if (!p) { pad.reset(); pad.down.clear(); return; } // the last pad went: stop pollingconst ax = (i) => Number(p.axes[i]) || 0;++ const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));++ if (buttonDrive) {++ // A robot made of buttons: the left stick or the D-pad points at a direction, and its button is pressed.++ // Like the drive: after a reset nothing is pressed again until the pad is let go of; B lets go; Start is the e-stop.++ const ok = g.focused && !document.hidden && joined && allowed.includes('button');++ let want = stickDir(dead(ax(0), PAD_DEADZONE), -dead(ax(1), PAD_DEADZONE));++ if (!want) for (const [b, d] of Object.entries({ 12: 'up', 13: 'down', 14: 'left', 15: 'right' })) if (now.has(Number(b))) { want = d; break; }++ if (!ok) {++ if (!pad.latched) pad.reset();++ } else if (pad.latched) {++ if (!want && !now.size) pad.latched = false;++ } else if (now.has(9) && !pad.down.has(9) && $('[data-estop]:not([hidden])')) {++ estop();++ } else if (now.has(1) && !pad.down.has(1)) {++ pad.reset();++ } else if (want !== pad.dir) {++ if (pad.dir) dirRelease(pad.dir, 'pad');++ pad.dir = want;++ if (want) dirPress(want, 'pad');++ }++ pad.down = now;++ pad.loop = requestAnimationFrame(padPoll);++ return;++ }const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);if (ROTATE) v[ROTATE] = scale(ROTATE, dead(ax(2), PAD_DEADZONE));const moving = Object.values(v).some((x) => x !== 0);-- const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));-- if (!document.hasFocus() || document.hidden || !driveEnabled()) {++ if (!g.focused || document.hidden || !driveEnabled()) {if (!pad.latched) pad.reset();} else if (pad.latched) {if (!moving && !now.size) pad.latched = false;@@@ -695,7 +836,30 @@for (const section of $$('[data-command]')) {const ok = joined && allowed.includes(JSON.parse(section.dataset.command).kind);for (const el of $$('button, input, select', section)) el.disabled = !ok;-- for (const el of $$('[data-joystick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }++ for (const el of $$('[data-joystick], [data-button-stick], [data-ptz-stick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }++ }++ announceControls();++ }++ // Framed by an allowed page (a Live channel), this panel tells it while its controls are live and which keys they++ // take, so the page sends those keys and its gamepad here when the person is on the page and not in the frame++ // (Maticus, 2026-10-27: "WASD keyboard AND game controller" on the stream page). Live public/js/bot-embed.js.++ let announced = '';++ function controlKeys() {++ const keys = new Set();++ if (driveEnabled()) { for (const code of Object.keys(KEYS)) if (keyValue(code)) keys.add(code); keys.add('Space'); }++ if (buttonDrive && joined && allowed.includes('button')) for (const [code, dir] of Object.entries(KEY_DIR)) if (dirButton(dir)) keys.add(code);++ for (const el of $$('[data-button-key]')) if (!el.disabled) keys.add(el.dataset.buttonKey);++ return [...keys];++ }++ function announceControls() {++ if (!framed) return;++ const keys = driving && joined ? controlKeys() : [];++ const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };++ const text = JSON.stringify(msg);++ if (text === announced) return;++ announced = text;++ for (const origin of PARENTS) {++ try { window.parent.postMessage(msg, origin); } catch { /* not that origin */ }}}// What a device's refusal means, in words (a nack's fault code). A deadline that passed in transit is not news.@@@ -905,6 +1069,7 @@// ── The embed's Drive ─────────────────────────────────────────────────────────────────────────────function stopDriving(text) {driving = false; watcher = true; ticket = null;++ announceControls();for (const el of $$('[data-estop]')) el.hidden = true;const label = $('[data-role-label]');if (label) label.textContent = 'Watching';@@@ -942,6 +1107,23 @@const fromParent = framed && e.source === window.parent && PARENTS.includes(e.origin);if (!fromPopup && !fromParent) return;const d = e.data || {};++ // The page's keys and gamepad while this panel drives (announceControls): the same handlers as this frame's own.++ // A key's release always goes through, so nothing stays held when driving ends between press and release.++ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {++ if (d.down && !(driving && joined)) return;++ const ev = { code: d.code.slice(0, 40), key: typeof d.key === 'string' ? d.key.slice(0, 20) : '', repeat: d.repeat === true,++ remote: true, target: document.body, preventDefault() {} };++ if (d.down) onKeyDown(ev); else onKeyUp(ev);++ return;++ }++ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {++ if (!(driving && joined)) return;++ const unit = (v) => (Number.isFinite(Number(v)) ? Math.max(-1, Math.min(1, Number(v))) : 0);++ remotePad = { axes: (Array.isArray(d.axes) ? d.axes.slice(0, 8) : []).map(unit),++ buttons: (Array.isArray(d.buttons) ? d.buttons.slice(0, 20) : []).map((b) => b === true), at: performance.now() };++ padStart();++ return;++ }if (fromParent && d.type === 'ov-bot-ticket-none' && d.robot_id === ROBOT_ID) {clearTimeout(asked);notice(typeof d.message === 'string' ? d.message.slice(0, 200) : 'You cannot drive this robot from here.');@@@ -104,6 +104,14 @@ function axisButtons(axes, enabled) {}).join('');}++/** The buttons that mean a direction on a robot made of buttons (panel.js presses them from keys, a stick and a pad). */++const DIRECTION_BUTTONS = [++ { dir: 'up', label: 'forward', names: ['forward', 'drive_forward', 'up'] },++ { dir: 'down', label: 'back', names: ['backward', 'back', 'reverse', 'down'] },++ { dir: 'left', label: 'left', names: ['turn_left', 'left', 'rotate_left'] },++ { dir: 'right', label: 'right', names: ['turn_right', 'right', 'rotate_right'] },++];++/** The drive pad: a touch joystick for two axes (pointer events, see panel.js), hold buttons for the rest. */function drivePad(axes, enabled) {const x = STICK_X.find((a) => axes[a]);@@@ -200,7 +208,13 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =}case 'ptz': {const axes = (commands[kind] && commands[kind].axes) || {};-- inner = `<div class="pad">${axisButtons(axes, enabled)}</div>`;++ // A camera that pans and tilts gets the joystick too (Maticus, 2026-10-27: "that cool virtual joystick from the++ // sim … would work killer for LOOI or any PTZ"): drag to move, let go to stop; zoom stays on its buttons.++ const ptzStick = axes.pan && axes.tilt++ ? `<div class="stick stick--ptz" data-ptz-stick="${data({ x: 'pan', y: 'tilt' })}" data-axes="${data(axes)}" role="img" aria-label="Joystick: pan and tilt. Drag and hold to move the camera."${enabled ? '' : ' aria-disabled="true"'}>`++ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'++ : '';++ inner = `<div class="ptz-pad">${ptzStick}<div class="pad">${axisButtons(axes, enabled)}</div></div>`;break;}case 'pan-tilt': case 'servo': case 'head': case 'lift': case 'lights': case 'horn': {@@@ -241,7 +255,16 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =break;case 'buttons': {const names = (commands.button && commands.button.names) || {};-- inner = `<div class="button-grid">${Object.entries(names).map(([name, button]) =>++ // A robot made of buttons (a relay profile: forward, backward, turn_left, turn_right …) with no drive of its own++ // drives from a joystick, the keyboard and a gamepad too: each direction presses its button (panel.js).++ const dirs = DIRECTION_BUTTONS.filter((d) => d.names.some((n) => names[n]));++ const hasDrive = (profile.widgets || []).some((x) => x.type === 'drive');++ const buttonStick = dirs.length >= 2 && !hasDrive++ ? `<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses ${esc(dirs.map((d) => d.label).join(', '))}. Drag and hold to drive."${enabled ? '' : ' aria-disabled="true"'}>`++ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'++ + `<p class="hint"><kbd>W A S D</kbd> or arrows press ${esc(dirs.map((d) => d.label).join(', '))}. <span class="pad-hint" data-pad-hint>${icon('gamepad')} A gamepad's stick and D-pad do too.</span></p>`++ : '';++ inner = `${buttonStick}<div class="button-grid">${Object.entries(names).map(([name, button]) =>`<button type="button" class="hold pad-button" data-button-name="${esc(name)}" data-button-hold="${button.hold ? 'true' : 'false'}"${button.key ? ` data-button-key="${esc(button.key)}"` : ''}${disabledUnless(enabled)}>`+ `<span>${esc(button.label)}</span>${button.key ? `<kbd>${esc(button.key)}</kbd>` : ''}</button>`).join('')}</div>`;break;@@@ -407,7 +430,7 @@ function localProfileForm(robot, profile, settings = {}) {</div></fieldset>`).join('');return `<section class="setting-card" aria-labelledby="buttons-h"><h2 id="buttons-h">Buttons</h2>--<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving.</p>++<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving. Buttons named forward, back (or backward), left (or turn_left) and right (or turn_right) also get a joystick, and W A S D, the arrows and a gamepad press them.</p>${settings.error ? `<p class="error" role="alert">${esc(settings.error)}</p>` : ''}<form method="post" action="/robots/${esc(robot.id)}/profile" data-profile-form><input type="hidden" name="count" value="${shown.length}">nnew file mode 100644@@@ -0,0 +1,95 @@++'use strict';++// Every way to drive from the panel (Maticus, 2026-10-27: "getting those buttons on openvibe.live stream page (along with++// WASD keyboard AND GaMe cOnTrOlLeR!! … maybe we can add that cool virtual joystick from the sim … would work killer for++// LOOI or any PTZ"):++// - A robot made of buttons (a relay profile: Cozmo's forward/back/left/right, Cozmo Hobo's forward/backward/turn_left/++// turn_right) with no drive of its own gets the joystick, and WASD, the arrows and a gamepad press its direction buttons.++// - A pan-tilt camera gets the joystick for pan and tilt; zoom stays on its buttons.++// - Framed by an allowed page (a Live channel), the panel says while its controls are live and which keys they take++// (ov-bot-controls), and takes that page's keys (ov-bot-key) and gamepad (ov-bot-pad) only from it, for its robot, while++// it drives; a key's release always goes through.++const fs = require('fs');++const path = require('path');++const assert = require('assert');++const { check, done } = require('./helpers/app');++const { loadProfiles } = require('../server/profiles');++const { renderWidget } = require('../server/web/render');++++const relay = (names) => {++ const base = loadProfiles().get('relay.generic');++ return { ...base, commands: { ...base.commands, button: { names: Object.fromEntries(names.map((n) => [n, { label: n }])) } } };++};++const buttonsOf = (profile, allowed = ['button']) => renderWidget(profile.widgets.find((w) => w.type === 'buttons'), { profile, allowed_commands: allowed });++++(async () => {++ await check('a robot made of direction buttons gets the joystick and the key hint, enabled only when it may press', () => {++ for (const names of [['back', 'left', 'light', 'right', 'forward', 'lift_up'], ['forward', 'backward', 'turn_left', 'turn_right', 'head_up']]) {++ const html = buttonsOf(relay(names));++ assert.match(html, /<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses forward, back, left, right\. Drag and hold to drive\.">/, names.join());++ assert.ok(html.includes('<kbd>W A S D</kbd> or arrows press forward, back, left, right.'), names.join());++ assert.ok(html.indexOf('data-button-stick') < html.indexOf('class="button-grid"'), 'the stick above the buttons');++ for (const n of names) assert.ok(html.includes(`data-button-name="${n}"`), `${n} is still its own button`);++ assert.match(buttonsOf(relay(names), []), /data-button-stick[^>]*aria-disabled="true"/, 'disabled without the button kind');++ }++ // Two directions are enough (forward and back on a slider); one is not.++ assert.ok(buttonsOf(relay(['forward', 'back'])).includes('Joystick: presses forward, back.'));++ assert.ok(!buttonsOf(relay(['forward', 'horn'])).includes('data-button-stick'));++ // OBS scenes are buttons with no direction: no stick.++ assert.ok(!buttonsOf(relay(['scene_cam1', 'scene_cam2', 'scene_obs_test'])).includes('data-button-stick'));++ // A robot that has a drive drives with that, never with its buttons.++ const withDrive = relay(['forward', 'back', 'left', 'right']);++ withDrive.widgets = [{ type: 'drive', label: 'Drive', order: 0, command: { kind: 'drive' } }, ...withDrive.widgets];++ assert.ok(!buttonsOf(withDrive).includes('data-button-stick'));++ });++++ await check('a pan-tilt camera gets the joystick for pan and tilt, its axes and zoom buttons kept', () => {++ const cam = loadProfiles().get('camera.onvif');++ const w = cam.widgets.find((x) => x.type === 'ptz');++ const html = renderWidget(w, { profile: cam, allowed_commands: ['ptz'] });++ const stick = /<div class="stick stick--ptz" data-ptz-stick="([^"]*)" data-axes="([^"]*)"[^>]*>/.exec(html);++ assert.ok(stick, html.slice(0, 400));++ const unq = (s) => JSON.parse(s.replace(/"/g, '"'));++ assert.deepStrictEqual(unq(stick[1]), { x: 'pan', y: 'tilt' });++ assert.deepStrictEqual(unq(stick[2]), cam.commands.ptz.axes);++ assert.ok(html.includes('class="ptz-pad"') && html.includes('class="pad"'), 'the buttons stay, zoom among them');++ assert.match(renderWidget(w, { profile: cam, allowed_commands: [] }), /data-ptz-stick[^>]*aria-disabled="true"/);++ // Zoom alone has nothing to steer.++ const zoomOnly = { ...cam, commands: { ...cam.commands, ptz: { axes: { zoom: [-1, 1] } } } };++ assert.ok(!renderWidget(w, { profile: zoomOnly, allowed_commands: ['ptz'] }).includes('data-ptz-stick'));++ });++++ await check('panel.js: keys, stick and gamepad press direction buttons; the PTZ stick holds pan and tilt', () => {++ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');++ assert.ok(js.includes("const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };"));++ assert.ok(js.includes("const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;"), 'only with no drive');++ assert.ok(/if \(buttonDrive && KEY_DIR\[e\.code\] && dirButton\(KEY_DIR\[e\.code\]\)\)/.test(js), 'keydown presses');++ assert.ok(js.includes('if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);'), 'keyup releases');++ assert.ok(/if \(el\.dataset\.buttonHold === 'true'\) \{ pressProfileButton\(el, source\); return true; \}/.test(js), 'a hold button is held');++ assert.ok(js.includes('dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));'), 'a plain one repeats');++ assert.ok(js.includes("holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });"));++ assert.ok(js.includes("() => holdRelease(kind, 'ptzstick'));"), 'let go stops the camera');++ assert.ok(js.includes("for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }"), 'releaseAll lets every repeat go');++ assert.ok(/\[data-joystick\], \[data-button-stick\], \[data-ptz-stick\]/.test(js), 'the new sticks follow the allowed state');++ // The pad on a button robot keeps the drive's latch, B lets go, Start is the e-stop.++ const branch = js.slice(js.indexOf('if (buttonDrive) {', js.indexOf('function padPoll')), js.indexOf('const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);'));++ assert.ok(branch.includes('if (!pad.latched) pad.reset();') && branch.includes('if (!want && !now.size) pad.latched = false;'));++ assert.ok(branch.includes('now.has(1) && !pad.down.has(1)') && branch.includes('now.has(9) && !pad.down.has(9)'));++ });++++ await check('panel.js: the page around the frame gets the controls state and may send keys and its gamepad, nothing else', () => {++ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');++ assert.ok(js.includes("const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };"));++ assert.ok(js.includes('const keys = driving && joined ? controlKeys() : [];'), 'active only while driving');++ assert.ok(js.includes('try { window.parent.postMessage(msg, origin); }'), 'only to the allowed origins');++ assert.ok(/stopDriving\(text\) \{\n\s+driving = false; watcher = true; ticket = null;\n\s+announceControls\(\);/.test(js), 'ending the drive says so');++ const listener = js.slice(js.indexOf("window.addEventListener('message', (e) => {"));++ assert.ok(listener.indexOf("if (!fromPopup && !fromParent) return;") < listener.indexOf("d.type === 'ov-bot-key'"), 'from the allowed page only');++ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {"));++ assert.ok(listener.includes('if (d.down && !(driving && joined)) return;'), 'a press only while driving; a release always');++ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {") && listener.includes('if (!(driving && joined)) return;'));++ assert.ok(listener.includes('Math.max(-1, Math.min(1, Number(v)))') && listener.includes('.map((b) => b === true)'), 'the pad is clamped and typed');++ assert.ok(js.includes('const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&'));++ });++++ done();++})();
@@ -1,7 +1,24 @@
@@ -1,7 +1,24 @@
# Moving OpenVibe.Live's stream controls into Bot (plan T15, row R9)
# Moving OpenVibe.Live's stream controls into Bot (plan T15, row R9)
-Status: steps 1–3 built in Contracts 0.109.0 and Bot, and step 5 (the conversion) built in Bot, 2026-10-07.
-Live still serves its controls; step 4 (the Node relay plugin) and step 6 (Live's deletion) remain.
+Status: **done** (updated 2026-10-27). Every step below is built and in production:
+- steps 1–3 and 5: Contracts 0.109.0 and Bot, 2026-10-07;
+- step 4: OpenVibe.Node's `relay` plugin (`plugins/relay`);
+- step 6: Live deleted its controls (`0011_drop_live_controls`). Its old hardware socket `/ws/control` answers
+ **410 Gone**, naming OpenVibe.Bot, so a client that still connects there learns where to go
+ (Live `server/net/ws-paths.js`).
+
+What changed since, from the first owner to move (Maticus, 2026-10-26: "Buttons on the stream. That's like my number
+one priority"):
+- **One press puts the buttons under a stream.** The robot page's **On your stream** card ("Put the buttons under my
+ OpenVibe.Live stream") turns the embed on and opens the owner's channel to confirm the binding. Who may drive is
+ chosen beside it (Bot#169).
+- **Drive under the stream needs no second sign-in.** The panel asks the Live page around it for a ticket
+ (`POST /api/v1/robots/:id/control-ticket`).
+- **Chat commands reach a button robot.** `!forward`, `!left` … press the robot's buttons of the same name when it
+ has no drive, and any other `!word` that is not a channel sound presses the button of that name (Live#383,
+ Chat#194).
+- **The Node's lifecycle:** `openvibe-node quit` ends a running Node cleanly with nothing latched (Node 0.16.0), for
+ launchers that start and stop it with the rest of a stream setup. `stop` stays the kill switch.
## What Live has today
## What Live has today
@@ -45,7 +62,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
@@ -45,7 +62,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
3. **Built — Bot gate and panel**: the gate accepts `button`/`point` only for names and shapes the profile
3. **Built — Bot gate and panel**: the gate accepts `button`/`point` only for names and shapes the profile
declares; the panel renders `buttons` (keyboard bindings, hold behaviour) and `video_click` (a click layer on the
declares; the panel renders `buttons` (keyboard bindings, hold behaviour) and `video_click` (a click layer on the
camera tile, normalised coordinates).
camera tile, normalised coordinates).
-4. **Node — the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live's
+4. **Built — Node, the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live's
hardware clients already understand — one JSON line per command on stdout of a supervised child process, or a
hardware clients already understand — one JSON line per command on stdout of a supervised child process, or a
loopback WebSocket that speaks Live's `{type:'command'|'key_held'|'key_released'|'video_click'}` messages — so an
loopback WebSocket that speaks Live's `{type:'command'|'key_held'|'key_released'|'video_click'}` messages — so an
existing Pi script keeps working with a one-line change of where it connects. Local policy and the kill switch
existing Pi script keeps working with a one-line change of where it connects. Local policy and the kill switch
@@ -63,7 +80,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
@@ -63,7 +80,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
and skipped (they must sign in to Network once). Idempotent on the Live config id (`live_conversions`,
and skipped (they must sign in to Network once). Idempotent on the Live config id (`live_conversions`,
migration 0007): a second `--apply` changes nothing and says "already converted". The owner then pairs a
migration 0007): a second `--apply` changes nothing and says "already converted". The owner then pairs a
Node (the five-minute path) and moves the script onto the relay plugin.
Node (the five-minute path) and moves the script onto the relay plugin.
-6. **Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has a
+6. **Done — Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has a
robot; then delete `server/controls/*`, the hardware WebSocket path and its stream-key auth, the controls tables
robot; then delete `server/controls/*`, the hardware WebSocket path and its stream-key auth, the controls tables
and `public/js/controls.js` / `onvif-dashboard.js` (the embed stays).
and `public/js/controls.js` / `onvif-dashboard.js` (the embed stays).
@@ -106,10 +123,11 @@ if anything, needs redoing by hand.
@@ -106,10 +123,11 @@ if anything, needs redoing by hand.
is one line.
is one line.
3. Point the Pi script at the paired Node instead of Live's stream-key socket. The stream key is no longer a
3. Point the Pi script at the paired Node instead of Live's stream-key socket. The stream key is no longer a
robot credential.
robot credential.
+4. Robot page → **On your stream** → **Put the buttons under my OpenVibe.Live stream**, and choose who may drive.
## Order and cutover
## Order and cutover
Contracts → robot-local profiles → gate/panel → relay plugin → conversion run (owners notified) → Live embed on →
Contracts → robot-local profiles → gate/panel → relay plugin → conversion run (owners notified) → Live embed on →
owners re-pair → Live deletion. The conversion (step 5) is built and runs once; the deletion waits for the three
owners re-pair → Live deletion. The conversion (step 5) is built and runs once; the deletion waits for the three
converted owners' robots to be paired, because the hardware clients are physical devices only their owners can
converted owners' robots to be paired, because the hardware clients are physical devices only their owners can
-move; it does not wait on a date.
+move; it does not wait on a date. (Done: Live's controls are gone, and its old socket answers 410 naming Bot.)
@@ -369,6 +369,21 @@ a spent or expired one closes **4002**. A panel on another site gets one in two
@@ -369,6 +369,21 @@ a spent or expired one closes **4002**. A panel on another site gets one in two
takes an answer only from `window.parent` on one of them: `{ type: "ov-bot-ticket", robot_id, ticket }`, or
takes an answer only from `window.parent` on one of them: `{ type: "ov-bot-ticket", robot_id, ticket }`, or
`{ type: "ov-bot-ticket-none", robot_id, message }` (shown as is: sign in, or the owner decides who drives).
`{ type: "ov-bot-ticket-none", robot_id, message }` (shown as is: sign in, or the owner decides who drives).
With no answer in 2.5 s the next Drive opens the popup.
With no answer in 2.5 s the next Drive opens the popup.
+
+While a framed panel drives, the page around it can drive with its own keyboard and gamepad. This is for when the
+person is on the page and not in the frame (a Live channel's player).
+- The panel tells each `BOT_EMBED_ORIGINS` origin when its controls go live, change or end:
+ `{ type: "ov-bot-controls", robot_id, active, keys, pad }`.
+ - `keys` lists the `KeyboardEvent.code`s it takes: the drive keys, WASD and the arrows on a robot made of buttons
+ (they press its forward/back/left/right buttons), and the profile's button keys.
+ - `active: false`, with no keys, ends it.
+- The page sends those keys as `{ type: "ov-bot-key", robot_id, code, key, down, repeat }`, never while the person
+ types.
+- It sends its gamepad at most about 30 times a second as `{ type: "ov-bot-pad", robot_id, axes, buttons }`: axes
+ -1…1 and buttons true while pressed. Pad input that has not arrived for 500 ms counts as let go.
+- The panel takes these messages only from `window.parent` on an allowed origin, for its own robot, and a press only
+ while it drives. They go through the same handlers and the same gate as its own input. A key's release always
+ goes through.
The gate then applies the **acted-for subject's** role — a service acting for a viewer is a viewer.
The gate then applies the **acted-for subject's** role — a service acting for a viewer is a viewer.
The same gate is reachable over HTTP as `POST /robots/:id/commands` (§3), so a bound channel's chat can
The same gate is reachable over HTTP as `POST /robots/:id/commands` (§3), so a bound channel's chat can
forward commands without holding a socket. The HTTP gate is the same gate but not the same join: a caller
forward commands without holding a socket. The HTTP gate is the same gate but not the same join: a caller
@@ -125,6 +125,11 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf
@@ -125,6 +125,11 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf
.stick.held .stick-knob { transition: none; }
.stick.held .stick-knob { transition: none; }
.stick.active .stick-knob { background: radial-gradient(circle at 40% 35%, #7fb2ff, var(--accent) 70%); border-color: #8fbaff; }
.stick.active .stick-knob { background: radial-gradient(circle at 40% 35%, #7fb2ff, var(--accent) 70%); border-color: #8fbaff; }
.stick[aria-disabled="true"] { opacity: 0.4; cursor: not-allowed; }
.stick[aria-disabled="true"] { opacity: 0.4; cursor: not-allowed; }
+/* The same joystick on a robot made of buttons (it presses forward, back, left and right) and on a pan-tilt camera. */
+.stick--buttons, .stick--ptz { width: min(12rem, 60vw); }
+.stick--buttons { margin: 0 auto; }
+.stick--buttons + .hint { margin-bottom: 0.9rem; }
+.ptz-pad { display: grid; justify-items: center; gap: 0.9rem; }
.pad { display: grid; gap: 0.5rem; width: 100%; max-width: 22rem; }
.pad { display: grid; gap: 0.5rem; width: 100%; max-width: 22rem; }
.axis { display: grid; grid-template-columns: 1fr auto 1fr; gap: 0.5rem; align-items: center; }
.axis { display: grid; grid-template-columns: 1fr auto 1fr; gap: 0.5rem; align-items: center; }
.axis .hold { min-height: 56px; user-select: none; -webkit-user-select: none; touch-action: none; }
.axis .hold { min-height: 56px; user-select: none; -webkit-user-select: none; touch-action: none; }
@@ -434,6 +434,7 @@
@@ -434,6 +434,7 @@
for (const el of [...tones.keys()]) releaseTone(el);
for (const el of [...tones.keys()]) releaseTone(el);
for (const [el, h] of [...buttonHolds]) for (const source of [...h.sources]) releaseProfileButton(el, source);
for (const [el, h] of [...buttonHolds]) for (const source of [...h.sources]) releaseProfileButton(el, source);
for (const s of sticks) s.release();
for (const s of sticks) s.release();
+ for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }
pad.reset();
pad.reset();
for (const el of $$('.held')) el.classList.remove('held');
for (const el of $$('.held')) el.classList.remove('held');
}
}
@@ -512,6 +513,102 @@
@@ -512,6 +513,102 @@
sticks.push({ release });
sticks.push({ release });
}
}
+ // ── A robot made of buttons drives too (Maticus, 2026-10-27: "WASD keyboard AND game controller … that cool virtual
+ // joystick from the sim") ──────────────────────────────────────────────────────────────────────────────────
+ // A relay robot's direction buttons (forward, backward, turn_left, turn_right …; render.js DIRECTION_BUTTONS) are
+ // pressed from WASD and the arrows, its joystick, and a gamepad's stick and D-pad, when the robot has no drive of its
+ // own. A hold button is held while the input is; a plain one is pressed again every REPEAT_MS while it is.
+ const DIR_NAMES = { up: ['forward', 'drive_forward', 'up'], down: ['backward', 'back', 'reverse', 'down'], left: ['turn_left', 'left', 'rotate_left'], right: ['turn_right', 'right', 'rotate_right'] };
+ const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };
+ const REPEAT_MS = 300;
+ const dirButton = (dir) => (DIR_NAMES[dir] || []).map((n) => $(`[data-button-name="${n}"]`)).find(Boolean) || null;
+ const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;
+ const dirRepeats = new Map(); // `${dir}|${source}` → the timer re-pressing a plain button
+ function dirPress(dir, source) {
+ const el = dirButton(dir);
+ if (!el || el.disabled || !joined || !allowed.includes('button')) return false;
+ if (el.dataset.buttonHold === 'true') { pressProfileButton(el, source); return true; }
+ const k = `${dir}|${source}`;
+ if (dirRepeats.has(k)) return true;
+ el.classList.add('held');
+ pressProfileButton(el, source);
+ dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));
+ return true;
+ }
+ function dirRelease(dir, source) {
+ const el = dirButton(dir);
+ if (!el) return;
+ if (el.dataset.buttonHold === 'true') { releaseProfileButton(el, source); return; }
+ const k = `${dir}|${source}`;
+ if (!dirRepeats.has(k)) return;
+ clearInterval(dirRepeats.get(k));
+ dirRepeats.delete(k);
+ if (![...dirRepeats.keys()].some((x) => x.startsWith(`${dir}|`))) el.classList.remove('held');
+ }
+ /** A stick position (x right, y up) → the direction it points in once past half way, or null. */
+ const stickDir = (x, y) => (Math.hypot(x, y) < 0.5 ? null : Math.abs(x) > Math.abs(y) ? (x > 0 ? 'right' : 'left') : (y > 0 ? 'up' : 'down'));
+ /** Where a pointer is on a stick: x right, y up, both -1…1, the knob drawn there. */
+ function stickAt(el, e) {
+ const r = el.getBoundingClientRect();
+ const radius = r.width / 2;
+ let x = (e.clientX - (r.left + radius)) / radius;
+ let y = -(e.clientY - (r.top + r.height / 2)) / radius;
+ const m = Math.hypot(x, y);
+ if (m > 1) { x /= m; y /= m; }
+ const knob = $('[data-knob]', el);
+ if (knob) knob.style.transform = `translate(${x * 75}%, ${-y * 75}%)`;
+ return { x, y };
+ }
+ /** A joystick other than the drive's: onMove({ x, y }) while dragged, onEnd() when let go of (or released). */
+ function extraStick(el, onMove, onEnd) {
+ let pointer = null;
+ const end = () => {
+ if (pointer == null) return;
+ pointer = null;
+ el.classList.remove('held', 'active');
+ const knob = $('[data-knob]', el);
+ if (knob) knob.style.transform = '';
+ onEnd();
+ };
+ el.addEventListener('pointerdown', (e) => {
+ if (el.getAttribute('aria-disabled') === 'true' || pointer != null) return;
+ e.preventDefault();
+ pointer = e.pointerId;
+ try { el.setPointerCapture(e.pointerId); } catch { /* not capturable */ }
+ el.classList.add('held', 'active');
+ onMove(stickAt(el, e));
+ });
+ el.addEventListener('pointermove', (e) => { if (e.pointerId === pointer) onMove(stickAt(el, e)); });
+ for (const ev of ['pointerup', 'pointercancel', 'lostpointercapture']) el.addEventListener(ev, (e) => { if (e.pointerId === pointer) end(); });
+ el.addEventListener('contextmenu', (e) => e.preventDefault());
+ sticks.push({ release: end });
+ }
+ const buttonStick = $('[data-button-stick]');
+ if (buttonStick) {
+ let dir = null;
+ extraStick(buttonStick, ({ x, y }) => {
+ const d = stickDir(x, y);
+ if (d === dir) return;
+ if (dir) dirRelease(dir, 'bstick');
+ dir = d;
+ if (d) dirPress(d, 'bstick');
+ }, () => { if (dir) dirRelease(dir, 'bstick'); dir = null; });
+ }
+ // A camera's pan and tilt from a joystick (the PTZ widget): the stick's lean is the speed, let go stops it.
+ const ptzStick = $('[data-ptz-stick]');
+ if (ptzStick) {
+ const kind = kindOf(ptzStick) || 'ptz';
+ const axes = JSON.parse(ptzStick.dataset.axes || '{}');
+ const map = JSON.parse(ptzStick.dataset.ptzStick);
+ const at = (axis, u) => { const r = axes[axis]; return r ? round(u >= 0 ? u * r[1] : -u * r[0]) : 0; };
+ extraStick(ptzStick, ({ x, y }) => {
+ const r = Math.hypot(x, y);
+ const k = r <= STICK_DEADZONE ? 0 : Math.min(1, (r - STICK_DEADZONE) / (1 - STICK_DEADZONE)) / r;
+ if (!joined || !allowed.includes(kind)) return;
+ holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });
+ }, () => holdRelease(kind, 'ptzstick'));
+ }
+
// ── Keyboard: arrows (and WASD) drive; Q/E rotate; Space stops ────────────────────────────────────
// ── Keyboard: arrows (and WASD) drive; Q/E rotate; Space stops ────────────────────────────────────
const KEYS = {
const KEYS = {
ArrowUp: [['throttle', 'y'], 1], KeyW: [['throttle', 'y'], 1], ArrowDown: [['throttle', 'y'], 0], KeyS: [['throttle', 'y'], 0],
ArrowUp: [['throttle', 'y'], 1], KeyW: [['throttle', 'y'], 1], ArrowDown: [['throttle', 'y'], 0], KeyS: [['throttle', 'y'], 0],
@@ -541,9 +638,10 @@
@@ -541,9 +638,10 @@
holdRelease(DRIVE, source);
holdRelease(DRIVE, source);
}
}
const typing = (e) => /^(INPUT|SELECT|TEXTAREA)$/.test(e.target.tagName);
const typing = (e) => /^(INPUT|SELECT|TEXTAREA)$/.test(e.target.tagName);
- const profileKey = (e) => main.contains(document.activeElement) &&
+ // A key from the page around the frame (e.remote) was already checked there: not while typing, only while driving.
+ const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&
$$('[data-button-key]').find((el) => !el.disabled && (el.dataset.buttonKey === e.code || el.dataset.buttonKey.toLowerCase() === e.key.toLowerCase()));
$$('[data-button-key]').find((el) => !el.disabled && (el.dataset.buttonKey === e.code || el.dataset.buttonKey.toLowerCase() === e.key.toLowerCase()));
- document.addEventListener('keydown', (e) => {
+ function onKeyDown(e) {
if (typing(e)) return;
if (typing(e)) return;
const focusedButton = e.target.closest && e.target.closest('[data-button-name]');
const focusedButton = e.target.closest && e.target.closest('[data-button-name]');
if (focusedButton && (e.code === 'Enter' || e.code === 'Space')) {
if (focusedButton && (e.code === 'Enter' || e.code === 'Space')) {
@@ -552,13 +650,18 @@
@@ -552,13 +650,18 @@
// Stop and the drive keys first: a button's key never shadows them.
// Stop and the drive keys first: a button's key never shadows them.
if (e.code === 'Space') { e.preventDefault(); stopDrive(); return; }
if (e.code === 'Space') { e.preventDefault(); stopDrive(); return; }
if (keyValue(e.code)) { e.preventDefault(); if (!e.repeat) keyPress(e.code, `key:${e.code}`); return; }
if (keyValue(e.code)) { e.preventDefault(); if (!e.repeat) keyPress(e.code, `key:${e.code}`); return; }
+ // A robot made of buttons: the drive keys press its direction buttons.
+ if (buttonDrive && KEY_DIR[e.code] && dirButton(KEY_DIR[e.code])) { e.preventDefault(); if (!e.repeat) dirPress(KEY_DIR[e.code], `key:${e.code}`); return; }
const button = profileKey(e);
const button = profileKey(e);
if (button) { e.preventDefault(); if (!e.repeat) pressProfileButton(button, `key:${e.code}`); }
if (button) { e.preventDefault(); if (!e.repeat) pressProfileButton(button, `key:${e.code}`); }
- });
- document.addEventListener('keyup', (e) => {
+ }
+ function onKeyUp(e) {
for (const [el, h] of buttonHolds) if (h.sources.has(`key:${e.code}`)) releaseProfileButton(el, `key:${e.code}`);
for (const [el, h] of buttonHolds) if (h.sources.has(`key:${e.code}`)) releaseProfileButton(el, `key:${e.code}`);
if (KEYS[e.code]) keyRelease(e.code, `key:${e.code}`);
if (KEYS[e.code]) keyRelease(e.code, `key:${e.code}`);
- });
+ if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);
+ }
+ document.addEventListener('keydown', onKeyDown);
+ document.addEventListener('keyup', onKeyUp);
function stopDrive() {
function stopDrive() {
const h = holds.get(DRIVE);
const h = holds.get(DRIVE);
@@ -581,23 +684,61 @@
@@ -581,23 +684,61 @@
// After a reset (blur, a hidden page, the e-stop) nothing is pressed again until the pad is let go of
// After a reset (blur, a hidden page, the e-stop) nothing is pressed again until the pad is let go of
// entirely: a stick still pushed or a button still held does not undo a release.
// entirely: a stick still pushed or a button still held does not undo a release.
latched: false,
latched: false,
+ dir: null, // the direction button the pad holds (a robot made of buttons)
reset() {
reset() {
holdRelease(DRIVE, 'pad');
holdRelease(DRIVE, 'pad');
+ if (this.dir) { dirRelease(this.dir, 'pad'); this.dir = null; }
for (const [b, code] of Object.entries(PAD_DPAD)) if (this.down.has(Number(b))) keyRelease(code, `pad:${b}`);
for (const [b, code] of Object.entries(PAD_DPAD)) if (this.down.has(Number(b))) keyRelease(code, `pad:${b}`);
if (this.tone) { releaseTone(this.tone); this.tone = null; }
if (this.tone) { releaseTone(this.tone); this.tone = null; }
this.latched = true;
this.latched = true;
},
},
};
};
+ // The page around the frame (a Live channel) sends its gamepad while this panel drives and the page, not the frame,
+ // has the focus (Live public/js/bot-embed.js): { axes, buttons: [pressed] }, at most about 30 times a second.
+ let remotePad = null;
+ const REMOTE_PAD_MS = 500;
+ function gamepad() {
+ const local = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);
+ if (local && document.hasFocus()) return { pad: local, focused: true };
+ if (remotePad && performance.now() - remotePad.at < REMOTE_PAD_MS) {
+ return { pad: { axes: remotePad.axes, buttons: remotePad.buttons.map((b) => ({ pressed: !!b })), connected: true }, focused: true };
+ }
+ return local ? { pad: local, focused: document.hasFocus() } : null;
+ }
function padPoll() {
function padPoll() {
pad.loop = 0;
pad.loop = 0;
- const p = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);
+ const g = gamepad();
+ const p = g && g.pad;
if (!p) { pad.reset(); pad.down.clear(); return; } // the last pad went: stop polling
if (!p) { pad.reset(); pad.down.clear(); return; } // the last pad went: stop polling
const ax = (i) => Number(p.axes[i]) || 0;
const ax = (i) => Number(p.axes[i]) || 0;
+ const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));
+ if (buttonDrive) {
+ // A robot made of buttons: the left stick or the D-pad points at a direction, and its button is pressed.
+ // Like the drive: after a reset nothing is pressed again until the pad is let go of; B lets go; Start is the e-stop.
+ const ok = g.focused && !document.hidden && joined && allowed.includes('button');
+ let want = stickDir(dead(ax(0), PAD_DEADZONE), -dead(ax(1), PAD_DEADZONE));
+ if (!want) for (const [b, d] of Object.entries({ 12: 'up', 13: 'down', 14: 'left', 15: 'right' })) if (now.has(Number(b))) { want = d; break; }
+ if (!ok) {
+ if (!pad.latched) pad.reset();
+ } else if (pad.latched) {
+ if (!want && !now.size) pad.latched = false;
+ } else if (now.has(9) && !pad.down.has(9) && $('[data-estop]:not([hidden])')) {
+ estop();
+ } else if (now.has(1) && !pad.down.has(1)) {
+ pad.reset();
+ } else if (want !== pad.dir) {
+ if (pad.dir) dirRelease(pad.dir, 'pad');
+ pad.dir = want;
+ if (want) dirPress(want, 'pad');
+ }
+ pad.down = now;
+ pad.loop = requestAnimationFrame(padPoll);
+ return;
+ }
const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);
const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);
if (ROTATE) v[ROTATE] = scale(ROTATE, dead(ax(2), PAD_DEADZONE));
if (ROTATE) v[ROTATE] = scale(ROTATE, dead(ax(2), PAD_DEADZONE));
const moving = Object.values(v).some((x) => x !== 0);
const moving = Object.values(v).some((x) => x !== 0);
- const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));
- if (!document.hasFocus() || document.hidden || !driveEnabled()) {
+ if (!g.focused || document.hidden || !driveEnabled()) {
if (!pad.latched) pad.reset();
if (!pad.latched) pad.reset();
} else if (pad.latched) {
} else if (pad.latched) {
if (!moving && !now.size) pad.latched = false;
if (!moving && !now.size) pad.latched = false;
@@ -695,7 +836,30 @@
@@ -695,7 +836,30 @@
for (const section of $$('[data-command]')) {
for (const section of $$('[data-command]')) {
const ok = joined && allowed.includes(JSON.parse(section.dataset.command).kind);
const ok = joined && allowed.includes(JSON.parse(section.dataset.command).kind);
for (const el of $$('button, input, select', section)) el.disabled = !ok;
for (const el of $$('button, input, select', section)) el.disabled = !ok;
- for (const el of $$('[data-joystick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }
+ for (const el of $$('[data-joystick], [data-button-stick], [data-ptz-stick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }
+ }
+ announceControls();
+ }
+ // Framed by an allowed page (a Live channel), this panel tells it while its controls are live and which keys they
+ // take, so the page sends those keys and its gamepad here when the person is on the page and not in the frame
+ // (Maticus, 2026-10-27: "WASD keyboard AND game controller" on the stream page). Live public/js/bot-embed.js.
+ let announced = '';
+ function controlKeys() {
+ const keys = new Set();
+ if (driveEnabled()) { for (const code of Object.keys(KEYS)) if (keyValue(code)) keys.add(code); keys.add('Space'); }
+ if (buttonDrive && joined && allowed.includes('button')) for (const [code, dir] of Object.entries(KEY_DIR)) if (dirButton(dir)) keys.add(code);
+ for (const el of $$('[data-button-key]')) if (!el.disabled) keys.add(el.dataset.buttonKey);
+ return [...keys];
+ }
+ function announceControls() {
+ if (!framed) return;
+ const keys = driving && joined ? controlKeys() : [];
+ const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };
+ const text = JSON.stringify(msg);
+ if (text === announced) return;
+ announced = text;
+ for (const origin of PARENTS) {
+ try { window.parent.postMessage(msg, origin); } catch { /* not that origin */ }
}
}
}
}
// What a device's refusal means, in words (a nack's fault code). A deadline that passed in transit is not news.
// What a device's refusal means, in words (a nack's fault code). A deadline that passed in transit is not news.
@@ -905,6 +1069,7 @@
@@ -905,6 +1069,7 @@
// ── The embed's Drive ─────────────────────────────────────────────────────────────────────────────
// ── The embed's Drive ─────────────────────────────────────────────────────────────────────────────
function stopDriving(text) {
function stopDriving(text) {
driving = false; watcher = true; ticket = null;
driving = false; watcher = true; ticket = null;
+ announceControls();
for (const el of $$('[data-estop]')) el.hidden = true;
for (const el of $$('[data-estop]')) el.hidden = true;
const label = $('[data-role-label]');
const label = $('[data-role-label]');
if (label) label.textContent = 'Watching';
if (label) label.textContent = 'Watching';
@@ -942,6 +1107,23 @@
@@ -942,6 +1107,23 @@
const fromParent = framed && e.source === window.parent && PARENTS.includes(e.origin);
const fromParent = framed && e.source === window.parent && PARENTS.includes(e.origin);
if (!fromPopup && !fromParent) return;
if (!fromPopup && !fromParent) return;
const d = e.data || {};
const d = e.data || {};
+ // The page's keys and gamepad while this panel drives (announceControls): the same handlers as this frame's own.
+ // A key's release always goes through, so nothing stays held when driving ends between press and release.
+ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {
+ if (d.down && !(driving && joined)) return;
+ const ev = { code: d.code.slice(0, 40), key: typeof d.key === 'string' ? d.key.slice(0, 20) : '', repeat: d.repeat === true,
+ remote: true, target: document.body, preventDefault() {} };
+ if (d.down) onKeyDown(ev); else onKeyUp(ev);
+ return;
+ }
+ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {
+ if (!(driving && joined)) return;
+ const unit = (v) => (Number.isFinite(Number(v)) ? Math.max(-1, Math.min(1, Number(v))) : 0);
+ remotePad = { axes: (Array.isArray(d.axes) ? d.axes.slice(0, 8) : []).map(unit),
+ buttons: (Array.isArray(d.buttons) ? d.buttons.slice(0, 20) : []).map((b) => b === true), at: performance.now() };
+ padStart();
+ return;
+ }
if (fromParent && d.type === 'ov-bot-ticket-none' && d.robot_id === ROBOT_ID) {
if (fromParent && d.type === 'ov-bot-ticket-none' && d.robot_id === ROBOT_ID) {
clearTimeout(asked);
clearTimeout(asked);
notice(typeof d.message === 'string' ? d.message.slice(0, 200) : 'You cannot drive this robot from here.');
notice(typeof d.message === 'string' ? d.message.slice(0, 200) : 'You cannot drive this robot from here.');
@@ -104,6 +104,14 @@ function axisButtons(axes, enabled) {
@@ -104,6 +104,14 @@ function axisButtons(axes, enabled) {
}).join('');
}).join('');
}
}
+/** The buttons that mean a direction on a robot made of buttons (panel.js presses them from keys, a stick and a pad). */
+const DIRECTION_BUTTONS = [
+ { dir: 'up', label: 'forward', names: ['forward', 'drive_forward', 'up'] },
+ { dir: 'down', label: 'back', names: ['backward', 'back', 'reverse', 'down'] },
+ { dir: 'left', label: 'left', names: ['turn_left', 'left', 'rotate_left'] },
+ { dir: 'right', label: 'right', names: ['turn_right', 'right', 'rotate_right'] },
+];
+
/** The drive pad: a touch joystick for two axes (pointer events, see panel.js), hold buttons for the rest. */
/** The drive pad: a touch joystick for two axes (pointer events, see panel.js), hold buttons for the rest. */
function drivePad(axes, enabled) {
function drivePad(axes, enabled) {
const x = STICK_X.find((a) => axes[a]);
const x = STICK_X.find((a) => axes[a]);
@@ -200,7 +208,13 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
@@ -200,7 +208,13 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
}
}
case 'ptz': {
case 'ptz': {
const axes = (commands[kind] && commands[kind].axes) || {};
const axes = (commands[kind] && commands[kind].axes) || {};
- inner = `<div class="pad">${axisButtons(axes, enabled)}</div>`;
+ // A camera that pans and tilts gets the joystick too (Maticus, 2026-10-27: "that cool virtual joystick from the
+ // sim … would work killer for LOOI or any PTZ"): drag to move, let go to stop; zoom stays on its buttons.
+ const ptzStick = axes.pan && axes.tilt
+ ? `<div class="stick stick--ptz" data-ptz-stick="${data({ x: 'pan', y: 'tilt' })}" data-axes="${data(axes)}" role="img" aria-label="Joystick: pan and tilt. Drag and hold to move the camera."${enabled ? '' : ' aria-disabled="true"'}>`
+ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'
+ : '';
+ inner = `<div class="ptz-pad">${ptzStick}<div class="pad">${axisButtons(axes, enabled)}</div></div>`;
break;
break;
}
}
case 'pan-tilt': case 'servo': case 'head': case 'lift': case 'lights': case 'horn': {
case 'pan-tilt': case 'servo': case 'head': case 'lift': case 'lights': case 'horn': {
@@ -241,7 +255,16 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
@@ -241,7 +255,16 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
break;
break;
case 'buttons': {
case 'buttons': {
const names = (commands.button && commands.button.names) || {};
const names = (commands.button && commands.button.names) || {};
- inner = `<div class="button-grid">${Object.entries(names).map(([name, button]) =>
+ // A robot made of buttons (a relay profile: forward, backward, turn_left, turn_right …) with no drive of its own
+ // drives from a joystick, the keyboard and a gamepad too: each direction presses its button (panel.js).
+ const dirs = DIRECTION_BUTTONS.filter((d) => d.names.some((n) => names[n]));
+ const hasDrive = (profile.widgets || []).some((x) => x.type === 'drive');
+ const buttonStick = dirs.length >= 2 && !hasDrive
+ ? `<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses ${esc(dirs.map((d) => d.label).join(', '))}. Drag and hold to drive."${enabled ? '' : ' aria-disabled="true"'}>`
+ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'
+ + `<p class="hint"><kbd>W A S D</kbd> or arrows press ${esc(dirs.map((d) => d.label).join(', '))}. <span class="pad-hint" data-pad-hint>${icon('gamepad')} A gamepad's stick and D-pad do too.</span></p>`
+ : '';
+ inner = `${buttonStick}<div class="button-grid">${Object.entries(names).map(([name, button]) =>
`<button type="button" class="hold pad-button" data-button-name="${esc(name)}" data-button-hold="${button.hold ? 'true' : 'false'}"${button.key ? ` data-button-key="${esc(button.key)}"` : ''}${disabledUnless(enabled)}>`
`<button type="button" class="hold pad-button" data-button-name="${esc(name)}" data-button-hold="${button.hold ? 'true' : 'false'}"${button.key ? ` data-button-key="${esc(button.key)}"` : ''}${disabledUnless(enabled)}>`
+ `<span>${esc(button.label)}</span>${button.key ? `<kbd>${esc(button.key)}</kbd>` : ''}</button>`).join('')}</div>`;
+ `<span>${esc(button.label)}</span>${button.key ? `<kbd>${esc(button.key)}</kbd>` : ''}</button>`).join('')}</div>`;
break;
break;
@@ -407,7 +430,7 @@ function localProfileForm(robot, profile, settings = {}) {
@@ -407,7 +430,7 @@ function localProfileForm(robot, profile, settings = {}) {
</div>
</div>
</fieldset>`).join('');
</fieldset>`).join('');
return `<section class="setting-card" aria-labelledby="buttons-h"><h2 id="buttons-h">Buttons</h2>
return `<section class="setting-card" aria-labelledby="buttons-h"><h2 id="buttons-h">Buttons</h2>
-<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving.</p>
+<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving. Buttons named forward, back (or backward), left (or turn_left) and right (or turn_right) also get a joystick, and W A S D, the arrows and a gamepad press them.</p>
${settings.error ? `<p class="error" role="alert">${esc(settings.error)}</p>` : ''}
${settings.error ? `<p class="error" role="alert">${esc(settings.error)}</p>` : ''}
<form method="post" action="/robots/${esc(robot.id)}/profile" data-profile-form>
<form method="post" action="/robots/${esc(robot.id)}/profile" data-profile-form>
<input type="hidden" name="count" value="${shown.length}">
<input type="hidden" name="count" value="${shown.length}">
new file mode 100644
new file mode 100644
@@ -0,0 +1,95 @@
@@ -0,0 +1,95 @@
+'use strict';
+// Every way to drive from the panel (Maticus, 2026-10-27: "getting those buttons on openvibe.live stream page (along with
+// WASD keyboard AND GaMe cOnTrOlLeR!! … maybe we can add that cool virtual joystick from the sim … would work killer for
+// LOOI or any PTZ"):
+// - A robot made of buttons (a relay profile: Cozmo's forward/back/left/right, Cozmo Hobo's forward/backward/turn_left/
+// turn_right) with no drive of its own gets the joystick, and WASD, the arrows and a gamepad press its direction buttons.
+// - A pan-tilt camera gets the joystick for pan and tilt; zoom stays on its buttons.
+// - Framed by an allowed page (a Live channel), the panel says while its controls are live and which keys they take
+// (ov-bot-controls), and takes that page's keys (ov-bot-key) and gamepad (ov-bot-pad) only from it, for its robot, while
+// it drives; a key's release always goes through.
+const fs = require('fs');
+const path = require('path');
+const assert = require('assert');
+const { check, done } = require('./helpers/app');
+const { loadProfiles } = require('../server/profiles');
+const { renderWidget } = require('../server/web/render');
+
+const relay = (names) => {
+ const base = loadProfiles().get('relay.generic');
+ return { ...base, commands: { ...base.commands, button: { names: Object.fromEntries(names.map((n) => [n, { label: n }])) } } };
+};
+const buttonsOf = (profile, allowed = ['button']) => renderWidget(profile.widgets.find((w) => w.type === 'buttons'), { profile, allowed_commands: allowed });
+
+(async () => {
+ await check('a robot made of direction buttons gets the joystick and the key hint, enabled only when it may press', () => {
+ for (const names of [['back', 'left', 'light', 'right', 'forward', 'lift_up'], ['forward', 'backward', 'turn_left', 'turn_right', 'head_up']]) {
+ const html = buttonsOf(relay(names));
+ assert.match(html, /<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses forward, back, left, right\. Drag and hold to drive\.">/, names.join());
+ assert.ok(html.includes('<kbd>W A S D</kbd> or arrows press forward, back, left, right.'), names.join());
+ assert.ok(html.indexOf('data-button-stick') < html.indexOf('class="button-grid"'), 'the stick above the buttons');
+ for (const n of names) assert.ok(html.includes(`data-button-name="${n}"`), `${n} is still its own button`);
+ assert.match(buttonsOf(relay(names), []), /data-button-stick[^>]*aria-disabled="true"/, 'disabled without the button kind');
+ }
+ // Two directions are enough (forward and back on a slider); one is not.
+ assert.ok(buttonsOf(relay(['forward', 'back'])).includes('Joystick: presses forward, back.'));
+ assert.ok(!buttonsOf(relay(['forward', 'horn'])).includes('data-button-stick'));
+ // OBS scenes are buttons with no direction: no stick.
+ assert.ok(!buttonsOf(relay(['scene_cam1', 'scene_cam2', 'scene_obs_test'])).includes('data-button-stick'));
+ // A robot that has a drive drives with that, never with its buttons.
+ const withDrive = relay(['forward', 'back', 'left', 'right']);
+ withDrive.widgets = [{ type: 'drive', label: 'Drive', order: 0, command: { kind: 'drive' } }, ...withDrive.widgets];
+ assert.ok(!buttonsOf(withDrive).includes('data-button-stick'));
+ });
+
+ await check('a pan-tilt camera gets the joystick for pan and tilt, its axes and zoom buttons kept', () => {
+ const cam = loadProfiles().get('camera.onvif');
+ const w = cam.widgets.find((x) => x.type === 'ptz');
+ const html = renderWidget(w, { profile: cam, allowed_commands: ['ptz'] });
+ const stick = /<div class="stick stick--ptz" data-ptz-stick="([^"]*)" data-axes="([^"]*)"[^>]*>/.exec(html);
+ assert.ok(stick, html.slice(0, 400));
+ const unq = (s) => JSON.parse(s.replace(/"/g, '"'));
+ assert.deepStrictEqual(unq(stick[1]), { x: 'pan', y: 'tilt' });
+ assert.deepStrictEqual(unq(stick[2]), cam.commands.ptz.axes);
+ assert.ok(html.includes('class="ptz-pad"') && html.includes('class="pad"'), 'the buttons stay, zoom among them');
+ assert.match(renderWidget(w, { profile: cam, allowed_commands: [] }), /data-ptz-stick[^>]*aria-disabled="true"/);
+ // Zoom alone has nothing to steer.
+ const zoomOnly = { ...cam, commands: { ...cam.commands, ptz: { axes: { zoom: [-1, 1] } } } };
+ assert.ok(!renderWidget(w, { profile: zoomOnly, allowed_commands: ['ptz'] }).includes('data-ptz-stick'));
+ });
+
+ await check('panel.js: keys, stick and gamepad press direction buttons; the PTZ stick holds pan and tilt', () => {
+ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');
+ assert.ok(js.includes("const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };"));
+ assert.ok(js.includes("const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;"), 'only with no drive');
+ assert.ok(/if \(buttonDrive && KEY_DIR\[e\.code\] && dirButton\(KEY_DIR\[e\.code\]\)\)/.test(js), 'keydown presses');
+ assert.ok(js.includes('if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);'), 'keyup releases');
+ assert.ok(/if \(el\.dataset\.buttonHold === 'true'\) \{ pressProfileButton\(el, source\); return true; \}/.test(js), 'a hold button is held');
+ assert.ok(js.includes('dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));'), 'a plain one repeats');
+ assert.ok(js.includes("holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });"));
+ assert.ok(js.includes("() => holdRelease(kind, 'ptzstick'));"), 'let go stops the camera');
+ assert.ok(js.includes("for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }"), 'releaseAll lets every repeat go');
+ assert.ok(/\[data-joystick\], \[data-button-stick\], \[data-ptz-stick\]/.test(js), 'the new sticks follow the allowed state');
+ // The pad on a button robot keeps the drive's latch, B lets go, Start is the e-stop.
+ const branch = js.slice(js.indexOf('if (buttonDrive) {', js.indexOf('function padPoll')), js.indexOf('const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);'));
+ assert.ok(branch.includes('if (!pad.latched) pad.reset();') && branch.includes('if (!want && !now.size) pad.latched = false;'));
+ assert.ok(branch.includes('now.has(1) && !pad.down.has(1)') && branch.includes('now.has(9) && !pad.down.has(9)'));
+ });
+
+ await check('panel.js: the page around the frame gets the controls state and may send keys and its gamepad, nothing else', () => {
+ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');
+ assert.ok(js.includes("const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };"));
+ assert.ok(js.includes('const keys = driving && joined ? controlKeys() : [];'), 'active only while driving');
+ assert.ok(js.includes('try { window.parent.postMessage(msg, origin); }'), 'only to the allowed origins');
+ assert.ok(/stopDriving\(text\) \{\n\s+driving = false; watcher = true; ticket = null;\n\s+announceControls\(\);/.test(js), 'ending the drive says so');
+ const listener = js.slice(js.indexOf("window.addEventListener('message', (e) => {"));
+ assert.ok(listener.indexOf("if (!fromPopup && !fromParent) return;") < listener.indexOf("d.type === 'ov-bot-key'"), 'from the allowed page only');
+ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {"));
+ assert.ok(listener.includes('if (d.down && !(driving && joined)) return;'), 'a press only while driving; a release always');
+ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {") && listener.includes('if (!(driving && joined)) return;'));
+ assert.ok(listener.includes('Math.max(-1, Math.min(1, Number(v)))') && listener.includes('.map((b) => b === true)'), 'the pad is clamped and typed');
+ assert.ok(js.includes('const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&'));
+ });
+
+ done();
+})();