openvibers/openvibe.bot Commits

Panel inputs: WASD, a gamepad and the joystick drive a robot made of buttons; the joystick on pan-tilt cameras; a framed panel takes its page's keys and gamepad (#171)

OP OpenVibers committed 17 days ago · ca2272ec39 · 6 files, +355 −17

* docs: the Live controls migration is done (relay plugin, Live's deletion), and what changed since for owners

* Panel inputs: a robot made of buttons drives from a joystick, WASD/arrows and a gamepad (its forward/back/left/right buttons); a pan-tilt camera gets the joystick; a framed panel takes its page's keys and gamepad while it drives (ov-bot-controls/key/pad)

Changes

ddiff --git a/docs/live-controls-migration.md b/docs/live-controls-migration.md
iindex 67f57c4..20bb59a 100644
---- a/docs/live-controls-migration.md
++++ b/docs/live-controls-migration.md
@@@ -1,7 +1,24 @@
# Moving OpenVibe.Live's stream controls into Bot (plan T15, row R9)
--Status: steps 1–3 built in Contracts 0.109.0 and Bot, and step 5 (the conversion) built in Bot, 2026-10-07.
--Live still serves its controls; step 4 (the Node relay plugin) and step 6 (Live's deletion) remain.
++Status: **done** (updated 2026-10-27). Every step below is built and in production:
++- steps 1–3 and 5: Contracts 0.109.0 and Bot, 2026-10-07;
++- step 4: OpenVibe.Node's `relay` plugin (`plugins/relay`);
++- step 6: Live deleted its controls (`0011_drop_live_controls`). Its old hardware socket `/ws/control` answers
++ **410 Gone**, naming OpenVibe.Bot, so a client that still connects there learns where to go
++ (Live `server/net/ws-paths.js`).
++
++What changed since, from the first owner to move (Maticus, 2026-10-26: "Buttons on the stream. That's like my number
++one priority"):
++- **One press puts the buttons under a stream.** The robot page's **On your stream** card ("Put the buttons under my
++ OpenVibe.Live stream") turns the embed on and opens the owner's channel to confirm the binding. Who may drive is
++ chosen beside it (Bot#169).
++- **Drive under the stream needs no second sign-in.** The panel asks the Live page around it for a ticket
++ (`POST /api/v1/robots/:id/control-ticket`).
++- **Chat commands reach a button robot.** `!forward`, `!left` … press the robot's buttons of the same name when it
++ has no drive, and any other `!word` that is not a channel sound presses the button of that name (Live#383,
++ Chat#194).
++- **The Node's lifecycle:** `openvibe-node quit` ends a running Node cleanly with nothing latched (Node 0.16.0), for
++ launchers that start and stop it with the rest of a stream setup. `stop` stays the kill switch.
## What Live has today
@@@ -45,7 +62,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
3. **Built — Bot gate and panel**: the gate accepts `button`/`point` only for names and shapes the profile
declares; the panel renders `buttons` (keyboard bindings, hold behaviour) and `video_click` (a click layer on the
camera tile, normalised coordinates).
--4. **Node — the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live's
++4. **Built — Node, the relay plugin**: forwards `button`/`point` commands to the owner's local script in the shape Live's
hardware clients already understand — one JSON line per command on stdout of a supervised child process, or a
loopback WebSocket that speaks Live's `{type:'command'|'key_held'|'key_released'|'video_click'}` messages — so an
existing Pi script keeps working with a one-line change of where it connects. Local policy and the kill switch
@@@ -63,7 +80,7 @@ lines) let viewers drive a streamer's hardware from the channel page:
and skipped (they must sign in to Network once). Idempotent on the Live config id (`live_conversions`,
migration 0007): a second `--apply` changes nothing and says "already converted". The owner then pairs a
Node (the five-minute path) and moves the script onto the relay plugin.
--6. **Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has a
++6. **Done — Live**: `LIVE_BOT_EMBED=1`; the channel page shows the robot's embedded panel for a stream whose owner has a
robot; then delete `server/controls/*`, the hardware WebSocket path and its stream-key auth, the controls tables
and `public/js/controls.js` / `onvif-dashboard.js` (the embed stays).
@@@ -106,10 +123,11 @@ if anything, needs redoing by hand.
is one line.
3. Point the Pi script at the paired Node instead of Live's stream-key socket. The stream key is no longer a
robot credential.
++4. Robot page → **On your stream** → **Put the buttons under my OpenVibe.Live stream**, and choose who may drive.
## Order and cutover
Contracts → robot-local profiles → gate/panel → relay plugin → conversion run (owners notified) → Live embed on →
owners re-pair → Live deletion. The conversion (step 5) is built and runs once; the deletion waits for the three
converted owners' robots to be paired, because the hardware clients are physical devices only their owners can
--move; it does not wait on a date.
++move; it does not wait on a date. (Done: Live's controls are gone, and its old socket answers 410 naming Bot.)
ddiff --git a/docs/protocol.md b/docs/protocol.md
iindex b333e7c..a3ec634 100644
---- a/docs/protocol.md
++++ b/docs/protocol.md
@@@ -369,6 +369,21 @@ a spent or expired one closes **4002**. A panel on another site gets one in two
takes an answer only from `window.parent` on one of them: `{ type: "ov-bot-ticket", robot_id, ticket }`, or
`{ type: "ov-bot-ticket-none", robot_id, message }` (shown as is: sign in, or the owner decides who drives).
With no answer in 2.5 s the next Drive opens the popup.
++
++While a framed panel drives, the page around it can drive with its own keyboard and gamepad. This is for when the
++person is on the page and not in the frame (a Live channel's player).
++- The panel tells each `BOT_EMBED_ORIGINS` origin when its controls go live, change or end:
++ `{ type: "ov-bot-controls", robot_id, active, keys, pad }`.
++ - `keys` lists the `KeyboardEvent.code`s it takes: the drive keys, WASD and the arrows on a robot made of buttons
++ (they press its forward/back/left/right buttons), and the profile's button keys.
++ - `active: false`, with no keys, ends it.
++- The page sends those keys as `{ type: "ov-bot-key", robot_id, code, key, down, repeat }`, never while the person
++ types.
++- It sends its gamepad at most about 30 times a second as `{ type: "ov-bot-pad", robot_id, axes, buttons }`: axes
++ -1…1 and buttons true while pressed. Pad input that has not arrived for 500 ms counts as let go.
++- The panel takes these messages only from `window.parent` on an allowed origin, for its own robot, and a press only
++ while it drives. They go through the same handlers and the same gate as its own input. A key's release always
++ goes through.
The gate then applies the **acted-for subject's** role — a service acting for a viewer is a viewer.
The same gate is reachable over HTTP as `POST /robots/:id/commands` (§3), so a bound channel's chat can
forward commands without holding a socket. The HTTP gate is the same gate but not the same join: a caller
ddiff --git a/public/panel.css b/public/panel.css
iindex 8bda658..1e76225 100644
---- a/public/panel.css
++++ b/public/panel.css
@@@ -125,6 +125,11 @@ main { padding: 1rem max(1rem, env(safe-area-inset-right)) calc(1.5rem + env(saf
.stick.held .stick-knob { transition: none; }
.stick.active .stick-knob { background: radial-gradient(circle at 40% 35%, #7fb2ff, var(--accent) 70%); border-color: #8fbaff; }
.stick[aria-disabled="true"] { opacity: 0.4; cursor: not-allowed; }
++/* The same joystick on a robot made of buttons (it presses forward, back, left and right) and on a pan-tilt camera. */
++.stick--buttons, .stick--ptz { width: min(12rem, 60vw); }
++.stick--buttons { margin: 0 auto; }
++.stick--buttons + .hint { margin-bottom: 0.9rem; }
++.ptz-pad { display: grid; justify-items: center; gap: 0.9rem; }
.pad { display: grid; gap: 0.5rem; width: 100%; max-width: 22rem; }
.axis { display: grid; grid-template-columns: 1fr auto 1fr; gap: 0.5rem; align-items: center; }
.axis .hold { min-height: 56px; user-select: none; -webkit-user-select: none; touch-action: none; }
ddiff --git a/public/panel.js b/public/panel.js
iindex 2e1eceb..5f973c6 100644
---- a/public/panel.js
++++ b/public/panel.js
@@@ -434,6 +434,7 @@
for (const el of [...tones.keys()]) releaseTone(el);
for (const [el, h] of [...buttonHolds]) for (const source of [...h.sources]) releaseProfileButton(el, source);
for (const s of sticks) s.release();
++ for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }
pad.reset();
for (const el of $$('.held')) el.classList.remove('held');
}
@@@ -512,6 +513,102 @@
sticks.push({ release });
}
++ // ── A robot made of buttons drives too (Maticus, 2026-10-27: "WASD keyboard AND game controller … that cool virtual
++ // joystick from the sim") ──────────────────────────────────────────────────────────────────────────────────
++ // A relay robot's direction buttons (forward, backward, turn_left, turn_right …; render.js DIRECTION_BUTTONS) are
++ // pressed from WASD and the arrows, its joystick, and a gamepad's stick and D-pad, when the robot has no drive of its
++ // own. A hold button is held while the input is; a plain one is pressed again every REPEAT_MS while it is.
++ const DIR_NAMES = { up: ['forward', 'drive_forward', 'up'], down: ['backward', 'back', 'reverse', 'down'], left: ['turn_left', 'left', 'rotate_left'], right: ['turn_right', 'right', 'rotate_right'] };
++ const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };
++ const REPEAT_MS = 300;
++ const dirButton = (dir) => (DIR_NAMES[dir] || []).map((n) => $(`[data-button-name="${n}"]`)).find(Boolean) || null;
++ const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;
++ const dirRepeats = new Map(); // `${dir}|${source}` → the timer re-pressing a plain button
++ function dirPress(dir, source) {
++ const el = dirButton(dir);
++ if (!el || el.disabled || !joined || !allowed.includes('button')) return false;
++ if (el.dataset.buttonHold === 'true') { pressProfileButton(el, source); return true; }
++ const k = `${dir}|${source}`;
++ if (dirRepeats.has(k)) return true;
++ el.classList.add('held');
++ pressProfileButton(el, source);
++ dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));
++ return true;
++ }
++ function dirRelease(dir, source) {
++ const el = dirButton(dir);
++ if (!el) return;
++ if (el.dataset.buttonHold === 'true') { releaseProfileButton(el, source); return; }
++ const k = `${dir}|${source}`;
++ if (!dirRepeats.has(k)) return;
++ clearInterval(dirRepeats.get(k));
++ dirRepeats.delete(k);
++ if (![...dirRepeats.keys()].some((x) => x.startsWith(`${dir}|`))) el.classList.remove('held');
++ }
++ /** A stick position (x right, y up) → the direction it points in once past half way, or null. */
++ const stickDir = (x, y) => (Math.hypot(x, y) < 0.5 ? null : Math.abs(x) > Math.abs(y) ? (x > 0 ? 'right' : 'left') : (y > 0 ? 'up' : 'down'));
++ /** Where a pointer is on a stick: x right, y up, both -1…1, the knob drawn there. */
++ function stickAt(el, e) {
++ const r = el.getBoundingClientRect();
++ const radius = r.width / 2;
++ let x = (e.clientX - (r.left + radius)) / radius;
++ let y = -(e.clientY - (r.top + r.height / 2)) / radius;
++ const m = Math.hypot(x, y);
++ if (m > 1) { x /= m; y /= m; }
++ const knob = $('[data-knob]', el);
++ if (knob) knob.style.transform = `translate(${x * 75}%, ${-y * 75}%)`;
++ return { x, y };
++ }
++ /** A joystick other than the drive's: onMove({ x, y }) while dragged, onEnd() when let go of (or released). */
++ function extraStick(el, onMove, onEnd) {
++ let pointer = null;
++ const end = () => {
++ if (pointer == null) return;
++ pointer = null;
++ el.classList.remove('held', 'active');
++ const knob = $('[data-knob]', el);
++ if (knob) knob.style.transform = '';
++ onEnd();
++ };
++ el.addEventListener('pointerdown', (e) => {
++ if (el.getAttribute('aria-disabled') === 'true' || pointer != null) return;
++ e.preventDefault();
++ pointer = e.pointerId;
++ try { el.setPointerCapture(e.pointerId); } catch { /* not capturable */ }
++ el.classList.add('held', 'active');
++ onMove(stickAt(el, e));
++ });
++ el.addEventListener('pointermove', (e) => { if (e.pointerId === pointer) onMove(stickAt(el, e)); });
++ for (const ev of ['pointerup', 'pointercancel', 'lostpointercapture']) el.addEventListener(ev, (e) => { if (e.pointerId === pointer) end(); });
++ el.addEventListener('contextmenu', (e) => e.preventDefault());
++ sticks.push({ release: end });
++ }
++ const buttonStick = $('[data-button-stick]');
++ if (buttonStick) {
++ let dir = null;
++ extraStick(buttonStick, ({ x, y }) => {
++ const d = stickDir(x, y);
++ if (d === dir) return;
++ if (dir) dirRelease(dir, 'bstick');
++ dir = d;
++ if (d) dirPress(d, 'bstick');
++ }, () => { if (dir) dirRelease(dir, 'bstick'); dir = null; });
++ }
++ // A camera's pan and tilt from a joystick (the PTZ widget): the stick's lean is the speed, let go stops it.
++ const ptzStick = $('[data-ptz-stick]');
++ if (ptzStick) {
++ const kind = kindOf(ptzStick) || 'ptz';
++ const axes = JSON.parse(ptzStick.dataset.axes || '{}');
++ const map = JSON.parse(ptzStick.dataset.ptzStick);
++ const at = (axis, u) => { const r = axes[axis]; return r ? round(u >= 0 ? u * r[1] : -u * r[0]) : 0; };
++ extraStick(ptzStick, ({ x, y }) => {
++ const r = Math.hypot(x, y);
++ const k = r <= STICK_DEADZONE ? 0 : Math.min(1, (r - STICK_DEADZONE) / (1 - STICK_DEADZONE)) / r;
++ if (!joined || !allowed.includes(kind)) return;
++ holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });
++ }, () => holdRelease(kind, 'ptzstick'));
++ }
++
// ── Keyboard: arrows (and WASD) drive; Q/E rotate; Space stops ────────────────────────────────────
const KEYS = {
ArrowUp: [['throttle', 'y'], 1], KeyW: [['throttle', 'y'], 1], ArrowDown: [['throttle', 'y'], 0], KeyS: [['throttle', 'y'], 0],
@@@ -541,9 +638,10 @@
holdRelease(DRIVE, source);
}
const typing = (e) => /^(INPUT|SELECT|TEXTAREA)$/.test(e.target.tagName);
-- const profileKey = (e) => main.contains(document.activeElement) &&
++ // A key from the page around the frame (e.remote) was already checked there: not while typing, only while driving.
++ const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&
$$('[data-button-key]').find((el) => !el.disabled && (el.dataset.buttonKey === e.code || el.dataset.buttonKey.toLowerCase() === e.key.toLowerCase()));
-- document.addEventListener('keydown', (e) => {
++ function onKeyDown(e) {
if (typing(e)) return;
const focusedButton = e.target.closest && e.target.closest('[data-button-name]');
if (focusedButton && (e.code === 'Enter' || e.code === 'Space')) {
@@@ -552,13 +650,18 @@
// Stop and the drive keys first: a button's key never shadows them.
if (e.code === 'Space') { e.preventDefault(); stopDrive(); return; }
if (keyValue(e.code)) { e.preventDefault(); if (!e.repeat) keyPress(e.code, `key:${e.code}`); return; }
++ // A robot made of buttons: the drive keys press its direction buttons.
++ if (buttonDrive && KEY_DIR[e.code] && dirButton(KEY_DIR[e.code])) { e.preventDefault(); if (!e.repeat) dirPress(KEY_DIR[e.code], `key:${e.code}`); return; }
const button = profileKey(e);
if (button) { e.preventDefault(); if (!e.repeat) pressProfileButton(button, `key:${e.code}`); }
-- });
-- document.addEventListener('keyup', (e) => {
++ }
++ function onKeyUp(e) {
for (const [el, h] of buttonHolds) if (h.sources.has(`key:${e.code}`)) releaseProfileButton(el, `key:${e.code}`);
if (KEYS[e.code]) keyRelease(e.code, `key:${e.code}`);
-- });
++ if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);
++ }
++ document.addEventListener('keydown', onKeyDown);
++ document.addEventListener('keyup', onKeyUp);
function stopDrive() {
const h = holds.get(DRIVE);
@@@ -581,23 +684,61 @@
// After a reset (blur, a hidden page, the e-stop) nothing is pressed again until the pad is let go of
// entirely: a stick still pushed or a button still held does not undo a release.
latched: false,
++ dir: null, // the direction button the pad holds (a robot made of buttons)
reset() {
holdRelease(DRIVE, 'pad');
++ if (this.dir) { dirRelease(this.dir, 'pad'); this.dir = null; }
for (const [b, code] of Object.entries(PAD_DPAD)) if (this.down.has(Number(b))) keyRelease(code, `pad:${b}`);
if (this.tone) { releaseTone(this.tone); this.tone = null; }
this.latched = true;
},
};
++ // The page around the frame (a Live channel) sends its gamepad while this panel drives and the page, not the frame,
++ // has the focus (Live public/js/bot-embed.js): { axes, buttons: [pressed] }, at most about 30 times a second.
++ let remotePad = null;
++ const REMOTE_PAD_MS = 500;
++ function gamepad() {
++ const local = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);
++ if (local && document.hasFocus()) return { pad: local, focused: true };
++ if (remotePad && performance.now() - remotePad.at < REMOTE_PAD_MS) {
++ return { pad: { axes: remotePad.axes, buttons: remotePad.buttons.map((b) => ({ pressed: !!b })), connected: true }, focused: true };
++ }
++ return local ? { pad: local, focused: document.hasFocus() } : null;
++ }
function padPoll() {
pad.loop = 0;
-- const p = [...(navigator.getGamepads ? navigator.getGamepads() : [])].find((g) => g && g.connected);
++ const g = gamepad();
++ const p = g && g.pad;
if (!p) { pad.reset(); pad.down.clear(); return; } // the last pad went: stop polling
const ax = (i) => Number(p.axes[i]) || 0;
++ const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));
++ if (buttonDrive) {
++ // A robot made of buttons: the left stick or the D-pad points at a direction, and its button is pressed.
++ // Like the drive: after a reset nothing is pressed again until the pad is let go of; B lets go; Start is the e-stop.
++ const ok = g.focused && !document.hidden && joined && allowed.includes('button');
++ let want = stickDir(dead(ax(0), PAD_DEADZONE), -dead(ax(1), PAD_DEADZONE));
++ if (!want) for (const [b, d] of Object.entries({ 12: 'up', 13: 'down', 14: 'left', 15: 'right' })) if (now.has(Number(b))) { want = d; break; }
++ if (!ok) {
++ if (!pad.latched) pad.reset();
++ } else if (pad.latched) {
++ if (!want && !now.size) pad.latched = false;
++ } else if (now.has(9) && !pad.down.has(9) && $('[data-estop]:not([hidden])')) {
++ estop();
++ } else if (now.has(1) && !pad.down.has(1)) {
++ pad.reset();
++ } else if (want !== pad.dir) {
++ if (pad.dir) dirRelease(pad.dir, 'pad');
++ pad.dir = want;
++ if (want) dirPress(want, 'pad');
++ }
++ pad.down = now;
++ pad.loop = requestAnimationFrame(padPoll);
++ return;
++ }
const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);
if (ROTATE) v[ROTATE] = scale(ROTATE, dead(ax(2), PAD_DEADZONE));
const moving = Object.values(v).some((x) => x !== 0);
-- const now = new Set(p.buttons.map((b, i) => (b && b.pressed ? i : -1)).filter((i) => i >= 0));
-- if (!document.hasFocus() || document.hidden || !driveEnabled()) {
++ if (!g.focused || document.hidden || !driveEnabled()) {
if (!pad.latched) pad.reset();
} else if (pad.latched) {
if (!moving && !now.size) pad.latched = false;
@@@ -695,7 +836,30 @@
for (const section of $$('[data-command]')) {
const ok = joined && allowed.includes(JSON.parse(section.dataset.command).kind);
for (const el of $$('button, input, select', section)) el.disabled = !ok;
-- for (const el of $$('[data-joystick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }
++ for (const el of $$('[data-joystick], [data-button-stick], [data-ptz-stick]', section)) { if (ok) el.removeAttribute('aria-disabled'); else el.setAttribute('aria-disabled', 'true'); }
++ }
++ announceControls();
++ }
++ // Framed by an allowed page (a Live channel), this panel tells it while its controls are live and which keys they
++ // take, so the page sends those keys and its gamepad here when the person is on the page and not in the frame
++ // (Maticus, 2026-10-27: "WASD keyboard AND game controller" on the stream page). Live public/js/bot-embed.js.
++ let announced = '';
++ function controlKeys() {
++ const keys = new Set();
++ if (driveEnabled()) { for (const code of Object.keys(KEYS)) if (keyValue(code)) keys.add(code); keys.add('Space'); }
++ if (buttonDrive && joined && allowed.includes('button')) for (const [code, dir] of Object.entries(KEY_DIR)) if (dirButton(dir)) keys.add(code);
++ for (const el of $$('[data-button-key]')) if (!el.disabled) keys.add(el.dataset.buttonKey);
++ return [...keys];
++ }
++ function announceControls() {
++ if (!framed) return;
++ const keys = driving && joined ? controlKeys() : [];
++ const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };
++ const text = JSON.stringify(msg);
++ if (text === announced) return;
++ announced = text;
++ for (const origin of PARENTS) {
++ try { window.parent.postMessage(msg, origin); } catch { /* not that origin */ }
}
}
// What a device's refusal means, in words (a nack's fault code). A deadline that passed in transit is not news.
@@@ -905,6 +1069,7 @@
// ── The embed's Drive ─────────────────────────────────────────────────────────────────────────────
function stopDriving(text) {
driving = false; watcher = true; ticket = null;
++ announceControls();
for (const el of $$('[data-estop]')) el.hidden = true;
const label = $('[data-role-label]');
if (label) label.textContent = 'Watching';
@@@ -942,6 +1107,23 @@
const fromParent = framed && e.source === window.parent && PARENTS.includes(e.origin);
if (!fromPopup && !fromParent) return;
const d = e.data || {};
++ // The page's keys and gamepad while this panel drives (announceControls): the same handlers as this frame's own.
++ // A key's release always goes through, so nothing stays held when driving ends between press and release.
++ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {
++ if (d.down && !(driving && joined)) return;
++ const ev = { code: d.code.slice(0, 40), key: typeof d.key === 'string' ? d.key.slice(0, 20) : '', repeat: d.repeat === true,
++ remote: true, target: document.body, preventDefault() {} };
++ if (d.down) onKeyDown(ev); else onKeyUp(ev);
++ return;
++ }
++ if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {
++ if (!(driving && joined)) return;
++ const unit = (v) => (Number.isFinite(Number(v)) ? Math.max(-1, Math.min(1, Number(v))) : 0);
++ remotePad = { axes: (Array.isArray(d.axes) ? d.axes.slice(0, 8) : []).map(unit),
++ buttons: (Array.isArray(d.buttons) ? d.buttons.slice(0, 20) : []).map((b) => b === true), at: performance.now() };
++ padStart();
++ return;
++ }
if (fromParent && d.type === 'ov-bot-ticket-none' && d.robot_id === ROBOT_ID) {
clearTimeout(asked);
notice(typeof d.message === 'string' ? d.message.slice(0, 200) : 'You cannot drive this robot from here.');
ddiff --git a/server/web/render.js b/server/web/render.js
iindex 99c430c..2f40fab 100644
---- a/server/web/render.js
++++ b/server/web/render.js
@@@ -104,6 +104,14 @@ function axisButtons(axes, enabled) {
}).join('');
}
++/** The buttons that mean a direction on a robot made of buttons (panel.js presses them from keys, a stick and a pad). */
++const DIRECTION_BUTTONS = [
++ { dir: 'up', label: 'forward', names: ['forward', 'drive_forward', 'up'] },
++ { dir: 'down', label: 'back', names: ['backward', 'back', 'reverse', 'down'] },
++ { dir: 'left', label: 'left', names: ['turn_left', 'left', 'rotate_left'] },
++ { dir: 'right', label: 'right', names: ['turn_right', 'right', 'rotate_right'] },
++];
++
/** The drive pad: a touch joystick for two axes (pointer events, see panel.js), hold buttons for the rest. */
function drivePad(axes, enabled) {
const x = STICK_X.find((a) => axes[a]);
@@@ -200,7 +208,13 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
}
case 'ptz': {
const axes = (commands[kind] && commands[kind].axes) || {};
-- inner = `<div class="pad">${axisButtons(axes, enabled)}</div>`;
++ // A camera that pans and tilts gets the joystick too (Maticus, 2026-10-27: "that cool virtual joystick from the
++ // sim … would work killer for LOOI or any PTZ"): drag to move, let go to stop; zoom stays on its buttons.
++ const ptzStick = axes.pan && axes.tilt
++ ? `<div class="stick stick--ptz" data-ptz-stick="${data({ x: 'pan', y: 'tilt' })}" data-axes="${data(axes)}" role="img" aria-label="Joystick: pan and tilt. Drag and hold to move the camera."${enabled ? '' : ' aria-disabled="true"'}>`
++ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'
++ : '';
++ inner = `<div class="ptz-pad">${ptzStick}<div class="pad">${axisButtons(axes, enabled)}</div></div>`;
break;
}
case 'pan-tilt': case 'servo': case 'head': case 'lift': case 'lights': case 'horn': {
@@@ -241,7 +255,16 @@ function renderWidget(w, { profile = {}, allowed_commands = [], signaling_url =
break;
case 'buttons': {
const names = (commands.button && commands.button.names) || {};
-- inner = `<div class="button-grid">${Object.entries(names).map(([name, button]) =>
++ // A robot made of buttons (a relay profile: forward, backward, turn_left, turn_right …) with no drive of its own
++ // drives from a joystick, the keyboard and a gamepad too: each direction presses its button (panel.js).
++ const dirs = DIRECTION_BUTTONS.filter((d) => d.names.some((n) => names[n]));
++ const hasDrive = (profile.widgets || []).some((x) => x.type === 'drive');
++ const buttonStick = dirs.length >= 2 && !hasDrive
++ ? `<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses ${esc(dirs.map((d) => d.label).join(', '))}. Drag and hold to drive."${enabled ? '' : ' aria-disabled="true"'}>`
++ + '<span class="stick-ring"></span><span class="stick-cross"></span><span class="stick-knob" data-knob></span></div>'
++ + `<p class="hint"><kbd>W A S D</kbd> or arrows press ${esc(dirs.map((d) => d.label).join(', '))}. <span class="pad-hint" data-pad-hint>${icon('gamepad')} A gamepad's stick and D-pad do too.</span></p>`
++ : '';
++ inner = `${buttonStick}<div class="button-grid">${Object.entries(names).map(([name, button]) =>
`<button type="button" class="hold pad-button" data-button-name="${esc(name)}" data-button-hold="${button.hold ? 'true' : 'false'}"${button.key ? ` data-button-key="${esc(button.key)}"` : ''}${disabledUnless(enabled)}>`
+ `<span>${esc(button.label)}</span>${button.key ? `<kbd>${esc(button.key)}</kbd>` : ''}</button>`).join('')}</div>`;
break;
@@@ -407,7 +430,7 @@ function localProfileForm(robot, profile, settings = {}) {
</div>
</fieldset>`).join('');
return `<section class="setting-card" aria-labelledby="buttons-h"><h2 id="buttons-h">Buttons</h2>
--<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving.</p>
++<p class="setting-note">Each button sends its name to the robot. A key works while the panel has focus. Space always stops, and on a robot that drives its drive keys stay with driving. Buttons named forward, back (or backward), left (or turn_left) and right (or turn_right) also get a joystick, and W A S D, the arrows and a gamepad press them.</p>
${settings.error ? `<p class="error" role="alert">${esc(settings.error)}</p>` : ''}
<form method="post" action="/robots/${esc(robot.id)}/profile" data-profile-form>
<input type="hidden" name="count" value="${shown.length}">
ddiff --git a/test/panel-inputs.test.js b/test/panel-inputs.test.js
nnew file mode 100644
iindex 0000000..5a0d4a3
---- /dev/null
++++ b/test/panel-inputs.test.js
@@@ -0,0 +1,95 @@
++'use strict';
++// Every way to drive from the panel (Maticus, 2026-10-27: "getting those buttons on openvibe.live stream page (along with
++// WASD keyboard AND GaMe cOnTrOlLeR!! … maybe we can add that cool virtual joystick from the sim … would work killer for
++// LOOI or any PTZ"):
++// - A robot made of buttons (a relay profile: Cozmo's forward/back/left/right, Cozmo Hobo's forward/backward/turn_left/
++// turn_right) with no drive of its own gets the joystick, and WASD, the arrows and a gamepad press its direction buttons.
++// - A pan-tilt camera gets the joystick for pan and tilt; zoom stays on its buttons.
++// - Framed by an allowed page (a Live channel), the panel says while its controls are live and which keys they take
++// (ov-bot-controls), and takes that page's keys (ov-bot-key) and gamepad (ov-bot-pad) only from it, for its robot, while
++// it drives; a key's release always goes through.
++const fs = require('fs');
++const path = require('path');
++const assert = require('assert');
++const { check, done } = require('./helpers/app');
++const { loadProfiles } = require('../server/profiles');
++const { renderWidget } = require('../server/web/render');
++
++const relay = (names) => {
++ const base = loadProfiles().get('relay.generic');
++ return { ...base, commands: { ...base.commands, button: { names: Object.fromEntries(names.map((n) => [n, { label: n }])) } } };
++};
++const buttonsOf = (profile, allowed = ['button']) => renderWidget(profile.widgets.find((w) => w.type === 'buttons'), { profile, allowed_commands: allowed });
++
++(async () => {
++ await check('a robot made of direction buttons gets the joystick and the key hint, enabled only when it may press', () => {
++ for (const names of [['back', 'left', 'light', 'right', 'forward', 'lift_up'], ['forward', 'backward', 'turn_left', 'turn_right', 'head_up']]) {
++ const html = buttonsOf(relay(names));
++ assert.match(html, /<div class="stick stick--buttons" data-button-stick role="img" aria-label="Joystick: presses forward, back, left, right\. Drag and hold to drive\.">/, names.join());
++ assert.ok(html.includes('<kbd>W A S D</kbd> or arrows press forward, back, left, right.'), names.join());
++ assert.ok(html.indexOf('data-button-stick') < html.indexOf('class="button-grid"'), 'the stick above the buttons');
++ for (const n of names) assert.ok(html.includes(`data-button-name="${n}"`), `${n} is still its own button`);
++ assert.match(buttonsOf(relay(names), []), /data-button-stick[^>]*aria-disabled="true"/, 'disabled without the button kind');
++ }
++ // Two directions are enough (forward and back on a slider); one is not.
++ assert.ok(buttonsOf(relay(['forward', 'back'])).includes('Joystick: presses forward, back.'));
++ assert.ok(!buttonsOf(relay(['forward', 'horn'])).includes('data-button-stick'));
++ // OBS scenes are buttons with no direction: no stick.
++ assert.ok(!buttonsOf(relay(['scene_cam1', 'scene_cam2', 'scene_obs_test'])).includes('data-button-stick'));
++ // A robot that has a drive drives with that, never with its buttons.
++ const withDrive = relay(['forward', 'back', 'left', 'right']);
++ withDrive.widgets = [{ type: 'drive', label: 'Drive', order: 0, command: { kind: 'drive' } }, ...withDrive.widgets];
++ assert.ok(!buttonsOf(withDrive).includes('data-button-stick'));
++ });
++
++ await check('a pan-tilt camera gets the joystick for pan and tilt, its axes and zoom buttons kept', () => {
++ const cam = loadProfiles().get('camera.onvif');
++ const w = cam.widgets.find((x) => x.type === 'ptz');
++ const html = renderWidget(w, { profile: cam, allowed_commands: ['ptz'] });
++ const stick = /<div class="stick stick--ptz" data-ptz-stick="([^"]*)" data-axes="([^"]*)"[^>]*>/.exec(html);
++ assert.ok(stick, html.slice(0, 400));
++ const unq = (s) => JSON.parse(s.replace(/&quot;/g, '"'));
++ assert.deepStrictEqual(unq(stick[1]), { x: 'pan', y: 'tilt' });
++ assert.deepStrictEqual(unq(stick[2]), cam.commands.ptz.axes);
++ assert.ok(html.includes('class="ptz-pad"') && html.includes('class="pad"'), 'the buttons stay, zoom among them');
++ assert.match(renderWidget(w, { profile: cam, allowed_commands: [] }), /data-ptz-stick[^>]*aria-disabled="true"/);
++ // Zoom alone has nothing to steer.
++ const zoomOnly = { ...cam, commands: { ...cam.commands, ptz: { axes: { zoom: [-1, 1] } } } };
++ assert.ok(!renderWidget(w, { profile: zoomOnly, allowed_commands: ['ptz'] }).includes('data-ptz-stick'));
++ });
++
++ await check('panel.js: keys, stick and gamepad press direction buttons; the PTZ stick holds pan and tilt', () => {
++ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');
++ assert.ok(js.includes("const KEY_DIR = { ArrowUp: 'up', KeyW: 'up', ArrowDown: 'down', KeyS: 'down', ArrowLeft: 'left', KeyA: 'left', ArrowRight: 'right', KeyD: 'right' };"));
++ assert.ok(js.includes("const buttonDrive = !Object.keys(driveAxes).length && Object.keys(DIR_NAMES).filter(dirButton).length >= 2;"), 'only with no drive');
++ assert.ok(/if \(buttonDrive && KEY_DIR\[e\.code\] && dirButton\(KEY_DIR\[e\.code\]\)\)/.test(js), 'keydown presses');
++ assert.ok(js.includes('if (KEY_DIR[e.code]) dirRelease(KEY_DIR[e.code], `key:${e.code}`);'), 'keyup releases');
++ assert.ok(/if \(el\.dataset\.buttonHold === 'true'\) \{ pressProfileButton\(el, source\); return true; \}/.test(js), 'a hold button is held');
++ assert.ok(js.includes('dirRepeats.set(k, setInterval(() => pressProfileButton(el, source), REPEAT_MS));'), 'a plain one repeats');
++ assert.ok(js.includes("holdSet(kind, 'ptzstick', { [map.x]: at(map.x, x * k), [map.y]: at(map.y, y * k) });"));
++ assert.ok(js.includes("() => holdRelease(kind, 'ptzstick'));"), 'let go stops the camera');
++ assert.ok(js.includes("for (const k of [...dirRepeats.keys()]) { const i = k.indexOf('|'); dirRelease(k.slice(0, i), k.slice(i + 1)); }"), 'releaseAll lets every repeat go');
++ assert.ok(/\[data-joystick\], \[data-button-stick\], \[data-ptz-stick\]/.test(js), 'the new sticks follow the allowed state');
++ // The pad on a button robot keeps the drive's latch, B lets go, Start is the e-stop.
++ const branch = js.slice(js.indexOf('if (buttonDrive) {', js.indexOf('function padPoll')), js.indexOf('const v = stickValue(ax(0), -ax(1), PAD_DEADZONE);'));
++ assert.ok(branch.includes('if (!pad.latched) pad.reset();') && branch.includes('if (!want && !now.size) pad.latched = false;'));
++ assert.ok(branch.includes('now.has(1) && !pad.down.has(1)') && branch.includes('now.has(9) && !pad.down.has(9)'));
++ });
++
++ await check('panel.js: the page around the frame gets the controls state and may send keys and its gamepad, nothing else', () => {
++ const js = fs.readFileSync(path.join(__dirname, '..', 'public', 'panel.js'), 'utf8');
++ assert.ok(js.includes("const msg = { type: 'ov-bot-controls', robot_id: ROBOT_ID, active: keys.length > 0, keys, pad: keys.length > 0 };"));
++ assert.ok(js.includes('const keys = driving && joined ? controlKeys() : [];'), 'active only while driving');
++ assert.ok(js.includes('try { window.parent.postMessage(msg, origin); }'), 'only to the allowed origins');
++ assert.ok(/stopDriving\(text\) \{\n\s+driving = false; watcher = true; ticket = null;\n\s+announceControls\(\);/.test(js), 'ending the drive says so');
++ const listener = js.slice(js.indexOf("window.addEventListener('message', (e) => {"));
++ assert.ok(listener.indexOf("if (!fromPopup && !fromParent) return;") < listener.indexOf("d.type === 'ov-bot-key'"), 'from the allowed page only');
++ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-key' && typeof d.code === 'string') {"));
++ assert.ok(listener.includes('if (d.down && !(driving && joined)) return;'), 'a press only while driving; a release always');
++ assert.ok(listener.includes("if (fromParent && d.robot_id === ROBOT_ID && d.type === 'ov-bot-pad') {") && listener.includes('if (!(driving && joined)) return;'));
++ assert.ok(listener.includes('Math.max(-1, Math.min(1, Number(v)))') && listener.includes('.map((b) => b === true)'), 'the pad is clamped and typed');
++ assert.ok(js.includes('const profileKey = (e) => (e.remote || main.contains(document.activeElement)) &&'));
++ });
++
++ done();
++})();