openvibers/openvibe.link Commits
Docs: protocol, plugins (with the Actor attach point), install; README
Changes
@@@ -1,5 +1,45 @@# OpenVibe Node--One service for your computers, servers, Raspberry Pis, phones and robots: pair it once with OpenVibe and products like OpenVibe.Bot (robots) and OpenVibe.Actor (AI agents) can use the device safely. It connects out, needs no open ports, and keeps a local kill switch.++One service for your computers, servers, Raspberry Pis, phones and robots: pair it once with OpenVibe and products++like **OpenVibe.Bot** (robots: drive, cameras, sensors, with safety) and, later, **OpenVibe.Actor** (computer control++for AI agents) can use the device. It connects out, needs no open ports, and keeps a local kill switch.--Status: being built. Start here: [docs/HANDOFF.md](docs/HANDOFF.md) and [docs/ADR-043-bot-devices-and-control.md](docs/ADR-043-bot-devices-and-control.md).++```sh++curl -fsSL https://openvibe.bot/install | sh -s -- ABCD-1234 --robot adeept++```++++Get the code on openvibe.bot. `--robot` is `adeept`, `adeept-mecanum`, `cozmo` or `none` (dry run). Details, manual++install, Windows and configuration: [docs/install.md](docs/install.md).++++```sh++openvibe-node status # link, plugins, video, battery, stop latch++openvibe-node stop # kill switch: stop every motor now and keep it stopped++openvibe-node resume++```++++## What is in here++++- **Core** (Go, one static binary per platform): pairing, one outbound WebSocket to `wss://openvibe.bot/device`++ (credential in the `Authorization` header only), heartbeats and a deadman, the safety rules (deadlines, owner limits++ clamped before any driver sees a value, a persisted e-stop, the local kill switch), the plugin supervisor, WHIP video++ publishing with pion, and the system service (systemd, launchd, Windows).++- **Plugins** (separate processes, JSON lines over stdin/stdout): [`dryrun`](plugins/dryrun), the Adeept 4WD Smart Car++ Kit [`adeept_adr036`](plugins/adeept_adr036/README.md), and [`cozmo`](plugins/cozmo/README.md) through PyCozmo. Each++ plugin stops its own actuators when stdin closes, at a command's deadline and after 1 s without a heartbeat, so the++ core dying never leaves motors running.++++Docs: [protocol](docs/protocol.md) (the wire format with OpenVibe.Bot), [plugins](docs/plugins.md) (the driver++protocol, the bundled drivers, and where Actor's computer-control plugin attaches), [install](docs/install.md).++Decisions: [ADR-043](docs/ADR-043-bot-devices-and-control.md).++++## Development++++```sh++go vet ./... && go test -race ./... # the end-to-end tests run the Python dry-run plugin++pip install pytest pillow numpy++for p in sdk dryrun adeept_adr036 cozmo; do python3 -m pytest -q plugins/$p/tests; done++scripts/dist.sh v0.0.0-dev # every binary, the plugin bundle and SHA256SUMS in dist/++```++++`internal/fakebot` is a fake OpenVibe.Bot (pairing, device WebSocket, WHIP) used by the tests. Licence: AGPL-3.0++([LICENSE](LICENSE)). Security reports: [SECURITY.md](SECURITY.md).nnew file mode 100644@@@ -0,0 +1,144 @@++# Installing the OpenVibe Node++++The Node is one program, `openvibe-node`, plus Python plugins for robots. It connects **out** to OpenVibe (no open++ports, works behind any home router), and a local kill switch stops everything it drives.++++## Quick start (Linux, Raspberry Pi, macOS)++++1. On openvibe.bot, add a robot. You get a code like `ABCD-1234` (valid 10 minutes).++2. On the device:++++ ```sh++ curl -fsSL https://openvibe.bot/install | sh -s -- ABCD-1234 --robot adeept++ ```++++ `--robot` is `adeept` (Adeept 4WD Smart Car, ordinary wheels), `adeept-mecanum`, `cozmo`, or `none` (the dry-run++ plugin with a test-pattern camera, for trying the Node on a laptop).++3. Confirm the new device on openvibe.bot.++++The installer detects the OS and CPU (linux amd64/arm64/armv7, macOS amd64/arm64), downloads the binary and the plugin++bundle and checks them against `SHA256SUMS`, creates a Python virtual environment for the plugins, writes++`config.json`, disables the Adeept kit's stock server if it finds it, creates the `openvibe-node` service account++(Linux; with the `gpio`, `i2c`, `spi`, `video`, `audio` groups), installs and starts the service, and pairs. Running it++again upgrades and keeps the credential. Options: `--no-service`, `--version vX.Y.Z`, `--local DIR` (install from a++directory holding the release files). `OPENVIBE_NODE_BASE` overrides the download location, `OPENVIBE_SERVER` the++server.++++Requirements: `python3` with `venv` (`sudo apt install python3 python3-venv`), `curl` or `wget`. For the Adeept camera,++`rpicam-apps` (preinstalled on Raspberry Pi OS). For Cozmo's voice, `espeak-ng`. `ffmpeg` is optional (better video++from plugins that send JPEG frames).++++## By hand++++```sh++sudo install -m 755 openvibe-node-linux-arm64 /usr/local/bin/openvibe-node++sudo openvibe-node pair ABCD-1234 # stores /etc/openvibe-node/credential.json (mode 600)++sudo openvibe-node install # systemd / launchd / Windows service, starts it++openvibe-node status++```++++Without root, everything goes to your user config directory (`~/.config/openvibe-node`), or to any directory with++`--home DIR` / `OPENVIBE_NODE_HOME`:++++```sh++openvibe-node --home ~/node pair ABCD-1234++openvibe-node --home ~/node run --dry-run # foreground; Ctrl-C stops it++```++++**Windows**: download `openvibe-node-windows-amd64.exe`, then in an Administrator prompt `openvibe-node pair <CODE>`++and `openvibe-node install` (the service runs as LocalSystem; files go to `%ProgramData%\OpenVibe Node`). Robot plugins++need Python; set `"python"` in `config.json`.++++## Commands++++| command | what it does |++|-------------------------------------------|----------------------------------------------------------------------------|++| `openvibe-node pair <CODE> [--force]` | redeem a pairing code; `--server`, `--kind onboard\|bridge` override config |++| `openvibe-node run [--dry-run]` | run in the foreground (what the service runs) |++| `openvibe-node install [--user NAME]` | install and start the service |++| `openvibe-node uninstall` | stop and remove the service (config and credential are kept) |++| `openvibe-node status [--json]` | link, stop latch, limits, plugins, video, battery |++| `openvibe-node stop` | **kill switch**: stop every actuator now and hold it stopped |++| `openvibe-node resume` | release the local stop (and a server e-stop) |++| `openvibe-node plugins [--json]` | plugins and their capabilities |++++`stop` works even when the Node does not answer: it latches `latch.json` in the state directory, which a running Node++obeys within a quarter second and a starting Node obeys before anything moves. The latch survives restarts.++++## Files++++| Linux | macOS / Windows | contents |++|---------------------------------------|-------------------------------------------------------|-----------------------------------|++| `/etc/openvibe-node/config.json` | `…/OpenVibe Node/config.json` | configuration |++| `/etc/openvibe-node/credential.json` | `…/OpenVibe Node/credential.json` | device credential (mode 600) |++| `/var/lib/openvibe-node/latch.json` | `…/OpenVibe Node/latch.json` | e-stop / kill switch latch |++| `/var/lib/openvibe-node/node.sock` | `…/OpenVibe Node/node.sock` | local control socket (mode 600) |++| `/var/lib/openvibe-node/venv` | `…/OpenVibe Node/venv` | plugin Python environment |++++macOS: `/Library/Application Support/OpenVibe Node`. Windows: `%ProgramData%\OpenVibe Node`.++++## Configuration++++```json++{++ "server": "https://openvibe.bot",++ "device_kind": "onboard",++ "plugins": [{"name": "adeept_adr036", "config": {"backend": "real", "wheels": "ordinary"}}],++ "video": {"source": "auto"},++ "limits": {"max_speed": 0.6, "max_turn": 0.5, "max_command_ms": 500},++ "log_level": "info"++}++```++++- `device_kind`: `onboard` or `bridge`.++- `plugins`: see [plugins.md](plugins.md#configuration).++- `video.source`: `auto` (the plugin's camera: its `h264_command` if the program exists, else its JPEG frames, else the++ test pattern if it asks for one), `test`, `command` (with `video.command`: a program writing H.264 Annex B to stdout,++ e.g. `["ffmpeg", "-f", "v4l2", "-i", "/dev/video0", "-c:v", "libx264", "-preset", "ultrafast", "-tune",++ "zerolatency", "-profile:v", "baseline", "-g", "30", "-f", "h264", "-"]`), `plugin` (with `video.plugin`), `off`.++ `fps`, `width`, `height`, `ffmpeg` tune the rest.++- `limits`: local caps on top of the owner's limits from openvibe.bot; the stricter value wins.++- `python`: interpreter for bundled plugins (default `<state>/venv/bin/python`, then `python3`).++++Unknown keys are rejected, so a typo does not silently change behaviour.++++## Robots++++### Adeept 4WD Smart Car Kit for Raspberry Pi (ADR036)++++Raspberry Pi OS Bookworm (64-bit recommended). Enable I²C and SPI: `sudo raspi-config` → Interface Options, or++`dtparam=i2c_arm=on` and `dtparam=spi=on` in `/boot/firmware/config.txt`, then reboot. Install with++`--robot adeept` (or `adeept-mecanum`).++++The kit's own software (`Adeept_Robot.service`, `WebServer.py` on `0.0.0.0:8888` with the fixed login++`admin:123456`, MJPEG on `:5000`) must not run: anyone on the network could drive the car. The installer disables it;++check with `systemctl status Adeept_Robot.service`. The Node replaces it.++++If a direction is wrong on your car, fix it in `config.json` → `plugins[0].config`: `motors.<wheel>.direction`,++`pan.invert`, `tilt.invert`, `line.invert`. See [the plugin README](../plugins/adeept_adr036/README.md).++++### Cozmo (bridge)++++Cozmo makes its own Wi-Fi network without internet. Use a Raspberry Pi or laptop with two network interfaces: one++joined to Cozmo's network (`Cozmo_…`; raise and lower the lift to see the password on its face), the other to the++internet (Ethernet, or a second Wi-Fi adapter). Install with `--robot cozmo`; the device kind is `bridge`. See++[the plugin README](../plugins/cozmo/README.md).++++## Troubleshooting++++- `openvibe-node status` shows the last link error. "the server refused the device credential" means it was revoked:++ `sudo openvibe-node pair --force <NEW-CODE>`.++- Logs: `journalctl -u openvibe-node -f` (Linux), `/var/log/openvibe-node.*.log` (macOS), Event Viewer (Windows).++- A plugin in `faulted` state says why in `status` (missing library, not a Raspberry Pi, robot not reachable, firmware).++- `openvibe-node plugins` works without the service running: it asks each plugin to describe itself without touching++ hardware.++++## Uninstall++++```sh++sudo openvibe-node uninstall++sudo rm -rf /usr/local/bin/openvibe-node /etc/openvibe-node /var/lib/openvibe-node++sudo userdel openvibe-node++```++++Then remove the device on openvibe.bot, which revokes its credential.nnew file mode 100644@@@ -0,0 +1,185 @@++# Plugins++++A plugin is a driver: a separate executable the Node core starts, speaking **JSON lines over stdin/stdout** (one JSON++object per line, UTF-8). Its stderr goes to the Node's log. The core never links driver code, so a driver can be written++in any language; the robot drivers are Python 3 because the kit libraries and PyCozmo are.++++Bundled plugins: [`dryrun`](../plugins/dryrun), [`adeept_adr036`](../plugins/adeept_adr036/README.md),++[`cozmo`](../plugins/cozmo/README.md). Python plugins use the runtime in [`plugins/sdk`](../plugins/sdk) (package++`openvibe_plugin`), which implements everything in "Safety rules" below, so a driver only says how to move its++hardware.++++## Configuration++++`config.json` lists the plugins (see [install.md](install.md#configuration)):++++```json++{"plugins": [++ {"name": "adeept_adr036", "config": {"backend": "real", "wheels": "mecanum"}},++ {"name": "my_arm", "command": ["/opt/arm/driver", "--serial", "/dev/ttyUSB0"], "env": {"ARM_DEBUG": "1"}}++]}++```++++A bundled plugin needs only its `name`; the core runs `<state>/venv/bin/python -m openvibe_<name>`. Anything else needs++`command`. `config` is sent to the plugin in `hello`. The core adds `frame_dir` (where to write camera frames) unless++set, and the environment variable `OPENVIBE_NODE_STATE`.++++## Lifecycle++++1. The core starts the process and sends `hello`.++2. The plugin answers `describe` **before touching any hardware**.++3. Unless `hello.probe` is true, the plugin opens its hardware with every actuator stopped and sends `ready` (or++ `fault` if it cannot).++4. The core sends `heartbeat` every 250 ms and commands as they arrive; the plugin answers each command with `ack` or++ `nack` and sends `telemetry`, `event` and `video` messages whenever it has them.++5. On shutdown the core sends `stop` and closes stdin; the plugin stops everything and exits. After 2 s it is killed.++++A plugin that exits is restarted with exponential backoff (1 s doubling to 30 s, reset after a minute of uptime). It++starts stopped; if the e-stop or the local kill switch is latched, the core sends `estop` right after `hello`.++A plugin that does not describe itself within 15 s, or stops reading stdin, is killed.++++Probe mode (`hello` with `"probe": true`) is used by `openvibe-node pair` and `openvibe-node plugins`: the plugin++describes itself and waits for EOF without opening any hardware.++++## Messages++++### Core → plugin++++| op | fields | the plugin must |++|-------------|---------------------------------------------------------------|--------------------------------------------------------|++| `hello` | `config` {…}, `probe` bool | answer `describe`; then set up (unless probe) |++| `command` | `id`, `kind`, `value`, `deadline_ms` | act, answer `ack` or `nack` with the same `id` |++| `heartbeat` | `t` (core clock, ms) | note the time |++| `stop` | | stop every actuator |++| `estop` | | stop every actuator and refuse motion until `resume` |++| `resume` | | accept motion again |++++`value` has already been validated and clamped to the owner's limits by the core ([protocol.md](protocol.md#commands)).++`deadline_ms` is relative to receipt and already capped.++++### Plugin → core++++| op | fields |++|-------------|---------------------------------------------------------------------------------------------|++| `describe` | `driver`, `version`, `protocol` (1), `capabilities` {…}, `motion_kinds` [kinds] |++| `ready` | — setup finished, every actuator stopped |++| `fault` | `fault_code`, `message` — the plugin runs but cannot drive (commands get this code as nack) |++| `ack` | `id` |++| `nack` | `id`, `fault_code`, `message` |++| `telemetry` | `battery` {`volts`, `percent`}, `rssi`, `sensors` {…}, `faults` […] |++| `event` | `name` (`cliff`, `picked_up`, `low_battery`, `heartbeat_lost`, `stop_failed`, …), any fields |++| `video` | `format` (`jpeg`), `path`, optional `width`, `height`, `seq` |++++For `video`, the plugin writes each frame to a file (write to a temporary name, then rename, and rotate a few names so++a slow reader never sees a partial file) and announces it. The core reads the file at once and encodes it: with++`ffmpeg` if installed, otherwise with its built-in encoder at up to 5 frames per second.++++## Safety rules (every plugin)++++A plugin must stop every actuator itself — the core dying must never leave motors running:++++- when its **stdin closes** (EOF), then exit;++- when a **motion command's deadline passes** without a newer command (`deadline_ms` after receipt);++- when it has **not heard a heartbeat for 1 s** (and refuse motion with `no_heartbeat` until heartbeats return);++- on `stop`, `estop` and a `halt` command;++- when its own sensors say so (Cozmo: cliff, pick-up).++++`stop` must be idempotent and safe at any time, including before setup. Handlers must not block: slow work (speech++synthesis, file I/O) goes on a thread. Nothing but protocol lines may go to stdout; the Python runtime enforces this by++pointing file descriptor 1 at stderr and keeping a private copy for the protocol, because hardware libraries print++there.++++## Capabilities++++`describe.capabilities` is an open object; the core uses these keys to route commands, and Bot uses the rest to build++the panel (ADR-043 decision 7: names are open strings checked against a registry in Bot).++++| key | meaning | routes |++|------------|------------------------------------------------------------------------------------------|-------------|++| `drive` | `{"type": "differential" \| "mecanum", …}` | `drive` |++| `ptz` | `{"pan": {"min", "max", …}, "tilt": {…}}` | `ptz` |++| `actuator` | `{"names": ["lift", …] \| ["any"], "ranges": {…}}` | `actuator` |++| `say` | `{…}` | `say` |++| `display` | `{"width", "height", "text", "faces", "image"}` | `display` |++| `camera` | `{"h264_command": [argv]}` (the core runs it), `{"jpeg": true}` (frames by `video`), or `{"source": "test_pattern"}` | video |++| `battery`, `sensors`, `lights`, `bridge` | informational | — |++++## The bundled drivers++++### `dryrun`++++Accepts every kind, logs it, reports simulated battery and distance, and asks the core for the test-pattern camera.++`config.record: "<path>"` appends one JSON line per `setup`, `command` and `stop` — end-to-end tests (here and in++OpenVibe.Bot) read it to check what reached the "robot".++++### `adeept_adr036` — Adeept 4WD Smart Car Kit for Raspberry Pi (on-board)++++Motors on the Robot HAT's PCA9685 (I²C `0x5f`, 50 Hz): M1 front-left (15, 14), M2 rear-left (12, 13), M3 rear-right++(11, 10), M4 front-right (8, 9), all with direction −1, as in the kit's `Move.py`; the table is plugin config.++`wheels: "ordinary"` mixes `throttle`/`steer`; `"mecanum"` mixes `x`/`y`/`rotation` (FL = x − y − r, FR = x + y + r,++RL = x + y − r, RR = x − y + r, normalised). Pan/tilt servos on channels 0/1 (500–2400 µs, 180°) take `ptz`++`pan`/`tilt` or actuators `pan`/`tilt`. Other actuators: `buzzer` `{"note": "A4"}` / `{"hz": 440}` / `null` (at most 2 s++per tone), `lights` `{"r", "g", "b", "index"?}` (WS2812 over SPI 0.0). Telemetry: `distance_cm` (ultrasonic 23/24),++`line` {left, middle, right} (GPIO 22/27/17), battery from the ADS7830 at `0x48` (2 × 18650: 6.0–8.4 V). Camera:++`rpicam-vid` H.264, published by the core. `backend: "fake"` runs it on any machine. It never starts or exposes the++kit's stock server.++++### `cozmo` — Anki/DDL Cozmo through PyCozmo (bridge)++++The computer running the Node joins Cozmo's Wi-Fi on one interface and the internet on another. `drive` →++`drive_wheels` (mm/s, `max_wheel_mmps` default 150); `ptz` `tilt` and actuator `head` (−1..1 → −25°..44.5°); actuator++`lift` (0..1 → 32..92 mm); `backpack_lights` and `cube_lights` `{"r", "g", "b", "cube"?}`; `head_light` bool (infrared);++`say` through `espeak-ng` or `pico2wave` → 22 kHz mono PCM; `display` text, faces or a PNG on the 128 × 32 screen;++camera frames (320 × 240) as JPEG files. A cliff or pick-up stops the wheels inside the event callback and emits an++event; while at a cliff only backing away is allowed, while picked up nothing drives. Firmware other than PyCozmo's++supported version (2381, `allow_firmware`) is reported as `firmware_unsupported`.++++## Writing a plugin in Python++++```python++from openvibe_plugin import Fault, Plugin, clamp, run++++class Arm(Plugin):++ driver, version = "my_arm", "0.1.0"++ motion_kinds = frozenset({"drive", "actuator"}) # stop these at their deadline++++ def describe(self, config):++ return {"actuator": {"names": ["shoulder", "gripper"]}}++++ def setup(self, config):++ self.bus = open_serial(config["port"]) # every actuator stopped when this returns++++ def handle(self, cmd):++ if cmd.kind != "actuator":++ raise Fault("unsupported")++ self.bus.speed(cmd.value["name"], clamp(cmd.value.get("value"), -1, 1))++++ def stop(self):++ if getattr(self, "bus", None):++ self.bus.stop_all()++++if __name__ == "__main__":++ raise SystemExit(run(Arm()))++```++++Test it with `openvibe_plugin.Runtime(plugin, infile, outfile, clock)` and a fake clock, as the bundled plugins do.++++## Where OpenVibe.Actor attaches (later)++++Actor's computer control (terminal, files, screen, keyboard and mouse, browser, the machine's own coding agents) will++be one more plugin, `computer`, on the same protocol; the core does not change:++++- **Capabilities** it will describe: `terminal`, `files`, `screen`, `input`, `browser`, `agents`, each with what the++ machine allows (for example `{"screen": {"displays": 2}, "agents": {"names": ["claude-code"]}}`).++- **Command kinds** it will add: `exec`, `fs`, `screen`, `input`, `browser`, `agent`. The core routes by capability++ key, so new kinds need only a line in `protocol.Kinds` and `plugins.capabilityFor`, plus clamping rules in++ `safety.Limits.Clamp` if they carry magnitudes. None of them are motion kinds, so deadlines do not apply; they are++ not in `GuardedKinds`, so an e-stop does not block them unless Actor asks for that.++- **Control levels** (Observe, Ask, Trusted, Full control) are enforced twice: by Bot/Actor when it issues a command++ (the operator's `role`), and in the plugin, which reads the owner's local level from its `config` and answers++ `nack not_allowed` for anything above it. `Ask` makes the plugin hold the command and raise an `event` `approval`++ that the local UI answers; the command's reply waits (the core's reply timeout is max(deadline, 1 s), so Actor++ commands will carry a long `deadline_ms` and the cap will come from Actor's own profile).++- **Output larger than a line** (screenshots, file contents, terminal streams) uses the `video` pattern: the plugin++ writes a file and sends its path in an `event`, and the core uploads it; the upload path is to be designed with Actor.++- The local kill switch (`openvibe-node stop`) will also make the `computer` plugin drop to Observe.nnew file mode 100644@@@ -0,0 +1,168 @@++# Node ↔ OpenVibe.Bot protocol (bot.device-message@1, device side)++++This is the wire format the Node speaks today, from ADR-043 decisions 1, 2, 4, 5 and 6. OpenVibe.Bot's own++`docs/protocol.md` is the source of truth; every field name below lives in one Go package, `internal/protocol/`, so++aligning with it is a change to that package only. Points still to be confirmed with the service are marked **(to++align)**.++++## 1. Pairing++++The owner adds a robot on openvibe.bot and gets an 8-character code (Crockford base32, shown `XXXX-XXXX`, 10 minutes,++single use, 5 wrong tries end it). The Node normalises it (upper case; dash and spaces removed; `I`/`L` → `1`, `O` → `0`)++and redeems it:++++```http++POST https://openvibe.bot/api/v1/pair++Content-Type: application/json++++{"code": "ABCD1234", "agent_version": "v0.1.0", "device_kind": "onboard",++ "drivers": ["adeept_adr036"],++ "capabilities": {"adeept_adr036": {"drive": {"type": "differential"}, "ptz": {...}, ...}},++ "os": "linux", "arch": "arm64", "hostname": "rover"}++```++++`device_kind` is `onboard` (the Node runs on the robot) or `bridge` (it drives the robot over the robot's own link, as++with Cozmo). `capabilities` is keyed by plugin name and holds each plugin's `describe` capabilities++([plugins.md](plugins.md)). ADR-043 writes the code field as `pair`; the Node sends `code` as the handoff brief says++**(to align)**.++++Answer `2xx`:++++```json++{"device_id": "dev_…", "credential": "<32 random bytes, encoded>", "publish_key": "…",++ "profile": {…bot.robot-profile@1…},++ "whip_url": "https://…/whip/…", "device_url": "wss://openvibe.bot/device",++ "ice_servers": [{"urls": ["turn:…"], "username": "…", "credential": "…"}]}++```++++`whip_url`, `device_url` and `ice_servers` are optional; without `device_url` the Node dials `wss://<origin>/device`++**(to align: where the WHIP URL comes from)**. Errors: `404`/`410` or `{"error":"invalid_code"|"expired"}` → "code wrong,++used or expired"; `429` → "too many tries". Plain `http://` is refused except for `localhost` (tests).++++The credential and publish key are stored in `credential.json` (mode 0600) and never appear in a URL, a log line, an++error, `openvibe-node status` or telemetry (the Go type `credentials.Secret` prints as `[redacted]` in every format).++++## 2. The control link++++One WebSocket per device, opened by the device:++++```http++GET wss://openvibe.bot/device++Authorization: Bearer <credential>++User-Agent: openvibe-node/<version>++```++++`401`/`403` means the credential was revoked or rotated away: the Node logs "pair again" and keeps retrying slowly.++++### Frames++++Every frame is one JSON object in one text message with the envelope fields at the top level:++++| field | type | meaning |++|--------|--------|------------------------------------------------------------------------|++| `v` | int | envelope version, `1`. Other versions are rejected. |++| `seq` | uint | per-direction counter, starting at 1 on each connection |++| `ts` | int | sender's clock, Unix milliseconds |++| `type` | string | message type |++++Unknown `type`s are ignored (a newer server may send more). Frames are at most 1 MiB.++++### Server → device++++| type | fields |++|-----------------|--------------------------------------------------------------------------------------------------|++| `hello` | `device_id`, `robot_id`, `session`, `server_time` |++| `config` | `limits` {`max_speed` 0..1, `max_turn` 0..1, `max_command_ms`}, `heartbeat_ms`, `deadman_ms`, `telemetry_ms` (≥ 500), `allowed` [kinds] |++| `command` | `id` (idempotency key), `kind`, `value`, `deadline_ms`, `operator`, `role`, `target` (optional plugin name) |++| `estop` | `reason`, `by` — latched |++| `estop_clear` | `by` — the owner clears the remote e-stop |++| `heartbeat_ack` | `t` (echo of the heartbeat's `t`; the Node measures RTT from it) |++++### Device → server++++| type | fields |++|---------------|-----------------------------------------------------------------------------------------------------|++| `status` | `agent_version`, `device_kind`, `os`, `arch`, `drivers` [{`name`, `driver`, `version`, `state`, `capabilities`}], `faults` [{`code`, `driver`, `message`}], `estop_latched`, `local_stop`, `video` |++| `telemetry` | `battery` {`volts`, `percent`}, `rssi`, `sensors` {…}, `events` [{`name`, `driver`, `ts`, `fields`}] |++| `ack` | `id`, `latency_ms` (receipt → plugin ack) |++| `nack` | `id`, `fault_code`, `message` |++| `heartbeat` | `t` (device clock, ms) |++| `estop_state` | `latched` (remote e-stop), `local_stop` (local kill switch), `reason` |++++`status` and `estop_state` are sent on every connect and whenever a plugin changes state, faults or the latch changes.++Plugin events (`cliff`, `picked_up`, `low_battery`, `heartbeat_lost`, …) travel in `telemetry.events` and are sent++within 100 ms; sensor telemetry is sent at most every `telemetry_ms` (default 500 ms, so ≤ 2 Hz) **(to align: a++separate `event` type?)**.++++### Commands++++| kind | value | motion |++|------------|---------------------------------------------------------------------------------------------------------|--------|++| `drive` | differential `{"throttle": -1..1, "steer": -1..1}`; mecanum `{"x": -1..1, "y": -1..1, "rotation": -1..1}` | yes |++| `ptz` | `{"pan": -1..1, "tilt": -1..1, "zoom": -1..1}` absolute positions (0 = centre) | guarded |++| `actuator` | `{"name": "lift", "value": …}` — number, object or bool per actuator ([plugins.md](plugins.md)) | guarded |++| `say` | `{"text": "…"}` (1–1000 bytes; plugins may cap lower) | no |++| `display` | `{"text": "…"}`, `{"face": "happy"}` or `{"image_png_b64": "…"}` | no |++| `halt` | `{}` — stop every actuator now (not latched); always accepted | — |++++Sign conventions: `throttle`/`x` positive = forward; `steer` positive = turn right; `y` positive = strafe left;++`rotation` positive = counter-clockwise (left); `pan` positive = right, `tilt` positive = up (plugins can invert a++servo in their config to make the hardware match).++++`deadline_ms` is **relative to receipt** (milliseconds the command stays valid), not an absolute timestamp: a device's++clock may be minutes off, and a relative deadline cannot be stretched by clock skew. Default 300 ms; capped at++`max_command_ms` (default 1000). A held control is re-sent by the panel every 150 ms, each with a new `id`++**(to align: ADR-043 says "absolute deadline")**.++++What the Node does with a command, in order:++++1. A repeated `id` is answered with the first result and never executed again (the cache lives for 10 minutes and++ across reconnects).++2. Unknown kind → `nack unsupported`. Kind not in `config.allowed` → `nack not_allowed` (`halt` is always allowed).++3. `halt` → every plugin stops, `ack`.++4. `drive`, `ptz`, `actuator` while the local kill switch is latched → `nack local_stop`; while the remote e-stop is++ latched → `nack estopped`.++5. The value is clamped in the core, before any plugin sees it: `throttle`, `x`, `y` to ±`max_speed`; `steer`,++ `rotation` to ±`max_turn`; `pan`, `tilt`, `zoom` and a numeric actuator `value` to ±1. The limits are the stricter of++ the server's `config.limits` and the local config's `limits`. Non-numbers → `nack bad_value`.++6. It goes to the plugin named by `target`, or the first plugin whose `describe` declares the kind (for actuators, the++ first listing the name). None → `nack unsupported`.++7. The plugin's `ack`/`nack` is relayed. No answer within max(deadline, 1 s) → the plugin is told to stop and the++ server gets `nack plugin_timeout`.++++Commands are never queued while offline and never replayed after a reconnect.++++### Fault codes++++`bad_frame`, `bad_value`, `unsupported`, `not_allowed`, `estopped`, `local_stop`, `no_heartbeat`,++`not_ready`, `plugin_down`, `plugin_timeout`, `hardware`, `firmware_unsupported`, `not_connected`, `cliff`,++`picked_up`, `shutting_down`. Plugins may add their own; they are passed through.++++### Heartbeat, deadman, reconnect++++- The device sends `heartbeat` every `heartbeat_ms` (default 1000). The server answers `heartbeat_ack`.++- If nothing at all arrives from the server for `deadman_ms` (default 2 × `heartbeat_ms`: two missed heartbeats), the++ Node closes the link. Every close stops every actuator at once.++- Reconnect: exponential backoff from 0.5 s to 30 s with full jitter; at least 10 s after a `401`/`403`.++++## 3. Safety paths (ADR-043 decision 6)++++| event | what stops it | latched? |++|----------------------------------------------|----------------------------------------------------|-----------------------------------------|++| motion command's deadline passes | the plugin (its own timer) | no |++| 2 missed heartbeats / link closed | the core → `stop` to every plugin | no |++| server `estop` | the core → `estop` to every plugin | yes, persisted; `estop_clear` or `openvibe-node resume` |++| `openvibe-node stop` (local kill switch) | the core → `estop` to every plugin | yes, persisted; only `openvibe-node resume` |++| core shutdown | `stop`, then stdin closed | no |++| core killed / crashed | the plugin: stdin EOF, or 1 s without a heartbeat | no |++| plugin crash | restarted with backoff; starts stopped; `estop` re-sent if latched | — |++| Cozmo cliff or pick-up | the Cozmo plugin, in the event callback | until the condition clears |++++The latch file is `latch.json` in the state directory. A corrupt latch file is read as "local stop latched".++++## 4. Video (WHIP)++++The Node publishes one H.264 track (Constrained Baseline, `profile-level-id=42e01f`, packetization-mode 1) to++`whip_url` with `Authorization: Bearer <publish_key>`: `POST` with `Content-Type: application/sdp` → `201` with the++answer and a `Location`; `DELETE <Location>` on shutdown. PLI/FIR from the far end request a keyframe. The session is++re-established with backoff; video runs on its own goroutines and never waits on control or vice versa.@@@ -66,7 +66,6 @@ const (FaultEstopped = "estopped" // the remote e-stop is latchedFaultLocalStop = "local_stop" // the local kill switch (`openvibe-node stop`) is latchedFaultNoHeartbeat = "no_heartbeat" // a plugin lost the core's heartbeat-- FaultExpired = "expired" // the deadline had already passed on arrivalFaultNotReady = "not_ready" // the driver is starting or restartingFaultPluginDown = "plugin_down" // the driver process is not runningFaultPluginTimeout = "plugin_timeout" // the driver did not answer in time@@@ -55,7 +55,8 @@ class DryRun(Plugin):if cmd.kind == "drive":if "x" in v or "y" in v:self.state["throttle"] = clamp(v.get("x", 0), -1, 1)-- self.state["steer"] = clamp(v.get("rotation", 0), -1, 1)++ # rotation is counter-clockwise positive; steer is right positive.++ self.state["steer"] = -clamp(v.get("rotation", 0), -1, 1)else:self.state["throttle"] = clamp(v.get("throttle", 0), -1, 1)self.state["steer"] = clamp(v.get("steer", 0), -1, 1)
@@ -1,5 +1,45 @@
@@ -1,5 +1,45 @@
# OpenVibe Node
# OpenVibe Node
-One service for your computers, servers, Raspberry Pis, phones and robots: pair it once with OpenVibe and products like OpenVibe.Bot (robots) and OpenVibe.Actor (AI agents) can use the device safely. It connects out, needs no open ports, and keeps a local kill switch.
+One service for your computers, servers, Raspberry Pis, phones and robots: pair it once with OpenVibe and products
+like **OpenVibe.Bot** (robots: drive, cameras, sensors, with safety) and, later, **OpenVibe.Actor** (computer control
+for AI agents) can use the device. It connects out, needs no open ports, and keeps a local kill switch.
-Status: being built. Start here: [docs/HANDOFF.md](docs/HANDOFF.md) and [docs/ADR-043-bot-devices-and-control.md](docs/ADR-043-bot-devices-and-control.md).
+```sh
+curl -fsSL https://openvibe.bot/install | sh -s -- ABCD-1234 --robot adeept
+```
+
+Get the code on openvibe.bot. `--robot` is `adeept`, `adeept-mecanum`, `cozmo` or `none` (dry run). Details, manual
+install, Windows and configuration: [docs/install.md](docs/install.md).
+
+```sh
+openvibe-node status # link, plugins, video, battery, stop latch
+openvibe-node stop # kill switch: stop every motor now and keep it stopped
+openvibe-node resume
+```
+
+## What is in here
+
+- **Core** (Go, one static binary per platform): pairing, one outbound WebSocket to `wss://openvibe.bot/device`
+ (credential in the `Authorization` header only), heartbeats and a deadman, the safety rules (deadlines, owner limits
+ clamped before any driver sees a value, a persisted e-stop, the local kill switch), the plugin supervisor, WHIP video
+ publishing with pion, and the system service (systemd, launchd, Windows).
+- **Plugins** (separate processes, JSON lines over stdin/stdout): [`dryrun`](plugins/dryrun), the Adeept 4WD Smart Car
+ Kit [`adeept_adr036`](plugins/adeept_adr036/README.md), and [`cozmo`](plugins/cozmo/README.md) through PyCozmo. Each
+ plugin stops its own actuators when stdin closes, at a command's deadline and after 1 s without a heartbeat, so the
+ core dying never leaves motors running.
+
+Docs: [protocol](docs/protocol.md) (the wire format with OpenVibe.Bot), [plugins](docs/plugins.md) (the driver
+protocol, the bundled drivers, and where Actor's computer-control plugin attaches), [install](docs/install.md).
+Decisions: [ADR-043](docs/ADR-043-bot-devices-and-control.md).
+
+## Development
+
+```sh
+go vet ./... && go test -race ./... # the end-to-end tests run the Python dry-run plugin
+pip install pytest pillow numpy
+for p in sdk dryrun adeept_adr036 cozmo; do python3 -m pytest -q plugins/$p/tests; done
+scripts/dist.sh v0.0.0-dev # every binary, the plugin bundle and SHA256SUMS in dist/
+```
+
+`internal/fakebot` is a fake OpenVibe.Bot (pairing, device WebSocket, WHIP) used by the tests. Licence: AGPL-3.0
+([LICENSE](LICENSE)). Security reports: [SECURITY.md](SECURITY.md).
new file mode 100644
new file mode 100644
@@ -0,0 +1,144 @@
@@ -0,0 +1,144 @@
+# Installing the OpenVibe Node
+
+The Node is one program, `openvibe-node`, plus Python plugins for robots. It connects **out** to OpenVibe (no open
+ports, works behind any home router), and a local kill switch stops everything it drives.
+
+## Quick start (Linux, Raspberry Pi, macOS)
+
+1. On openvibe.bot, add a robot. You get a code like `ABCD-1234` (valid 10 minutes).
+2. On the device:
+
+ ```sh
+ curl -fsSL https://openvibe.bot/install | sh -s -- ABCD-1234 --robot adeept
+ ```
+
+ `--robot` is `adeept` (Adeept 4WD Smart Car, ordinary wheels), `adeept-mecanum`, `cozmo`, or `none` (the dry-run
+ plugin with a test-pattern camera, for trying the Node on a laptop).
+3. Confirm the new device on openvibe.bot.
+
+The installer detects the OS and CPU (linux amd64/arm64/armv7, macOS amd64/arm64), downloads the binary and the plugin
+bundle and checks them against `SHA256SUMS`, creates a Python virtual environment for the plugins, writes
+`config.json`, disables the Adeept kit's stock server if it finds it, creates the `openvibe-node` service account
+(Linux; with the `gpio`, `i2c`, `spi`, `video`, `audio` groups), installs and starts the service, and pairs. Running it
+again upgrades and keeps the credential. Options: `--no-service`, `--version vX.Y.Z`, `--local DIR` (install from a
+directory holding the release files). `OPENVIBE_NODE_BASE` overrides the download location, `OPENVIBE_SERVER` the
+server.
+
+Requirements: `python3` with `venv` (`sudo apt install python3 python3-venv`), `curl` or `wget`. For the Adeept camera,
+`rpicam-apps` (preinstalled on Raspberry Pi OS). For Cozmo's voice, `espeak-ng`. `ffmpeg` is optional (better video
+from plugins that send JPEG frames).
+
+## By hand
+
+```sh
+sudo install -m 755 openvibe-node-linux-arm64 /usr/local/bin/openvibe-node
+sudo openvibe-node pair ABCD-1234 # stores /etc/openvibe-node/credential.json (mode 600)
+sudo openvibe-node install # systemd / launchd / Windows service, starts it
+openvibe-node status
+```
+
+Without root, everything goes to your user config directory (`~/.config/openvibe-node`), or to any directory with
+`--home DIR` / `OPENVIBE_NODE_HOME`:
+
+```sh
+openvibe-node --home ~/node pair ABCD-1234
+openvibe-node --home ~/node run --dry-run # foreground; Ctrl-C stops it
+```
+
+**Windows**: download `openvibe-node-windows-amd64.exe`, then in an Administrator prompt `openvibe-node pair <CODE>`
+and `openvibe-node install` (the service runs as LocalSystem; files go to `%ProgramData%\OpenVibe Node`). Robot plugins
+need Python; set `"python"` in `config.json`.
+
+## Commands
+
+| command | what it does |
+|-------------------------------------------|----------------------------------------------------------------------------|
+| `openvibe-node pair <CODE> [--force]` | redeem a pairing code; `--server`, `--kind onboard\|bridge` override config |
+| `openvibe-node run [--dry-run]` | run in the foreground (what the service runs) |
+| `openvibe-node install [--user NAME]` | install and start the service |
+| `openvibe-node uninstall` | stop and remove the service (config and credential are kept) |
+| `openvibe-node status [--json]` | link, stop latch, limits, plugins, video, battery |
+| `openvibe-node stop` | **kill switch**: stop every actuator now and hold it stopped |
+| `openvibe-node resume` | release the local stop (and a server e-stop) |
+| `openvibe-node plugins [--json]` | plugins and their capabilities |
+
+`stop` works even when the Node does not answer: it latches `latch.json` in the state directory, which a running Node
+obeys within a quarter second and a starting Node obeys before anything moves. The latch survives restarts.
+
+## Files
+
+| Linux | macOS / Windows | contents |
+|---------------------------------------|-------------------------------------------------------|-----------------------------------|
+| `/etc/openvibe-node/config.json` | `…/OpenVibe Node/config.json` | configuration |
+| `/etc/openvibe-node/credential.json` | `…/OpenVibe Node/credential.json` | device credential (mode 600) |
+| `/var/lib/openvibe-node/latch.json` | `…/OpenVibe Node/latch.json` | e-stop / kill switch latch |
+| `/var/lib/openvibe-node/node.sock` | `…/OpenVibe Node/node.sock` | local control socket (mode 600) |
+| `/var/lib/openvibe-node/venv` | `…/OpenVibe Node/venv` | plugin Python environment |
+
+macOS: `/Library/Application Support/OpenVibe Node`. Windows: `%ProgramData%\OpenVibe Node`.
+
+## Configuration
+
+```json
+{
+ "server": "https://openvibe.bot",
+ "device_kind": "onboard",
+ "plugins": [{"name": "adeept_adr036", "config": {"backend": "real", "wheels": "ordinary"}}],
+ "video": {"source": "auto"},
+ "limits": {"max_speed": 0.6, "max_turn": 0.5, "max_command_ms": 500},
+ "log_level": "info"
+}
+```
+
+- `device_kind`: `onboard` or `bridge`.
+- `plugins`: see [plugins.md](plugins.md#configuration).
+- `video.source`: `auto` (the plugin's camera: its `h264_command` if the program exists, else its JPEG frames, else the
+ test pattern if it asks for one), `test`, `command` (with `video.command`: a program writing H.264 Annex B to stdout,
+ e.g. `["ffmpeg", "-f", "v4l2", "-i", "/dev/video0", "-c:v", "libx264", "-preset", "ultrafast", "-tune",
+ "zerolatency", "-profile:v", "baseline", "-g", "30", "-f", "h264", "-"]`), `plugin` (with `video.plugin`), `off`.
+ `fps`, `width`, `height`, `ffmpeg` tune the rest.
+- `limits`: local caps on top of the owner's limits from openvibe.bot; the stricter value wins.
+- `python`: interpreter for bundled plugins (default `<state>/venv/bin/python`, then `python3`).
+
+Unknown keys are rejected, so a typo does not silently change behaviour.
+
+## Robots
+
+### Adeept 4WD Smart Car Kit for Raspberry Pi (ADR036)
+
+Raspberry Pi OS Bookworm (64-bit recommended). Enable I²C and SPI: `sudo raspi-config` → Interface Options, or
+`dtparam=i2c_arm=on` and `dtparam=spi=on` in `/boot/firmware/config.txt`, then reboot. Install with
+`--robot adeept` (or `adeept-mecanum`).
+
+The kit's own software (`Adeept_Robot.service`, `WebServer.py` on `0.0.0.0:8888` with the fixed login
+`admin:123456`, MJPEG on `:5000`) must not run: anyone on the network could drive the car. The installer disables it;
+check with `systemctl status Adeept_Robot.service`. The Node replaces it.
+
+If a direction is wrong on your car, fix it in `config.json` → `plugins[0].config`: `motors.<wheel>.direction`,
+`pan.invert`, `tilt.invert`, `line.invert`. See [the plugin README](../plugins/adeept_adr036/README.md).
+
+### Cozmo (bridge)
+
+Cozmo makes its own Wi-Fi network without internet. Use a Raspberry Pi or laptop with two network interfaces: one
+joined to Cozmo's network (`Cozmo_…`; raise and lower the lift to see the password on its face), the other to the
+internet (Ethernet, or a second Wi-Fi adapter). Install with `--robot cozmo`; the device kind is `bridge`. See
+[the plugin README](../plugins/cozmo/README.md).
+
+## Troubleshooting
+
+- `openvibe-node status` shows the last link error. "the server refused the device credential" means it was revoked:
+ `sudo openvibe-node pair --force <NEW-CODE>`.
+- Logs: `journalctl -u openvibe-node -f` (Linux), `/var/log/openvibe-node.*.log` (macOS), Event Viewer (Windows).
+- A plugin in `faulted` state says why in `status` (missing library, not a Raspberry Pi, robot not reachable, firmware).
+- `openvibe-node plugins` works without the service running: it asks each plugin to describe itself without touching
+ hardware.
+
+## Uninstall
+
+```sh
+sudo openvibe-node uninstall
+sudo rm -rf /usr/local/bin/openvibe-node /etc/openvibe-node /var/lib/openvibe-node
+sudo userdel openvibe-node
+```
+
+Then remove the device on openvibe.bot, which revokes its credential.
new file mode 100644
new file mode 100644
@@ -0,0 +1,185 @@
@@ -0,0 +1,185 @@
+# Plugins
+
+A plugin is a driver: a separate executable the Node core starts, speaking **JSON lines over stdin/stdout** (one JSON
+object per line, UTF-8). Its stderr goes to the Node's log. The core never links driver code, so a driver can be written
+in any language; the robot drivers are Python 3 because the kit libraries and PyCozmo are.
+
+Bundled plugins: [`dryrun`](../plugins/dryrun), [`adeept_adr036`](../plugins/adeept_adr036/README.md),
+[`cozmo`](../plugins/cozmo/README.md). Python plugins use the runtime in [`plugins/sdk`](../plugins/sdk) (package
+`openvibe_plugin`), which implements everything in "Safety rules" below, so a driver only says how to move its
+hardware.
+
+## Configuration
+
+`config.json` lists the plugins (see [install.md](install.md#configuration)):
+
+```json
+{"plugins": [
+ {"name": "adeept_adr036", "config": {"backend": "real", "wheels": "mecanum"}},
+ {"name": "my_arm", "command": ["/opt/arm/driver", "--serial", "/dev/ttyUSB0"], "env": {"ARM_DEBUG": "1"}}
+]}
+```
+
+A bundled plugin needs only its `name`; the core runs `<state>/venv/bin/python -m openvibe_<name>`. Anything else needs
+`command`. `config` is sent to the plugin in `hello`. The core adds `frame_dir` (where to write camera frames) unless
+set, and the environment variable `OPENVIBE_NODE_STATE`.
+
+## Lifecycle
+
+1. The core starts the process and sends `hello`.
+2. The plugin answers `describe` **before touching any hardware**.
+3. Unless `hello.probe` is true, the plugin opens its hardware with every actuator stopped and sends `ready` (or
+ `fault` if it cannot).
+4. The core sends `heartbeat` every 250 ms and commands as they arrive; the plugin answers each command with `ack` or
+ `nack` and sends `telemetry`, `event` and `video` messages whenever it has them.
+5. On shutdown the core sends `stop` and closes stdin; the plugin stops everything and exits. After 2 s it is killed.
+
+A plugin that exits is restarted with exponential backoff (1 s doubling to 30 s, reset after a minute of uptime). It
+starts stopped; if the e-stop or the local kill switch is latched, the core sends `estop` right after `hello`.
+A plugin that does not describe itself within 15 s, or stops reading stdin, is killed.
+
+Probe mode (`hello` with `"probe": true`) is used by `openvibe-node pair` and `openvibe-node plugins`: the plugin
+describes itself and waits for EOF without opening any hardware.
+
+## Messages
+
+### Core → plugin
+
+| op | fields | the plugin must |
+|-------------|---------------------------------------------------------------|--------------------------------------------------------|
+| `hello` | `config` {…}, `probe` bool | answer `describe`; then set up (unless probe) |
+| `command` | `id`, `kind`, `value`, `deadline_ms` | act, answer `ack` or `nack` with the same `id` |
+| `heartbeat` | `t` (core clock, ms) | note the time |
+| `stop` | | stop every actuator |
+| `estop` | | stop every actuator and refuse motion until `resume` |
+| `resume` | | accept motion again |
+
+`value` has already been validated and clamped to the owner's limits by the core ([protocol.md](protocol.md#commands)).
+`deadline_ms` is relative to receipt and already capped.
+
+### Plugin → core
+
+| op | fields |
+|-------------|---------------------------------------------------------------------------------------------|
+| `describe` | `driver`, `version`, `protocol` (1), `capabilities` {…}, `motion_kinds` [kinds] |
+| `ready` | — setup finished, every actuator stopped |
+| `fault` | `fault_code`, `message` — the plugin runs but cannot drive (commands get this code as nack) |
+| `ack` | `id` |
+| `nack` | `id`, `fault_code`, `message` |
+| `telemetry` | `battery` {`volts`, `percent`}, `rssi`, `sensors` {…}, `faults` […] |
+| `event` | `name` (`cliff`, `picked_up`, `low_battery`, `heartbeat_lost`, `stop_failed`, …), any fields |
+| `video` | `format` (`jpeg`), `path`, optional `width`, `height`, `seq` |
+
+For `video`, the plugin writes each frame to a file (write to a temporary name, then rename, and rotate a few names so
+a slow reader never sees a partial file) and announces it. The core reads the file at once and encodes it: with
+`ffmpeg` if installed, otherwise with its built-in encoder at up to 5 frames per second.
+
+## Safety rules (every plugin)
+
+A plugin must stop every actuator itself — the core dying must never leave motors running:
+
+- when its **stdin closes** (EOF), then exit;
+- when a **motion command's deadline passes** without a newer command (`deadline_ms` after receipt);
+- when it has **not heard a heartbeat for 1 s** (and refuse motion with `no_heartbeat` until heartbeats return);
+- on `stop`, `estop` and a `halt` command;
+- when its own sensors say so (Cozmo: cliff, pick-up).
+
+`stop` must be idempotent and safe at any time, including before setup. Handlers must not block: slow work (speech
+synthesis, file I/O) goes on a thread. Nothing but protocol lines may go to stdout; the Python runtime enforces this by
+pointing file descriptor 1 at stderr and keeping a private copy for the protocol, because hardware libraries print
+there.
+
+## Capabilities
+
+`describe.capabilities` is an open object; the core uses these keys to route commands, and Bot uses the rest to build
+the panel (ADR-043 decision 7: names are open strings checked against a registry in Bot).
+
+| key | meaning | routes |
+|------------|------------------------------------------------------------------------------------------|-------------|
+| `drive` | `{"type": "differential" \| "mecanum", …}` | `drive` |
+| `ptz` | `{"pan": {"min", "max", …}, "tilt": {…}}` | `ptz` |
+| `actuator` | `{"names": ["lift", …] \| ["any"], "ranges": {…}}` | `actuator` |
+| `say` | `{…}` | `say` |
+| `display` | `{"width", "height", "text", "faces", "image"}` | `display` |
+| `camera` | `{"h264_command": [argv]}` (the core runs it), `{"jpeg": true}` (frames by `video`), or `{"source": "test_pattern"}` | video |
+| `battery`, `sensors`, `lights`, `bridge` | informational | — |
+
+## The bundled drivers
+
+### `dryrun`
+
+Accepts every kind, logs it, reports simulated battery and distance, and asks the core for the test-pattern camera.
+`config.record: "<path>"` appends one JSON line per `setup`, `command` and `stop` — end-to-end tests (here and in
+OpenVibe.Bot) read it to check what reached the "robot".
+
+### `adeept_adr036` — Adeept 4WD Smart Car Kit for Raspberry Pi (on-board)
+
+Motors on the Robot HAT's PCA9685 (I²C `0x5f`, 50 Hz): M1 front-left (15, 14), M2 rear-left (12, 13), M3 rear-right
+(11, 10), M4 front-right (8, 9), all with direction −1, as in the kit's `Move.py`; the table is plugin config.
+`wheels: "ordinary"` mixes `throttle`/`steer`; `"mecanum"` mixes `x`/`y`/`rotation` (FL = x − y − r, FR = x + y + r,
+RL = x + y − r, RR = x − y + r, normalised). Pan/tilt servos on channels 0/1 (500–2400 µs, 180°) take `ptz`
+`pan`/`tilt` or actuators `pan`/`tilt`. Other actuators: `buzzer` `{"note": "A4"}` / `{"hz": 440}` / `null` (at most 2 s
+per tone), `lights` `{"r", "g", "b", "index"?}` (WS2812 over SPI 0.0). Telemetry: `distance_cm` (ultrasonic 23/24),
+`line` {left, middle, right} (GPIO 22/27/17), battery from the ADS7830 at `0x48` (2 × 18650: 6.0–8.4 V). Camera:
+`rpicam-vid` H.264, published by the core. `backend: "fake"` runs it on any machine. It never starts or exposes the
+kit's stock server.
+
+### `cozmo` — Anki/DDL Cozmo through PyCozmo (bridge)
+
+The computer running the Node joins Cozmo's Wi-Fi on one interface and the internet on another. `drive` →
+`drive_wheels` (mm/s, `max_wheel_mmps` default 150); `ptz` `tilt` and actuator `head` (−1..1 → −25°..44.5°); actuator
+`lift` (0..1 → 32..92 mm); `backpack_lights` and `cube_lights` `{"r", "g", "b", "cube"?}`; `head_light` bool (infrared);
+`say` through `espeak-ng` or `pico2wave` → 22 kHz mono PCM; `display` text, faces or a PNG on the 128 × 32 screen;
+camera frames (320 × 240) as JPEG files. A cliff or pick-up stops the wheels inside the event callback and emits an
+event; while at a cliff only backing away is allowed, while picked up nothing drives. Firmware other than PyCozmo's
+supported version (2381, `allow_firmware`) is reported as `firmware_unsupported`.
+
+## Writing a plugin in Python
+
+```python
+from openvibe_plugin import Fault, Plugin, clamp, run
+
+class Arm(Plugin):
+ driver, version = "my_arm", "0.1.0"
+ motion_kinds = frozenset({"drive", "actuator"}) # stop these at their deadline
+
+ def describe(self, config):
+ return {"actuator": {"names": ["shoulder", "gripper"]}}
+
+ def setup(self, config):
+ self.bus = open_serial(config["port"]) # every actuator stopped when this returns
+
+ def handle(self, cmd):
+ if cmd.kind != "actuator":
+ raise Fault("unsupported")
+ self.bus.speed(cmd.value["name"], clamp(cmd.value.get("value"), -1, 1))
+
+ def stop(self):
+ if getattr(self, "bus", None):
+ self.bus.stop_all()
+
+if __name__ == "__main__":
+ raise SystemExit(run(Arm()))
+```
+
+Test it with `openvibe_plugin.Runtime(plugin, infile, outfile, clock)` and a fake clock, as the bundled plugins do.
+
+## Where OpenVibe.Actor attaches (later)
+
+Actor's computer control (terminal, files, screen, keyboard and mouse, browser, the machine's own coding agents) will
+be one more plugin, `computer`, on the same protocol; the core does not change:
+
+- **Capabilities** it will describe: `terminal`, `files`, `screen`, `input`, `browser`, `agents`, each with what the
+ machine allows (for example `{"screen": {"displays": 2}, "agents": {"names": ["claude-code"]}}`).
+- **Command kinds** it will add: `exec`, `fs`, `screen`, `input`, `browser`, `agent`. The core routes by capability
+ key, so new kinds need only a line in `protocol.Kinds` and `plugins.capabilityFor`, plus clamping rules in
+ `safety.Limits.Clamp` if they carry magnitudes. None of them are motion kinds, so deadlines do not apply; they are
+ not in `GuardedKinds`, so an e-stop does not block them unless Actor asks for that.
+- **Control levels** (Observe, Ask, Trusted, Full control) are enforced twice: by Bot/Actor when it issues a command
+ (the operator's `role`), and in the plugin, which reads the owner's local level from its `config` and answers
+ `nack not_allowed` for anything above it. `Ask` makes the plugin hold the command and raise an `event` `approval`
+ that the local UI answers; the command's reply waits (the core's reply timeout is max(deadline, 1 s), so Actor
+ commands will carry a long `deadline_ms` and the cap will come from Actor's own profile).
+- **Output larger than a line** (screenshots, file contents, terminal streams) uses the `video` pattern: the plugin
+ writes a file and sends its path in an `event`, and the core uploads it; the upload path is to be designed with Actor.
+- The local kill switch (`openvibe-node stop`) will also make the `computer` plugin drop to Observe.
new file mode 100644
new file mode 100644
@@ -0,0 +1,168 @@
@@ -0,0 +1,168 @@
+# Node ↔ OpenVibe.Bot protocol (bot.device-message@1, device side)
+
+This is the wire format the Node speaks today, from ADR-043 decisions 1, 2, 4, 5 and 6. OpenVibe.Bot's own
+`docs/protocol.md` is the source of truth; every field name below lives in one Go package, `internal/protocol/`, so
+aligning with it is a change to that package only. Points still to be confirmed with the service are marked **(to
+align)**.
+
+## 1. Pairing
+
+The owner adds a robot on openvibe.bot and gets an 8-character code (Crockford base32, shown `XXXX-XXXX`, 10 minutes,
+single use, 5 wrong tries end it). The Node normalises it (upper case; dash and spaces removed; `I`/`L` → `1`, `O` → `0`)
+and redeems it:
+
+```http
+POST https://openvibe.bot/api/v1/pair
+Content-Type: application/json
+
+{"code": "ABCD1234", "agent_version": "v0.1.0", "device_kind": "onboard",
+ "drivers": ["adeept_adr036"],
+ "capabilities": {"adeept_adr036": {"drive": {"type": "differential"}, "ptz": {...}, ...}},
+ "os": "linux", "arch": "arm64", "hostname": "rover"}
+```
+
+`device_kind` is `onboard` (the Node runs on the robot) or `bridge` (it drives the robot over the robot's own link, as
+with Cozmo). `capabilities` is keyed by plugin name and holds each plugin's `describe` capabilities
+([plugins.md](plugins.md)). ADR-043 writes the code field as `pair`; the Node sends `code` as the handoff brief says
+**(to align)**.
+
+Answer `2xx`:
+
+```json
+{"device_id": "dev_…", "credential": "<32 random bytes, encoded>", "publish_key": "…",
+ "profile": {…bot.robot-profile@1…},
+ "whip_url": "https://…/whip/…", "device_url": "wss://openvibe.bot/device",
+ "ice_servers": [{"urls": ["turn:…"], "username": "…", "credential": "…"}]}
+```
+
+`whip_url`, `device_url` and `ice_servers` are optional; without `device_url` the Node dials `wss://<origin>/device`
+**(to align: where the WHIP URL comes from)**. Errors: `404`/`410` or `{"error":"invalid_code"|"expired"}` → "code wrong,
+used or expired"; `429` → "too many tries". Plain `http://` is refused except for `localhost` (tests).
+
+The credential and publish key are stored in `credential.json` (mode 0600) and never appear in a URL, a log line, an
+error, `openvibe-node status` or telemetry (the Go type `credentials.Secret` prints as `[redacted]` in every format).
+
+## 2. The control link
+
+One WebSocket per device, opened by the device:
+
+```http
+GET wss://openvibe.bot/device
+Authorization: Bearer <credential>
+User-Agent: openvibe-node/<version>
+```
+
+`401`/`403` means the credential was revoked or rotated away: the Node logs "pair again" and keeps retrying slowly.
+
+### Frames
+
+Every frame is one JSON object in one text message with the envelope fields at the top level:
+
+| field | type | meaning |
+|--------|--------|------------------------------------------------------------------------|
+| `v` | int | envelope version, `1`. Other versions are rejected. |
+| `seq` | uint | per-direction counter, starting at 1 on each connection |
+| `ts` | int | sender's clock, Unix milliseconds |
+| `type` | string | message type |
+
+Unknown `type`s are ignored (a newer server may send more). Frames are at most 1 MiB.
+
+### Server → device
+
+| type | fields |
+|-----------------|--------------------------------------------------------------------------------------------------|
+| `hello` | `device_id`, `robot_id`, `session`, `server_time` |
+| `config` | `limits` {`max_speed` 0..1, `max_turn` 0..1, `max_command_ms`}, `heartbeat_ms`, `deadman_ms`, `telemetry_ms` (≥ 500), `allowed` [kinds] |
+| `command` | `id` (idempotency key), `kind`, `value`, `deadline_ms`, `operator`, `role`, `target` (optional plugin name) |
+| `estop` | `reason`, `by` — latched |
+| `estop_clear` | `by` — the owner clears the remote e-stop |
+| `heartbeat_ack` | `t` (echo of the heartbeat's `t`; the Node measures RTT from it) |
+
+### Device → server
+
+| type | fields |
+|---------------|-----------------------------------------------------------------------------------------------------|
+| `status` | `agent_version`, `device_kind`, `os`, `arch`, `drivers` [{`name`, `driver`, `version`, `state`, `capabilities`}], `faults` [{`code`, `driver`, `message`}], `estop_latched`, `local_stop`, `video` |
+| `telemetry` | `battery` {`volts`, `percent`}, `rssi`, `sensors` {…}, `events` [{`name`, `driver`, `ts`, `fields`}] |
+| `ack` | `id`, `latency_ms` (receipt → plugin ack) |
+| `nack` | `id`, `fault_code`, `message` |
+| `heartbeat` | `t` (device clock, ms) |
+| `estop_state` | `latched` (remote e-stop), `local_stop` (local kill switch), `reason` |
+
+`status` and `estop_state` are sent on every connect and whenever a plugin changes state, faults or the latch changes.
+Plugin events (`cliff`, `picked_up`, `low_battery`, `heartbeat_lost`, …) travel in `telemetry.events` and are sent
+within 100 ms; sensor telemetry is sent at most every `telemetry_ms` (default 500 ms, so ≤ 2 Hz) **(to align: a
+separate `event` type?)**.
+
+### Commands
+
+| kind | value | motion |
+|------------|---------------------------------------------------------------------------------------------------------|--------|
+| `drive` | differential `{"throttle": -1..1, "steer": -1..1}`; mecanum `{"x": -1..1, "y": -1..1, "rotation": -1..1}` | yes |
+| `ptz` | `{"pan": -1..1, "tilt": -1..1, "zoom": -1..1}` absolute positions (0 = centre) | guarded |
+| `actuator` | `{"name": "lift", "value": …}` — number, object or bool per actuator ([plugins.md](plugins.md)) | guarded |
+| `say` | `{"text": "…"}` (1–1000 bytes; plugins may cap lower) | no |
+| `display` | `{"text": "…"}`, `{"face": "happy"}` or `{"image_png_b64": "…"}` | no |
+| `halt` | `{}` — stop every actuator now (not latched); always accepted | — |
+
+Sign conventions: `throttle`/`x` positive = forward; `steer` positive = turn right; `y` positive = strafe left;
+`rotation` positive = counter-clockwise (left); `pan` positive = right, `tilt` positive = up (plugins can invert a
+servo in their config to make the hardware match).
+
+`deadline_ms` is **relative to receipt** (milliseconds the command stays valid), not an absolute timestamp: a device's
+clock may be minutes off, and a relative deadline cannot be stretched by clock skew. Default 300 ms; capped at
+`max_command_ms` (default 1000). A held control is re-sent by the panel every 150 ms, each with a new `id`
+**(to align: ADR-043 says "absolute deadline")**.
+
+What the Node does with a command, in order:
+
+1. A repeated `id` is answered with the first result and never executed again (the cache lives for 10 minutes and
+ across reconnects).
+2. Unknown kind → `nack unsupported`. Kind not in `config.allowed` → `nack not_allowed` (`halt` is always allowed).
+3. `halt` → every plugin stops, `ack`.
+4. `drive`, `ptz`, `actuator` while the local kill switch is latched → `nack local_stop`; while the remote e-stop is
+ latched → `nack estopped`.
+5. The value is clamped in the core, before any plugin sees it: `throttle`, `x`, `y` to ±`max_speed`; `steer`,
+ `rotation` to ±`max_turn`; `pan`, `tilt`, `zoom` and a numeric actuator `value` to ±1. The limits are the stricter of
+ the server's `config.limits` and the local config's `limits`. Non-numbers → `nack bad_value`.
+6. It goes to the plugin named by `target`, or the first plugin whose `describe` declares the kind (for actuators, the
+ first listing the name). None → `nack unsupported`.
+7. The plugin's `ack`/`nack` is relayed. No answer within max(deadline, 1 s) → the plugin is told to stop and the
+ server gets `nack plugin_timeout`.
+
+Commands are never queued while offline and never replayed after a reconnect.
+
+### Fault codes
+
+`bad_frame`, `bad_value`, `unsupported`, `not_allowed`, `estopped`, `local_stop`, `no_heartbeat`,
+`not_ready`, `plugin_down`, `plugin_timeout`, `hardware`, `firmware_unsupported`, `not_connected`, `cliff`,
+`picked_up`, `shutting_down`. Plugins may add their own; they are passed through.
+
+### Heartbeat, deadman, reconnect
+
+- The device sends `heartbeat` every `heartbeat_ms` (default 1000). The server answers `heartbeat_ack`.
+- If nothing at all arrives from the server for `deadman_ms` (default 2 × `heartbeat_ms`: two missed heartbeats), the
+ Node closes the link. Every close stops every actuator at once.
+- Reconnect: exponential backoff from 0.5 s to 30 s with full jitter; at least 10 s after a `401`/`403`.
+
+## 3. Safety paths (ADR-043 decision 6)
+
+| event | what stops it | latched? |
+|----------------------------------------------|----------------------------------------------------|-----------------------------------------|
+| motion command's deadline passes | the plugin (its own timer) | no |
+| 2 missed heartbeats / link closed | the core → `stop` to every plugin | no |
+| server `estop` | the core → `estop` to every plugin | yes, persisted; `estop_clear` or `openvibe-node resume` |
+| `openvibe-node stop` (local kill switch) | the core → `estop` to every plugin | yes, persisted; only `openvibe-node resume` |
+| core shutdown | `stop`, then stdin closed | no |
+| core killed / crashed | the plugin: stdin EOF, or 1 s without a heartbeat | no |
+| plugin crash | restarted with backoff; starts stopped; `estop` re-sent if latched | — |
+| Cozmo cliff or pick-up | the Cozmo plugin, in the event callback | until the condition clears |
+
+The latch file is `latch.json` in the state directory. A corrupt latch file is read as "local stop latched".
+
+## 4. Video (WHIP)
+
+The Node publishes one H.264 track (Constrained Baseline, `profile-level-id=42e01f`, packetization-mode 1) to
+`whip_url` with `Authorization: Bearer <publish_key>`: `POST` with `Content-Type: application/sdp` → `201` with the
+answer and a `Location`; `DELETE <Location>` on shutdown. PLI/FIR from the far end request a keyframe. The session is
+re-established with backoff; video runs on its own goroutines and never waits on control or vice versa.
@@ -66,7 +66,6 @@ const (
@@ -66,7 +66,6 @@ const (
FaultEstopped = "estopped" // the remote e-stop is latched
FaultEstopped = "estopped" // the remote e-stop is latched
FaultLocalStop = "local_stop" // the local kill switch (`openvibe-node stop`) is latched
FaultLocalStop = "local_stop" // the local kill switch (`openvibe-node stop`) is latched
FaultNoHeartbeat = "no_heartbeat" // a plugin lost the core's heartbeat
FaultNoHeartbeat = "no_heartbeat" // a plugin lost the core's heartbeat
- FaultExpired = "expired" // the deadline had already passed on arrival
FaultNotReady = "not_ready" // the driver is starting or restarting
FaultNotReady = "not_ready" // the driver is starting or restarting
FaultPluginDown = "plugin_down" // the driver process is not running
FaultPluginDown = "plugin_down" // the driver process is not running
FaultPluginTimeout = "plugin_timeout" // the driver did not answer in time
FaultPluginTimeout = "plugin_timeout" // the driver did not answer in time
@@ -55,7 +55,8 @@ class DryRun(Plugin):
@@ -55,7 +55,8 @@ class DryRun(Plugin):
if cmd.kind == "drive":
if cmd.kind == "drive":
if "x" in v or "y" in v:
if "x" in v or "y" in v:
self.state["throttle"] = clamp(v.get("x", 0), -1, 1)
self.state["throttle"] = clamp(v.get("x", 0), -1, 1)
- self.state["steer"] = clamp(v.get("rotation", 0), -1, 1)
+ # rotation is counter-clockwise positive; steer is right positive.
+ self.state["steer"] = -clamp(v.get("rotation", 0), -1, 1)
else:
else:
self.state["throttle"] = clamp(v.get("throttle", 0), -1, 1)
self.state["throttle"] = clamp(v.get("throttle", 0), -1, 1)
self.state["steer"] = clamp(v.get("steer", 0), -1, 1)
self.state["steer"] = clamp(v.get("steer", 0), -1, 1)