openvibers/openvibe.link Commits
Node: install --worker, a machine that runs its owner's jobs (job user, public egress, the OpenVibe.Codes job runner) (#38)
Changes
@@@ -99,9 +99,10 @@ jobs:runs-on: ubuntu-lateststeps:- uses: actions/checkout@v4-- - run: shellcheck -s sh install/install.sh scripts/dist.sh scripts/test-install-stock-server.sh scripts/test-install-pair-args.sh++ - run: shellcheck -s sh install/install.sh scripts/dist.sh scripts/test-install-stock-server.sh scripts/test-install-pair-args.sh scripts/test-install-worker-config.sh- run: sh scripts/test-install-stock-server.sh- run: sh scripts/test-install-pair-args.sh++ - run: sh scripts/test-install-worker-config.shesp32-conformance:name: esp32 protocol conformance (desktop)@@@ -34,6 +34,10 @@ openvibe-node resume- **ESP32 library** ([`esp32/`](esp32/README.md), `OpenVibeNode`): the device side of the protocol for ESP32 boards —pairing, the control link, drive/actuator commands, telemetry, heartbeat, deadman and e-stop — but no jobs or video.++**A machine that runs your jobs:** add it on openvibe.bot/machines, which prints the same installer with `--worker`: the++job worker runs each job as an unprivileged user in a sandbox, and the OpenVibe.Codes job runner lets you start a coding++task on openvibe.codes/run from any browser and have this machine do it ([docs/worker.md](docs/worker.md)).++Docs: [protocol](docs/protocol.md) (the wire format with OpenVibe.Bot), [plugins](docs/plugins.md) (the driverprotocol, the bundled drivers, and where Actor's computer-control plugin attaches), [install](docs/install.md).Decisions: [ADR-043](docs/ADR-043-bot-devices-and-control.md).@@@ -20,6 +20,11 @@ ports, works behind any home router), and a local kill switch stops everything inames the device on openvibe.bot (default: the hostname).3. Confirm the new device on openvibe.bot.++**A machine that runs your jobs, not a robot:** openvibe.bot/machines prints the Network form with `--worker` (and no++`--driver`). It turns the job worker on with a dedicated `openvibe-job` user and the OpenVibe.Codes job runner, so a++coding task you start on openvibe.codes/run runs on this machine. [worker.md](worker.md#a-machine-that-runs-your-coding-tasks---worker)++has the details.++**Two pairing paths.** Which one the command uses is up to openvibe.bot; the installer takes both.- **Bot pairing** (the form above): `--robot rob_… --code CODE`. The Node redeems the code at@@@ -9,6 +9,27 @@ runtime: it runs through the same isolate path as a `code` entry, with the sameIt is single-shot like a `code` job: the Node does not offer the persistent Linux environment that lives acrosscommands, which `platform.runtime-class@1` also lets `linux` describe.++## A machine that runs your coding tasks (`--worker`)++++The simplest way to turn the worker on is the installer's `--worker` (Linux), which openvibe.bot/machines prints with a++one-time code: `curl -fsSL https://openvibe.bot/install | sh -s -- --network … --pairing pair_… --code … --worker`. It:++++- pairs the machine for its owner's own jobs (a Network pairing with ref `worker:<owner>`; OpenVibe.Bot reports it to++ the network as a `user-owned` offer naming the owner, OpenVibe.Run places only that person's jobs on it, and Bot++ refuses anyone else's);++- runs the service as root and every job as the unprivileged system user `openvibe-job` (`run_as`), with public egress++ (it installs `nftables`, `iproute2` and `util-linux` with apt when they are missing and turns IPv4 forwarding on);++- installs the OpenVibe.Codes job runner (`openvibe-codes job`, artifact `[email protected]`) into++ `/opt/openvibe-codes` when Node.js 20 or newer is installed system-wide, and declares it as a `code` entry;++- sizes the caps for a coding task (an hour, 4 GiB of memory, 3 GiB of scratch, one job at a time). Your own values win++ on a second run, and so do the keys you added.++++Then add your agent keys to the runner's `env` in `config.json` (`DEEPSEEK_API_KEY`, `ANTHROPIC_API_KEY`,++`OPENAI_API_KEY`, and `GITHUB_TOKEN` to push branches and open pull requests), install any agent CLIs system-wide++(`sudo npm install -g @anthropic-ai/claude-code @openai/codex`), and restart the service. A task started on++openvibe.codes/run then clones the repository inside the job's sandbox, runs the agents there, and hands back the++summary, the changed files and the patch; it never pushes to the branch it started from.++## Config`config.json`, section `worker` (defaults shown; a zero cap means the default):@@@ -3,6 +3,7 @@## curl -fsSL https://openvibe.bot/install | sh -s -- --robot <rob_…> --code <CODE> [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none]# curl -fsSL https://openvibe.bot/install | sh -s -- --network <URL> --pairing <pair_…> --code <CODE> [--driver …]++# curl -fsSL https://openvibe.bot/install | sh -s -- --network <URL> --pairing <pair_…> --code <CODE> --worker## Detects the OS and CPU, downloads the openvibe-node binary and the plugin bundle (checking their SHA-256), creates# a Python virtual environment for the plugins, disables the Adeept kit's stock control server if it finds it,@@@ -19,6 +20,10 @@# --driver KIND adeept (ordinary wheels), adeept-mecanum, cozmo (bridge), looi (bridge, Bluetooth), ros2 (a ROS 2# robot through rosbridge), ros2-holonomic (mecanum or omni wheels), none (dry run).# Default: none.++# --worker this machine runs its owner's jobs (openvibe.bot/machines): the job worker is turned on, jobs run++# as the unprivileged user openvibe-job in the worker's sandbox with public egress (the service++# runs as root, which run_as and the egress need), and the OpenVibe.Codes job runner is added when++# Node.js 20+ is installed system-wide. Linux only. docs/worker.md has the details.# --no-service install and pair, but do not install the service.# --local DIR install from DIR (openvibe-node-<os>-<arch> and openvibe-node-plugins.tar.gz) instead of# downloading.@@@ -33,6 +38,8 @@# OPENVIBE_CRONTAB_USER=FILE edit FILE as the invoking user's crontab instead of the real one.# OPENVIBE_RC_LOCAL=FILE edit FILE as rc.local instead of /etc/rc.local.# OPENVIBE_INSTALL_PAIR_ARGS_ONLY=1 prints the `openvibe-node pair` arguments the flags make, one per line, and exits.++# OPENVIBE_INSTALL_WORKER_CONFIG_ONLY=<config.json> writes the --worker section into that file (OPENVIBE_JOB_UID,++# OPENVIBE_JOB_GID, OPENVIBE_NODEJS and OPENVIBE_CODES_DIR stand in for the job user and the runner) and exits.set -euREPO_URL="https://github.com/OpenVibers/OpenVibe.Node"@@@ -45,10 +52,15 @@ PAIRING=""SERVICE=1LOCAL=""TAG="latest"++WORKER=0++# The OpenVibe.Codes release whose `openvibe-codes job` the worker runs (protocol [email protected]).++CODES_VERSION="${OPENVIBE_CODES_VERSION:-v0.3.0}"++CODES_DIR=/opt/openvibe-codesusage() {echo "usage: install.sh [--robot rob_… | --network URL --pairing pair_…] [--code CODE | CODE] [--name NAME]"-- echo " [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none] [--no-service] [--local DIR] [--version TAG]"++ echo " [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none] [--worker] [--no-service]"++ echo " [--local DIR] [--version TAG]"echo " curl -fsSL https://openvibe.bot/install | sh -s -- --robot rob_… --code ABCD-1234 --driver adeept"echo " curl -fsSL https://openvibe.bot/install | sh -s -- --network https://openvibe.network --pairing pair_… --code ABCD-1234"}@@@ -76,6 +88,7 @@ while [ $# -gt 0 ]; do--network=*) NETWORK="${1#*=}"; shift ;;--pairing) PAIRING="${2:-}"; shift 2 ;;--pairing=*) PAIRING="${1#*=}"; shift ;;++ --worker) WORKER=1; shift ;;--no-service) SERVICE=0; shift ;;--local) LOCAL="${2:-}"; shift 2 ;;--version) TAG="${2:-}"; shift 2 ;;@@@ -111,6 +124,65 @@ if [ "${OPENVIBE_INSTALL_PAIR_ARGS_ONLY:-0}" = 1 ]; thenexit 0fi++# write_worker_config <config.json> <job uid> <job gid> [<node.js> <codes dir>]: turn the job worker on in that config++# (kept as it is otherwise, created when missing): jobs as the job user, public egress, caps sized for a coding task,++# and, with a Node.js and a Codes install, the `[email protected]` job runner. Keys stay empty for the owner to add.++write_worker_config() {++ ${SUDO:-} python3 - "$@" <<'PY'++import json, os, sys++path, uid, gid = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])++node = sys.argv[4] if len(sys.argv) > 4 else ''++codes = sys.argv[5] if len(sys.argv) > 5 else ''++try:++ with open(path) as f:++ conf = json.load(f)++except FileNotFoundError:++ conf = {}++worker = conf.get('worker') or {}++worker.update({++ 'enabled': True,++ 'run_as': {'uid': uid, 'gid': gid},++ 'allow_same_user': False,++ 'egress': 'public',++})++caps = worker.get('caps') or {}++for k, v in {++ 'max_ttl_ms': 3600000, 'max_wall_ms': 3600000, 'max_cpu_ms': 3600000, 'max_mem_bytes': 4 << 30,++ 'max_output_bytes': 1 << 20, 'max_jobs': 1, 'max_cpu_millis': 2000, 'max_pids': 512,++ 'max_disk_bytes': 3 << 30, 'max_vm_bytes': 64 << 40,++}.items():++ caps.setdefault(k, v)++worker['caps'] = caps++functions = [f for f in worker.get('functions') or [] if f.get('name') != 'openvibe-codes']++if node and codes:++ previous = next((f for f in worker.get('functions') or [] if f.get('name') == 'openvibe-codes'), {})++ env = {'HOME': '/tmp', 'OPENVIBE_CODES_HOME': '/tmp/.openvibe-codes', 'PATH': '/usr/local/bin:/usr/bin:/bin'}++ for k, v in (previous.get('env') or {}).items():++ env.setdefault(k, v)++ for k in ('DEEPSEEK_API_KEY', 'ANTHROPIC_API_KEY', 'OPENAI_API_KEY', 'GITHUB_TOKEN'):++ env.setdefault(k, '')++ functions.append({++ 'name': 'openvibe-codes', 'version': '1.0.0', 'class': 'code',++ 'command': [node, os.path.join(codes, 'node_modules', 'openvibe-codes', 'bin', 'openvibe-codes.js'), 'job'],++ 'artifact_dir': codes, 'env': env,++ })++worker['functions'] = functions++conf['worker'] = worker++old = os.umask(0o077)++with open(path + '.tmp', 'w') as f:++ json.dump(conf, f, indent=2)++ f.write('\n')++os.umask(old)++os.replace(path + '.tmp', path)++PY++}++++if [ -n "${OPENVIBE_INSTALL_WORKER_CONFIG_ONLY:-}" ]; then++ write_worker_config "$OPENVIBE_INSTALL_WORKER_CONFIG_ONLY" "${OPENVIBE_JOB_UID:?}" "${OPENVIBE_JOB_GID:?}" \++ ${OPENVIBE_NODEJS:+"$OPENVIBE_NODEJS"} ${OPENVIBE_CODES_DIR:+"$OPENVIBE_CODES_DIR"}++ exit 0++fi++# Root for the service, /usr/local/bin and /etc.SUDO=""@@@ -274,8 +346,11 @@ $SUDO install -m 0755 "$TMP/$BIN_NAME" "$BIN_DIR/openvibe-node"say "installed $BIN_DIR/openvibe-node ($("$BIN_DIR/openvibe-node" version))"# ---- service account (Linux): hardware groups, no login ----++# A worker (--worker) runs the service as root: the job sandbox's run_as and its public egress (a veth, NAT and the++# Node's nftables table) need it, and each job still runs as the unprivileged openvibe-job user.SVC_USER=""--if [ "$OS" = linux ] && [ "$SERVICE" = 1 ]; then++if [ "$WORKER" = 1 ] && [ "$OS" != linux ]; then die "--worker is for Linux"; fi++if [ "$OS" = linux ] && [ "$SERVICE" = 1 ] && [ "$WORKER" = 0 ]; thenSVC_USER=openvibe-nodeif ! id "$SVC_USER" >/dev/null 2>&1; then$SUDO useradd --system --home-dir "$STATE" --no-create-home --shell /usr/sbin/nologin "$SVC_USER" 2>/dev/null ||@@@ -351,6 +426,52 @@ elsesay "keeping the existing $CONF/config.json"fi++# ---- the job worker (--worker): the job user, the egress tools, the Codes job runner, the worker config ----++if [ "$WORKER" = 1 ]; then++ if ! id openvibe-job >/dev/null 2>&1; then++ $SUDO useradd --system --no-create-home --shell /usr/sbin/nologin --user-group openvibe-job 2>/dev/null ||++ $SUDO adduser --system --no-create-home --shell /usr/sbin/nologin --group openvibe-job++ fi++ JOB_UID="$(id -u openvibe-job)"++ JOB_GID="$(id -g openvibe-job)"++ missing=""++ for tool in nft ip nsenter; do command -v "$tool" >/dev/null 2>&1 || missing="$missing $tool"; done++ if [ -n "$missing" ] && command -v apt-get >/dev/null 2>&1; then++ say "installing what the worker's network sandbox needs:$missing"++ $SUDO apt-get install -y nftables iproute2 util-linux >/dev/null 2>&1 || say "warning: apt-get could not install nftables iproute2 util-linux"++ fi++ for tool in nft ip nsenter; do command -v "$tool" >/dev/null 2>&1 || say "warning: $tool is missing: jobs that need the internet are refused until it is installed"; done++ if [ "$(cat /proc/sys/net/ipv4/ip_forward 2>/dev/null || echo 0)" != 1 ]; then++ say "turning on IPv4 forwarding (the worker NATs each job's own network; /etc/sysctl.d/90-openvibe-node-worker.conf)"++ $SUDO sh -c "printf 'net.ipv4.ip_forward = 1\n' > /etc/sysctl.d/90-openvibe-node-worker.conf"++ $SUDO sysctl -q -p /etc/sysctl.d/90-openvibe-node-worker.conf || say "warning: could not turn IPv4 forwarding on"++ fi++ NODEJS=""++ for candidate in /usr/bin/node /usr/local/bin/node; do++ if [ -x "$candidate" ] && [ "$("$candidate" -p 'process.versions.node.split(".")[0]' 2>/dev/null || echo 0)" -ge 20 ]; then NODEJS="$candidate"; break; fi++ done++ if [ -n "$NODEJS" ]; then++ say "adding the OpenVibe.Codes job runner $CODES_VERSION in $CODES_DIR"++ $SUDO mkdir -p "$CODES_DIR"++ $SUDO "$(dirname "$NODEJS")/npm" install --prefix "$CODES_DIR" --omit=dev --no-audit --no-fund --no-save --loglevel=error \++ "https://codeload.github.com/OpenVibers/OpenVibe.Codes/tar.gz/refs/tags/$CODES_VERSION" >/dev/null ||++ die "could not install OpenVibe.Codes $CODES_VERSION into $CODES_DIR"++ $SUDO chown -R root:root "$CODES_DIR"++ $SUDO chmod -R go-w "$CODES_DIR"++ $SUDO touch "$CONF/config.json"++ write_worker_config "$CONF/config.json" "$JOB_UID" "$JOB_GID" "$NODEJS" "$CODES_DIR"++ else++ say "no Node.js 20+ installed system-wide (/usr/bin/node or /usr/local/bin/node): the worker is on, but the coding"++ say "job runner is not. Install Node.js 22 (https://nodejs.org) and run this installer again with --worker."++ $SUDO touch "$CONF/config.json"++ write_worker_config "$CONF/config.json" "$JOB_UID" "$JOB_GID"++ fi++ say "the job worker is on: jobs run as openvibe-job in the worker's sandbox, with public egress"++ say "add your agent keys (DEEPSEEK_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, and GITHUB_TOKEN to push branches) to"++ say " worker.functions[openvibe-codes].env in $CONF/config.json, and install agent CLIs system-wide"++ say " (sudo npm install -g @anthropic-ai/claude-code @openai/codex), then: sudo systemctl restart openvibe-node"++fi++# ---- the Adeept kit's stock server: never run it (fixed admin:123456 login on 0.0.0.0:8888, MJPEG on :5000) ----disable_stock_servernnew file mode 100755@@@ -0,0 +1,53 @@++#!/bin/sh++# Tests install.sh --worker's config: OPENVIBE_INSTALL_WORKER_CONFIG_ONLY=<file> merges the job worker into that++# config.json (what was there stays) and exits before it downloads, installs or pairs anything. With a Node.js and a++# Codes install it declares the [email protected] job runner, and a second run keeps the owner's keys.++set -eu++++here="$(cd "$(dirname "$0")" && pwd)"++installer="$here/../install/install.sh"++tmp="$(mktemp -d)"++trap 'rm -rf "$tmp"' EXIT++fail() { printf 'FAIL: %s\n' "$*" >&2; exit 1; }++++conf="$tmp/config.json"++printf '{"server": "https://openvibe.bot", "plugins": [{"name": "dryrun"}]}\n' > "$conf"++OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 \++ OPENVIBE_NODEJS=/usr/bin/node OPENVIBE_CODES_DIR=/opt/openvibe-codes sh "$installer" --worker++python3 - "$conf" <<'PY' || fail "first run"++import json, sys++c = json.load(open(sys.argv[1]))++w = c['worker']++assert c['server'] == 'https://openvibe.bot' and c['plugins'] == [{'name': 'dryrun'}], 'the rest of the config stays'++assert w['enabled'] is True and w['run_as'] == {'uid': 2001, 'gid': 2002} and w['egress'] == 'public' and w['allow_same_user'] is False++assert w['caps']['max_wall_ms'] == 3600000 and w['caps']['max_mem_bytes'] == 4 << 30 and w['caps']['max_jobs'] == 1++[f] = w['functions']++assert (f['name'], f['version'], f['class'], f['artifact_dir']) == ('openvibe-codes', '1.0.0', 'code', '/opt/openvibe-codes')++assert f['command'] == ['/usr/bin/node', '/opt/openvibe-codes/node_modules/openvibe-codes/bin/openvibe-codes.js', 'job']++assert f['env']['DEEPSEEK_API_KEY'] == '' and f['env']['HOME'] == '/tmp'++PY++# The owner adds a key and changes a cap; running the installer again keeps both and declares the runner once.++python3 - "$conf" <<'PY'++import json, sys++c = json.load(open(sys.argv[1]))++c['worker']['functions'][0]['env']['DEEPSEEK_API_KEY'] = 'kept'++c['worker']['caps']['max_jobs'] = 2++json.dump(c, open(sys.argv[1], 'w'))++PY++OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 \++ OPENVIBE_NODEJS=/usr/bin/node OPENVIBE_CODES_DIR=/opt/openvibe-codes sh "$installer" --worker++python3 - "$conf" <<'PY' || fail "second run"++import json, sys++w = json.load(open(sys.argv[1]))['worker']++assert len(w['functions']) == 1 and w['functions'][0]['env']['DEEPSEEK_API_KEY'] == 'kept' and w['caps']['max_jobs'] == 2++PY++# Without Node.js the worker is on and declares no runner.++conf2="$tmp/bare.json"++OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf2" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 sh "$installer" --worker++python3 - "$conf2" <<'PY' || fail "no Node.js"++import json, os, stat, sys++w = json.load(open(sys.argv[1]))['worker']++assert w['enabled'] is True and w['functions'] == []++assert stat.S_IMODE(os.stat(sys.argv[1]).st_mode) == 0o600, 'the config stays private'++PY++echo "install --worker config: ok"
@@ -99,9 +99,10 @@ jobs:
@@ -99,9 +99,10 @@ jobs:
runs-on: ubuntu-latest
runs-on: ubuntu-latest
steps:
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v4
- - run: shellcheck -s sh install/install.sh scripts/dist.sh scripts/test-install-stock-server.sh scripts/test-install-pair-args.sh
+ - run: shellcheck -s sh install/install.sh scripts/dist.sh scripts/test-install-stock-server.sh scripts/test-install-pair-args.sh scripts/test-install-worker-config.sh
- run: sh scripts/test-install-stock-server.sh
- run: sh scripts/test-install-stock-server.sh
- run: sh scripts/test-install-pair-args.sh
- run: sh scripts/test-install-pair-args.sh
+ - run: sh scripts/test-install-worker-config.sh
esp32-conformance:
esp32-conformance:
name: esp32 protocol conformance (desktop)
name: esp32 protocol conformance (desktop)
@@ -34,6 +34,10 @@ openvibe-node resume
@@ -34,6 +34,10 @@ openvibe-node resume
- **ESP32 library** ([`esp32/`](esp32/README.md), `OpenVibeNode`): the device side of the protocol for ESP32 boards —
- **ESP32 library** ([`esp32/`](esp32/README.md), `OpenVibeNode`): the device side of the protocol for ESP32 boards —
pairing, the control link, drive/actuator commands, telemetry, heartbeat, deadman and e-stop — but no jobs or video.
pairing, the control link, drive/actuator commands, telemetry, heartbeat, deadman and e-stop — but no jobs or video.
+**A machine that runs your jobs:** add it on openvibe.bot/machines, which prints the same installer with `--worker`: the
+job worker runs each job as an unprivileged user in a sandbox, and the OpenVibe.Codes job runner lets you start a coding
+task on openvibe.codes/run from any browser and have this machine do it ([docs/worker.md](docs/worker.md)).
+
Docs: [protocol](docs/protocol.md) (the wire format with OpenVibe.Bot), [plugins](docs/plugins.md) (the driver
Docs: [protocol](docs/protocol.md) (the wire format with OpenVibe.Bot), [plugins](docs/plugins.md) (the driver
protocol, the bundled drivers, and where Actor's computer-control plugin attaches), [install](docs/install.md).
protocol, the bundled drivers, and where Actor's computer-control plugin attaches), [install](docs/install.md).
Decisions: [ADR-043](docs/ADR-043-bot-devices-and-control.md).
Decisions: [ADR-043](docs/ADR-043-bot-devices-and-control.md).
@@ -20,6 +20,11 @@ ports, works behind any home router), and a local kill switch stops everything i
@@ -20,6 +20,11 @@ ports, works behind any home router), and a local kill switch stops everything i
names the device on openvibe.bot (default: the hostname).
names the device on openvibe.bot (default: the hostname).
3. Confirm the new device on openvibe.bot.
3. Confirm the new device on openvibe.bot.
+**A machine that runs your jobs, not a robot:** openvibe.bot/machines prints the Network form with `--worker` (and no
+`--driver`). It turns the job worker on with a dedicated `openvibe-job` user and the OpenVibe.Codes job runner, so a
+coding task you start on openvibe.codes/run runs on this machine. [worker.md](worker.md#a-machine-that-runs-your-coding-tasks---worker)
+has the details.
+
**Two pairing paths.** Which one the command uses is up to openvibe.bot; the installer takes both.
**Two pairing paths.** Which one the command uses is up to openvibe.bot; the installer takes both.
- **Bot pairing** (the form above): `--robot rob_… --code CODE`. The Node redeems the code at
- **Bot pairing** (the form above): `--robot rob_… --code CODE`. The Node redeems the code at
@@ -9,6 +9,27 @@ runtime: it runs through the same isolate path as a `code` entry, with the same
@@ -9,6 +9,27 @@ runtime: it runs through the same isolate path as a `code` entry, with the same
It is single-shot like a `code` job: the Node does not offer the persistent Linux environment that lives across
It is single-shot like a `code` job: the Node does not offer the persistent Linux environment that lives across
commands, which `platform.runtime-class@1` also lets `linux` describe.
commands, which `platform.runtime-class@1` also lets `linux` describe.
+## A machine that runs your coding tasks (`--worker`)
+
+The simplest way to turn the worker on is the installer's `--worker` (Linux), which openvibe.bot/machines prints with a
+one-time code: `curl -fsSL https://openvibe.bot/install | sh -s -- --network … --pairing pair_… --code … --worker`. It:
+
+- pairs the machine for its owner's own jobs (a Network pairing with ref `worker:<owner>`; OpenVibe.Bot reports it to
+ the network as a `user-owned` offer naming the owner, OpenVibe.Run places only that person's jobs on it, and Bot
+ refuses anyone else's);
+- runs the service as root and every job as the unprivileged system user `openvibe-job` (`run_as`), with public egress
+ (it installs `nftables`, `iproute2` and `util-linux` with apt when they are missing and turns IPv4 forwarding on);
+- installs the OpenVibe.Codes job runner (`openvibe-codes job`, artifact `[email protected]`) into
+ `/opt/openvibe-codes` when Node.js 20 or newer is installed system-wide, and declares it as a `code` entry;
+- sizes the caps for a coding task (an hour, 4 GiB of memory, 3 GiB of scratch, one job at a time). Your own values win
+ on a second run, and so do the keys you added.
+
+Then add your agent keys to the runner's `env` in `config.json` (`DEEPSEEK_API_KEY`, `ANTHROPIC_API_KEY`,
+`OPENAI_API_KEY`, and `GITHUB_TOKEN` to push branches and open pull requests), install any agent CLIs system-wide
+(`sudo npm install -g @anthropic-ai/claude-code @openai/codex`), and restart the service. A task started on
+openvibe.codes/run then clones the repository inside the job's sandbox, runs the agents there, and hands back the
+summary, the changed files and the patch; it never pushes to the branch it started from.
+
## Config
## Config
`config.json`, section `worker` (defaults shown; a zero cap means the default):
`config.json`, section `worker` (defaults shown; a zero cap means the default):
@@ -3,6 +3,7 @@
@@ -3,6 +3,7 @@
#
#
# curl -fsSL https://openvibe.bot/install | sh -s -- --robot <rob_…> --code <CODE> [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none]
# curl -fsSL https://openvibe.bot/install | sh -s -- --robot <rob_…> --code <CODE> [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none]
# curl -fsSL https://openvibe.bot/install | sh -s -- --network <URL> --pairing <pair_…> --code <CODE> [--driver …]
# curl -fsSL https://openvibe.bot/install | sh -s -- --network <URL> --pairing <pair_…> --code <CODE> [--driver …]
+# curl -fsSL https://openvibe.bot/install | sh -s -- --network <URL> --pairing <pair_…> --code <CODE> --worker
#
#
# Detects the OS and CPU, downloads the openvibe-node binary and the plugin bundle (checking their SHA-256), creates
# Detects the OS and CPU, downloads the openvibe-node binary and the plugin bundle (checking their SHA-256), creates
# a Python virtual environment for the plugins, disables the Adeept kit's stock control server if it finds it,
# a Python virtual environment for the plugins, disables the Adeept kit's stock control server if it finds it,
@@ -19,6 +20,10 @@
@@ -19,6 +20,10 @@
# --driver KIND adeept (ordinary wheels), adeept-mecanum, cozmo (bridge), looi (bridge, Bluetooth), ros2 (a ROS 2
# --driver KIND adeept (ordinary wheels), adeept-mecanum, cozmo (bridge), looi (bridge, Bluetooth), ros2 (a ROS 2
# robot through rosbridge), ros2-holonomic (mecanum or omni wheels), none (dry run).
# robot through rosbridge), ros2-holonomic (mecanum or omni wheels), none (dry run).
# Default: none.
# Default: none.
+# --worker this machine runs its owner's jobs (openvibe.bot/machines): the job worker is turned on, jobs run
+# as the unprivileged user openvibe-job in the worker's sandbox with public egress (the service
+# runs as root, which run_as and the egress need), and the OpenVibe.Codes job runner is added when
+# Node.js 20+ is installed system-wide. Linux only. docs/worker.md has the details.
# --no-service install and pair, but do not install the service.
# --no-service install and pair, but do not install the service.
# --local DIR install from DIR (openvibe-node-<os>-<arch> and openvibe-node-plugins.tar.gz) instead of
# --local DIR install from DIR (openvibe-node-<os>-<arch> and openvibe-node-plugins.tar.gz) instead of
# downloading.
# downloading.
@@ -33,6 +38,8 @@
@@ -33,6 +38,8 @@
# OPENVIBE_CRONTAB_USER=FILE edit FILE as the invoking user's crontab instead of the real one.
# OPENVIBE_CRONTAB_USER=FILE edit FILE as the invoking user's crontab instead of the real one.
# OPENVIBE_RC_LOCAL=FILE edit FILE as rc.local instead of /etc/rc.local.
# OPENVIBE_RC_LOCAL=FILE edit FILE as rc.local instead of /etc/rc.local.
# OPENVIBE_INSTALL_PAIR_ARGS_ONLY=1 prints the `openvibe-node pair` arguments the flags make, one per line, and exits.
# OPENVIBE_INSTALL_PAIR_ARGS_ONLY=1 prints the `openvibe-node pair` arguments the flags make, one per line, and exits.
+# OPENVIBE_INSTALL_WORKER_CONFIG_ONLY=<config.json> writes the --worker section into that file (OPENVIBE_JOB_UID,
+# OPENVIBE_JOB_GID, OPENVIBE_NODEJS and OPENVIBE_CODES_DIR stand in for the job user and the runner) and exits.
set -eu
set -eu
REPO_URL="https://github.com/OpenVibers/OpenVibe.Node"
REPO_URL="https://github.com/OpenVibers/OpenVibe.Node"
@@ -45,10 +52,15 @@ PAIRING=""
@@ -45,10 +52,15 @@ PAIRING=""
SERVICE=1
SERVICE=1
LOCAL=""
LOCAL=""
TAG="latest"
TAG="latest"
+WORKER=0
+# The OpenVibe.Codes release whose `openvibe-codes job` the worker runs (protocol [email protected]).
+CODES_VERSION="${OPENVIBE_CODES_VERSION:-v0.3.0}"
+CODES_DIR=/opt/openvibe-codes
usage() {
usage() {
echo "usage: install.sh [--robot rob_… | --network URL --pairing pair_…] [--code CODE | CODE] [--name NAME]"
echo "usage: install.sh [--robot rob_… | --network URL --pairing pair_…] [--code CODE | CODE] [--name NAME]"
- echo " [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none] [--no-service] [--local DIR] [--version TAG]"
+ echo " [--driver adeept|adeept-mecanum|cozmo|looi|ros2|ros2-holonomic|none] [--worker] [--no-service]"
+ echo " [--local DIR] [--version TAG]"
echo " curl -fsSL https://openvibe.bot/install | sh -s -- --robot rob_… --code ABCD-1234 --driver adeept"
echo " curl -fsSL https://openvibe.bot/install | sh -s -- --robot rob_… --code ABCD-1234 --driver adeept"
echo " curl -fsSL https://openvibe.bot/install | sh -s -- --network https://openvibe.network --pairing pair_… --code ABCD-1234"
echo " curl -fsSL https://openvibe.bot/install | sh -s -- --network https://openvibe.network --pairing pair_… --code ABCD-1234"
}
}
@@ -76,6 +88,7 @@ while [ $# -gt 0 ]; do
@@ -76,6 +88,7 @@ while [ $# -gt 0 ]; do
--network=*) NETWORK="${1#*=}"; shift ;;
--network=*) NETWORK="${1#*=}"; shift ;;
--pairing) PAIRING="${2:-}"; shift 2 ;;
--pairing) PAIRING="${2:-}"; shift 2 ;;
--pairing=*) PAIRING="${1#*=}"; shift ;;
--pairing=*) PAIRING="${1#*=}"; shift ;;
+ --worker) WORKER=1; shift ;;
--no-service) SERVICE=0; shift ;;
--no-service) SERVICE=0; shift ;;
--local) LOCAL="${2:-}"; shift 2 ;;
--local) LOCAL="${2:-}"; shift 2 ;;
--version) TAG="${2:-}"; shift 2 ;;
--version) TAG="${2:-}"; shift 2 ;;
@@ -111,6 +124,65 @@ if [ "${OPENVIBE_INSTALL_PAIR_ARGS_ONLY:-0}" = 1 ]; then
@@ -111,6 +124,65 @@ if [ "${OPENVIBE_INSTALL_PAIR_ARGS_ONLY:-0}" = 1 ]; then
exit 0
exit 0
fi
fi
+# write_worker_config <config.json> <job uid> <job gid> [<node.js> <codes dir>]: turn the job worker on in that config
+# (kept as it is otherwise, created when missing): jobs as the job user, public egress, caps sized for a coding task,
+# and, with a Node.js and a Codes install, the `[email protected]` job runner. Keys stay empty for the owner to add.
+write_worker_config() {
+ ${SUDO:-} python3 - "$@" <<'PY'
+import json, os, sys
+path, uid, gid = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
+node = sys.argv[4] if len(sys.argv) > 4 else ''
+codes = sys.argv[5] if len(sys.argv) > 5 else ''
+try:
+ with open(path) as f:
+ conf = json.load(f)
+except FileNotFoundError:
+ conf = {}
+worker = conf.get('worker') or {}
+worker.update({
+ 'enabled': True,
+ 'run_as': {'uid': uid, 'gid': gid},
+ 'allow_same_user': False,
+ 'egress': 'public',
+})
+caps = worker.get('caps') or {}
+for k, v in {
+ 'max_ttl_ms': 3600000, 'max_wall_ms': 3600000, 'max_cpu_ms': 3600000, 'max_mem_bytes': 4 << 30,
+ 'max_output_bytes': 1 << 20, 'max_jobs': 1, 'max_cpu_millis': 2000, 'max_pids': 512,
+ 'max_disk_bytes': 3 << 30, 'max_vm_bytes': 64 << 40,
+}.items():
+ caps.setdefault(k, v)
+worker['caps'] = caps
+functions = [f for f in worker.get('functions') or [] if f.get('name') != 'openvibe-codes']
+if node and codes:
+ previous = next((f for f in worker.get('functions') or [] if f.get('name') == 'openvibe-codes'), {})
+ env = {'HOME': '/tmp', 'OPENVIBE_CODES_HOME': '/tmp/.openvibe-codes', 'PATH': '/usr/local/bin:/usr/bin:/bin'}
+ for k, v in (previous.get('env') or {}).items():
+ env.setdefault(k, v)
+ for k in ('DEEPSEEK_API_KEY', 'ANTHROPIC_API_KEY', 'OPENAI_API_KEY', 'GITHUB_TOKEN'):
+ env.setdefault(k, '')
+ functions.append({
+ 'name': 'openvibe-codes', 'version': '1.0.0', 'class': 'code',
+ 'command': [node, os.path.join(codes, 'node_modules', 'openvibe-codes', 'bin', 'openvibe-codes.js'), 'job'],
+ 'artifact_dir': codes, 'env': env,
+ })
+worker['functions'] = functions
+conf['worker'] = worker
+old = os.umask(0o077)
+with open(path + '.tmp', 'w') as f:
+ json.dump(conf, f, indent=2)
+ f.write('\n')
+os.umask(old)
+os.replace(path + '.tmp', path)
+PY
+}
+
+if [ -n "${OPENVIBE_INSTALL_WORKER_CONFIG_ONLY:-}" ]; then
+ write_worker_config "$OPENVIBE_INSTALL_WORKER_CONFIG_ONLY" "${OPENVIBE_JOB_UID:?}" "${OPENVIBE_JOB_GID:?}" \
+ ${OPENVIBE_NODEJS:+"$OPENVIBE_NODEJS"} ${OPENVIBE_CODES_DIR:+"$OPENVIBE_CODES_DIR"}
+ exit 0
+fi
+
# Root for the service, /usr/local/bin and /etc.
# Root for the service, /usr/local/bin and /etc.
SUDO=""
SUDO=""
@@ -274,8 +346,11 @@ $SUDO install -m 0755 "$TMP/$BIN_NAME" "$BIN_DIR/openvibe-node"
@@ -274,8 +346,11 @@ $SUDO install -m 0755 "$TMP/$BIN_NAME" "$BIN_DIR/openvibe-node"
say "installed $BIN_DIR/openvibe-node ($("$BIN_DIR/openvibe-node" version))"
say "installed $BIN_DIR/openvibe-node ($("$BIN_DIR/openvibe-node" version))"
# ---- service account (Linux): hardware groups, no login ----
# ---- service account (Linux): hardware groups, no login ----
+# A worker (--worker) runs the service as root: the job sandbox's run_as and its public egress (a veth, NAT and the
+# Node's nftables table) need it, and each job still runs as the unprivileged openvibe-job user.
SVC_USER=""
SVC_USER=""
-if [ "$OS" = linux ] && [ "$SERVICE" = 1 ]; then
+if [ "$WORKER" = 1 ] && [ "$OS" != linux ]; then die "--worker is for Linux"; fi
+if [ "$OS" = linux ] && [ "$SERVICE" = 1 ] && [ "$WORKER" = 0 ]; then
SVC_USER=openvibe-node
SVC_USER=openvibe-node
if ! id "$SVC_USER" >/dev/null 2>&1; then
if ! id "$SVC_USER" >/dev/null 2>&1; then
$SUDO useradd --system --home-dir "$STATE" --no-create-home --shell /usr/sbin/nologin "$SVC_USER" 2>/dev/null ||
$SUDO useradd --system --home-dir "$STATE" --no-create-home --shell /usr/sbin/nologin "$SVC_USER" 2>/dev/null ||
@@ -351,6 +426,52 @@ else
@@ -351,6 +426,52 @@ else
say "keeping the existing $CONF/config.json"
say "keeping the existing $CONF/config.json"
fi
fi
+# ---- the job worker (--worker): the job user, the egress tools, the Codes job runner, the worker config ----
+if [ "$WORKER" = 1 ]; then
+ if ! id openvibe-job >/dev/null 2>&1; then
+ $SUDO useradd --system --no-create-home --shell /usr/sbin/nologin --user-group openvibe-job 2>/dev/null ||
+ $SUDO adduser --system --no-create-home --shell /usr/sbin/nologin --group openvibe-job
+ fi
+ JOB_UID="$(id -u openvibe-job)"
+ JOB_GID="$(id -g openvibe-job)"
+ missing=""
+ for tool in nft ip nsenter; do command -v "$tool" >/dev/null 2>&1 || missing="$missing $tool"; done
+ if [ -n "$missing" ] && command -v apt-get >/dev/null 2>&1; then
+ say "installing what the worker's network sandbox needs:$missing"
+ $SUDO apt-get install -y nftables iproute2 util-linux >/dev/null 2>&1 || say "warning: apt-get could not install nftables iproute2 util-linux"
+ fi
+ for tool in nft ip nsenter; do command -v "$tool" >/dev/null 2>&1 || say "warning: $tool is missing: jobs that need the internet are refused until it is installed"; done
+ if [ "$(cat /proc/sys/net/ipv4/ip_forward 2>/dev/null || echo 0)" != 1 ]; then
+ say "turning on IPv4 forwarding (the worker NATs each job's own network; /etc/sysctl.d/90-openvibe-node-worker.conf)"
+ $SUDO sh -c "printf 'net.ipv4.ip_forward = 1\n' > /etc/sysctl.d/90-openvibe-node-worker.conf"
+ $SUDO sysctl -q -p /etc/sysctl.d/90-openvibe-node-worker.conf || say "warning: could not turn IPv4 forwarding on"
+ fi
+ NODEJS=""
+ for candidate in /usr/bin/node /usr/local/bin/node; do
+ if [ -x "$candidate" ] && [ "$("$candidate" -p 'process.versions.node.split(".")[0]' 2>/dev/null || echo 0)" -ge 20 ]; then NODEJS="$candidate"; break; fi
+ done
+ if [ -n "$NODEJS" ]; then
+ say "adding the OpenVibe.Codes job runner $CODES_VERSION in $CODES_DIR"
+ $SUDO mkdir -p "$CODES_DIR"
+ $SUDO "$(dirname "$NODEJS")/npm" install --prefix "$CODES_DIR" --omit=dev --no-audit --no-fund --no-save --loglevel=error \
+ "https://codeload.github.com/OpenVibers/OpenVibe.Codes/tar.gz/refs/tags/$CODES_VERSION" >/dev/null ||
+ die "could not install OpenVibe.Codes $CODES_VERSION into $CODES_DIR"
+ $SUDO chown -R root:root "$CODES_DIR"
+ $SUDO chmod -R go-w "$CODES_DIR"
+ $SUDO touch "$CONF/config.json"
+ write_worker_config "$CONF/config.json" "$JOB_UID" "$JOB_GID" "$NODEJS" "$CODES_DIR"
+ else
+ say "no Node.js 20+ installed system-wide (/usr/bin/node or /usr/local/bin/node): the worker is on, but the coding"
+ say "job runner is not. Install Node.js 22 (https://nodejs.org) and run this installer again with --worker."
+ $SUDO touch "$CONF/config.json"
+ write_worker_config "$CONF/config.json" "$JOB_UID" "$JOB_GID"
+ fi
+ say "the job worker is on: jobs run as openvibe-job in the worker's sandbox, with public egress"
+ say "add your agent keys (DEEPSEEK_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, and GITHUB_TOKEN to push branches) to"
+ say " worker.functions[openvibe-codes].env in $CONF/config.json, and install agent CLIs system-wide"
+ say " (sudo npm install -g @anthropic-ai/claude-code @openai/codex), then: sudo systemctl restart openvibe-node"
+fi
+
# ---- the Adeept kit's stock server: never run it (fixed admin:123456 login on 0.0.0.0:8888, MJPEG on :5000) ----
# ---- the Adeept kit's stock server: never run it (fixed admin:123456 login on 0.0.0.0:8888, MJPEG on :5000) ----
disable_stock_server
disable_stock_server
new file mode 100755
new file mode 100755
@@ -0,0 +1,53 @@
@@ -0,0 +1,53 @@
+#!/bin/sh
+# Tests install.sh --worker's config: OPENVIBE_INSTALL_WORKER_CONFIG_ONLY=<file> merges the job worker into that
+# config.json (what was there stays) and exits before it downloads, installs or pairs anything. With a Node.js and a
+# Codes install it declares the [email protected] job runner, and a second run keeps the owner's keys.
+set -eu
+
+here="$(cd "$(dirname "$0")" && pwd)"
+installer="$here/../install/install.sh"
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+fail() { printf 'FAIL: %s\n' "$*" >&2; exit 1; }
+
+conf="$tmp/config.json"
+printf '{"server": "https://openvibe.bot", "plugins": [{"name": "dryrun"}]}\n' > "$conf"
+OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 \
+ OPENVIBE_NODEJS=/usr/bin/node OPENVIBE_CODES_DIR=/opt/openvibe-codes sh "$installer" --worker
+python3 - "$conf" <<'PY' || fail "first run"
+import json, sys
+c = json.load(open(sys.argv[1]))
+w = c['worker']
+assert c['server'] == 'https://openvibe.bot' and c['plugins'] == [{'name': 'dryrun'}], 'the rest of the config stays'
+assert w['enabled'] is True and w['run_as'] == {'uid': 2001, 'gid': 2002} and w['egress'] == 'public' and w['allow_same_user'] is False
+assert w['caps']['max_wall_ms'] == 3600000 and w['caps']['max_mem_bytes'] == 4 << 30 and w['caps']['max_jobs'] == 1
+[f] = w['functions']
+assert (f['name'], f['version'], f['class'], f['artifact_dir']) == ('openvibe-codes', '1.0.0', 'code', '/opt/openvibe-codes')
+assert f['command'] == ['/usr/bin/node', '/opt/openvibe-codes/node_modules/openvibe-codes/bin/openvibe-codes.js', 'job']
+assert f['env']['DEEPSEEK_API_KEY'] == '' and f['env']['HOME'] == '/tmp'
+PY
+# The owner adds a key and changes a cap; running the installer again keeps both and declares the runner once.
+python3 - "$conf" <<'PY'
+import json, sys
+c = json.load(open(sys.argv[1]))
+c['worker']['functions'][0]['env']['DEEPSEEK_API_KEY'] = 'kept'
+c['worker']['caps']['max_jobs'] = 2
+json.dump(c, open(sys.argv[1], 'w'))
+PY
+OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 \
+ OPENVIBE_NODEJS=/usr/bin/node OPENVIBE_CODES_DIR=/opt/openvibe-codes sh "$installer" --worker
+python3 - "$conf" <<'PY' || fail "second run"
+import json, sys
+w = json.load(open(sys.argv[1]))['worker']
+assert len(w['functions']) == 1 and w['functions'][0]['env']['DEEPSEEK_API_KEY'] == 'kept' and w['caps']['max_jobs'] == 2
+PY
+# Without Node.js the worker is on and declares no runner.
+conf2="$tmp/bare.json"
+OPENVIBE_INSTALL_WORKER_CONFIG_ONLY="$conf2" OPENVIBE_JOB_UID=2001 OPENVIBE_JOB_GID=2002 sh "$installer" --worker
+python3 - "$conf2" <<'PY' || fail "no Node.js"
+import json, os, stat, sys
+w = json.load(open(sys.argv[1]))['worker']
+assert w['enabled'] is True and w['functions'] == []
+assert stat.S_IMODE(os.stat(sys.argv[1]).st_mode) == 0o600, 'the config stays private'
+PY
+echo "install --worker config: ok"