openvibers/openvibe.link Commits

Node: the live screen — screen_stream publishes the machine's screen to its owner's private stream (v0.12.0) (#49)

OP OpenVibers committed 29 days ago · tag v0.12. · 0 files, +0 −0

* Node: the live screen — screen_stream publishes the machine's screen to its owner's private stream (v0.12.0)

The live screen (plan T14, "screen streaming to the Console"): a machine's owner watches its screen live on openvibe.bot. Contracts 0.182.0 adds the device kind `screen_stream`.

**Core** (`internal/node/screen.go`):
- `screen_stream` start carries the WHIP URL and publish key of the machine's own private OpenRestream stream, which Bot fills in. The core asks the computer plugin first, since it applies the owner's level and has the capture tool, and never passes it the URL or key. Once the plugin agrees, the core publishes the plugin's frames on a second WHIP session, separate from a robot's camera.
- One stream at a time; a new start replaces the running one.
- A stream ends when:
  - its time is up (30 s to 30 min, 10 min by default);
  - it gets stop;
  - the plugin reports it ended (matched by the stream's id, so a newer stream is never stopped);
  - within 250 ms of the local kill switch or an e-stop.
- A start is refused while a stop is latched.
- The WHIP URL must be https with no credentials in it (http only to loopback, in tests).
- `openvibe-node status` shows `Screen: live … until …` while it runs.

**Computer plugin 0.7.0:**
- Captures a JPEG frame 1 to 5 times a second (2 by default) with the same tools as `screen`.
  - Pillow resizes frames to at most 1600 px wide.
  - Without Pillow, the tool writes JPEG itself: screencapture, grim, import and scrot can; gnome-screenshot is refused.
- Frames go to the core as video with `stream: "screen"`, from a private temporary folder that is removed afterwards.
- Runs at every level, like a screenshot. `allow_screen_stream: false` turns it off.
- Notifies the desktop when it starts.
- Ends on stop, an e-stop or the kill switch.
- Telemetry carries `computer_screen_stream` (`live`/`off`), and `describe` reports `screen.stream`.

**Tests:**
- Go (`screen_test.go`): the real Python plugin with a fake `grim` and a test WHIP receiver.
  - Frames are published with the key as the bearer.
  - Status shows live, and stop ends it (WHIP DELETE).
  - The kill switch ends it within 3 s and refuses a new start.
  - Bad targets are refused: http to a public address, credentials in the URL, no key, an unknown action.
- Python: frames at observe, stop and cleanup, e-stop, replacement and time-up with stream ids, refusals, capability and native JPEG argv.
- `go test ./...` and the plugin tests pass.

Docs: computer.md (The live screen), protocol.md, README.

* Node: the screen stream tests stub the plugin with plain functions (Python 3.9 cannot call a staticmethod object)