openvibers/openrestream Commits
OpenRe carries JSMPEG (plan T4, transports step 1): one protocol name per transport (rtmp, webrtc, jsmpeg; whip accepted on input as webrtc; worker kinds rtmp-ingest, restream, webrtc, jsmpeg; migration 0002 rewrites stored names), and a real jsmpeg worker: MPEG-TS POST /<key>/<w>/<h>/ admitted by key like RTMP, WebSocket viewers by playback id (viewer count in heartbeats and the session), a loopback tap for restream, generations/drain/lease like the RTMP worker; the playback descriptor carries jsmpeg { ws_url, width, height }; restream reads the tap; a jsmpeg recording request fails with a stated reason (openre.recording.failed), never silently. The webrtc-ingest and sfu stubs become one webrtc stub. [email protected]; cutover.md gains the JSMPEG slot procedure. npm test 19/23 (4 need PostgreSQL), test:pg 23/23.
Changes
@@@ -43,11 +43,11 @@ The ingest/restream control plane and runtime extracted from OpenVibe.Live. Live|---|---|---|---|| RTMP ingest | `server/streaming/rtmp-server.js` (node-media-server) | `workers/rtmp-ingest.js` — same NMS session code; OpenRe's own keys; publish renamed to `/live/<session id>` so no key reaches a play URL; loopback RTMP play + HTTP-FLV per generation; SO_REUSEPORT drain | **ported, tested with real ffmpeg** || Restream (RTMP source) | `server/streaming/restream-manager.js` | `workers/restream/*` — same ffmpeg arguments (codec copy from HTTP-FLV), Twitch→RTMPS, Kick `/app`, SRT options, live ACK by `-progress`, backoff, rapid-crash circuit breaker, destination cooldown | **ported, tested with real ffmpeg** |--| Restream from JSMPEG / WebRTC sources | same | encoder args ported (`encodingArgs`), sources not (they need those ingests) | not ported |++| Restream from a JSMPEG source | same | `jsmpegArgs()` — re-encodes the worker's MPEG-TS data tap (`http://127.0.0.1:<tapPort>/tap/<session>.ts`) | **ported, tested with real ffmpeg** |++| Restream from a WebRTC source | same | encoder args ported (`encodingArgs`), source not (it needs that ingest) | not ported || Recording | `server/streaming/recorder.js` + `server/media-client.js` | `server/store/recordings.js` + `server/media-client.js` — same Media API v1 calls (create VOD, `ingest/rtmp`, finalize, delete shell / clips-only) driven by the coordinator; Media pulls the loopback play URL | **ported (RTMP)**, tested against a stub Media that really pulls the URL |--| WHIP / WebRTC ingest | `whip-handler.js` (werift) | `workers/webrtc-ingest.js`: worker interface + coordinator registration only | **not ported** |--| SFU | `webrtc-sfu.js` + `broadcast-server.js` (mediasoup) | `workers/sfu.js`: worker interface only | **not ported** |--| JSMPEG | `jsmpeg-relay.js` | `workers/jsmpeg.js`: worker interface only | **not ported** |++| WebRTC (WHIP ingest + SFU + viewer signaling) | `whip-handler.js` + `webrtc-sfu.js` + `broadcast-server.js` (werift + mediasoup; one worker owns all of it) | `workers/webrtc.js`: worker interface + coordinator registration only | **not ported** |++| JSMPEG | `jsmpeg-relay.js` | `workers/jsmpeg.js` — MPEG-TS POST admission by key in path, WebSocket viewers by playback id, loopback MPEG-TS data tap for restream; SO_REUSEPORT drain | **ported, tested with real ffmpeg** || OAuth-linked destinations (per go-live Twitch/YouTube key refresh, YouTube broadcast creation) | `restream-manager._refreshDestFromConnection` | no (Live owns platform OAuth); OpenRe pushes to the stored key | **not ported** || Viewer counts from platforms, chat relay, PowerChat | restream-manager / integrations | no (product features, stay in Live) | not in scope |@@@ -134,11 +134,12 @@ Written to `event_outbox` in the same transaction as the change (openvibe-sdk `c| `openre.session.failed` | `ingest_session` | worker lost, lease expired, … (`was_live`, reason) || `openre.output.healthy` / `openre.output.failed` | `output` | an output confirmed live / gave up (error, cooldown) || `openre.recording.requested` | `recording` | Media accepted the recording request (Media app + VOD id) |++| `openre.recording.failed` | `recording` | a recording could not start (e.g. `recording is not available for jsmpeg`) || `openre.key.rotated` | `stream` | new key id + hint, retired key ids, grace end |## Playback--`GET /api/v1/sessions/:id/playback` returns a descriptor: `flv.internal_url` (loopback HTTP-FLV on the worker that holds the session — for services on this host), `flv.public_url` (`https://openre.stream/play/<session>.flv`, proxied by the API for non-private streams), `rtmp.internal_url` (loopback RTMP play, what Media records from). `webrtc` and `hls` are `null` (not produced). Live's player keeps using its own `/api/streams/rtmp-proxy/:id.flv`, which for an OpenRe session proxies `flv.internal_url` (see the Live patch).++`GET /api/v1/sessions/:id/playback` returns a descriptor: `flv.internal_url` (loopback HTTP-FLV on the worker that holds the session — for services on this host), `flv.public_url` (`https://openre.stream/play/<session>.flv`, proxied by the API for non-private streams), `rtmp.internal_url` (loopback RTMP play, what Media records from). A **jsmpeg** session's descriptor is `jsmpeg: { ws_url, width, height }` (`ws_url` is the worker's public WebSocket path carrying the session's playback id, never the key; `width`/`height` come from the ffmpeg POST path), plus `jsmpeg.tap_internal_url`, the loopback MPEG-TS tap the restream worker reads. `webrtc` and `hls` are `null` (not produced). The session API also reports `viewers` (connected JSMPEG viewers, from the worker heartbeat). Live's player keeps using its own `/api/streams/rtmp-proxy/:id.flv`, which for an OpenRe session proxies `flv.internal_url` (see the Live patch).## Recording@@@ -162,6 +163,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).| 9935/tcp (loopback) | Live's HTTP-FLV | same || **1936/tcp** | OpenRe `openre-rtmp-ingest` (all generations, SO_REUSEPORT) | permanent: URLs handed out as `rtmp://ingest.openre.stream:1936/live` keep working || 19360–19399/tcp (loopback) | OpenRe per-generation RTMP play + HTTP-FLV | permanent |++| **9736/tcp** | OpenRe `openre-jsmpeg` (all generations, SO_REUSEPORT): MPEG-TS POST + WS viewers | permanent: `http://ingest.openre.stream:9736/<key>/<w>/<h>/` and `ws://…/<session>` |++| 19710–19749/tcp (loopback) | OpenRe per-generation JSMPEG MPEG-TS data tap | permanent || 4500/tcp (loopback) | `openre-api` | permanent |The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_PORTS=1935` and deploy a worker generation (`deploy.sh workers`); from then on `rtmp://ingest.openre.stream/live` works too, and 1936 keeps working. `ingest.openre.stream` is a DNS-only record (RTMP cannot go through Cloudflare's proxy); open 1936/tcp at the host firewall and the provider edge.@@@ -183,7 +186,7 @@ sudo deploy/scripts/deploy.sh status # deploy-legacy.sh- ovhost refuses an API restart while an ingest session is open (`--wait-idle` holds it, `--force` goes ahead), records every attempt (`ovhost releases openre`), prunes releases beyond five but never one a worker generation runs from, and announces the release. When ovhost is missing, too old or does not deploy OpenRe with `release-layout`, the wrapper runs [deploy/scripts/deploy-legacy.sh](deploy/scripts/deploy-legacy.sh), the previous script, unchanged (`OVHOST_LEGACY=1` forces it). `workers`, `status` and `prune` always run it.- An **API deploy** never restarts a worker unit (no unit depends on another). Viewers of `openre.stream/play/…` reconnect; encoders, restreams and recordings do not notice.--- A **worker deploy** starts `openre-rtmp-ingest@<sha>` and `openre-restream-worker@<sha>`; older instances are disabled (not stopped) and exit on their own when drained. Never `systemctl restart` a worker instance during a broadcast; `systemctl stop` starts a drain and waits up to 30 min (`TimeoutStopSec`), then kills.++- A **worker deploy** starts `openre-rtmp-ingest@<sha>`, `openre-restream-worker@<sha>` and `openre-jsmpeg@<sha>`; older instances are disabled (not stopped) and exit on their own when drained. Never `systemctl restart` a worker instance during a broadcast; `systemctl stop` starts a drain and waits up to 30 min (`TimeoutStopSec`), then kills.- Worker instances are named after the release. `deploy.sh workers <sha>` for a release that already runs a generation does nothing. An env change that workers read, such as `OPENRE_RTMP_BIND`, takes effect with the next release's generation (docs/cutover.md A3).- **Restore drills** (`ovhost drill openre`, OpenVibe.Host) start only `openre-api`, with `OPENRE_DRILL=1`, on a restored copy of the database. In drill mode the API serves reads and refuses writes, `/play/` and sign-in with 503 `openre.drill_read_only`. The coordinator and every worker exit before they open the database. The event relay and Media calls are off (`test/drill-mode.test.js`). Never set `OPENRE_DRILL` in `/etc/openvibe/openre.env`; the preflight's `env` check fails if it is set.- Rollback: when `openre-api` is not ready within 60 s ovhost switches `current` back to the previous@@@ -237,9 +240,13 @@ the env, the bind, DNS, the port from outside, the database, Live's env, the Eveone slot. Status on 2026-09-23: OpenRe is deployed (`655b98a10aaa`) and DNS is done. The release with`6dc78a5`, the public bind, the provider port, Live's settings and the subscription are not.--### WHIP, JSMPEG, SFU (not ready)++### JSMPEG (worker ready)--Each needs its transport ported into its worker (`workers/webrtc-ingest.js`, `workers/jsmpeg.js`, `workers/sfu.js` have the interface and registration), a restream source for it, a playback descriptor (`webrtc`), recording (RTP to Media for WebRTC; JSMPEG has no Media ingest), and `OPENRE_PROTOCOLS` in Live's `server/openre/authority.js` extended. The same per-slot switch, window and rollback apply.++`workers/jsmpeg.js` is ported (key-in-path admission, WebSocket viewers by playback id, the MPEG-TS data tap, generations/drain), and `jsmpegArgs()` restreams it. docs/cutover.md has the JSMPEG slot procedure (J-A once, then J-B1–B6 per slot): set `OPENRE_JSMPEG_BIND=0.0.0.0`, open 9736 at the provider edge, add `jsmpeg` to Live's `OPENRE_PROTOCOLS`, then the per-slot window/switch/rollback. JSMPEG has **no recording** (parity with Live): a jsmpeg definition with recording on fails the request explicitly.++++### WebRTC (not ready)++++One `workers/webrtc.js` will own WHIP ingest, the mediasoup producers/consumers and viewer signaling (decision 1: mediasoup is single-process, so ingest and consumption cannot be split). It needs its ported transport, a restream source for it, a `webrtc` playback descriptor, recording (RTP to Media) and `webrtc` in Live's `OPENRE_PROTOCOLS`. The same per-slot switch, window and rollback apply.## Acceptance@@@ -12,9 +12,8 @@"protocols": {"rtmp": "ingest ported (openre-rtmp-ingest)","restream": "RTMP source to RTMP/RTMPS/SRT destinations ported (openre-restream-worker)",-- "whip": "not ported (worker interface only)",-- "webrtc-sfu": "not ported (worker interface only)",-- "jsmpeg": "not ported (worker interface only)"++ "webrtc": "not ported (workers/webrtc.js: one worker will own WHIP ingest, the SFU and viewer signaling; worker interface only)",++ "jsmpeg": "worker ported (workers/jsmpeg.js): MPEG-TS POST admission by key, WS viewers, loopback data tap, restream (re-encode), generations/drain; no recording (parity)"},"capabilities": ["openre.stream.read",@@@ -77,9 +77,9 @@ cmd_workers() {local sha=${1:-$(latest_release)}[ -n "$sha" ] && [ -d "$ROOT/releases/$sha" ] || die "no release $sha"say "starting worker generation from release $sha (older generations drain by themselves)"-- systemctl enable --now "openre-rtmp-ingest@$sha.service" "openre-restream-worker@$sha.service"++ systemctl enable --now "openre-rtmp-ingest@$sha.service" "openre-restream-worker@$sha.service" "openre-jsmpeg@$sha.service"# Disable (not stop) older instances so a reboot does not bring them back; they exit on their own.-- for unit in $(systemctl list-units --plain --no-legend 'openre-rtmp-ingest@*' 'openre-restream-worker@*' | awk '{print $1}'); do++ for unit in $(systemctl list-units --plain --no-legend 'openre-rtmp-ingest@*' 'openre-restream-worker@*' 'openre-jsmpeg@*' | awk '{print $1}'); docase "$unit" in *"@$sha.service") ;; *) systemctl disable "$unit" >/dev/null 2>&1 || true; say "draining: $unit";; esacdone}nnew file mode 100644@@@ -0,0 +1,37 @@++[Unit]++Description=OpenRe.Stream JSMPEG ingest+relay worker, release %i++# One instance per release (instance name = the release directory). A worker deploy STARTS a new++# instance; the coordinator then drains the older generation, which closes its public listener,++# keeps its sessions until they leave (or OPENRE_DRAIN_MAX_MS), and exits 0 by itself.++# Nothing here depends on openre-api or openre-session-coordinator: an API deploy never restarts++# this unit. Never `systemctl restart` it during a broadcast; see README "Deploying".++After=network.target postgresql.service pgbouncer.service valkey-server.service++Wants=postgresql.service pgbouncer.service valkey-server.service++++[Service]++Type=simple++User=ubuntu++Group=ubuntu++WorkingDirectory=/opt/openre.stream/releases/%i++EnvironmentFile=/etc/openvibe/openre.env++Environment=NODE_ENV=production++Environment=OPENRE_RELEASE=%i++Environment=PATH=/usr/local/bin:/usr/bin:/bin++ExecStart=/usr/bin/env node workers/jsmpeg.js++# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.++Restart=on-failure++RestartSec=2++# SIGTERM starts a drain; it does not drop sessions. Only the main process gets it.++KillSignal=SIGTERM++KillMode=mixed++# An explicit stop (or host shutdown) waits this long for sessions to leave, then kills.++TimeoutStopSec=30min++LimitNOFILE=65536++NoNewPrivileges=true++PrivateTmp=true++ProtectSystem=strict++ProtectHome=true++ReadWritePaths=/var/lib/openre++++[Install]++WantedBy=multi-user.target@@@ -333,6 +333,46 @@ first, then B3 to B6.and restart Live. Every switched slot then behaves as a Live slot. Its Live key was rotated at theswitch, so each broadcaster regenerates on the Go Live page.++## JSMPEG protocol cutover (after RTMP)++++JSMPEG reuses the RTMP runbook above, with one **protocol precondition** (brief §4.5): the slot's++`managed_streams.protocol` must be `jsmpeg` and its OpenRe definition carries the `jsmpeg` protocol.++Live keeps its own `jsmpeg-relay.js` (9710/9711) until the last JSMPEG slot is cut over.++++### J-A. Once, before the first JSMPEG slot (AGENT)++++```bash++# The release that carries workers/jsmpeg.js, deploy/scripts/deploy.sh, systemd/[email protected].++sudo deploy/scripts/deploy.sh release origin/main++sudo deploy/scripts/deploy.sh api <sha>++# Public JSMPEG bind + a new generation. deploy.sh workers now starts openre-jsmpeg@<sha>.service too.++# OPENRE_JSMPEG_BIND=0.0.0.0 and OPENRE_JSMPEG_PORT=9736 in /etc/openvibe/openre.env++sudo deploy/scripts/deploy.sh workers <sha>++```++++The **OWNER** opens `OPENRE_JSMPEG_PORT` (9736 until the cutover) at the provider edge; the host++`ingest.openre.stream` is the DNS-only record RTMP already uses (raw MPEG-TS cannot go through++Cloudflare's proxy). `openre.stream`/nginx gain the JSMPEG route only when Live's relay is retired.++++### J-B1–B6. Per JSMPEG slot++++Identical to B1–B6 with these substitutions:++++- **B2 Migrate:** `migrate-from-live.js` imports a `jsmpeg` slot as a `jsmpeg` definition (it is no++ longer refused; the checklist flags it). `pf --only slot --slot <id>` must show the protocol++ `jsmpeg`.++- **B3 Switch:** `PUT /api/admin/openre/managed/<id>/ingest-authority {"authority":"openre"}` needs++ the slot offline, the `jsmpeg` protocol in Live's `OPENRE_PROTOCOLS`, and the JSMPEG worker live.++- **B4 New key:** the broadcaster regenerates the key, then POSTs MPEG-TS to++ `http://ingest.openre.stream:9736/<key>/<width>/<height>/` (its ffmpeg command, from the profile++ route). Viewers connect to the `jsmpeg.ws_url` in the session playback descriptor++ (`GET /api/v1/sessions/<id>/playback`); the viewer canvas shows the stream.++- **B5/B6:** unchanged (personal-key rotation; a test broadcast checks the viewer and a restream++ destination — JSMPEG restreams re-encode the worker's MPEG-TS tap).++- **B-rollback:** `{"authority":"live"}` + `set-definition-state.js --state disabled`, then the++ broadcaster points ffmpeg back at Live's relay. JSMPEG has **no recording** (parity with Live): a++ jsmpeg definition with recording on fails the request with `recording is not available for jsmpeg`.++## Phase C: bookkeeping (AGENT, after A2)- **ovhost inventory.** Copy the `openre` entry of OpenVibe.Host's `host.example.json` into@@@ -30,11 +30,12 @@"openre.output.healthy","openre.output.failed","openre.recording.requested",++ "openre.recording.failed","openre.key.rotated"],"eventsConsumed": [],"namespacesOwned": ["openre.*"],-- "notes": "Ingest and restream authority (ADR-009). Alpha: RTMP ingest and RTMP/SRT restream run in transport workers with generations and leases, separate from the API; WHIP, WebRTC/SFU and JSMPEG are still ingested by Live. Runs in parallel with Live; slots move one at a time behind Live's per-slot ingest_authority switch."++ "notes": "Ingest and restream authority (ADR-009). Alpha: RTMP ingest, JSMPEG ingest+relay, and RTMP/SRT/JSMPEG-source restream run in transport workers with generations and leases, separate from the API; WebRTC (WHIP + SFU) is still ingested by Live. Runs in parallel with Live; slots move one at a time behind Live's per-slot ingest_authority switch."}nnew file mode 100644@@@ -0,0 +1,43 @@++-- phase: migrate++-- One protocol name per transport (T4 decision 2): 'whip' is an input alias, stored as 'webrtc', and++-- the 'webrtc-ingest'/'sfu' worker kinds are one 'webrtc' worker. Rewrite the rows written before the++-- rename. Never edited after it runs; a further rename is a new migration.++--++-- Protocols: map 'whip' to 'webrtc' and drop duplicates, keeping the first occurrence's position.++-- (Stream definitions are the only place protocols are stored; workers.store validates on write.)++UPDATE stream_definitions++SET protocols = (++ SELECT jsonb_agg(x.p ORDER BY x.ord)::text++ FROM (++ SELECT DISTINCT ON (mapped) mapped AS p, ord++ FROM (++ SELECT (CASE WHEN v = 'whip' THEN 'webrtc' ELSE v END) AS mapped, ord++ FROM jsonb_array_elements_text(protocols::jsonb) WITH ORDINALITY AS e(v, ord)++ ) m++ ORDER BY mapped, ord++ ) x++)++WHERE protocols LIKE '%"whip"%';++++-- Workers: both legacy kinds become 'webrtc'. Their generations were unique only within each old++-- kind, so they are renumbered into one clean run for UNIQUE (kind, generation). No 'webrtc' row can++-- exist before this migration (the kind is introduced with it), so the merge cannot collide.++WITH legacy AS (++ SELECT id, -(row_number() OVER (ORDER BY generation, started_at, id)) AS tmp++ FROM workers++ WHERE kind IN ('webrtc-ingest', 'sfu')++)++UPDATE workers w++SET kind = 'webrtc', generation = legacy.tmp++FROM legacy++WHERE w.id = legacy.id;++++WITH merged AS (++ SELECT id, row_number() OVER (ORDER BY generation, started_at, id) AS rn++ FROM workers++ WHERE kind = 'webrtc'++)++UPDATE workers w++SET generation = merged.rn++FROM merged++WHERE w.id = merged.id;@@@ -83,6 +83,7 @@ function createV1Router({ rt, auth }) {end_reason: s.end_reason,failure_reason: s.failure_reason,media_info: s.media_info,++ viewers: s.viewers || 0,revision: s.revision,created_at: iso(s.created_at),live_at: iso(s.live_at),@@@ -22,6 +22,7 @@ function load(env = process.env) {const isProduction = nodeEnv === 'production';const port = int(env.PORT, 4500);const rtmpPort = int(env.OPENRE_RTMP_PORT, 1936);++ const jsmpegPort = int(env.OPENRE_JSMPEG_PORT, 9736);return {service: 'openre',port,@@@ -92,6 +93,22 @@ function load(env = process.env) {maxPublishersPerWorker: int(env.OPENRE_RTMP_MAX_PUBLISHERS, 64),},++ // ── Ingest (JSMPEG) ─────────────────────────────────────++ jsmpeg: {++ // Public port for both the MPEG-TS HTTP POST from the broadcaster's ffmpeg and the++ // WebSocket viewers. Live's relay owns 9710/9711 until the JSMPEG cutover; OpenRe++ // listens on 9736 until then, SO_REUSEPORT so two generations can listen during a drain.++ port: jsmpegPort,++ bindHost: env.OPENRE_JSMPEG_BIND || '0.0.0.0',++ publicHost: env.OPENRE_JSMPEG_PUBLIC_HOST || (isProduction ? 'ingest.openre.stream' : '127.0.0.1'),++ publicPort: int(env.OPENRE_JSMPEG_PUBLIC_PORT, jsmpegPort),++ // Loopback data tap a generation serves the raw MPEG-TS from (the restream worker reads++ // it). First free port from this range; never Live's 9710/9711.++ internalPortMin: int(env.OPENRE_JSMPEG_INTERNAL_PORT_MIN, 19710),++ internalPortMax: int(env.OPENRE_JSMPEG_INTERNAL_PORT_MAX, 19749),++ maxPublishersPerWorker: int(env.OPENRE_JSMPEG_MAX_PUBLISHERS, 64),++ },++// ── Workers, leases, generations ────────────────────────workers: {heartbeatMs: int(env.OPENRE_WORKER_HEARTBEAT_MS, 2000),@@@ -14,6 +14,7 @@* openre.output.healthy a restream output confirmed live subject output* openre.output.failed an output gave up (circuit breaker) subject output* openre.recording.requested Media accepted a recording request subject recording++ * openre.recording.failed a recording could not be started subject recording* openre.key.rotated an ingest key was rotated subject stream* Payloads never carry an ingest key or a destination key.*/@@@ -28,6 +29,7 @@ const TYPES = Object.freeze({outputHealthy: 'openre.output.healthy',outputFailed: 'openre.output.failed',recordingRequested: 'openre.recording.requested',++ recordingFailed: 'openre.recording.failed',keyRotated: 'openre.key.rotated',});@@@ -22,6 +22,18 @@ async function columns(db, table) {try { return new Set((await db.prepare(`PRAGMA table_info(${table})`).all()).map(c => c.name)); } catch { return new Set(); }}++/**++ * The definition protocols for a Live slot (T4 decision 2): 'whip' is an input alias for 'webrtc',++ * 'webrtc' and 'jsmpeg' slots import as themselves (they are unadmittable until their worker ships++ * — the per-slot switch is the real gate), anything else is treated as RTMP.++ */++function protocolOf(ms) {++ const p = String(ms.protocol || '').toLowerCase();++ if (p === 'whip' || p === 'webrtc') return ['webrtc'];++ if (p === 'jsmpeg') return ['jsmpeg'];++ return ['rtmp'];++}++function recordingModeOf(ms, channel) {let vod = !channel ? true : Boolean(channel.vod_recording_enabled) && !channel.force_vod_recording_disabled;if (vod && ms.slot_vod_recording_enabled === 0) vod = false;@@@ -100,7 +112,7 @@ async function migrate({ liveDb, rt, apply = false, onlySlots = null, now = () =owner_subject: subject,title: ms.title || `${ms.display_name || ms.username}'s stream`,description: ms.description || '',-- protocols: ['rtmp'],++ protocols: protocolOf(ms),recording_mode: recordingModeOf(ms, chRow),recording_visibility: visibilityOf(ms, chRow),mirror_to_live: true,@@@ -206,8 +218,10 @@ function checklist(report, { openreUrl = 'https://openre.stream', rtmpUrl } = {}lines.push(`### Slot ${s.live_id}${s.slug ? ` "${s.slug}"` : ''}: ${s.title || ''} — ${s.status}${s.definition_id ? ` → ${s.definition_id}` : ''}`);if (s.reason && s.status !== 'imported') lines.push(`- ${s.status}: ${s.reason}`);if (s.status === 'imported') {-- if (s.protocol !== 'rtmp' && !['rtmp', 'obs'].includes(String(s.streaming_method || '').toLowerCase())) {-- lines.push(`- Slot protocol is ${s.protocol}${s.streaming_method ? ` / ${s.streaming_method}` : ''}: only RTMP moves to OpenRe; leave this slot on Live unless the broadcaster streams it with OBS/RTMP.`);++ if (s.protocol === 'webrtc' || s.protocol === 'whip') {++ lines.push(`- Slot protocol is ${s.protocol}${s.streaming_method ? ` / ${s.streaming_method}` : ''}: the webrtc worker is not ported yet; leave this slot on Live until it ships.`);++ } else if (s.protocol === 'jsmpeg') {++ lines.push(`- Slot protocol is jsmpeg${s.streaming_method ? ` / ${s.streaming_method}` : ''}: switch it after the JSMPEG worker generation is running (docs/cutover.md, JSMPEG).`);}lines.push(`- [ ] Maintenance window agreed with @${ch.username} (the slot must be offline when it is switched).`);lines.push(`- [ ] Destinations reviewed on ${openreUrl}/streams/${s.definition_id || '<id>'}: ${s.destinations.length ? s.destinations.map(d => `${d.name} (${d.platform}) ${d.status}${d.reason ? `: ${d.reason}` : ''}`).join('; ') : 'none'}`);@@@ -11,7 +11,10 @@ const { newId, isId } = require('../ids');const { newIngestKey, hashIngestKey, isIngestKeyShape, hintOf } = require('../secrets');const { TYPES } = require('../events');--const PROTOCOLS = Object.freeze(['rtmp', 'whip', 'webrtc', 'jsmpeg']);++// The definition protocols (T4 decision 2): one name per transport. 'whip' is accepted on input++// only (API create/update, migrate-from-live) and stored as 'webrtc'.++const PROTOCOLS = Object.freeze(['rtmp', 'webrtc', 'jsmpeg']);++const PROTOCOL_ALIASES = Object.freeze({ whip: 'webrtc' });const RECORDING_MODES = Object.freeze(['vod', 'clips', 'none']);const VISIBILITIES = Object.freeze(['public', 'unlisted', 'private']);const REF_SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;@@@ -106,7 +109,8 @@ function createDefinitions({ db, config, events, clock }) {if (input.title !== undefined || base.title === undefined) out.title = cleanText(input.title, 140, '') || base.title || 'Untitled stream';if (input.description !== undefined) out.description = cleanText(input.description, 2000, '');if (input.protocols !== undefined) {-- const p = Array.isArray(input.protocols) ? [...new Set(input.protocols.map(String))] : null;++ const named = Array.isArray(input.protocols) ? input.protocols.map(String).map(x => PROTOCOL_ALIASES[x] || x) : null;++ const p = named ? [...new Set(named)] : null;if (!p || !p.length || p.some(x => !PROTOCOLS.includes(x))) throw new StoreError(400, 'openre.invalid_protocols', `protocols must be a non-empty subset of ${PROTOCOLS.join(', ')}`);out.protocols = JSON.stringify(p);}@@@ -259,6 +263,15 @@ function createDefinitions({ db, config, events, clock }) {key_id: active ? active.id : null,};}++ if (definition.protocols.includes('jsmpeg')) {++ const j = config.jsmpeg;++ out.jsmpeg = {++ // The broadcaster's ffmpeg POSTs MPEG-TS to <url>/<key>/<width>/<height>/.++ url: `http://${j.publicHost}:${j.publicPort}`,++ key_hint: active ? active.hint : null,++ key_id: active ? active.id : null,++ };++ }return out;}@@@ -14,6 +14,9 @@ const { newId } = require('../ids');const { TYPES } = require('../events');const { isTerminal } = require('../state-machine');++// JSMPEG has no recording (Live has none either, decision 8: parity). A jsmpeg definition with++// recording on gets an explicit failure, never a silent no-op.++const RECORDING_UNSUPPORTED = 'recording is not available for jsmpeg';const BACKOFF_MS = [5000, 15000, 30000, 60000, 120000, 300000];const MAX_ATTEMPTS = 10;// Media refuses a recording while its disk is critically low and frees space within minutes;@@@ -68,6 +71,29 @@ function createRecordings({ db, config, events, clock, definitions, sessions, loconst definition = await definitions.row(session.definition_id);const ended = isTerminal(session.state) || session.state === 'ending';++ if (session.protocol === 'jsmpeg') {++ return await db.tx(async () => {++ const r = await update(rec.id, { state: 'failed', last_error: RECORDING_UNSUPPORTED });++ await events.enqueue({++ event_type: TYPES.recordingFailed,++ actor: { type: 'service', id: 'openre' },++ subject: { type: 'recording', id: rec.id, revision: 1 },++ visibility: 'internal',++ priority: 'important',++ payload: {++ recording_id: rec.id,++ session_id: session.id,++ stream_id: definition.id,++ owner: { type: 'user', id: definition.owner_subject },++ protocol: session.protocol,++ reason: RECORDING_UNSUPPORTED,++ external_refs: definition.external_refs,++ },++ });++ return r;++ });++ }++if (rec.state === 'pending') {if (ended) return await update(rec.id, { state: 'cancelled' });const pb = await sessions.playback(session);@@@ -28,6 +28,9 @@ function createSessions({ db, config, events, clock, definitions, workers }) {transitionRow: db.prepare('INSERT INTO session_transitions (session_id, from_state, to_state, reason, actor, at) VALUES (?, ?, ?, ?, ?, ?)'),transitions: db.prepare('SELECT from_state, to_state, reason, actor, at FROM session_transitions WHERE session_id = ? ORDER BY id'),mediaInfo: db.prepare(`UPDATE ingest_sessions SET media_info = ? WHERE id = ? AND state IN ${OPEN}`),++ // Viewer counts live inside media_info so no column is needed: merge, never replace, so the++ // width/height the JSMPEG ingest path reported survive.++ viewers: db.prepare(`UPDATE ingest_sessions SET media_info = jsonb_set(COALESCE(NULLIF(media_info, '')::jsonb, '{}'::jsonb), '{viewers}', to_jsonb(?::bigint), true)::text WHERE id = ? AND state IN ${OPEN}`),requestEnd: db.prepare(`UPDATE ingest_sessions SET desired_state = 'end', end_requested_by = ?, updated_at = ? WHERE id = ? AND state IN ${OPEN}`),endRequests: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND desired_state = 'end' AND state IN ${OPEN}`),ofWorker: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND state IN ${OPEN}`),@@@ -46,7 +49,8 @@ function createSessions({ db, config, events, clock, definitions, workers }) {function shape(s) {if (!s) return null;-- return { ...s, media_info: parseJson(s.media_info, null) };++ const media_info = parseJson(s.media_info, null);++ return { ...s, media_info, viewers: (media_info && Number(media_info.viewers)) || 0 };}async function get(id) {@@@ -200,8 +204,20 @@ function createSessions({ db, config, events, clock, definitions, workers }) {flv: null,rtmp: null,webrtc: null,++ jsmpeg: null,hls: null,};++ if (session.protocol === 'jsmpeg' && ep.publicPort) {++ const mi = session.media_info || {};++ descriptor.jsmpeg = {++ // Viewers connect over WebSocket to the worker that holds the session, by playback++ // id (never the key). tap_internal_url is the loopback MPEG-TS feed restream reads.++ ws_url: `ws://${config.jsmpeg.publicHost}:${ep.publicPort}/${session.id}`,++ width: mi.width || null,++ height: mi.height || null,++ tap_internal_url: ep.tapPort ? `http://127.0.0.1:${ep.tapPort}/tap/${session.id}.ts` : null,++ };++ }if (session.protocol === 'rtmp' && ep.flvPort) {descriptor.flv = {internal_url: `http://127.0.0.1:${ep.flvPort}/live/${session.id}.flv`,@@@ -225,6 +241,7 @@ function createSessions({ db, config, events, clock, definitions, workers }) {requestEnd,playback,setMediaInfo: async (id, info) => (await q.mediaInfo.run(JSON.stringify(info), id)).changes > 0,++ setViewers: async (id, count) => (await q.viewers.run(Math.max(0, Number(count) || 0), id)).changes > 0,transitions: async (id) => await q.transitions.all(id),endRequestsFor: async (workerId) => (await q.endRequests.all(workerId)).map(shape),ofWorker: async (workerId) => (await q.ofWorker.all(workerId)).map(shape),@@@ -18,7 +18,9 @@ const os = require('os');const { newId } = require('../ids');const { parseJson } = require('./definitions');--const KINDS = Object.freeze(['rtmp-ingest', 'restream', 'webrtc-ingest', 'sfu', 'jsmpeg']);++// T4 decision 2: one kind per transport. The 'webrtc-ingest' and 'sfu' kinds collapsed into the one++// 'webrtc' worker (migration 0002 rewrites any stored rows).++const KINDS = Object.freeze(['rtmp-ingest', 'restream', 'webrtc', 'jsmpeg']);const ALIVE = "('starting', 'ready', 'draining')";function createWorkers({ db, config, clock }) {@@@ -12,6 +12,7 @@ const { createCoordinator } = require('../server/coordinator');const { createRtmpIngest } = require('../workers/rtmp-ingest');const { createRestreamWorker } = require('../workers/restream-worker');const { createUnportedTransport } = require('../workers/unported-transport');++const { createJsmpeg } = require('../workers/jsmpeg');const { openDb } = require('../server/db');const { runtime, bootApi, request, serviceToken, tmpDir, testEnv, child, suite, silent, testDb } = require('./helpers');@@@ -45,13 +46,14 @@ t('the coordinator and every transport worker refuse to be created in a drill',drill(() => createCoordinator({ rt, media: null, log: silent }));drill(() => createRtmpIngest({ rt, log: silent, exit: () => {} }));drill(() => createRestreamWorker({ rt, log: silent, exit: () => {} }));-- for (const kind of ['webrtc-ingest', 'sfu', 'jsmpeg']) drill(() => createUnportedTransport({ rt, kind, log: silent, exit: () => {} }));++ drill(() => createUnportedTransport({ rt, kind: 'webrtc', log: silent, exit: () => {} }));++ drill(() => createJsmpeg({ rt, log: silent, exit: () => {} }));assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM workers').get()).n, 0, 'no generation registered');assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM leases').get()).n, 0, 'no coordinator lease taken');});t('the entry points exit before opening the database', async () => {-- for (const script of ['workers/coordinator.js', 'workers/rtmp-ingest.js', 'workers/restream-worker.js', 'workers/sfu.js']) {++ for (const script of ['workers/coordinator.js', 'workers/rtmp-ingest.js', 'workers/restream-worker.js', 'workers/webrtc.js', 'workers/jsmpeg.js']) {const dir = tmpDir();const p = child(script, await testEnv(dir, { ...PROD_LIKE, OPENRE_DRILL: '1' }));const r = await p.exited;nnew file mode 100644@@@ -0,0 +1,268 @@++'use strict';++// End to end with real processes and real ffmpeg: openre-api, openre-jsmpeg, openre-restream-worker++// and openre-session-coordinator as separate child processes, plus a stub OpenVibe.Network, a stub++// OpenVibe.Events and a stub OpenVibe.Media.++//++// Proves: the broadcaster's ffmpeg POSTs MPEG-TS to /<key>/<w>/<h>/ and the session goes live on the++// newest generation (wrong key and wrong protocol refused), a WebSocket viewer receives the bytes and++// is counted in the session heartbeat, the loopback data tap restreams to a working destination,++// recording is refused with a clear reason, a duplicate publisher is refused, and a second generation++// takes new sessions while the first drains, keeps its session and exits when it ends; a killed worker++// fails its session after the lease expires.++const assert = require('assert');++const http = require('http');++const { spawn } = require('child_process');++const WebSocket = require('ws');++const {++ tmpDir, testEnv, multiProcess, child, waitFor, request, userToken, freePort, sleep, suite, hasFfmpeg, rtmpSink, OWNER,++} = require('./helpers');++++if (!hasFfmpeg()) {++ console.log('jsmpeg-e2e: skipped (no ffmpeg on PATH)');++ process.exit(0);++}++if (!multiProcess()) {++ console.log('jsmpeg-e2e: skipped (the OpenRe processes share one database: PostgreSQL only, npm run test:pg)');++ process.exit(0);++}++const t = suite('jsmpeg-e2e');++const token = userToken({ subjectId: OWNER });++const procs = [];++const mediaCalls = [];++let env;++let api;++let coordinator;++let base;++let jsmpegPort;++let ingest1;++let stubs = [];++++function stubServer(handler) {++ return new Promise((resolve) => {++ const s = http.createServer((req, res) => {++ let body = '';++ req.on('data', (c) => { body += c; });++ req.on('end', () => handler(req, res, body));++ });++ s.listen(0, '127.0.0.1', () => { stubs.push(s); resolve(`http://127.0.0.1:${s.address().port}`); });++ });++}++const json = (res, status, obj) => { res.statusCode = status; res.setHeader('Content-Type', 'application/json'); res.end(JSON.stringify(obj)); };++++function startChild(script, extra = {}) {++ const p = child(script, { ...env, ...extra });++ procs.push(p);++ return p;++}++async function startApi() {++ const p = startChild('server/index.js');++ await waitFor(async () => { try { return (await request(base, 'GET', '/api/ready')).status === 200; } catch { return false; } }, { what: 'api ready' });++ return p;++}++++/** ffmpeg POSTs MPEG-TS to http://127.0.0.1:<port>/<key>/<w>/<h>/ — Live's JSMPEG broadcaster command. */++function publishJsmpeg(port, key, w, h, { seconds } = {}) {++ const url = `http://127.0.0.1:${port}/${key}/${w}/${h}/`;++ const args = ['-hide_banner', '-loglevel', 'error', '-re', '-f', 'lavfi', '-i', 'testsrc=size=320x240:rate=30',++ '-c:v', 'mpeg1video', '-b:v', '600k', '-r', '30', '-f', 'mpegts'];++ if (seconds) args.push('-t', String(seconds));++ args.push(url);++ const p = spawn('ffmpeg', args, { stdio: ['ignore', 'ignore', 'pipe'] });++ p.err = '';++ p.stderr.on('data', (c) => { p.err += c; });++ p.exited = new Promise(r => p.on('exit', (code, signal) => r({ code, signal })));++ return p;++}++++async function openSession(streamId, state = 'live') {++ const r = await request(base, 'GET', `/api/v1/sessions?stream_id=${streamId}`, { token });++ return (r.body.sessions || []).find(s => s.state === state) || null;++}++async function detail(id) {++ const r = await request(base, 'GET', `/api/v1/sessions/${id}`, { token });++ return r.body.session;++}++++let streamA;++let keyA;++let sessionA;++let pubA;++let sink;++let viewer;++++t('setup: stubs and the four OpenRe processes', async () => {++ const network = await stubServer((req, res, body) => {++ if (req.url === '/oauth/token') {++ const p = new URLSearchParams(body);++ return json(res, 200, { access_token: `tok-${p.get('audience')}`, token_type: 'Bearer', expires_in: 300 });++ }++ return json(res, 404, {});++ });++ const events = await stubServer((req, res, body) => {++ if (req.url === '/api/v1/events' && req.method === 'POST') return json(res, 201, { event_id: 'e', seq: 1, duplicate: false });++ return json(res, 404, {});++ });++ const media = await stubServer((req, res, body) => {++ mediaCalls.push({ method: req.method, url: req.url, body: body ? JSON.parse(body) : null });++ return json(res, 404, { error: 'Media must not be called for a jsmpeg recording' });++ });++ jsmpegPort = await freePort();++ const dir = tmpDir();++ env = await testEnv(dir, {++ PORT: String(await freePort()),++ OPENRE_JSMPEG_PORT: String(jsmpegPort),++ OPENRE_JSMPEG_BIND: '127.0.0.1',++ OPENRE_JSMPEG_INTERNAL_PORT_MIN: String(22000 + Math.floor(Math.random() * 20000)),++ OV_NETWORK_INTERNAL_URL: network,++ OV_OAUTH_CLIENT_SECRET: 'test-secret',++ EVENTS_URL: events,++ EVENTS_RELAY_INTERVAL_MS: '200',++ MEDIA_URL: media,++ MEDIA_API_KEY: 'media-app-key',++ OPENRE_RECORDING_START_DELAY_MS: '200',++ OPENRE_WORKER_HEARTBEAT_MS: '250',++ OPENRE_WORKER_LEASE_MS: '4000',++ OPENRE_COORDINATOR_INTERVAL_MS: '250',++ OPENRE_RESTREAM_POLL_MS: '250',++ OPENRE_OUTPUT_START_DELAY_MS: '500',++ OPENRE_RESTART_BASE_MS: '200',++ OPENRE_MAX_RESTARTS: '3',++ });++ env.OPENRE_JSMPEG_INTERNAL_PORT_MAX = String(Number(env.OPENRE_JSMPEG_INTERNAL_PORT_MIN) + 20);++ base = `http://127.0.0.1:${env.PORT}`;++ coordinator = startChild('workers/coordinator.js');++ ingest1 = startChild('workers/jsmpeg.js');++ startChild('workers/restream-worker.js');++ api = await startApi();++ await waitFor(() => /generation 1 ready: ingest\+ws/.test(ingest1.output), { what: 'jsmpeg worker ready' });++});++++t('a bad key and a wrong-protocol key are refused', async () => {++ const bad = publishJsmpeg(jsmpegPort, `ork_${'x'.repeat(43)}`, 320, 240, { seconds: 2 });++ procs.push(bad);++ assert.notStrictEqual((await bad.exited).code, 0, 'ffmpeg with a wrong key fails');++++ // A definition that does not list jsmpeg: resolveIngestKey answers protocol_not_allowed.++ const r = await request(base, 'POST', '/api/v1/streams', { token, body: { title: 'RTMP only', protocols: ['rtmp'] } });++ assert.strictEqual(r.status, 201);++ const wrong = publishJsmpeg(jsmpegPort, r.body.key.key, 320, 240, { seconds: 2 });++ procs.push(wrong);++ assert.notStrictEqual((await wrong.exited).code, 0, 'an rtmp-only definition is refused on the jsmpeg listener');++});++++t('the ffmpeg MPEG-TS POST goes live, a WS viewer receives bytes and is counted', async () => {++ const r = await request(base, 'POST', '/api/v1/streams', {++ token,++ body: { title: 'JSMPEG', protocols: ['jsmpeg'], recording_mode: 'vod', external_refs: [{ service: 'live', type: 'managed_stream', id: '9' }, { service: 'live', type: 'user', id: '42' }] },++ });++ assert.strictEqual(r.status, 201);++ streamA = r.body.stream.id;++ keyA = r.body.key.key;++ const sinkPort = await freePort();++ sink = rtmpSink(sinkPort);++ procs.push(sink);++ assert.strictEqual((await request(base, 'POST', `/api/v1/streams/${streamA}/destinations`, { token, body: { platform: 'custom', name: 'sink', server_url: `rtmp://127.0.0.1:${sinkPort}/app`, stream_key: 'sinkkey' } })).status, 201);++++ pubA = publishJsmpeg(jsmpegPort, keyA, 320, 240);++ procs.push(pubA);++ sessionA = await waitFor(async () => await openSession(streamA), { what: 'session A live', timeoutMs: 20000 });++ assert.strictEqual(sessionA.worker.generation, 1);++++ const d = await detail(sessionA.id);++ assert.deepStrictEqual({ w: d.playback.jsmpeg.width, h: d.playback.jsmpeg.height }, { w: 320, h: 240 });++ assert.ok(!d.playback.jsmpeg.ws_url.includes(keyA), 'the key is never in the viewer URL');++++ viewer = new WebSocket(d.playback.jsmpeg.ws_url);++ viewer.frames = [];++ viewer.on('message', (m) => viewer.frames.push(m));++ await new Promise((res, rej) => { viewer.once('open', res); viewer.once('error', rej); });++ await waitFor(() => viewer.frames.length > 0, { what: 'viewer bytes', timeoutMs: 10000 });++ assert.ok(viewer.frames[0].length > 0);++++ const v = await waitFor(async () => { const s = await detail(sessionA.id); return s.viewers >= 1 ? s : null; }, { what: 'viewer counted', timeoutMs: 10000 });++ assert.ok(v.viewers >= 1);++});++++t('the JSMPEG data tap restreams to a destination', async () => {++ const outputs = await waitFor(async () => {++ const r = await request(base, 'GET', `/api/v1/sessions/${sessionA.id}/outputs`, { token });++ const o = r.body.outputs || [];++ return o.some(x => x.state === 'live') ? o : null;++ }, { what: 'restream live', timeoutMs: 45000 });++ assert.ok(outputs.some(o => o.state === 'live'));++});++++t('recording is refused with a clear reason, without calling Media', async () => {++ const s = await waitFor(async () => { const d = await detail(sessionA.id); return d.recording && d.recording.state === 'failed' ? d : null; }, { what: 'recording failed', timeoutMs: 10000 });++ assert.strictEqual(s.recording.last_error, 'recording is not available for jsmpeg');++ assert.strictEqual(mediaCalls.length, 0, 'Media was never asked to record a jsmpeg session');++});++++t('a second publisher with the same key is refused while the first is live', async () => {++ const dup = publishJsmpeg(jsmpegPort, keyA, 320, 240, { seconds: 2 });++ procs.push(dup);++ assert.notStrictEqual((await dup.exited).code, 0);++ assert.strictEqual((await openSession(streamA)).id, sessionA.id, 'the source session is unaffected');++});++++let streamB;++let pubB;++let ingest2;++++t('a new jsmpeg generation takes new sessions; generation 1 drains but keeps A', async () => {++ ingest2 = startChild('workers/jsmpeg.js');++ await waitFor(() => /generation 2 ready: ingest\+ws/.test(ingest2.output), { what: 'generation 2 ready' });++ await waitFor(() => /draining/.test(ingest1.output) && /public listener closed/.test(ingest1.output), { what: 'generation 1 draining', timeoutMs: 10000 });++++ const r = await request(base, 'POST', '/api/v1/streams', { token, body: { title: 'B', protocols: ['jsmpeg'], recording_mode: 'none' } });++ streamB = r.body.stream.id;++ pubB = publishJsmpeg(jsmpegPort, r.body.key.key, 320, 240);++ procs.push(pubB);++ const sB = await waitFor(async () => await openSession(streamB), { what: 'session B live', timeoutMs: 20000 });++ assert.strictEqual(sB.worker.generation, 2, 'new sessions reach the newest generation');++ const sA = await openSession(streamA);++ assert.strictEqual(sA.id, sessionA.id);++ assert.strictEqual(sA.worker.generation, 1, 'the running session stays on its generation');++ assert.strictEqual(ingest1.exitCode, null);++});++++t('when session A ends, generation 1 exits by itself; session B is unaffected', async () => {++ pubA.kill('SIGKILL');++ const end = await waitFor(async () => (await detail(sessionA.id)).state === 'ended', { what: 'session A ended', timeoutMs: 30000 }).then(() => true, () => false);++ assert.ok(end, `session A did not end: ${JSON.stringify(await detail(sessionA.id))}`);++ const exit = await Promise.race([ingest1.exited, sleep(10000).then(() => ({ code: 'timeout' }))]);++ assert.strictEqual(exit.code, 0, 'the drained generation exited cleanly');++ const idB = (await openSession(streamB)).id;++ assert.strictEqual((await detail(idB)).state, 'live');++});++++t('a killed worker fails its session after the lease expires', async () => {++ ingest2.kill('SIGKILL');++ await ingest2.exited;++ const idB = (await openSession(streamB)).id;++ const s = await waitFor(async () => {++ const d = await detail(idB);++ return d.state === 'failed' ? d : null;++ }, { what: 'session B failed after lease loss', timeoutMs: 20000 });++ assert.match(s.failure_reason, /lease|lost|expired/i);++});++++t('no OpenRe process crashed along the way, and no key reached a log', () => {++ assert.strictEqual(coordinator.exitCode, null, `coordinator exited:\n${coordinator.output}`);++ assert.strictEqual(api.exitCode, null, `api exited:\n${api.output}`);++ for (const p of procs) {++ if (!p.output) continue;++ assert.ok(!p.output.includes(keyA), 'no ingest key in logs');++ assert.ok(!p.output.includes('sinkkey'), 'no destination key in logs');++ }++});++++t('teardown', async () => {++ if (process.exitCode || process.env.DEBUG) for (const p of procs) if (p.output) console.log(p.output);++ if (viewer) { try { viewer.close(); } catch { /* */ } }++ for (const p of procs) { try { p.kill('SIGKILL'); } catch { /* */ } }++ for (const s of stubs) { s.closeAllConnections?.(); s.close(); }++ stubs = [];++});++++t.run();nnew file mode 100644@@@ -0,0 +1,44 @@++'use strict';++// The 0002 naming migration, run for real: a fresh database is built from migrations/0001_initial.sql,++// legacy rows ('whip' protocols, 'webrtc-ingest'/'sfu' workers) are inserted, the 0002 SQL is applied,++// and the stored rows are asserted rewritten (protocols deduplicated, worker kinds merged and++// renumbered so UNIQUE (kind, generation) holds).++const assert = require('assert');++const fs = require('fs');++const path = require('path');++const { createDb } = require('openvibe-sdk/db');++const { suite, ROOT } = require('./helpers');++++const MIG1 = path.join(ROOT, 'migrations', '0001_initial.sql');++const MIG2 = path.join(ROOT, 'migrations', '0002_protocol_names.sql');++++const t = suite('jsmpeg-migration');++const log = { log() {}, warn() {}, error: console.error };++++t('stored whip protocols become webrtc (deduplicated); webrtc-ingest/sfu worker rows become webrtc', async () => {++ const db = createDb({ pglite: true, service: 'jsmpeg-migration', log });++ try {++ await db.exec(fs.readFileSync(MIG1, 'utf8'));++ await db.exec(`INSERT INTO stream_definitions (id, owner_subject, protocols, created_at, updated_at) VALUES++ ('stream_a', 'usr_01J0000000000000000000000A', '["whip","webrtc","rtmp"]', 1, 1),++ ('stream_b', 'usr_01J0000000000000000000000A', '["whip","whip"]', 1, 1),++ ('stream_c', 'usr_01J0000000000000000000000A', '["rtmp","jsmpeg"]', 1, 1)`);++ await db.exec(`INSERT INTO workers (id, kind, generation, state, started_at, heartbeat_at) VALUES++ ('w_wi1','webrtc-ingest',1,'stopped',1,1),++ ('w_wi2','webrtc-ingest',2,'stopped',2,2),++ ('w_sfu','sfu',1,'stopped',3,3)`);++ await db.exec(fs.readFileSync(MIG2, 'utf8'));++++ const defs = Object.fromEntries((await db.prepare('SELECT id, protocols FROM stream_definitions').all()).map(r => [r.id, r.protocols]));++ assert.deepStrictEqual(JSON.parse(defs.stream_a), ['webrtc', 'rtmp']);++ assert.deepStrictEqual(JSON.parse(defs.stream_b), ['webrtc']);++ assert.deepStrictEqual(JSON.parse(defs.stream_c), ['rtmp', 'jsmpeg'], 'a row without whip is untouched');++ const workers = await db.prepare('SELECT kind, generation FROM workers ORDER BY generation').all();++ assert.ok(workers.every(w => w.kind === 'webrtc'), 'every legacy kind is webrtc');++ assert.deepStrictEqual(workers.map(w => w.generation).sort((a, b) => a - b), [1, 2, 3], 'generations stay unique');++ } finally {++ await db.close();++ }++});++++t.run();nnew file mode 100644@@@ -0,0 +1,74 @@++'use strict';++// T4 naming + the JSMPEG worker's store-level behaviour, without ffmpeg or PostgreSQL: 'whip' is++// an input alias for 'webrtc'; the worker kinds are one per transport; a jsmpeg session's playback++// descriptor is { jsmpeg: { ws_url, width, height } } (plus the loopback tap) and its viewers are++// reported; recording for jsmpeg is refused with a clear reason and an event.++const assert = require('assert');++const { runtime, tmpDir, suite, OWNER, outboxEnvelopes } = require('./helpers');++++const t = suite('jsmpeg');++++t('naming: whip is an input alias, and the worker kinds are one per transport', async () => {++ const rt = await runtime({ dir: tmpDir() });++ assert.deepStrictEqual(rt.store.workers.KINDS, ['rtmp-ingest', 'restream', 'webrtc', 'jsmpeg']);++++ const a = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['whip'] });++ assert.deepStrictEqual(a.definition.protocols, ['webrtc'], 'whip is stored as webrtc');++ const b = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['whip', 'webrtc', 'jsmpeg'] });++ assert.deepStrictEqual(b.definition.protocols, ['webrtc', 'jsmpeg'], 'deduplicated after aliasing');++ await assert.rejects(() => rt.store.definitions.create({ owner_subject: OWNER, protocols: ['sfu'] }), /protocols must be/);++++ await assert.rejects(() => rt.store.workers.register({ kind: 'webrtc-ingest' }), /unknown worker kind/);++ await assert.rejects(() => rt.store.workers.register({ kind: 'sfu' }), /unknown worker kind/);++ const w = await rt.store.workers.register({ kind: 'webrtc', endpoints: {} });++ assert.strictEqual(w.kind, 'webrtc');++ rt.db.close();++});++++t('a jsmpeg session playback descriptor: ws_url by playback id, width/height, the tap, viewers', async () => {++ const rt = await runtime({ dir: tmpDir() });++ const { definition, key } = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['jsmpeg'] });++ const w = await rt.store.workers.register({ kind: 'jsmpeg', endpoints: { publicPort: 9736, publicPorts: [9736], tapPort: 19710 } });++ await rt.store.workers.ready(w.id);++++ const a = await rt.store.sessions.admit({ definition, key, protocol: 'jsmpeg', worker: await rt.store.workers.get(w.id) });++ await rt.store.sessions.setMediaInfo(a.session.id, { width: 640, height: 360 });++ await rt.store.sessions.transition(a.session.id, 'live', { reason: 'media_flowing', actor: 'worker:w' });++ await rt.store.sessions.setViewers(a.session.id, 3);++++ const s = await rt.store.sessions.get(a.session.id);++ assert.strictEqual(s.viewers, 3, 'the heartbeat viewer count is on the session');++ const pb = await rt.store.sessions.playback(s);++ assert.deepStrictEqual(Object.keys(pb.jsmpeg).sort(), ['height', 'tap_internal_url', 'width', 'ws_url']);++ assert.strictEqual(pb.jsmpeg.ws_url, `ws://127.0.0.1:9736/${a.session.id}`);++ assert.ok(!pb.jsmpeg.ws_url.includes(key.key), 'the key is never in the viewer URL');++ assert.strictEqual(pb.jsmpeg.width, 640);++ assert.strictEqual(pb.jsmpeg.height, 360);++ assert.strictEqual(pb.jsmpeg.tap_internal_url, `http://127.0.0.1:19710/tap/${a.session.id}.ts`);++ rt.db.close();++});++++t('a jsmpeg session with recording on is refused with a clear reason and an event', async () => {++ const rt = await runtime({ dir: tmpDir(), env: { OPENRE_RECORDING_START_DELAY_MS: '0' } });++ const { definition } = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['jsmpeg'], recording_mode: 'vod' });++ const w = await rt.store.workers.register({ kind: 'jsmpeg', endpoints: { publicPort: 9736, tapPort: 19710 } });++ await rt.store.workers.ready(w.id);++ const a = await rt.store.sessions.admit({ definition, key: null, protocol: 'jsmpeg', worker: await rt.store.workers.get(w.id) });++ await rt.store.sessions.transition(a.session.id, 'live', { reason: 'media_flowing', actor: 'worker:w' });++++ const never = () => { throw new Error('Media must not be called for a jsmpeg recording'); };++ const media = { configured: true, createVod: never, ingestRtmp: never, finalizeVod: never, deleteVod: never };++ assert.strictEqual(await rt.store.recordings.ensureRequests(), 1);++ await rt.store.recordings.process(media);++++ const rec = await rt.store.recordings.bySession(a.session.id);++ assert.strictEqual(rec.state, 'failed');++ assert.strictEqual(rec.last_error, 'recording is not available for jsmpeg');++ const ev = (await outboxEnvelopes(rt.db)).find(e => e.event_type === 'openre.recording.failed');++ assert.ok(ev, 'a recording.failed event is emitted');++ assert.strictEqual(ev.payload.reason, 'recording is not available for jsmpeg');++ assert.strictEqual(ev.payload.protocol, 'jsmpeg');++ rt.db.close();++});++++t.run();@@@ -29,7 +29,8 @@ t('unknown, malformed and wrong-protocol keys are refused', async () => {const { key } = await rt.store.definitions.create({ owner_subject: OWNER });assert.strictEqual((await rt.store.definitions.resolveIngestKey('ork_' + 'A'.repeat(43), 'rtmp')).error, 'unknown_key');assert.strictEqual((await rt.store.definitions.resolveIngestKey('a3f9c2', 'rtmp')).error, 'malformed_key');-- assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'whip')).error, 'protocol_not_allowed');++ assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'webrtc')).error, 'protocol_not_allowed');++ assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'jsmpeg')).error, 'protocol_not_allowed');});t('rotate revokes the old key immediately and emits openre.key.rotated without any key', async () => {@@@ -85,6 +85,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held orassert.strictEqual((await rt.store.definitions.get(map['managed_stream:13'].target_id)).recording_mode, 'none', 'VOD off on the channel, clips off on the slot');assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_mode, 'clips');assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_visibility, 'private');++ assert.deepStrictEqual((await rt.store.definitions.get(map['managed_stream:13'].target_id)).protocols, ['webrtc'], 'a webrtc slot imports as a webrtc definition (unadmittable until the worker ships)');// Old keys: none of them authenticates, and none is stored in any form.for (const old of ['a3f9c2leakedkey0000000000000000000000000', 'personalkey111']) {@@@ -114,7 +115,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held orassert.match(md, /## @japaneseoldguy/);assert.match(md, /ingest-authority/);assert.match(md, /OAuth-linked/);-- assert.match(md, /only RTMP moves to OpenRe/, 'the WebRTC slot is flagged');++ assert.match(md, /webrtc worker is not ported/, 'the WebRTC slot is flagged');for (const secret of ['a3f9c2leakedkey', 'live_twitchsecret_1', 'yt-secret-2222', 'personalkey111', 'kick-secret-44']) assert.ok(!md.includes(secret), `no ${secret} in the checklist`);});@@@ -1,3 +1,317 @@'use strict';--/** openre-jsmpeg: worker interface only, the transport is not ported yet (see unported-transport.js). */--if (require.main === module) require('./unported-transport').main('jsmpeg').catch((err) => { console.error(`[jsmpeg] failed to start: ${err && err.stack || err}`); process.exit(1); });++/**++ * openre-jsmpeg — the JSMPEG transport worker (unit openre-jsmpeg@<release>.service).++ *++ * The relay Live runs in-process (OpenVibe.Live server/streaming/jsmpeg-relay.js), with the++ * lifecycle and ownership moved here:++ *++ * public 0.0.0.0:OPENRE_JSMPEG_PORT (9736 until the JSMPEG cutover), SO_REUSEPORT so two++ * generations can listen during a drain. The broadcaster's ffmpeg POSTs MPEG-TS to++ * http://<host>:<port>/<key>/<w>/<h>/, and viewers connect over WebSocket to++ * ws://<host>:<port>/<session playback id> (never the key).++ * internal 127.0.0.1:<tapPort> GET /tap/<session>.ts streams the raw MPEG-TS for restream.++ * The port is this generation's own (from OPENRE_JSMPEG_INTERNAL_PORT_MIN..MAX) and is++ * published in its worker row, so consumers always reach the process that holds the++ * publisher.++ *++ * Admission is key-in-path exactly like RTMP: resolveIngestKey with protocol 'jsmpeg', one open++ * session per definition. The session goes live on the first bytes and ends when the POST closes.++ *++ * Viewers only ever receive, so the WebSocket server is the ~30 lines that takes the upgrade and++ * writes unmasked binary frames; no ws dependency is needed.++ *++ * Drain: when the coordinator marks this generation draining (a newer one is ready) the public++ * listener closes — new encoders and viewers reach the new generation through the shared port — and++ * running sessions stay until they leave or the drain deadline passes. Then the process exits by++ * itself. SIGTERM starts the same drain; it never drops a live session.++ */++const http = require('http');++const crypto = require('crypto');++const path = require('path');++const { createWorkerRuntime } = require('./runtime');++++const SESSION_ID_RE = /^ses_[0-9A-HJKMNP-TV-Z]{26}$/;++const INGEST_PATH_RE = /^\/([^/]+)\/(\d{1,5})\/(\d{1,5})\/?$/;++const TAP_PATH_RE = /^\/tap\/(ses_[0-9A-HJKMNP-TV-Z]{26})\.ts$/;++const MAX_WS_BACKPRESSURE = 512 * 1024;++const MAX_ADMISSION_BUFFER = 4 * 1024 * 1024;++const WS_GUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11';++const OP_BINARY = 0x2;++const OP_CLOSE = 0x8;++++/** One unmasked WebSocket frame (the server never masks). */++function wsFrame(opcode, payload) {++ const len = payload.length;++ let header;++ if (len < 126) { header = Buffer.allocUnsafe(2); header[0] = 0x80 | opcode; header[1] = len; }++ else if (len < 65536) { header = Buffer.allocUnsafe(4); header[0] = 0x80 | opcode; header[1] = 126; header.writeUInt16BE(len, 2); }++ else { header = Buffer.allocUnsafe(10); header[0] = 0x80 | opcode; header[1] = 127; header.writeBigUInt64BE(BigInt(len), 2); }++ return Buffer.concat([header, payload]);++}++++/** Bind the first free loopback port in [min, max] (skipping `skip`). */++function listenInRange(server, min, max, skip = new Set()) {++ return new Promise((resolve, reject) => {++ let port = min;++ const tryNext = () => {++ while (skip.has(port)) port++;++ if (port > max) { reject(new Error(`no free loopback port in ${min}-${max}`)); return; }++ const onError = (err) => {++ server.removeListener('listening', onListening);++ if (err.code === 'EADDRINUSE' || err.code === 'EACCES') { port++; tryNext(); } else reject(err);++ };++ const onListening = () => { server.removeListener('error', onError); resolve(server.address().port); };++ server.once('error', onError);++ server.once('listening', onListening);++ server.listen(port, '127.0.0.1');++ };++ tryNext();++ });++}++++function createJsmpeg({ rt, log = console, exit = (code) => process.exit(code) }) {++ const { store, config } = rt;++ const j = config.jsmpeg;++ /** session id → { sessionId, definitionId, width, height, req, ended, live, endReason, viewers:Set, taps:Set } */++ const channels = new Map();++ // Ends in flight. The heartbeat calls exitWhenIdle() every beat; without this an ended session++ // whose `finish` write has not committed yet would let the drained process exit before it lands,++ // leaving the session live until the coordinator fails it lease_expired.++ let pendingEnds = 0;++ let ingestServer = null;++ let tapServer = null;++ let endpoints = null;++++ const runtime = createWorkerRuntime({++ rt, kind: 'jsmpeg', log, exit,++ hooks: {++ onDrain: () => closePublic(),++ onDrainDeadline: async () => await endAll('drain_deadline'),++ onEndRequested: async (s) => await endSession(s.id, 'end_requested'),++ onLost: async () => { for (const ch of [...channels.values()]) await endChannel(ch, 'worker_lost'); return closeAll(); },++ onHeartbeat: async () => { for (const ch of channels.values()) await store.sessions.setViewers(ch.sessionId, ch.viewers.size); },++ activeCount: () => channels.size + pendingEnds,++ onExit: () => closeAll(),++ },++ });++++ const actor = () => `worker:${runtime.me && runtime.me.id}`;++ /** Never let a rejected handler — Node 22 ends the worker on an unhandled rejection — drop every session on it. */++ function safe(promise, what) {++ return Promise.resolve(promise).catch((err) => log.error(`[jsmpeg] ${what}: ${err && (err.stack || err.message) || err}`));++ }++++ async function admit(key, width, height) {++ if (runtime.draining || !runtime.me || runtime.me.state !== 'ready') return { error: 'generation not taking sessions' };++ if (channels.size >= j.maxPublishersPerWorker) return { error: 'worker full' };++ const r = await store.definitions.resolveIngestKey(key, 'jsmpeg');++ if (r.error) return { error: r.error };++ const a = await store.sessions.admit({ definition: r.definition, key: r.key, protocol: 'jsmpeg', worker: runtime.me });++ if (a.error) return { error: a.error };++ return { sessionId: a.session.id, definitionId: r.definition.id, hint: r.key.hint, width, height };++ }++++ function handleIngest(req, res, key, width, height) {++ const buffered = [];++ let bufferedBytes = 0;++ let ch = null;++ let done = false;++++ req.on('data', (chunk) => {++ if (ch) { feed(ch, chunk); return; }++ if (bufferedBytes + chunk.length > MAX_ADMISSION_BUFFER) return;++ buffered.push(chunk);++ bufferedBytes += chunk.length;++ });++ req.on('error', () => { if (ch) safe(endChannel(ch, 'publisher_disconnected'), 'error end'); });++++ safe((async () => {++ const adm = await admit(key, width, height);++ if (done) return;++ if (adm.error) {++ log.log(`[jsmpeg] publish refused (${adm.error}) from ${req.socket.remoteAddress || '?'}`);++ res.statusCode = 404;++ res.setHeader('content-type', 'text/plain');++ res.end(`refused: ${adm.error}\n`);++ req.destroy();++ return;++ }++ ch = { sessionId: adm.sessionId, definitionId: adm.definitionId, req, width, height, live: false, ended: false, endReason: null, viewers: new Set(), taps: new Set() };++ channels.set(ch.sessionId, ch);++ log.log(`[jsmpeg] session ${ch.sessionId} accepted for ${ch.definitionId} (key …${adm.hint}) at ${width}x${height}`);++ res.writeHead(200, { 'content-type': 'video/mp2t' });++ for (const c of buffered.splice(0)) feed(ch, c);++ bufferedBytes = 0;++ })(), 'admission failed');++++ req.on('end', () => {++ done = true;++ try { res.end(); } catch { /* already closed */ }++ if (ch) safe(endChannel(ch, ch.endReason || 'publisher_disconnected'), 'end');++ });++ req.on('close', () => {++ done = true;++ if (ch && !ch.ended) safe(endChannel(ch, ch.endReason || 'publisher_disconnected'), 'close');++ });++ }++++ async function ensureLive(ch) {++ if (ch.live || ch.ended) return;++ ch.live = true;++ await store.sessions.setMediaInfo(ch.sessionId, { width: ch.width, height: ch.height });++ const t = await store.sessions.transition(ch.sessionId, 'live', { reason: 'media_flowing', actor: actor() });++ if (!t.ok) {++ // The coordinator failed it meanwhile (lease) — the transport must not outlive its record.++ log.warn(`[jsmpeg] session ${ch.sessionId} could not go live (${t.code}); closing`);++ ch.endReason = 'coordinator_failed';++ try { if (ch.req && !ch.req.destroyed) ch.req.destroy(); } catch { /* gone */ }++ }++ }++++ function feed(ch, chunk) {++ if (ch.ended) return;++ if (!ch.live) safe(ensureLive(ch), 'go live');++ for (const v of ch.viewers) {++ if (v.socket.destroyed || v.socket.writableLength > MAX_WS_BACKPRESSURE) continue;++ try { v.socket.write(wsFrame(OP_BINARY, chunk)); } catch { /* socket raced with close */ }++ }++ for (const tap of ch.taps) { try { tap(chunk); } catch { /* tap closed */ } }++ }++++ async function endChannel(ch, reason) {++ if (ch.ended) return;++ ch.ended = true;++ channels.delete(ch.sessionId);++ pendingEnds++;++ try {++ for (const v of ch.viewers) { try { v.socket.end(wsFrame(OP_CLOSE, Buffer.alloc(0))); } catch { /* gone */ } }++ ch.viewers.clear();++ for (const end of ch.taps) { try { end(); } catch { /* gone */ } }++ ch.taps.clear();++ try { if (ch.req && !ch.req.destroyed) ch.req.destroy(); } catch { /* gone */ }++ await store.sessions.finish(ch.sessionId, { reason, actor: actor() });++ log.log(`[jsmpeg] session ${ch.sessionId} ended (${reason})`);++ } finally {++ pendingEnds--;++ }++ if (runtime.draining) runtime.exitWhenIdle();++ }++++ async function endSession(sessionId, reason) {++ const ch = channels.get(sessionId);++ if (ch) { ch.endReason = reason; await endChannel(ch, reason); return true; }++ // Not ours any more (publisher already gone): make sure the record is closed. Counted so a++ // drained generation does not exit before the write lands.++ pendingEnds++;++ try { await store.sessions.finish(sessionId, { reason, actor: actor() }); } finally { pendingEnds--; }++ if (runtime.draining) runtime.exitWhenIdle();++ return false;++ }++++ async function endAll(reason) {++ for (const ch of [...channels.values()]) { ch.endReason = reason; await endChannel(ch, reason); }++ }++++ function handleUpgrade(req, socket) {++ let pathname;++ try { pathname = new URL(req.url, 'http://127.0.0.1').pathname; } catch { socket.destroy(); return; }++ const id = pathname.replace(/^\//, '');++ const ch = SESSION_ID_RE.test(id) ? channels.get(id) : null;++ const key = req.headers['sec-websocket-key'];++ if (!ch || typeof key !== 'string') { socket.destroy(); return; }++ const accept = crypto.createHash('sha1').update(key + WS_GUID).digest('base64');++ socket.write('HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n'++ + `Sec-WebSocket-Accept: ${accept}\r\n\r\n`);++ socket.setNoDelay(true);++ const viewer = { socket };++ ch.viewers.add(viewer);++ log.log(`[jsmpeg] viewer connected to ${ch.sessionId} (${ch.viewers.size} total)`);++ const drop = () => { ch.viewers.delete(viewer); };++ socket.on('close', drop);++ socket.on('error', drop);++ // Viewers never send: consume and ignore anything inbound (ping/close frames).++ socket.on('data', () => {});++ }++++ function handleTap(req, res) {++ const m = TAP_PATH_RE.exec(new URL(req.url, 'http://127.0.0.1').pathname);++ const ch = m && req.method === 'GET' ? channels.get(m[1]) : null;++ if (!ch) { res.statusCode = 404; return res.end(); }++ res.writeHead(200, { 'content-type': 'video/mp2t', 'cache-control': 'no-store' });++ const tap = (chunk) => { try { if (!res.writableEnded) res.write(chunk); } catch { /* client gone */ } };++ const end = () => { ch.taps.delete(tap); try { if (!res.writableEnded) res.end(); } catch { /* gone */ } };++ ch.taps.add(tap);++ req.on('close', end);++ return undefined;++ }++++ function closePublic() {++ if (!ingestServer) return;++ ingestServer.close();++ log.log('[jsmpeg] public listener closed; running sessions stay until they leave');++ }++++ function closeAll() {++ closePublic();++ const closing = [];++ for (const srv of [ingestServer, tapServer]) {++ if (!srv) continue;++ srv.closeAllConnections?.();++ closing.push(new Promise(r => srv.close(() => r())));++ }++ ingestServer = null;++ tapServer = null;++ return Promise.all(closing);++ }++++ async function start() {++ // Internal endpoint first: it is how this generation is reached, so it goes in the worker++ // row before it can take a session.++ tapServer = http.createServer((req, res) => { handleTap(req, res); });++ const tapPort = await listenInRange(tapServer, j.internalPortMin, j.internalPortMax);++ endpoints = { publicPort: j.port, publicPorts: [j.port], tapPort };++ await runtime.register(endpoints);++++ ingestServer = http.createServer((req, res) => {++ if (req.method !== 'POST') { res.statusCode = 404; return res.end(); }++ const m = INGEST_PATH_RE.exec(new URL(req.url, 'http://127.0.0.1').pathname);++ if (!m) { res.statusCode = 404; return res.end(); }++ handleIngest(req, res, m[1], Number(m[2]), Number(m[3]));++ return undefined;++ });++ ingestServer.on('upgrade', handleUpgrade);++ await new Promise((resolve, reject) => {++ ingestServer.once('error', reject);++ ingestServer.listen({ port: j.port, host: j.bindHost, reusePort: true }, () => resolve());++ });++ ingestServer.on('error', (err) => log.error(`[jsmpeg] public listener ${j.port}: ${err.message}`));++ await runtime.ready();++ log.log(`[jsmpeg] generation ${runtime.me.generation} ready: ingest+ws ${j.bindHost}:${j.port}, tap 127.0.0.1:${tapPort}`);++ return endpoints;++ }++++ return {++ start,++ runtime,++ endpoints: () => endpoints,++ channels: () => [...channels.values()],++ drain: async (reason) => await runtime.drainNow(reason),++ close: closeAll,++ };++}++++if (require.main === module) {++ (async () => {++ require('dotenv').config({ path: process.env.OPENRE_ENV_FILE || path.join(process.cwd(), '.env') });++ const { load, exitIfDrill } = require('../server/config');++ const { openRuntime } = require('../server/store');++ const config = load();++ exitIfDrill(config, 'openre-jsmpeg');++ const rt = await openRuntime({ config });++ const worker = createJsmpeg({ rt });++ await worker.start();++ for (const sig of ['SIGTERM', 'SIGINT']) process.on(sig, () => worker.drain(sig).catch((err) => console.error(`[jsmpeg] drain failed on ${sig}: ${err && (err.stack || err.message) || err}`)));++ })().catch((err) => { console.error(`[jsmpeg] failed to start: ${err && err.stack || err}`); process.exit(1); });++}++++module.exports = { createJsmpeg, listenInRange };@@@ -31,10 +31,17 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exit},});++ /** The loopback source URL for a session's protocol, or null when OpenRe cannot restream it yet. */async function inputUrlFor(session) {-- if (session.protocol !== 'rtmp') return null;-- const pb = await store.sessions.playback(session);-- return pb && pb.flv ? pb.flv.internal_url : null;++ if (session.protocol === 'rtmp') {++ const pb = await store.sessions.playback(session);++ return pb && pb.flv ? pb.flv.internal_url : null;++ }++ if (session.protocol === 'jsmpeg') {++ const pb = await store.sessions.playback(session);++ return pb && pb.jsmpeg ? pb.jsmpeg.tap_internal_url : null;++ }++ return null;}async function poll() {@@@ -59,7 +66,7 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exitif (!input) { await store.outputs.report(row.id, { state: 'failed', last_error: `restream from ${session.protocol} sessions is not supported by OpenRe yet`, ended_at: Date.now() }, { workerId: runtime.me.id }); continue; }// Let the ingest settle before pulling (Live waits 3 s for node-media-server's FLV).if (session.live_at && Date.now() - session.live_at < config.outputs.startDelayMs) continue;-- runner = new OutputRunner({ outputId: row.id, destinationId: row.destination_id, inputUrl: input, store, config, log, spawnImpl, lookup, workerId: runtime.me.id });++ runner = new OutputRunner({ outputId: row.id, destinationId: row.destination_id, inputUrl: input, source: session.protocol, store, config, log, spawnImpl, lookup, workerId: runtime.me.id });runners.set(row.id, runner);runner.start().catch((err) => runner.fail(err.message, { cooldown: false }));}@@@ -5,9 +5,9 @@* output flags, friendly errors). Kept byte-for-byte equivalent where it matters so a stream that* restreams correctly from Live restreams the same way from OpenRe.*-- * Only the RTMP source is ported: RTMP → destination is a codec copy (zero CPU). The JSMPEG-- * (MPEG-TS on stdin) and WebRTC (mediasoup PlainRTP → SDP) sources stay in Live until those-- * ingest protocols move; encodingArgs() is here for them and for a future re-encode option.++ * RTMP → destination is a codec copy (zero CPU); a JSMPEG source (the worker's MPEG-TS data tap,++ * read over HTTP) is re-encoded with encodingArgs(). The WebRTC (mediasoup PlainRTP → SDP) source++ * stays in Live until that ingest protocol moves.*/const SRT_DEFAULT_LATENCY_MS = 120;@@@ -69,6 +69,19 @@ function rtmpCopyArgs(flvUrl, destUrl) {];}++/** MPEG-TS source (JSMPEG data tap) → destination, re-encoded (Live's _startJsmpegRestream). */++function jsmpegArgs(tsUrl, destUrl, preset, { overrides = {} } = {}) {++ return [++ '-hide_banner',++ '-loglevel', 'warning',++ '-thread_queue_size', '1024',++ '-f', 'mpegts',++ '-i', tsUrl,++ ...encodingArgs(preset, { hasAudio: true, overrides }),++ ...outputArgs(destUrl),++ ];++}++/** `-progress pipe:1 -stats_period 1` after the global flags: the live ACK and the health line. */function withProgress(args) {return args[0] === '-hide_banner'@@@ -105,7 +118,8 @@ function encodingArgs(preset, { hasAudio = true, overrides = {} } = {}) {'-sc_threshold', '0', '-flags', '+cgop', '-pix_fmt', 'yuv420p', '-threads', '2', '-x264-params', 'nal-hrd=cbr:force-cfr=1'];if (preset.scale) args.push('-vf', `scale=${preset.scale}:force_original_aspect_ratio=decrease,pad=${preset.scale}:(ow-iw)/2:(oh-ih)/2`);if (fps > 0) args.push('-r', String(fps), '-fps_mode', 'cfr');-- if (hasAudio) args.push('-map', '0:a:0', '-c:a', 'aac', '-b:a', audioBitrate, '-ar', '48000', '-ac', '2');++ // The trailing '?' keeps a video-only source (a jsmpeg tap without audio) restreamable.++ if (hasAudio) args.push('-map', '0:a:0?', '-c:a', 'aac', '-b:a', audioBitrate, '-ar', '48000', '-ac', '2');return args;}@@@ -150,5 +164,5 @@ function redactUrl(value) {module.exports = {QUALITY_PRESETS, PLATFORM_DEFAULT_PRESET, ENCODER_PRESETS, SRT_DEFAULT_LATENCY_MS,-- isSrtUrl, buildDestUrl, buildSrtUrl, outputArgs, rtmpCopyArgs, withProgress, resolvePreset, customOverrides, encodingArgs, friendlyError, redactUrl, redactText,++ isSrtUrl, buildDestUrl, buildSrtUrl, outputArgs, rtmpCopyArgs, jsmpegArgs, withProgress, resolvePreset, customOverrides, encodingArgs, friendlyError, redactUrl, redactText,};@@@ -8,16 +8,17 @@*/const { spawn } = require('child_process');const fs = require('fs');--const { rtmpCopyArgs, withProgress, buildDestUrl, friendlyError, redactUrl, redactText } = require('./ffmpeg-args');++const { rtmpCopyArgs, jsmpegArgs, withProgress, buildDestUrl, resolvePreset, customOverrides, friendlyError, redactUrl, redactText } = require('./ffmpeg-args');const { validateDestinationUrl, checkResolvedHost } = require('../../server/destination-url');class OutputRunner {-- constructor({ outputId, destinationId, inputUrl, store, config, log = console, spawnImpl = spawn, lookup, workerId = null }) {++ constructor({ outputId, destinationId, inputUrl, source = 'rtmp', store, config, log = console, spawnImpl = spawn, lookup, workerId = null }) {this.outputId = outputId;this.workerId = workerId;this.secrets = [];this.destinationId = destinationId;this.inputUrl = inputUrl;++ this.source = source;this.store = store;this.config = config;this.log = log;@@@ -68,11 +69,15 @@ class OutputRunner {if (!resolved.ok) return this.fail(`destination refused: ${resolved.error}`, { cooldown: false });const destUrl = buildDestUrl(dest);if (!destUrl) return this.fail('destination has no usable URL or stream key', { cooldown: false });++ this.preset = resolvePreset(dest);++ this.overrides = customOverrides(dest);this.spawn(destUrl);}spawn(destUrl) {-- const args = withProgress(rtmpCopyArgs(this.inputUrl, destUrl));++ const args = withProgress(this.source === 'jsmpeg'++ ? jsmpegArgs(this.inputUrl, destUrl, this.preset || resolvePreset({}), this.overrides ? { overrides: this.overrides } : {})++ : rtmpCopyArgs(this.inputUrl, destUrl));const rtmps = destUrl.startsWith('rtmps://');const bin = rtmps && this.o.ffmpegOpenSslPath && fs.existsSync(this.o.ffmpegOpenSslPath) ? this.o.ffmpegOpenSslPath : this.o.ffmpegPath;this.note('info', `starting ffmpeg → ${redactUrl(destUrl)}${bin !== this.o.ffmpegPath ? ' (openssl build)' : ''}`);ddeleted file mode 100644@@@ -1,3 +0,0 @@--'use strict';--/** openre-sfu: worker interface only, the transport is not ported yet (see unported-transport.js). */--if (require.main === module) require('./unported-transport').main('sfu').catch((err) => { console.error(`[sfu] failed to start: ${err && err.stack || err}`); process.exit(1); });@@@ -1,22 +1,20 @@'use strict';/**-- * The worker interface for the transports OpenRe does not carry yet: WHIP/WebRTC ingest-- * (werift), the mediasoup SFU and JSMPEG. Each registers a generation with the coordinator, takes-- * part in the drain protocol and heartbeats like a real transport worker, and carries nothing:-- * activeCount() is always 0, and no session of these protocols is ever admitted (their stream-- * definitions may list the protocol, but no worker accepts it, so encoders keep using Live).++ * The worker interface for the transports OpenRe does not carry yet. Today only WebRTC: the port++ * (WHIP ingest + mediasoup SFU + viewer signaling) is T4's next job (decision 1, one worker owns++ * all of it). It registers a generation with the coordinator, takes part in the drain protocol and++ * heartbeats like a real transport worker, and carries nothing: activeCount() is always 0, and no++ * 'webrtc' session is ever admitted (a definition may list the protocol, but its HTTP error is++ * 'protocol_not_allowed' — no worker accepts it, so encoders keep using Live).*-- * This is the seam the port lands in: a ported transport replaces `admit nothing` with its-- * session code and reports its sessions through store.sessions exactly like rtmp-ingest.js.-- * Nothing in production runs these (deploy/ has no unit for them).++ * This is the seam the port lands in: it replaces `admit nothing` with its session code and reports++ * sessions through store.sessions exactly like rtmp-ingest.js / jsmpeg.js.*/const path = require('path');const { createWorkerRuntime } = require('./runtime');const PORT_STATUS = Object.freeze({-- 'webrtc-ingest': 'WHIP/WebRTC ingest (Live: server/streaming/whip-handler.js, werift) — not ported',-- sfu: 'WebRTC SFU (Live: server/streaming/webrtc-sfu.js + broadcast-server.js, mediasoup) — not ported',-- jsmpeg: 'JSMPEG relay (Live: server/streaming/jsmpeg-relay.js) — not ported',++ webrtc: 'WebRTC (WHIP ingest + mediasoup SFU + viewer signaling; Live: server/streaming/whip-handler.js + webrtc-sfu.js + broadcast-server.js) — not ported',});function createUnportedTransport({ rt, kind, log = console, exit = (code) => process.exit(code) }) {ddeleted file mode 100644@@@ -1,3 +0,0 @@--'use strict';--/** openre-webrtc-ingest: worker interface only, the transport is not ported yet (see unported-transport.js). */--if (require.main === module) require('./unported-transport').main('webrtc-ingest').catch((err) => { console.error(`[webrtc-ingest] failed to start: ${err && err.stack || err}`); process.exit(1); });nnew file mode 100644@@@ -0,0 +1,13 @@++'use strict';++/**++ * openre-webrtc — the WebRTC transport worker (unit openre-webrtc@<release>.service).++ *++ * One worker owns WHIP ingest, the mediasoup producers/consumers and viewer signaling (T4 decision++ * 1: mediasoup is single-process, so ingest and consumption cannot be split). This replaces the++ * webrtc-ingest.js and sfu.js stubs.++ *++ * Not ported yet: it registers a real generation and takes part in the drain protocol, but carries++ * nothing (see unported-transport.js). The next T4 job ports it.++ */++if (require.main === module) require('./unported-transport').main('webrtc').catch((err) => { console.error(`[webrtc] failed to start: ${err && err.stack || err}`); process.exit(1); });++module.exports = require('./unported-transport');
@@ -43,11 +43,11 @@ The ingest/restream control plane and runtime extracted from OpenVibe.Live. Live
@@ -43,11 +43,11 @@ The ingest/restream control plane and runtime extracted from OpenVibe.Live. Live
|---|---|---|---|
|---|---|---|---|
| RTMP ingest | `server/streaming/rtmp-server.js` (node-media-server) | `workers/rtmp-ingest.js` — same NMS session code; OpenRe's own keys; publish renamed to `/live/<session id>` so no key reaches a play URL; loopback RTMP play + HTTP-FLV per generation; SO_REUSEPORT drain | **ported, tested with real ffmpeg** |
| RTMP ingest | `server/streaming/rtmp-server.js` (node-media-server) | `workers/rtmp-ingest.js` — same NMS session code; OpenRe's own keys; publish renamed to `/live/<session id>` so no key reaches a play URL; loopback RTMP play + HTTP-FLV per generation; SO_REUSEPORT drain | **ported, tested with real ffmpeg** |
| Restream (RTMP source) | `server/streaming/restream-manager.js` | `workers/restream/*` — same ffmpeg arguments (codec copy from HTTP-FLV), Twitch→RTMPS, Kick `/app`, SRT options, live ACK by `-progress`, backoff, rapid-crash circuit breaker, destination cooldown | **ported, tested with real ffmpeg** |
| Restream (RTMP source) | `server/streaming/restream-manager.js` | `workers/restream/*` — same ffmpeg arguments (codec copy from HTTP-FLV), Twitch→RTMPS, Kick `/app`, SRT options, live ACK by `-progress`, backoff, rapid-crash circuit breaker, destination cooldown | **ported, tested with real ffmpeg** |
-| Restream from JSMPEG / WebRTC sources | same | encoder args ported (`encodingArgs`), sources not (they need those ingests) | not ported |
+| Restream from a JSMPEG source | same | `jsmpegArgs()` — re-encodes the worker's MPEG-TS data tap (`http://127.0.0.1:<tapPort>/tap/<session>.ts`) | **ported, tested with real ffmpeg** |
+| Restream from a WebRTC source | same | encoder args ported (`encodingArgs`), source not (it needs that ingest) | not ported |
| Recording | `server/streaming/recorder.js` + `server/media-client.js` | `server/store/recordings.js` + `server/media-client.js` — same Media API v1 calls (create VOD, `ingest/rtmp`, finalize, delete shell / clips-only) driven by the coordinator; Media pulls the loopback play URL | **ported (RTMP)**, tested against a stub Media that really pulls the URL |
| Recording | `server/streaming/recorder.js` + `server/media-client.js` | `server/store/recordings.js` + `server/media-client.js` — same Media API v1 calls (create VOD, `ingest/rtmp`, finalize, delete shell / clips-only) driven by the coordinator; Media pulls the loopback play URL | **ported (RTMP)**, tested against a stub Media that really pulls the URL |
-| WHIP / WebRTC ingest | `whip-handler.js` (werift) | `workers/webrtc-ingest.js`: worker interface + coordinator registration only | **not ported** |
-| SFU | `webrtc-sfu.js` + `broadcast-server.js` (mediasoup) | `workers/sfu.js`: worker interface only | **not ported** |
-| JSMPEG | `jsmpeg-relay.js` | `workers/jsmpeg.js`: worker interface only | **not ported** |
+| WebRTC (WHIP ingest + SFU + viewer signaling) | `whip-handler.js` + `webrtc-sfu.js` + `broadcast-server.js` (werift + mediasoup; one worker owns all of it) | `workers/webrtc.js`: worker interface + coordinator registration only | **not ported** |
+| JSMPEG | `jsmpeg-relay.js` | `workers/jsmpeg.js` — MPEG-TS POST admission by key in path, WebSocket viewers by playback id, loopback MPEG-TS data tap for restream; SO_REUSEPORT drain | **ported, tested with real ffmpeg** |
| OAuth-linked destinations (per go-live Twitch/YouTube key refresh, YouTube broadcast creation) | `restream-manager._refreshDestFromConnection` | no (Live owns platform OAuth); OpenRe pushes to the stored key | **not ported** |
| OAuth-linked destinations (per go-live Twitch/YouTube key refresh, YouTube broadcast creation) | `restream-manager._refreshDestFromConnection` | no (Live owns platform OAuth); OpenRe pushes to the stored key | **not ported** |
| Viewer counts from platforms, chat relay, PowerChat | restream-manager / integrations | no (product features, stay in Live) | not in scope |
| Viewer counts from platforms, chat relay, PowerChat | restream-manager / integrations | no (product features, stay in Live) | not in scope |
@@ -134,11 +134,12 @@ Written to `event_outbox` in the same transaction as the change (openvibe-sdk `c
@@ -134,11 +134,12 @@ Written to `event_outbox` in the same transaction as the change (openvibe-sdk `c
| `openre.session.failed` | `ingest_session` | worker lost, lease expired, … (`was_live`, reason) |
| `openre.session.failed` | `ingest_session` | worker lost, lease expired, … (`was_live`, reason) |
| `openre.output.healthy` / `openre.output.failed` | `output` | an output confirmed live / gave up (error, cooldown) |
| `openre.output.healthy` / `openre.output.failed` | `output` | an output confirmed live / gave up (error, cooldown) |
| `openre.recording.requested` | `recording` | Media accepted the recording request (Media app + VOD id) |
| `openre.recording.requested` | `recording` | Media accepted the recording request (Media app + VOD id) |
+| `openre.recording.failed` | `recording` | a recording could not start (e.g. `recording is not available for jsmpeg`) |
| `openre.key.rotated` | `stream` | new key id + hint, retired key ids, grace end |
| `openre.key.rotated` | `stream` | new key id + hint, retired key ids, grace end |
## Playback
## Playback
-`GET /api/v1/sessions/:id/playback` returns a descriptor: `flv.internal_url` (loopback HTTP-FLV on the worker that holds the session — for services on this host), `flv.public_url` (`https://openre.stream/play/<session>.flv`, proxied by the API for non-private streams), `rtmp.internal_url` (loopback RTMP play, what Media records from). `webrtc` and `hls` are `null` (not produced). Live's player keeps using its own `/api/streams/rtmp-proxy/:id.flv`, which for an OpenRe session proxies `flv.internal_url` (see the Live patch).
+`GET /api/v1/sessions/:id/playback` returns a descriptor: `flv.internal_url` (loopback HTTP-FLV on the worker that holds the session — for services on this host), `flv.public_url` (`https://openre.stream/play/<session>.flv`, proxied by the API for non-private streams), `rtmp.internal_url` (loopback RTMP play, what Media records from). A **jsmpeg** session's descriptor is `jsmpeg: { ws_url, width, height }` (`ws_url` is the worker's public WebSocket path carrying the session's playback id, never the key; `width`/`height` come from the ffmpeg POST path), plus `jsmpeg.tap_internal_url`, the loopback MPEG-TS tap the restream worker reads. `webrtc` and `hls` are `null` (not produced). The session API also reports `viewers` (connected JSMPEG viewers, from the worker heartbeat). Live's player keeps using its own `/api/streams/rtmp-proxy/:id.flv`, which for an OpenRe session proxies `flv.internal_url` (see the Live patch).
## Recording
## Recording
@@ -162,6 +163,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).
@@ -162,6 +163,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).
| 9935/tcp (loopback) | Live's HTTP-FLV | same |
| 9935/tcp (loopback) | Live's HTTP-FLV | same |
| **1936/tcp** | OpenRe `openre-rtmp-ingest` (all generations, SO_REUSEPORT) | permanent: URLs handed out as `rtmp://ingest.openre.stream:1936/live` keep working |
| **1936/tcp** | OpenRe `openre-rtmp-ingest` (all generations, SO_REUSEPORT) | permanent: URLs handed out as `rtmp://ingest.openre.stream:1936/live` keep working |
| 19360–19399/tcp (loopback) | OpenRe per-generation RTMP play + HTTP-FLV | permanent |
| 19360–19399/tcp (loopback) | OpenRe per-generation RTMP play + HTTP-FLV | permanent |
+| **9736/tcp** | OpenRe `openre-jsmpeg` (all generations, SO_REUSEPORT): MPEG-TS POST + WS viewers | permanent: `http://ingest.openre.stream:9736/<key>/<w>/<h>/` and `ws://…/<session>` |
+| 19710–19749/tcp (loopback) | OpenRe per-generation JSMPEG MPEG-TS data tap | permanent |
| 4500/tcp (loopback) | `openre-api` | permanent |
| 4500/tcp (loopback) | `openre-api` | permanent |
The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_PORTS=1935` and deploy a worker generation (`deploy.sh workers`); from then on `rtmp://ingest.openre.stream/live` works too, and 1936 keeps working. `ingest.openre.stream` is a DNS-only record (RTMP cannot go through Cloudflare's proxy); open 1936/tcp at the host firewall and the provider edge.
The switch to 1935: once Live no longer listens on 1935, set `OPENRE_RTMP_EXTRA_PORTS=1935` and deploy a worker generation (`deploy.sh workers`); from then on `rtmp://ingest.openre.stream/live` works too, and 1936 keeps working. `ingest.openre.stream` is a DNS-only record (RTMP cannot go through Cloudflare's proxy); open 1936/tcp at the host firewall and the provider edge.
@@ -183,7 +186,7 @@ sudo deploy/scripts/deploy.sh status # deploy-legacy.sh
@@ -183,7 +186,7 @@ sudo deploy/scripts/deploy.sh status # deploy-legacy.sh
- ovhost refuses an API restart while an ingest session is open (`--wait-idle` holds it, `--force` goes ahead), records every attempt (`ovhost releases openre`), prunes releases beyond five but never one a worker generation runs from, and announces the release. When ovhost is missing, too old or does not deploy OpenRe with `release-layout`, the wrapper runs [deploy/scripts/deploy-legacy.sh](deploy/scripts/deploy-legacy.sh), the previous script, unchanged (`OVHOST_LEGACY=1` forces it). `workers`, `status` and `prune` always run it.
- ovhost refuses an API restart while an ingest session is open (`--wait-idle` holds it, `--force` goes ahead), records every attempt (`ovhost releases openre`), prunes releases beyond five but never one a worker generation runs from, and announces the release. When ovhost is missing, too old or does not deploy OpenRe with `release-layout`, the wrapper runs [deploy/scripts/deploy-legacy.sh](deploy/scripts/deploy-legacy.sh), the previous script, unchanged (`OVHOST_LEGACY=1` forces it). `workers`, `status` and `prune` always run it.
- An **API deploy** never restarts a worker unit (no unit depends on another). Viewers of `openre.stream/play/…` reconnect; encoders, restreams and recordings do not notice.
- An **API deploy** never restarts a worker unit (no unit depends on another). Viewers of `openre.stream/play/…` reconnect; encoders, restreams and recordings do not notice.
-- A **worker deploy** starts `openre-rtmp-ingest@<sha>` and `openre-restream-worker@<sha>`; older instances are disabled (not stopped) and exit on their own when drained. Never `systemctl restart` a worker instance during a broadcast; `systemctl stop` starts a drain and waits up to 30 min (`TimeoutStopSec`), then kills.
+- A **worker deploy** starts `openre-rtmp-ingest@<sha>`, `openre-restream-worker@<sha>` and `openre-jsmpeg@<sha>`; older instances are disabled (not stopped) and exit on their own when drained. Never `systemctl restart` a worker instance during a broadcast; `systemctl stop` starts a drain and waits up to 30 min (`TimeoutStopSec`), then kills.
- Worker instances are named after the release. `deploy.sh workers <sha>` for a release that already runs a generation does nothing. An env change that workers read, such as `OPENRE_RTMP_BIND`, takes effect with the next release's generation (docs/cutover.md A3).
- Worker instances are named after the release. `deploy.sh workers <sha>` for a release that already runs a generation does nothing. An env change that workers read, such as `OPENRE_RTMP_BIND`, takes effect with the next release's generation (docs/cutover.md A3).
- **Restore drills** (`ovhost drill openre`, OpenVibe.Host) start only `openre-api`, with `OPENRE_DRILL=1`, on a restored copy of the database. In drill mode the API serves reads and refuses writes, `/play/` and sign-in with 503 `openre.drill_read_only`. The coordinator and every worker exit before they open the database. The event relay and Media calls are off (`test/drill-mode.test.js`). Never set `OPENRE_DRILL` in `/etc/openvibe/openre.env`; the preflight's `env` check fails if it is set.
- **Restore drills** (`ovhost drill openre`, OpenVibe.Host) start only `openre-api`, with `OPENRE_DRILL=1`, on a restored copy of the database. In drill mode the API serves reads and refuses writes, `/play/` and sign-in with 503 `openre.drill_read_only`. The coordinator and every worker exit before they open the database. The event relay and Media calls are off (`test/drill-mode.test.js`). Never set `OPENRE_DRILL` in `/etc/openvibe/openre.env`; the preflight's `env` check fails if it is set.
- Rollback: when `openre-api` is not ready within 60 s ovhost switches `current` back to the previous
- Rollback: when `openre-api` is not ready within 60 s ovhost switches `current` back to the previous
@@ -237,9 +240,13 @@ the env, the bind, DNS, the port from outside, the database, Live's env, the Eve
@@ -237,9 +240,13 @@ the env, the bind, DNS, the port from outside, the database, Live's env, the Eve
one slot. Status on 2026-09-23: OpenRe is deployed (`655b98a10aaa`) and DNS is done. The release with
one slot. Status on 2026-09-23: OpenRe is deployed (`655b98a10aaa`) and DNS is done. The release with
`6dc78a5`, the public bind, the provider port, Live's settings and the subscription are not.
`6dc78a5`, the public bind, the provider port, Live's settings and the subscription are not.
-### WHIP, JSMPEG, SFU (not ready)
+### JSMPEG (worker ready)
-Each needs its transport ported into its worker (`workers/webrtc-ingest.js`, `workers/jsmpeg.js`, `workers/sfu.js` have the interface and registration), a restream source for it, a playback descriptor (`webrtc`), recording (RTP to Media for WebRTC; JSMPEG has no Media ingest), and `OPENRE_PROTOCOLS` in Live's `server/openre/authority.js` extended. The same per-slot switch, window and rollback apply.
+`workers/jsmpeg.js` is ported (key-in-path admission, WebSocket viewers by playback id, the MPEG-TS data tap, generations/drain), and `jsmpegArgs()` restreams it. docs/cutover.md has the JSMPEG slot procedure (J-A once, then J-B1–B6 per slot): set `OPENRE_JSMPEG_BIND=0.0.0.0`, open 9736 at the provider edge, add `jsmpeg` to Live's `OPENRE_PROTOCOLS`, then the per-slot window/switch/rollback. JSMPEG has **no recording** (parity with Live): a jsmpeg definition with recording on fails the request explicitly.
+
+### WebRTC (not ready)
+
+One `workers/webrtc.js` will own WHIP ingest, the mediasoup producers/consumers and viewer signaling (decision 1: mediasoup is single-process, so ingest and consumption cannot be split). It needs its ported transport, a restream source for it, a `webrtc` playback descriptor, recording (RTP to Media) and `webrtc` in Live's `OPENRE_PROTOCOLS`. The same per-slot switch, window and rollback apply.
## Acceptance
## Acceptance
@@ -12,9 +12,8 @@
@@ -12,9 +12,8 @@
"protocols": {
"protocols": {
"rtmp": "ingest ported (openre-rtmp-ingest)",
"rtmp": "ingest ported (openre-rtmp-ingest)",
"restream": "RTMP source to RTMP/RTMPS/SRT destinations ported (openre-restream-worker)",
"restream": "RTMP source to RTMP/RTMPS/SRT destinations ported (openre-restream-worker)",
- "whip": "not ported (worker interface only)",
- "webrtc-sfu": "not ported (worker interface only)",
- "jsmpeg": "not ported (worker interface only)"
+ "webrtc": "not ported (workers/webrtc.js: one worker will own WHIP ingest, the SFU and viewer signaling; worker interface only)",
+ "jsmpeg": "worker ported (workers/jsmpeg.js): MPEG-TS POST admission by key, WS viewers, loopback data tap, restream (re-encode), generations/drain; no recording (parity)"
},
},
"capabilities": [
"capabilities": [
"openre.stream.read",
"openre.stream.read",
@@ -77,9 +77,9 @@ cmd_workers() {
@@ -77,9 +77,9 @@ cmd_workers() {
local sha=${1:-$(latest_release)}
local sha=${1:-$(latest_release)}
[ -n "$sha" ] && [ -d "$ROOT/releases/$sha" ] || die "no release $sha"
[ -n "$sha" ] && [ -d "$ROOT/releases/$sha" ] || die "no release $sha"
say "starting worker generation from release $sha (older generations drain by themselves)"
say "starting worker generation from release $sha (older generations drain by themselves)"
- systemctl enable --now "openre-rtmp-ingest@$sha.service" "openre-restream-worker@$sha.service"
+ systemctl enable --now "openre-rtmp-ingest@$sha.service" "openre-restream-worker@$sha.service" "openre-jsmpeg@$sha.service"
# Disable (not stop) older instances so a reboot does not bring them back; they exit on their own.
# Disable (not stop) older instances so a reboot does not bring them back; they exit on their own.
- for unit in $(systemctl list-units --plain --no-legend 'openre-rtmp-ingest@*' 'openre-restream-worker@*' | awk '{print $1}'); do
+ for unit in $(systemctl list-units --plain --no-legend 'openre-rtmp-ingest@*' 'openre-restream-worker@*' 'openre-jsmpeg@*' | awk '{print $1}'); do
case "$unit" in *"@$sha.service") ;; *) systemctl disable "$unit" >/dev/null 2>&1 || true; say "draining: $unit";; esac
case "$unit" in *"@$sha.service") ;; *) systemctl disable "$unit" >/dev/null 2>&1 || true; say "draining: $unit";; esac
done
done
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,37 @@
@@ -0,0 +1,37 @@
+[Unit]
+Description=OpenRe.Stream JSMPEG ingest+relay worker, release %i
+# One instance per release (instance name = the release directory). A worker deploy STARTS a new
+# instance; the coordinator then drains the older generation, which closes its public listener,
+# keeps its sessions until they leave (or OPENRE_DRAIN_MAX_MS), and exits 0 by itself.
+# Nothing here depends on openre-api or openre-session-coordinator: an API deploy never restarts
+# this unit. Never `systemctl restart` it during a broadcast; see README "Deploying".
+After=network.target postgresql.service pgbouncer.service valkey-server.service
+Wants=postgresql.service pgbouncer.service valkey-server.service
+
+[Service]
+Type=simple
+User=ubuntu
+Group=ubuntu
+WorkingDirectory=/opt/openre.stream/releases/%i
+EnvironmentFile=/etc/openvibe/openre.env
+Environment=NODE_ENV=production
+Environment=OPENRE_RELEASE=%i
+Environment=PATH=/usr/local/bin:/usr/bin:/bin
+ExecStart=/usr/bin/env node workers/jsmpeg.js
+# A crash restarts the instance as a new generation; a clean exit (drained) stays stopped.
+Restart=on-failure
+RestartSec=2
+# SIGTERM starts a drain; it does not drop sessions. Only the main process gets it.
+KillSignal=SIGTERM
+KillMode=mixed
+# An explicit stop (or host shutdown) waits this long for sessions to leave, then kills.
+TimeoutStopSec=30min
+LimitNOFILE=65536
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=true
+ReadWritePaths=/var/lib/openre
+
+[Install]
+WantedBy=multi-user.target
@@ -333,6 +333,46 @@ first, then B3 to B6.
@@ -333,6 +333,46 @@ first, then B3 to B6.
and restart Live. Every switched slot then behaves as a Live slot. Its Live key was rotated at the
and restart Live. Every switched slot then behaves as a Live slot. Its Live key was rotated at the
switch, so each broadcaster regenerates on the Go Live page.
switch, so each broadcaster regenerates on the Go Live page.
+## JSMPEG protocol cutover (after RTMP)
+
+JSMPEG reuses the RTMP runbook above, with one **protocol precondition** (brief §4.5): the slot's
+`managed_streams.protocol` must be `jsmpeg` and its OpenRe definition carries the `jsmpeg` protocol.
+Live keeps its own `jsmpeg-relay.js` (9710/9711) until the last JSMPEG slot is cut over.
+
+### J-A. Once, before the first JSMPEG slot (AGENT)
+
+```bash
+# The release that carries workers/jsmpeg.js, deploy/scripts/deploy.sh, systemd/[email protected].
+sudo deploy/scripts/deploy.sh release origin/main
+sudo deploy/scripts/deploy.sh api <sha>
+# Public JSMPEG bind + a new generation. deploy.sh workers now starts openre-jsmpeg@<sha>.service too.
+# OPENRE_JSMPEG_BIND=0.0.0.0 and OPENRE_JSMPEG_PORT=9736 in /etc/openvibe/openre.env
+sudo deploy/scripts/deploy.sh workers <sha>
+```
+
+The **OWNER** opens `OPENRE_JSMPEG_PORT` (9736 until the cutover) at the provider edge; the host
+`ingest.openre.stream` is the DNS-only record RTMP already uses (raw MPEG-TS cannot go through
+Cloudflare's proxy). `openre.stream`/nginx gain the JSMPEG route only when Live's relay is retired.
+
+### J-B1–B6. Per JSMPEG slot
+
+Identical to B1–B6 with these substitutions:
+
+- **B2 Migrate:** `migrate-from-live.js` imports a `jsmpeg` slot as a `jsmpeg` definition (it is no
+ longer refused; the checklist flags it). `pf --only slot --slot <id>` must show the protocol
+ `jsmpeg`.
+- **B3 Switch:** `PUT /api/admin/openre/managed/<id>/ingest-authority {"authority":"openre"}` needs
+ the slot offline, the `jsmpeg` protocol in Live's `OPENRE_PROTOCOLS`, and the JSMPEG worker live.
+- **B4 New key:** the broadcaster regenerates the key, then POSTs MPEG-TS to
+ `http://ingest.openre.stream:9736/<key>/<width>/<height>/` (its ffmpeg command, from the profile
+ route). Viewers connect to the `jsmpeg.ws_url` in the session playback descriptor
+ (`GET /api/v1/sessions/<id>/playback`); the viewer canvas shows the stream.
+- **B5/B6:** unchanged (personal-key rotation; a test broadcast checks the viewer and a restream
+ destination — JSMPEG restreams re-encode the worker's MPEG-TS tap).
+- **B-rollback:** `{"authority":"live"}` + `set-definition-state.js --state disabled`, then the
+ broadcaster points ffmpeg back at Live's relay. JSMPEG has **no recording** (parity with Live): a
+ jsmpeg definition with recording on fails the request with `recording is not available for jsmpeg`.
+
## Phase C: bookkeeping (AGENT, after A2)
## Phase C: bookkeeping (AGENT, after A2)
- **ovhost inventory.** Copy the `openre` entry of OpenVibe.Host's `host.example.json` into
- **ovhost inventory.** Copy the `openre` entry of OpenVibe.Host's `host.example.json` into
@@ -30,11 +30,12 @@
@@ -30,11 +30,12 @@
"openre.output.healthy",
"openre.output.healthy",
"openre.output.failed",
"openre.output.failed",
"openre.recording.requested",
"openre.recording.requested",
+ "openre.recording.failed",
"openre.key.rotated"
"openre.key.rotated"
],
],
"eventsConsumed": [],
"eventsConsumed": [],
"namespacesOwned": [
"namespacesOwned": [
"openre.*"
"openre.*"
],
],
- "notes": "Ingest and restream authority (ADR-009). Alpha: RTMP ingest and RTMP/SRT restream run in transport workers with generations and leases, separate from the API; WHIP, WebRTC/SFU and JSMPEG are still ingested by Live. Runs in parallel with Live; slots move one at a time behind Live's per-slot ingest_authority switch."
+ "notes": "Ingest and restream authority (ADR-009). Alpha: RTMP ingest, JSMPEG ingest+relay, and RTMP/SRT/JSMPEG-source restream run in transport workers with generations and leases, separate from the API; WebRTC (WHIP + SFU) is still ingested by Live. Runs in parallel with Live; slots move one at a time behind Live's per-slot ingest_authority switch."
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,43 @@
@@ -0,0 +1,43 @@
+-- phase: migrate
+-- One protocol name per transport (T4 decision 2): 'whip' is an input alias, stored as 'webrtc', and
+-- the 'webrtc-ingest'/'sfu' worker kinds are one 'webrtc' worker. Rewrite the rows written before the
+-- rename. Never edited after it runs; a further rename is a new migration.
+--
+-- Protocols: map 'whip' to 'webrtc' and drop duplicates, keeping the first occurrence's position.
+-- (Stream definitions are the only place protocols are stored; workers.store validates on write.)
+UPDATE stream_definitions
+SET protocols = (
+ SELECT jsonb_agg(x.p ORDER BY x.ord)::text
+ FROM (
+ SELECT DISTINCT ON (mapped) mapped AS p, ord
+ FROM (
+ SELECT (CASE WHEN v = 'whip' THEN 'webrtc' ELSE v END) AS mapped, ord
+ FROM jsonb_array_elements_text(protocols::jsonb) WITH ORDINALITY AS e(v, ord)
+ ) m
+ ORDER BY mapped, ord
+ ) x
+)
+WHERE protocols LIKE '%"whip"%';
+
+-- Workers: both legacy kinds become 'webrtc'. Their generations were unique only within each old
+-- kind, so they are renumbered into one clean run for UNIQUE (kind, generation). No 'webrtc' row can
+-- exist before this migration (the kind is introduced with it), so the merge cannot collide.
+WITH legacy AS (
+ SELECT id, -(row_number() OVER (ORDER BY generation, started_at, id)) AS tmp
+ FROM workers
+ WHERE kind IN ('webrtc-ingest', 'sfu')
+)
+UPDATE workers w
+SET kind = 'webrtc', generation = legacy.tmp
+FROM legacy
+WHERE w.id = legacy.id;
+
+WITH merged AS (
+ SELECT id, row_number() OVER (ORDER BY generation, started_at, id) AS rn
+ FROM workers
+ WHERE kind = 'webrtc'
+)
+UPDATE workers w
+SET generation = merged.rn
+FROM merged
+WHERE w.id = merged.id;
@@ -83,6 +83,7 @@ function createV1Router({ rt, auth }) {
@@ -83,6 +83,7 @@ function createV1Router({ rt, auth }) {
end_reason: s.end_reason,
end_reason: s.end_reason,
failure_reason: s.failure_reason,
failure_reason: s.failure_reason,
media_info: s.media_info,
media_info: s.media_info,
+ viewers: s.viewers || 0,
revision: s.revision,
revision: s.revision,
created_at: iso(s.created_at),
created_at: iso(s.created_at),
live_at: iso(s.live_at),
live_at: iso(s.live_at),
@@ -22,6 +22,7 @@ function load(env = process.env) {
@@ -22,6 +22,7 @@ function load(env = process.env) {
const isProduction = nodeEnv === 'production';
const isProduction = nodeEnv === 'production';
const port = int(env.PORT, 4500);
const port = int(env.PORT, 4500);
const rtmpPort = int(env.OPENRE_RTMP_PORT, 1936);
const rtmpPort = int(env.OPENRE_RTMP_PORT, 1936);
+ const jsmpegPort = int(env.OPENRE_JSMPEG_PORT, 9736);
return {
return {
service: 'openre',
service: 'openre',
port,
port,
@@ -92,6 +93,22 @@ function load(env = process.env) {
@@ -92,6 +93,22 @@ function load(env = process.env) {
maxPublishersPerWorker: int(env.OPENRE_RTMP_MAX_PUBLISHERS, 64),
maxPublishersPerWorker: int(env.OPENRE_RTMP_MAX_PUBLISHERS, 64),
},
},
+ // ── Ingest (JSMPEG) ─────────────────────────────────────
+ jsmpeg: {
+ // Public port for both the MPEG-TS HTTP POST from the broadcaster's ffmpeg and the
+ // WebSocket viewers. Live's relay owns 9710/9711 until the JSMPEG cutover; OpenRe
+ // listens on 9736 until then, SO_REUSEPORT so two generations can listen during a drain.
+ port: jsmpegPort,
+ bindHost: env.OPENRE_JSMPEG_BIND || '0.0.0.0',
+ publicHost: env.OPENRE_JSMPEG_PUBLIC_HOST || (isProduction ? 'ingest.openre.stream' : '127.0.0.1'),
+ publicPort: int(env.OPENRE_JSMPEG_PUBLIC_PORT, jsmpegPort),
+ // Loopback data tap a generation serves the raw MPEG-TS from (the restream worker reads
+ // it). First free port from this range; never Live's 9710/9711.
+ internalPortMin: int(env.OPENRE_JSMPEG_INTERNAL_PORT_MIN, 19710),
+ internalPortMax: int(env.OPENRE_JSMPEG_INTERNAL_PORT_MAX, 19749),
+ maxPublishersPerWorker: int(env.OPENRE_JSMPEG_MAX_PUBLISHERS, 64),
+ },
+
// ── Workers, leases, generations ────────────────────────
// ── Workers, leases, generations ────────────────────────
workers: {
workers: {
heartbeatMs: int(env.OPENRE_WORKER_HEARTBEAT_MS, 2000),
heartbeatMs: int(env.OPENRE_WORKER_HEARTBEAT_MS, 2000),
@@ -14,6 +14,7 @@
@@ -14,6 +14,7 @@
* openre.output.healthy a restream output confirmed live subject output
* openre.output.healthy a restream output confirmed live subject output
* openre.output.failed an output gave up (circuit breaker) subject output
* openre.output.failed an output gave up (circuit breaker) subject output
* openre.recording.requested Media accepted a recording request subject recording
* openre.recording.requested Media accepted a recording request subject recording
+ * openre.recording.failed a recording could not be started subject recording
* openre.key.rotated an ingest key was rotated subject stream
* openre.key.rotated an ingest key was rotated subject stream
* Payloads never carry an ingest key or a destination key.
* Payloads never carry an ingest key or a destination key.
*/
*/
@@ -28,6 +29,7 @@ const TYPES = Object.freeze({
@@ -28,6 +29,7 @@ const TYPES = Object.freeze({
outputHealthy: 'openre.output.healthy',
outputHealthy: 'openre.output.healthy',
outputFailed: 'openre.output.failed',
outputFailed: 'openre.output.failed',
recordingRequested: 'openre.recording.requested',
recordingRequested: 'openre.recording.requested',
+ recordingFailed: 'openre.recording.failed',
keyRotated: 'openre.key.rotated',
keyRotated: 'openre.key.rotated',
});
});
@@ -22,6 +22,18 @@ async function columns(db, table) {
@@ -22,6 +22,18 @@ async function columns(db, table) {
try { return new Set((await db.prepare(`PRAGMA table_info(${table})`).all()).map(c => c.name)); } catch { return new Set(); }
try { return new Set((await db.prepare(`PRAGMA table_info(${table})`).all()).map(c => c.name)); } catch { return new Set(); }
}
}
+/**
+ * The definition protocols for a Live slot (T4 decision 2): 'whip' is an input alias for 'webrtc',
+ * 'webrtc' and 'jsmpeg' slots import as themselves (they are unadmittable until their worker ships
+ * — the per-slot switch is the real gate), anything else is treated as RTMP.
+ */
+function protocolOf(ms) {
+ const p = String(ms.protocol || '').toLowerCase();
+ if (p === 'whip' || p === 'webrtc') return ['webrtc'];
+ if (p === 'jsmpeg') return ['jsmpeg'];
+ return ['rtmp'];
+}
+
function recordingModeOf(ms, channel) {
function recordingModeOf(ms, channel) {
let vod = !channel ? true : Boolean(channel.vod_recording_enabled) && !channel.force_vod_recording_disabled;
let vod = !channel ? true : Boolean(channel.vod_recording_enabled) && !channel.force_vod_recording_disabled;
if (vod && ms.slot_vod_recording_enabled === 0) vod = false;
if (vod && ms.slot_vod_recording_enabled === 0) vod = false;
@@ -100,7 +112,7 @@ async function migrate({ liveDb, rt, apply = false, onlySlots = null, now = () =
@@ -100,7 +112,7 @@ async function migrate({ liveDb, rt, apply = false, onlySlots = null, now = () =
owner_subject: subject,
owner_subject: subject,
title: ms.title || `${ms.display_name || ms.username}'s stream`,
title: ms.title || `${ms.display_name || ms.username}'s stream`,
description: ms.description || '',
description: ms.description || '',
- protocols: ['rtmp'],
+ protocols: protocolOf(ms),
recording_mode: recordingModeOf(ms, chRow),
recording_mode: recordingModeOf(ms, chRow),
recording_visibility: visibilityOf(ms, chRow),
recording_visibility: visibilityOf(ms, chRow),
mirror_to_live: true,
mirror_to_live: true,
@@ -206,8 +218,10 @@ function checklist(report, { openreUrl = 'https://openre.stream', rtmpUrl } = {}
@@ -206,8 +218,10 @@ function checklist(report, { openreUrl = 'https://openre.stream', rtmpUrl } = {}
lines.push(`### Slot ${s.live_id}${s.slug ? ` "${s.slug}"` : ''}: ${s.title || ''} — ${s.status}${s.definition_id ? ` → ${s.definition_id}` : ''}`);
lines.push(`### Slot ${s.live_id}${s.slug ? ` "${s.slug}"` : ''}: ${s.title || ''} — ${s.status}${s.definition_id ? ` → ${s.definition_id}` : ''}`);
if (s.reason && s.status !== 'imported') lines.push(`- ${s.status}: ${s.reason}`);
if (s.reason && s.status !== 'imported') lines.push(`- ${s.status}: ${s.reason}`);
if (s.status === 'imported') {
if (s.status === 'imported') {
- if (s.protocol !== 'rtmp' && !['rtmp', 'obs'].includes(String(s.streaming_method || '').toLowerCase())) {
- lines.push(`- Slot protocol is ${s.protocol}${s.streaming_method ? ` / ${s.streaming_method}` : ''}: only RTMP moves to OpenRe; leave this slot on Live unless the broadcaster streams it with OBS/RTMP.`);
+ if (s.protocol === 'webrtc' || s.protocol === 'whip') {
+ lines.push(`- Slot protocol is ${s.protocol}${s.streaming_method ? ` / ${s.streaming_method}` : ''}: the webrtc worker is not ported yet; leave this slot on Live until it ships.`);
+ } else if (s.protocol === 'jsmpeg') {
+ lines.push(`- Slot protocol is jsmpeg${s.streaming_method ? ` / ${s.streaming_method}` : ''}: switch it after the JSMPEG worker generation is running (docs/cutover.md, JSMPEG).`);
}
}
lines.push(`- [ ] Maintenance window agreed with @${ch.username} (the slot must be offline when it is switched).`);
lines.push(`- [ ] Maintenance window agreed with @${ch.username} (the slot must be offline when it is switched).`);
lines.push(`- [ ] Destinations reviewed on ${openreUrl}/streams/${s.definition_id || '<id>'}: ${s.destinations.length ? s.destinations.map(d => `${d.name} (${d.platform}) ${d.status}${d.reason ? `: ${d.reason}` : ''}`).join('; ') : 'none'}`);
lines.push(`- [ ] Destinations reviewed on ${openreUrl}/streams/${s.definition_id || '<id>'}: ${s.destinations.length ? s.destinations.map(d => `${d.name} (${d.platform}) ${d.status}${d.reason ? `: ${d.reason}` : ''}`).join('; ') : 'none'}`);
@@ -11,7 +11,10 @@ const { newId, isId } = require('../ids');
@@ -11,7 +11,10 @@ const { newId, isId } = require('../ids');
const { newIngestKey, hashIngestKey, isIngestKeyShape, hintOf } = require('../secrets');
const { newIngestKey, hashIngestKey, isIngestKeyShape, hintOf } = require('../secrets');
const { TYPES } = require('../events');
const { TYPES } = require('../events');
-const PROTOCOLS = Object.freeze(['rtmp', 'whip', 'webrtc', 'jsmpeg']);
+// The definition protocols (T4 decision 2): one name per transport. 'whip' is accepted on input
+// only (API create/update, migrate-from-live) and stored as 'webrtc'.
+const PROTOCOLS = Object.freeze(['rtmp', 'webrtc', 'jsmpeg']);
+const PROTOCOL_ALIASES = Object.freeze({ whip: 'webrtc' });
const RECORDING_MODES = Object.freeze(['vod', 'clips', 'none']);
const RECORDING_MODES = Object.freeze(['vod', 'clips', 'none']);
const VISIBILITIES = Object.freeze(['public', 'unlisted', 'private']);
const VISIBILITIES = Object.freeze(['public', 'unlisted', 'private']);
const REF_SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;
const REF_SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;
@@ -106,7 +109,8 @@ function createDefinitions({ db, config, events, clock }) {
@@ -106,7 +109,8 @@ function createDefinitions({ db, config, events, clock }) {
if (input.title !== undefined || base.title === undefined) out.title = cleanText(input.title, 140, '') || base.title || 'Untitled stream';
if (input.title !== undefined || base.title === undefined) out.title = cleanText(input.title, 140, '') || base.title || 'Untitled stream';
if (input.description !== undefined) out.description = cleanText(input.description, 2000, '');
if (input.description !== undefined) out.description = cleanText(input.description, 2000, '');
if (input.protocols !== undefined) {
if (input.protocols !== undefined) {
- const p = Array.isArray(input.protocols) ? [...new Set(input.protocols.map(String))] : null;
+ const named = Array.isArray(input.protocols) ? input.protocols.map(String).map(x => PROTOCOL_ALIASES[x] || x) : null;
+ const p = named ? [...new Set(named)] : null;
if (!p || !p.length || p.some(x => !PROTOCOLS.includes(x))) throw new StoreError(400, 'openre.invalid_protocols', `protocols must be a non-empty subset of ${PROTOCOLS.join(', ')}`);
if (!p || !p.length || p.some(x => !PROTOCOLS.includes(x))) throw new StoreError(400, 'openre.invalid_protocols', `protocols must be a non-empty subset of ${PROTOCOLS.join(', ')}`);
out.protocols = JSON.stringify(p);
out.protocols = JSON.stringify(p);
}
}
@@ -259,6 +263,15 @@ function createDefinitions({ db, config, events, clock }) {
@@ -259,6 +263,15 @@ function createDefinitions({ db, config, events, clock }) {
key_id: active ? active.id : null,
key_id: active ? active.id : null,
};
};
}
}
+ if (definition.protocols.includes('jsmpeg')) {
+ const j = config.jsmpeg;
+ out.jsmpeg = {
+ // The broadcaster's ffmpeg POSTs MPEG-TS to <url>/<key>/<width>/<height>/.
+ url: `http://${j.publicHost}:${j.publicPort}`,
+ key_hint: active ? active.hint : null,
+ key_id: active ? active.id : null,
+ };
+ }
return out;
return out;
}
}
@@ -14,6 +14,9 @@ const { newId } = require('../ids');
@@ -14,6 +14,9 @@ const { newId } = require('../ids');
const { TYPES } = require('../events');
const { TYPES } = require('../events');
const { isTerminal } = require('../state-machine');
const { isTerminal } = require('../state-machine');
+// JSMPEG has no recording (Live has none either, decision 8: parity). A jsmpeg definition with
+// recording on gets an explicit failure, never a silent no-op.
+const RECORDING_UNSUPPORTED = 'recording is not available for jsmpeg';
const BACKOFF_MS = [5000, 15000, 30000, 60000, 120000, 300000];
const BACKOFF_MS = [5000, 15000, 30000, 60000, 120000, 300000];
const MAX_ATTEMPTS = 10;
const MAX_ATTEMPTS = 10;
// Media refuses a recording while its disk is critically low and frees space within minutes;
// Media refuses a recording while its disk is critically low and frees space within minutes;
@@ -68,6 +71,29 @@ function createRecordings({ db, config, events, clock, definitions, sessions, lo
@@ -68,6 +71,29 @@ function createRecordings({ db, config, events, clock, definitions, sessions, lo
const definition = await definitions.row(session.definition_id);
const definition = await definitions.row(session.definition_id);
const ended = isTerminal(session.state) || session.state === 'ending';
const ended = isTerminal(session.state) || session.state === 'ending';
+ if (session.protocol === 'jsmpeg') {
+ return await db.tx(async () => {
+ const r = await update(rec.id, { state: 'failed', last_error: RECORDING_UNSUPPORTED });
+ await events.enqueue({
+ event_type: TYPES.recordingFailed,
+ actor: { type: 'service', id: 'openre' },
+ subject: { type: 'recording', id: rec.id, revision: 1 },
+ visibility: 'internal',
+ priority: 'important',
+ payload: {
+ recording_id: rec.id,
+ session_id: session.id,
+ stream_id: definition.id,
+ owner: { type: 'user', id: definition.owner_subject },
+ protocol: session.protocol,
+ reason: RECORDING_UNSUPPORTED,
+ external_refs: definition.external_refs,
+ },
+ });
+ return r;
+ });
+ }
+
if (rec.state === 'pending') {
if (rec.state === 'pending') {
if (ended) return await update(rec.id, { state: 'cancelled' });
if (ended) return await update(rec.id, { state: 'cancelled' });
const pb = await sessions.playback(session);
const pb = await sessions.playback(session);
@@ -28,6 +28,9 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
@@ -28,6 +28,9 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
transitionRow: db.prepare('INSERT INTO session_transitions (session_id, from_state, to_state, reason, actor, at) VALUES (?, ?, ?, ?, ?, ?)'),
transitionRow: db.prepare('INSERT INTO session_transitions (session_id, from_state, to_state, reason, actor, at) VALUES (?, ?, ?, ?, ?, ?)'),
transitions: db.prepare('SELECT from_state, to_state, reason, actor, at FROM session_transitions WHERE session_id = ? ORDER BY id'),
transitions: db.prepare('SELECT from_state, to_state, reason, actor, at FROM session_transitions WHERE session_id = ? ORDER BY id'),
mediaInfo: db.prepare(`UPDATE ingest_sessions SET media_info = ? WHERE id = ? AND state IN ${OPEN}`),
mediaInfo: db.prepare(`UPDATE ingest_sessions SET media_info = ? WHERE id = ? AND state IN ${OPEN}`),
+ // Viewer counts live inside media_info so no column is needed: merge, never replace, so the
+ // width/height the JSMPEG ingest path reported survive.
+ viewers: db.prepare(`UPDATE ingest_sessions SET media_info = jsonb_set(COALESCE(NULLIF(media_info, '')::jsonb, '{}'::jsonb), '{viewers}', to_jsonb(?::bigint), true)::text WHERE id = ? AND state IN ${OPEN}`),
requestEnd: db.prepare(`UPDATE ingest_sessions SET desired_state = 'end', end_requested_by = ?, updated_at = ? WHERE id = ? AND state IN ${OPEN}`),
requestEnd: db.prepare(`UPDATE ingest_sessions SET desired_state = 'end', end_requested_by = ?, updated_at = ? WHERE id = ? AND state IN ${OPEN}`),
endRequests: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND desired_state = 'end' AND state IN ${OPEN}`),
endRequests: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND desired_state = 'end' AND state IN ${OPEN}`),
ofWorker: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND state IN ${OPEN}`),
ofWorker: db.prepare(`SELECT * FROM ingest_sessions WHERE worker_id = ? AND state IN ${OPEN}`),
@@ -46,7 +49,8 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
@@ -46,7 +49,8 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
function shape(s) {
function shape(s) {
if (!s) return null;
if (!s) return null;
- return { ...s, media_info: parseJson(s.media_info, null) };
+ const media_info = parseJson(s.media_info, null);
+ return { ...s, media_info, viewers: (media_info && Number(media_info.viewers)) || 0 };
}
}
async function get(id) {
async function get(id) {
@@ -200,8 +204,20 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
@@ -200,8 +204,20 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
flv: null,
flv: null,
rtmp: null,
rtmp: null,
webrtc: null,
webrtc: null,
+ jsmpeg: null,
hls: null,
hls: null,
};
};
+ if (session.protocol === 'jsmpeg' && ep.publicPort) {
+ const mi = session.media_info || {};
+ descriptor.jsmpeg = {
+ // Viewers connect over WebSocket to the worker that holds the session, by playback
+ // id (never the key). tap_internal_url is the loopback MPEG-TS feed restream reads.
+ ws_url: `ws://${config.jsmpeg.publicHost}:${ep.publicPort}/${session.id}`,
+ width: mi.width || null,
+ height: mi.height || null,
+ tap_internal_url: ep.tapPort ? `http://127.0.0.1:${ep.tapPort}/tap/${session.id}.ts` : null,
+ };
+ }
if (session.protocol === 'rtmp' && ep.flvPort) {
if (session.protocol === 'rtmp' && ep.flvPort) {
descriptor.flv = {
descriptor.flv = {
internal_url: `http://127.0.0.1:${ep.flvPort}/live/${session.id}.flv`,
internal_url: `http://127.0.0.1:${ep.flvPort}/live/${session.id}.flv`,
@@ -225,6 +241,7 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
@@ -225,6 +241,7 @@ function createSessions({ db, config, events, clock, definitions, workers }) {
requestEnd,
requestEnd,
playback,
playback,
setMediaInfo: async (id, info) => (await q.mediaInfo.run(JSON.stringify(info), id)).changes > 0,
setMediaInfo: async (id, info) => (await q.mediaInfo.run(JSON.stringify(info), id)).changes > 0,
+ setViewers: async (id, count) => (await q.viewers.run(Math.max(0, Number(count) || 0), id)).changes > 0,
transitions: async (id) => await q.transitions.all(id),
transitions: async (id) => await q.transitions.all(id),
endRequestsFor: async (workerId) => (await q.endRequests.all(workerId)).map(shape),
endRequestsFor: async (workerId) => (await q.endRequests.all(workerId)).map(shape),
ofWorker: async (workerId) => (await q.ofWorker.all(workerId)).map(shape),
ofWorker: async (workerId) => (await q.ofWorker.all(workerId)).map(shape),
@@ -18,7 +18,9 @@ const os = require('os');
@@ -18,7 +18,9 @@ const os = require('os');
const { newId } = require('../ids');
const { newId } = require('../ids');
const { parseJson } = require('./definitions');
const { parseJson } = require('./definitions');
-const KINDS = Object.freeze(['rtmp-ingest', 'restream', 'webrtc-ingest', 'sfu', 'jsmpeg']);
+// T4 decision 2: one kind per transport. The 'webrtc-ingest' and 'sfu' kinds collapsed into the one
+// 'webrtc' worker (migration 0002 rewrites any stored rows).
+const KINDS = Object.freeze(['rtmp-ingest', 'restream', 'webrtc', 'jsmpeg']);
const ALIVE = "('starting', 'ready', 'draining')";
const ALIVE = "('starting', 'ready', 'draining')";
function createWorkers({ db, config, clock }) {
function createWorkers({ db, config, clock }) {
@@ -12,6 +12,7 @@ const { createCoordinator } = require('../server/coordinator');
@@ -12,6 +12,7 @@ const { createCoordinator } = require('../server/coordinator');
const { createRtmpIngest } = require('../workers/rtmp-ingest');
const { createRtmpIngest } = require('../workers/rtmp-ingest');
const { createRestreamWorker } = require('../workers/restream-worker');
const { createRestreamWorker } = require('../workers/restream-worker');
const { createUnportedTransport } = require('../workers/unported-transport');
const { createUnportedTransport } = require('../workers/unported-transport');
+const { createJsmpeg } = require('../workers/jsmpeg');
const { openDb } = require('../server/db');
const { openDb } = require('../server/db');
const { runtime, bootApi, request, serviceToken, tmpDir, testEnv, child, suite, silent, testDb } = require('./helpers');
const { runtime, bootApi, request, serviceToken, tmpDir, testEnv, child, suite, silent, testDb } = require('./helpers');
@@ -45,13 +46,14 @@ t('the coordinator and every transport worker refuse to be created in a drill',
@@ -45,13 +46,14 @@ t('the coordinator and every transport worker refuse to be created in a drill',
drill(() => createCoordinator({ rt, media: null, log: silent }));
drill(() => createCoordinator({ rt, media: null, log: silent }));
drill(() => createRtmpIngest({ rt, log: silent, exit: () => {} }));
drill(() => createRtmpIngest({ rt, log: silent, exit: () => {} }));
drill(() => createRestreamWorker({ rt, log: silent, exit: () => {} }));
drill(() => createRestreamWorker({ rt, log: silent, exit: () => {} }));
- for (const kind of ['webrtc-ingest', 'sfu', 'jsmpeg']) drill(() => createUnportedTransport({ rt, kind, log: silent, exit: () => {} }));
+ drill(() => createUnportedTransport({ rt, kind: 'webrtc', log: silent, exit: () => {} }));
+ drill(() => createJsmpeg({ rt, log: silent, exit: () => {} }));
assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM workers').get()).n, 0, 'no generation registered');
assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM workers').get()).n, 0, 'no generation registered');
assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM leases').get()).n, 0, 'no coordinator lease taken');
assert.strictEqual((await rt.db.prepare('SELECT count(*) AS n FROM leases').get()).n, 0, 'no coordinator lease taken');
});
});
t('the entry points exit before opening the database', async () => {
t('the entry points exit before opening the database', async () => {
- for (const script of ['workers/coordinator.js', 'workers/rtmp-ingest.js', 'workers/restream-worker.js', 'workers/sfu.js']) {
+ for (const script of ['workers/coordinator.js', 'workers/rtmp-ingest.js', 'workers/restream-worker.js', 'workers/webrtc.js', 'workers/jsmpeg.js']) {
const dir = tmpDir();
const dir = tmpDir();
const p = child(script, await testEnv(dir, { ...PROD_LIKE, OPENRE_DRILL: '1' }));
const p = child(script, await testEnv(dir, { ...PROD_LIKE, OPENRE_DRILL: '1' }));
const r = await p.exited;
const r = await p.exited;
new file mode 100644
new file mode 100644
@@ -0,0 +1,268 @@
@@ -0,0 +1,268 @@
+'use strict';
+// End to end with real processes and real ffmpeg: openre-api, openre-jsmpeg, openre-restream-worker
+// and openre-session-coordinator as separate child processes, plus a stub OpenVibe.Network, a stub
+// OpenVibe.Events and a stub OpenVibe.Media.
+//
+// Proves: the broadcaster's ffmpeg POSTs MPEG-TS to /<key>/<w>/<h>/ and the session goes live on the
+// newest generation (wrong key and wrong protocol refused), a WebSocket viewer receives the bytes and
+// is counted in the session heartbeat, the loopback data tap restreams to a working destination,
+// recording is refused with a clear reason, a duplicate publisher is refused, and a second generation
+// takes new sessions while the first drains, keeps its session and exits when it ends; a killed worker
+// fails its session after the lease expires.
+const assert = require('assert');
+const http = require('http');
+const { spawn } = require('child_process');
+const WebSocket = require('ws');
+const {
+ tmpDir, testEnv, multiProcess, child, waitFor, request, userToken, freePort, sleep, suite, hasFfmpeg, rtmpSink, OWNER,
+} = require('./helpers');
+
+if (!hasFfmpeg()) {
+ console.log('jsmpeg-e2e: skipped (no ffmpeg on PATH)');
+ process.exit(0);
+}
+if (!multiProcess()) {
+ console.log('jsmpeg-e2e: skipped (the OpenRe processes share one database: PostgreSQL only, npm run test:pg)');
+ process.exit(0);
+}
+const t = suite('jsmpeg-e2e');
+const token = userToken({ subjectId: OWNER });
+const procs = [];
+const mediaCalls = [];
+let env;
+let api;
+let coordinator;
+let base;
+let jsmpegPort;
+let ingest1;
+let stubs = [];
+
+function stubServer(handler) {
+ return new Promise((resolve) => {
+ const s = http.createServer((req, res) => {
+ let body = '';
+ req.on('data', (c) => { body += c; });
+ req.on('end', () => handler(req, res, body));
+ });
+ s.listen(0, '127.0.0.1', () => { stubs.push(s); resolve(`http://127.0.0.1:${s.address().port}`); });
+ });
+}
+const json = (res, status, obj) => { res.statusCode = status; res.setHeader('Content-Type', 'application/json'); res.end(JSON.stringify(obj)); };
+
+function startChild(script, extra = {}) {
+ const p = child(script, { ...env, ...extra });
+ procs.push(p);
+ return p;
+}
+async function startApi() {
+ const p = startChild('server/index.js');
+ await waitFor(async () => { try { return (await request(base, 'GET', '/api/ready')).status === 200; } catch { return false; } }, { what: 'api ready' });
+ return p;
+}
+
+/** ffmpeg POSTs MPEG-TS to http://127.0.0.1:<port>/<key>/<w>/<h>/ — Live's JSMPEG broadcaster command. */
+function publishJsmpeg(port, key, w, h, { seconds } = {}) {
+ const url = `http://127.0.0.1:${port}/${key}/${w}/${h}/`;
+ const args = ['-hide_banner', '-loglevel', 'error', '-re', '-f', 'lavfi', '-i', 'testsrc=size=320x240:rate=30',
+ '-c:v', 'mpeg1video', '-b:v', '600k', '-r', '30', '-f', 'mpegts'];
+ if (seconds) args.push('-t', String(seconds));
+ args.push(url);
+ const p = spawn('ffmpeg', args, { stdio: ['ignore', 'ignore', 'pipe'] });
+ p.err = '';
+ p.stderr.on('data', (c) => { p.err += c; });
+ p.exited = new Promise(r => p.on('exit', (code, signal) => r({ code, signal })));
+ return p;
+}
+
+async function openSession(streamId, state = 'live') {
+ const r = await request(base, 'GET', `/api/v1/sessions?stream_id=${streamId}`, { token });
+ return (r.body.sessions || []).find(s => s.state === state) || null;
+}
+async function detail(id) {
+ const r = await request(base, 'GET', `/api/v1/sessions/${id}`, { token });
+ return r.body.session;
+}
+
+let streamA;
+let keyA;
+let sessionA;
+let pubA;
+let sink;
+let viewer;
+
+t('setup: stubs and the four OpenRe processes', async () => {
+ const network = await stubServer((req, res, body) => {
+ if (req.url === '/oauth/token') {
+ const p = new URLSearchParams(body);
+ return json(res, 200, { access_token: `tok-${p.get('audience')}`, token_type: 'Bearer', expires_in: 300 });
+ }
+ return json(res, 404, {});
+ });
+ const events = await stubServer((req, res, body) => {
+ if (req.url === '/api/v1/events' && req.method === 'POST') return json(res, 201, { event_id: 'e', seq: 1, duplicate: false });
+ return json(res, 404, {});
+ });
+ const media = await stubServer((req, res, body) => {
+ mediaCalls.push({ method: req.method, url: req.url, body: body ? JSON.parse(body) : null });
+ return json(res, 404, { error: 'Media must not be called for a jsmpeg recording' });
+ });
+ jsmpegPort = await freePort();
+ const dir = tmpDir();
+ env = await testEnv(dir, {
+ PORT: String(await freePort()),
+ OPENRE_JSMPEG_PORT: String(jsmpegPort),
+ OPENRE_JSMPEG_BIND: '127.0.0.1',
+ OPENRE_JSMPEG_INTERNAL_PORT_MIN: String(22000 + Math.floor(Math.random() * 20000)),
+ OV_NETWORK_INTERNAL_URL: network,
+ OV_OAUTH_CLIENT_SECRET: 'test-secret',
+ EVENTS_URL: events,
+ EVENTS_RELAY_INTERVAL_MS: '200',
+ MEDIA_URL: media,
+ MEDIA_API_KEY: 'media-app-key',
+ OPENRE_RECORDING_START_DELAY_MS: '200',
+ OPENRE_WORKER_HEARTBEAT_MS: '250',
+ OPENRE_WORKER_LEASE_MS: '4000',
+ OPENRE_COORDINATOR_INTERVAL_MS: '250',
+ OPENRE_RESTREAM_POLL_MS: '250',
+ OPENRE_OUTPUT_START_DELAY_MS: '500',
+ OPENRE_RESTART_BASE_MS: '200',
+ OPENRE_MAX_RESTARTS: '3',
+ });
+ env.OPENRE_JSMPEG_INTERNAL_PORT_MAX = String(Number(env.OPENRE_JSMPEG_INTERNAL_PORT_MIN) + 20);
+ base = `http://127.0.0.1:${env.PORT}`;
+ coordinator = startChild('workers/coordinator.js');
+ ingest1 = startChild('workers/jsmpeg.js');
+ startChild('workers/restream-worker.js');
+ api = await startApi();
+ await waitFor(() => /generation 1 ready: ingest\+ws/.test(ingest1.output), { what: 'jsmpeg worker ready' });
+});
+
+t('a bad key and a wrong-protocol key are refused', async () => {
+ const bad = publishJsmpeg(jsmpegPort, `ork_${'x'.repeat(43)}`, 320, 240, { seconds: 2 });
+ procs.push(bad);
+ assert.notStrictEqual((await bad.exited).code, 0, 'ffmpeg with a wrong key fails');
+
+ // A definition that does not list jsmpeg: resolveIngestKey answers protocol_not_allowed.
+ const r = await request(base, 'POST', '/api/v1/streams', { token, body: { title: 'RTMP only', protocols: ['rtmp'] } });
+ assert.strictEqual(r.status, 201);
+ const wrong = publishJsmpeg(jsmpegPort, r.body.key.key, 320, 240, { seconds: 2 });
+ procs.push(wrong);
+ assert.notStrictEqual((await wrong.exited).code, 0, 'an rtmp-only definition is refused on the jsmpeg listener');
+});
+
+t('the ffmpeg MPEG-TS POST goes live, a WS viewer receives bytes and is counted', async () => {
+ const r = await request(base, 'POST', '/api/v1/streams', {
+ token,
+ body: { title: 'JSMPEG', protocols: ['jsmpeg'], recording_mode: 'vod', external_refs: [{ service: 'live', type: 'managed_stream', id: '9' }, { service: 'live', type: 'user', id: '42' }] },
+ });
+ assert.strictEqual(r.status, 201);
+ streamA = r.body.stream.id;
+ keyA = r.body.key.key;
+ const sinkPort = await freePort();
+ sink = rtmpSink(sinkPort);
+ procs.push(sink);
+ assert.strictEqual((await request(base, 'POST', `/api/v1/streams/${streamA}/destinations`, { token, body: { platform: 'custom', name: 'sink', server_url: `rtmp://127.0.0.1:${sinkPort}/app`, stream_key: 'sinkkey' } })).status, 201);
+
+ pubA = publishJsmpeg(jsmpegPort, keyA, 320, 240);
+ procs.push(pubA);
+ sessionA = await waitFor(async () => await openSession(streamA), { what: 'session A live', timeoutMs: 20000 });
+ assert.strictEqual(sessionA.worker.generation, 1);
+
+ const d = await detail(sessionA.id);
+ assert.deepStrictEqual({ w: d.playback.jsmpeg.width, h: d.playback.jsmpeg.height }, { w: 320, h: 240 });
+ assert.ok(!d.playback.jsmpeg.ws_url.includes(keyA), 'the key is never in the viewer URL');
+
+ viewer = new WebSocket(d.playback.jsmpeg.ws_url);
+ viewer.frames = [];
+ viewer.on('message', (m) => viewer.frames.push(m));
+ await new Promise((res, rej) => { viewer.once('open', res); viewer.once('error', rej); });
+ await waitFor(() => viewer.frames.length > 0, { what: 'viewer bytes', timeoutMs: 10000 });
+ assert.ok(viewer.frames[0].length > 0);
+
+ const v = await waitFor(async () => { const s = await detail(sessionA.id); return s.viewers >= 1 ? s : null; }, { what: 'viewer counted', timeoutMs: 10000 });
+ assert.ok(v.viewers >= 1);
+});
+
+t('the JSMPEG data tap restreams to a destination', async () => {
+ const outputs = await waitFor(async () => {
+ const r = await request(base, 'GET', `/api/v1/sessions/${sessionA.id}/outputs`, { token });
+ const o = r.body.outputs || [];
+ return o.some(x => x.state === 'live') ? o : null;
+ }, { what: 'restream live', timeoutMs: 45000 });
+ assert.ok(outputs.some(o => o.state === 'live'));
+});
+
+t('recording is refused with a clear reason, without calling Media', async () => {
+ const s = await waitFor(async () => { const d = await detail(sessionA.id); return d.recording && d.recording.state === 'failed' ? d : null; }, { what: 'recording failed', timeoutMs: 10000 });
+ assert.strictEqual(s.recording.last_error, 'recording is not available for jsmpeg');
+ assert.strictEqual(mediaCalls.length, 0, 'Media was never asked to record a jsmpeg session');
+});
+
+t('a second publisher with the same key is refused while the first is live', async () => {
+ const dup = publishJsmpeg(jsmpegPort, keyA, 320, 240, { seconds: 2 });
+ procs.push(dup);
+ assert.notStrictEqual((await dup.exited).code, 0);
+ assert.strictEqual((await openSession(streamA)).id, sessionA.id, 'the source session is unaffected');
+});
+
+let streamB;
+let pubB;
+let ingest2;
+
+t('a new jsmpeg generation takes new sessions; generation 1 drains but keeps A', async () => {
+ ingest2 = startChild('workers/jsmpeg.js');
+ await waitFor(() => /generation 2 ready: ingest\+ws/.test(ingest2.output), { what: 'generation 2 ready' });
+ await waitFor(() => /draining/.test(ingest1.output) && /public listener closed/.test(ingest1.output), { what: 'generation 1 draining', timeoutMs: 10000 });
+
+ const r = await request(base, 'POST', '/api/v1/streams', { token, body: { title: 'B', protocols: ['jsmpeg'], recording_mode: 'none' } });
+ streamB = r.body.stream.id;
+ pubB = publishJsmpeg(jsmpegPort, r.body.key.key, 320, 240);
+ procs.push(pubB);
+ const sB = await waitFor(async () => await openSession(streamB), { what: 'session B live', timeoutMs: 20000 });
+ assert.strictEqual(sB.worker.generation, 2, 'new sessions reach the newest generation');
+ const sA = await openSession(streamA);
+ assert.strictEqual(sA.id, sessionA.id);
+ assert.strictEqual(sA.worker.generation, 1, 'the running session stays on its generation');
+ assert.strictEqual(ingest1.exitCode, null);
+});
+
+t('when session A ends, generation 1 exits by itself; session B is unaffected', async () => {
+ pubA.kill('SIGKILL');
+ const end = await waitFor(async () => (await detail(sessionA.id)).state === 'ended', { what: 'session A ended', timeoutMs: 30000 }).then(() => true, () => false);
+ assert.ok(end, `session A did not end: ${JSON.stringify(await detail(sessionA.id))}`);
+ const exit = await Promise.race([ingest1.exited, sleep(10000).then(() => ({ code: 'timeout' }))]);
+ assert.strictEqual(exit.code, 0, 'the drained generation exited cleanly');
+ const idB = (await openSession(streamB)).id;
+ assert.strictEqual((await detail(idB)).state, 'live');
+});
+
+t('a killed worker fails its session after the lease expires', async () => {
+ ingest2.kill('SIGKILL');
+ await ingest2.exited;
+ const idB = (await openSession(streamB)).id;
+ const s = await waitFor(async () => {
+ const d = await detail(idB);
+ return d.state === 'failed' ? d : null;
+ }, { what: 'session B failed after lease loss', timeoutMs: 20000 });
+ assert.match(s.failure_reason, /lease|lost|expired/i);
+});
+
+t('no OpenRe process crashed along the way, and no key reached a log', () => {
+ assert.strictEqual(coordinator.exitCode, null, `coordinator exited:\n${coordinator.output}`);
+ assert.strictEqual(api.exitCode, null, `api exited:\n${api.output}`);
+ for (const p of procs) {
+ if (!p.output) continue;
+ assert.ok(!p.output.includes(keyA), 'no ingest key in logs');
+ assert.ok(!p.output.includes('sinkkey'), 'no destination key in logs');
+ }
+});
+
+t('teardown', async () => {
+ if (process.exitCode || process.env.DEBUG) for (const p of procs) if (p.output) console.log(p.output);
+ if (viewer) { try { viewer.close(); } catch { /* */ } }
+ for (const p of procs) { try { p.kill('SIGKILL'); } catch { /* */ } }
+ for (const s of stubs) { s.closeAllConnections?.(); s.close(); }
+ stubs = [];
+});
+
+t.run();
new file mode 100644
new file mode 100644
@@ -0,0 +1,44 @@
@@ -0,0 +1,44 @@
+'use strict';
+// The 0002 naming migration, run for real: a fresh database is built from migrations/0001_initial.sql,
+// legacy rows ('whip' protocols, 'webrtc-ingest'/'sfu' workers) are inserted, the 0002 SQL is applied,
+// and the stored rows are asserted rewritten (protocols deduplicated, worker kinds merged and
+// renumbered so UNIQUE (kind, generation) holds).
+const assert = require('assert');
+const fs = require('fs');
+const path = require('path');
+const { createDb } = require('openvibe-sdk/db');
+const { suite, ROOT } = require('./helpers');
+
+const MIG1 = path.join(ROOT, 'migrations', '0001_initial.sql');
+const MIG2 = path.join(ROOT, 'migrations', '0002_protocol_names.sql');
+
+const t = suite('jsmpeg-migration');
+const log = { log() {}, warn() {}, error: console.error };
+
+t('stored whip protocols become webrtc (deduplicated); webrtc-ingest/sfu worker rows become webrtc', async () => {
+ const db = createDb({ pglite: true, service: 'jsmpeg-migration', log });
+ try {
+ await db.exec(fs.readFileSync(MIG1, 'utf8'));
+ await db.exec(`INSERT INTO stream_definitions (id, owner_subject, protocols, created_at, updated_at) VALUES
+ ('stream_a', 'usr_01J0000000000000000000000A', '["whip","webrtc","rtmp"]', 1, 1),
+ ('stream_b', 'usr_01J0000000000000000000000A', '["whip","whip"]', 1, 1),
+ ('stream_c', 'usr_01J0000000000000000000000A', '["rtmp","jsmpeg"]', 1, 1)`);
+ await db.exec(`INSERT INTO workers (id, kind, generation, state, started_at, heartbeat_at) VALUES
+ ('w_wi1','webrtc-ingest',1,'stopped',1,1),
+ ('w_wi2','webrtc-ingest',2,'stopped',2,2),
+ ('w_sfu','sfu',1,'stopped',3,3)`);
+ await db.exec(fs.readFileSync(MIG2, 'utf8'));
+
+ const defs = Object.fromEntries((await db.prepare('SELECT id, protocols FROM stream_definitions').all()).map(r => [r.id, r.protocols]));
+ assert.deepStrictEqual(JSON.parse(defs.stream_a), ['webrtc', 'rtmp']);
+ assert.deepStrictEqual(JSON.parse(defs.stream_b), ['webrtc']);
+ assert.deepStrictEqual(JSON.parse(defs.stream_c), ['rtmp', 'jsmpeg'], 'a row without whip is untouched');
+ const workers = await db.prepare('SELECT kind, generation FROM workers ORDER BY generation').all();
+ assert.ok(workers.every(w => w.kind === 'webrtc'), 'every legacy kind is webrtc');
+ assert.deepStrictEqual(workers.map(w => w.generation).sort((a, b) => a - b), [1, 2, 3], 'generations stay unique');
+ } finally {
+ await db.close();
+ }
+});
+
+t.run();
new file mode 100644
new file mode 100644
@@ -0,0 +1,74 @@
@@ -0,0 +1,74 @@
+'use strict';
+// T4 naming + the JSMPEG worker's store-level behaviour, without ffmpeg or PostgreSQL: 'whip' is
+// an input alias for 'webrtc'; the worker kinds are one per transport; a jsmpeg session's playback
+// descriptor is { jsmpeg: { ws_url, width, height } } (plus the loopback tap) and its viewers are
+// reported; recording for jsmpeg is refused with a clear reason and an event.
+const assert = require('assert');
+const { runtime, tmpDir, suite, OWNER, outboxEnvelopes } = require('./helpers');
+
+const t = suite('jsmpeg');
+
+t('naming: whip is an input alias, and the worker kinds are one per transport', async () => {
+ const rt = await runtime({ dir: tmpDir() });
+ assert.deepStrictEqual(rt.store.workers.KINDS, ['rtmp-ingest', 'restream', 'webrtc', 'jsmpeg']);
+
+ const a = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['whip'] });
+ assert.deepStrictEqual(a.definition.protocols, ['webrtc'], 'whip is stored as webrtc');
+ const b = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['whip', 'webrtc', 'jsmpeg'] });
+ assert.deepStrictEqual(b.definition.protocols, ['webrtc', 'jsmpeg'], 'deduplicated after aliasing');
+ await assert.rejects(() => rt.store.definitions.create({ owner_subject: OWNER, protocols: ['sfu'] }), /protocols must be/);
+
+ await assert.rejects(() => rt.store.workers.register({ kind: 'webrtc-ingest' }), /unknown worker kind/);
+ await assert.rejects(() => rt.store.workers.register({ kind: 'sfu' }), /unknown worker kind/);
+ const w = await rt.store.workers.register({ kind: 'webrtc', endpoints: {} });
+ assert.strictEqual(w.kind, 'webrtc');
+ rt.db.close();
+});
+
+t('a jsmpeg session playback descriptor: ws_url by playback id, width/height, the tap, viewers', async () => {
+ const rt = await runtime({ dir: tmpDir() });
+ const { definition, key } = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['jsmpeg'] });
+ const w = await rt.store.workers.register({ kind: 'jsmpeg', endpoints: { publicPort: 9736, publicPorts: [9736], tapPort: 19710 } });
+ await rt.store.workers.ready(w.id);
+
+ const a = await rt.store.sessions.admit({ definition, key, protocol: 'jsmpeg', worker: await rt.store.workers.get(w.id) });
+ await rt.store.sessions.setMediaInfo(a.session.id, { width: 640, height: 360 });
+ await rt.store.sessions.transition(a.session.id, 'live', { reason: 'media_flowing', actor: 'worker:w' });
+ await rt.store.sessions.setViewers(a.session.id, 3);
+
+ const s = await rt.store.sessions.get(a.session.id);
+ assert.strictEqual(s.viewers, 3, 'the heartbeat viewer count is on the session');
+ const pb = await rt.store.sessions.playback(s);
+ assert.deepStrictEqual(Object.keys(pb.jsmpeg).sort(), ['height', 'tap_internal_url', 'width', 'ws_url']);
+ assert.strictEqual(pb.jsmpeg.ws_url, `ws://127.0.0.1:9736/${a.session.id}`);
+ assert.ok(!pb.jsmpeg.ws_url.includes(key.key), 'the key is never in the viewer URL');
+ assert.strictEqual(pb.jsmpeg.width, 640);
+ assert.strictEqual(pb.jsmpeg.height, 360);
+ assert.strictEqual(pb.jsmpeg.tap_internal_url, `http://127.0.0.1:19710/tap/${a.session.id}.ts`);
+ rt.db.close();
+});
+
+t('a jsmpeg session with recording on is refused with a clear reason and an event', async () => {
+ const rt = await runtime({ dir: tmpDir(), env: { OPENRE_RECORDING_START_DELAY_MS: '0' } });
+ const { definition } = await rt.store.definitions.create({ owner_subject: OWNER, protocols: ['jsmpeg'], recording_mode: 'vod' });
+ const w = await rt.store.workers.register({ kind: 'jsmpeg', endpoints: { publicPort: 9736, tapPort: 19710 } });
+ await rt.store.workers.ready(w.id);
+ const a = await rt.store.sessions.admit({ definition, key: null, protocol: 'jsmpeg', worker: await rt.store.workers.get(w.id) });
+ await rt.store.sessions.transition(a.session.id, 'live', { reason: 'media_flowing', actor: 'worker:w' });
+
+ const never = () => { throw new Error('Media must not be called for a jsmpeg recording'); };
+ const media = { configured: true, createVod: never, ingestRtmp: never, finalizeVod: never, deleteVod: never };
+ assert.strictEqual(await rt.store.recordings.ensureRequests(), 1);
+ await rt.store.recordings.process(media);
+
+ const rec = await rt.store.recordings.bySession(a.session.id);
+ assert.strictEqual(rec.state, 'failed');
+ assert.strictEqual(rec.last_error, 'recording is not available for jsmpeg');
+ const ev = (await outboxEnvelopes(rt.db)).find(e => e.event_type === 'openre.recording.failed');
+ assert.ok(ev, 'a recording.failed event is emitted');
+ assert.strictEqual(ev.payload.reason, 'recording is not available for jsmpeg');
+ assert.strictEqual(ev.payload.protocol, 'jsmpeg');
+ rt.db.close();
+});
+
+t.run();
@@ -29,7 +29,8 @@ t('unknown, malformed and wrong-protocol keys are refused', async () => {
@@ -29,7 +29,8 @@ t('unknown, malformed and wrong-protocol keys are refused', async () => {
const { key } = await rt.store.definitions.create({ owner_subject: OWNER });
const { key } = await rt.store.definitions.create({ owner_subject: OWNER });
assert.strictEqual((await rt.store.definitions.resolveIngestKey('ork_' + 'A'.repeat(43), 'rtmp')).error, 'unknown_key');
assert.strictEqual((await rt.store.definitions.resolveIngestKey('ork_' + 'A'.repeat(43), 'rtmp')).error, 'unknown_key');
assert.strictEqual((await rt.store.definitions.resolveIngestKey('a3f9c2', 'rtmp')).error, 'malformed_key');
assert.strictEqual((await rt.store.definitions.resolveIngestKey('a3f9c2', 'rtmp')).error, 'malformed_key');
- assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'whip')).error, 'protocol_not_allowed');
+ assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'webrtc')).error, 'protocol_not_allowed');
+ assert.strictEqual((await rt.store.definitions.resolveIngestKey(key.key, 'jsmpeg')).error, 'protocol_not_allowed');
});
});
t('rotate revokes the old key immediately and emits openre.key.rotated without any key', async () => {
t('rotate revokes the old key immediately and emits openre.key.rotated without any key', async () => {
@@ -85,6 +85,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held or
@@ -85,6 +85,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held or
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:13'].target_id)).recording_mode, 'none', 'VOD off on the channel, clips off on the slot');
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:13'].target_id)).recording_mode, 'none', 'VOD off on the channel, clips off on the slot');
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_mode, 'clips');
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_mode, 'clips');
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_visibility, 'private');
assert.strictEqual((await rt.store.definitions.get(map['managed_stream:14'].target_id)).recording_visibility, 'private');
+ assert.deepStrictEqual((await rt.store.definitions.get(map['managed_stream:13'].target_id)).protocols, ['webrtc'], 'a webrtc slot imports as a webrtc definition (unadmittable until the worker ships)');
// Old keys: none of them authenticates, and none is stored in any form.
// Old keys: none of them authenticates, and none is stored in any form.
for (const old of ['a3f9c2leakedkey0000000000000000000000000', 'personalkey111']) {
for (const old of ['a3f9c2leakedkey0000000000000000000000000', 'personalkey111']) {
@@ -114,7 +115,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held or
@@ -114,7 +115,7 @@ t('apply: slots with a subject are imported with new keys; the rest are held or
assert.match(md, /## @japaneseoldguy/);
assert.match(md, /## @japaneseoldguy/);
assert.match(md, /ingest-authority/);
assert.match(md, /ingest-authority/);
assert.match(md, /OAuth-linked/);
assert.match(md, /OAuth-linked/);
- assert.match(md, /only RTMP moves to OpenRe/, 'the WebRTC slot is flagged');
+ assert.match(md, /webrtc worker is not ported/, 'the WebRTC slot is flagged');
for (const secret of ['a3f9c2leakedkey', 'live_twitchsecret_1', 'yt-secret-2222', 'personalkey111', 'kick-secret-44']) assert.ok(!md.includes(secret), `no ${secret} in the checklist`);
for (const secret of ['a3f9c2leakedkey', 'live_twitchsecret_1', 'yt-secret-2222', 'personalkey111', 'kick-secret-44']) assert.ok(!md.includes(secret), `no ${secret} in the checklist`);
});
});
@@ -1,3 +1,317 @@
@@ -1,3 +1,317 @@
'use strict';
'use strict';
-/** openre-jsmpeg: worker interface only, the transport is not ported yet (see unported-transport.js). */
-if (require.main === module) require('./unported-transport').main('jsmpeg').catch((err) => { console.error(`[jsmpeg] failed to start: ${err && err.stack || err}`); process.exit(1); });
+/**
+ * openre-jsmpeg — the JSMPEG transport worker (unit openre-jsmpeg@<release>.service).
+ *
+ * The relay Live runs in-process (OpenVibe.Live server/streaming/jsmpeg-relay.js), with the
+ * lifecycle and ownership moved here:
+ *
+ * public 0.0.0.0:OPENRE_JSMPEG_PORT (9736 until the JSMPEG cutover), SO_REUSEPORT so two
+ * generations can listen during a drain. The broadcaster's ffmpeg POSTs MPEG-TS to
+ * http://<host>:<port>/<key>/<w>/<h>/, and viewers connect over WebSocket to
+ * ws://<host>:<port>/<session playback id> (never the key).
+ * internal 127.0.0.1:<tapPort> GET /tap/<session>.ts streams the raw MPEG-TS for restream.
+ * The port is this generation's own (from OPENRE_JSMPEG_INTERNAL_PORT_MIN..MAX) and is
+ * published in its worker row, so consumers always reach the process that holds the
+ * publisher.
+ *
+ * Admission is key-in-path exactly like RTMP: resolveIngestKey with protocol 'jsmpeg', one open
+ * session per definition. The session goes live on the first bytes and ends when the POST closes.
+ *
+ * Viewers only ever receive, so the WebSocket server is the ~30 lines that takes the upgrade and
+ * writes unmasked binary frames; no ws dependency is needed.
+ *
+ * Drain: when the coordinator marks this generation draining (a newer one is ready) the public
+ * listener closes — new encoders and viewers reach the new generation through the shared port — and
+ * running sessions stay until they leave or the drain deadline passes. Then the process exits by
+ * itself. SIGTERM starts the same drain; it never drops a live session.
+ */
+const http = require('http');
+const crypto = require('crypto');
+const path = require('path');
+const { createWorkerRuntime } = require('./runtime');
+
+const SESSION_ID_RE = /^ses_[0-9A-HJKMNP-TV-Z]{26}$/;
+const INGEST_PATH_RE = /^\/([^/]+)\/(\d{1,5})\/(\d{1,5})\/?$/;
+const TAP_PATH_RE = /^\/tap\/(ses_[0-9A-HJKMNP-TV-Z]{26})\.ts$/;
+const MAX_WS_BACKPRESSURE = 512 * 1024;
+const MAX_ADMISSION_BUFFER = 4 * 1024 * 1024;
+const WS_GUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11';
+const OP_BINARY = 0x2;
+const OP_CLOSE = 0x8;
+
+/** One unmasked WebSocket frame (the server never masks). */
+function wsFrame(opcode, payload) {
+ const len = payload.length;
+ let header;
+ if (len < 126) { header = Buffer.allocUnsafe(2); header[0] = 0x80 | opcode; header[1] = len; }
+ else if (len < 65536) { header = Buffer.allocUnsafe(4); header[0] = 0x80 | opcode; header[1] = 126; header.writeUInt16BE(len, 2); }
+ else { header = Buffer.allocUnsafe(10); header[0] = 0x80 | opcode; header[1] = 127; header.writeBigUInt64BE(BigInt(len), 2); }
+ return Buffer.concat([header, payload]);
+}
+
+/** Bind the first free loopback port in [min, max] (skipping `skip`). */
+function listenInRange(server, min, max, skip = new Set()) {
+ return new Promise((resolve, reject) => {
+ let port = min;
+ const tryNext = () => {
+ while (skip.has(port)) port++;
+ if (port > max) { reject(new Error(`no free loopback port in ${min}-${max}`)); return; }
+ const onError = (err) => {
+ server.removeListener('listening', onListening);
+ if (err.code === 'EADDRINUSE' || err.code === 'EACCES') { port++; tryNext(); } else reject(err);
+ };
+ const onListening = () => { server.removeListener('error', onError); resolve(server.address().port); };
+ server.once('error', onError);
+ server.once('listening', onListening);
+ server.listen(port, '127.0.0.1');
+ };
+ tryNext();
+ });
+}
+
+function createJsmpeg({ rt, log = console, exit = (code) => process.exit(code) }) {
+ const { store, config } = rt;
+ const j = config.jsmpeg;
+ /** session id → { sessionId, definitionId, width, height, req, ended, live, endReason, viewers:Set, taps:Set } */
+ const channels = new Map();
+ // Ends in flight. The heartbeat calls exitWhenIdle() every beat; without this an ended session
+ // whose `finish` write has not committed yet would let the drained process exit before it lands,
+ // leaving the session live until the coordinator fails it lease_expired.
+ let pendingEnds = 0;
+ let ingestServer = null;
+ let tapServer = null;
+ let endpoints = null;
+
+ const runtime = createWorkerRuntime({
+ rt, kind: 'jsmpeg', log, exit,
+ hooks: {
+ onDrain: () => closePublic(),
+ onDrainDeadline: async () => await endAll('drain_deadline'),
+ onEndRequested: async (s) => await endSession(s.id, 'end_requested'),
+ onLost: async () => { for (const ch of [...channels.values()]) await endChannel(ch, 'worker_lost'); return closeAll(); },
+ onHeartbeat: async () => { for (const ch of channels.values()) await store.sessions.setViewers(ch.sessionId, ch.viewers.size); },
+ activeCount: () => channels.size + pendingEnds,
+ onExit: () => closeAll(),
+ },
+ });
+
+ const actor = () => `worker:${runtime.me && runtime.me.id}`;
+ /** Never let a rejected handler — Node 22 ends the worker on an unhandled rejection — drop every session on it. */
+ function safe(promise, what) {
+ return Promise.resolve(promise).catch((err) => log.error(`[jsmpeg] ${what}: ${err && (err.stack || err.message) || err}`));
+ }
+
+ async function admit(key, width, height) {
+ if (runtime.draining || !runtime.me || runtime.me.state !== 'ready') return { error: 'generation not taking sessions' };
+ if (channels.size >= j.maxPublishersPerWorker) return { error: 'worker full' };
+ const r = await store.definitions.resolveIngestKey(key, 'jsmpeg');
+ if (r.error) return { error: r.error };
+ const a = await store.sessions.admit({ definition: r.definition, key: r.key, protocol: 'jsmpeg', worker: runtime.me });
+ if (a.error) return { error: a.error };
+ return { sessionId: a.session.id, definitionId: r.definition.id, hint: r.key.hint, width, height };
+ }
+
+ function handleIngest(req, res, key, width, height) {
+ const buffered = [];
+ let bufferedBytes = 0;
+ let ch = null;
+ let done = false;
+
+ req.on('data', (chunk) => {
+ if (ch) { feed(ch, chunk); return; }
+ if (bufferedBytes + chunk.length > MAX_ADMISSION_BUFFER) return;
+ buffered.push(chunk);
+ bufferedBytes += chunk.length;
+ });
+ req.on('error', () => { if (ch) safe(endChannel(ch, 'publisher_disconnected'), 'error end'); });
+
+ safe((async () => {
+ const adm = await admit(key, width, height);
+ if (done) return;
+ if (adm.error) {
+ log.log(`[jsmpeg] publish refused (${adm.error}) from ${req.socket.remoteAddress || '?'}`);
+ res.statusCode = 404;
+ res.setHeader('content-type', 'text/plain');
+ res.end(`refused: ${adm.error}\n`);
+ req.destroy();
+ return;
+ }
+ ch = { sessionId: adm.sessionId, definitionId: adm.definitionId, req, width, height, live: false, ended: false, endReason: null, viewers: new Set(), taps: new Set() };
+ channels.set(ch.sessionId, ch);
+ log.log(`[jsmpeg] session ${ch.sessionId} accepted for ${ch.definitionId} (key …${adm.hint}) at ${width}x${height}`);
+ res.writeHead(200, { 'content-type': 'video/mp2t' });
+ for (const c of buffered.splice(0)) feed(ch, c);
+ bufferedBytes = 0;
+ })(), 'admission failed');
+
+ req.on('end', () => {
+ done = true;
+ try { res.end(); } catch { /* already closed */ }
+ if (ch) safe(endChannel(ch, ch.endReason || 'publisher_disconnected'), 'end');
+ });
+ req.on('close', () => {
+ done = true;
+ if (ch && !ch.ended) safe(endChannel(ch, ch.endReason || 'publisher_disconnected'), 'close');
+ });
+ }
+
+ async function ensureLive(ch) {
+ if (ch.live || ch.ended) return;
+ ch.live = true;
+ await store.sessions.setMediaInfo(ch.sessionId, { width: ch.width, height: ch.height });
+ const t = await store.sessions.transition(ch.sessionId, 'live', { reason: 'media_flowing', actor: actor() });
+ if (!t.ok) {
+ // The coordinator failed it meanwhile (lease) — the transport must not outlive its record.
+ log.warn(`[jsmpeg] session ${ch.sessionId} could not go live (${t.code}); closing`);
+ ch.endReason = 'coordinator_failed';
+ try { if (ch.req && !ch.req.destroyed) ch.req.destroy(); } catch { /* gone */ }
+ }
+ }
+
+ function feed(ch, chunk) {
+ if (ch.ended) return;
+ if (!ch.live) safe(ensureLive(ch), 'go live');
+ for (const v of ch.viewers) {
+ if (v.socket.destroyed || v.socket.writableLength > MAX_WS_BACKPRESSURE) continue;
+ try { v.socket.write(wsFrame(OP_BINARY, chunk)); } catch { /* socket raced with close */ }
+ }
+ for (const tap of ch.taps) { try { tap(chunk); } catch { /* tap closed */ } }
+ }
+
+ async function endChannel(ch, reason) {
+ if (ch.ended) return;
+ ch.ended = true;
+ channels.delete(ch.sessionId);
+ pendingEnds++;
+ try {
+ for (const v of ch.viewers) { try { v.socket.end(wsFrame(OP_CLOSE, Buffer.alloc(0))); } catch { /* gone */ } }
+ ch.viewers.clear();
+ for (const end of ch.taps) { try { end(); } catch { /* gone */ } }
+ ch.taps.clear();
+ try { if (ch.req && !ch.req.destroyed) ch.req.destroy(); } catch { /* gone */ }
+ await store.sessions.finish(ch.sessionId, { reason, actor: actor() });
+ log.log(`[jsmpeg] session ${ch.sessionId} ended (${reason})`);
+ } finally {
+ pendingEnds--;
+ }
+ if (runtime.draining) runtime.exitWhenIdle();
+ }
+
+ async function endSession(sessionId, reason) {
+ const ch = channels.get(sessionId);
+ if (ch) { ch.endReason = reason; await endChannel(ch, reason); return true; }
+ // Not ours any more (publisher already gone): make sure the record is closed. Counted so a
+ // drained generation does not exit before the write lands.
+ pendingEnds++;
+ try { await store.sessions.finish(sessionId, { reason, actor: actor() }); } finally { pendingEnds--; }
+ if (runtime.draining) runtime.exitWhenIdle();
+ return false;
+ }
+
+ async function endAll(reason) {
+ for (const ch of [...channels.values()]) { ch.endReason = reason; await endChannel(ch, reason); }
+ }
+
+ function handleUpgrade(req, socket) {
+ let pathname;
+ try { pathname = new URL(req.url, 'http://127.0.0.1').pathname; } catch { socket.destroy(); return; }
+ const id = pathname.replace(/^\//, '');
+ const ch = SESSION_ID_RE.test(id) ? channels.get(id) : null;
+ const key = req.headers['sec-websocket-key'];
+ if (!ch || typeof key !== 'string') { socket.destroy(); return; }
+ const accept = crypto.createHash('sha1').update(key + WS_GUID).digest('base64');
+ socket.write('HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n'
+ + `Sec-WebSocket-Accept: ${accept}\r\n\r\n`);
+ socket.setNoDelay(true);
+ const viewer = { socket };
+ ch.viewers.add(viewer);
+ log.log(`[jsmpeg] viewer connected to ${ch.sessionId} (${ch.viewers.size} total)`);
+ const drop = () => { ch.viewers.delete(viewer); };
+ socket.on('close', drop);
+ socket.on('error', drop);
+ // Viewers never send: consume and ignore anything inbound (ping/close frames).
+ socket.on('data', () => {});
+ }
+
+ function handleTap(req, res) {
+ const m = TAP_PATH_RE.exec(new URL(req.url, 'http://127.0.0.1').pathname);
+ const ch = m && req.method === 'GET' ? channels.get(m[1]) : null;
+ if (!ch) { res.statusCode = 404; return res.end(); }
+ res.writeHead(200, { 'content-type': 'video/mp2t', 'cache-control': 'no-store' });
+ const tap = (chunk) => { try { if (!res.writableEnded) res.write(chunk); } catch { /* client gone */ } };
+ const end = () => { ch.taps.delete(tap); try { if (!res.writableEnded) res.end(); } catch { /* gone */ } };
+ ch.taps.add(tap);
+ req.on('close', end);
+ return undefined;
+ }
+
+ function closePublic() {
+ if (!ingestServer) return;
+ ingestServer.close();
+ log.log('[jsmpeg] public listener closed; running sessions stay until they leave');
+ }
+
+ function closeAll() {
+ closePublic();
+ const closing = [];
+ for (const srv of [ingestServer, tapServer]) {
+ if (!srv) continue;
+ srv.closeAllConnections?.();
+ closing.push(new Promise(r => srv.close(() => r())));
+ }
+ ingestServer = null;
+ tapServer = null;
+ return Promise.all(closing);
+ }
+
+ async function start() {
+ // Internal endpoint first: it is how this generation is reached, so it goes in the worker
+ // row before it can take a session.
+ tapServer = http.createServer((req, res) => { handleTap(req, res); });
+ const tapPort = await listenInRange(tapServer, j.internalPortMin, j.internalPortMax);
+ endpoints = { publicPort: j.port, publicPorts: [j.port], tapPort };
+ await runtime.register(endpoints);
+
+ ingestServer = http.createServer((req, res) => {
+ if (req.method !== 'POST') { res.statusCode = 404; return res.end(); }
+ const m = INGEST_PATH_RE.exec(new URL(req.url, 'http://127.0.0.1').pathname);
+ if (!m) { res.statusCode = 404; return res.end(); }
+ handleIngest(req, res, m[1], Number(m[2]), Number(m[3]));
+ return undefined;
+ });
+ ingestServer.on('upgrade', handleUpgrade);
+ await new Promise((resolve, reject) => {
+ ingestServer.once('error', reject);
+ ingestServer.listen({ port: j.port, host: j.bindHost, reusePort: true }, () => resolve());
+ });
+ ingestServer.on('error', (err) => log.error(`[jsmpeg] public listener ${j.port}: ${err.message}`));
+ await runtime.ready();
+ log.log(`[jsmpeg] generation ${runtime.me.generation} ready: ingest+ws ${j.bindHost}:${j.port}, tap 127.0.0.1:${tapPort}`);
+ return endpoints;
+ }
+
+ return {
+ start,
+ runtime,
+ endpoints: () => endpoints,
+ channels: () => [...channels.values()],
+ drain: async (reason) => await runtime.drainNow(reason),
+ close: closeAll,
+ };
+}
+
+if (require.main === module) {
+ (async () => {
+ require('dotenv').config({ path: process.env.OPENRE_ENV_FILE || path.join(process.cwd(), '.env') });
+ const { load, exitIfDrill } = require('../server/config');
+ const { openRuntime } = require('../server/store');
+ const config = load();
+ exitIfDrill(config, 'openre-jsmpeg');
+ const rt = await openRuntime({ config });
+ const worker = createJsmpeg({ rt });
+ await worker.start();
+ for (const sig of ['SIGTERM', 'SIGINT']) process.on(sig, () => worker.drain(sig).catch((err) => console.error(`[jsmpeg] drain failed on ${sig}: ${err && (err.stack || err.message) || err}`)));
+ })().catch((err) => { console.error(`[jsmpeg] failed to start: ${err && err.stack || err}`); process.exit(1); });
+}
+
+module.exports = { createJsmpeg, listenInRange };
@@ -31,10 +31,17 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exit
@@ -31,10 +31,17 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exit
},
},
});
});
+ /** The loopback source URL for a session's protocol, or null when OpenRe cannot restream it yet. */
async function inputUrlFor(session) {
async function inputUrlFor(session) {
- if (session.protocol !== 'rtmp') return null;
- const pb = await store.sessions.playback(session);
- return pb && pb.flv ? pb.flv.internal_url : null;
+ if (session.protocol === 'rtmp') {
+ const pb = await store.sessions.playback(session);
+ return pb && pb.flv ? pb.flv.internal_url : null;
+ }
+ if (session.protocol === 'jsmpeg') {
+ const pb = await store.sessions.playback(session);
+ return pb && pb.jsmpeg ? pb.jsmpeg.tap_internal_url : null;
+ }
+ return null;
}
}
async function poll() {
async function poll() {
@@ -59,7 +66,7 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exit
@@ -59,7 +66,7 @@ function createRestreamWorker({ rt, log = console, exit = (code) => process.exit
if (!input) { await store.outputs.report(row.id, { state: 'failed', last_error: `restream from ${session.protocol} sessions is not supported by OpenRe yet`, ended_at: Date.now() }, { workerId: runtime.me.id }); continue; }
if (!input) { await store.outputs.report(row.id, { state: 'failed', last_error: `restream from ${session.protocol} sessions is not supported by OpenRe yet`, ended_at: Date.now() }, { workerId: runtime.me.id }); continue; }
// Let the ingest settle before pulling (Live waits 3 s for node-media-server's FLV).
// Let the ingest settle before pulling (Live waits 3 s for node-media-server's FLV).
if (session.live_at && Date.now() - session.live_at < config.outputs.startDelayMs) continue;
if (session.live_at && Date.now() - session.live_at < config.outputs.startDelayMs) continue;
- runner = new OutputRunner({ outputId: row.id, destinationId: row.destination_id, inputUrl: input, store, config, log, spawnImpl, lookup, workerId: runtime.me.id });
+ runner = new OutputRunner({ outputId: row.id, destinationId: row.destination_id, inputUrl: input, source: session.protocol, store, config, log, spawnImpl, lookup, workerId: runtime.me.id });
runners.set(row.id, runner);
runners.set(row.id, runner);
runner.start().catch((err) => runner.fail(err.message, { cooldown: false }));
runner.start().catch((err) => runner.fail(err.message, { cooldown: false }));
}
}
@@ -5,9 +5,9 @@
@@ -5,9 +5,9 @@
* output flags, friendly errors). Kept byte-for-byte equivalent where it matters so a stream that
* output flags, friendly errors). Kept byte-for-byte equivalent where it matters so a stream that
* restreams correctly from Live restreams the same way from OpenRe.
* restreams correctly from Live restreams the same way from OpenRe.
*
*
- * Only the RTMP source is ported: RTMP → destination is a codec copy (zero CPU). The JSMPEG
- * (MPEG-TS on stdin) and WebRTC (mediasoup PlainRTP → SDP) sources stay in Live until those
- * ingest protocols move; encodingArgs() is here for them and for a future re-encode option.
+ * RTMP → destination is a codec copy (zero CPU); a JSMPEG source (the worker's MPEG-TS data tap,
+ * read over HTTP) is re-encoded with encodingArgs(). The WebRTC (mediasoup PlainRTP → SDP) source
+ * stays in Live until that ingest protocol moves.
*/
*/
const SRT_DEFAULT_LATENCY_MS = 120;
const SRT_DEFAULT_LATENCY_MS = 120;
@@ -69,6 +69,19 @@ function rtmpCopyArgs(flvUrl, destUrl) {
@@ -69,6 +69,19 @@ function rtmpCopyArgs(flvUrl, destUrl) {
];
];
}
}
+/** MPEG-TS source (JSMPEG data tap) → destination, re-encoded (Live's _startJsmpegRestream). */
+function jsmpegArgs(tsUrl, destUrl, preset, { overrides = {} } = {}) {
+ return [
+ '-hide_banner',
+ '-loglevel', 'warning',
+ '-thread_queue_size', '1024',
+ '-f', 'mpegts',
+ '-i', tsUrl,
+ ...encodingArgs(preset, { hasAudio: true, overrides }),
+ ...outputArgs(destUrl),
+ ];
+}
+
/** `-progress pipe:1 -stats_period 1` after the global flags: the live ACK and the health line. */
/** `-progress pipe:1 -stats_period 1` after the global flags: the live ACK and the health line. */
function withProgress(args) {
function withProgress(args) {
return args[0] === '-hide_banner'
return args[0] === '-hide_banner'
@@ -105,7 +118,8 @@ function encodingArgs(preset, { hasAudio = true, overrides = {} } = {}) {
@@ -105,7 +118,8 @@ function encodingArgs(preset, { hasAudio = true, overrides = {} } = {}) {
'-sc_threshold', '0', '-flags', '+cgop', '-pix_fmt', 'yuv420p', '-threads', '2', '-x264-params', 'nal-hrd=cbr:force-cfr=1'];
'-sc_threshold', '0', '-flags', '+cgop', '-pix_fmt', 'yuv420p', '-threads', '2', '-x264-params', 'nal-hrd=cbr:force-cfr=1'];
if (preset.scale) args.push('-vf', `scale=${preset.scale}:force_original_aspect_ratio=decrease,pad=${preset.scale}:(ow-iw)/2:(oh-ih)/2`);
if (preset.scale) args.push('-vf', `scale=${preset.scale}:force_original_aspect_ratio=decrease,pad=${preset.scale}:(ow-iw)/2:(oh-ih)/2`);
if (fps > 0) args.push('-r', String(fps), '-fps_mode', 'cfr');
if (fps > 0) args.push('-r', String(fps), '-fps_mode', 'cfr');
- if (hasAudio) args.push('-map', '0:a:0', '-c:a', 'aac', '-b:a', audioBitrate, '-ar', '48000', '-ac', '2');
+ // The trailing '?' keeps a video-only source (a jsmpeg tap without audio) restreamable.
+ if (hasAudio) args.push('-map', '0:a:0?', '-c:a', 'aac', '-b:a', audioBitrate, '-ar', '48000', '-ac', '2');
return args;
return args;
}
}
@@ -150,5 +164,5 @@ function redactUrl(value) {
@@ -150,5 +164,5 @@ function redactUrl(value) {
module.exports = {
module.exports = {
QUALITY_PRESETS, PLATFORM_DEFAULT_PRESET, ENCODER_PRESETS, SRT_DEFAULT_LATENCY_MS,
QUALITY_PRESETS, PLATFORM_DEFAULT_PRESET, ENCODER_PRESETS, SRT_DEFAULT_LATENCY_MS,
- isSrtUrl, buildDestUrl, buildSrtUrl, outputArgs, rtmpCopyArgs, withProgress, resolvePreset, customOverrides, encodingArgs, friendlyError, redactUrl, redactText,
+ isSrtUrl, buildDestUrl, buildSrtUrl, outputArgs, rtmpCopyArgs, jsmpegArgs, withProgress, resolvePreset, customOverrides, encodingArgs, friendlyError, redactUrl, redactText,
};
};
@@ -8,16 +8,17 @@
@@ -8,16 +8,17 @@
*/
*/
const { spawn } = require('child_process');
const { spawn } = require('child_process');
const fs = require('fs');
const fs = require('fs');
-const { rtmpCopyArgs, withProgress, buildDestUrl, friendlyError, redactUrl, redactText } = require('./ffmpeg-args');
+const { rtmpCopyArgs, jsmpegArgs, withProgress, buildDestUrl, resolvePreset, customOverrides, friendlyError, redactUrl, redactText } = require('./ffmpeg-args');
const { validateDestinationUrl, checkResolvedHost } = require('../../server/destination-url');
const { validateDestinationUrl, checkResolvedHost } = require('../../server/destination-url');
class OutputRunner {
class OutputRunner {
- constructor({ outputId, destinationId, inputUrl, store, config, log = console, spawnImpl = spawn, lookup, workerId = null }) {
+ constructor({ outputId, destinationId, inputUrl, source = 'rtmp', store, config, log = console, spawnImpl = spawn, lookup, workerId = null }) {
this.outputId = outputId;
this.outputId = outputId;
this.workerId = workerId;
this.workerId = workerId;
this.secrets = [];
this.secrets = [];
this.destinationId = destinationId;
this.destinationId = destinationId;
this.inputUrl = inputUrl;
this.inputUrl = inputUrl;
+ this.source = source;
this.store = store;
this.store = store;
this.config = config;
this.config = config;
this.log = log;
this.log = log;
@@ -68,11 +69,15 @@ class OutputRunner {
@@ -68,11 +69,15 @@ class OutputRunner {
if (!resolved.ok) return this.fail(`destination refused: ${resolved.error}`, { cooldown: false });
if (!resolved.ok) return this.fail(`destination refused: ${resolved.error}`, { cooldown: false });
const destUrl = buildDestUrl(dest);
const destUrl = buildDestUrl(dest);
if (!destUrl) return this.fail('destination has no usable URL or stream key', { cooldown: false });
if (!destUrl) return this.fail('destination has no usable URL or stream key', { cooldown: false });
+ this.preset = resolvePreset(dest);
+ this.overrides = customOverrides(dest);
this.spawn(destUrl);
this.spawn(destUrl);
}
}
spawn(destUrl) {
spawn(destUrl) {
- const args = withProgress(rtmpCopyArgs(this.inputUrl, destUrl));
+ const args = withProgress(this.source === 'jsmpeg'
+ ? jsmpegArgs(this.inputUrl, destUrl, this.preset || resolvePreset({}), this.overrides ? { overrides: this.overrides } : {})
+ : rtmpCopyArgs(this.inputUrl, destUrl));
const rtmps = destUrl.startsWith('rtmps://');
const rtmps = destUrl.startsWith('rtmps://');
const bin = rtmps && this.o.ffmpegOpenSslPath && fs.existsSync(this.o.ffmpegOpenSslPath) ? this.o.ffmpegOpenSslPath : this.o.ffmpegPath;
const bin = rtmps && this.o.ffmpegOpenSslPath && fs.existsSync(this.o.ffmpegOpenSslPath) ? this.o.ffmpegOpenSslPath : this.o.ffmpegPath;
this.note('info', `starting ffmpeg → ${redactUrl(destUrl)}${bin !== this.o.ffmpegPath ? ' (openssl build)' : ''}`);
this.note('info', `starting ffmpeg → ${redactUrl(destUrl)}${bin !== this.o.ffmpegPath ? ' (openssl build)' : ''}`);
deleted file mode 100644
deleted file mode 100644
@@ -1,3 +0,0 @@
@@ -1,3 +0,0 @@
-'use strict';
-/** openre-sfu: worker interface only, the transport is not ported yet (see unported-transport.js). */
-if (require.main === module) require('./unported-transport').main('sfu').catch((err) => { console.error(`[sfu] failed to start: ${err && err.stack || err}`); process.exit(1); });
@@ -1,22 +1,20 @@
@@ -1,22 +1,20 @@
'use strict';
'use strict';
/**
/**
- * The worker interface for the transports OpenRe does not carry yet: WHIP/WebRTC ingest
- * (werift), the mediasoup SFU and JSMPEG. Each registers a generation with the coordinator, takes
- * part in the drain protocol and heartbeats like a real transport worker, and carries nothing:
- * activeCount() is always 0, and no session of these protocols is ever admitted (their stream
- * definitions may list the protocol, but no worker accepts it, so encoders keep using Live).
+ * The worker interface for the transports OpenRe does not carry yet. Today only WebRTC: the port
+ * (WHIP ingest + mediasoup SFU + viewer signaling) is T4's next job (decision 1, one worker owns
+ * all of it). It registers a generation with the coordinator, takes part in the drain protocol and
+ * heartbeats like a real transport worker, and carries nothing: activeCount() is always 0, and no
+ * 'webrtc' session is ever admitted (a definition may list the protocol, but its HTTP error is
+ * 'protocol_not_allowed' — no worker accepts it, so encoders keep using Live).
*
*
- * This is the seam the port lands in: a ported transport replaces `admit nothing` with its
- * session code and reports its sessions through store.sessions exactly like rtmp-ingest.js.
- * Nothing in production runs these (deploy/ has no unit for them).
+ * This is the seam the port lands in: it replaces `admit nothing` with its session code and reports
+ * sessions through store.sessions exactly like rtmp-ingest.js / jsmpeg.js.
*/
*/
const path = require('path');
const path = require('path');
const { createWorkerRuntime } = require('./runtime');
const { createWorkerRuntime } = require('./runtime');
const PORT_STATUS = Object.freeze({
const PORT_STATUS = Object.freeze({
- 'webrtc-ingest': 'WHIP/WebRTC ingest (Live: server/streaming/whip-handler.js, werift) — not ported',
- sfu: 'WebRTC SFU (Live: server/streaming/webrtc-sfu.js + broadcast-server.js, mediasoup) — not ported',
- jsmpeg: 'JSMPEG relay (Live: server/streaming/jsmpeg-relay.js) — not ported',
+ webrtc: 'WebRTC (WHIP ingest + mediasoup SFU + viewer signaling; Live: server/streaming/whip-handler.js + webrtc-sfu.js + broadcast-server.js) — not ported',
});
});
function createUnportedTransport({ rt, kind, log = console, exit = (code) => process.exit(code) }) {
function createUnportedTransport({ rt, kind, log = console, exit = (code) => process.exit(code) }) {
deleted file mode 100644
deleted file mode 100644
@@ -1,3 +0,0 @@
@@ -1,3 +0,0 @@
-'use strict';
-/** openre-webrtc-ingest: worker interface only, the transport is not ported yet (see unported-transport.js). */
-if (require.main === module) require('./unported-transport').main('webrtc-ingest').catch((err) => { console.error(`[webrtc-ingest] failed to start: ${err && err.stack || err}`); process.exit(1); });
new file mode 100644
new file mode 100644
@@ -0,0 +1,13 @@
@@ -0,0 +1,13 @@
+'use strict';
+/**
+ * openre-webrtc — the WebRTC transport worker (unit openre-webrtc@<release>.service).
+ *
+ * One worker owns WHIP ingest, the mediasoup producers/consumers and viewer signaling (T4 decision
+ * 1: mediasoup is single-process, so ingest and consumption cannot be split). This replaces the
+ * webrtc-ingest.js and sfu.js stubs.
+ *
+ * Not ported yet: it registers a real generation and takes part in the drain protocol, but carries
+ * nothing (see unported-transport.js). The next T4 job ports it.
+ */
+if (require.main === module) require('./unported-transport').main('webrtc').catch((err) => { console.error(`[webrtc] failed to start: ${err && err.stack || err}`); process.exit(1); });
+module.exports = require('./unported-transport');