openvibers/openvibe.live README.md
README.md
| 1 |  |
| 2 | |
| 3 | # OpenVibe.Live |
| 4 | |
| 5 | **Open source live streaming** — [openvibe.live](https://openvibe.live). Go live from a browser or OBS with no follower minimum: chat, clips, recordings, restream and The Rundown. |
| 6 | |
| 7 | **Live site:** https://openvibe.live · **API docs:** https://openvibe.services · **For AI:** https://openvibe.live/llms.txt |
| 8 | **Run it:** `npm ci && npm run dev` (Node 18 or newer) · **Tests:** `npm test` |
| 9 | |
| 10 | Part of the [OpenVibe network](https://openvibe.network); [help](https://openvibe.help), [docs](https://openvibe.live/docs/architecture) and the [developer API](https://openvibe.services). |
| 11 | |
| 12 | ## Purpose |
| 13 | |
| 14 | Live is where people go live and watch. It takes a streamer's video in, plays it to viewers, and gives the channel its page, chat, clips and tools. The pieces other services own well (identity, chat storage, recordings, pastes, comments, AI, money, search) are theirs; Live calls them and keeps the streaming, the watch experience and the streamer's controls. |
| 15 | |
| 16 | --- |
| 17 | |
| 18 | ## What this repo contains |
| 19 | |
| 20 | - `server/` — Node/Express backend, WebSocket handling, streaming routes, chat, auth, monetization, and the Media/Network clients. |
| 21 | - `server/media-client.js` — OpenVibe.Media API v1 client (VODs/clips/pastes/files/thumbnails + public URL builders). |
| 22 | - `server/media-proxy/` — thin routers that preserve the SPA's `/api/vods`, `/api/clips`, `/api/pastes`, `/api/thumbnails` paths by forwarding to Media, plus the Media webhook receiver and clip-notify sweeper. |
| 23 | - `server/streaming/recorder.js` — Media-backed recorder (RTMP pull + RTP ingest wiring). |
| 24 | - `server/monetization/wallet-client.js` — OpenCoins wallet client (Network internal API). |
| 25 | - `public/` — static browser UI assets (no build step). |
| 26 | - `data/` — Live-local runtime files (live thumbnails, emotes, avatars, song-request cache). The database is PostgreSQL (`DATABASE_URL`, schema in `migrations/`). |
| 27 | - `node_modules/openvibe-shared/` — pinned OpenVibe.Shared release (`"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/vX.Y.Z"`), served at `/shared/*`. Change it in OpenVibe.Shared and bump the tag; never edit `node_modules`. |
| 28 | - `deploy/` — nginx / systemd / fail2ban reference configs. |
| 29 | - `.env.example` — runtime configuration template. |
| 30 | |
| 31 | --- |
| 32 | |
| 33 | ## Runtime architecture |
| 34 | |
| 35 | ### Core server |
| 36 | |
| 37 | `server/index.js` is the entrypoint (port **3000**). It loads environment configuration, initializes the database, and starts the HTTP server and WebSocket upgrade handler. |
| 38 | |
| 39 | ### Streaming support |
| 40 | |
| 41 | - RTMP ingest using `node-media-server` (ports 1935 / 9935, public host `ingest.openvibe.live`, DNS-only: Cloudflare does not carry RTMP). |
| 42 | - optional WebRTC SFU via `mediasoup` (ports 11000-11300 in production). |
| 43 | - JSMPEG relay (TLS relay ports 9710-9789 via nginx stream). |
| 44 | - WHIP/HTTP ingestion support (`whip.openvibe.live`, behind Cloudflare; `ingest.openvibe.live` still answers for older encoder configs) — open CORS, so even a backend-less web page can publish; see [docs/whip.md → Publishing from a browser](docs/whip.md#publishing-from-a-browser) and the hosted [browser publisher](https://openvibe.live/whip-publisher.html). |
| 45 | - real-time broadcast and control channels. |
| 46 | |
| 47 | ### Media (VODs / clips / pastes / thumbnails) |
| 48 | |
| 49 | The media subsystem lives in **OpenVibe.Media**: |
| 50 | |
| 51 | - On stream start, Live creates a VOD in Media and starts ingest — RTMP streams are pulled by Media from `rtmp://127.0.0.1:1935/live/<key>`; WebRTC/WHIP streams are forwarded over RTP to ports Media allocates (UDP 12000-12199); browser MediaRecorder chunks are proxied to Media's chunks endpoints. |
| 52 | - The SPA's existing `/api/vods…`, `/api/clips…`, `/api/pastes…`, `/api/thumbnails/:filename` calls are preserved by thin proxies; big media files 302-redirect to `https://openvibe.media`. |
| 53 | - Media reports `vod.ready` / `clip.ready` (and failures, storage alerts) as `media.*` OpenVibe.Events events, delivered to `POST /internal/media-events`, driving recording state, AI jobs and clip chat announcements. Each outcome is applied once (see [docs/architecture.md](docs/architecture.md#media-outcomes-over-events)). |
| 54 | - Live-owned AI/transcript state for Media-hosted content lives in `vod_ai_state` / `clip_ai_state` in Live's PostgreSQL database. |
| 55 | - Recordings and clips are watched on **OpenVibe.Video** (`openvibe.video`, served by OpenVibe.MediaHub) since 2026-10-19: it owns their pages (player, AI chapters and captions, chat replay, comments, clipping), and Live's old `/vod/:id` and `/clip/:id` answer a permanent redirect there with any `?t=` (`server/web/video-handover.js`). `/vods` and `/clips` stay on Live as the Content feed with a filter. |
| 56 | |
| 57 | ### Authentication & currencies |
| 58 | |
| 59 | - User auth via OpenVibe.Network OAuth2 (client id `live`); RS256 tokens verified offline with the Network public key; local user records join Network identities via `linked_accounts`. |
| 60 | - **OpenCoins** (network-wide) — earn/spend goes through the Network wallet API (`/internal/coins/credit|debit|transfer`, idempotency keys `live:<event>:<id>`). The legacy local balance column is frozen for migration. |
| 61 | - **Vibes** (tips, checkouts, cash-outs, balances) are OpenVibe.Billing's since the cutover of 2026-10-17: every money action is a Billing call (`server/monetization/billing-client.js`), and provider webhooks go to Billing. Per-streamer channel points stay Live-local. |
| 62 | |
| 63 | ### Data storage |
| 64 | |
| 65 | - PostgreSQL `ov_live` (since 2026-10-08; `migrations/0001_analytics.sql`, `0002_live.sql`) — users, streams, channel state, AI state, request analytics. Chat storage is OpenVibe.Chat's. |
| 66 | - `data/live-thumbs` — ephemeral live-stream thumbnails. |
| 67 | - `data/emotes`, `data/avatars`, `data/offline` — Live-local assets. |
| 68 | - `data/media/cache` — song-request (watch-party) downloads. |
| 69 | - request analytics (the `analytics_*` tables; ADR-021, `openvibe-shared/analytics` since v1.4.0: no IPs or user ids, route templates, raw rows ≤ 30 days; a request with `Sec-GPC: 1` or `DNT: 1` is not recorded at all; `scripts/analytics-prune.js`; see [docs/architecture.md](docs/architecture.md#analytics-adr-021)). |
| 70 | - VOD/clip/paste **files** live in OpenVibe.Media's storage. |
| 71 | |
| 72 | --- |
| 73 | |
| 74 | ## Owns |
| 75 | |
| 76 | - Streaming ingest and viewer playback: RTMP (`node-media-server`), WHIP and WebRTC (`mediasoup`), JSMPEG, and the broadcast page. |
| 77 | - Channels and streams: go-live, stream keys, managed streams, heartbeats, channel pages (`/@username`), offline screens, panels and goals. |
| 78 | - The watch experience: the SPA in `public/`, the chat surface and its effects (TTS, sounds, emotes, overlays), anonymous chat identities. |
| 79 | - Streamer tools: the dashboard, restreaming (via OpenRestream for managed ingest), song requests (watch party), AI viewers (the context Live builds; the runs are OpenVibe.AI's). |
| 80 | - Channel points, Vibes tipping flows (OpenVibe.Billing keeps the ledger since 2026-10-17), moderation of Live's own surfaces, The Rundown (what streamers talked about, written up from their speech: topics, takes in their own words and every stream as an episode; docs/rundown.md, replacing the Arena on 2026-10-19) and after-show recaps. Live transcripts come from NVIDIA Parakeet TDT on the server (whisper for other languages). |
| 81 | - Live-local state: PostgreSQL `ov_live` (users' Live profiles, streams, channel state, AI state for Media-hosted recordings in `vod_ai_state` / `clip_ai_state`, request analytics). |
| 82 | |
| 83 | ## Does not own |
| 84 | |
| 85 | - **Identity, sessions, follows, blocks, notifications, OpenCoins**: OpenVibe.Network. Live verifies its RS256 tokens offline and follows `network.user.*`, `network.follow.*`, deletion and export events. |
| 86 | - **Chat, its messages and every chat table**: OpenVibe.Chat (since 2026-09-23). Live keeps no copy: reads and writes go through `server/chat/chat-reads.js` and `chat-delivery.js`, and the twelve tables it once held do not exist on PostgreSQL (`migrations/0002_live.sql` leaves them out; SQLite-era migration 007 dropped them). |
| 87 | - **Recordings, clips, thumbnails, files**: OpenVibe.Media; their watch pages are OpenVibe.Video's. The local `vods`/`clips`/`pastes` tables are frozen (`test/frozen-tables.test.js`). |
| 88 | - **Pastes and comments**: OpenVibe.Community. |
| 89 | - **AI**: OpenVibe.AI. Live calls no model provider itself. Every prompt is an AI template; streamers' own keys are stored there; VOD and clip transcripts run there. Live-stream captions run in Live's own speech-to-text (`server/ai/transcribe.js`). Both use NVIDIA Parakeet TDT 0.6B v3 for English and 24 European languages and whisper.cpp for the rest. The AI's own clip/moment text is sorted by `server/ai/explicit.js` into exactly three categories: `fine` (nudity is allowed), `sex_acts` (never public; the clip is private and kept for its owner) and `minor_sexual` (always blocked, private even when a person owns it, and a report record is made through OpenVibe.Vision with `VISION_URL` + Live's service principal). The moments job, both auto-clip paths, both clip overview passes and clip evaluation use the same classifier. |
| 90 | - **Search, events, tips, VIP, money ledger**: OpenVibe.Search, Events, Tips, VIP and Billing. |
| 91 | |
| 92 | ## Depends on |
| 93 | |
| 94 | - **OpenVibe.Network**: sign-in (OAuth2 client `live`), the signing key, service tokens (`server/net/network-principal.js`), follows, notifications, the wallet, the OpenVibe Frame (`openvibe-shared`, served at `/shared/*`). |
| 95 | - **OpenVibe.Media**: VOD recording, storage (local, B2, R2), playback, clips, thumbnails, signed playback URLs. Outcomes arrive as `media.*` events at `POST /internal/media-events`. |
| 96 | - **OpenVibe.Chat**, **OpenVibe.Community**, **OpenVibe.AI**, **OpenVibe.Events**, **OpenRestream**, **OpenVibe.Billing**, **OpenVibe.Tips**, **OpenVibe.VIP**, **OpenVibe.Tools**, **OpenVibe.Search**: through their APIs with Live's service token. |
| 97 | - **Libraries**: `openvibe-contracts` (pinned tag), `openvibe-sdk`, `openvibe-shared`; FFmpeg and whisper.cpp on the host. |
| 98 | |
| 99 | ## Capabilities |
| 100 | |
| 101 | Live implements, for other services' tokens (manifest `manifests/services/live.json` in OpenVibe.Contracts): `live.chat_context.read`, `live.chat_effects.write`, `live.lineage.resolve`, `live.tips_delivery.write`, `live.channel.read`, `live.stream.read`, `live.discovery.read`, `live.owner.resolve`. |
| 102 | |
| 103 | Live's own principal (`live`) holds grants to call: |
| 104 | |
| 105 | - **Network**: `identity.subject.resolve`, `network.follows.read|write`, `network.modules.read|write`, `network.notifications.push`, `network.coins.credit|debit`, `network.analytics.creator.read`, `network.account.export.contribute`, `network.account.deletion.confirm`. |
| 106 | - **AI**: `ai.run.create|read` (namespaces `live.*`, `network.site_copy`, `media.analyze`), `ai.credential.manage`, `ai.quota.attribution.manage`. |
| 107 | - **Chat**: `chat.message.send`, `chat.event.publish`, `chat.moderation.write`, `chat.cache.invalidate`, `chat.presence.read`; `chat.live_bridge.write` for the call channels (`server/streaming/calls-authority.js`) and the one-off outbox delivery (`scripts/chat-bridge-outbox-drop.js`). |
| 108 | - **Community**: `community.paste.*`, `community.comment.*`, `community.pulse.write`. |
| 109 | - **Events**: `events.event.publish|read`, `events.subscription.manage`. |
| 110 | - **OpenRestream**: `openre.stream.*`, `openre.key.rotate`, `openre.session.read`. |
| 111 | - **Billing**: `billing.*` (intents, transfers, subscriptions, cash-outs, balances, entitlements). |
| 112 | - **Elsewhere**: `tips.interaction.record`, `vip.entitlement.check`, `tools.tool.run`, `tools.job.read`. |
| 113 | |
| 114 | **Chat ingress.** With `CHAT_AUTHORITY=chat`, every chat producer in Live goes through one seam, `server/chat/chat-delivery.js`, to OpenVibe.Chat's typed service-token ingress (`server/chat/chat-client.js`: `POST /internal/chat/messages|events|moderation|invalidate`, `GET /internal/chat/presence`) — the only path; the old `chat-remote.js` bridge is gone (T3 J2). Each operation carries one idempotency key, reused on every retry. A Chat 4xx is logged, counted and dropped; a 5xx or timeout is retried with the same key past Chat's five-minute delivery lease, then logged, counted and dropped. Neither is thrown into the caller. The old bridge's `chat_bridge_outbox` is dropped by an operator step once the release before is out of rollback range: `node scripts/chat-bridge-outbox-drop.js` (dry run), `--deliver` (hands Chat every queued write it never acknowledged), then `--apply` (backup, then `op_002_drop_chat_bridge_outbox`, which refuses while a chat write is still queued). |
| 115 | |
| 116 | API writes are limited per person (`server/net/actor-limits.js`, `LIVE_LIMITS_MINUTE` / `_HOUR`). |
| 117 | |
| 118 | **Robots are OpenVibe.Bot's (plan T15).** Live has no control socket, hardware API or ONVIF code. A bound channel's chat commands (`!forward`, `!backward`, `!left`, `!right`, `!headup`/`!headdown`, `!liftup`/`!liftdown`, `!say`) reach `POST /internal/chat-effects/hardware` from OpenVibe.Chat and go to Bot as the viewer who typed them (`server/bot/bot-client.js`): Bot's roles, queue and cooldowns decide. On a robot made of buttons (Bot's generic relay or a local profile), which has no drive or actuator, the chat word is pressed as its button of the same name (`forward`, `turn_left`, `head_up` …; a hold button is pressed for the 600 ms pulse and released), and `button:<name>` (Chat's `!<word>` that is not a channel sound) presses any other button. |
| 119 | |
| 120 | **Bot panel embed (`LIVE_BOT_EMBED`, off by default; on in production).** A channel can show its OpenVibe.Bot robot's embeddable panel (`<LIVE_BOT_URL>/panel/<robot id>/embed`). With `LIVE_BOT_EMBED=1` the channel owner (not mods, not staff) binds a robot with `PUT /api/streams/channel/:username/bot` `{ "robot_id": "rob_…" | null }` (null or `""` unbinds; stored in `channels.bot_robot_id`; limited per person as `live.bot.bind`), and `GET /api/streams/channel/:username` carries `bot_embed: { enabled: true, robot_id, url }`. Live never calls Bot to bind: the robot owner's `embed_public` toggle in Bot is the gate, so binding someone else's robot shows nothing until its owner makes it public. With the flag off the route answers 404 and the channel JSON has no `bot_embed` key (the stored id is kept). Turn it on in this order: Bot deployed with `GET /panel/:id/embed` and its `BOT_EMBED_ORIGINS` including this site's origins → set `LIVE_BOT_EMBED=1` → bind a channel. The owner binds from the dashboard's Controls tab (a "Bot robot id" field shown only with the flag on), and a bound channel's page shows the panel in a sandboxed, lazy iframe under the player (`public/js/bot-embed.js`, its own `botEmbed` feature, loaded only for a bound channel; the frame is removed when the route ends). Unbound channels and the flag off load nothing new. The easier way to bind is from Bot: the robot page's **Put the buttons under my OpenVibe.Live stream** turns its embedding on and opens the owner's channel with `?bind_robot=rob_…`, where they confirm (`public/js/bot-bind.js`). The panel's Drive asks the page around it for a ticket (`postMessage` `ov-bot-ticket-request`, answered only to that frame on Bot's origin): `POST /api/streams/channel/:username/bot/ticket` `{ robot_id }` (signed in; the channel's own robot only; limited as `live.bot.ticket`) gets one from Bot as the viewer (`controlTicket`, `bot.robot.control`), so a viewer signed in here drives with no popup or second sign-in; Bot's refusal is said in the panel. When the robot's owner has not turned the panel on, Bot's framed refusal tells the page, which hides the box from viewers (the owner keeps it, with the link to the switch). |
| 121 | |
| 122 | | Variable | Default | Meaning | |
| 123 | | --- | --- | --- | |
| 124 | | `LIVE_BOT_EMBED` | off | `1` turns on the channel ↔ Bot robot binding and `bot_embed` on the channel JSON (`server/bot/`) | |
| 125 | | `LIVE_BOT_INTERNAL_URL` | `http://127.0.0.1:4630` | where Live sends a bound channel's chat robot commands (Bot's `POST /api/v1/robots/:id/commands`, as the viewer, Network grant `bot.robot.control`) | |
| 126 | | `LIVE_BOT_URL` | `https://openvibe.bot` | Bot's origin for the embed URL; a bare https origin (http://localhost* outside production), else the default with a warning | |
| 127 | |
| 128 | ## Acceptance |
| 129 | |
| 130 | `npm test` runs every file in `test/` (`test/run.js`), each in its own process against a temp database: 150 files covering auth and revocation, security crawls (stream keys, secrets, SSRF, open redirect, private objects), the frozen tables, chat bridging, AI (every run is an OpenVibe.AI template; nothing calls a provider directly), transcripts by window, per-actor limits, account export and deletion, drill mode, N-1 compatibility fixtures, and the home page's size budgets. The browser smoke is `BASE=http://127.0.0.1:3000 npm run test:browser` (add `-- --a11y` for axe). |
| 131 | |
| 132 | ## Security |
| 133 | |
| 134 | See [SECURITY.md](SECURITY.md) and [SECURITY_AUDIT.md](SECURITY_AUDIT.md). The main rules: |
| 135 | |
| 136 | - Tokens are verified offline with the Network key. Sessions end on `network.user.token_valid_after`, and API tokens (`hbt_`) are scoped. |
| 137 | - Stream keys never leave the owner's own pages; a crawler test checks every GET route as every role. |
| 138 | - User-chosen URLs are fetched only through `server/net/egress.js`. |
| 139 | - `/metrics` answers loopback only. Secrets come from `/etc/openvibe/live.env` and are never logged. |
| 140 | - Private recordings look missing, and IP bans cover CIDR ranges. |
| 141 | |
| 142 | --- |
| 143 | |
| 144 | ## Package scripts |
| 145 | |
| 146 | - `npm install` — install dependencies. |
| 147 | - `npm start` — start the server. |
| 148 | - `npm run dev` — start in development mode (`NODE_ENV=development`). |
| 149 | - `npm run test:pg` — the tests on the PostgreSQL + PgBouncer containers. |
| 150 | - `npm test` — every test in `test/` (see Acceptance). |
| 151 | |
| 152 | --- |
| 153 | |
| 154 | ## Quick start |
| 155 | |
| 156 | Requirements: Node.js 22, npm, FFmpeg, a running OpenVibe.Network, and (for media features) a running OpenVibe.Media. Linux preferred for production. |
| 157 | |
| 158 | ```bash |
| 159 | npm install |
| 160 | cp .env.example .env # then edit — see SETUP.md (no DATABASE_URL: an embedded PGlite database under DATA_DIR) |
| 161 | npm run dev |
| 162 | ``` |
| 163 | |
| 164 | Minimum `.env`: `BASE_URL`, `JWT_SECRET`, `OV_NETWORK_URL`, `OV_NETWORK_INTERNAL_URL`, `OV_OAUTH_CLIENT_ID`, `OV_OAUTH_CLIENT_SECRET`, `OV_NETWORK_PUBLIC_KEY`, `MEDIA_URL`, `MEDIA_PUBLIC_URL`, `MEDIA_EVENTS_SECRET`. Media calls use Live's service token from that OAuth client (audience `openvibe.media`, namespace `live`, grants `media.object.read`/`list`/`upload`/`delete`). |
| 165 | |
| 166 | --- |
| 167 | |
| 168 | ## Deploy |
| 169 | |
| 170 | - **Production**: release layout under `/opt/openvibe.live` (`current` → `releases/<time>-<sha>`); env `/etc/openvibe/live.env` (0600) plus the unit's `Environment=`; unit `openvibe-live.service` on the systemd socket `127.0.0.1:3000`. |
| 171 | - **Deploy on the host**: `sudo ovhost deploy live --wait-idle` (OpenVibe.Host, strategy `release-layout`; `deploy/scripts/deploy.sh` wraps it). Static-only changes do not restart; server changes restart behind the socket with readiness checks, and a failed release rolls back to the previous one. `ovhost` also announces the release on Events. |
| 172 | - **Rollback**: `sudo ovhost rollback live --wait-idle` (the previous release), or `--to <sha>` for a known one. Details: [docs/deploy.md](docs/deploy.md). |
| 173 | - **nginx**: `deploy/nginx/openvibe.live.conf` (`openvibe.live`, `www.openvibe.live`, `ingest.openvibe.live`, `whip.openvibe.live`). |
| 174 | - **After a deploy**: `npm run n-1:record` refreshes the N-1 compatibility fixtures from the release in production. |
| 175 | |
| 176 | --- |
| 177 | |
| 178 | ## Additional docs |
| 179 | |
| 180 | - [SETUP.md](SETUP.md) — first-time setup, local development, and architecture details. |
| 181 | - [docs/broadcasting.md](docs/broadcasting.md) — streaming method and broadcast page guide. |
| 182 | - [docs/restream-branding.md](docs/restream-branding.md) — branding guide for restream channels. |
| 183 | |
| 184 | <!-- versions:start --> |
| 185 | - openvibe-contracts: v0.419.0 |
| 186 | - openvibe-sdk: v0.161.0 |
| 187 | - openvibe-shared: v4.241.0 |
| 188 | <!-- versions:end --> |
| 189 |