openvibers/openvibe.events Commits

Ops and docs: .env.example, systemd unit, nginx vhost for SSE, CI on Node 22.22.1, README usage, STATUS alpha W3

OP OpenVibers committed 23 Sept 2026 · a5fda929fc · 6 files, +325 −28


Changes

ddiff --git a/.env.example b/.env.example
nnew file mode 100644
iindex 0000000..cd30b94
---- /dev/null
++++ b/.env.example
@@@ -0,0 +1,63 @@
++# ═══════════════════════════════════════════════════════════════
++# OpenVibe.Events — Environment Configuration
++# Copy to .env and fill in values.
++# In production this lives at /etc/openvibe/events.env (0600); the systemd unit sets
++# NODE_ENV, PORT and EVENTS_DB_PATH itself.
++# ═══════════════════════════════════════════════════════════════
++
++# Server
++PORT=4300
++HOST=127.0.0.1
++NODE_ENV=development
++#BASE_URL=https://events.openvibe.network
++
++# Identity — OpenVibe.Network signs service tokens (client_credentials, audience
++# openvibe.events) and user JWTs. The public key is fetched from
++# <url>/api/.well-known/jwks (internal URL first), retried every 30 s until it loads.
++OV_NETWORK_URL=https://openvibe.network
++OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000
++# Expected `iss` of every token (defaults to OV_NETWORK_URL).
++#OV_NETWORK_ISSUER=https://openvibe.network
++# Pin the key instead of fetching it (PEM; \n escapes allowed).
++#OV_NETWORK_PUBLIC_KEY=
++# A browser's user JWT is accepted when its aud contains one of these.
++#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network
++
++# Store (SQLite). Production: /var/lib/openvibe-events/events.db (systemd StateDirectory).
++EVENTS_DB_PATH=./data/events.db
++# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.
++EVENTS_RETENTION_DAYS=30
++# Accepted event ids are remembered this long, so late re-publishes stay duplicates.
++EVENTS_RECEIPT_RETENTION_DAYS=90
++#EVENTS_PRUNE_INTERVAL_MS=3600000
++
++# Publishing
++#EVENTS_MAX_PAYLOAD_BYTES=65536
++# A trace may carry this many hops before a publish is refused (events.loop_detected).
++#EVENTS_MAX_HOPS=8
++# Which source may publish which event_type prefixes. JSON; replaces the default, which is
++# { "<source>": ["<source>."] } for live media network community chat openre billing tips vip
++# ai games tools codes host. Every prefix must start with "<source>.".
++#EVENTS_SOURCE_PREFIXES={"live":["live."],"media":["media."]}
++
++# Delivery worker
++#EVENTS_WORKER=on
++#EVENTS_WORKER_INTERVAL_MS=500
++#EVENTS_MAX_INFLIGHT=20
++#EVENTS_DELIVERY_TIMEOUT_MS=10000
++#EVENTS_MAX_ATTEMPTS=8
++#EVENTS_BACKOFF_MS=1000,5000,30000,120000,600000,3600000
++#EVENTS_MAX_SUBSCRIPTIONS=100
++# Hostnames a subscription endpoint may use (SSRF guard). `*.openvibe.*` = openvibe.<tld>
++# and its subdomains; IP literals only when listed exactly.
++#EVENTS_ENDPOINT_HOSTS=127.0.0.1,*.openvibe.*
++
++# Realtime (SSE) gateway
++#REALTIME_MAX_CONNECTIONS=2000
++#REALTIME_MAX_TOPICS=20
++#REALTIME_HEARTBEAT_MS=25000
++#REALTIME_REPLAY_MAX=1000
++# Signed-out browsers may receive public events.
++#REALTIME_ALLOW_ANONYMOUS=true
++# Exact extra CORS origins besides https://*.openvibe.* (development).
++#REALTIME_CORS_ORIGINS=http://localhost:3000
ddiff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
nnew file mode 100644
iindex 0000000..1ba60fe
---- /dev/null
++++ b/.github/workflows/ci.yml
@@@ -0,0 +1,19 @@
++name: ci
++on:
++ push:
++ branches: [main]
++ pull_request:
++jobs:
++ test:
++ runs-on: ubuntu-latest
++ steps:
++ - uses: actions/checkout@v4
++ - uses: actions/setup-node@v4
++ with:
++ node-version: 22.22.1 # production runs Node 22
++ cache: npm
++ - run: npm ci --no-audit --no-fund
++ - run: npm test
++ # Add once the events.* capabilities and the events service manifest ship in openvibe-contracts
++ # (proposals in docs/capabilities-proposal/):
++ # - run: npx openvibe-contracts-check --service events --src server
ddiff --git a/README.md b/README.md
iindex 8620781..94411c3 100644
---- a/README.md
++++ b/README.md
@@@ -2,8 +2,8 @@
> Durable events, subscriptions, delivery, retry, dead letters and replay for the network.
--**Status:** placeholder — planning only, no runnable code yet.
--**Domain:** `events.openvibe.network (+ realtime.openvibe.network runtime)`
++**Status:** alpha (roadmap Wave 3). Runs and is tested; no service publishes to it in production yet.
++**Domain:** `events.openvibe.network` (the realtime gateway is served here too until ADR-005 decides where `realtime.openvibe.network` lives)
**Plan:** OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §6 and §6.3.
**License:** AGPL-3.0 (same as every OpenVibe service).
@@@ -11,45 +11,125 @@
The durable event backbone the network does not have yet: every authoritative service writes domain mutations and outbox events in one transaction, a relay publishes them here, consumers use inbox/idempotency receipts, and browsers get authorized, resumable projections through the Realtime delivery plane.
--## Owns
++## Running it
--- `events`, `subscriptions`, `outbox_deliveries`, `consumer_checkpoints`, `idempotency_receipts` (PostgreSQL first; Redis only for fanout/leases)
--- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)
--- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay
--- Realtime: WS/SSE topics, `Last-Event-ID`/cursor resume, gap detection, presence (ephemeral)
++```bash
++npm install
++cp .env.example .env
++npm run dev # http://127.0.0.1:4300
++npm test # every test/*.test.js, temp databases, no network needed
++```
--## Does not own
++Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store `/var/lib/openvibe-events/events.db`, nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream` and health only; services on the host call `127.0.0.1:4300`).
--- media bytes or game/media transport packets
--- product business rules
++`GET /api/health` is liveness. `GET /api/ready` is 200 only when the database answers, the delivery worker runs and the Network signing key has loaded (it retries every 30 s while Network boots).
++
++## Auth
++
++Services call with an OpenVibe.Network client-credentials token (`POST /oauth/token`, `audience=openvibe.events`), verified offline against the Network JWKS. Each route checks one capability:
++
++| Capability | Routes |
++|---|---|
++| `events.publish` | `POST /api/v1/events` |
++| `events.subscribe` | `/api/v1/subscriptions…` (own subscriptions only) |
++| `events.read` | `GET /api/v1/events`, `GET /api/v1/events/:id`, `/api/v1/checkpoints`, realtime as a service |
++| `events.admin` | `GET /api/v1/deliveries`, `POST /api/v1/deliveries/replay` |
++
++These ids are not in `openvibe-contracts` yet; their manifests are proposed in [docs/capabilities-proposal/](docs/capabilities-proposal/) (with the service manifest). Until the contracts release defines them, `server/auth.js` grants them with the contracts rule (exact id or a `family.*` grant) and hands the decision to `capabilities.check()` as soon as contracts know the id.
++
++Errors are RFC 9457 problem+json (`openvibe-contracts` `http.problem`) with a stable `code`.
++
++## Publishing
++
++`POST /api/v1/events` takes one `events.event-envelope@1` envelope, or `{ "events": [ … ] }` (up to 100, stored atomically).
++
++- `source` must be the calling service (`svc:live` publishes `source: "live"`), and `event_type` must start with a prefix that source owns (`live.*`, `media.*`, `network.*`, `community.*`, `chat.*`, `openre.*`, `billing.*`, `tips.*`, `vip.*`, `ai.*`, `games.*`, `tools.*`, `codes.*`, `host.*`; `EVENTS_SOURCE_PREFIXES` overrides).
++- Idempotent on `event_id`: a repeat answers `200 { event_id, seq, duplicate: true }` and is never stored twice (even after retention, for `EVENTS_RECEIPT_RETENTION_DAYS`).
++- Missing `trace_id` is taken from the request's `traceparent`; `priority` defaults to `important`, `visibility` to `internal`.
++- Loop guard: an event whose trace already carries 8 hops (the cross-service chain depth, or the same source/type/subject repeating in the trace) is refused with `409 events.loop_detected`.
++- Each accepted event gets a global `seq` and is committed with one delivery row per matching subscription before anything is sent.
++
++## Subscriptions and delivery
++
++```http
++POST /api/v1/subscriptions
++{ "topic_pattern": "media.vod.*", "endpoint": "http://127.0.0.1:3000/internal/events", "retry_policy": { "max_attempts": 8 } }
++→ 201 { "id": "sub_…", "secret": "whsec_…", … } # the secret is shown once
++```
--## Planned surfaces
++Topic patterns are dot-separated segments where `*` stands for one or more whole segments (`media.vod.*`, `*.created`, `media.*.ready`, `*`). Endpoints must be `http(s)` on `127.0.0.1` or `openvibe.<tld>`/its subdomains (`EVENTS_ENDPOINT_HOSTS`); redirects are never followed.
--- publish/subscribe/replay APIs; signed webhook deliveries
--- operator DLQ inspect/replay and queue diagnostics
--- `realtime.openvibe.network` gateway
++Each delivery is `POST <endpoint>` with body `{ "event": <envelope>, "seq": n }` and headers `X-OpenVibe-Event-Id`, `X-OpenVibe-Event-Type`, `X-OpenVibe-Seq`, `X-OpenVibe-Subscription-Id`, `X-OpenVibe-Delivery-Attempt`, `X-OpenVibe-Signature: sha256=<HMAC-SHA256 of the raw body with the subscription secret>` and `traceparent` (the event's trace). Any 2xx is delivered. Anything else is retried after 1 s, 5 s, 30 s, 2 min, 10 min, 1 h (then hourly) up to 8 attempts, after which the delivery is `dead`. Priority classes go first (`critical`, `important`, `low`, then seq); one delivery per subscription is in flight at a time and at most 20 overall. Delivery is at least once and not strictly ordered; consumers dedupe with an inbox and order with `subject.revision`.
--## Data (authority tables / families)
++Operators: `GET /api/v1/deliveries?status=dead` is the dead-letter queue; `POST /api/v1/deliveries/replay { subscription_id, event_ids: [...] }` or `{ subscription_id, from_seq }` requeues retained events (that is also how a new subscription catches up on history).
--- see above
++Pull consumers: `GET /api/v1/events?topic=media.vod.*&after_seq=<cursor>&limit=100` returns `{ events: [{ seq, event }], next_after_seq, latest_seq }`, plus `gap: { from_seq, to_seq }` when the cursor is older than retention. `PUT /api/v1/checkpoints { topic, cursor }` stores a consumer's cursor here if it has nowhere better.
--## Capabilities and events
++## Client library
--- `events.publish`, `events.subscribe`, `events.replay`, `realtime.subscribe`
++`require('openvibe-events')` (package `main` is [lib/client.js](lib/client.js); services can depend on this repo by tarball):
--Events: `all `*.created|updated|deleted` families from every service`
++```js
++const events = require('openvibe-events');
++const tokenClient = contracts.serviceAuth.createTokenClient({ tokenUrl: `${NETWORK}/oauth/token`, clientId, clientSecret, audience: 'openvibe.events' });
++
++// Producer: transactional outbox in the service's own better-sqlite3 database
++const outbox = events.createOutbox(db, { publisher: events.createPublisher({ eventsUrl: 'http://127.0.0.1:4300', tokenClient }) });
++outbox.ensureSchema();
++db.transaction(() => {
++ db.prepare('UPDATE vods SET status = ? WHERE id = ?').run('ready', id);
++ outbox.enqueue({ event_type: 'media.vod.ready', source: 'media', actor, subject: { type: 'vod', id, revision }, payload });
++})(); // the event exists if and only if the change committed
++outbox.start(); // relay: publishes pending rows, marks them sent, backs off on failure
++
++// Consumer: signed webhook + exactly-once effects in the consumer's own database
++const inbox = events.createInbox(db);
++inbox.ensureSchema();
++app.post('/internal/events', express.json({ verify: (req, _res, buf) => { req.rawBody = buf; } }), (req, res) => {
++ if (!events.verifyDelivery(req.rawBody, req.get('X-OpenVibe-Signature'), SECRET)) return res.sendStatus(401);
++ inbox.once('live', req.body.event.event_id, () => { /* synchronous db writes */ });
++ res.sendStatus(204);
++});
++```
++
++`enqueue()` refuses to run outside a transaction. `once()` records the receipt and runs the handler in one SQLite transaction, so a crash before commit leaves nothing behind and the redelivery runs it again, and a crash after commit makes the redelivery a no-op.
++
++## Realtime (SSE)
++
++```js
++const es = new EventSource('https://events.openvibe.network/realtime/stream?topics=live.stream.*,network.notification.*', { withCredentials: true });
++es.onmessage = (m) => { const { seq, event } = JSON.parse(m.data); };
++es.addEventListener('gap', (m) => { /* events were missed: refetch state */ });
++```
++
++- Auth: the Network `ov_token` cookie or a Bearer user JWT; a service token with `events.read`; or nobody (public events only, `REALTIME_ALLOW_ANONYMOUS`). An expired cookie degrades to anonymous; a bad Bearer is a 401.
++- Visibility: `public` events go to anyone subscribed to the topic; `subject` events only to the user whose subject id (`usr_…`) is the event's `actor.id` or its user `subject.id`; `internal` events never reach a browser. A guessed topic yields nothing.
++- Resume: the SSE `id` is the seq, so the browser's automatic `Last-Event-ID` (or `?last_event_id=`) replays what was missed. A cursor older than retention first gets `event: gap` (`{ reason, from_seq, to_seq }`), as does a replay that hits `REALTIME_REPLAY_MAX`.
++- Heartbeat comment every 25 s; at most 20 topics per connection and `REALTIME_MAX_CONNECTIONS` (2000) overall; CORS with credentials for `https://*.openvibe.*` only.
++
++## Owns
++
++- `events`, `subscriptions`, `deliveries`, `consumer_checkpoints`, `idempotency_receipts` (SQLite today; the plan's PostgreSQL + Redis fanout is a later step)
++- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)
++- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay
++- Realtime: SSE topics, `Last-Event-ID`/cursor resume, gap detection (WS and presence are not built yet)
++
++## Does not own
++
++- media bytes or game/media transport packets
++- product business rules
## Depends on
--- OpenVibe.Contracts
--- OpenVibe.Network (topic authorization per subject)
++- OpenVibe.Contracts (`events.event-envelope@1`, service tokens, problem details, ids)
++- OpenVibe.Network (token signing key; user subject ids for topic authorization)
## Acceptance (must be true before "done")
--- event persists before consumer delivery
--- kill a consumer mid-processing; replay creates exactly one effect
--- browser reconnect resumes from a cursor or reports a gap
--- a guessed private topic yields no data
++- event persists before consumer delivery (`test/delivery.test.js`)
++- kill a consumer mid-processing; replay creates exactly one effect (`test/outbox-inbox.test.js`)
++- browser reconnect resumes from a cursor or reports a gap (`test/realtime.test.js`)
++- a guessed private topic yields no data (`test/realtime.test.js`)
## Bootstrap / extraction source
ddiff --git a/STATUS.json b/STATUS.json
iindex c86eb2b..7003d0e 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -1,7 +1,15 @@
{
"repository": "OpenVibers/OpenVibe.Events",
-- "stage": "placeholder",
-- "code": false,
++ "stage": "alpha",
++ "code": true,
++ "wave": "W3",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
-- "updated": "2026-09-21"
++ "runtime": "node 22, express, better-sqlite3",
++ "port": 4300,
++ "domain": "events.openvibe.network",
++ "contracts": "openvibe-contracts v0.5.1",
++ "capabilities": ["events.publish", "events.subscribe", "events.read", "events.admin"],
++ "capabilitiesRegistered": false,
++ "deployed": false,
++ "updated": "2026-09-22"
}
ddiff --git a/deploy/nginx/events.openvibe.network.conf b/deploy/nginx/events.openvibe.network.conf
nnew file mode 100644
iindex 0000000..3ae7e91
---- /dev/null
++++ b/deploy/nginx/events.openvibe.network.conf
@@@ -0,0 +1,93 @@
++## ═══════════════════════════════════════════════════════════════
++## events.openvibe.network — Nginx Configuration (REFERENCE)
++##
++## Deployed to /etc/nginx/sites-available/events.openvibe.network.conf — keep them in sync,
++## and reload nginx after changing it.
++##
++## The Node service (port 4300, systemd unit openvibe-events) serves:
++## /realtime/stream browser Server-Sent Events (long-lived; must not be buffered)
++## /api/health, /api/ready
++## /api/v1/* service-to-service API. Services on this host call 127.0.0.1:4300
++## directly, so the public vhost refuses it; open the location below only
++## when a service on another host needs it (it still requires a token).
++## ═══════════════════════════════════════════════════════════════
++
++limit_req_zone $binary_remote_addr zone=ovevents_connect:10m rate=2r/s;
++limit_conn_zone $binary_remote_addr zone=ovevents_streams:10m;
++
++# ── HTTP → HTTPS redirect ──────────────────────────────────
++server {
++ listen 80;
++ listen [::]:80;
++ server_name events.openvibe.network;
++
++ location /.well-known/acme-challenge/ { root /var/www/certbot; }
++ location / { return 301 https://events.openvibe.network$request_uri; }
++}
++
++# ── HTTPS server ───────────────────────────────────────────
++server {
++ listen 443 ssl;
++ listen [::]:443 ssl;
++ http2 on;
++ server_name events.openvibe.network;
++
++ ssl_certificate /etc/letsencrypt/live/events.openvibe.network/fullchain.pem;
++ ssl_certificate_key /etc/letsencrypt/live/events.openvibe.network/privkey.pem;
++ ssl_protocols TLSv1.2 TLSv1.3;
++
++ access_log /var/log/nginx/events.openvibe.network.access.log;
++ error_log /var/log/nginx/events.openvibe.network.error.log;
++
++ client_max_body_size 2m;
++
++ add_header X-Content-Type-Options nosniff always;
++ add_header Referrer-Policy strict-origin-when-cross-origin always;
++ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
++
++ # ── Realtime (SSE): no buffering, no compression, long reads ──
++ # The service sends a heartbeat comment every 25 s, well inside these timeouts and
++ # Cloudflare's 100 s idle limit. CORS is answered by the service (https://*.openvibe.* only).
++ location = /realtime/stream {
++ limit_req zone=ovevents_connect burst=20 nodelay;
++ limit_conn ovevents_streams 20;
++ proxy_pass http://127.0.0.1:4300;
++ proxy_http_version 1.1;
++ proxy_set_header Connection "";
++ proxy_set_header Host $host;
++ proxy_set_header X-Real-IP $remote_addr;
++ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
++ proxy_set_header X-Forwarded-Proto $scheme;
++ proxy_set_header Last-Event-ID $http_last_event_id;
++ proxy_buffering off;
++ proxy_cache off;
++ gzip off;
++ chunked_transfer_encoding on;
++ proxy_read_timeout 1h;
++ proxy_send_timeout 1h;
++ send_timeout 1h;
++ }
++
++ location = /api/health { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }
++ location = /api/ready { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }
++
++ # ── Service API: host-local only (see header) ───────────
++ location /api/ {
++ return 403;
++ # To expose it (token-guarded) to off-host services instead:
++ # proxy_pass http://127.0.0.1:4300;
++ # proxy_http_version 1.1;
++ # proxy_set_header Host $host;
++ # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
++ # proxy_set_header X-Forwarded-Proto $scheme;
++ }
++
++ location = / {
++ proxy_pass http://127.0.0.1:4300;
++ proxy_set_header Host $host;
++ }
++
++ location / {
++ return 404;
++ }
++}
ddiff --git a/deploy/systemd/openvibe-events.service b/deploy/systemd/openvibe-events.service
nnew file mode 100644
iindex 0000000..55bdb15
---- /dev/null
++++ b/deploy/systemd/openvibe-events.service
@@@ -0,0 +1,34 @@
++[Unit]
++Description=OpenVibe.Events — durable events, signed delivery, dead letters, replay and the SSE realtime gateway
++# Network (openvibe-network) signs the tokens this service verifies. Events starts without it
++# and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.
++After=network.target
++Wants=openvibe-network.service
++
++[Service]
++Type=simple
++User=ubuntu
++Group=ubuntu
++WorkingDirectory=/opt/openvibe.events
++EnvironmentFile=/etc/openvibe/events.env
++Environment=NODE_ENV=production
++Environment=PORT=4300
++Environment=PATH=/usr/local/bin:/usr/bin:/bin
++ExecStart=/usr/bin/env node /opt/openvibe.events/server/index.js
++Restart=always
++RestartSec=5
++# SIGTERM: stop accepting, let in-flight deliveries finish (they are retried anyway if cut short).
++KillSignal=SIGTERM
++TimeoutStopSec=15
++# Up to REALTIME_MAX_CONNECTIONS long-lived SSE sockets plus delivery sockets.
++LimitNOFILE=16384
++NoNewPrivileges=true
++PrivateTmp=true
++ProtectSystem=strict
++ProtectHome=true
++# The code tree stays read-only; the event store lives in /var/lib/openvibe-events.
++StateDirectory=openvibe-events
++Environment=EVENTS_DB_PATH=/var/lib/openvibe-events/events.db
++
++[Install]
++WantedBy=multi-user.target