openvibers/openvibe.events Commits
Ops and docs: .env.example, systemd unit, nginx vhost for SSE, CI on Node 22.22.1, README usage, STATUS alpha W3
Changes
nnew file mode 100644@@@ -0,0 +1,63 @@++# ═══════════════════════════════════════════════════════════════++# OpenVibe.Events — Environment Configuration++# Copy to .env and fill in values.++# In production this lives at /etc/openvibe/events.env (0600); the systemd unit sets++# NODE_ENV, PORT and EVENTS_DB_PATH itself.++# ═══════════════════════════════════════════════════════════════++++# Server++PORT=4300++HOST=127.0.0.1++NODE_ENV=development++#BASE_URL=https://events.openvibe.network++++# Identity — OpenVibe.Network signs service tokens (client_credentials, audience++# openvibe.events) and user JWTs. The public key is fetched from++# <url>/api/.well-known/jwks (internal URL first), retried every 30 s until it loads.++OV_NETWORK_URL=https://openvibe.network++OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000++# Expected `iss` of every token (defaults to OV_NETWORK_URL).++#OV_NETWORK_ISSUER=https://openvibe.network++# Pin the key instead of fetching it (PEM; \n escapes allowed).++#OV_NETWORK_PUBLIC_KEY=++# A browser's user JWT is accepted when its aud contains one of these.++#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network++++# Store (SQLite). Production: /var/lib/openvibe-events/events.db (systemd StateDirectory).++EVENTS_DB_PATH=./data/events.db++# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.++EVENTS_RETENTION_DAYS=30++# Accepted event ids are remembered this long, so late re-publishes stay duplicates.++EVENTS_RECEIPT_RETENTION_DAYS=90++#EVENTS_PRUNE_INTERVAL_MS=3600000++++# Publishing++#EVENTS_MAX_PAYLOAD_BYTES=65536++# A trace may carry this many hops before a publish is refused (events.loop_detected).++#EVENTS_MAX_HOPS=8++# Which source may publish which event_type prefixes. JSON; replaces the default, which is++# { "<source>": ["<source>."] } for live media network community chat openre billing tips vip++# ai games tools codes host. Every prefix must start with "<source>.".++#EVENTS_SOURCE_PREFIXES={"live":["live."],"media":["media."]}++++# Delivery worker++#EVENTS_WORKER=on++#EVENTS_WORKER_INTERVAL_MS=500++#EVENTS_MAX_INFLIGHT=20++#EVENTS_DELIVERY_TIMEOUT_MS=10000++#EVENTS_MAX_ATTEMPTS=8++#EVENTS_BACKOFF_MS=1000,5000,30000,120000,600000,3600000++#EVENTS_MAX_SUBSCRIPTIONS=100++# Hostnames a subscription endpoint may use (SSRF guard). `*.openvibe.*` = openvibe.<tld>++# and its subdomains; IP literals only when listed exactly.++#EVENTS_ENDPOINT_HOSTS=127.0.0.1,*.openvibe.*++++# Realtime (SSE) gateway++#REALTIME_MAX_CONNECTIONS=2000++#REALTIME_MAX_TOPICS=20++#REALTIME_HEARTBEAT_MS=25000++#REALTIME_REPLAY_MAX=1000++# Signed-out browsers may receive public events.++#REALTIME_ALLOW_ANONYMOUS=true++# Exact extra CORS origins besides https://*.openvibe.* (development).++#REALTIME_CORS_ORIGINS=http://localhost:3000nnew file mode 100644@@@ -0,0 +1,19 @@++name: ci++on:++ push:++ branches: [main]++ pull_request:++jobs:++ test:++ runs-on: ubuntu-latest++ steps:++ - uses: actions/checkout@v4++ - uses: actions/setup-node@v4++ with:++ node-version: 22.22.1 # production runs Node 22++ cache: npm++ - run: npm ci --no-audit --no-fund++ - run: npm test++ # Add once the events.* capabilities and the events service manifest ship in openvibe-contracts++ # (proposals in docs/capabilities-proposal/):++ # - run: npx openvibe-contracts-check --service events --src server@@@ -2,8 +2,8 @@> Durable events, subscriptions, delivery, retry, dead letters and replay for the network.--**Status:** placeholder — planning only, no runnable code yet.--**Domain:** `events.openvibe.network (+ realtime.openvibe.network runtime)`++**Status:** alpha (roadmap Wave 3). Runs and is tested; no service publishes to it in production yet.++**Domain:** `events.openvibe.network` (the realtime gateway is served here too until ADR-005 decides where `realtime.openvibe.network` lives)**Plan:** OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §6 and §6.3.**License:** AGPL-3.0 (same as every OpenVibe service).@@@ -11,45 +11,125 @@The durable event backbone the network does not have yet: every authoritative service writes domain mutations and outbox events in one transaction, a relay publishes them here, consumers use inbox/idempotency receipts, and browsers get authorized, resumable projections through the Realtime delivery plane.--## Owns++## Running it--- `events`, `subscriptions`, `outbox_deliveries`, `consumer_checkpoints`, `idempotency_receipts` (PostgreSQL first; Redis only for fanout/leases)--- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)--- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay--- Realtime: WS/SSE topics, `Last-Event-ID`/cursor resume, gap detection, presence (ephemeral)++```bash++npm install++cp .env.example .env++npm run dev # http://127.0.0.1:4300++npm test # every test/*.test.js, temp databases, no network needed++```--## Does not own++Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store `/var/lib/openvibe-events/events.db`, nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream` and health only; services on the host call `127.0.0.1:4300`).--- media bytes or game/media transport packets--- product business rules++`GET /api/health` is liveness. `GET /api/ready` is 200 only when the database answers, the delivery worker runs and the Network signing key has loaded (it retries every 30 s while Network boots).++++## Auth++++Services call with an OpenVibe.Network client-credentials token (`POST /oauth/token`, `audience=openvibe.events`), verified offline against the Network JWKS. Each route checks one capability:++++| Capability | Routes |++|---|---|++| `events.publish` | `POST /api/v1/events` |++| `events.subscribe` | `/api/v1/subscriptions…` (own subscriptions only) |++| `events.read` | `GET /api/v1/events`, `GET /api/v1/events/:id`, `/api/v1/checkpoints`, realtime as a service |++| `events.admin` | `GET /api/v1/deliveries`, `POST /api/v1/deliveries/replay` |++++These ids are not in `openvibe-contracts` yet; their manifests are proposed in [docs/capabilities-proposal/](docs/capabilities-proposal/) (with the service manifest). Until the contracts release defines them, `server/auth.js` grants them with the contracts rule (exact id or a `family.*` grant) and hands the decision to `capabilities.check()` as soon as contracts know the id.++++Errors are RFC 9457 problem+json (`openvibe-contracts` `http.problem`) with a stable `code`.++++## Publishing++++`POST /api/v1/events` takes one `events.event-envelope@1` envelope, or `{ "events": [ … ] }` (up to 100, stored atomically).++++- `source` must be the calling service (`svc:live` publishes `source: "live"`), and `event_type` must start with a prefix that source owns (`live.*`, `media.*`, `network.*`, `community.*`, `chat.*`, `openre.*`, `billing.*`, `tips.*`, `vip.*`, `ai.*`, `games.*`, `tools.*`, `codes.*`, `host.*`; `EVENTS_SOURCE_PREFIXES` overrides).++- Idempotent on `event_id`: a repeat answers `200 { event_id, seq, duplicate: true }` and is never stored twice (even after retention, for `EVENTS_RECEIPT_RETENTION_DAYS`).++- Missing `trace_id` is taken from the request's `traceparent`; `priority` defaults to `important`, `visibility` to `internal`.++- Loop guard: an event whose trace already carries 8 hops (the cross-service chain depth, or the same source/type/subject repeating in the trace) is refused with `409 events.loop_detected`.++- Each accepted event gets a global `seq` and is committed with one delivery row per matching subscription before anything is sent.++++## Subscriptions and delivery++++```http++POST /api/v1/subscriptions++{ "topic_pattern": "media.vod.*", "endpoint": "http://127.0.0.1:3000/internal/events", "retry_policy": { "max_attempts": 8 } }++→ 201 { "id": "sub_…", "secret": "whsec_…", … } # the secret is shown once++```--## Planned surfaces++Topic patterns are dot-separated segments where `*` stands for one or more whole segments (`media.vod.*`, `*.created`, `media.*.ready`, `*`). Endpoints must be `http(s)` on `127.0.0.1` or `openvibe.<tld>`/its subdomains (`EVENTS_ENDPOINT_HOSTS`); redirects are never followed.--- publish/subscribe/replay APIs; signed webhook deliveries--- operator DLQ inspect/replay and queue diagnostics--- `realtime.openvibe.network` gateway++Each delivery is `POST <endpoint>` with body `{ "event": <envelope>, "seq": n }` and headers `X-OpenVibe-Event-Id`, `X-OpenVibe-Event-Type`, `X-OpenVibe-Seq`, `X-OpenVibe-Subscription-Id`, `X-OpenVibe-Delivery-Attempt`, `X-OpenVibe-Signature: sha256=<HMAC-SHA256 of the raw body with the subscription secret>` and `traceparent` (the event's trace). Any 2xx is delivered. Anything else is retried after 1 s, 5 s, 30 s, 2 min, 10 min, 1 h (then hourly) up to 8 attempts, after which the delivery is `dead`. Priority classes go first (`critical`, `important`, `low`, then seq); one delivery per subscription is in flight at a time and at most 20 overall. Delivery is at least once and not strictly ordered; consumers dedupe with an inbox and order with `subject.revision`.--## Data (authority tables / families)++Operators: `GET /api/v1/deliveries?status=dead` is the dead-letter queue; `POST /api/v1/deliveries/replay { subscription_id, event_ids: [...] }` or `{ subscription_id, from_seq }` requeues retained events (that is also how a new subscription catches up on history).--- see above++Pull consumers: `GET /api/v1/events?topic=media.vod.*&after_seq=<cursor>&limit=100` returns `{ events: [{ seq, event }], next_after_seq, latest_seq }`, plus `gap: { from_seq, to_seq }` when the cursor is older than retention. `PUT /api/v1/checkpoints { topic, cursor }` stores a consumer's cursor here if it has nowhere better.--## Capabilities and events++## Client library--- `events.publish`, `events.subscribe`, `events.replay`, `realtime.subscribe`++`require('openvibe-events')` (package `main` is [lib/client.js](lib/client.js); services can depend on this repo by tarball):--Events: `all `*.created|updated|deleted` families from every service`++```js++const events = require('openvibe-events');++const tokenClient = contracts.serviceAuth.createTokenClient({ tokenUrl: `${NETWORK}/oauth/token`, clientId, clientSecret, audience: 'openvibe.events' });++++// Producer: transactional outbox in the service's own better-sqlite3 database++const outbox = events.createOutbox(db, { publisher: events.createPublisher({ eventsUrl: 'http://127.0.0.1:4300', tokenClient }) });++outbox.ensureSchema();++db.transaction(() => {++ db.prepare('UPDATE vods SET status = ? WHERE id = ?').run('ready', id);++ outbox.enqueue({ event_type: 'media.vod.ready', source: 'media', actor, subject: { type: 'vod', id, revision }, payload });++})(); // the event exists if and only if the change committed++outbox.start(); // relay: publishes pending rows, marks them sent, backs off on failure++++// Consumer: signed webhook + exactly-once effects in the consumer's own database++const inbox = events.createInbox(db);++inbox.ensureSchema();++app.post('/internal/events', express.json({ verify: (req, _res, buf) => { req.rawBody = buf; } }), (req, res) => {++ if (!events.verifyDelivery(req.rawBody, req.get('X-OpenVibe-Signature'), SECRET)) return res.sendStatus(401);++ inbox.once('live', req.body.event.event_id, () => { /* synchronous db writes */ });++ res.sendStatus(204);++});++```++++`enqueue()` refuses to run outside a transaction. `once()` records the receipt and runs the handler in one SQLite transaction, so a crash before commit leaves nothing behind and the redelivery runs it again, and a crash after commit makes the redelivery a no-op.++++## Realtime (SSE)++++```js++const es = new EventSource('https://events.openvibe.network/realtime/stream?topics=live.stream.*,network.notification.*', { withCredentials: true });++es.onmessage = (m) => { const { seq, event } = JSON.parse(m.data); };++es.addEventListener('gap', (m) => { /* events were missed: refetch state */ });++```++++- Auth: the Network `ov_token` cookie or a Bearer user JWT; a service token with `events.read`; or nobody (public events only, `REALTIME_ALLOW_ANONYMOUS`). An expired cookie degrades to anonymous; a bad Bearer is a 401.++- Visibility: `public` events go to anyone subscribed to the topic; `subject` events only to the user whose subject id (`usr_…`) is the event's `actor.id` or its user `subject.id`; `internal` events never reach a browser. A guessed topic yields nothing.++- Resume: the SSE `id` is the seq, so the browser's automatic `Last-Event-ID` (or `?last_event_id=`) replays what was missed. A cursor older than retention first gets `event: gap` (`{ reason, from_seq, to_seq }`), as does a replay that hits `REALTIME_REPLAY_MAX`.++- Heartbeat comment every 25 s; at most 20 topics per connection and `REALTIME_MAX_CONNECTIONS` (2000) overall; CORS with credentials for `https://*.openvibe.*` only.++++## Owns++++- `events`, `subscriptions`, `deliveries`, `consumer_checkpoints`, `idempotency_receipts` (SQLite today; the plan's PostgreSQL + Redis fanout is a later step)++- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)++- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay++- Realtime: SSE topics, `Last-Event-ID`/cursor resume, gap detection (WS and presence are not built yet)++++## Does not own++++- media bytes or game/media transport packets++- product business rules## Depends on--- OpenVibe.Contracts--- OpenVibe.Network (topic authorization per subject)++- OpenVibe.Contracts (`events.event-envelope@1`, service tokens, problem details, ids)++- OpenVibe.Network (token signing key; user subject ids for topic authorization)## Acceptance (must be true before "done")--- event persists before consumer delivery--- kill a consumer mid-processing; replay creates exactly one effect--- browser reconnect resumes from a cursor or reports a gap--- a guessed private topic yields no data++- event persists before consumer delivery (`test/delivery.test.js`)++- kill a consumer mid-processing; replay creates exactly one effect (`test/outbox-inbox.test.js`)++- browser reconnect resumes from a cursor or reports a gap (`test/realtime.test.js`)++- a guessed private topic yields no data (`test/realtime.test.js`)## Bootstrap / extraction source@@@ -1,7 +1,15 @@{"repository": "OpenVibers/OpenVibe.Events",-- "stage": "placeholder",-- "code": false,++ "stage": "alpha",++ "code": true,++ "wave": "W3","plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",-- "updated": "2026-09-21"++ "runtime": "node 22, express, better-sqlite3",++ "port": 4300,++ "domain": "events.openvibe.network",++ "contracts": "openvibe-contracts v0.5.1",++ "capabilities": ["events.publish", "events.subscribe", "events.read", "events.admin"],++ "capabilitiesRegistered": false,++ "deployed": false,++ "updated": "2026-09-22"}nnew file mode 100644@@@ -0,0 +1,93 @@++## ═══════════════════════════════════════════════════════════════++## events.openvibe.network — Nginx Configuration (REFERENCE)++##++## Deployed to /etc/nginx/sites-available/events.openvibe.network.conf — keep them in sync,++## and reload nginx after changing it.++##++## The Node service (port 4300, systemd unit openvibe-events) serves:++## /realtime/stream browser Server-Sent Events (long-lived; must not be buffered)++## /api/health, /api/ready++## /api/v1/* service-to-service API. Services on this host call 127.0.0.1:4300++## directly, so the public vhost refuses it; open the location below only++## when a service on another host needs it (it still requires a token).++## ═══════════════════════════════════════════════════════════════++++limit_req_zone $binary_remote_addr zone=ovevents_connect:10m rate=2r/s;++limit_conn_zone $binary_remote_addr zone=ovevents_streams:10m;++++# ── HTTP → HTTPS redirect ──────────────────────────────────++server {++ listen 80;++ listen [::]:80;++ server_name events.openvibe.network;++++ location /.well-known/acme-challenge/ { root /var/www/certbot; }++ location / { return 301 https://events.openvibe.network$request_uri; }++}++++# ── HTTPS server ───────────────────────────────────────────++server {++ listen 443 ssl;++ listen [::]:443 ssl;++ http2 on;++ server_name events.openvibe.network;++++ ssl_certificate /etc/letsencrypt/live/events.openvibe.network/fullchain.pem;++ ssl_certificate_key /etc/letsencrypt/live/events.openvibe.network/privkey.pem;++ ssl_protocols TLSv1.2 TLSv1.3;++++ access_log /var/log/nginx/events.openvibe.network.access.log;++ error_log /var/log/nginx/events.openvibe.network.error.log;++++ client_max_body_size 2m;++++ add_header X-Content-Type-Options nosniff always;++ add_header Referrer-Policy strict-origin-when-cross-origin always;++ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;++++ # ── Realtime (SSE): no buffering, no compression, long reads ──++ # The service sends a heartbeat comment every 25 s, well inside these timeouts and++ # Cloudflare's 100 s idle limit. CORS is answered by the service (https://*.openvibe.* only).++ location = /realtime/stream {++ limit_req zone=ovevents_connect burst=20 nodelay;++ limit_conn ovevents_streams 20;++ proxy_pass http://127.0.0.1:4300;++ proxy_http_version 1.1;++ proxy_set_header Connection "";++ proxy_set_header Host $host;++ proxy_set_header X-Real-IP $remote_addr;++ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;++ proxy_set_header X-Forwarded-Proto $scheme;++ proxy_set_header Last-Event-ID $http_last_event_id;++ proxy_buffering off;++ proxy_cache off;++ gzip off;++ chunked_transfer_encoding on;++ proxy_read_timeout 1h;++ proxy_send_timeout 1h;++ send_timeout 1h;++ }++++ location = /api/health { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }++ location = /api/ready { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }++++ # ── Service API: host-local only (see header) ───────────++ location /api/ {++ return 403;++ # To expose it (token-guarded) to off-host services instead:++ # proxy_pass http://127.0.0.1:4300;++ # proxy_http_version 1.1;++ # proxy_set_header Host $host;++ # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;++ # proxy_set_header X-Forwarded-Proto $scheme;++ }++++ location = / {++ proxy_pass http://127.0.0.1:4300;++ proxy_set_header Host $host;++ }++++ location / {++ return 404;++ }++}nnew file mode 100644@@@ -0,0 +1,34 @@++[Unit]++Description=OpenVibe.Events — durable events, signed delivery, dead letters, replay and the SSE realtime gateway++# Network (openvibe-network) signs the tokens this service verifies. Events starts without it++# and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.++After=network.target++Wants=openvibe-network.service++++[Service]++Type=simple++User=ubuntu++Group=ubuntu++WorkingDirectory=/opt/openvibe.events++EnvironmentFile=/etc/openvibe/events.env++Environment=NODE_ENV=production++Environment=PORT=4300++Environment=PATH=/usr/local/bin:/usr/bin:/bin++ExecStart=/usr/bin/env node /opt/openvibe.events/server/index.js++Restart=always++RestartSec=5++# SIGTERM: stop accepting, let in-flight deliveries finish (they are retried anyway if cut short).++KillSignal=SIGTERM++TimeoutStopSec=15++# Up to REALTIME_MAX_CONNECTIONS long-lived SSE sockets plus delivery sockets.++LimitNOFILE=16384++NoNewPrivileges=true++PrivateTmp=true++ProtectSystem=strict++ProtectHome=true++# The code tree stays read-only; the event store lives in /var/lib/openvibe-events.++StateDirectory=openvibe-events++Environment=EVENTS_DB_PATH=/var/lib/openvibe-events/events.db++++[Install]++WantedBy=multi-user.target
new file mode 100644
new file mode 100644
@@ -0,0 +1,63 @@
@@ -0,0 +1,63 @@
+# ═══════════════════════════════════════════════════════════════
+# OpenVibe.Events — Environment Configuration
+# Copy to .env and fill in values.
+# In production this lives at /etc/openvibe/events.env (0600); the systemd unit sets
+# NODE_ENV, PORT and EVENTS_DB_PATH itself.
+# ═══════════════════════════════════════════════════════════════
+
+# Server
+PORT=4300
+HOST=127.0.0.1
+NODE_ENV=development
+#BASE_URL=https://events.openvibe.network
+
+# Identity — OpenVibe.Network signs service tokens (client_credentials, audience
+# openvibe.events) and user JWTs. The public key is fetched from
+# <url>/api/.well-known/jwks (internal URL first), retried every 30 s until it loads.
+OV_NETWORK_URL=https://openvibe.network
+OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000
+# Expected `iss` of every token (defaults to OV_NETWORK_URL).
+#OV_NETWORK_ISSUER=https://openvibe.network
+# Pin the key instead of fetching it (PEM; \n escapes allowed).
+#OV_NETWORK_PUBLIC_KEY=
+# A browser's user JWT is accepted when its aud contains one of these.
+#EVENTS_USER_AUDIENCES=openvibe.events,openvibe.network
+
+# Store (SQLite). Production: /var/lib/openvibe-events/events.db (systemd StateDirectory).
+EVENTS_DB_PATH=./data/events.db
+# Events (and their deliveries) older than this are pruned; resuming from before it yields a gap.
+EVENTS_RETENTION_DAYS=30
+# Accepted event ids are remembered this long, so late re-publishes stay duplicates.
+EVENTS_RECEIPT_RETENTION_DAYS=90
+#EVENTS_PRUNE_INTERVAL_MS=3600000
+
+# Publishing
+#EVENTS_MAX_PAYLOAD_BYTES=65536
+# A trace may carry this many hops before a publish is refused (events.loop_detected).
+#EVENTS_MAX_HOPS=8
+# Which source may publish which event_type prefixes. JSON; replaces the default, which is
+# { "<source>": ["<source>."] } for live media network community chat openre billing tips vip
+# ai games tools codes host. Every prefix must start with "<source>.".
+#EVENTS_SOURCE_PREFIXES={"live":["live."],"media":["media."]}
+
+# Delivery worker
+#EVENTS_WORKER=on
+#EVENTS_WORKER_INTERVAL_MS=500
+#EVENTS_MAX_INFLIGHT=20
+#EVENTS_DELIVERY_TIMEOUT_MS=10000
+#EVENTS_MAX_ATTEMPTS=8
+#EVENTS_BACKOFF_MS=1000,5000,30000,120000,600000,3600000
+#EVENTS_MAX_SUBSCRIPTIONS=100
+# Hostnames a subscription endpoint may use (SSRF guard). `*.openvibe.*` = openvibe.<tld>
+# and its subdomains; IP literals only when listed exactly.
+#EVENTS_ENDPOINT_HOSTS=127.0.0.1,*.openvibe.*
+
+# Realtime (SSE) gateway
+#REALTIME_MAX_CONNECTIONS=2000
+#REALTIME_MAX_TOPICS=20
+#REALTIME_HEARTBEAT_MS=25000
+#REALTIME_REPLAY_MAX=1000
+# Signed-out browsers may receive public events.
+#REALTIME_ALLOW_ANONYMOUS=true
+# Exact extra CORS origins besides https://*.openvibe.* (development).
+#REALTIME_CORS_ORIGINS=http://localhost:3000
new file mode 100644
new file mode 100644
@@ -0,0 +1,19 @@
@@ -0,0 +1,19 @@
+name: ci
+on:
+ push:
+ branches: [main]
+ pull_request:
+jobs:
+ test:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-node@v4
+ with:
+ node-version: 22.22.1 # production runs Node 22
+ cache: npm
+ - run: npm ci --no-audit --no-fund
+ - run: npm test
+ # Add once the events.* capabilities and the events service manifest ship in openvibe-contracts
+ # (proposals in docs/capabilities-proposal/):
+ # - run: npx openvibe-contracts-check --service events --src server
@@ -2,8 +2,8 @@
@@ -2,8 +2,8 @@
> Durable events, subscriptions, delivery, retry, dead letters and replay for the network.
> Durable events, subscriptions, delivery, retry, dead letters and replay for the network.
-**Status:** placeholder — planning only, no runnable code yet.
-**Domain:** `events.openvibe.network (+ realtime.openvibe.network runtime)`
+**Status:** alpha (roadmap Wave 3). Runs and is tested; no service publishes to it in production yet.
+**Domain:** `events.openvibe.network` (the realtime gateway is served here too until ADR-005 decides where `realtime.openvibe.network` lives)
**Plan:** OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §6 and §6.3.
**Plan:** OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §6 and §6.3.
**License:** AGPL-3.0 (same as every OpenVibe service).
**License:** AGPL-3.0 (same as every OpenVibe service).
@@ -11,45 +11,125 @@
@@ -11,45 +11,125 @@
The durable event backbone the network does not have yet: every authoritative service writes domain mutations and outbox events in one transaction, a relay publishes them here, consumers use inbox/idempotency receipts, and browsers get authorized, resumable projections through the Realtime delivery plane.
The durable event backbone the network does not have yet: every authoritative service writes domain mutations and outbox events in one transaction, a relay publishes them here, consumers use inbox/idempotency receipts, and browsers get authorized, resumable projections through the Realtime delivery plane.
-## Owns
+## Running it
-- `events`, `subscriptions`, `outbox_deliveries`, `consumer_checkpoints`, `idempotency_receipts` (PostgreSQL first; Redis only for fanout/leases)
-- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)
-- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay
-- Realtime: WS/SSE topics, `Last-Event-ID`/cursor resume, gap detection, presence (ephemeral)
+```bash
+npm install
+cp .env.example .env
+npm run dev # http://127.0.0.1:4300
+npm test # every test/*.test.js, temp databases, no network needed
+```
-## Does not own
+Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.events`, env `/etc/openvibe/events.env`, unit [deploy/systemd/openvibe-events.service](deploy/systemd/openvibe-events.service), store `/var/lib/openvibe-events/events.db`, nginx [deploy/nginx/events.openvibe.network.conf](deploy/nginx/events.openvibe.network.conf) (public vhost exposes `/realtime/stream` and health only; services on the host call `127.0.0.1:4300`).
-- media bytes or game/media transport packets
-- product business rules
+`GET /api/health` is liveness. `GET /api/ready` is 200 only when the database answers, the delivery worker runs and the Network signing key has loaded (it retries every 30 s while Network boots).
+
+## Auth
+
+Services call with an OpenVibe.Network client-credentials token (`POST /oauth/token`, `audience=openvibe.events`), verified offline against the Network JWKS. Each route checks one capability:
+
+| Capability | Routes |
+|---|---|
+| `events.publish` | `POST /api/v1/events` |
+| `events.subscribe` | `/api/v1/subscriptions…` (own subscriptions only) |
+| `events.read` | `GET /api/v1/events`, `GET /api/v1/events/:id`, `/api/v1/checkpoints`, realtime as a service |
+| `events.admin` | `GET /api/v1/deliveries`, `POST /api/v1/deliveries/replay` |
+
+These ids are not in `openvibe-contracts` yet; their manifests are proposed in [docs/capabilities-proposal/](docs/capabilities-proposal/) (with the service manifest). Until the contracts release defines them, `server/auth.js` grants them with the contracts rule (exact id or a `family.*` grant) and hands the decision to `capabilities.check()` as soon as contracts know the id.
+
+Errors are RFC 9457 problem+json (`openvibe-contracts` `http.problem`) with a stable `code`.
+
+## Publishing
+
+`POST /api/v1/events` takes one `events.event-envelope@1` envelope, or `{ "events": [ … ] }` (up to 100, stored atomically).
+
+- `source` must be the calling service (`svc:live` publishes `source: "live"`), and `event_type` must start with a prefix that source owns (`live.*`, `media.*`, `network.*`, `community.*`, `chat.*`, `openre.*`, `billing.*`, `tips.*`, `vip.*`, `ai.*`, `games.*`, `tools.*`, `codes.*`, `host.*`; `EVENTS_SOURCE_PREFIXES` overrides).
+- Idempotent on `event_id`: a repeat answers `200 { event_id, seq, duplicate: true }` and is never stored twice (even after retention, for `EVENTS_RECEIPT_RETENTION_DAYS`).
+- Missing `trace_id` is taken from the request's `traceparent`; `priority` defaults to `important`, `visibility` to `internal`.
+- Loop guard: an event whose trace already carries 8 hops (the cross-service chain depth, or the same source/type/subject repeating in the trace) is refused with `409 events.loop_detected`.
+- Each accepted event gets a global `seq` and is committed with one delivery row per matching subscription before anything is sent.
+
+## Subscriptions and delivery
+
+```http
+POST /api/v1/subscriptions
+{ "topic_pattern": "media.vod.*", "endpoint": "http://127.0.0.1:3000/internal/events", "retry_policy": { "max_attempts": 8 } }
+→ 201 { "id": "sub_…", "secret": "whsec_…", … } # the secret is shown once
+```
-## Planned surfaces
+Topic patterns are dot-separated segments where `*` stands for one or more whole segments (`media.vod.*`, `*.created`, `media.*.ready`, `*`). Endpoints must be `http(s)` on `127.0.0.1` or `openvibe.<tld>`/its subdomains (`EVENTS_ENDPOINT_HOSTS`); redirects are never followed.
-- publish/subscribe/replay APIs; signed webhook deliveries
-- operator DLQ inspect/replay and queue diagnostics
-- `realtime.openvibe.network` gateway
+Each delivery is `POST <endpoint>` with body `{ "event": <envelope>, "seq": n }` and headers `X-OpenVibe-Event-Id`, `X-OpenVibe-Event-Type`, `X-OpenVibe-Seq`, `X-OpenVibe-Subscription-Id`, `X-OpenVibe-Delivery-Attempt`, `X-OpenVibe-Signature: sha256=<HMAC-SHA256 of the raw body with the subscription secret>` and `traceparent` (the event's trace). Any 2xx is delivered. Anything else is retried after 1 s, 5 s, 30 s, 2 min, 10 min, 1 h (then hourly) up to 8 attempts, after which the delivery is `dead`. Priority classes go first (`critical`, `important`, `low`, then seq); one delivery per subscription is in flight at a time and at most 20 overall. Delivery is at least once and not strictly ordered; consumers dedupe with an inbox and order with `subject.revision`.
-## Data (authority tables / families)
+Operators: `GET /api/v1/deliveries?status=dead` is the dead-letter queue; `POST /api/v1/deliveries/replay { subscription_id, event_ids: [...] }` or `{ subscription_id, from_seq }` requeues retained events (that is also how a new subscription catches up on history).
-- see above
+Pull consumers: `GET /api/v1/events?topic=media.vod.*&after_seq=<cursor>&limit=100` returns `{ events: [{ seq, event }], next_after_seq, latest_seq }`, plus `gap: { from_seq, to_seq }` when the cursor is older than retention. `PUT /api/v1/checkpoints { topic, cursor }` stores a consumer's cursor here if it has nowhere better.
-## Capabilities and events
+## Client library
-- `events.publish`, `events.subscribe`, `events.replay`, `realtime.subscribe`
+`require('openvibe-events')` (package `main` is [lib/client.js](lib/client.js); services can depend on this repo by tarball):
-Events: `all `*.created|updated|deleted` families from every service`
+```js
+const events = require('openvibe-events');
+const tokenClient = contracts.serviceAuth.createTokenClient({ tokenUrl: `${NETWORK}/oauth/token`, clientId, clientSecret, audience: 'openvibe.events' });
+
+// Producer: transactional outbox in the service's own better-sqlite3 database
+const outbox = events.createOutbox(db, { publisher: events.createPublisher({ eventsUrl: 'http://127.0.0.1:4300', tokenClient }) });
+outbox.ensureSchema();
+db.transaction(() => {
+ db.prepare('UPDATE vods SET status = ? WHERE id = ?').run('ready', id);
+ outbox.enqueue({ event_type: 'media.vod.ready', source: 'media', actor, subject: { type: 'vod', id, revision }, payload });
+})(); // the event exists if and only if the change committed
+outbox.start(); // relay: publishes pending rows, marks them sent, backs off on failure
+
+// Consumer: signed webhook + exactly-once effects in the consumer's own database
+const inbox = events.createInbox(db);
+inbox.ensureSchema();
+app.post('/internal/events', express.json({ verify: (req, _res, buf) => { req.rawBody = buf; } }), (req, res) => {
+ if (!events.verifyDelivery(req.rawBody, req.get('X-OpenVibe-Signature'), SECRET)) return res.sendStatus(401);
+ inbox.once('live', req.body.event.event_id, () => { /* synchronous db writes */ });
+ res.sendStatus(204);
+});
+```
+
+`enqueue()` refuses to run outside a transaction. `once()` records the receipt and runs the handler in one SQLite transaction, so a crash before commit leaves nothing behind and the redelivery runs it again, and a crash after commit makes the redelivery a no-op.
+
+## Realtime (SSE)
+
+```js
+const es = new EventSource('https://events.openvibe.network/realtime/stream?topics=live.stream.*,network.notification.*', { withCredentials: true });
+es.onmessage = (m) => { const { seq, event } = JSON.parse(m.data); };
+es.addEventListener('gap', (m) => { /* events were missed: refetch state */ });
+```
+
+- Auth: the Network `ov_token` cookie or a Bearer user JWT; a service token with `events.read`; or nobody (public events only, `REALTIME_ALLOW_ANONYMOUS`). An expired cookie degrades to anonymous; a bad Bearer is a 401.
+- Visibility: `public` events go to anyone subscribed to the topic; `subject` events only to the user whose subject id (`usr_…`) is the event's `actor.id` or its user `subject.id`; `internal` events never reach a browser. A guessed topic yields nothing.
+- Resume: the SSE `id` is the seq, so the browser's automatic `Last-Event-ID` (or `?last_event_id=`) replays what was missed. A cursor older than retention first gets `event: gap` (`{ reason, from_seq, to_seq }`), as does a replay that hits `REALTIME_REPLAY_MAX`.
+- Heartbeat comment every 25 s; at most 20 topics per connection and `REALTIME_MAX_CONNECTIONS` (2000) overall; CORS with credentials for `https://*.openvibe.*` only.
+
+## Owns
+
+- `events`, `subscriptions`, `deliveries`, `consumer_checkpoints`, `idempotency_receipts` (SQLite today; the plan's PostgreSQL + Redis fanout is a later step)
+- canonical event envelope (event_id, trace_id, type, version, source, actor, subject + revision, payload)
+- priority classes `critical|important|low`, loop guards, backpressure, DLQ and replay
+- Realtime: SSE topics, `Last-Event-ID`/cursor resume, gap detection (WS and presence are not built yet)
+
+## Does not own
+
+- media bytes or game/media transport packets
+- product business rules
## Depends on
## Depends on
-- OpenVibe.Contracts
-- OpenVibe.Network (topic authorization per subject)
+- OpenVibe.Contracts (`events.event-envelope@1`, service tokens, problem details, ids)
+- OpenVibe.Network (token signing key; user subject ids for topic authorization)
## Acceptance (must be true before "done")
## Acceptance (must be true before "done")
-- event persists before consumer delivery
-- kill a consumer mid-processing; replay creates exactly one effect
-- browser reconnect resumes from a cursor or reports a gap
-- a guessed private topic yields no data
+- event persists before consumer delivery (`test/delivery.test.js`)
+- kill a consumer mid-processing; replay creates exactly one effect (`test/outbox-inbox.test.js`)
+- browser reconnect resumes from a cursor or reports a gap (`test/realtime.test.js`)
+- a guessed private topic yields no data (`test/realtime.test.js`)
## Bootstrap / extraction source
## Bootstrap / extraction source
@@ -1,7 +1,15 @@
@@ -1,7 +1,15 @@
{
{
"repository": "OpenVibers/OpenVibe.Events",
"repository": "OpenVibers/OpenVibe.Events",
- "stage": "placeholder",
- "code": false,
+ "stage": "alpha",
+ "code": true,
+ "wave": "W3",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
- "updated": "2026-09-21"
+ "runtime": "node 22, express, better-sqlite3",
+ "port": 4300,
+ "domain": "events.openvibe.network",
+ "contracts": "openvibe-contracts v0.5.1",
+ "capabilities": ["events.publish", "events.subscribe", "events.read", "events.admin"],
+ "capabilitiesRegistered": false,
+ "deployed": false,
+ "updated": "2026-09-22"
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,93 @@
@@ -0,0 +1,93 @@
+## ═══════════════════════════════════════════════════════════════
+## events.openvibe.network — Nginx Configuration (REFERENCE)
+##
+## Deployed to /etc/nginx/sites-available/events.openvibe.network.conf — keep them in sync,
+## and reload nginx after changing it.
+##
+## The Node service (port 4300, systemd unit openvibe-events) serves:
+## /realtime/stream browser Server-Sent Events (long-lived; must not be buffered)
+## /api/health, /api/ready
+## /api/v1/* service-to-service API. Services on this host call 127.0.0.1:4300
+## directly, so the public vhost refuses it; open the location below only
+## when a service on another host needs it (it still requires a token).
+## ═══════════════════════════════════════════════════════════════
+
+limit_req_zone $binary_remote_addr zone=ovevents_connect:10m rate=2r/s;
+limit_conn_zone $binary_remote_addr zone=ovevents_streams:10m;
+
+# ── HTTP → HTTPS redirect ──────────────────────────────────
+server {
+ listen 80;
+ listen [::]:80;
+ server_name events.openvibe.network;
+
+ location /.well-known/acme-challenge/ { root /var/www/certbot; }
+ location / { return 301 https://events.openvibe.network$request_uri; }
+}
+
+# ── HTTPS server ───────────────────────────────────────────
+server {
+ listen 443 ssl;
+ listen [::]:443 ssl;
+ http2 on;
+ server_name events.openvibe.network;
+
+ ssl_certificate /etc/letsencrypt/live/events.openvibe.network/fullchain.pem;
+ ssl_certificate_key /etc/letsencrypt/live/events.openvibe.network/privkey.pem;
+ ssl_protocols TLSv1.2 TLSv1.3;
+
+ access_log /var/log/nginx/events.openvibe.network.access.log;
+ error_log /var/log/nginx/events.openvibe.network.error.log;
+
+ client_max_body_size 2m;
+
+ add_header X-Content-Type-Options nosniff always;
+ add_header Referrer-Policy strict-origin-when-cross-origin always;
+ add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
+
+ # ── Realtime (SSE): no buffering, no compression, long reads ──
+ # The service sends a heartbeat comment every 25 s, well inside these timeouts and
+ # Cloudflare's 100 s idle limit. CORS is answered by the service (https://*.openvibe.* only).
+ location = /realtime/stream {
+ limit_req zone=ovevents_connect burst=20 nodelay;
+ limit_conn ovevents_streams 20;
+ proxy_pass http://127.0.0.1:4300;
+ proxy_http_version 1.1;
+ proxy_set_header Connection "";
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_set_header Last-Event-ID $http_last_event_id;
+ proxy_buffering off;
+ proxy_cache off;
+ gzip off;
+ chunked_transfer_encoding on;
+ proxy_read_timeout 1h;
+ proxy_send_timeout 1h;
+ send_timeout 1h;
+ }
+
+ location = /api/health { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }
+ location = /api/ready { proxy_pass http://127.0.0.1:4300; proxy_set_header Host $host; }
+
+ # ── Service API: host-local only (see header) ───────────
+ location /api/ {
+ return 403;
+ # To expose it (token-guarded) to off-host services instead:
+ # proxy_pass http://127.0.0.1:4300;
+ # proxy_http_version 1.1;
+ # proxy_set_header Host $host;
+ # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ # proxy_set_header X-Forwarded-Proto $scheme;
+ }
+
+ location = / {
+ proxy_pass http://127.0.0.1:4300;
+ proxy_set_header Host $host;
+ }
+
+ location / {
+ return 404;
+ }
+}
new file mode 100644
new file mode 100644
@@ -0,0 +1,34 @@
@@ -0,0 +1,34 @@
+[Unit]
+Description=OpenVibe.Events — durable events, signed delivery, dead letters, replay and the SSE realtime gateway
+# Network (openvibe-network) signs the tokens this service verifies. Events starts without it
+# and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.
+After=network.target
+Wants=openvibe-network.service
+
+[Service]
+Type=simple
+User=ubuntu
+Group=ubuntu
+WorkingDirectory=/opt/openvibe.events
+EnvironmentFile=/etc/openvibe/events.env
+Environment=NODE_ENV=production
+Environment=PORT=4300
+Environment=PATH=/usr/local/bin:/usr/bin:/bin
+ExecStart=/usr/bin/env node /opt/openvibe.events/server/index.js
+Restart=always
+RestartSec=5
+# SIGTERM: stop accepting, let in-flight deliveries finish (they are retried anyway if cut short).
+KillSignal=SIGTERM
+TimeoutStopSec=15
+# Up to REALTIME_MAX_CONNECTIONS long-lived SSE sockets plus delivery sockets.
+LimitNOFILE=16384
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=strict
+ProtectHome=true
+# The code tree stays read-only; the event store lives in /var/lib/openvibe-events.
+StateDirectory=openvibe-events
+Environment=EVENTS_DB_PATH=/var/lib/openvibe-events/events.db
+
+[Install]
+WantedBy=multi-user.target