openvibers/openvibe.media Commits
Media: no per-app webhooks; outcomes are OpenVibe.Events events only (C-75) (#87)
Changes
@@@ -20,7 +20,7 @@ VALKEY_PREFIX=MEDIA_DATA_DIR=./data# Restore-drill mode, set only by `ovhost drill media` (OpenVibe.Host docs/restore-drills.md): reads--# only from a restored copy, no stored bytes, no jobs, webhooks, events or ffmpeg; refuses to start++# only from a restored copy, no stored bytes, no jobs, events or ffmpeg; refuses to start# unless DATABASE_URL names the restored copy (never ov_media on 5432/6432), DATABASE_DIRECT_URL is unset,# HOST is loopback and PORT is not 4100 (server/drill.js).# MEDIA_DRILL=1@@@ -68,8 +68,7 @@ Called elsewhere, as the service principal `media` (the OAuth client `media`):| OpenVibe.Live | `live.lineage.resolve` | which channel and person a recording belongs to || OpenVibe.Events | `events.event.publish`, `events.subscription.manage` | the outbox relay; its subscriptions |--Outbound webhooks to tenants (`webhook_url`, HMAC with `webhook_secret`) stay until every consumer--reads events.++Media announces outcomes as OpenVibe.Events events (`media.vod.ready`, …) and sends no webhooks since 2026-10-11 (C-75).## Layout@@@ -79,7 +78,7 @@ server/config.js env config (PORT, MEDIA_DATA_DIR, *_PATH, DATABASE_URL/DATABASE_DIRECT_URL, MEDIA_B2_*/R2_*,RTP pool, OV_NETWORK_URL)auth.js tenancy + auth middleware, JWKS fetch, app seeding-- webhooks.js HMAC-signed outbound webhooks++ announce.js transactional OpenVibe.Events outcome announcementsdrill.js MEDIA_DRILL: restore-drill mode (reads only, no bytes, no jobs, nothing leaves)db/database.js PostgreSQL helpers (openvibe-sdk/db, async), all app_id-scopeddb/init.js applies migrations/ (DATABASE_URL/DATABASE_DIRECT_URL, or embedded PGlite)@@@ -87,7 +86,7 @@ server/media_objects/locations/relationships/variants/jobs/holds/invariant_violationsvod/recorder.js ffmpeg recording: RTMP pull + RTP (SDP) ingest, codec passthroughvod/media-tools.js probes, seekable remux, DVR sidecar, chunk-segment concat-- vod/finalize.js finalize pipeline (remux → probe → master recovery → thumbnail → webhook)++ vod/finalize.js finalize pipeline (remux → probe → master recovery → thumbnail → event)vod/routes.js /api/v1/:app/vods (+ ingest + chunks)vod/clip-cutter.js re-encoding clip cutter (local file or presigned URL source)vod/clips-routes.js /api/v1/:app/clips@@@ -147,8 +146,7 @@ other OpenVibe services call the API.subject, in every tenant: uploads in progress, copies, derivatives, visibility, lifecycle and their ownusage per tenant and namespace (`GET /api/v2/me/objects`, `/objects/:id`, `/usage`). Read-only: changesstay with the apps. Network staff with `staff.site.view` get the operator views at `/me/ops` (failed--jobs, missing media, projection, tiering, outbound webhooks — who is configured and what this process--has sent — and usage recompute with `staff.site.configure`). Details in++jobs, missing media, projection, tiering and usage recompute with `staff.site.configure`). Details in[docs/object-model.md](docs/object-model.md#object-explorer).## Tenancy & auth@@@ -159,7 +157,7 @@ Tenant = the `:app` path segment. Rows in the `apps` table define tenants:|---|---|| `app_id` | tenant id (`live`, `games`, …) || `api_key_hash` | sha256 of the app API key (never stored in plaintext) |--| `webhook_url` / `webhook_secret` | outbound event delivery (HMAC) |++| `webhook_url` / `webhook_secret` | retained for a separate schema migration; unused for delivery || `allowed_origins` | JSON array; guards browser (user-JWT) calls || `quota_bytes` | storage quota: v1 files + native v2 objects (0 = unlimited) || `project_id` / `env` | set only on developer-project tenants (below) |@@@ -240,7 +238,7 @@ and in `scripts/smoke-test.sh`.| POST | `/vods/:id/ingest/rtp/stop` | finalizes the recording || POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow || POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) |--| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, webhook |++| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, event || GET | `/vods/:id` | `{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness); `hls_url`, with `hls_expires_at` when signed: the object's master playlist with `MEDIA_HLS_ENABLED`, docs/media-fabric.md §3) || GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) || PUT | `/vods/:id` | `{ title?, description?, visibility? }` |@@@ -354,22 +352,17 @@ directories are shared across apps) and responses carry a `note` saying so.| GET | `/admin/storage/ops` | this app's operator report (as `/me/ops` shows it to Network staff): failed jobs, missing media, projection, tiering, the namespaces' usage snapshot. Not for calls acting for a user (403) || POST | `/admin/storage/ops/recompute` | refresh this app's namespaces' usage snapshot from the rows |--### Webhooks (outbound)++### Outcome events--`POST` to the app's `webhook_url` with body--`{ "event": "vod.ready"|"vod.failed"|"clip.ready"|"clip.failed"|"media.object.uploaded"|"storage.alert"|"storage.recovered", "app_id", "data", "event_id"? }`--and header `X-OVMedia-Signature: sha256=<hex hmac-sha256 of the raw body with--the app's webhook_secret>`. 3 attempts with backoff, 10 s timeout.++Media announces outcomes as OpenVibe.Events events (`media.vod.ready`, …) and sends no webhooks since 2026-10-11 (C-75).--Each outcome is also a durable OpenVibe.Events event (`media.vod.ready|failed`,++Each outcome is a durable OpenVibe.Events event (`media.vod.ready|failed`,`media.clip.ready|failed`, `media.object.uploaded`, `media.storage.alert|recovered`;`server/events.js`). The outbox row is written in the **same transaction** as the state--change it describes (`webhooks.announce()`), so an outcome is never lost or announced for a--rolled-back change; the webhook follows the commit and carries `event_id` = that event's id--(absent when the outbox is off), so an app that reads both paths (Live during its webhook →--Events transition) handles each outcome once. Webhooks stay until every consumer reads Events.++change it describes (`announce()`), so an outcome is never lost or announced for a++rolled-back change.--Two object events go to OpenVibe.Events only (no webhook): **`media.object.deleted`**++Two additional object events go to OpenVibe.Events: **`media.object.deleted`**`{ object_id, app_id, kind, legacy_ref, deleted_at }` and **`media.object.visibility_changed`**`{ object_id, app_id, kind, legacy_ref, visibility, previous_visibility, changed_at }` (subject`object <id>`, `important`, times ISO 8601 UTC, `legacy_ref` null for native objects). Nothing@@@ -422,7 +415,7 @@ mode (`server/drill.js`) Media:4100 (before the database is opened), and opens the copy without migrating it;- starts only its HTTP server: no app seeding, JWKS refresh, tiering sweep, health job, junk sweep,clip re-cuts, copy verification, jobs worker, disk guardian, thumbnail cleanup, object purge,-- orphan-recording finalize or Events relay; webhooks are never sent;++ orphan-recording finalize or Events relay;- writes no file and creates no directory (the storage checks leave `/api/ready`, as do the remotetiers; it reports `"mode": "drill"`);- never opens a file path from the database: every byte route (`/v` and `/c` bytes, `/t`, `/a`, `/f`,@@@ -432,7 +425,7 @@ mode (`server/drill.js`) Media:listens on nothing but its port;- answers 403 to every method but GET, HEAD and OPTIONS, and 503 to `/auth/*`.--`test/drill-mode.test.js` boots the real server that way against a copy whose app has a webhook URL++`test/drill-mode.test.js` boots the real server that way against a restored copyand checks each point.## N-1: the previous release against this one@@@ -660,7 +653,7 @@ shutdown.- **Only real files are candidates.** Rows without a file (a recording that was refused or never produced output, legacy paths) are quarantined as `health_status='missing_file'` and excluded — they used to fill the candidate list and starve the drain.- **Cadence** — while a drain is still needed the sweep re-runs every `pressureRetryMs` (2 min), otherwise every `sweepIntervalMs` (15 min). A VOD whose upload failed is skipped for 30 min so one bad file cannot pin the drain.- **Bounded uploads** — each upload is aborted past `uploadTimeoutFloorMs + size / uploadMinThroughputMBps`; a watchdog logs a sweep that has run for hours.--- **Stall alerts** — after `alertAfterStalledPasses` (2) consecutive pressure passes that freed nothing, and whenever the disk is critical, Media sends `storage.alert` to every app webhook (Live logs it and forwards to `OPS_ALERT_WEBHOOK_URL` / the `ops_alert_webhook_url` setting); `storage.recovered` follows once the drain works again. Each pass under pressure logs candidates / uploaded / failed / backed-off / quarantined.++- **Stall alerts** — after `alertAfterStalledPasses` (2) consecutive pressure passes that freed nothing, and whenever the disk is critical, Media queues a `media.storage.alert` event; `media.storage.recovered` follows once the drain works again. Each pass under pressure logs candidates / uploaded / failed / backed-off / quarantined.- **Status** — `GET /api/v1/:app/admin/storage` includes `sweep` (last result, next run, stalled flag, needsDrain/critical).Run `npm test` for the policy regression tests.@@@ -33,7 +33,7 @@"multi-tenant apps and developer-project tenants (sandbox separated)","public serving with presigned URLs; signed access for private and sandbox items","the object explorer (/me) and operator views",-- "media.* events through the SDK outbox, webhooks during the transition",++ "media.* events through the SDK outbox as the only outcome announcement","restore-drill mode and N-1 compatibility tests","per-actor limits (openvibe-sdk/limits)","placement demand rollups on Valkey (F2.4): reads per object × region × 5-minute bucket and, for a timeline segment, per segment too (object total is the sum over its segments; hotness over the hour, media.object.hot once per object per hour)",@@@ -277,7 +277,7 @@ formats (AVIF/WebP), preview clips and AI thumbnails are `rebuildable = true`:only, report only) reads those rows every `MEDIA_MOVE_CLEANUP_MINUTES` (60; `0` = never; not in a restore drill): anobject whose move failed three times in the last 7 days (a later `done` of that move resets the count) raises one`storage.alert` of kind `move_cleanup_failed`, listing the objects (`media_storage_alerts_total{kind}`, the Events-- outbox and the per-app webhook, once per 6 h cooldown like every storage alert).++ outbox, once per 6 h cooldown like every storage alert).- **Dry-run and simulator:** every policy can run in dry-run mode (logs what it would do and what it would cost), and asimulator replays recorded telemetry against a candidate policy before it is enabled.- **Demand rollups (F2.4, shipped):** every served viewer read (`router.route()` for playback or download, not a derive@@@ -87,7 +87,7 @@ Every write to a projected row writes its object in the same PostgreSQL transact(`model.withObject(kind, ids, write)`; roadmap WS-G task 1; compatibility shim C-75 was retired on 2026-10-10). The row change and the re-projection commit together ornot at all, so a crash or an error between them can no longer leave the object behind its row. Awrite whose object cannot be written fails as a whole and changes nothing. Inside--`webhooks.announce()` the object joins the outcome's transaction, and its object events are queued++`announce()` the object joins the outcome's transaction, and its object events are queuedafter the outcome event.| path | where |@@@ -180,7 +180,7 @@ All routes live under `/api/v2/:app/objects`.| POST | `/` | init. `{ namespace (default the tenant's root; a name below it, full or relative), kind, visibility (default private), mime_type, size_bytes, filename, content_hash (expected sha256), metadata (≤ 16 KB; `metadata.class` declares the object's storage class, see [Tiering of native objects](#tiering-of-native-objects)), user_id, upload_ttl (60-86400 s), multipart, part_size }` returns 201 `{ id, object, upload: { method: 'PUT', url, token, expires_at, max_bytes, content_type, complete_url, multipart_url } }`. `url` is a [presigned PUT URL](#presigned-uploads). With `multipart: true` (required above `MEDIA_OBJECT_MAX_MB`): `method: 'multipart'`, `url: null`, and `upload.multipart` is the new [session](#multipart-uploads). Checks at init: `size_bytes` against `MEDIA_OBJECT_MAX_MB` (single part) or `MEDIA_MULTIPART_MAX_MB` (413), the namespace's policy (422 `media.namespace.policy_denied`, 413 `media.object.too_large`), every quota from the namespace up to the tenant including reservations (413 `media.quota.exceeded` / `media.quota.objects_exceeded`; the declared size is then reserved), the public-size invariant (422) and the [content type](#content-types) for the kind (415 `media.object.unsupported_type`) || POST | `/:id/upload-url` | a fresh presigned single-PUT URL for an uploading object (`{ ttl }` 60-86400 s) || PUT | `/:id/content` | the bytes, single part. Auth is the upload token (`?token=` from init, or `X-Upload-Token`) or the usual credential. Streams to disk computing sha256. Refuses more than the declared size or the limit (413; an object declared above `MEDIA_OBJECT_MAX_MB` goes multipart), a size mismatch (400), quota overrun (413), a type that does not suit the kind (415) and a PUT while a multipart session is open (409). Stored at `OBJECTS_PATH/<app>/<id>` with a verified local location. Mounted ahead of the JSON body parser, so any Content-Type is taken as raw bytes; the Content-Type becomes `mime_type` when init set none. Can be repeated while the object is `uploading` |--| POST | `/:id/complete` | checks the expected hash (422 `media.object.hash_mismatch`), the invariant, the quota and the bytes against the type (415 `media.object.content_mismatch`). Sets `ready` and sends the `media.object.uploaded` webhook. Also accepts the upload token |++| POST | `/:id/complete` | checks the expected hash (422 `media.object.hash_mismatch`), the invariant, the quota and the bytes against the type (415 `media.object.content_mismatch`). Sets `ready` and queues the `media.object.uploaded` event. Also accepts the upload token || POST | `/:id/multipart` | start a [multipart session](#multipart-uploads) `{ part_size, size_bytes (when init declared none) }` → 201 || GET | `/:id/multipart/:uploadId` | the session: `parts` received (number, size, sha256), `missing`, `received_bytes`, `expires_at` || PUT | `/:id/multipart/:uploadId/parts/:n` | one part, raw bytes of exactly its size; optional `X-Content-SHA256` |@@@ -416,15 +416,13 @@ cheap and renders in a drill.- `projection`: rows with no object yet; objects whose owner subject is unresolved; whether upload reservations were seeded.- `tiering`: providers and policy; VODs and clips per provider; objects per canonical copy (native or projected); VODs eligible to offload; R2 decisions of the last 24 h and recent refusals and failures; the sweep's state in this process; and `native_objects`: the [object tiering](#tiering-of-native-objects)'s activation gate and policy, native objects in R2, how many are eligible to promote now, its decisions of the last 24 h (dry runs included) and the most recent ones, and its last sweep.- `namespaces`: every namespace with quota and usage snapshot.--- `webhooks`: the apps with a configured outbound webhook (app id and host only — never the URL or secret) and, per app, what this process has sent since it loaded (`sent`, `failed`, the last event and attempt/ok times, the last error). In-memory: counts reset on restart and scope to the app when the report is scoped.## ReadinessCode: `server/objects/readiness.js`. Test: `test/readiness.test.js`.Every object answer (v2 `GET /:id`, the v2 list, the `media.object.uploaded` payload) and the v1 VOD--and clip answers (`GET`, list, update, finalize, upload; not the `vod.*`/`clip.*` webhook payloads,--which are sent before the object is re-projected) carry `readiness`:++and clip answers (`GET`, list, update, finalize, upload) carry `readiness`:```json{ "metadata": true, "bytes_verified": true, "playable": true, "hash_verified": false, "verified_copies": ["local"], "reason": null }@@@ -647,10 +645,9 @@ Code: `server/jobs/`. Tests: `test/jobs.test.js`, `test/jobs-invariant.test.js`,**Events.** Every state change and its `media.job.<transition>` event (`proposed`, `queued`,`started`, `retrying`, `succeeded`, `failed`, `cancelled`; subject `job <id>`) commit in one PostgreSQL--transaction through Media's outbox, the same rule as the outcome events (`webhooks.announce()`): no++transaction through Media's outbox, the same rule as the outcome events (`announce()`): nochange without its event, no event for a change that rolled back. Progress events are `low`--priority; outcomes and proposals are `important`. Job events go to OpenVibe.Events only, not to app--webhooks.++priority; outcomes and proposals are `important`. Job events go to OpenVibe.Events.The payload is the event projection (`queue.jobEvent`, contracts `media.job.<transition>@1`), notthe job as the API answers it:@@@ -704,7 +701,7 @@ job over only with the token it read and, past boot, only while the lease is sti| `invariant.scan` | light | The [size-invariant validator](#public-object-size-invariant): records violations, proposes `object.split` / `object.remux`, withdraws moot proposals. Tenant-wide (no object) || `object.split` | heavy | Stream-copies a vod/clip (or a video/audio object) into parts (`params.parts` 2-1000 or `segment_seconds`), each a new **private** object with a `derived_from` relationship (`job_id`, `part`, `start_seconds`, `duration_seconds`). Cuts land on keyframes, so neighbouring parts can overlap slightly. The source is never changed. Checkpointed per part || `object.remux` | heavy | Stream-copy remux of the whole source (seek index, duration, MP4 faststart) into one new private object; also the source's `remux` variant |--| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips` and `/:id/recut` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` and the webhook come from the cut as before. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording |++| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips` and `/:id/recut` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` events come from the cut. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording || `object.waveform` | heavy | The audio waveform as a PNG (`showwavespic`, 1800×140 by default; `width`, `height`). It is a private `asset` object and the source's `waveform` variant. A remote-only source over `MEDIA_PREVIEW_REMOTE_MAX_MB` (2048) is refused, because it decodes the whole audio track. `no_audio` is permanent || `object.sprite` | heavy | A seek-preview sprite sheet as a JPEG: 100 frames by default, at most one per 2 s, 10 columns and 160 px tiles (`frames`, `columns`, `tile_width`). With `MEDIA_HLS_ENABLED` and a source timeline it is cut from the timeline's rows — each sample time's row decoded from this node's segment or the packed chunk's byte range, checked against the row's sha256 — so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame over the source. It is a private `asset` object and the source's `sprite` variant. The layout goes in its metadata as `sprite: { count, interval_seconds, columns, rows, tile_width, tile_height }`: frame *i* covers [*i*·interval, (*i*+1)·interval) at column *i* mod columns, row ⌊*i*/columns⌋. `no_video` is permanent || `object.hash` | light | sha256 of ready objects' present local copies that have no hash yet, onto the object (`content_hash`, `metadata.hash_basis`) and the local location (`checksum`). Tenant-wide batch (`params { limit (1-500, 50), budget_mb }`) or one object. Scheduled per tenant every `MEDIA_HASH_INTERVAL_MIN`. Result `{ hashed, bytes, skipped, remaining }`. See [Scheduled verification](#scheduled-verification) |nnew file mode 100644@@@ -0,0 +1,31 @@++/**++ * Outcomes are announced as OpenVibe.Events events written in the state change's own++ * transaction. Per-app webhooks are gone (C-75).++ */++'use strict';++++const db = require('./db/database');++++/**++ * Commit an outcome and announce it. `change()` (synchronous DB writes, optional) and the outbox++ * row for `event` run in ONE PostgreSQL transaction, so the durable event exists if and only if the++ * change committed: never lost after a commit, never announced for a rollback. `payload()` is read++ * inside that transaction (the row as committed). After the commit the relay is woken.++ * Returns { data, eventId }; throws (nothing changed, nothing queued) when the transaction fails.++ */++async function announce(appId, event, { change = null, payload }) {++ const events = require('./events');++ let data = null;++ let env = null;++ await db.getDb().tx(async () => {++ if (change) await change();++ data = typeof payload === 'function' ? await payload() : payload;++ env = await events.record(event, appId, data);++ await events.recordObjectChanges(); // a row the change deleted: media.object.deleted, same transaction++ });++ events.kick();++ const eventId = env ? env.event_id : null;++ return { data, eventId };++}++++module.exports = { announce };@@@ -108,7 +108,7 @@ async function storeAvatar({ db, pic, subject = null, username = null, sourceHosconst model = require('../objects/model');const namespaces = require('../objects/namespaces');const invariant = require('../objects/invariant');-- const { announce } = require('../webhooks');++ const { announce } = require('../announce');const app = await db.getApp(AVATAR_APP);if (!app) throw new Error('Picture storage is not configured on this server');const { namespace } = namespaces.resolveName(app, AVATAR_NAMESPACE);@@@ -131,7 +131,7 @@ async function getApp(appId) {}async function listApps() {-- return await all('SELECT app_id, name, webhook_url, allowed_origins, quota_bytes, created_at FROM apps ORDER BY app_id');++ return await all('SELECT app_id, name, allowed_origins, quota_bytes, created_at FROM apps ORDER BY app_id');}async function upsertApp({ app_id, name, api_key, webhook_url, webhook_secret, allowed_origins, quota_bytes }) {@@@ -10,7 +10,7 @@* - assertSafe(): Media refuses to start unless DATABASE_URL is set and does not name production's database* (ov_media on the host's 5432 or PgBouncer's 6432), DATABASE_DIRECT_URL is unset (a drill never migrates),* PORT is set and is not production's 4100, and HOST is loopback. It runs before the database is opened.-- * - installGuards(): no outbound connection (net.Socket#connect, fetch), so no webhook, event, B2/R2++ * - installGuards(): no outbound connection (net.Socket#connect, fetch), so no event, B2/R2* or Network call leaves the process; no program other than git (child_process: no ffmpeg or* ffprobe ever; git only reads the checkout for /release.json); no UDP socket (RTP ingest); nothing* listens but the drill's own HTTP port.@@@ -21,7 +21,7 @@* - server/index.js opens the database and serves HTTP, and starts nothing else: no app seeding, no* JWKS refresh, no health job, junk sweep, tiering sweep, clip re-cuts, copy verification, jobs* worker (thumbnails, split/remux, invariant scans, storage orphan scans), disk guardian, thumbnail cleanup, object purge,-- * orphan-recording finalize or Events relay; webhooks are never sent. Timers that modules++ * orphan-recording finalize or Events relay. Timers that modules* start when they are loaded check `enabled` as well. Readiness does not probe storage (a drill* writes none) or the remote tiers.*@@@ -1,27 +1,25 @@'use strict';/**-- * Media → OpenVibe.Events (roadmap Wave 3, ADR-004): the outcomes Media already announces by-- * per-app webhook are also written as durable events to Media's own `event_outbox` table and-- * relayed to OpenVibe.Events with Media's service token (audience openvibe.events, capability-- * events.event.publish). The webhooks stay as they are until every consumer reads events.++ * Media → OpenVibe.Events (roadmap Wave 3, ADR-004): Media announces outcomes as durable events++ * written to its own `event_outbox` table and relayed to OpenVibe.Events with Media's service++ * token (audience openvibe.events, capability++ * events.event.publish). Events are the only announcement path.** vod.ready | vod.failed → media.vod.ready | media.vod.failed (subject vod <id>)* clip.ready | clip.failed → media.clip.ready | media.clip.failed (subject clip <id>)* media.object.uploaded → media.object.uploaded (subject object <id>)* storage.alert | storage.recovered → media.storage.alert | media.storage.recovered* job state changes (server/jobs/queue.js) → media.job.proposed | queued | started | retrying |-- * succeeded | failed | cancelled (subject job <id>; Events only, no webhook;++ * succeeded | failed | cancelled (subject job <id>;* payload queue.jobEvent, not the tenant's params/result/error)* an object deleted / its visibility changed → media.object.deleted | media.object.visibility_changed-- * (subject object <id>; Events only; payload object id, tenant,++ * (subject object <id>; payload object id, tenant,* kind, legacy_ref and the times/visibilities, nothing else)** The outbox row is written INSIDE the PostgreSQL transaction that makes the state change it describes-- * (record(), called from webhooks.announce()): the event exists if and only if the change committed,-- * so a crash can neither lose an outcome nor announce one that rolled back. The webhook for the same-- * outcome is sent after the commit and carries the envelope's event_id, so a consumer receiving it by-- * both paths (Live during its webhook → Events transition) handles it once. Delivery from the outbox-- * on is at least once. Every event is `internal` visibility: a VOD's own visibility decides who may++ * (record(), called from announce()): the event exists if and only if the change committed,++ * so a crash can neither lose an outcome nor announce one that rolled back. Delivery from the outbox++ * is at least once. Every event is `internal` visibility: a VOD's own visibility decides who may* see it, and consumers (Live, Search) apply it. Lifecycle outcomes are `important` (roadmap §6.3);* only storage.recovered is `low`.*@@@ -42,7 +40,7 @@ const TYPES = {'storage.recovered': ['media.storage.recovered', 'storage'],};--// Placement decisions have no webhook twin; callers record them in the outbox transaction.++// Placement decisions are recorded in the outbox transaction.const PLACEMENT_TYPES = new Set(['media.replica.requested', 'media.replica.ready', 'media.replica.draining', 'media.replica.evicted','media.object.hot', 'media.region.hot', 'media.delivery.surge', 'media.delivery.degraded',@@@ -111,10 +109,10 @@ function slim(appId, data) {* Queue the durable event for an outcome. MUST run inside the transaction that makes the change* (the SDK outbox refuses otherwise), and throws if the insert fails, so the change rolls back with* it. Returns the envelope (with event_id), or null when the outbox is off, the tenant is a sandbox-- * or the webhook event has no durable twin.++ * or the outcome has no mapped event type.*/--async function record(webhookEvent, appId, data) {-- const map = TYPES[webhookEvent];++async function record(outcome, appId, data) {++ const map = TYPES[outcome];if (!map) return null;const [eventType, subjectType] = map;const id = data && (data.object_id || data.id);@@@ -3,7 +3,7 @@** Multi-tenant media service: VOD ingest/recording (RTMP pull, RTP, browser* chunks), local/B2/R2 storage tiering with presigned-302 playback, clips,-- * files, thumbnails, legacy screenshot redirects, and HMAC-signed webhooks. See CONTRACTS.md++ * files, thumbnails, legacy screenshot redirects, and OpenVibe.Events announcements. See CONTRACTS.md* (Media API v1) and README.md.*/'use strict';@@@ -338,7 +338,7 @@ const ready = (async () => {console.log(`[Media] OpenVibe.Media listening on ${config.host}:${config.port} (${config.nodeEnv})`);console.log(`[Media] Data: db=${config.db.url ? 'postgresql' : 'pglite'} vods=${config.vod.path} clips=${config.vod.clipsPath}`);console.log(`[Media] RTP ingest pool: udp ${config.rtp.portMin}-${config.rtp.portMax} (127.0.0.1)`);-- // Durable events (roadmap Wave 3): webhook outcomes also go to OpenVibe.Events. Off without EVENTS_URL.++ // Durable outcome events (roadmap Wave 3) go to OpenVibe.Events. Off without EVENTS_URL.try { require('./events').init(); } catch (err) { console.warn('[Events] not started:', err.message); }// Media's own watch pages in OpenVibe.Search (WS-O task 10): media.index_document.* through the same outbox.require('./public/search-documents').start().catch((err) => console.warn('[Search] documents not started:', err.message));@@@ -5,7 +5,7 @@* in media_object_tier_decisions. This job reads those rows: an object whose move failed FAIL_THRESHOLD times* within the lookback (a later `done` of the same move starts the count over) raises one storage.alert of kind* move_cleanup_failed, which vod-storage's emitStorageEvent counts in media_storage_alerts_total{kind} and-- * delivers as the Events outbox row and per-app webhook. Report only: it never moves, deletes or changes a row.++ * delivers as an Events outbox row. Report only: it never moves, deletes or changes a row.** params {} service-wide; runs only under queue.SYSTEM_APP (no tenant can enqueue it)* result { scanned, alerted, objects } objects = the first 50 (object_id, action, fails, first_at, last_at, last_error)@@@ -9,8 +9,8 @@* succeeded | failed | cancelled finished** Every state change and its media.job.<transition> event commit in ONE PostgreSQL transaction (the same-- * rule as webhooks.announce(), see events.js): an event exists if and only if its change committed.-- * The relay is woken after the commit. Job events go to OpenVibe.Events only (no app webhook).++ * rule as announce(), see events.js): an event exists if and only if its change committed.++ * The relay is woken after the commit. Job events go to OpenVibe.Events.** Fencing: a claim writes a fresh random lease_token on the row and hands it to its holder. Renewing* the lease, saving a checkpoint and every way out of 'running' (succeed, fail, markCancelled) match@@@ -3,7 +3,7 @@* docs/object-model.md#operator-views).** report({ appId, limit }) failed jobs, missing media, projection status, tiering diagnostics, the-- * namespaces' usage snapshot and the outbound-webhooks section; appId++ * namespaces' usage snapshot; appId* narrows everything that has a tenant (the tiering sweep and the provider* switches are service-wide)* recompute({ appId }) refresh the namespaces' usage snapshot from the rows (namespaces.reconcile),@@@ -19,7 +19,6 @@const db = require('../db/database');const namespaces = require('../objects/namespaces');const copyReport = require('../objects/copy-report');--const webhookStats = require('../webhooks');const iso = require('./explorer').iso;const clip = (s, n) => (s == null ? null : String(s).slice(0, n));@@@ -198,29 +197,6 @@ async function namespaceSnapshot(appId) {})));}--// ── Webhooks ─────────────────────────────────────────────────----/** Who is configured to receive outbound webhooks (host only, never the URL or secret) and what this-- * process has sent them since it loaded. */--async function webhooks(appId) {-- const s = scoped(appId);-- const configured = (await db.all(`SELECT app_id, webhook_url FROM apps WHERE COALESCE(webhook_url, '') != ''${s.sql} ORDER BY app_id`, s.params))-- .map((r) => {-- let host = null;-- try { host = new URL(r.webhook_url).host; } catch { /* an unparseable URL shows as null */ }-- return { app_id: r.app_id, host };-- });-- const all = webhookStats.stats();-- const sends = appId-- ? { since: all.since, by_app: all.by_app[appId] ? { [appId]: all.by_app[appId] } : {} }-- : all;-- return {-- configured,-- sends,-- note: 'Outbound webhooks (C-83) retire when every configured app reads OpenVibe.Events; sends counts reset on restart.',-- };--}--/** The whole operator report (see the header). `limit` bounds each list (1-200, default 50). */async function report({ appId = null, limit = 50 } = {}) {const n = Math.min(Math.max(parseInt(limit, 10) || 50, 1), 200);@@@ -233,7 +209,6 @@ async function report({ appId = null, limit = 50 } = {}) {projection: await projection(appId),tiering: await tiering(appId, n),namespaces: await namespaceSnapshot(appId),-- webhooks: await webhooks(appId),};}@@@ -430,17 +430,8 @@ function renderOps({ person, report, canRecompute, recomputed = null }) {])) || '<p class="empty">None.</p>'}` : ''}</section>-- <section aria-labelledby="webhooks-h">-- <h2 id="webhooks-h">Outbound webhooks</h2>-- <p class="hint">${esc(r.webhooks.note)}</p>-- <p class="hint">Sends counted since ${esc(fmtWhen(r.webhooks.sends.since))} in this process.</p>-- ${table('Apps configured to receive webhooks, host only', ['Tenant', 'Host', 'Sent', 'Failed', 'Last success', 'Last error'], r.webhooks.configured.map(c => {-- const s = r.webhooks.sends.by_app[c.app_id] || {};-- return [esc(c.app_id), esc(c.host || 'unparseable URL'), num(s.sent || 0), num(s.failed || 0), esc(fmtWhen(s.last_ok_at) || 'never'), s.last_error ? `<pre class="err">${esc(s.last_error)}</pre>` : ''];-- }), { numeric: [2, 3] }) || '<p class="empty">No app is configured to receive webhooks.</p>'}-- </section><p class="hint">JSON: <a href="/api/v2/me/ops${scopeQs}">/api/v2/me/ops</a>.</p>`;-- return doc({ title: 'Media operations', description: 'Operator views of OpenVibe.Media: failed jobs, usage, missing media, projection, tiering and webhooks.', body });++ return doc({ title: 'Media operations', description: 'Operator views of OpenVibe.Media: failed jobs, usage, missing media, projection and tiering.', body });}module.exports = {@@@ -448,7 +448,7 @@ async function syncThumbnail(parentRow, parentObjectId, parentKind) {/*** Projection work in one transaction, with the object-change events it caused (server/events.js).-- * Nested in an outer transaction (webhooks.announce, another withObject) it is a savepoint, and the++ * Nested in an outer transaction (announce, another withObject) it is a savepoint, and the* outer one records the changes at its end, in the same transaction and after its own outcome event,* the order they had when the object was synced after the commit.*/@@@ -494,7 +494,7 @@ async function safeSync(kind, id) {* an object behind its row. kind vod|clip|file; ids the row id (a file's key), a list of them,* or a function of write()'s result that gives them (an INSERT's lastInsertRowid). write() must be* database work (awaited). Returns write()'s result; throws, with nothing written, when either-- * part fails. Inside an outer transaction (webhooks.announce) it is a savepoint of that one.++ * part fails. Inside an outer transaction (announce) it is a savepoint of that one.** The projection stats local files (sizes, present/missing copies) but never reads them: content* hashes stay with the content-hash job and remote copies stay 'pending' until reconciliation or a@@@ -58,7 +58,7 @@ const multipart = require('./multipart');const namespaces = require('./namespaces');const { contentDisposition } = require('./disposition');const { tenantAuth, tenantCors, tenantPath, namespaceGrant } = require('../auth');--const { announce } = require('../webhooks');++const { announce } = require('../announce');const { limits } = require('../actor-limits');const MB = 1024 * 1024;@@@ -401,7 +401,7 @@ router.post('/', upload, limits('media.object.upload', UPLOAD), async (req, res)/*** The checks every upload ends with (single PUT and multipart): expected sha256, the public-size* invariant, the quota, and whether the bytes are what their type says; then the ready transition,-- * its invariant row and the media.object.uploaded event commit together (the webhook follows).++ * its invariant row and the media.object.uploaded event commit together.*/async function completeUpload(req, res, obj, { expectedHash = null } = {}) {const loc = (await model.listLocations(obj.id)).find(l => l.provider === 'local' && l.state === 'present');@@@ -9,7 +9,7 @@* local storage directory accepts a write. Optional (one capability degrades, the rest keeps* working): the Network public key (user-JWT and service-token routes; app-key tenants are* unaffected), each CONFIGURED remote tier (B2 cold storage, R2 cache; a HeadBucket at most once a-- * minute), the Events outbox (webhooks still deliver when it backs up), and object_copies: every++ * minute), the Events outbox, and object_copies: every* ready object has a good copy somewhere (server/objects/verify-job.js records it; objects with no* good copy are listed by scripts/no-good-copy-report.js). object_copies never fails readiness —* Media keeps serving everything else.@@@ -145,7 +145,7 @@ function createMediaReadiness({ release, db, config, auth, recorder, events, remchecks.push({ name: `remote_${name}`, required: false, cacheMs: 60 * 1000, timeoutMs: 5000, check: () => remote.probe(name) });}checks.push({-- name: 'events_outbox', required: false, description: 'durable events to OpenVibe.Events (webhooks are separate)',++ name: 'events_outbox', required: false, description: 'durable outcome events to OpenVibe.Events',check: async () => {const s = await events.status();if (!s.enabled) return skip('relay off (EVENTS_URL or the service credentials unset)', { enabled: false }); // verified nothing: never ok (WS-Q task 7)@@@ -67,7 +67,7 @@ function bind(registry) {});alerts = registry.counter({name: 'media_storage_alerts_total',-- help: 'Storage alerts emitted to /metrics, mirroring the storage.alert webhook kinds',++ help: 'Storage alerts emitted to /metrics, mirroring the storage.alert event kinds',labelNames: ['kind'],});@@@ -97,4 +97,4 @@ function observeLatency(provider, op, ms) { if (latency && ms > 0) latency.obserfunction observeError(provider, op) { errors && errors.inc({ provider, op }, 1); }function alert(kind) { alerts && alerts.inc({ kind }, 1); }--module.exports = { bind, inc, observeLatency, observeError, alert, _reset: () => { latency = errors = breakerGauge = decisions = presign = alerts = demandSource = null; } };\\ No newline at end of file++module.exports = { bind, inc, observeLatency, observeError, alert, _reset: () => { latency = errors = breakerGauge = decisions = presign = alerts = demandSource = null; } };@@@ -3,7 +3,7 @@** recutClip(clipId, { reason }) the ONE code path that (re)cuts an existing clip row:* resolves the source, cuts, updates the row, thumbnails,-- * commits the outcome with its event, fires the webhook. Records cut_error / cut_attempts and++ * commits the outcome with its event. Records cut_error / cut_attempts and* schedules an automatic retry on failure.* spec the media job clip.cut (lane clips; roadmap WS-G task 3): a new clip's* cut and every re-cut run as a job, so they carry media.job.* events and a@@@ -31,7 +31,7 @@ const db = require('../db/database');const config = require('../config');const cutter = require('./clip-cutter');const vodStorage = require('./vod-storage');--const { announce } = require('../webhooks');++const { announce } = require('../announce');const objects = () => require('../objects/model');const MAX_ATTEMPTS = 4;@@@ -113,7 +113,7 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false, job = nullconst cut = await cutter.cutClipFile({ source: source.value, startTime, duration });if (!cut.ok) return await fail(clipId, clip, attempt, cut.error, { viaJob });// Thumbnail first (the clip.ready payload carries it), then the ready transition, the clip's-- // object and its event in one transaction (webhooks.announce), then the webhook.++ // object and its event in one transaction (announce).try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }await announce(clip.app_id, 'clip.ready', {change: async () => {@@@ -205,7 +205,7 @@ async function timelineWindow(vod, startTime, endTime) {return { objectId: obj.id, endTime: startTime + Number(last.end_ms) / 1000 };}--/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction, then the webhook. */++/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction. */async function makeVirtual(clipId, win) {const clip = await db.getClipById(clipId);if (!clip) return { ok: false, error: 'Clip not found' };@@@ -3,7 +3,7 @@** POST / { vod_id, start_s, end_s, title?, user_id? } → { id, status }* The cut runs in the background (works on the local file or a-- * presigned B2/R2 URL); webhook clip.ready|clip.failed fires on++ * presigned B2/R2 URL); clip.ready|clip.failed events are queued on* completion. Also accepts a live cut against a still-recording* VOD (clamped to flushed footage, like the predecessor).* GET /:id clip meta (status: processing | ready | failed)@@@ -143,8 +143,8 @@ const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].incl// ── Create clip ──────────────────────────────────────────────// JSON body: a window of a VOD. Over a source with a CMAF timeline (MEDIA_HLS_ENABLED) the clip is virtual: ready at--// once (201, clip.ready webhook), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).--// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut + webhook).++// once (201, clip.ready event), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).++// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut event).// Multipart `video`: direct upload of an already-cut blob (201, ready) —// inherited browser-MediaRecorder clip path.router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {@@@ -241,7 +241,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.-- // clip.ready / clip.failed and the webhook come from the cut itself, as before.++ // clip.ready / clip.failed events come from the cut itself.const { job } = await clipJobs.enqueueCut(appId, clipId, {reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,});@@@ -257,7 +257,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing/*** Direct clip upload: the blob is already cut client-side — remux it for* seeking (rebases cluster timestamps hours from zero on long streams),-- * probe the real duration, store as ready. No webhook (nothing async).++ * probe the real duration, store as ready. There is no asynchronous outcome.*/async function _createUploadedClip(req, res) {const body = req.body || {};@@@ -5,7 +5,7 @@* vodId (there is no stream registry here — apps own stream state):* merge pending chunk segments → seekable remux → probe → master recovery* (truncated-webm rebuild) → thumbnail → DB update + vod.ready event (one-- * transaction) → webhook vod.ready.++ * transaction).** Guarded against double-invocation; refuses to finalize while the recorder* still holds the file open (stops it gracefully instead — its ffmpeg exit@@@ -19,7 +19,7 @@ const { spawn } = require('child_process');const db = require('../db/database');const config = require('../config');const tools = require('./media-tools');--const { announce } = require('../webhooks');++const { announce } = require('../announce');const objects = () => require('../objects/model');/** Absolute-ize a stored Media-relative URL (/t/…, /api/thumbnails/…). */@@@ -35,9 +35,9 @@ function isFinalizing(vodId) {return _finalizing.has(Number(vodId));}--// Public shape of a vod row for API responses + webhooks. { readiness: true } (API responses) adds the--// object's readiness levels (server/objects/readiness.js); webhook payloads leave them out (their shape--// predates the object model and stays as it is). `hls` selects the playlist fields: undefined = discover++// Public shape of a vod row for API responses + events. { readiness: true } (API responses) adds the++// object's readiness levels (server/objects/readiness.js); event payloads leave them out.++// `hls` selects the playlist fields: undefined = discover// them for this row (a single-item read), false = omit them (an event payload must never carry a signed// URL into the outbox), an object = the answer the list route already batched (objects/hls.discoveryMany).async function vodPublic(vod, { readiness = false, hls } = {}) {@@@ -119,8 +119,8 @@ function rebuildWebmFromMaster(masterPath, webmPath) {}/**-- * The state change for a VOD outcome, the VOD's object and the event/webhook, committed together-- * (webhooks.announce around objects.withObject). `row()` gives the row the payload describes (read++ * The state change for a VOD outcome, the VOD's object and the event commit together++ * (announce around objects.withObject). `row()` gives the row the payload describes (read* inside the transaction). Returns the payload, or null when the transaction failed (then nothing* changed and nothing was announced). A deleted row's object is marked deleted by its trigger.*/@@@ -381,8 +381,8 @@ async function _doFinalize(vodId, opts) {// recording is back to the visibility its owner chose.const lift = _finalizeQuarantined(await db.getVodById(vodId) || vod);-- // The ready transition, the VOD's object and the vod.ready event commit together (then the-- // webhook goes out). A failed commit throws, as the plain UPDATE did before.++ // The ready transition, the VOD's object and the vod.ready event commit together.++ // A failed commit throws, as the plain UPDATE did before.await announce(vod.app_id, 'vod.ready', {change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),@@@ -269,7 +269,7 @@ class StreamRecorder {ffmpegCorrupted: rec?._ffmpegCorrupted || false,};this._teardown(vodId);-- // Let finalizeVod handle remux, probe, thumbnail, webhook.++ // Let finalizeVod handle remux, probe, thumbnail and outcome event.// Short delay to ensure the file is fully flushed to disk.setTimeout(() => {require('./finalize').finalizeVod(vodId, finalizeOpts).catch(err => {@@@ -974,9 +974,7 @@ const sweepState = {const alertLastSentAt = new Map(); // kind → epoch ms/**-- * Tell every app with a webhook that storage needs a human (or that it recovered).-- * Delivered as the regular Media→app webhook (`storage.alert` / `storage.recovered`),-- * so Live can log it loudly and page whatever ops channel it has configured.++ * Announce that storage needs a human (or that it recovered) through OpenVibe.Events.*/async function emitStorageEvent(event, kind, data, settings = getSettings()) {const cooldownKey = `${event}:${kind}`;@@@ -989,19 +987,13 @@ async function emitStorageEvent(event, kind, data, settings = getSettings()) {const payload = { kind, ...data, at: new Date().toISOString() };(event === 'storage.alert' ? console.error : console.warn)(`[VodStorage] ${event} (${kind}): ${JSON.stringify(data)}`);// No database state changes here (the cooldown is in memory): the outbox row is the whole-- // commit, queued once for every app; each app's webhook carries its event_id.-- let eventId = null;++ // commit, queued once for all consumers.++ // The alert fired once it passed the cooldown (counted, logged); the event is best effort, as before.try {const events = require('../events');-- const env = await db.getDb().tx(async () => await events.record(event, null, payload));-- eventId = env ? env.event_id : null;++ await db.getDb().tx(async () => await events.record(event, null, payload));events.kick();} catch (err) { console.warn(`[VodStorage] ${event} not queued for Events:`, err.message); }-- let apps = [];-- try { apps = (await db.listApps()).filter(a => a.webhook_url); } catch { apps = []; }-- let webhooks;-- try { webhooks = require('../webhooks'); } catch { return false; }-- await Promise.all(apps.map(app => webhooks.sendWebhook(app.app_id, event, payload, { eventId }).catch(() => false)));return true;}@@@ -1095,7 +1087,7 @@ async function runSweep() {// Stall detection: a pressure pass that freed nothing is the failure mode that// went unnoticed for weeks. Count them, and after a couple in a row raise an-- // alert through the app webhooks; announce recovery once the drain works again.++ // alert; announce recovery once the drain works again.const diskAfter = diskUsage(config.vod.path);const stillNeedsDrain = underPressure && !drainSatisfied(diskAfter, settings);if (underPressure && !migrated) {ddeleted file mode 100644@@@ -1,138 +0,0 @@--/**-- * OpenVibe.Media — Outbound webhooks-- *-- * POSTs { event, app_id, data } to the app's configured webhook_url with-- * `X-OVMedia-Signature: sha256=<hmac>` — HMAC-SHA256 of the raw body using the-- * app's webhook_secret (per CONTRACTS.md). Fire-and-forget with small retries.-- *-- * Events: vod.ready | vod.failed | clip.ready | clip.failed | media.object.uploaded |-- * storage.alert | storage.recovered-- *-- * Outcomes go through announce(): the state change and its durable OpenVibe.Events outbox row-- * commit in one PostgreSQL transaction, then the webhook is sent with `event_id` = that event's id-- * (absent when the outbox is off), so a consumer that also reads Events dedupes the pair.-- */--'use strict';----const crypto = require('crypto');--const db = require('./db/database');----const MAX_ATTEMPTS = 3;--const RETRY_BASE_MS = 3000;--const TIMEOUT_MS = 10000;----// In-memory tally of what this process actually tried to send, per app (C-83: who still gets webhooks--// is what decides when the outbound path can retire). Never holds the URL, secret, body or signature.--const SINCE = new Date().toISOString();--const tallies = new Map();----function tallyFor(appId) {-- let t = tallies.get(appId);-- if (!t) { t = { sent: 0, failed: 0, last_event: null, last_attempt_at: null, last_ok_at: null, last_error: null }; tallies.set(appId, t); }-- return t;--}----/** Fresh copy of the in-process tallies: { since, by_app: { [app_id]: {...} } }. */--function stats() {-- const by_app = {};-- for (const [appId, t] of tallies) by_app[appId] = { ...t };-- return { since: SINCE, by_app };--}----/** Test-only: forget every tally. */--function _resetStats() { tallies.clear(); }----// Test-only: the base of the retry wait (null restores RETRY_BASE_MS), so a test never patches the--// global setTimeout the database pool's own timers use.--let retryBaseMs = RETRY_BASE_MS;--function _setRetryBaseMs(ms) { retryBaseMs = ms == null ? RETRY_BASE_MS : ms; }----function sign(secret, rawBody) {-- return 'sha256=' + crypto.createHmac('sha256', String(secret || '')).update(rawBody).digest('hex');--}----async function _post(url, rawBody, signature) {-- const res = await fetch(url, {-- method: 'POST',-- headers: {-- 'Content-Type': 'application/json',-- 'X-OVMedia-Signature': signature,-- },-- body: rawBody,-- signal: AbortSignal.timeout(TIMEOUT_MS),-- });-- if (!res.ok) throw new Error(`HTTP ${res.status}`);--}----/**-- * Send an event to an app's webhook. Silently no-ops when the app has no-- * webhook_url. Never throws. Queues nothing: the durable event is written by-- * announce() (or the caller) inside the state change's transaction.-- * @param {string|object} appOrId app_id or apps row-- * @param {string} event e.g. 'vod.ready'-- * @param {object} data event payload-- * @param {object} [opts]-- * @param {string} [opts.eventId] the OpenVibe.Events event_id of the same outcome-- */--async function sendWebhook(appOrId, event, data, { eventId = null } = {}) {-- // A restore drill (MEDIA_DRILL) never calls an app: the URLs in its apps table are production's.-- if (require('./drill').enabled) return false;-- let app = appOrId;-- if (typeof appOrId === 'string') {-- try { app = await db.getApp(appOrId); } catch { app = null; }-- }-- if (!app || !app.webhook_url) return false;---- const body = { event, app_id: app.app_id, data };-- if (eventId) body.event_id = eventId;-- const rawBody = JSON.stringify(body);-- const signature = sign(app.webhook_secret, rawBody);---- const t = tallyFor(app.app_id);-- for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {-- t.last_event = event;-- t.last_attempt_at = new Date().toISOString();-- try {-- await _post(app.webhook_url, rawBody, signature);-- t.sent++;-- t.last_ok_at = new Date().toISOString();-- t.last_error = null;-- return true;-- } catch (err) {-- if (attempt === MAX_ATTEMPTS) {-- t.failed++;-- t.last_error = String((err && err.message) || err).slice(0, 200);-- console.warn(`[Webhooks] ${event} → ${app.app_id} failed after ${MAX_ATTEMPTS} attempts: ${err.message}`);-- return false;-- }-- await new Promise(r => setTimeout(r, retryBaseMs * attempt));-- }-- }-- return false;--}----/**-- * Commit an outcome and announce it. `change()` (synchronous DB writes, optional) and the outbox-- * row for `event` run in ONE PostgreSQL transaction, so the durable event exists if and only if the-- * change committed: never lost after a commit, never announced for a rollback. `payload()` is read-- * inside that transaction (the row as committed). After the commit the relay is woken and the-- * app's webhook is sent with the same event_id. Returns { data, eventId }; throws (nothing changed,-- * nothing queued, no webhook) when the transaction fails.-- */--async function announce(appId, event, { change = null, payload }) {-- const events = require('./events');-- let data = null;-- let env = null;-- await db.getDb().tx(async () => {-- if (change) await change();-- data = typeof payload === 'function' ? await payload() : payload;-- env = await events.record(event, appId, data);-- await events.recordObjectChanges(); // a row the change deleted: media.object.deleted, same transaction-- });-- events.kick();-- const eventId = env ? env.event_id : null;-- sendWebhook(appId, event, data, { eventId }).catch(() => {});-- return { data, eventId };--}----module.exports = { sendWebhook, announce, sign, stats, _resetStats, _setRetryBaseMs };@@@ -7,10 +7,10 @@* DATABASE_DIRECT_URL is set, when HOST is not loopback or PORT is production's 4100, before the* database is opened (a real* `node server/index.js`, in a child process).-- * - Booted for real (server/index.js, in this process) on a copy whose `live` app has a webhook-- * URL, with production-like settings (EVENTS_URL + client secret, B2/R2, app seeding, VOD_PATH …):++ * - Booted for real (server/index.js, in this process) on a copy with production-like settings++ * (EVENTS_URL + client secret, B2/R2, app seeding, VOD_PATH …):* one listener, its HTTP port on 127.0.0.1; no setInterval and no long timer from Media's code;-- * no program but git (no ffmpeg/ffprobe); no outbound connection; no webhook reaches the app;++ * no program but git (no ffmpeg/ffprobe); no outbound connection;* nothing written or created outside the copy's directory.* - The endpoints the Host inventory compares (/release.json, the /browse index) and the watch pages* render from the copy; every route that would stream a stored file answers 503 without opening@@@ -105,11 +105,10 @@ const { spawnSync, execFileSync } = require('child_process');});// ── Production's database and files (a normal-mode process writes them) ──-- // The `live` app's webhook goes to a sink here: a drill must never call it.++ // A local sink helps verify the drill's connection guard.const sinkHits = [];const sink = http.createServer((req, res) => { sinkHits.push(`${req.method} ${req.url}`); res.end('ok'); });await new Promise((r) => sink.listen(0, '127.0.0.1', r));-- const SINK = `http://127.0.0.1:${sink.address().port}/media-webhook`;for (const d of Object.values(P)) fs.mkdirSync(d, { recursive: true });const bytes = { vod: path.join(P.vods, 'vod-1.mp4'), rec: path.join(P.vods, 'vod-2.mp4'), clip: path.join(P.clips, 'clip-1.mp4'), asset: path.join(P.assets, 'wave.png'), shot: path.join(P.pastes, 'shot-1.png'), file: path.join(P.files, 'live', 'f1.txt') };fs.mkdirSync(path.dirname(bytes.file), { recursive: true });@@@ -133,7 +132,7 @@ const { spawnSync, execFileSync } = require('child_process');// The restored copy's rows (written before the drill boots, as production wrote them).const db = require('../server/db/database');await db.initDb();-- await db.run("INSERT INTO apps (app_id, name, api_key_hash, webhook_url, webhook_secret) VALUES ('live', 'OpenVibe.Live', ?, ?, 'whsec')", [require('crypto').createHash('sha256').update('k'.repeat(40)).digest('hex'), SINK]);++ await db.run("INSERT INTO apps (app_id, name, api_key_hash) VALUES ('live', 'OpenVibe.Live', ?)", [require('crypto').createHash('sha256').update('k'.repeat(40)).digest('hex')]);await db.run("INSERT INTO vods (id, app_id, title, file_path, thumbnail_url, duration_seconds, is_public, visibility) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 'Restored VOD', ?, '/t/vod-1-1.jpg', 60, 1, 'public')", [bytes.vod]);await db.run("INSERT INTO vods (id, app_id, title, file_path, is_public, visibility, is_recording) OVERRIDING SYSTEM VALUE VALUES (2, 'live', 'Still recording', ?, 1, 'public', 1)", [bytes.rec]);await db.run("INSERT INTO clips (id, app_id, vod_id, title, file_path, duration_seconds, status) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 1, 'Restored clip', ?, 10, 'ready')", [bytes.clip]);@@@ -206,7 +205,7 @@ const { spawnSync, execFileSync } = require('child_process');assert.deepStrictEqual(spied.intervals, [], JSON.stringify(spied.intervals, null, 1));assert.deepStrictEqual(spied.longTimeouts, [], JSON.stringify(spied.longTimeouts, null, 1));});-- await check('no program but git ran, nothing tried to leave, no webhook reached the app', () => {++ await check('no program but git ran, nothing tried to leave', () => {assert.ok(spied.programs.every((p) => path.basename(p) === 'git'), spied.programs.join(', '));assert.deepStrictEqual(drill.blocked, []);assert.deepStrictEqual(sinkHits, []);@@@ -272,13 +271,12 @@ const { spawnSync, execFileSync } = require('child_process');assert.deepStrictEqual(drill.blocked, [], 'nothing tried to leave');assert.deepStrictEqual(await counts(), countsBefore);});-- await check('webhooks and the Events relay stay off even when called', async () => {-- assert.strictEqual(await require('../server/webhooks').sendWebhook('live', 'vod.ready', { id: 1 }), false);++ await check('the Events relay stays off even when called', async () => {assert.strictEqual(require('../server/events').init(), null);assert.deepStrictEqual(sinkHits, []);});await check('the guards refuse a connection, ffmpeg, a listener and a UDP socket', async () => {-- await assert.rejects(fetch('https://openvibe.live/internal/media-webhook'), /fetch failed/);++ await assert.rejects(fetch('https://openvibe.live/internal/probe'), /fetch failed/);const sock = net.connect({ host: '127.0.0.1', port: sink.address().port });const err = await new Promise((resolve) => sock.on('error', resolve));assert.strictEqual(err.code, 'ECONNREFUSED');@@@ -41,7 +41,7 @@ const http = require('http');const db = require('../server/db/database');db.getDb();const events = require('../server/events');-- const { sendWebhook, announce } = require('../server/webhooks');++ const { announce } = require('../server/announce');const inTx = async (fn) => await db.getDb().tx(fn);assert.strictEqual(events.init({ eventsUrl: '', clientSecret: '' }), null, 'off without EVENTS_URL');@@@ -50,12 +50,10 @@ const http = require('http');const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 50 });assert.ok(outbox);-- // An app with no webhook URL still gets the durable event.++ // An outcome gets the durable event.await announce('no-such-app', 'vod.ready', { payload: { id: 42, title: 'Stream', meta: { big: 'x'.repeat(100000) }, ai_overview: 'long text' } });await announce('no-such-app', 'clip.failed', { payload: { id: 7, status: 'failed' } });assert.strictEqual((await announce('no-such-app', 'unknown.event', { payload: { id: 1 } })).eventId, null, 'no durable twin');-- // A bare webhook queues nothing (the event is written inside the state change's transaction).-- await sendWebhook('no-such-app', 'vod.ready', { id: 43 });await outbox.flush();assert.strictEqual(published.length, 2);assert.strictEqual(published[0].event_type, 'media.vod.ready');@@@ -68,10 +66,7 @@ const http = require('http');assert.strictEqual(published[1].event_type, 'media.clip.failed');assert.strictEqual(published[1].priority, 'important');-- // Storage alerts: queued once by the storage module, never per app by sendWebhook.-- await sendWebhook('no-such-app', 'storage.alert', { kind: 'disk' });-- await outbox.flush();-- assert.strictEqual(published.length, 2);++ // Storage alerts are queued once for all consumers.await inTx(async () => await events.record('storage.alert', null, { kind: 'disk', free_gb: 3 }));await outbox.flush();assert.strictEqual(published.length, 3);@@@ -26,14 +26,12 @@ const http = require('http');process.env.MEDIA_INVARIANT_SCAN_HOURS = '0';-- const hooks = [];const stub = http.createServer((req, res) => {let body = '';req.on('data', (c) => { body += c; });req.on('end', () => {res.setHeader('Content-Type', 'application/json');if (req.url === '/oauth/token') return res.end(JSON.stringify({ access_token: 'tok', token_type: 'Bearer', expires_in: 300 }));-- if (req.url === '/hook') { hooks.push(JSON.parse(body)); return res.end('{}'); }res.statusCode = 404; res.end('{}');});});@@@ -52,7 +50,7 @@ const http = require('http');const raw = db.getDb();-- await db.upsertApp({ app_id: 'live', api_key: 'live-key', webhook_url: `${base}/hook`, webhook_secret: 's' });++ await db.upsertApp({ app_id: 'live', api_key: 'live-key' });await db.upsertApp({ app_id: 'games', api_key: 'games-key' });const events = require('../server/events');const relay = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });@@@ -123,7 +121,6 @@ const http = require('http');await assert.rejects(async () => await queue.cancel(q2.job.id, { by: 'x' }), /outbox insert failed/);failOutbox(false);assert.strictEqual((await queue.get(q2.job.id)).status, 'queued', 'a cancel whose event cannot be written changes nothing');-- assert.ok(hooks.length === 0, 'job events go to Events only, not the app webhook');console.log('✅ job state changes commit with their media.job.* outbox row, or not at all');// ── 3. Idempotency keys ──@@@ -37,11 +37,6 @@ const http = require('http');await db.upsertApp({ app_id: 'live', name: 'OpenVibe.Live', api_key: 'live-key' });await db.upsertApp({ app_id: 'tools', name: 'OpenVibe.Tools', api_key: 'tools-key' });-- // The `live` app has an outbound webhook (tools has none): the operator report must show the host-- // only and never the URL or the secret.-- const HOOK_URL = 'https://hooks.example.test/ov-private-hook-path';-- const HOOK_SECRET = 'whsec_TESTONLY-MUST-NOT-LEAK';-- await db.run("UPDATE apps SET webhook_url = ?, webhook_secret = ? WHERE app_id = 'live'", [HOOK_URL, HOOK_SECRET]);for (const a of ['live', 'tools']) await db.ensureRootNamespace(await db.getApp(a));await namespaces.ensure(await db.getApp('live'), 'live.avatars');@@@ -339,19 +334,10 @@ const http = require('http');assert.ok(rep.projection.unprojected && rep.projection.owner_subject.missing >= 1);assert.ok(rep.tiering.objects.some(o => o.origin === 'native') && rep.tiering.sweep && 'pending_offload' in rep.tiering);assert.ok(rep.namespaces.some(n => n.namespace === 'live.avatars'));-- // Webhooks section: host only, no URL path and no secret anywhere in the payload; counts are-- // this process's own (none of these calls sends one).-- assert.deepStrictEqual(rep.webhooks.configured, [{ app_id: 'live', host: 'hooks.example.test' }], 'configured webhooks, host only');-- assert.ok(rep.webhooks.sends.since && !Object.keys(rep.webhooks.sends.by_app).length, 'no sends from the explorer itself');-- assert.ok(rep.webhooks.note.includes('OpenVibe.Events') && rep.webhooks.note.includes('reset on restart'));-- assert.ok(!r.text.includes(HOOK_URL) && !r.text.includes('ov-private-hook-path') && !r.text.includes(HOOK_SECRET), 'the report never carries the webhook URL or secret');r = await call('GET', '/api/v2/me/ops?app=tools', { token: 'admin' });assert.deepStrictEqual([r.json.scope, r.json.jobs.failed.total, r.json.namespaces.map(n => n.namespace)], ['tools', 0, ['tools']]);-- assert.deepStrictEqual(r.json.webhooks.configured, [], 'tools has no webhook');r = await call('GET', '/me/ops', { cookie: 'admin' });assert.ok(r.status === 200 && r.text.includes('mjob_TESTFAILED') && r.text.includes('action="/me/ops/recompute"') && r.text.includes('noindex'));-- assert.ok(r.text.includes('webhooks-h') && r.text.includes('hooks.example.test'), 'the page shows the webhooks section and the configured host');-- assert.ok(!r.text.includes(HOOK_URL) && !r.text.includes('ov-private-hook-path') && !r.text.includes(HOOK_SECRET), 'the page never shows the webhook URL path or secret');console.log('✅ operator views: staff.site.view only (not users, not global mods); failed jobs, missing media, projection, tiering, namespaces');// ── Recompute: staff.site.configure, same-origin ──@@@ -378,53 +364,12 @@ const http = require('http');assert.strictEqual(r.status, 200, r.text);assert.strictEqual(r.json.scope, 'live');assert.ok(r.json.namespaces.every(n => n.app_id === 'live'));-- assert.deepStrictEqual(r.json.webhooks.configured, [{ app_id: 'live', host: 'hooks.example.test' }], 'the app-key twin shows its own webhook');-- assert.ok(r.json.webhooks.sends.by_app === undefined || Object.keys(r.json.webhooks.sends.by_app).every(a => a === 'live'), 'sends narrowed to the app');assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'live-key', headers: { 'x-ov-user-id': '5' } })).status, 403, 'never acting for a user');assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'ana' })).status, 401);r = await call('POST', '/api/v1/tools/admin/storage/ops/recompute', { token: 'tools-key' });assert.deepStrictEqual([r.status, r.json.scope, r.json.namespaces], [200, 'tools', 1]);console.log('✅ /api/v1/:app/admin/storage/ops: the same report for the app key, scoped to its tenant');-- // ── Webhook tallies: one success, one failure ──-- {-- const hooks = require('../server/webhooks');-- hooks._resetStats();-- await db.run("UPDATE apps SET webhook_url = 'https://hooks-down.example.test/never' WHERE app_id = 'tools'");-- const realFetch = global.fetch;-- hooks._setRetryBaseMs(0); // retries must not slow the suite (never a global setTimeout patch: the pg pool's timers use it)-- let down = true;-- global.fetch = async (url, opts) => {-- const u = String(url);-- if (!u.includes('hooks.example.test') && !u.includes('hooks-down.example.test')) return realFetch(url, opts);-- const ok = u.includes('hooks.example.test') || !down;-- return { ok, status: ok ? 200 : 500 };-- };-- try {-- assert.strictEqual(await hooks.sendWebhook('live', 'vod.ready', { id: 1 }), true);-- assert.strictEqual(await hooks.sendWebhook('tools', 'clip.ready', { id: 2 }), false);-- // The same app answers 200 on a later send: the recorded error must be cleared.-- const ghost = { app_id: 'ghost', webhook_url: 'https://hooks-down.example.test/x', webhook_secret: 's' };-- assert.strictEqual(await hooks.sendWebhook(ghost, 'clip.ready', { id: 3 }), false);-- assert.strictEqual(hooks.stats().by_app.ghost.last_error, 'HTTP 500');-- down = false;-- assert.strictEqual(await hooks.sendWebhook(ghost, 'clip.ready', { id: 4 }), true);-- assert.strictEqual(hooks.stats().by_app.ghost.last_error, null, 'a later success clears the previous error');-- } finally {-- global.fetch = realFetch; hooks._setRetryBaseMs(null);-- await db.run("UPDATE apps SET webhook_url = NULL, webhook_secret = NULL WHERE app_id = 'tools'");-- }-- const st = hooks.stats();-- assert.deepStrictEqual([st.by_app.live.sent, st.by_app.live.failed, st.by_app.live.last_event], [1, 0, 'vod.ready']);-- assert.ok(st.by_app.live.last_ok_at && st.by_app.live.last_attempt_at && !st.by_app.live.last_error);-- assert.deepStrictEqual([st.by_app.tools.sent, st.by_app.tools.failed, st.by_app.tools.last_error, st.by_app.tools.last_event], [0, 1, 'HTTP 500', 'clip.ready']);-- st.by_app.live.sent = 999;-- assert.strictEqual(hooks.stats().by_app.live.sent, 1, 'stats() is a fresh copy');-- hooks._resetStats();-- assert.deepStrictEqual(hooks.stats().by_app, {});-- console.log('✅ webhook tallies: a 200 counts sent, a 500 after retries counts failed with its error; stats() copies');-- }--// ── Routes under /api/v2/me are the explorer's only ──r = await call('GET', '/api/v2/me/namespaces', { token: 'ana' });assert.deepStrictEqual([r.status, r.json.code], [404, 'media.not_found']);@@@ -88,7 +88,7 @@ const http = require('http');console.log('✅ a rolled-back change leaves neither an event nor a staged row');// ── 3. A row deleted inside announce(): media.object.deleted in the same transaction ──-- const { announce } = require('../server/webhooks');++ const { announce } = require('../server/announce');await announce('live', 'vod.failed', { change: async () => await db.run('DELETE FROM vods WHERE id = ?', [vodId]), payload: { id: vodId } });evs = await objectEvents();assert.strictEqual(evs.length, 2);@@@ -1,16 +1,14 @@'use strict';// Roadmap Wave 3 (audit item 7): Media's outbox row is written in the SAME PostgreSQL transaction as// the state change it describes. An outcome is never lost (the change committed, the event did--// not) and never a phantom (the event exists, the change rolled back). The webhook for the same--// outcome follows the commit and carries the event's id, so a consumer reading both paths--// handles it once. Covers webhooks.announce() and the real call sites for clip.failed++// not) and never a phantom (the event exists, the change rolled back). Covers announce() and++// the real call sites for clip.failed// (clip-jobs) and vod.failed (finalize), with an outbox insert that fails and a change that fails.const assert = require('assert');const fs = require('fs');const os = require('os');const path = require('path');const http = require('http');--const crypto = require('crypto');(async () => {const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-outbox-tx-'));@@@ -22,7 +20,6 @@ const crypto = require('crypto');process.env.PASTES_PATH = dir('pastes');process.env.OBJECTS_PATH = dir('objects');-- const hooks = [];const stub = http.createServer((req, res) => {let body = '';req.on('data', (c) => { body += c; });@@@ -35,29 +32,19 @@ const crypto = require('crypto');const results = list.map((e, i) => ({ event_id: e.event_id, seq: i + 1, duplicate: false }));return res.end(JSON.stringify(parsed.events ? { results } : results[0]));}-- if (req.url === '/hook') {-- hooks.push({ body, signature: req.headers['x-ovmedia-signature'] });-- return res.end('{"ok":true}');-- }res.statusCode = 404; res.end('{}');});});-- const waitFor = async (fn, ms = 3000) => {-- const end = Date.now() + ms;-- while (Date.now() < end) { if (fn()) return true; await new Promise((r) => setTimeout(r, 20)); }-- return false;-- };--(async () => {await new Promise((r) => stub.listen(0, '127.0.0.1', r));const base = `http://127.0.0.1:${stub.address().port}`;const db = require('../server/db/database');const raw = db.getDb();-- await db.upsertApp({ app_id: 'live', api_key: 'k', webhook_url: `${base}/hook`, webhook_secret: 'hook-secret' });++ await db.upsertApp({ app_id: 'live', api_key: 'k' });const events = require('../server/events');-- const { announce } = require('../server/webhooks');++ const { announce } = require('../server/announce');// A long relay interval: the test reads the outbox table, the relay is not under test here.const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });assert.ok(outbox);@@@ -72,7 +59,7 @@ const crypto = require('crypto');const { clipPublic } = require('../server/vod/clips-routes');const clipStatus = async () => (await raw.prepare('SELECT status FROM clips WHERE id = ?').get(clipId)).status;-- // 1. The change and its event commit together; the webhook carries the same event_id.++ // 1. The change and its event commit together; announce returns the event_id.const out = await announce('live', 'clip.ready', {change: async () => await db.run("UPDATE clips SET status = 'ready' WHERE id = ?", [clipId]),payload: async () => await clipPublic(await db.getClipById(clipId)),@@@ -84,14 +71,7 @@ const crypto = require('crypto');assert.deepStrictEqual(rows[0].env.subject, { type: 'clip', id: String(clipId) });assert.strictEqual(rows[0].env.payload.status, 'ready', 'payload is the row as committed');assert.strictEqual(out.eventId, rows[0].id);-- assert.ok(await waitFor(() => hooks.length === 1), 'webhook sent after the commit');-- const hook = JSON.parse(hooks[0].body);-- assert.strictEqual(hook.event, 'clip.ready');-- assert.strictEqual(hook.event_id, rows[0].id, 'the webhook names the durable event');-- const sig = 'sha256=' + crypto.createHmac('sha256', 'hook-secret').update(hooks[0].body).digest('hex');-- assert.strictEqual(hooks[0].signature, sig, 'the signature covers event_id');---- // 2. No phantom: a change that fails leaves no event and sends no webhook.++ // 2. No phantom: a change that fails leaves no event.await assert.rejects(async () => await announce('live', 'clip.failed', {change: async () => { await db.run("UPDATE clips SET status = 'failed' WHERE id = ?", [clipId]); throw new Error('disk full'); },payload: async () => await clipPublic(await db.getClipById(clipId)),@@@ -150,11 +130,10 @@ const crypto = require('crypto');assert.strictEqual(rows[2].env.event_type, 'media.vod.failed');assert.deepStrictEqual(rows[2].env.subject, { type: 'vod', id: '20' });assert.strictEqual(rows[2].env.priority, 'important');-- assert.ok(await waitFor(() => hooks.some((h) => JSON.parse(h.body).event_id === rows[2].id)), 'vod.failed webhook names the event');events._reset();stub.close();-- console.log('✅ media outbox: state change and event commit together (no lost or phantom outcome), webhook carries event_id');++ console.log('✅ media outbox: state change and event commit together (no lost or phantom outcome)');process.exit(0);})().catch((err) => { console.error(err); process.exit(1); });})().catch((err) => { console.error(err); process.exit(1); });@@@ -6,9 +6,9 @@// - POST /api/v1/:app/vods/:id/ingest/rtmp makes ffmpeg pull an app-supplied rtmp_url, which must name an// allow-listed host:port (MEDIA_RTMP_PULL_ALLOW; test/rtmp-allowlist.test.js has the URL grammar).// Every other outbound request goes to an operator-configured address (Network, Events, Live's internal--// URL, B2/R2, an app's operator-set webhook_url row, IndexNow's fixed api.indexnow.org from++// URL, B2/R2, IndexNow's fixed api.indexnow.org from// INDEXNOW_KEY). The inventory below fails when a new outbound call appears in server/ until it is--// classified here, and no route lets a tenant set a webhook URL.++// classified here. No server path sends to a webhook URL.// No test here needs the network: literals and internal names are refused before DNS, and DNS answers// and the socket are stubbed.const assert = require('assert');@@@ -31,7 +31,6 @@ const https = require('https');'server/avatars/ingest.js': 'a person\'s picture URL: safeFetchImage, tested below','server/account-data.js': 'Network internal API (OV_NETWORK_INTERNAL_URL), fixed paths: export parts and deletion confirmations','server/user-auth.js': 'Network OAuth (OV_NETWORK_URL / OV_NETWORK_INTERNAL_URL); the signing keys come from openvibe-sdk/auth',-- 'server/webhooks.js': 'an app\'s webhook_url, an operator-set row (no seeding since C-71)','server/thumbnails/live-frame-service.js': 'the app\'s internal URL (APP_INTERNAL_URLS), fixed path','server/public/dev-data.js': 'the app\'s internal URL (APP_INTERNAL_URLS), ids and encoded names in the path','server/jobs/pack.js': 'a presigned GET on B2/R2 (operator-configured buckets) for a segment\'s durable copy',@@@ -52,8 +51,8 @@ const https = require('https');walk(path.join(ROOT, 'server'));for (const f of found) assert.ok(OUTBOUND[f], `${f} makes an outbound request: classify it in test/security-ssrf.test.js (a user-chosen address must go through a guard)`);for (const f of Object.keys(OUTBOUND)) assert.ok(found.has(f), `${f} no longer makes an outbound request: drop it from the inventory`);-- // Webhook targets are operator configuration: the only SQL that writes an app's webhook_url is-- // db.upsertApp, which no server code calls since app seeding went (C-71): tests and the smoke script only.++ // The schema and db.upsertApp retain the legacy columns, but no server code uses them++ // as an outbound destination.const sqlWriters = [], upserters = [];const scan = (d) => {for (const e of fs.readdirSync(d, { withFileTypes: true })) {@@@ -67,14 +66,24 @@ const https = require('https');};scan(path.join(ROOT, 'server'));assert.deepStrictEqual(sqlWriters, ['server/db/database.js'], 'only db.upsertApp writes an app\'s webhook_url');-- assert.deepStrictEqual(upserters, [], 'and no server code calls it: app seeding is gone (C-71), so a webhook URL is an operator-set row');++ assert.deepStrictEqual(upserters, [], 'no server code calls db.upsertApp');++ const webhookReaders = [];++ const scanReaders = (d) => {++ for (const e of fs.readdirSync(d, { withFileTypes: true })) {++ const p = path.join(d, e.name);++ if (e.isDirectory()) { scanReaders(p); continue; }++ if (p.endsWith('.js') && !p.endsWith(path.join('db', 'database.js')) && /\bwebhook_(url|secret)\b/.test(fs.readFileSync(p, 'utf8'))) webhookReaders.push(path.relative(ROOT, p));++ }++ };++ scanReaders(path.join(ROOT, 'server'));++ assert.deepStrictEqual(webhookReaders, [], 'no server code sends to or reads a webhook URL or secret');// IndexNow (openvibe-shared/indexnow, wired in server/index.js): the only outbound request made// from a dependency. Its endpoint is the fixed api.indexnow.org and host/key are operator config// (MEDIA_PUBLIC_URL / INDEXNOW_KEY) — never a tenant or a user-supplied URL, and no route reaches it.const idxSrc = fs.readFileSync(path.join(ROOT, 'server', 'index.js'), 'utf8');assert.ok(idxSrc.includes('createIndexNow(') && idxSrc.includes('host: config.publicUrl') && idxSrc.includes('key: config.indexnow.key'),'IndexNow is wired from operator config (host/key), the one outbound dependency');-- console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); webhook URLs are operator-set; IndexNow posts to the fixed api.indexnow.org`);++ console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); no webhook sends; IndexNow posts to the fixed api.indexnow.org`);}const ingest = require('../server/avatars/ingest');
@@ -20,7 +20,7 @@ VALKEY_PREFIX=
@@ -20,7 +20,7 @@ VALKEY_PREFIX=
MEDIA_DATA_DIR=./data
MEDIA_DATA_DIR=./data
# Restore-drill mode, set only by `ovhost drill media` (OpenVibe.Host docs/restore-drills.md): reads
# Restore-drill mode, set only by `ovhost drill media` (OpenVibe.Host docs/restore-drills.md): reads
-# only from a restored copy, no stored bytes, no jobs, webhooks, events or ffmpeg; refuses to start
+# only from a restored copy, no stored bytes, no jobs, events or ffmpeg; refuses to start
# unless DATABASE_URL names the restored copy (never ov_media on 5432/6432), DATABASE_DIRECT_URL is unset,
# unless DATABASE_URL names the restored copy (never ov_media on 5432/6432), DATABASE_DIRECT_URL is unset,
# HOST is loopback and PORT is not 4100 (server/drill.js).
# HOST is loopback and PORT is not 4100 (server/drill.js).
# MEDIA_DRILL=1
# MEDIA_DRILL=1
@@ -68,8 +68,7 @@ Called elsewhere, as the service principal `media` (the OAuth client `media`):
@@ -68,8 +68,7 @@ Called elsewhere, as the service principal `media` (the OAuth client `media`):
| OpenVibe.Live | `live.lineage.resolve` | which channel and person a recording belongs to |
| OpenVibe.Live | `live.lineage.resolve` | which channel and person a recording belongs to |
| OpenVibe.Events | `events.event.publish`, `events.subscription.manage` | the outbox relay; its subscriptions |
| OpenVibe.Events | `events.event.publish`, `events.subscription.manage` | the outbox relay; its subscriptions |
-Outbound webhooks to tenants (`webhook_url`, HMAC with `webhook_secret`) stay until every consumer
-reads events.
+Media announces outcomes as OpenVibe.Events events (`media.vod.ready`, …) and sends no webhooks since 2026-10-11 (C-75).
## Layout
## Layout
@@ -79,7 +78,7 @@ server/
@@ -79,7 +78,7 @@ server/
config.js env config (PORT, MEDIA_DATA_DIR, *_PATH, DATABASE_URL/DATABASE_DIRECT_URL, MEDIA_B2_*/R2_*,
config.js env config (PORT, MEDIA_DATA_DIR, *_PATH, DATABASE_URL/DATABASE_DIRECT_URL, MEDIA_B2_*/R2_*,
RTP pool, OV_NETWORK_URL)
RTP pool, OV_NETWORK_URL)
auth.js tenancy + auth middleware, JWKS fetch, app seeding
auth.js tenancy + auth middleware, JWKS fetch, app seeding
- webhooks.js HMAC-signed outbound webhooks
+ announce.js transactional OpenVibe.Events outcome announcements
drill.js MEDIA_DRILL: restore-drill mode (reads only, no bytes, no jobs, nothing leaves)
drill.js MEDIA_DRILL: restore-drill mode (reads only, no bytes, no jobs, nothing leaves)
db/database.js PostgreSQL helpers (openvibe-sdk/db, async), all app_id-scoped
db/database.js PostgreSQL helpers (openvibe-sdk/db, async), all app_id-scoped
db/init.js applies migrations/ (DATABASE_URL/DATABASE_DIRECT_URL, or embedded PGlite)
db/init.js applies migrations/ (DATABASE_URL/DATABASE_DIRECT_URL, or embedded PGlite)
@@ -87,7 +86,7 @@ server/
@@ -87,7 +86,7 @@ server/
media_objects/locations/relationships/variants/jobs/holds/invariant_violations
media_objects/locations/relationships/variants/jobs/holds/invariant_violations
vod/recorder.js ffmpeg recording: RTMP pull + RTP (SDP) ingest, codec passthrough
vod/recorder.js ffmpeg recording: RTMP pull + RTP (SDP) ingest, codec passthrough
vod/media-tools.js probes, seekable remux, DVR sidecar, chunk-segment concat
vod/media-tools.js probes, seekable remux, DVR sidecar, chunk-segment concat
- vod/finalize.js finalize pipeline (remux → probe → master recovery → thumbnail → webhook)
+ vod/finalize.js finalize pipeline (remux → probe → master recovery → thumbnail → event)
vod/routes.js /api/v1/:app/vods (+ ingest + chunks)
vod/routes.js /api/v1/:app/vods (+ ingest + chunks)
vod/clip-cutter.js re-encoding clip cutter (local file or presigned URL source)
vod/clip-cutter.js re-encoding clip cutter (local file or presigned URL source)
vod/clips-routes.js /api/v1/:app/clips
vod/clips-routes.js /api/v1/:app/clips
@@ -147,8 +146,7 @@ other OpenVibe services call the API.
@@ -147,8 +146,7 @@ other OpenVibe services call the API.
subject, in every tenant: uploads in progress, copies, derivatives, visibility, lifecycle and their own
subject, in every tenant: uploads in progress, copies, derivatives, visibility, lifecycle and their own
usage per tenant and namespace (`GET /api/v2/me/objects`, `/objects/:id`, `/usage`). Read-only: changes
usage per tenant and namespace (`GET /api/v2/me/objects`, `/objects/:id`, `/usage`). Read-only: changes
stay with the apps. Network staff with `staff.site.view` get the operator views at `/me/ops` (failed
stay with the apps. Network staff with `staff.site.view` get the operator views at `/me/ops` (failed
-jobs, missing media, projection, tiering, outbound webhooks — who is configured and what this process
-has sent — and usage recompute with `staff.site.configure`). Details in
+jobs, missing media, projection, tiering and usage recompute with `staff.site.configure`). Details in
[docs/object-model.md](docs/object-model.md#object-explorer).
[docs/object-model.md](docs/object-model.md#object-explorer).
## Tenancy & auth
## Tenancy & auth
@@ -159,7 +157,7 @@ Tenant = the `:app` path segment. Rows in the `apps` table define tenants:
@@ -159,7 +157,7 @@ Tenant = the `:app` path segment. Rows in the `apps` table define tenants:
|---|---|
|---|---|
| `app_id` | tenant id (`live`, `games`, …) |
| `app_id` | tenant id (`live`, `games`, …) |
| `api_key_hash` | sha256 of the app API key (never stored in plaintext) |
| `api_key_hash` | sha256 of the app API key (never stored in plaintext) |
-| `webhook_url` / `webhook_secret` | outbound event delivery (HMAC) |
+| `webhook_url` / `webhook_secret` | retained for a separate schema migration; unused for delivery |
| `allowed_origins` | JSON array; guards browser (user-JWT) calls |
| `allowed_origins` | JSON array; guards browser (user-JWT) calls |
| `quota_bytes` | storage quota: v1 files + native v2 objects (0 = unlimited) |
| `quota_bytes` | storage quota: v1 files + native v2 objects (0 = unlimited) |
| `project_id` / `env` | set only on developer-project tenants (below) |
| `project_id` / `env` | set only on developer-project tenants (below) |
@@ -240,7 +238,7 @@ and in `scripts/smoke-test.sh`.
@@ -240,7 +238,7 @@ and in `scripts/smoke-test.sh`.
| POST | `/vods/:id/ingest/rtp/stop` | finalizes the recording |
| POST | `/vods/:id/ingest/rtp/stop` | finalizes the recording |
| POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow |
| POST | `/vods/:id/chunks` | multipart `chunk` (+`segmentId`), user JWT ok — browser MediaRecorder append flow |
| POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) |
| POST | `/vods/:id/complete` | finalize chunked upload (user JWT ok) |
-| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, webhook |
+| POST | `/vods/:id/finalize` | close recording; remux, probe, thumbnail, event |
| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness); `hls_url`, with `hls_expires_at` when signed: the object's master playlist with `MEDIA_HLS_ENABLED`, docs/media-fabric.md §3) |
| GET | `/vods/:id` | `{ id, title, status, duration, playback_url, hls_url, thumbnail_url, storage_provider, readiness, … }` (`readiness`: [below](#readiness); `hls_url`, with `hls_expires_at` when signed: the object's master playlist with `MEDIA_HLS_ENABLED`, docs/media-fabric.md §3) |
| GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) |
| GET | `/vods?limit&offset&user_id&stream_id&managed_stream_id&include_private&order&since` | list; `include_private` app-key only; `order` = newest\|oldest\|views; `since` = created at or after (ISO 8601 or `YYYY-MM-DD HH:MM:SS`, UTC) |
| PUT | `/vods/:id` | `{ title?, description?, visibility? }` |
| PUT | `/vods/:id` | `{ title?, description?, visibility? }` |
@@ -354,22 +352,17 @@ directories are shared across apps) and responses carry a `note` saying so.
@@ -354,22 +352,17 @@ directories are shared across apps) and responses carry a `note` saying so.
| GET | `/admin/storage/ops` | this app's operator report (as `/me/ops` shows it to Network staff): failed jobs, missing media, projection, tiering, the namespaces' usage snapshot. Not for calls acting for a user (403) |
| GET | `/admin/storage/ops` | this app's operator report (as `/me/ops` shows it to Network staff): failed jobs, missing media, projection, tiering, the namespaces' usage snapshot. Not for calls acting for a user (403) |
| POST | `/admin/storage/ops/recompute` | refresh this app's namespaces' usage snapshot from the rows |
| POST | `/admin/storage/ops/recompute` | refresh this app's namespaces' usage snapshot from the rows |
-### Webhooks (outbound)
+### Outcome events
-`POST` to the app's `webhook_url` with body
-`{ "event": "vod.ready"|"vod.failed"|"clip.ready"|"clip.failed"|"media.object.uploaded"|"storage.alert"|"storage.recovered", "app_id", "data", "event_id"? }`
-and header `X-OVMedia-Signature: sha256=<hex hmac-sha256 of the raw body with
-the app's webhook_secret>`. 3 attempts with backoff, 10 s timeout.
+Media announces outcomes as OpenVibe.Events events (`media.vod.ready`, …) and sends no webhooks since 2026-10-11 (C-75).
-Each outcome is also a durable OpenVibe.Events event (`media.vod.ready|failed`,
+Each outcome is a durable OpenVibe.Events event (`media.vod.ready|failed`,
`media.clip.ready|failed`, `media.object.uploaded`, `media.storage.alert|recovered`;
`media.clip.ready|failed`, `media.object.uploaded`, `media.storage.alert|recovered`;
`server/events.js`). The outbox row is written in the **same transaction** as the state
`server/events.js`). The outbox row is written in the **same transaction** as the state
-change it describes (`webhooks.announce()`), so an outcome is never lost or announced for a
-rolled-back change; the webhook follows the commit and carries `event_id` = that event's id
-(absent when the outbox is off), so an app that reads both paths (Live during its webhook →
-Events transition) handles each outcome once. Webhooks stay until every consumer reads Events.
+change it describes (`announce()`), so an outcome is never lost or announced for a
+rolled-back change.
-Two object events go to OpenVibe.Events only (no webhook): **`media.object.deleted`**
+Two additional object events go to OpenVibe.Events: **`media.object.deleted`**
`{ object_id, app_id, kind, legacy_ref, deleted_at }` and **`media.object.visibility_changed`**
`{ object_id, app_id, kind, legacy_ref, deleted_at }` and **`media.object.visibility_changed`**
`{ object_id, app_id, kind, legacy_ref, visibility, previous_visibility, changed_at }` (subject
`{ object_id, app_id, kind, legacy_ref, visibility, previous_visibility, changed_at }` (subject
`object <id>`, `important`, times ISO 8601 UTC, `legacy_ref` null for native objects). Nothing
`object <id>`, `important`, times ISO 8601 UTC, `legacy_ref` null for native objects). Nothing
@@ -422,7 +415,7 @@ mode (`server/drill.js`) Media:
@@ -422,7 +415,7 @@ mode (`server/drill.js`) Media:
4100 (before the database is opened), and opens the copy without migrating it;
4100 (before the database is opened), and opens the copy without migrating it;
- starts only its HTTP server: no app seeding, JWKS refresh, tiering sweep, health job, junk sweep,
- starts only its HTTP server: no app seeding, JWKS refresh, tiering sweep, health job, junk sweep,
clip re-cuts, copy verification, jobs worker, disk guardian, thumbnail cleanup, object purge,
clip re-cuts, copy verification, jobs worker, disk guardian, thumbnail cleanup, object purge,
- orphan-recording finalize or Events relay; webhooks are never sent;
+ orphan-recording finalize or Events relay;
- writes no file and creates no directory (the storage checks leave `/api/ready`, as do the remote
- writes no file and creates no directory (the storage checks leave `/api/ready`, as do the remote
tiers; it reports `"mode": "drill"`);
tiers; it reports `"mode": "drill"`);
- never opens a file path from the database: every byte route (`/v` and `/c` bytes, `/t`, `/a`, `/f`,
- never opens a file path from the database: every byte route (`/v` and `/c` bytes, `/t`, `/a`, `/f`,
@@ -432,7 +425,7 @@ mode (`server/drill.js`) Media:
@@ -432,7 +425,7 @@ mode (`server/drill.js`) Media:
listens on nothing but its port;
listens on nothing but its port;
- answers 403 to every method but GET, HEAD and OPTIONS, and 503 to `/auth/*`.
- answers 403 to every method but GET, HEAD and OPTIONS, and 503 to `/auth/*`.
-`test/drill-mode.test.js` boots the real server that way against a copy whose app has a webhook URL
+`test/drill-mode.test.js` boots the real server that way against a restored copy
and checks each point.
and checks each point.
## N-1: the previous release against this one
## N-1: the previous release against this one
@@ -660,7 +653,7 @@ shutdown.
@@ -660,7 +653,7 @@ shutdown.
- **Only real files are candidates.** Rows without a file (a recording that was refused or never produced output, legacy paths) are quarantined as `health_status='missing_file'` and excluded — they used to fill the candidate list and starve the drain.
- **Only real files are candidates.** Rows without a file (a recording that was refused or never produced output, legacy paths) are quarantined as `health_status='missing_file'` and excluded — they used to fill the candidate list and starve the drain.
- **Cadence** — while a drain is still needed the sweep re-runs every `pressureRetryMs` (2 min), otherwise every `sweepIntervalMs` (15 min). A VOD whose upload failed is skipped for 30 min so one bad file cannot pin the drain.
- **Cadence** — while a drain is still needed the sweep re-runs every `pressureRetryMs` (2 min), otherwise every `sweepIntervalMs` (15 min). A VOD whose upload failed is skipped for 30 min so one bad file cannot pin the drain.
- **Bounded uploads** — each upload is aborted past `uploadTimeoutFloorMs + size / uploadMinThroughputMBps`; a watchdog logs a sweep that has run for hours.
- **Bounded uploads** — each upload is aborted past `uploadTimeoutFloorMs + size / uploadMinThroughputMBps`; a watchdog logs a sweep that has run for hours.
-- **Stall alerts** — after `alertAfterStalledPasses` (2) consecutive pressure passes that freed nothing, and whenever the disk is critical, Media sends `storage.alert` to every app webhook (Live logs it and forwards to `OPS_ALERT_WEBHOOK_URL` / the `ops_alert_webhook_url` setting); `storage.recovered` follows once the drain works again. Each pass under pressure logs candidates / uploaded / failed / backed-off / quarantined.
+- **Stall alerts** — after `alertAfterStalledPasses` (2) consecutive pressure passes that freed nothing, and whenever the disk is critical, Media queues a `media.storage.alert` event; `media.storage.recovered` follows once the drain works again. Each pass under pressure logs candidates / uploaded / failed / backed-off / quarantined.
- **Status** — `GET /api/v1/:app/admin/storage` includes `sweep` (last result, next run, stalled flag, needsDrain/critical).
- **Status** — `GET /api/v1/:app/admin/storage` includes `sweep` (last result, next run, stalled flag, needsDrain/critical).
Run `npm test` for the policy regression tests.
Run `npm test` for the policy regression tests.
@@ -33,7 +33,7 @@
@@ -33,7 +33,7 @@
"multi-tenant apps and developer-project tenants (sandbox separated)",
"multi-tenant apps and developer-project tenants (sandbox separated)",
"public serving with presigned URLs; signed access for private and sandbox items",
"public serving with presigned URLs; signed access for private and sandbox items",
"the object explorer (/me) and operator views",
"the object explorer (/me) and operator views",
- "media.* events through the SDK outbox, webhooks during the transition",
+ "media.* events through the SDK outbox as the only outcome announcement",
"restore-drill mode and N-1 compatibility tests",
"restore-drill mode and N-1 compatibility tests",
"per-actor limits (openvibe-sdk/limits)",
"per-actor limits (openvibe-sdk/limits)",
"placement demand rollups on Valkey (F2.4): reads per object × region × 5-minute bucket and, for a timeline segment, per segment too (object total is the sum over its segments; hotness over the hour, media.object.hot once per object per hour)",
"placement demand rollups on Valkey (F2.4): reads per object × region × 5-minute bucket and, for a timeline segment, per segment too (object total is the sum over its segments; hotness over the hour, media.object.hot once per object per hour)",
@@ -277,7 +277,7 @@ formats (AVIF/WebP), preview clips and AI thumbnails are `rebuildable = true`:
@@ -277,7 +277,7 @@ formats (AVIF/WebP), preview clips and AI thumbnails are `rebuildable = true`:
only, report only) reads those rows every `MEDIA_MOVE_CLEANUP_MINUTES` (60; `0` = never; not in a restore drill): an
only, report only) reads those rows every `MEDIA_MOVE_CLEANUP_MINUTES` (60; `0` = never; not in a restore drill): an
object whose move failed three times in the last 7 days (a later `done` of that move resets the count) raises one
object whose move failed three times in the last 7 days (a later `done` of that move resets the count) raises one
`storage.alert` of kind `move_cleanup_failed`, listing the objects (`media_storage_alerts_total{kind}`, the Events
`storage.alert` of kind `move_cleanup_failed`, listing the objects (`media_storage_alerts_total{kind}`, the Events
- outbox and the per-app webhook, once per 6 h cooldown like every storage alert).
+ outbox, once per 6 h cooldown like every storage alert).
- **Dry-run and simulator:** every policy can run in dry-run mode (logs what it would do and what it would cost), and a
- **Dry-run and simulator:** every policy can run in dry-run mode (logs what it would do and what it would cost), and a
simulator replays recorded telemetry against a candidate policy before it is enabled.
simulator replays recorded telemetry against a candidate policy before it is enabled.
- **Demand rollups (F2.4, shipped):** every served viewer read (`router.route()` for playback or download, not a derive
- **Demand rollups (F2.4, shipped):** every served viewer read (`router.route()` for playback or download, not a derive
@@ -87,7 +87,7 @@ Every write to a projected row writes its object in the same PostgreSQL transact
@@ -87,7 +87,7 @@ Every write to a projected row writes its object in the same PostgreSQL transact
(`model.withObject(kind, ids, write)`; roadmap WS-G task 1; compatibility shim C-75 was retired on 2026-10-10). The row change and the re-projection commit together or
(`model.withObject(kind, ids, write)`; roadmap WS-G task 1; compatibility shim C-75 was retired on 2026-10-10). The row change and the re-projection commit together or
not at all, so a crash or an error between them can no longer leave the object behind its row. A
not at all, so a crash or an error between them can no longer leave the object behind its row. A
write whose object cannot be written fails as a whole and changes nothing. Inside
write whose object cannot be written fails as a whole and changes nothing. Inside
-`webhooks.announce()` the object joins the outcome's transaction, and its object events are queued
+`announce()` the object joins the outcome's transaction, and its object events are queued
after the outcome event.
after the outcome event.
| path | where |
| path | where |
@@ -180,7 +180,7 @@ All routes live under `/api/v2/:app/objects`.
@@ -180,7 +180,7 @@ All routes live under `/api/v2/:app/objects`.
| POST | `/` | init. `{ namespace (default the tenant's root; a name below it, full or relative), kind, visibility (default private), mime_type, size_bytes, filename, content_hash (expected sha256), metadata (≤ 16 KB; `metadata.class` declares the object's storage class, see [Tiering of native objects](#tiering-of-native-objects)), user_id, upload_ttl (60-86400 s), multipart, part_size }` returns 201 `{ id, object, upload: { method: 'PUT', url, token, expires_at, max_bytes, content_type, complete_url, multipart_url } }`. `url` is a [presigned PUT URL](#presigned-uploads). With `multipart: true` (required above `MEDIA_OBJECT_MAX_MB`): `method: 'multipart'`, `url: null`, and `upload.multipart` is the new [session](#multipart-uploads). Checks at init: `size_bytes` against `MEDIA_OBJECT_MAX_MB` (single part) or `MEDIA_MULTIPART_MAX_MB` (413), the namespace's policy (422 `media.namespace.policy_denied`, 413 `media.object.too_large`), every quota from the namespace up to the tenant including reservations (413 `media.quota.exceeded` / `media.quota.objects_exceeded`; the declared size is then reserved), the public-size invariant (422) and the [content type](#content-types) for the kind (415 `media.object.unsupported_type`) |
| POST | `/` | init. `{ namespace (default the tenant's root; a name below it, full or relative), kind, visibility (default private), mime_type, size_bytes, filename, content_hash (expected sha256), metadata (≤ 16 KB; `metadata.class` declares the object's storage class, see [Tiering of native objects](#tiering-of-native-objects)), user_id, upload_ttl (60-86400 s), multipart, part_size }` returns 201 `{ id, object, upload: { method: 'PUT', url, token, expires_at, max_bytes, content_type, complete_url, multipart_url } }`. `url` is a [presigned PUT URL](#presigned-uploads). With `multipart: true` (required above `MEDIA_OBJECT_MAX_MB`): `method: 'multipart'`, `url: null`, and `upload.multipart` is the new [session](#multipart-uploads). Checks at init: `size_bytes` against `MEDIA_OBJECT_MAX_MB` (single part) or `MEDIA_MULTIPART_MAX_MB` (413), the namespace's policy (422 `media.namespace.policy_denied`, 413 `media.object.too_large`), every quota from the namespace up to the tenant including reservations (413 `media.quota.exceeded` / `media.quota.objects_exceeded`; the declared size is then reserved), the public-size invariant (422) and the [content type](#content-types) for the kind (415 `media.object.unsupported_type`) |
| POST | `/:id/upload-url` | a fresh presigned single-PUT URL for an uploading object (`{ ttl }` 60-86400 s) |
| POST | `/:id/upload-url` | a fresh presigned single-PUT URL for an uploading object (`{ ttl }` 60-86400 s) |
| PUT | `/:id/content` | the bytes, single part. Auth is the upload token (`?token=` from init, or `X-Upload-Token`) or the usual credential. Streams to disk computing sha256. Refuses more than the declared size or the limit (413; an object declared above `MEDIA_OBJECT_MAX_MB` goes multipart), a size mismatch (400), quota overrun (413), a type that does not suit the kind (415) and a PUT while a multipart session is open (409). Stored at `OBJECTS_PATH/<app>/<id>` with a verified local location. Mounted ahead of the JSON body parser, so any Content-Type is taken as raw bytes; the Content-Type becomes `mime_type` when init set none. Can be repeated while the object is `uploading` |
| PUT | `/:id/content` | the bytes, single part. Auth is the upload token (`?token=` from init, or `X-Upload-Token`) or the usual credential. Streams to disk computing sha256. Refuses more than the declared size or the limit (413; an object declared above `MEDIA_OBJECT_MAX_MB` goes multipart), a size mismatch (400), quota overrun (413), a type that does not suit the kind (415) and a PUT while a multipart session is open (409). Stored at `OBJECTS_PATH/<app>/<id>` with a verified local location. Mounted ahead of the JSON body parser, so any Content-Type is taken as raw bytes; the Content-Type becomes `mime_type` when init set none. Can be repeated while the object is `uploading` |
-| POST | `/:id/complete` | checks the expected hash (422 `media.object.hash_mismatch`), the invariant, the quota and the bytes against the type (415 `media.object.content_mismatch`). Sets `ready` and sends the `media.object.uploaded` webhook. Also accepts the upload token |
+| POST | `/:id/complete` | checks the expected hash (422 `media.object.hash_mismatch`), the invariant, the quota and the bytes against the type (415 `media.object.content_mismatch`). Sets `ready` and queues the `media.object.uploaded` event. Also accepts the upload token |
| POST | `/:id/multipart` | start a [multipart session](#multipart-uploads) `{ part_size, size_bytes (when init declared none) }` → 201 |
| POST | `/:id/multipart` | start a [multipart session](#multipart-uploads) `{ part_size, size_bytes (when init declared none) }` → 201 |
| GET | `/:id/multipart/:uploadId` | the session: `parts` received (number, size, sha256), `missing`, `received_bytes`, `expires_at` |
| GET | `/:id/multipart/:uploadId` | the session: `parts` received (number, size, sha256), `missing`, `received_bytes`, `expires_at` |
| PUT | `/:id/multipart/:uploadId/parts/:n` | one part, raw bytes of exactly its size; optional `X-Content-SHA256` |
| PUT | `/:id/multipart/:uploadId/parts/:n` | one part, raw bytes of exactly its size; optional `X-Content-SHA256` |
@@ -416,15 +416,13 @@ cheap and renders in a drill.
@@ -416,15 +416,13 @@ cheap and renders in a drill.
- `projection`: rows with no object yet; objects whose owner subject is unresolved; whether upload reservations were seeded.
- `projection`: rows with no object yet; objects whose owner subject is unresolved; whether upload reservations were seeded.
- `tiering`: providers and policy; VODs and clips per provider; objects per canonical copy (native or projected); VODs eligible to offload; R2 decisions of the last 24 h and recent refusals and failures; the sweep's state in this process; and `native_objects`: the [object tiering](#tiering-of-native-objects)'s activation gate and policy, native objects in R2, how many are eligible to promote now, its decisions of the last 24 h (dry runs included) and the most recent ones, and its last sweep.
- `tiering`: providers and policy; VODs and clips per provider; objects per canonical copy (native or projected); VODs eligible to offload; R2 decisions of the last 24 h and recent refusals and failures; the sweep's state in this process; and `native_objects`: the [object tiering](#tiering-of-native-objects)'s activation gate and policy, native objects in R2, how many are eligible to promote now, its decisions of the last 24 h (dry runs included) and the most recent ones, and its last sweep.
- `namespaces`: every namespace with quota and usage snapshot.
- `namespaces`: every namespace with quota and usage snapshot.
-- `webhooks`: the apps with a configured outbound webhook (app id and host only — never the URL or secret) and, per app, what this process has sent since it loaded (`sent`, `failed`, the last event and attempt/ok times, the last error). In-memory: counts reset on restart and scope to the app when the report is scoped.
## Readiness
## Readiness
Code: `server/objects/readiness.js`. Test: `test/readiness.test.js`.
Code: `server/objects/readiness.js`. Test: `test/readiness.test.js`.
Every object answer (v2 `GET /:id`, the v2 list, the `media.object.uploaded` payload) and the v1 VOD
Every object answer (v2 `GET /:id`, the v2 list, the `media.object.uploaded` payload) and the v1 VOD
-and clip answers (`GET`, list, update, finalize, upload; not the `vod.*`/`clip.*` webhook payloads,
-which are sent before the object is re-projected) carry `readiness`:
+and clip answers (`GET`, list, update, finalize, upload) carry `readiness`:
```json
```json
{ "metadata": true, "bytes_verified": true, "playable": true, "hash_verified": false, "verified_copies": ["local"], "reason": null }
{ "metadata": true, "bytes_verified": true, "playable": true, "hash_verified": false, "verified_copies": ["local"], "reason": null }
@@ -647,10 +645,9 @@ Code: `server/jobs/`. Tests: `test/jobs.test.js`, `test/jobs-invariant.test.js`,
@@ -647,10 +645,9 @@ Code: `server/jobs/`. Tests: `test/jobs.test.js`, `test/jobs-invariant.test.js`,
**Events.** Every state change and its `media.job.<transition>` event (`proposed`, `queued`,
**Events.** Every state change and its `media.job.<transition>` event (`proposed`, `queued`,
`started`, `retrying`, `succeeded`, `failed`, `cancelled`; subject `job <id>`) commit in one PostgreSQL
`started`, `retrying`, `succeeded`, `failed`, `cancelled`; subject `job <id>`) commit in one PostgreSQL
-transaction through Media's outbox, the same rule as the outcome events (`webhooks.announce()`): no
+transaction through Media's outbox, the same rule as the outcome events (`announce()`): no
change without its event, no event for a change that rolled back. Progress events are `low`
change without its event, no event for a change that rolled back. Progress events are `low`
-priority; outcomes and proposals are `important`. Job events go to OpenVibe.Events only, not to app
-webhooks.
+priority; outcomes and proposals are `important`. Job events go to OpenVibe.Events.
The payload is the event projection (`queue.jobEvent`, contracts `media.job.<transition>@1`), not
The payload is the event projection (`queue.jobEvent`, contracts `media.job.<transition>@1`), not
the job as the API answers it:
the job as the API answers it:
@@ -704,7 +701,7 @@ job over only with the token it read and, past boot, only while the lease is sti
@@ -704,7 +701,7 @@ job over only with the token it read and, past boot, only while the lease is sti
| `invariant.scan` | light | The [size-invariant validator](#public-object-size-invariant): records violations, proposes `object.split` / `object.remux`, withdraws moot proposals. Tenant-wide (no object) |
| `invariant.scan` | light | The [size-invariant validator](#public-object-size-invariant): records violations, proposes `object.split` / `object.remux`, withdraws moot proposals. Tenant-wide (no object) |
| `object.split` | heavy | Stream-copies a vod/clip (or a video/audio object) into parts (`params.parts` 2-1000 or `segment_seconds`), each a new **private** object with a `derived_from` relationship (`job_id`, `part`, `start_seconds`, `duration_seconds`). Cuts land on keyframes, so neighbouring parts can overlap slightly. The source is never changed. Checkpointed per part |
| `object.split` | heavy | Stream-copies a vod/clip (or a video/audio object) into parts (`params.parts` 2-1000 or `segment_seconds`), each a new **private** object with a `derived_from` relationship (`job_id`, `part`, `start_seconds`, `duration_seconds`). Cuts land on keyframes, so neighbouring parts can overlap slightly. The source is never changed. Checkpointed per part |
| `object.remux` | heavy | Stream-copy remux of the whole source (seek index, duration, MP4 faststart) into one new private object; also the source's `remux` variant |
| `object.remux` | heavy | Stream-copy remux of the whole source (seek index, duration, MP4 faststart) into one new private object; also the source's `remux` variant |
-| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips` and `/:id/recut` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` and the webhook come from the cut as before. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording |
+| `clip.cut` | clips | Cut (or re-cut) a clip row from its VOD: `POST /api/v1/:app/clips` and `/:id/recut` answer 202 with `job_id`, which a UI polls and reattaches to (`GET /api/v2/:app/jobs/:id`). `clip.ready`/`clip.failed` events come from the cut. It retries after 2, 10 and 30 min (from attempt 2 a cloud VOD is fetched home first); a window with no footage fails at once. The `clips` lane (`MEDIA_JOBS_CLIPS_CONCURRENCY`, 2) runs while recording |
| `object.waveform` | heavy | The audio waveform as a PNG (`showwavespic`, 1800×140 by default; `width`, `height`). It is a private `asset` object and the source's `waveform` variant. A remote-only source over `MEDIA_PREVIEW_REMOTE_MAX_MB` (2048) is refused, because it decodes the whole audio track. `no_audio` is permanent |
| `object.waveform` | heavy | The audio waveform as a PNG (`showwavespic`, 1800×140 by default; `width`, `height`). It is a private `asset` object and the source's `waveform` variant. A remote-only source over `MEDIA_PREVIEW_REMOTE_MAX_MB` (2048) is refused, because it decodes the whole audio track. `no_audio` is permanent |
| `object.sprite` | heavy | A seek-preview sprite sheet as a JPEG: 100 frames by default, at most one per 2 s, 10 columns and 160 px tiles (`frames`, `columns`, `tile_width`). With `MEDIA_HLS_ENABLED` and a source timeline it is cut from the timeline's rows — each sample time's row decoded from this node's segment or the packed chunk's byte range, checked against the row's sha256 — so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame over the source. It is a private `asset` object and the source's `sprite` variant. The layout goes in its metadata as `sprite: { count, interval_seconds, columns, rows, tile_width, tile_height }`: frame *i* covers [*i*·interval, (*i*+1)·interval) at column *i* mod columns, row ⌊*i*/columns⌋. `no_video` is permanent |
| `object.sprite` | heavy | A seek-preview sprite sheet as a JPEG: 100 frames by default, at most one per 2 s, 10 columns and 160 px tiles (`frames`, `columns`, `tile_width`). With `MEDIA_HLS_ENABLED` and a source timeline it is cut from the timeline's rows — each sample time's row decoded from this node's segment or the packed chunk's byte range, checked against the row's sha256 — so a packed long VOD is never decoded end to end; a source without a timeline keeps one fast seek per frame over the source. It is a private `asset` object and the source's `sprite` variant. The layout goes in its metadata as `sprite: { count, interval_seconds, columns, rows, tile_width, tile_height }`: frame *i* covers [*i*·interval, (*i*+1)·interval) at column *i* mod columns, row ⌊*i*/columns⌋. `no_video` is permanent |
| `object.hash` | light | sha256 of ready objects' present local copies that have no hash yet, onto the object (`content_hash`, `metadata.hash_basis`) and the local location (`checksum`). Tenant-wide batch (`params { limit (1-500, 50), budget_mb }`) or one object. Scheduled per tenant every `MEDIA_HASH_INTERVAL_MIN`. Result `{ hashed, bytes, skipped, remaining }`. See [Scheduled verification](#scheduled-verification) |
| `object.hash` | light | sha256 of ready objects' present local copies that have no hash yet, onto the object (`content_hash`, `metadata.hash_basis`) and the local location (`checksum`). Tenant-wide batch (`params { limit (1-500, 50), budget_mb }`) or one object. Scheduled per tenant every `MEDIA_HASH_INTERVAL_MIN`. Result `{ hashed, bytes, skipped, remaining }`. See [Scheduled verification](#scheduled-verification) |
new file mode 100644
new file mode 100644
@@ -0,0 +1,31 @@
@@ -0,0 +1,31 @@
+/**
+ * Outcomes are announced as OpenVibe.Events events written in the state change's own
+ * transaction. Per-app webhooks are gone (C-75).
+ */
+'use strict';
+
+const db = require('./db/database');
+
+/**
+ * Commit an outcome and announce it. `change()` (synchronous DB writes, optional) and the outbox
+ * row for `event` run in ONE PostgreSQL transaction, so the durable event exists if and only if the
+ * change committed: never lost after a commit, never announced for a rollback. `payload()` is read
+ * inside that transaction (the row as committed). After the commit the relay is woken.
+ * Returns { data, eventId }; throws (nothing changed, nothing queued) when the transaction fails.
+ */
+async function announce(appId, event, { change = null, payload }) {
+ const events = require('./events');
+ let data = null;
+ let env = null;
+ await db.getDb().tx(async () => {
+ if (change) await change();
+ data = typeof payload === 'function' ? await payload() : payload;
+ env = await events.record(event, appId, data);
+ await events.recordObjectChanges(); // a row the change deleted: media.object.deleted, same transaction
+ });
+ events.kick();
+ const eventId = env ? env.event_id : null;
+ return { data, eventId };
+}
+
+module.exports = { announce };
@@ -108,7 +108,7 @@ async function storeAvatar({ db, pic, subject = null, username = null, sourceHos
@@ -108,7 +108,7 @@ async function storeAvatar({ db, pic, subject = null, username = null, sourceHos
const model = require('../objects/model');
const model = require('../objects/model');
const namespaces = require('../objects/namespaces');
const namespaces = require('../objects/namespaces');
const invariant = require('../objects/invariant');
const invariant = require('../objects/invariant');
- const { announce } = require('../webhooks');
+ const { announce } = require('../announce');
const app = await db.getApp(AVATAR_APP);
const app = await db.getApp(AVATAR_APP);
if (!app) throw new Error('Picture storage is not configured on this server');
if (!app) throw new Error('Picture storage is not configured on this server');
const { namespace } = namespaces.resolveName(app, AVATAR_NAMESPACE);
const { namespace } = namespaces.resolveName(app, AVATAR_NAMESPACE);
@@ -131,7 +131,7 @@ async function getApp(appId) {
@@ -131,7 +131,7 @@ async function getApp(appId) {
}
}
async function listApps() {
async function listApps() {
- return await all('SELECT app_id, name, webhook_url, allowed_origins, quota_bytes, created_at FROM apps ORDER BY app_id');
+ return await all('SELECT app_id, name, allowed_origins, quota_bytes, created_at FROM apps ORDER BY app_id');
}
}
async function upsertApp({ app_id, name, api_key, webhook_url, webhook_secret, allowed_origins, quota_bytes }) {
async function upsertApp({ app_id, name, api_key, webhook_url, webhook_secret, allowed_origins, quota_bytes }) {
@@ -10,7 +10,7 @@
@@ -10,7 +10,7 @@
* - assertSafe(): Media refuses to start unless DATABASE_URL is set and does not name production's database
* - assertSafe(): Media refuses to start unless DATABASE_URL is set and does not name production's database
* (ov_media on the host's 5432 or PgBouncer's 6432), DATABASE_DIRECT_URL is unset (a drill never migrates),
* (ov_media on the host's 5432 or PgBouncer's 6432), DATABASE_DIRECT_URL is unset (a drill never migrates),
* PORT is set and is not production's 4100, and HOST is loopback. It runs before the database is opened.
* PORT is set and is not production's 4100, and HOST is loopback. It runs before the database is opened.
- * - installGuards(): no outbound connection (net.Socket#connect, fetch), so no webhook, event, B2/R2
+ * - installGuards(): no outbound connection (net.Socket#connect, fetch), so no event, B2/R2
* or Network call leaves the process; no program other than git (child_process: no ffmpeg or
* or Network call leaves the process; no program other than git (child_process: no ffmpeg or
* ffprobe ever; git only reads the checkout for /release.json); no UDP socket (RTP ingest); nothing
* ffprobe ever; git only reads the checkout for /release.json); no UDP socket (RTP ingest); nothing
* listens but the drill's own HTTP port.
* listens but the drill's own HTTP port.
@@ -21,7 +21,7 @@
@@ -21,7 +21,7 @@
* - server/index.js opens the database and serves HTTP, and starts nothing else: no app seeding, no
* - server/index.js opens the database and serves HTTP, and starts nothing else: no app seeding, no
* JWKS refresh, no health job, junk sweep, tiering sweep, clip re-cuts, copy verification, jobs
* JWKS refresh, no health job, junk sweep, tiering sweep, clip re-cuts, copy verification, jobs
* worker (thumbnails, split/remux, invariant scans, storage orphan scans), disk guardian, thumbnail cleanup, object purge,
* worker (thumbnails, split/remux, invariant scans, storage orphan scans), disk guardian, thumbnail cleanup, object purge,
- * orphan-recording finalize or Events relay; webhooks are never sent. Timers that modules
+ * orphan-recording finalize or Events relay. Timers that modules
* start when they are loaded check `enabled` as well. Readiness does not probe storage (a drill
* start when they are loaded check `enabled` as well. Readiness does not probe storage (a drill
* writes none) or the remote tiers.
* writes none) or the remote tiers.
*
*
@@ -1,27 +1,25 @@
@@ -1,27 +1,25 @@
'use strict';
'use strict';
/**
/**
- * Media → OpenVibe.Events (roadmap Wave 3, ADR-004): the outcomes Media already announces by
- * per-app webhook are also written as durable events to Media's own `event_outbox` table and
- * relayed to OpenVibe.Events with Media's service token (audience openvibe.events, capability
- * events.event.publish). The webhooks stay as they are until every consumer reads events.
+ * Media → OpenVibe.Events (roadmap Wave 3, ADR-004): Media announces outcomes as durable events
+ * written to its own `event_outbox` table and relayed to OpenVibe.Events with Media's service
+ * token (audience openvibe.events, capability
+ * events.event.publish). Events are the only announcement path.
*
*
* vod.ready | vod.failed → media.vod.ready | media.vod.failed (subject vod <id>)
* vod.ready | vod.failed → media.vod.ready | media.vod.failed (subject vod <id>)
* clip.ready | clip.failed → media.clip.ready | media.clip.failed (subject clip <id>)
* clip.ready | clip.failed → media.clip.ready | media.clip.failed (subject clip <id>)
* media.object.uploaded → media.object.uploaded (subject object <id>)
* media.object.uploaded → media.object.uploaded (subject object <id>)
* storage.alert | storage.recovered → media.storage.alert | media.storage.recovered
* storage.alert | storage.recovered → media.storage.alert | media.storage.recovered
* job state changes (server/jobs/queue.js) → media.job.proposed | queued | started | retrying |
* job state changes (server/jobs/queue.js) → media.job.proposed | queued | started | retrying |
- * succeeded | failed | cancelled (subject job <id>; Events only, no webhook;
+ * succeeded | failed | cancelled (subject job <id>;
* payload queue.jobEvent, not the tenant's params/result/error)
* payload queue.jobEvent, not the tenant's params/result/error)
* an object deleted / its visibility changed → media.object.deleted | media.object.visibility_changed
* an object deleted / its visibility changed → media.object.deleted | media.object.visibility_changed
- * (subject object <id>; Events only; payload object id, tenant,
+ * (subject object <id>; payload object id, tenant,
* kind, legacy_ref and the times/visibilities, nothing else)
* kind, legacy_ref and the times/visibilities, nothing else)
*
*
* The outbox row is written INSIDE the PostgreSQL transaction that makes the state change it describes
* The outbox row is written INSIDE the PostgreSQL transaction that makes the state change it describes
- * (record(), called from webhooks.announce()): the event exists if and only if the change committed,
- * so a crash can neither lose an outcome nor announce one that rolled back. The webhook for the same
- * outcome is sent after the commit and carries the envelope's event_id, so a consumer receiving it by
- * both paths (Live during its webhook → Events transition) handles it once. Delivery from the outbox
- * on is at least once. Every event is `internal` visibility: a VOD's own visibility decides who may
+ * (record(), called from announce()): the event exists if and only if the change committed,
+ * so a crash can neither lose an outcome nor announce one that rolled back. Delivery from the outbox
+ * is at least once. Every event is `internal` visibility: a VOD's own visibility decides who may
* see it, and consumers (Live, Search) apply it. Lifecycle outcomes are `important` (roadmap §6.3);
* see it, and consumers (Live, Search) apply it. Lifecycle outcomes are `important` (roadmap §6.3);
* only storage.recovered is `low`.
* only storage.recovered is `low`.
*
*
@@ -42,7 +40,7 @@ const TYPES = {
@@ -42,7 +40,7 @@ const TYPES = {
'storage.recovered': ['media.storage.recovered', 'storage'],
'storage.recovered': ['media.storage.recovered', 'storage'],
};
};
-// Placement decisions have no webhook twin; callers record them in the outbox transaction.
+// Placement decisions are recorded in the outbox transaction.
const PLACEMENT_TYPES = new Set([
const PLACEMENT_TYPES = new Set([
'media.replica.requested', 'media.replica.ready', 'media.replica.draining', 'media.replica.evicted',
'media.replica.requested', 'media.replica.ready', 'media.replica.draining', 'media.replica.evicted',
'media.object.hot', 'media.region.hot', 'media.delivery.surge', 'media.delivery.degraded',
'media.object.hot', 'media.region.hot', 'media.delivery.surge', 'media.delivery.degraded',
@@ -111,10 +109,10 @@ function slim(appId, data) {
@@ -111,10 +109,10 @@ function slim(appId, data) {
* Queue the durable event for an outcome. MUST run inside the transaction that makes the change
* Queue the durable event for an outcome. MUST run inside the transaction that makes the change
* (the SDK outbox refuses otherwise), and throws if the insert fails, so the change rolls back with
* (the SDK outbox refuses otherwise), and throws if the insert fails, so the change rolls back with
* it. Returns the envelope (with event_id), or null when the outbox is off, the tenant is a sandbox
* it. Returns the envelope (with event_id), or null when the outbox is off, the tenant is a sandbox
- * or the webhook event has no durable twin.
+ * or the outcome has no mapped event type.
*/
*/
-async function record(webhookEvent, appId, data) {
- const map = TYPES[webhookEvent];
+async function record(outcome, appId, data) {
+ const map = TYPES[outcome];
if (!map) return null;
if (!map) return null;
const [eventType, subjectType] = map;
const [eventType, subjectType] = map;
const id = data && (data.object_id || data.id);
const id = data && (data.object_id || data.id);
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
*
*
* Multi-tenant media service: VOD ingest/recording (RTMP pull, RTP, browser
* Multi-tenant media service: VOD ingest/recording (RTMP pull, RTP, browser
* chunks), local/B2/R2 storage tiering with presigned-302 playback, clips,
* chunks), local/B2/R2 storage tiering with presigned-302 playback, clips,
- * files, thumbnails, legacy screenshot redirects, and HMAC-signed webhooks. See CONTRACTS.md
+ * files, thumbnails, legacy screenshot redirects, and OpenVibe.Events announcements. See CONTRACTS.md
* (Media API v1) and README.md.
* (Media API v1) and README.md.
*/
*/
'use strict';
'use strict';
@@ -338,7 +338,7 @@ const ready = (async () => {
@@ -338,7 +338,7 @@ const ready = (async () => {
console.log(`[Media] OpenVibe.Media listening on ${config.host}:${config.port} (${config.nodeEnv})`);
console.log(`[Media] OpenVibe.Media listening on ${config.host}:${config.port} (${config.nodeEnv})`);
console.log(`[Media] Data: db=${config.db.url ? 'postgresql' : 'pglite'} vods=${config.vod.path} clips=${config.vod.clipsPath}`);
console.log(`[Media] Data: db=${config.db.url ? 'postgresql' : 'pglite'} vods=${config.vod.path} clips=${config.vod.clipsPath}`);
console.log(`[Media] RTP ingest pool: udp ${config.rtp.portMin}-${config.rtp.portMax} (127.0.0.1)`);
console.log(`[Media] RTP ingest pool: udp ${config.rtp.portMin}-${config.rtp.portMax} (127.0.0.1)`);
- // Durable events (roadmap Wave 3): webhook outcomes also go to OpenVibe.Events. Off without EVENTS_URL.
+ // Durable outcome events (roadmap Wave 3) go to OpenVibe.Events. Off without EVENTS_URL.
try { require('./events').init(); } catch (err) { console.warn('[Events] not started:', err.message); }
try { require('./events').init(); } catch (err) { console.warn('[Events] not started:', err.message); }
// Media's own watch pages in OpenVibe.Search (WS-O task 10): media.index_document.* through the same outbox.
// Media's own watch pages in OpenVibe.Search (WS-O task 10): media.index_document.* through the same outbox.
require('./public/search-documents').start().catch((err) => console.warn('[Search] documents not started:', err.message));
require('./public/search-documents').start().catch((err) => console.warn('[Search] documents not started:', err.message));
@@ -5,7 +5,7 @@
@@ -5,7 +5,7 @@
* in media_object_tier_decisions. This job reads those rows: an object whose move failed FAIL_THRESHOLD times
* in media_object_tier_decisions. This job reads those rows: an object whose move failed FAIL_THRESHOLD times
* within the lookback (a later `done` of the same move starts the count over) raises one storage.alert of kind
* within the lookback (a later `done` of the same move starts the count over) raises one storage.alert of kind
* move_cleanup_failed, which vod-storage's emitStorageEvent counts in media_storage_alerts_total{kind} and
* move_cleanup_failed, which vod-storage's emitStorageEvent counts in media_storage_alerts_total{kind} and
- * delivers as the Events outbox row and per-app webhook. Report only: it never moves, deletes or changes a row.
+ * delivers as an Events outbox row. Report only: it never moves, deletes or changes a row.
*
*
* params {} service-wide; runs only under queue.SYSTEM_APP (no tenant can enqueue it)
* params {} service-wide; runs only under queue.SYSTEM_APP (no tenant can enqueue it)
* result { scanned, alerted, objects } objects = the first 50 (object_id, action, fails, first_at, last_at, last_error)
* result { scanned, alerted, objects } objects = the first 50 (object_id, action, fails, first_at, last_at, last_error)
@@ -9,8 +9,8 @@
@@ -9,8 +9,8 @@
* succeeded | failed | cancelled finished
* succeeded | failed | cancelled finished
*
*
* Every state change and its media.job.<transition> event commit in ONE PostgreSQL transaction (the same
* Every state change and its media.job.<transition> event commit in ONE PostgreSQL transaction (the same
- * rule as webhooks.announce(), see events.js): an event exists if and only if its change committed.
- * The relay is woken after the commit. Job events go to OpenVibe.Events only (no app webhook).
+ * rule as announce(), see events.js): an event exists if and only if its change committed.
+ * The relay is woken after the commit. Job events go to OpenVibe.Events.
*
*
* Fencing: a claim writes a fresh random lease_token on the row and hands it to its holder. Renewing
* Fencing: a claim writes a fresh random lease_token on the row and hands it to its holder. Renewing
* the lease, saving a checkpoint and every way out of 'running' (succeed, fail, markCancelled) match
* the lease, saving a checkpoint and every way out of 'running' (succeed, fail, markCancelled) match
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
* docs/object-model.md#operator-views).
* docs/object-model.md#operator-views).
*
*
* report({ appId, limit }) failed jobs, missing media, projection status, tiering diagnostics, the
* report({ appId, limit }) failed jobs, missing media, projection status, tiering diagnostics, the
- * namespaces' usage snapshot and the outbound-webhooks section; appId
+ * namespaces' usage snapshot; appId
* narrows everything that has a tenant (the tiering sweep and the provider
* narrows everything that has a tenant (the tiering sweep and the provider
* switches are service-wide)
* switches are service-wide)
* recompute({ appId }) refresh the namespaces' usage snapshot from the rows (namespaces.reconcile),
* recompute({ appId }) refresh the namespaces' usage snapshot from the rows (namespaces.reconcile),
@@ -19,7 +19,6 @@
@@ -19,7 +19,6 @@
const db = require('../db/database');
const db = require('../db/database');
const namespaces = require('../objects/namespaces');
const namespaces = require('../objects/namespaces');
const copyReport = require('../objects/copy-report');
const copyReport = require('../objects/copy-report');
-const webhookStats = require('../webhooks');
const iso = require('./explorer').iso;
const iso = require('./explorer').iso;
const clip = (s, n) => (s == null ? null : String(s).slice(0, n));
const clip = (s, n) => (s == null ? null : String(s).slice(0, n));
@@ -198,29 +197,6 @@ async function namespaceSnapshot(appId) {
@@ -198,29 +197,6 @@ async function namespaceSnapshot(appId) {
})));
})));
}
}
-// ── Webhooks ─────────────────────────────────────────────────
-
-/** Who is configured to receive outbound webhooks (host only, never the URL or secret) and what this
- * process has sent them since it loaded. */
-async function webhooks(appId) {
- const s = scoped(appId);
- const configured = (await db.all(`SELECT app_id, webhook_url FROM apps WHERE COALESCE(webhook_url, '') != ''${s.sql} ORDER BY app_id`, s.params))
- .map((r) => {
- let host = null;
- try { host = new URL(r.webhook_url).host; } catch { /* an unparseable URL shows as null */ }
- return { app_id: r.app_id, host };
- });
- const all = webhookStats.stats();
- const sends = appId
- ? { since: all.since, by_app: all.by_app[appId] ? { [appId]: all.by_app[appId] } : {} }
- : all;
- return {
- configured,
- sends,
- note: 'Outbound webhooks (C-83) retire when every configured app reads OpenVibe.Events; sends counts reset on restart.',
- };
-}
-
/** The whole operator report (see the header). `limit` bounds each list (1-200, default 50). */
/** The whole operator report (see the header). `limit` bounds each list (1-200, default 50). */
async function report({ appId = null, limit = 50 } = {}) {
async function report({ appId = null, limit = 50 } = {}) {
const n = Math.min(Math.max(parseInt(limit, 10) || 50, 1), 200);
const n = Math.min(Math.max(parseInt(limit, 10) || 50, 1), 200);
@@ -233,7 +209,6 @@ async function report({ appId = null, limit = 50 } = {}) {
@@ -233,7 +209,6 @@ async function report({ appId = null, limit = 50 } = {}) {
projection: await projection(appId),
projection: await projection(appId),
tiering: await tiering(appId, n),
tiering: await tiering(appId, n),
namespaces: await namespaceSnapshot(appId),
namespaces: await namespaceSnapshot(appId),
- webhooks: await webhooks(appId),
};
};
}
}
@@ -430,17 +430,8 @@ function renderOps({ person, report, canRecompute, recomputed = null }) {
@@ -430,17 +430,8 @@ function renderOps({ person, report, canRecompute, recomputed = null }) {
])) || '<p class="empty">None.</p>'}` : ''}
])) || '<p class="empty">None.</p>'}` : ''}
</section>
</section>
- <section aria-labelledby="webhooks-h">
- <h2 id="webhooks-h">Outbound webhooks</h2>
- <p class="hint">${esc(r.webhooks.note)}</p>
- <p class="hint">Sends counted since ${esc(fmtWhen(r.webhooks.sends.since))} in this process.</p>
- ${table('Apps configured to receive webhooks, host only', ['Tenant', 'Host', 'Sent', 'Failed', 'Last success', 'Last error'], r.webhooks.configured.map(c => {
- const s = r.webhooks.sends.by_app[c.app_id] || {};
- return [esc(c.app_id), esc(c.host || 'unparseable URL'), num(s.sent || 0), num(s.failed || 0), esc(fmtWhen(s.last_ok_at) || 'never'), s.last_error ? `<pre class="err">${esc(s.last_error)}</pre>` : ''];
- }), { numeric: [2, 3] }) || '<p class="empty">No app is configured to receive webhooks.</p>'}
- </section>
<p class="hint">JSON: <a href="/api/v2/me/ops${scopeQs}">/api/v2/me/ops</a>.</p>`;
<p class="hint">JSON: <a href="/api/v2/me/ops${scopeQs}">/api/v2/me/ops</a>.</p>`;
- return doc({ title: 'Media operations', description: 'Operator views of OpenVibe.Media: failed jobs, usage, missing media, projection, tiering and webhooks.', body });
+ return doc({ title: 'Media operations', description: 'Operator views of OpenVibe.Media: failed jobs, usage, missing media, projection and tiering.', body });
}
}
module.exports = {
module.exports = {
@@ -448,7 +448,7 @@ async function syncThumbnail(parentRow, parentObjectId, parentKind) {
@@ -448,7 +448,7 @@ async function syncThumbnail(parentRow, parentObjectId, parentKind) {
/**
/**
* Projection work in one transaction, with the object-change events it caused (server/events.js).
* Projection work in one transaction, with the object-change events it caused (server/events.js).
- * Nested in an outer transaction (webhooks.announce, another withObject) it is a savepoint, and the
+ * Nested in an outer transaction (announce, another withObject) it is a savepoint, and the
* outer one records the changes at its end, in the same transaction and after its own outcome event,
* outer one records the changes at its end, in the same transaction and after its own outcome event,
* the order they had when the object was synced after the commit.
* the order they had when the object was synced after the commit.
*/
*/
@@ -494,7 +494,7 @@ async function safeSync(kind, id) {
@@ -494,7 +494,7 @@ async function safeSync(kind, id) {
* an object behind its row. kind vod|clip|file; ids the row id (a file's key), a list of them,
* an object behind its row. kind vod|clip|file; ids the row id (a file's key), a list of them,
* or a function of write()'s result that gives them (an INSERT's lastInsertRowid). write() must be
* or a function of write()'s result that gives them (an INSERT's lastInsertRowid). write() must be
* database work (awaited). Returns write()'s result; throws, with nothing written, when either
* database work (awaited). Returns write()'s result; throws, with nothing written, when either
- * part fails. Inside an outer transaction (webhooks.announce) it is a savepoint of that one.
+ * part fails. Inside an outer transaction (announce) it is a savepoint of that one.
*
*
* The projection stats local files (sizes, present/missing copies) but never reads them: content
* The projection stats local files (sizes, present/missing copies) but never reads them: content
* hashes stay with the content-hash job and remote copies stay 'pending' until reconciliation or a
* hashes stay with the content-hash job and remote copies stay 'pending' until reconciliation or a
@@ -58,7 +58,7 @@ const multipart = require('./multipart');
@@ -58,7 +58,7 @@ const multipart = require('./multipart');
const namespaces = require('./namespaces');
const namespaces = require('./namespaces');
const { contentDisposition } = require('./disposition');
const { contentDisposition } = require('./disposition');
const { tenantAuth, tenantCors, tenantPath, namespaceGrant } = require('../auth');
const { tenantAuth, tenantCors, tenantPath, namespaceGrant } = require('../auth');
-const { announce } = require('../webhooks');
+const { announce } = require('../announce');
const { limits } = require('../actor-limits');
const { limits } = require('../actor-limits');
const MB = 1024 * 1024;
const MB = 1024 * 1024;
@@ -401,7 +401,7 @@ router.post('/', upload, limits('media.object.upload', UPLOAD), async (req, res)
@@ -401,7 +401,7 @@ router.post('/', upload, limits('media.object.upload', UPLOAD), async (req, res)
/**
/**
* The checks every upload ends with (single PUT and multipart): expected sha256, the public-size
* The checks every upload ends with (single PUT and multipart): expected sha256, the public-size
* invariant, the quota, and whether the bytes are what their type says; then the ready transition,
* invariant, the quota, and whether the bytes are what their type says; then the ready transition,
- * its invariant row and the media.object.uploaded event commit together (the webhook follows).
+ * its invariant row and the media.object.uploaded event commit together.
*/
*/
async function completeUpload(req, res, obj, { expectedHash = null } = {}) {
async function completeUpload(req, res, obj, { expectedHash = null } = {}) {
const loc = (await model.listLocations(obj.id)).find(l => l.provider === 'local' && l.state === 'present');
const loc = (await model.listLocations(obj.id)).find(l => l.provider === 'local' && l.state === 'present');
@@ -9,7 +9,7 @@
@@ -9,7 +9,7 @@
* local storage directory accepts a write. Optional (one capability degrades, the rest keeps
* local storage directory accepts a write. Optional (one capability degrades, the rest keeps
* working): the Network public key (user-JWT and service-token routes; app-key tenants are
* working): the Network public key (user-JWT and service-token routes; app-key tenants are
* unaffected), each CONFIGURED remote tier (B2 cold storage, R2 cache; a HeadBucket at most once a
* unaffected), each CONFIGURED remote tier (B2 cold storage, R2 cache; a HeadBucket at most once a
- * minute), the Events outbox (webhooks still deliver when it backs up), and object_copies: every
+ * minute), the Events outbox, and object_copies: every
* ready object has a good copy somewhere (server/objects/verify-job.js records it; objects with no
* ready object has a good copy somewhere (server/objects/verify-job.js records it; objects with no
* good copy are listed by scripts/no-good-copy-report.js). object_copies never fails readiness —
* good copy are listed by scripts/no-good-copy-report.js). object_copies never fails readiness —
* Media keeps serving everything else.
* Media keeps serving everything else.
@@ -145,7 +145,7 @@ function createMediaReadiness({ release, db, config, auth, recorder, events, rem
@@ -145,7 +145,7 @@ function createMediaReadiness({ release, db, config, auth, recorder, events, rem
checks.push({ name: `remote_${name}`, required: false, cacheMs: 60 * 1000, timeoutMs: 5000, check: () => remote.probe(name) });
checks.push({ name: `remote_${name}`, required: false, cacheMs: 60 * 1000, timeoutMs: 5000, check: () => remote.probe(name) });
}
}
checks.push({
checks.push({
- name: 'events_outbox', required: false, description: 'durable events to OpenVibe.Events (webhooks are separate)',
+ name: 'events_outbox', required: false, description: 'durable outcome events to OpenVibe.Events',
check: async () => {
check: async () => {
const s = await events.status();
const s = await events.status();
if (!s.enabled) return skip('relay off (EVENTS_URL or the service credentials unset)', { enabled: false }); // verified nothing: never ok (WS-Q task 7)
if (!s.enabled) return skip('relay off (EVENTS_URL or the service credentials unset)', { enabled: false }); // verified nothing: never ok (WS-Q task 7)
@@ -67,7 +67,7 @@ function bind(registry) {
@@ -67,7 +67,7 @@ function bind(registry) {
});
});
alerts = registry.counter({
alerts = registry.counter({
name: 'media_storage_alerts_total',
name: 'media_storage_alerts_total',
- help: 'Storage alerts emitted to /metrics, mirroring the storage.alert webhook kinds',
+ help: 'Storage alerts emitted to /metrics, mirroring the storage.alert event kinds',
labelNames: ['kind'],
labelNames: ['kind'],
});
});
@@ -97,4 +97,4 @@ function observeLatency(provider, op, ms) { if (latency && ms > 0) latency.obser
@@ -97,4 +97,4 @@ function observeLatency(provider, op, ms) { if (latency && ms > 0) latency.obser
function observeError(provider, op) { errors && errors.inc({ provider, op }, 1); }
function observeError(provider, op) { errors && errors.inc({ provider, op }, 1); }
function alert(kind) { alerts && alerts.inc({ kind }, 1); }
function alert(kind) { alerts && alerts.inc({ kind }, 1); }
-module.exports = { bind, inc, observeLatency, observeError, alert, _reset: () => { latency = errors = breakerGauge = decisions = presign = alerts = demandSource = null; } };
\ No newline at end of file
\ No newline at end of file
+module.exports = { bind, inc, observeLatency, observeError, alert, _reset: () => { latency = errors = breakerGauge = decisions = presign = alerts = demandSource = null; } };
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
*
*
* recutClip(clipId, { reason }) the ONE code path that (re)cuts an existing clip row:
* recutClip(clipId, { reason }) the ONE code path that (re)cuts an existing clip row:
* resolves the source, cuts, updates the row, thumbnails,
* resolves the source, cuts, updates the row, thumbnails,
- * commits the outcome with its event, fires the webhook. Records cut_error / cut_attempts and
+ * commits the outcome with its event. Records cut_error / cut_attempts and
* schedules an automatic retry on failure.
* schedules an automatic retry on failure.
* spec the media job clip.cut (lane clips; roadmap WS-G task 3): a new clip's
* spec the media job clip.cut (lane clips; roadmap WS-G task 3): a new clip's
* cut and every re-cut run as a job, so they carry media.job.* events and a
* cut and every re-cut run as a job, so they carry media.job.* events and a
@@ -31,7 +31,7 @@ const db = require('../db/database');
@@ -31,7 +31,7 @@ const db = require('../db/database');
const config = require('../config');
const config = require('../config');
const cutter = require('./clip-cutter');
const cutter = require('./clip-cutter');
const vodStorage = require('./vod-storage');
const vodStorage = require('./vod-storage');
-const { announce } = require('../webhooks');
+const { announce } = require('../announce');
const objects = () => require('../objects/model');
const objects = () => require('../objects/model');
const MAX_ATTEMPTS = 4;
const MAX_ATTEMPTS = 4;
@@ -113,7 +113,7 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false, job = null
@@ -113,7 +113,7 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false, job = null
const cut = await cutter.cutClipFile({ source: source.value, startTime, duration });
const cut = await cutter.cutClipFile({ source: source.value, startTime, duration });
if (!cut.ok) return await fail(clipId, clip, attempt, cut.error, { viaJob });
if (!cut.ok) return await fail(clipId, clip, attempt, cut.error, { viaJob });
// Thumbnail first (the clip.ready payload carries it), then the ready transition, the clip's
// Thumbnail first (the clip.ready payload carries it), then the ready transition, the clip's
- // object and its event in one transaction (webhooks.announce), then the webhook.
+ // object and its event in one transaction (announce).
try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }
try { await require('../thumbnails/thumbnail-service').generateClipThumbnail(clipId, cut.filePath); } catch { /* */ }
await announce(clip.app_id, 'clip.ready', {
await announce(clip.app_id, 'clip.ready', {
change: async () => {
change: async () => {
@@ -205,7 +205,7 @@ async function timelineWindow(vod, startTime, endTime) {
@@ -205,7 +205,7 @@ async function timelineWindow(vod, startTime, endTime) {
return { objectId: obj.id, endTime: startTime + Number(last.end_ms) / 1000 };
return { objectId: obj.id, endTime: startTime + Number(last.end_ms) / 1000 };
}
}
-/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction, then the webhook. */
+/** Make a clip row virtual and ready (no ffmpeg): row, object and clip.ready event in one transaction. */
async function makeVirtual(clipId, win) {
async function makeVirtual(clipId, win) {
const clip = await db.getClipById(clipId);
const clip = await db.getClipById(clipId);
if (!clip) return { ok: false, error: 'Clip not found' };
if (!clip) return { ok: false, error: 'Clip not found' };
@@ -3,7 +3,7 @@
@@ -3,7 +3,7 @@
*
*
* POST / { vod_id, start_s, end_s, title?, user_id? } → { id, status }
* POST / { vod_id, start_s, end_s, title?, user_id? } → { id, status }
* The cut runs in the background (works on the local file or a
* The cut runs in the background (works on the local file or a
- * presigned B2/R2 URL); webhook clip.ready|clip.failed fires on
+ * presigned B2/R2 URL); clip.ready|clip.failed events are queued on
* completion. Also accepts a live cut against a still-recording
* completion. Also accepts a live cut against a still-recording
* VOD (clamped to flushed footage, like the predecessor).
* VOD (clamped to flushed footage, like the predecessor).
* GET /:id clip meta (status: processing | ready | failed)
* GET /:id clip meta (status: processing | ready | failed)
@@ -143,8 +143,8 @@ const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].incl
@@ -143,8 +143,8 @@ const flag = (v) => (['1', 'true'].includes(String(v)) ? 1 : ['0', 'false'].incl
// ── Create clip ──────────────────────────────────────────────
// ── Create clip ──────────────────────────────────────────────
// JSON body: a window of a VOD. Over a source with a CMAF timeline (MEDIA_HLS_ENABLED) the clip is virtual: ready at
// JSON body: a window of a VOD. Over a source with a CMAF timeline (MEDIA_HLS_ENABLED) the clip is virtual: ready at
-// once (201, clip.ready webhook), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).
-// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut + webhook).
+// once (201, clip.ready event), no ffmpeg, played as HLS over the source's segments (docs/media-fabric.md §4, F3.5).
+// With `materialize: true`, or a source without a timeline, it is cut into its own file (202, background clip.cut event).
// Multipart `video`: direct upload of an already-cut blob (201, ready) —
// Multipart `video`: direct upload of an already-cut blob (201, ready) —
// inherited browser-MediaRecorder clip path.
// inherited browser-MediaRecorder clip path.
router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {
router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.single('video'), async (req, res) => {
@@ -241,7 +241,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
@@ -241,7 +241,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a
// The cut is a media job (clip.cut, WS-G task 3): media.job.* events, retries with backoff, and a
// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.
// job id the caller's UI follows (GET /api/v2/:app/jobs/:id) and reattaches to after a reload.
- // clip.ready / clip.failed and the webhook come from the cut itself, as before.
+ // clip.ready / clip.failed events come from the cut itself.
const { job } = await clipJobs.enqueueCut(appId, clipId, {
const { job } = await clipJobs.enqueueCut(appId, clipId, {
reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,
reason: 'cut', createdBy: req.authType === 'user' ? `app:${appId}:user:${req.userId}` : `app:${appId}`, ownerUserId: userId ?? null,
});
});
@@ -257,7 +257,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
@@ -257,7 +257,7 @@ router.post('/', tenantAuth({ verb: 'write', allowUser: true }), clipUpload.sing
/**
/**
* Direct clip upload: the blob is already cut client-side — remux it for
* Direct clip upload: the blob is already cut client-side — remux it for
* seeking (rebases cluster timestamps hours from zero on long streams),
* seeking (rebases cluster timestamps hours from zero on long streams),
- * probe the real duration, store as ready. No webhook (nothing async).
+ * probe the real duration, store as ready. There is no asynchronous outcome.
*/
*/
async function _createUploadedClip(req, res) {
async function _createUploadedClip(req, res) {
const body = req.body || {};
const body = req.body || {};
@@ -5,7 +5,7 @@
@@ -5,7 +5,7 @@
* vodId (there is no stream registry here — apps own stream state):
* vodId (there is no stream registry here — apps own stream state):
* merge pending chunk segments → seekable remux → probe → master recovery
* merge pending chunk segments → seekable remux → probe → master recovery
* (truncated-webm rebuild) → thumbnail → DB update + vod.ready event (one
* (truncated-webm rebuild) → thumbnail → DB update + vod.ready event (one
- * transaction) → webhook vod.ready.
+ * transaction).
*
*
* Guarded against double-invocation; refuses to finalize while the recorder
* Guarded against double-invocation; refuses to finalize while the recorder
* still holds the file open (stops it gracefully instead — its ffmpeg exit
* still holds the file open (stops it gracefully instead — its ffmpeg exit
@@ -19,7 +19,7 @@ const { spawn } = require('child_process');
@@ -19,7 +19,7 @@ const { spawn } = require('child_process');
const db = require('../db/database');
const db = require('../db/database');
const config = require('../config');
const config = require('../config');
const tools = require('./media-tools');
const tools = require('./media-tools');
-const { announce } = require('../webhooks');
+const { announce } = require('../announce');
const objects = () => require('../objects/model');
const objects = () => require('../objects/model');
/** Absolute-ize a stored Media-relative URL (/t/…, /api/thumbnails/…). */
/** Absolute-ize a stored Media-relative URL (/t/…, /api/thumbnails/…). */
@@ -35,9 +35,9 @@ function isFinalizing(vodId) {
@@ -35,9 +35,9 @@ function isFinalizing(vodId) {
return _finalizing.has(Number(vodId));
return _finalizing.has(Number(vodId));
}
}
-// Public shape of a vod row for API responses + webhooks. { readiness: true } (API responses) adds the
-// object's readiness levels (server/objects/readiness.js); webhook payloads leave them out (their shape
-// predates the object model and stays as it is). `hls` selects the playlist fields: undefined = discover
+// Public shape of a vod row for API responses + events. { readiness: true } (API responses) adds the
+// object's readiness levels (server/objects/readiness.js); event payloads leave them out.
+// `hls` selects the playlist fields: undefined = discover
// them for this row (a single-item read), false = omit them (an event payload must never carry a signed
// them for this row (a single-item read), false = omit them (an event payload must never carry a signed
// URL into the outbox), an object = the answer the list route already batched (objects/hls.discoveryMany).
// URL into the outbox), an object = the answer the list route already batched (objects/hls.discoveryMany).
async function vodPublic(vod, { readiness = false, hls } = {}) {
async function vodPublic(vod, { readiness = false, hls } = {}) {
@@ -119,8 +119,8 @@ function rebuildWebmFromMaster(masterPath, webmPath) {
@@ -119,8 +119,8 @@ function rebuildWebmFromMaster(masterPath, webmPath) {
}
}
/**
/**
- * The state change for a VOD outcome, the VOD's object and the event/webhook, committed together
- * (webhooks.announce around objects.withObject). `row()` gives the row the payload describes (read
+ * The state change for a VOD outcome, the VOD's object and the event commit together
+ * (announce around objects.withObject). `row()` gives the row the payload describes (read
* inside the transaction). Returns the payload, or null when the transaction failed (then nothing
* inside the transaction). Returns the payload, or null when the transaction failed (then nothing
* changed and nothing was announced). A deleted row's object is marked deleted by its trigger.
* changed and nothing was announced). A deleted row's object is marked deleted by its trigger.
*/
*/
@@ -381,8 +381,8 @@ async function _doFinalize(vodId, opts) {
@@ -381,8 +381,8 @@ async function _doFinalize(vodId, opts) {
// recording is back to the visibility its owner chose.
// recording is back to the visibility its owner chose.
const lift = _finalizeQuarantined(await db.getVodById(vodId) || vod);
const lift = _finalizeQuarantined(await db.getVodById(vodId) || vod);
- // The ready transition, the VOD's object and the vod.ready event commit together (then the
- // webhook goes out). A failed commit throws, as the plain UPDATE did before.
+ // The ready transition, the VOD's object and the vod.ready event commit together.
+ // A failed commit throws, as the plain UPDATE did before.
await announce(vod.app_id, 'vod.ready', {
await announce(vod.app_id, 'vod.ready', {
change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,
change: async () => await objects().withObject('vod', vodId, async () => await db.run(`UPDATE vods SET is_recording = 0, duration_seconds = ?, duration_source = ?, file_size = ?, probe_duration_seconds = ?, probe_format_json = ?, health_status = ?, health_issues_json = ?${lift ? ", quarantined_at = NULL, is_public = CASE WHEN COALESCE(visibility, 'public') = 'public' THEN 1 ELSE 0 END" : ''} WHERE id = ?`,
[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),
[stored, durationSource, stat.size, measured, probeFormatJson, 'ok', JSON.stringify(issues), vodId])),
@@ -269,7 +269,7 @@ class StreamRecorder {
@@ -269,7 +269,7 @@ class StreamRecorder {
ffmpegCorrupted: rec?._ffmpegCorrupted || false,
ffmpegCorrupted: rec?._ffmpegCorrupted || false,
};
};
this._teardown(vodId);
this._teardown(vodId);
- // Let finalizeVod handle remux, probe, thumbnail, webhook.
+ // Let finalizeVod handle remux, probe, thumbnail and outcome event.
// Short delay to ensure the file is fully flushed to disk.
// Short delay to ensure the file is fully flushed to disk.
setTimeout(() => {
setTimeout(() => {
require('./finalize').finalizeVod(vodId, finalizeOpts).catch(err => {
require('./finalize').finalizeVod(vodId, finalizeOpts).catch(err => {
@@ -974,9 +974,7 @@ const sweepState = {
@@ -974,9 +974,7 @@ const sweepState = {
const alertLastSentAt = new Map(); // kind → epoch ms
const alertLastSentAt = new Map(); // kind → epoch ms
/**
/**
- * Tell every app with a webhook that storage needs a human (or that it recovered).
- * Delivered as the regular Media→app webhook (`storage.alert` / `storage.recovered`),
- * so Live can log it loudly and page whatever ops channel it has configured.
+ * Announce that storage needs a human (or that it recovered) through OpenVibe.Events.
*/
*/
async function emitStorageEvent(event, kind, data, settings = getSettings()) {
async function emitStorageEvent(event, kind, data, settings = getSettings()) {
const cooldownKey = `${event}:${kind}`;
const cooldownKey = `${event}:${kind}`;
@@ -989,19 +987,13 @@ async function emitStorageEvent(event, kind, data, settings = getSettings()) {
@@ -989,19 +987,13 @@ async function emitStorageEvent(event, kind, data, settings = getSettings()) {
const payload = { kind, ...data, at: new Date().toISOString() };
const payload = { kind, ...data, at: new Date().toISOString() };
(event === 'storage.alert' ? console.error : console.warn)(`[VodStorage] ${event} (${kind}): ${JSON.stringify(data)}`);
(event === 'storage.alert' ? console.error : console.warn)(`[VodStorage] ${event} (${kind}): ${JSON.stringify(data)}`);
// No database state changes here (the cooldown is in memory): the outbox row is the whole
// No database state changes here (the cooldown is in memory): the outbox row is the whole
- // commit, queued once for every app; each app's webhook carries its event_id.
- let eventId = null;
+ // commit, queued once for all consumers.
+ // The alert fired once it passed the cooldown (counted, logged); the event is best effort, as before.
try {
try {
const events = require('../events');
const events = require('../events');
- const env = await db.getDb().tx(async () => await events.record(event, null, payload));
- eventId = env ? env.event_id : null;
+ await db.getDb().tx(async () => await events.record(event, null, payload));
events.kick();
events.kick();
} catch (err) { console.warn(`[VodStorage] ${event} not queued for Events:`, err.message); }
} catch (err) { console.warn(`[VodStorage] ${event} not queued for Events:`, err.message); }
- let apps = [];
- try { apps = (await db.listApps()).filter(a => a.webhook_url); } catch { apps = []; }
- let webhooks;
- try { webhooks = require('../webhooks'); } catch { return false; }
- await Promise.all(apps.map(app => webhooks.sendWebhook(app.app_id, event, payload, { eventId }).catch(() => false)));
return true;
return true;
}
}
@@ -1095,7 +1087,7 @@ async function runSweep() {
@@ -1095,7 +1087,7 @@ async function runSweep() {
// Stall detection: a pressure pass that freed nothing is the failure mode that
// Stall detection: a pressure pass that freed nothing is the failure mode that
// went unnoticed for weeks. Count them, and after a couple in a row raise an
// went unnoticed for weeks. Count them, and after a couple in a row raise an
- // alert through the app webhooks; announce recovery once the drain works again.
+ // alert; announce recovery once the drain works again.
const diskAfter = diskUsage(config.vod.path);
const diskAfter = diskUsage(config.vod.path);
const stillNeedsDrain = underPressure && !drainSatisfied(diskAfter, settings);
const stillNeedsDrain = underPressure && !drainSatisfied(diskAfter, settings);
if (underPressure && !migrated) {
if (underPressure && !migrated) {
deleted file mode 100644
deleted file mode 100644
@@ -1,138 +0,0 @@
@@ -1,138 +0,0 @@
-/**
- * OpenVibe.Media — Outbound webhooks
- *
- * POSTs { event, app_id, data } to the app's configured webhook_url with
- * `X-OVMedia-Signature: sha256=<hmac>` — HMAC-SHA256 of the raw body using the
- * app's webhook_secret (per CONTRACTS.md). Fire-and-forget with small retries.
- *
- * Events: vod.ready | vod.failed | clip.ready | clip.failed | media.object.uploaded |
- * storage.alert | storage.recovered
- *
- * Outcomes go through announce(): the state change and its durable OpenVibe.Events outbox row
- * commit in one PostgreSQL transaction, then the webhook is sent with `event_id` = that event's id
- * (absent when the outbox is off), so a consumer that also reads Events dedupes the pair.
- */
-'use strict';
-
-const crypto = require('crypto');
-const db = require('./db/database');
-
-const MAX_ATTEMPTS = 3;
-const RETRY_BASE_MS = 3000;
-const TIMEOUT_MS = 10000;
-
-// In-memory tally of what this process actually tried to send, per app (C-83: who still gets webhooks
-// is what decides when the outbound path can retire). Never holds the URL, secret, body or signature.
-const SINCE = new Date().toISOString();
-const tallies = new Map();
-
-function tallyFor(appId) {
- let t = tallies.get(appId);
- if (!t) { t = { sent: 0, failed: 0, last_event: null, last_attempt_at: null, last_ok_at: null, last_error: null }; tallies.set(appId, t); }
- return t;
-}
-
-/** Fresh copy of the in-process tallies: { since, by_app: { [app_id]: {...} } }. */
-function stats() {
- const by_app = {};
- for (const [appId, t] of tallies) by_app[appId] = { ...t };
- return { since: SINCE, by_app };
-}
-
-/** Test-only: forget every tally. */
-function _resetStats() { tallies.clear(); }
-
-// Test-only: the base of the retry wait (null restores RETRY_BASE_MS), so a test never patches the
-// global setTimeout the database pool's own timers use.
-let retryBaseMs = RETRY_BASE_MS;
-function _setRetryBaseMs(ms) { retryBaseMs = ms == null ? RETRY_BASE_MS : ms; }
-
-function sign(secret, rawBody) {
- return 'sha256=' + crypto.createHmac('sha256', String(secret || '')).update(rawBody).digest('hex');
-}
-
-async function _post(url, rawBody, signature) {
- const res = await fetch(url, {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'X-OVMedia-Signature': signature,
- },
- body: rawBody,
- signal: AbortSignal.timeout(TIMEOUT_MS),
- });
- if (!res.ok) throw new Error(`HTTP ${res.status}`);
-}
-
-/**
- * Send an event to an app's webhook. Silently no-ops when the app has no
- * webhook_url. Never throws. Queues nothing: the durable event is written by
- * announce() (or the caller) inside the state change's transaction.
- * @param {string|object} appOrId app_id or apps row
- * @param {string} event e.g. 'vod.ready'
- * @param {object} data event payload
- * @param {object} [opts]
- * @param {string} [opts.eventId] the OpenVibe.Events event_id of the same outcome
- */
-async function sendWebhook(appOrId, event, data, { eventId = null } = {}) {
- // A restore drill (MEDIA_DRILL) never calls an app: the URLs in its apps table are production's.
- if (require('./drill').enabled) return false;
- let app = appOrId;
- if (typeof appOrId === 'string') {
- try { app = await db.getApp(appOrId); } catch { app = null; }
- }
- if (!app || !app.webhook_url) return false;
-
- const body = { event, app_id: app.app_id, data };
- if (eventId) body.event_id = eventId;
- const rawBody = JSON.stringify(body);
- const signature = sign(app.webhook_secret, rawBody);
-
- const t = tallyFor(app.app_id);
- for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
- t.last_event = event;
- t.last_attempt_at = new Date().toISOString();
- try {
- await _post(app.webhook_url, rawBody, signature);
- t.sent++;
- t.last_ok_at = new Date().toISOString();
- t.last_error = null;
- return true;
- } catch (err) {
- if (attempt === MAX_ATTEMPTS) {
- t.failed++;
- t.last_error = String((err && err.message) || err).slice(0, 200);
- console.warn(`[Webhooks] ${event} → ${app.app_id} failed after ${MAX_ATTEMPTS} attempts: ${err.message}`);
- return false;
- }
- await new Promise(r => setTimeout(r, retryBaseMs * attempt));
- }
- }
- return false;
-}
-
-/**
- * Commit an outcome and announce it. `change()` (synchronous DB writes, optional) and the outbox
- * row for `event` run in ONE PostgreSQL transaction, so the durable event exists if and only if the
- * change committed: never lost after a commit, never announced for a rollback. `payload()` is read
- * inside that transaction (the row as committed). After the commit the relay is woken and the
- * app's webhook is sent with the same event_id. Returns { data, eventId }; throws (nothing changed,
- * nothing queued, no webhook) when the transaction fails.
- */
-async function announce(appId, event, { change = null, payload }) {
- const events = require('./events');
- let data = null;
- let env = null;
- await db.getDb().tx(async () => {
- if (change) await change();
- data = typeof payload === 'function' ? await payload() : payload;
- env = await events.record(event, appId, data);
- await events.recordObjectChanges(); // a row the change deleted: media.object.deleted, same transaction
- });
- events.kick();
- const eventId = env ? env.event_id : null;
- sendWebhook(appId, event, data, { eventId }).catch(() => {});
- return { data, eventId };
-}
-
-module.exports = { sendWebhook, announce, sign, stats, _resetStats, _setRetryBaseMs };
@@ -7,10 +7,10 @@
@@ -7,10 +7,10 @@
* DATABASE_DIRECT_URL is set, when HOST is not loopback or PORT is production's 4100, before the
* DATABASE_DIRECT_URL is set, when HOST is not loopback or PORT is production's 4100, before the
* database is opened (a real
* database is opened (a real
* `node server/index.js`, in a child process).
* `node server/index.js`, in a child process).
- * - Booted for real (server/index.js, in this process) on a copy whose `live` app has a webhook
- * URL, with production-like settings (EVENTS_URL + client secret, B2/R2, app seeding, VOD_PATH …):
+ * - Booted for real (server/index.js, in this process) on a copy with production-like settings
+ * (EVENTS_URL + client secret, B2/R2, app seeding, VOD_PATH …):
* one listener, its HTTP port on 127.0.0.1; no setInterval and no long timer from Media's code;
* one listener, its HTTP port on 127.0.0.1; no setInterval and no long timer from Media's code;
- * no program but git (no ffmpeg/ffprobe); no outbound connection; no webhook reaches the app;
+ * no program but git (no ffmpeg/ffprobe); no outbound connection;
* nothing written or created outside the copy's directory.
* nothing written or created outside the copy's directory.
* - The endpoints the Host inventory compares (/release.json, the /browse index) and the watch pages
* - The endpoints the Host inventory compares (/release.json, the /browse index) and the watch pages
* render from the copy; every route that would stream a stored file answers 503 without opening
* render from the copy; every route that would stream a stored file answers 503 without opening
@@ -105,11 +105,10 @@ const { spawnSync, execFileSync } = require('child_process');
@@ -105,11 +105,10 @@ const { spawnSync, execFileSync } = require('child_process');
});
});
// ── Production's database and files (a normal-mode process writes them) ──
// ── Production's database and files (a normal-mode process writes them) ──
- // The `live` app's webhook goes to a sink here: a drill must never call it.
+ // A local sink helps verify the drill's connection guard.
const sinkHits = [];
const sinkHits = [];
const sink = http.createServer((req, res) => { sinkHits.push(`${req.method} ${req.url}`); res.end('ok'); });
const sink = http.createServer((req, res) => { sinkHits.push(`${req.method} ${req.url}`); res.end('ok'); });
await new Promise((r) => sink.listen(0, '127.0.0.1', r));
await new Promise((r) => sink.listen(0, '127.0.0.1', r));
- const SINK = `http://127.0.0.1:${sink.address().port}/media-webhook`;
for (const d of Object.values(P)) fs.mkdirSync(d, { recursive: true });
for (const d of Object.values(P)) fs.mkdirSync(d, { recursive: true });
const bytes = { vod: path.join(P.vods, 'vod-1.mp4'), rec: path.join(P.vods, 'vod-2.mp4'), clip: path.join(P.clips, 'clip-1.mp4'), asset: path.join(P.assets, 'wave.png'), shot: path.join(P.pastes, 'shot-1.png'), file: path.join(P.files, 'live', 'f1.txt') };
const bytes = { vod: path.join(P.vods, 'vod-1.mp4'), rec: path.join(P.vods, 'vod-2.mp4'), clip: path.join(P.clips, 'clip-1.mp4'), asset: path.join(P.assets, 'wave.png'), shot: path.join(P.pastes, 'shot-1.png'), file: path.join(P.files, 'live', 'f1.txt') };
fs.mkdirSync(path.dirname(bytes.file), { recursive: true });
fs.mkdirSync(path.dirname(bytes.file), { recursive: true });
@@ -133,7 +132,7 @@ const { spawnSync, execFileSync } = require('child_process');
@@ -133,7 +132,7 @@ const { spawnSync, execFileSync } = require('child_process');
// The restored copy's rows (written before the drill boots, as production wrote them).
// The restored copy's rows (written before the drill boots, as production wrote them).
const db = require('../server/db/database');
const db = require('../server/db/database');
await db.initDb();
await db.initDb();
- await db.run("INSERT INTO apps (app_id, name, api_key_hash, webhook_url, webhook_secret) VALUES ('live', 'OpenVibe.Live', ?, ?, 'whsec')", [require('crypto').createHash('sha256').update('k'.repeat(40)).digest('hex'), SINK]);
+ await db.run("INSERT INTO apps (app_id, name, api_key_hash) VALUES ('live', 'OpenVibe.Live', ?)", [require('crypto').createHash('sha256').update('k'.repeat(40)).digest('hex')]);
await db.run("INSERT INTO vods (id, app_id, title, file_path, thumbnail_url, duration_seconds, is_public, visibility) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 'Restored VOD', ?, '/t/vod-1-1.jpg', 60, 1, 'public')", [bytes.vod]);
await db.run("INSERT INTO vods (id, app_id, title, file_path, thumbnail_url, duration_seconds, is_public, visibility) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 'Restored VOD', ?, '/t/vod-1-1.jpg', 60, 1, 'public')", [bytes.vod]);
await db.run("INSERT INTO vods (id, app_id, title, file_path, is_public, visibility, is_recording) OVERRIDING SYSTEM VALUE VALUES (2, 'live', 'Still recording', ?, 1, 'public', 1)", [bytes.rec]);
await db.run("INSERT INTO vods (id, app_id, title, file_path, is_public, visibility, is_recording) OVERRIDING SYSTEM VALUE VALUES (2, 'live', 'Still recording', ?, 1, 'public', 1)", [bytes.rec]);
await db.run("INSERT INTO clips (id, app_id, vod_id, title, file_path, duration_seconds, status) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 1, 'Restored clip', ?, 10, 'ready')", [bytes.clip]);
await db.run("INSERT INTO clips (id, app_id, vod_id, title, file_path, duration_seconds, status) OVERRIDING SYSTEM VALUE VALUES (1, 'live', 1, 'Restored clip', ?, 10, 'ready')", [bytes.clip]);
@@ -206,7 +205,7 @@ const { spawnSync, execFileSync } = require('child_process');
@@ -206,7 +205,7 @@ const { spawnSync, execFileSync } = require('child_process');
assert.deepStrictEqual(spied.intervals, [], JSON.stringify(spied.intervals, null, 1));
assert.deepStrictEqual(spied.intervals, [], JSON.stringify(spied.intervals, null, 1));
assert.deepStrictEqual(spied.longTimeouts, [], JSON.stringify(spied.longTimeouts, null, 1));
assert.deepStrictEqual(spied.longTimeouts, [], JSON.stringify(spied.longTimeouts, null, 1));
});
});
- await check('no program but git ran, nothing tried to leave, no webhook reached the app', () => {
+ await check('no program but git ran, nothing tried to leave', () => {
assert.ok(spied.programs.every((p) => path.basename(p) === 'git'), spied.programs.join(', '));
assert.ok(spied.programs.every((p) => path.basename(p) === 'git'), spied.programs.join(', '));
assert.deepStrictEqual(drill.blocked, []);
assert.deepStrictEqual(drill.blocked, []);
assert.deepStrictEqual(sinkHits, []);
assert.deepStrictEqual(sinkHits, []);
@@ -272,13 +271,12 @@ const { spawnSync, execFileSync } = require('child_process');
@@ -272,13 +271,12 @@ const { spawnSync, execFileSync } = require('child_process');
assert.deepStrictEqual(drill.blocked, [], 'nothing tried to leave');
assert.deepStrictEqual(drill.blocked, [], 'nothing tried to leave');
assert.deepStrictEqual(await counts(), countsBefore);
assert.deepStrictEqual(await counts(), countsBefore);
});
});
- await check('webhooks and the Events relay stay off even when called', async () => {
- assert.strictEqual(await require('../server/webhooks').sendWebhook('live', 'vod.ready', { id: 1 }), false);
+ await check('the Events relay stays off even when called', async () => {
assert.strictEqual(require('../server/events').init(), null);
assert.strictEqual(require('../server/events').init(), null);
assert.deepStrictEqual(sinkHits, []);
assert.deepStrictEqual(sinkHits, []);
});
});
await check('the guards refuse a connection, ffmpeg, a listener and a UDP socket', async () => {
await check('the guards refuse a connection, ffmpeg, a listener and a UDP socket', async () => {
- await assert.rejects(fetch('https://openvibe.live/internal/media-webhook'), /fetch failed/);
+ await assert.rejects(fetch('https://openvibe.live/internal/probe'), /fetch failed/);
const sock = net.connect({ host: '127.0.0.1', port: sink.address().port });
const sock = net.connect({ host: '127.0.0.1', port: sink.address().port });
const err = await new Promise((resolve) => sock.on('error', resolve));
const err = await new Promise((resolve) => sock.on('error', resolve));
assert.strictEqual(err.code, 'ECONNREFUSED');
assert.strictEqual(err.code, 'ECONNREFUSED');
@@ -41,7 +41,7 @@ const http = require('http');
@@ -41,7 +41,7 @@ const http = require('http');
const db = require('../server/db/database');
const db = require('../server/db/database');
db.getDb();
db.getDb();
const events = require('../server/events');
const events = require('../server/events');
- const { sendWebhook, announce } = require('../server/webhooks');
+ const { announce } = require('../server/announce');
const inTx = async (fn) => await db.getDb().tx(fn);
const inTx = async (fn) => await db.getDb().tx(fn);
assert.strictEqual(events.init({ eventsUrl: '', clientSecret: '' }), null, 'off without EVENTS_URL');
assert.strictEqual(events.init({ eventsUrl: '', clientSecret: '' }), null, 'off without EVENTS_URL');
@@ -50,12 +50,10 @@ const http = require('http');
@@ -50,12 +50,10 @@ const http = require('http');
const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 50 });
const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 50 });
assert.ok(outbox);
assert.ok(outbox);
- // An app with no webhook URL still gets the durable event.
+ // An outcome gets the durable event.
await announce('no-such-app', 'vod.ready', { payload: { id: 42, title: 'Stream', meta: { big: 'x'.repeat(100000) }, ai_overview: 'long text' } });
await announce('no-such-app', 'vod.ready', { payload: { id: 42, title: 'Stream', meta: { big: 'x'.repeat(100000) }, ai_overview: 'long text' } });
await announce('no-such-app', 'clip.failed', { payload: { id: 7, status: 'failed' } });
await announce('no-such-app', 'clip.failed', { payload: { id: 7, status: 'failed' } });
assert.strictEqual((await announce('no-such-app', 'unknown.event', { payload: { id: 1 } })).eventId, null, 'no durable twin');
assert.strictEqual((await announce('no-such-app', 'unknown.event', { payload: { id: 1 } })).eventId, null, 'no durable twin');
- // A bare webhook queues nothing (the event is written inside the state change's transaction).
- await sendWebhook('no-such-app', 'vod.ready', { id: 43 });
await outbox.flush();
await outbox.flush();
assert.strictEqual(published.length, 2);
assert.strictEqual(published.length, 2);
assert.strictEqual(published[0].event_type, 'media.vod.ready');
assert.strictEqual(published[0].event_type, 'media.vod.ready');
@@ -68,10 +66,7 @@ const http = require('http');
@@ -68,10 +66,7 @@ const http = require('http');
assert.strictEqual(published[1].event_type, 'media.clip.failed');
assert.strictEqual(published[1].event_type, 'media.clip.failed');
assert.strictEqual(published[1].priority, 'important');
assert.strictEqual(published[1].priority, 'important');
- // Storage alerts: queued once by the storage module, never per app by sendWebhook.
- await sendWebhook('no-such-app', 'storage.alert', { kind: 'disk' });
- await outbox.flush();
- assert.strictEqual(published.length, 2);
+ // Storage alerts are queued once for all consumers.
await inTx(async () => await events.record('storage.alert', null, { kind: 'disk', free_gb: 3 }));
await inTx(async () => await events.record('storage.alert', null, { kind: 'disk', free_gb: 3 }));
await outbox.flush();
await outbox.flush();
assert.strictEqual(published.length, 3);
assert.strictEqual(published.length, 3);
@@ -26,14 +26,12 @@ const http = require('http');
@@ -26,14 +26,12 @@ const http = require('http');
process.env.MEDIA_INVARIANT_SCAN_HOURS = '0';
process.env.MEDIA_INVARIANT_SCAN_HOURS = '0';
- const hooks = [];
const stub = http.createServer((req, res) => {
const stub = http.createServer((req, res) => {
let body = '';
let body = '';
req.on('data', (c) => { body += c; });
req.on('data', (c) => { body += c; });
req.on('end', () => {
req.on('end', () => {
res.setHeader('Content-Type', 'application/json');
res.setHeader('Content-Type', 'application/json');
if (req.url === '/oauth/token') return res.end(JSON.stringify({ access_token: 'tok', token_type: 'Bearer', expires_in: 300 }));
if (req.url === '/oauth/token') return res.end(JSON.stringify({ access_token: 'tok', token_type: 'Bearer', expires_in: 300 }));
- if (req.url === '/hook') { hooks.push(JSON.parse(body)); return res.end('{}'); }
res.statusCode = 404; res.end('{}');
res.statusCode = 404; res.end('{}');
});
});
});
});
@@ -52,7 +50,7 @@ const http = require('http');
@@ -52,7 +50,7 @@ const http = require('http');
const raw = db.getDb();
const raw = db.getDb();
- await db.upsertApp({ app_id: 'live', api_key: 'live-key', webhook_url: `${base}/hook`, webhook_secret: 's' });
+ await db.upsertApp({ app_id: 'live', api_key: 'live-key' });
await db.upsertApp({ app_id: 'games', api_key: 'games-key' });
await db.upsertApp({ app_id: 'games', api_key: 'games-key' });
const events = require('../server/events');
const events = require('../server/events');
const relay = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
const relay = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
@@ -123,7 +121,6 @@ const http = require('http');
@@ -123,7 +121,6 @@ const http = require('http');
await assert.rejects(async () => await queue.cancel(q2.job.id, { by: 'x' }), /outbox insert failed/);
await assert.rejects(async () => await queue.cancel(q2.job.id, { by: 'x' }), /outbox insert failed/);
failOutbox(false);
failOutbox(false);
assert.strictEqual((await queue.get(q2.job.id)).status, 'queued', 'a cancel whose event cannot be written changes nothing');
assert.strictEqual((await queue.get(q2.job.id)).status, 'queued', 'a cancel whose event cannot be written changes nothing');
- assert.ok(hooks.length === 0, 'job events go to Events only, not the app webhook');
console.log('✅ job state changes commit with their media.job.* outbox row, or not at all');
console.log('✅ job state changes commit with their media.job.* outbox row, or not at all');
// ── 3. Idempotency keys ──
// ── 3. Idempotency keys ──
@@ -37,11 +37,6 @@ const http = require('http');
@@ -37,11 +37,6 @@ const http = require('http');
await db.upsertApp({ app_id: 'live', name: 'OpenVibe.Live', api_key: 'live-key' });
await db.upsertApp({ app_id: 'live', name: 'OpenVibe.Live', api_key: 'live-key' });
await db.upsertApp({ app_id: 'tools', name: 'OpenVibe.Tools', api_key: 'tools-key' });
await db.upsertApp({ app_id: 'tools', name: 'OpenVibe.Tools', api_key: 'tools-key' });
- // The `live` app has an outbound webhook (tools has none): the operator report must show the host
- // only and never the URL or the secret.
- const HOOK_URL = 'https://hooks.example.test/ov-private-hook-path';
- const HOOK_SECRET = 'whsec_TESTONLY-MUST-NOT-LEAK';
- await db.run("UPDATE apps SET webhook_url = ?, webhook_secret = ? WHERE app_id = 'live'", [HOOK_URL, HOOK_SECRET]);
for (const a of ['live', 'tools']) await db.ensureRootNamespace(await db.getApp(a));
for (const a of ['live', 'tools']) await db.ensureRootNamespace(await db.getApp(a));
await namespaces.ensure(await db.getApp('live'), 'live.avatars');
await namespaces.ensure(await db.getApp('live'), 'live.avatars');
@@ -339,19 +334,10 @@ const http = require('http');
@@ -339,19 +334,10 @@ const http = require('http');
assert.ok(rep.projection.unprojected && rep.projection.owner_subject.missing >= 1);
assert.ok(rep.projection.unprojected && rep.projection.owner_subject.missing >= 1);
assert.ok(rep.tiering.objects.some(o => o.origin === 'native') && rep.tiering.sweep && 'pending_offload' in rep.tiering);
assert.ok(rep.tiering.objects.some(o => o.origin === 'native') && rep.tiering.sweep && 'pending_offload' in rep.tiering);
assert.ok(rep.namespaces.some(n => n.namespace === 'live.avatars'));
assert.ok(rep.namespaces.some(n => n.namespace === 'live.avatars'));
- // Webhooks section: host only, no URL path and no secret anywhere in the payload; counts are
- // this process's own (none of these calls sends one).
- assert.deepStrictEqual(rep.webhooks.configured, [{ app_id: 'live', host: 'hooks.example.test' }], 'configured webhooks, host only');
- assert.ok(rep.webhooks.sends.since && !Object.keys(rep.webhooks.sends.by_app).length, 'no sends from the explorer itself');
- assert.ok(rep.webhooks.note.includes('OpenVibe.Events') && rep.webhooks.note.includes('reset on restart'));
- assert.ok(!r.text.includes(HOOK_URL) && !r.text.includes('ov-private-hook-path') && !r.text.includes(HOOK_SECRET), 'the report never carries the webhook URL or secret');
r = await call('GET', '/api/v2/me/ops?app=tools', { token: 'admin' });
r = await call('GET', '/api/v2/me/ops?app=tools', { token: 'admin' });
assert.deepStrictEqual([r.json.scope, r.json.jobs.failed.total, r.json.namespaces.map(n => n.namespace)], ['tools', 0, ['tools']]);
assert.deepStrictEqual([r.json.scope, r.json.jobs.failed.total, r.json.namespaces.map(n => n.namespace)], ['tools', 0, ['tools']]);
- assert.deepStrictEqual(r.json.webhooks.configured, [], 'tools has no webhook');
r = await call('GET', '/me/ops', { cookie: 'admin' });
r = await call('GET', '/me/ops', { cookie: 'admin' });
assert.ok(r.status === 200 && r.text.includes('mjob_TESTFAILED') && r.text.includes('action="/me/ops/recompute"') && r.text.includes('noindex'));
assert.ok(r.status === 200 && r.text.includes('mjob_TESTFAILED') && r.text.includes('action="/me/ops/recompute"') && r.text.includes('noindex'));
- assert.ok(r.text.includes('webhooks-h') && r.text.includes('hooks.example.test'), 'the page shows the webhooks section and the configured host');
- assert.ok(!r.text.includes(HOOK_URL) && !r.text.includes('ov-private-hook-path') && !r.text.includes(HOOK_SECRET), 'the page never shows the webhook URL path or secret');
console.log('✅ operator views: staff.site.view only (not users, not global mods); failed jobs, missing media, projection, tiering, namespaces');
console.log('✅ operator views: staff.site.view only (not users, not global mods); failed jobs, missing media, projection, tiering, namespaces');
// ── Recompute: staff.site.configure, same-origin ──
// ── Recompute: staff.site.configure, same-origin ──
@@ -378,53 +364,12 @@ const http = require('http');
@@ -378,53 +364,12 @@ const http = require('http');
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.json.scope, 'live');
assert.strictEqual(r.json.scope, 'live');
assert.ok(r.json.namespaces.every(n => n.app_id === 'live'));
assert.ok(r.json.namespaces.every(n => n.app_id === 'live'));
- assert.deepStrictEqual(r.json.webhooks.configured, [{ app_id: 'live', host: 'hooks.example.test' }], 'the app-key twin shows its own webhook');
- assert.ok(r.json.webhooks.sends.by_app === undefined || Object.keys(r.json.webhooks.sends.by_app).every(a => a === 'live'), 'sends narrowed to the app');
assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'live-key', headers: { 'x-ov-user-id': '5' } })).status, 403, 'never acting for a user');
assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'live-key', headers: { 'x-ov-user-id': '5' } })).status, 403, 'never acting for a user');
assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'ana' })).status, 401);
assert.strictEqual((await call('GET', '/api/v1/live/admin/storage/ops', { token: 'ana' })).status, 401);
r = await call('POST', '/api/v1/tools/admin/storage/ops/recompute', { token: 'tools-key' });
r = await call('POST', '/api/v1/tools/admin/storage/ops/recompute', { token: 'tools-key' });
assert.deepStrictEqual([r.status, r.json.scope, r.json.namespaces], [200, 'tools', 1]);
assert.deepStrictEqual([r.status, r.json.scope, r.json.namespaces], [200, 'tools', 1]);
console.log('✅ /api/v1/:app/admin/storage/ops: the same report for the app key, scoped to its tenant');
console.log('✅ /api/v1/:app/admin/storage/ops: the same report for the app key, scoped to its tenant');
- // ── Webhook tallies: one success, one failure ──
- {
- const hooks = require('../server/webhooks');
- hooks._resetStats();
- await db.run("UPDATE apps SET webhook_url = 'https://hooks-down.example.test/never' WHERE app_id = 'tools'");
- const realFetch = global.fetch;
- hooks._setRetryBaseMs(0); // retries must not slow the suite (never a global setTimeout patch: the pg pool's timers use it)
- let down = true;
- global.fetch = async (url, opts) => {
- const u = String(url);
- if (!u.includes('hooks.example.test') && !u.includes('hooks-down.example.test')) return realFetch(url, opts);
- const ok = u.includes('hooks.example.test') || !down;
- return { ok, status: ok ? 200 : 500 };
- };
- try {
- assert.strictEqual(await hooks.sendWebhook('live', 'vod.ready', { id: 1 }), true);
- assert.strictEqual(await hooks.sendWebhook('tools', 'clip.ready', { id: 2 }), false);
- // The same app answers 200 on a later send: the recorded error must be cleared.
- const ghost = { app_id: 'ghost', webhook_url: 'https://hooks-down.example.test/x', webhook_secret: 's' };
- assert.strictEqual(await hooks.sendWebhook(ghost, 'clip.ready', { id: 3 }), false);
- assert.strictEqual(hooks.stats().by_app.ghost.last_error, 'HTTP 500');
- down = false;
- assert.strictEqual(await hooks.sendWebhook(ghost, 'clip.ready', { id: 4 }), true);
- assert.strictEqual(hooks.stats().by_app.ghost.last_error, null, 'a later success clears the previous error');
- } finally {
- global.fetch = realFetch; hooks._setRetryBaseMs(null);
- await db.run("UPDATE apps SET webhook_url = NULL, webhook_secret = NULL WHERE app_id = 'tools'");
- }
- const st = hooks.stats();
- assert.deepStrictEqual([st.by_app.live.sent, st.by_app.live.failed, st.by_app.live.last_event], [1, 0, 'vod.ready']);
- assert.ok(st.by_app.live.last_ok_at && st.by_app.live.last_attempt_at && !st.by_app.live.last_error);
- assert.deepStrictEqual([st.by_app.tools.sent, st.by_app.tools.failed, st.by_app.tools.last_error, st.by_app.tools.last_event], [0, 1, 'HTTP 500', 'clip.ready']);
- st.by_app.live.sent = 999;
- assert.strictEqual(hooks.stats().by_app.live.sent, 1, 'stats() is a fresh copy');
- hooks._resetStats();
- assert.deepStrictEqual(hooks.stats().by_app, {});
- console.log('✅ webhook tallies: a 200 counts sent, a 500 after retries counts failed with its error; stats() copies');
- }
-
// ── Routes under /api/v2/me are the explorer's only ──
// ── Routes under /api/v2/me are the explorer's only ──
r = await call('GET', '/api/v2/me/namespaces', { token: 'ana' });
r = await call('GET', '/api/v2/me/namespaces', { token: 'ana' });
assert.deepStrictEqual([r.status, r.json.code], [404, 'media.not_found']);
assert.deepStrictEqual([r.status, r.json.code], [404, 'media.not_found']);
@@ -88,7 +88,7 @@ const http = require('http');
@@ -88,7 +88,7 @@ const http = require('http');
console.log('✅ a rolled-back change leaves neither an event nor a staged row');
console.log('✅ a rolled-back change leaves neither an event nor a staged row');
// ── 3. A row deleted inside announce(): media.object.deleted in the same transaction ──
// ── 3. A row deleted inside announce(): media.object.deleted in the same transaction ──
- const { announce } = require('../server/webhooks');
+ const { announce } = require('../server/announce');
await announce('live', 'vod.failed', { change: async () => await db.run('DELETE FROM vods WHERE id = ?', [vodId]), payload: { id: vodId } });
await announce('live', 'vod.failed', { change: async () => await db.run('DELETE FROM vods WHERE id = ?', [vodId]), payload: { id: vodId } });
evs = await objectEvents();
evs = await objectEvents();
assert.strictEqual(evs.length, 2);
assert.strictEqual(evs.length, 2);
@@ -1,16 +1,14 @@
@@ -1,16 +1,14 @@
'use strict';
'use strict';
// Roadmap Wave 3 (audit item 7): Media's outbox row is written in the SAME PostgreSQL transaction as
// Roadmap Wave 3 (audit item 7): Media's outbox row is written in the SAME PostgreSQL transaction as
// the state change it describes. An outcome is never lost (the change committed, the event did
// the state change it describes. An outcome is never lost (the change committed, the event did
-// not) and never a phantom (the event exists, the change rolled back). The webhook for the same
-// outcome follows the commit and carries the event's id, so a consumer reading both paths
-// handles it once. Covers webhooks.announce() and the real call sites for clip.failed
+// not) and never a phantom (the event exists, the change rolled back). Covers announce() and
+// the real call sites for clip.failed
// (clip-jobs) and vod.failed (finalize), with an outbox insert that fails and a change that fails.
// (clip-jobs) and vod.failed (finalize), with an outbox insert that fails and a change that fails.
const assert = require('assert');
const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const os = require('os');
const os = require('os');
const path = require('path');
const path = require('path');
const http = require('http');
const http = require('http');
-const crypto = require('crypto');
(async () => {
(async () => {
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-outbox-tx-'));
const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-outbox-tx-'));
@@ -22,7 +20,6 @@ const crypto = require('crypto');
@@ -22,7 +20,6 @@ const crypto = require('crypto');
process.env.PASTES_PATH = dir('pastes');
process.env.PASTES_PATH = dir('pastes');
process.env.OBJECTS_PATH = dir('objects');
process.env.OBJECTS_PATH = dir('objects');
- const hooks = [];
const stub = http.createServer((req, res) => {
const stub = http.createServer((req, res) => {
let body = '';
let body = '';
req.on('data', (c) => { body += c; });
req.on('data', (c) => { body += c; });
@@ -35,29 +32,19 @@ const crypto = require('crypto');
@@ -35,29 +32,19 @@ const crypto = require('crypto');
const results = list.map((e, i) => ({ event_id: e.event_id, seq: i + 1, duplicate: false }));
const results = list.map((e, i) => ({ event_id: e.event_id, seq: i + 1, duplicate: false }));
return res.end(JSON.stringify(parsed.events ? { results } : results[0]));
return res.end(JSON.stringify(parsed.events ? { results } : results[0]));
}
}
- if (req.url === '/hook') {
- hooks.push({ body, signature: req.headers['x-ovmedia-signature'] });
- return res.end('{"ok":true}');
- }
res.statusCode = 404; res.end('{}');
res.statusCode = 404; res.end('{}');
});
});
});
});
- const waitFor = async (fn, ms = 3000) => {
- const end = Date.now() + ms;
- while (Date.now() < end) { if (fn()) return true; await new Promise((r) => setTimeout(r, 20)); }
- return false;
- };
-
(async () => {
(async () => {
await new Promise((r) => stub.listen(0, '127.0.0.1', r));
await new Promise((r) => stub.listen(0, '127.0.0.1', r));
const base = `http://127.0.0.1:${stub.address().port}`;
const base = `http://127.0.0.1:${stub.address().port}`;
const db = require('../server/db/database');
const db = require('../server/db/database');
const raw = db.getDb();
const raw = db.getDb();
- await db.upsertApp({ app_id: 'live', api_key: 'k', webhook_url: `${base}/hook`, webhook_secret: 'hook-secret' });
+ await db.upsertApp({ app_id: 'live', api_key: 'k' });
const events = require('../server/events');
const events = require('../server/events');
- const { announce } = require('../server/webhooks');
+ const { announce } = require('../server/announce');
// A long relay interval: the test reads the outbox table, the relay is not under test here.
// A long relay interval: the test reads the outbox table, the relay is not under test here.
const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
const outbox = events.init({ eventsUrl: base, clientSecret: 's', networkUrl: base, intervalMs: 60000 });
assert.ok(outbox);
assert.ok(outbox);
@@ -72,7 +59,7 @@ const crypto = require('crypto');
@@ -72,7 +59,7 @@ const crypto = require('crypto');
const { clipPublic } = require('../server/vod/clips-routes');
const { clipPublic } = require('../server/vod/clips-routes');
const clipStatus = async () => (await raw.prepare('SELECT status FROM clips WHERE id = ?').get(clipId)).status;
const clipStatus = async () => (await raw.prepare('SELECT status FROM clips WHERE id = ?').get(clipId)).status;
- // 1. The change and its event commit together; the webhook carries the same event_id.
+ // 1. The change and its event commit together; announce returns the event_id.
const out = await announce('live', 'clip.ready', {
const out = await announce('live', 'clip.ready', {
change: async () => await db.run("UPDATE clips SET status = 'ready' WHERE id = ?", [clipId]),
change: async () => await db.run("UPDATE clips SET status = 'ready' WHERE id = ?", [clipId]),
payload: async () => await clipPublic(await db.getClipById(clipId)),
payload: async () => await clipPublic(await db.getClipById(clipId)),
@@ -84,14 +71,7 @@ const crypto = require('crypto');
@@ -84,14 +71,7 @@ const crypto = require('crypto');
assert.deepStrictEqual(rows[0].env.subject, { type: 'clip', id: String(clipId) });
assert.deepStrictEqual(rows[0].env.subject, { type: 'clip', id: String(clipId) });
assert.strictEqual(rows[0].env.payload.status, 'ready', 'payload is the row as committed');
assert.strictEqual(rows[0].env.payload.status, 'ready', 'payload is the row as committed');
assert.strictEqual(out.eventId, rows[0].id);
assert.strictEqual(out.eventId, rows[0].id);
- assert.ok(await waitFor(() => hooks.length === 1), 'webhook sent after the commit');
- const hook = JSON.parse(hooks[0].body);
- assert.strictEqual(hook.event, 'clip.ready');
- assert.strictEqual(hook.event_id, rows[0].id, 'the webhook names the durable event');
- const sig = 'sha256=' + crypto.createHmac('sha256', 'hook-secret').update(hooks[0].body).digest('hex');
- assert.strictEqual(hooks[0].signature, sig, 'the signature covers event_id');
-
- // 2. No phantom: a change that fails leaves no event and sends no webhook.
+ // 2. No phantom: a change that fails leaves no event.
await assert.rejects(async () => await announce('live', 'clip.failed', {
await assert.rejects(async () => await announce('live', 'clip.failed', {
change: async () => { await db.run("UPDATE clips SET status = 'failed' WHERE id = ?", [clipId]); throw new Error('disk full'); },
change: async () => { await db.run("UPDATE clips SET status = 'failed' WHERE id = ?", [clipId]); throw new Error('disk full'); },
payload: async () => await clipPublic(await db.getClipById(clipId)),
payload: async () => await clipPublic(await db.getClipById(clipId)),
@@ -150,11 +130,10 @@ const crypto = require('crypto');
@@ -150,11 +130,10 @@ const crypto = require('crypto');
assert.strictEqual(rows[2].env.event_type, 'media.vod.failed');
assert.strictEqual(rows[2].env.event_type, 'media.vod.failed');
assert.deepStrictEqual(rows[2].env.subject, { type: 'vod', id: '20' });
assert.deepStrictEqual(rows[2].env.subject, { type: 'vod', id: '20' });
assert.strictEqual(rows[2].env.priority, 'important');
assert.strictEqual(rows[2].env.priority, 'important');
- assert.ok(await waitFor(() => hooks.some((h) => JSON.parse(h.body).event_id === rows[2].id)), 'vod.failed webhook names the event');
events._reset();
events._reset();
stub.close();
stub.close();
- console.log('✅ media outbox: state change and event commit together (no lost or phantom outcome), webhook carries event_id');
+ console.log('✅ media outbox: state change and event commit together (no lost or phantom outcome)');
process.exit(0);
process.exit(0);
})().catch((err) => { console.error(err); process.exit(1); });
})().catch((err) => { console.error(err); process.exit(1); });
})().catch((err) => { console.error(err); process.exit(1); });
})().catch((err) => { console.error(err); process.exit(1); });
@@ -6,9 +6,9 @@
@@ -6,9 +6,9 @@
// - POST /api/v1/:app/vods/:id/ingest/rtmp makes ffmpeg pull an app-supplied rtmp_url, which must name an
// - POST /api/v1/:app/vods/:id/ingest/rtmp makes ffmpeg pull an app-supplied rtmp_url, which must name an
// allow-listed host:port (MEDIA_RTMP_PULL_ALLOW; test/rtmp-allowlist.test.js has the URL grammar).
// allow-listed host:port (MEDIA_RTMP_PULL_ALLOW; test/rtmp-allowlist.test.js has the URL grammar).
// Every other outbound request goes to an operator-configured address (Network, Events, Live's internal
// Every other outbound request goes to an operator-configured address (Network, Events, Live's internal
-// URL, B2/R2, an app's operator-set webhook_url row, IndexNow's fixed api.indexnow.org from
+// URL, B2/R2, IndexNow's fixed api.indexnow.org from
// INDEXNOW_KEY). The inventory below fails when a new outbound call appears in server/ until it is
// INDEXNOW_KEY). The inventory below fails when a new outbound call appears in server/ until it is
-// classified here, and no route lets a tenant set a webhook URL.
+// classified here. No server path sends to a webhook URL.
// No test here needs the network: literals and internal names are refused before DNS, and DNS answers
// No test here needs the network: literals and internal names are refused before DNS, and DNS answers
// and the socket are stubbed.
// and the socket are stubbed.
const assert = require('assert');
const assert = require('assert');
@@ -31,7 +31,6 @@ const https = require('https');
@@ -31,7 +31,6 @@ const https = require('https');
'server/avatars/ingest.js': 'a person\'s picture URL: safeFetchImage, tested below',
'server/avatars/ingest.js': 'a person\'s picture URL: safeFetchImage, tested below',
'server/account-data.js': 'Network internal API (OV_NETWORK_INTERNAL_URL), fixed paths: export parts and deletion confirmations',
'server/account-data.js': 'Network internal API (OV_NETWORK_INTERNAL_URL), fixed paths: export parts and deletion confirmations',
'server/user-auth.js': 'Network OAuth (OV_NETWORK_URL / OV_NETWORK_INTERNAL_URL); the signing keys come from openvibe-sdk/auth',
'server/user-auth.js': 'Network OAuth (OV_NETWORK_URL / OV_NETWORK_INTERNAL_URL); the signing keys come from openvibe-sdk/auth',
- 'server/webhooks.js': 'an app\'s webhook_url, an operator-set row (no seeding since C-71)',
'server/thumbnails/live-frame-service.js': 'the app\'s internal URL (APP_INTERNAL_URLS), fixed path',
'server/thumbnails/live-frame-service.js': 'the app\'s internal URL (APP_INTERNAL_URLS), fixed path',
'server/public/dev-data.js': 'the app\'s internal URL (APP_INTERNAL_URLS), ids and encoded names in the path',
'server/public/dev-data.js': 'the app\'s internal URL (APP_INTERNAL_URLS), ids and encoded names in the path',
'server/jobs/pack.js': 'a presigned GET on B2/R2 (operator-configured buckets) for a segment\'s durable copy',
'server/jobs/pack.js': 'a presigned GET on B2/R2 (operator-configured buckets) for a segment\'s durable copy',
@@ -52,8 +51,8 @@ const https = require('https');
@@ -52,8 +51,8 @@ const https = require('https');
walk(path.join(ROOT, 'server'));
walk(path.join(ROOT, 'server'));
for (const f of found) assert.ok(OUTBOUND[f], `${f} makes an outbound request: classify it in test/security-ssrf.test.js (a user-chosen address must go through a guard)`);
for (const f of found) assert.ok(OUTBOUND[f], `${f} makes an outbound request: classify it in test/security-ssrf.test.js (a user-chosen address must go through a guard)`);
for (const f of Object.keys(OUTBOUND)) assert.ok(found.has(f), `${f} no longer makes an outbound request: drop it from the inventory`);
for (const f of Object.keys(OUTBOUND)) assert.ok(found.has(f), `${f} no longer makes an outbound request: drop it from the inventory`);
- // Webhook targets are operator configuration: the only SQL that writes an app's webhook_url is
- // db.upsertApp, which no server code calls since app seeding went (C-71): tests and the smoke script only.
+ // The schema and db.upsertApp retain the legacy columns, but no server code uses them
+ // as an outbound destination.
const sqlWriters = [], upserters = [];
const sqlWriters = [], upserters = [];
const scan = (d) => {
const scan = (d) => {
for (const e of fs.readdirSync(d, { withFileTypes: true })) {
for (const e of fs.readdirSync(d, { withFileTypes: true })) {
@@ -67,14 +66,24 @@ const https = require('https');
@@ -67,14 +66,24 @@ const https = require('https');
};
};
scan(path.join(ROOT, 'server'));
scan(path.join(ROOT, 'server'));
assert.deepStrictEqual(sqlWriters, ['server/db/database.js'], 'only db.upsertApp writes an app\'s webhook_url');
assert.deepStrictEqual(sqlWriters, ['server/db/database.js'], 'only db.upsertApp writes an app\'s webhook_url');
- assert.deepStrictEqual(upserters, [], 'and no server code calls it: app seeding is gone (C-71), so a webhook URL is an operator-set row');
+ assert.deepStrictEqual(upserters, [], 'no server code calls db.upsertApp');
+ const webhookReaders = [];
+ const scanReaders = (d) => {
+ for (const e of fs.readdirSync(d, { withFileTypes: true })) {
+ const p = path.join(d, e.name);
+ if (e.isDirectory()) { scanReaders(p); continue; }
+ if (p.endsWith('.js') && !p.endsWith(path.join('db', 'database.js')) && /\bwebhook_(url|secret)\b/.test(fs.readFileSync(p, 'utf8'))) webhookReaders.push(path.relative(ROOT, p));
+ }
+ };
+ scanReaders(path.join(ROOT, 'server'));
+ assert.deepStrictEqual(webhookReaders, [], 'no server code sends to or reads a webhook URL or secret');
// IndexNow (openvibe-shared/indexnow, wired in server/index.js): the only outbound request made
// IndexNow (openvibe-shared/indexnow, wired in server/index.js): the only outbound request made
// from a dependency. Its endpoint is the fixed api.indexnow.org and host/key are operator config
// from a dependency. Its endpoint is the fixed api.indexnow.org and host/key are operator config
// (MEDIA_PUBLIC_URL / INDEXNOW_KEY) — never a tenant or a user-supplied URL, and no route reaches it.
// (MEDIA_PUBLIC_URL / INDEXNOW_KEY) — never a tenant or a user-supplied URL, and no route reaches it.
const idxSrc = fs.readFileSync(path.join(ROOT, 'server', 'index.js'), 'utf8');
const idxSrc = fs.readFileSync(path.join(ROOT, 'server', 'index.js'), 'utf8');
assert.ok(idxSrc.includes('createIndexNow(') && idxSrc.includes('host: config.publicUrl') && idxSrc.includes('key: config.indexnow.key'),
assert.ok(idxSrc.includes('createIndexNow(') && idxSrc.includes('host: config.publicUrl') && idxSrc.includes('key: config.indexnow.key'),
'IndexNow is wired from operator config (host/key), the one outbound dependency');
'IndexNow is wired from operator config (host/key), the one outbound dependency');
- console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); webhook URLs are operator-set; IndexNow posts to the fixed api.indexnow.org`);
+ console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); no webhook sends; IndexNow posts to the fixed api.indexnow.org`);
}
}
const ingest = require('../server/avatars/ingest');
const ingest = require('../server/avatars/ingest');