openvibers/openvibe.mediahub AGENTS.md

AGENTS.md

main 12.1 KB · 165 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.MediaHub
2
3## Project Overview
4
5OpenVibe.MediaHub is the network's file service. One process serves three hosts, chosen by Host header:
6
7- **openvibe.download** — a private drive: upload, folders, share links, resumable uploads, sync keys, agent grants.
8- **openvibe.video** — OpenVibe.Video: uploads and browser recordings, AI captions/chapters/cuts, watch pages,
9 playlists, and OpenVibe.Live's stream recordings and clips.
10- **openvibe.pics** — OpenVibe.Pics: picture hosting with resize/convert, alt text, albums, the public gallery and a
11 takedown blocklist.
12
13- **Port:** 4990 · **Service id:** `media-hub` · **Env prefix:** `MEDIAHUB`
14- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development/tests), Valkey 9.
15- **Read first:** [README.md](README.md), [STATUS.json](STATUS.json), [.env.example](.env.example).
16
17**Network context:** subjects, usernames and JWKS are OpenVibe.Network (OAuth client `media-hub`). The **bytes** are
18OpenVibe.Media objects (`med_…`); MediaHub stores ids and metadata and asks Media to create/read/delete. Live's public
19JSON API supplies recordings/clips (`MEDIAHUB_LIVE_URL`). Comments on videos and pictures are OpenVibe.Community
20threads through `server/comments/`. Sharing is `openvibe-sdk/share`. Events go through `event_outbox` to
21OpenVibe.Events; account export/deletion arrives at `/internal/events`; public ready videos go to OpenVibe.Search.
22Nothing here is identity, media bytes, money or event delivery.
23
24## Commands
25
26```bash
27npm ci # install (contracts/sdk/shared pinned by tag)
28cp .env.example .env # development names/values
29npm run dev # http://127.0.0.1:4990, PGlite under data/
30npm start # node server/index.js
31npm test # every test/*.test.js on PGlite with stand-ins for Network/Media
32npm run test:pg # the same through PostgreSQL + PgBouncer
33npm test -- video- # only test files whose name contains the word
34node --check <file.js> # syntax check (no linter configured)
35```
36
37No build step: pages are server-rendered; browser code is plain JS under [public/js/](public/js/) plus the vendored
38HLS player [public/vendor/hls.light.min.js](public/vendor/hls.light.min.js).
39
40## Architecture at a Glance
41
42- **Entry:** [server/index.js](server/index.js) — listener, telemetry flush/retention, Events subscriptions; graceful
43 stop in `createLifecycle()`. `MEDIAHUB_DRILL` keeps boot-time work off.
44- **App factory:** [server/app.js](server/app.js) — helmet/CSP, SSO, **host routing** (`videoHost`, `picsHost`; every
45 other host is the drive), `/auth`, `/internal/events`, `/shared` + static, `/api/v1`, the agent layer, pages, 404 and
46 error handler. Tests build `createApp({ store, fetchImpl, ... })`.
47- **Config:** [server/config.js](server/config.js) — `load(env)` is pure; production `/etc/openvibe/media-hub.env`.
48- **Drive:** [server/drive/](server/drive/) — `service.js`, `store.js`, `upload.js` (chunked/resumable),
49 `syncKeys.js`, `drops.js`, `ui.js`; API [server/http/api.js](server/http/api.js). Bytes: [server/media/](server/media/).
50- **Video:** [server/video/](server/video/) — `pages.js`, `uploads.js`, `playlists.js`, `ai-edit.js` (captions,
51 chapters, cut/trim), `cut.js`, `downloads.js`, `watch-ui.js`, `studio.js`, `search-index.js`, `comments-client.js`;
52 watch routes and `/u/` owner pages.
53- **Pics:** [server/pics/](server/pics/) — `service.js`, `pages.js`, `api.js`, `store.js`, `describe.js` (AI alt text),
54 `gallery.js`, `policy.js`, `variants.js`, `images.js`, `resumable.js`; [server/comments/](server/comments/)
55 (`form.js`, `inline.js`) for Community threads.
56- **Radio:** [server/radio/routes.js](server/radio/routes.js) — the shared radio tracks/listener.
57- **Auth:** [server/auth/sso.js](server/auth/sso.js), [server/auth/keys.js](server/auth/keys.js),
58 [server/http/principal.js](server/http/principal.js) (person, sync key, app/agent/service capability).
59- **Frontend:** server-rendered through [server/render/](server/render/) and `openvibe-shared/shell|frame`, with
60 assets [public/css/](public/css/) (`app`, `drive`, `video`, `pics`, `comments`, `labels`, `radio`) and
61 [public/js/](public/js/) (`upload`, `video`, `video-upload`, `video-record`, `pics`, `pics-gallery`, `drop`,
62 `comments`, `my-videos`, `video-home`). Pages work without JavaScript; the one drive script is a progress meter.
63- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/) — drive (`mh_files`, `mh_folders`,
64 `mh_shares`, `mh_reports`, `mh_usage`, `mh_uploads`, `mh_sync_keys`, `mh_drops`, `mh_agent_grants`,
65 `mh_share_defaults`, `mh_drop_reports`, `mh_drop_blocklist`), video (`video_uploads`, `video_playlists`,
66 `video_playlist_items`, `video_cuts`, `video_likes`, `video_search_state`, `video_upload_ai`,
67 `video_upload_reports`, `video_usage`), pics (`pics_images`, `pics_albums`, `pics_album_images`, `pics_reports`,
68 `pics_blocklist`, `pics_usage`, `pics_views`, `pics_likes`, `pics_uploads`, `pics_upload_chunks`), radio,
69 sharing (`shares`, `share_*`), `event_outbox`, `account_data_events`, `telemetry_samples`.
70- **Deploy:** [deploy/systemd/openvibe-media-hub.service](deploy/systemd/openvibe-media-hub.service)
71 (`/etc/openvibe/media-hub.env`, `/opt/openvibe.download`) and
72 [deploy/nginx/openvibe.download.conf](deploy/nginx/openvibe.download.conf) — one vhost for openvibe.download and
73 openvibe.video (both proxy to 127.0.0.1:4990), openvibe.pics parked with a 302 to `https://openvibe.network/`;
74 uploads and downloads stream unbuffered. `sudo ovhost deploy media-hub`; database `ov_media_hub`.
75
76## Ownership and Boundaries
77
78MediaHub owns the drive's metadata, share links, sync keys, agent grants, upload sessions, reports and usage; the
79video upload/playlist/cut metadata and search state; the pics metadata, albums, gallery, labels and blocklist; the
80radio tracks; its share rows and outbox. It does **not** own the bytes (Media), identity (Network), Live's channels or
81recordings (Live), comments (Community), event delivery (Events), or the search index (Search). CSAM hash matching
82against NCMEC/PhotoDNA is not implemented: Pics matches only its own takedown blocklist, and child-safety reports hide
83at once. Never claim otherwise in UI or docs.
84
85## Auth and Capabilities
86
87The Contracts manifest lists ten capabilities: `video.vod.read`, `video.playlist.read`, `video.playlist.manage`,
88`pics.image.read`, `pics.image.upload`, `pics.album.read`, `pics.album.manage`, `download.file.read`,
89`download.file.upload`, `download.share.create`. The two playlist capabilities are registered in
90[server/http/principal.js](server/http/principal.js). `download.file.read` is agent-only and requires that agent's
91active per-file grant — a capability never lists the drive or replaces the grant. Every other drive write is a
92signed-in person or the owner's scoped sync key.
93
94- Cookie-authenticated writes must be same-origin. Uploads and downloads take the same per-caller budgets as the API
95 route they stand for, so a plain form is never a way round a limit.
96- Outbound: `identity.subject.resolve` (Network), Media `media.object.upload|read` on namespace `media-hub`,
97 `events.subscription.manage` + `events.event.publish` (`mediahub.video.published`, `mediahub.picture.uploaded`),
98 `search.document.write`.
99- Errors are RFC 9457 problem+json with stable codes; never leak Media's internals or a person's email/subject.
100
101## API (drive, `/api/v1`)
102
103`GET /files`, `POST /files`, `GET|DELETE /files/:id`, `GET /folders`, `POST /folders`,
104`POST /shares`, `DELETE /shares/:slug`, `GET|POST /agent-grants`, `DELETE /agent-grants/:id`,
105`GET /agent/files/:id/content` (agent + grant, byte ranges), `GET|PUT|POST|DELETE /uploads/:id[/parts/:n|/complete]`,
106`GET /usage`. Video and Pics have their own APIs on their hosts (`/api/v1/videos`, playlists, pics/images,
107albums, gallery, comments), plus public reads, `/llms.txt`, sitemaps and IndexNow for openvibe.video. Machine
108endpoints `/api/health`, `/api/ready`, `/release.json`, `/metrics` are loopback only.
109
110## Conventions
111
112- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter. Async handlers through the router so a
113 rejection reaches the error handler.
114- New schema = a new `migrations/NNNN_name.sql`; never edit one that has run. Prefix tables by product (`mh_`, `pics_`,
115 `video_`) and keep the 40+ existing numbers' order.
116- Never render a file's bytes into a page; escape every owner-supplied name with the `html` template.
117- CSP: add every external origin to the `helmet` directives in [server/app.js](server/app.js) (Media images, Live HLS,
118 Community, Events, Search). Do not widen `frameAncestors`/`objectSrc`. The HLS player is vendored on purpose.
119- **No Font Awesome** — use `openvibe-shared/glyphs` and `openvibe-shared/brands`; the no-FA guard
120 checks every page and stylesheet, and a new `fa-` name must be one the glyphs/brands draw.
121- **No looks filter** — `openvibe-sdk/no-looks` is gone; AI text may describe people. Only sex acts/porn stay gated
122 ([test/no-looks-guard.test.js](test/no-looks-guard.test.js)).
123- Phone first: watch, drive, pics and gallery pages must fit 390 px; the video mini-player and the watch page are
124 first-class phone surfaces.
125- "Calm reduces, only Off removes": audience/share preferences de-emphasize, they do not silently delete. Only the
126 explicit Off/delete path removes.
127- Empty states use the shared kit (`ovk-empty`), not bespoke markup ([test/empty-states.test.js](test/empty-states.test.js)).
128
129## Key Pitfalls
130
131- **Host header is the product switch.** openvibe.video and openvibe.pics must never serve drive pages/API and vice
132 versa; add new routes to the right host router in [server/app.js](server/app.js).
133- **Bytes are Media's.** Never write a media file to local disk as the source of truth; Media controls retention, and
134 a soft-deleted drive row stays until its object is gone.
135- **Streaming:** uploads and downloads must not be buffered by nginx or by the app; the vhost is deliberately
136 `proxy_request_buffering off`/`proxy_buffering off`. Range requests must keep working for the player and downloads.
137- **Share links** have expiry, allow-lists and a report high-water mark; revocation is immediate. Agent grants expire
138 after 1–720 h and cover one file — do not turn one into a listing right.
139- **Pics policy:** signed-out uploads exist only with `MEDIAHUB_PICS_ANONYMOUS`; the anonymous delete key is stored as
140 its sha256; the blocklist matches sha256 and difference hashes, not perceptual guesses. Reports hide first, review
141 later.
142- **Video imports** from Live are public-API reads (signed out); a recording in progress is handled by Media, and a
143 stale/private upload is tombstoned from Search. Never call Media's private routes with a person's token.
144- **Drill mode** (`MEDIAHUB_DRILL`) keeps the backfills, search sync, indexnow and timers off; add new background work
145 there.
146- **`event_outbox`** needs its rejected partial index ([migrations/0025_outbox_rejected_index.sql](migrations/0025_outbox_rejected_index.sql)).
147- Nginx parks openvibe.pics with a 302 today; do not "fix" that without the Pics deployment decision.
148
149## Testing
150
151`npm test` runs each file in its own process on PGlite with Network/Media stand-ins. The suite covers drive
152upload/share/download/range, sync keys, agent grants, drops, video upload/record/playlists/AI/cut/trim/search/comments,
153pics upload/gallery/album/labels/policy/sharing/rotate, radio, sharing, account data, host routing, no Font Awesome,
154no-looks, empty states, phone/contrast, secret-leak/redirect and size budgets. Run the narrowest file first, then the
155suite; `npm run test:pg` for the PostgreSQL path. Always `node --check` changed files.
156
157## Documentation
158
159- [README.md](README.md) — three products in one process, owns/does-not-own, capabilities, drive/video/pics pages,
160 sync keys, API, playlists, account data, configuration, acceptance, deploy, security notes.
161- [docs/screenshots/](docs/screenshots/) — recorded UI states.
162- [STATUS.json](STATUS.json) — stage, deployed flag, features and not-yet list.
163- [.env.example](.env.example) — variable names only; never commit values.
164- [CHANGELOG.md](CHANGELOG.md), [SECURITY.md](SECURITY.md).
165