openvibers/openvibe.report AGENTS.md

AGENTS.md

main 11.0 KB · 150 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.Report
2
3## Project Overview
4
5OpenVibe.Report is the network's analytics room and its analytics authority in shadow: the open numbers every site
6publishes, Live's Rundown and after-show reports quoted in one place, Bench's AI boards rendered here, each person's
7own private stats with per-section publishing, session reports with sharing, the AI usage meter and the AI Exchange
8aggregates. `REPORT_SOURCE=report` switches the pages from Network's numbers to Report's own tables when the cutover is
9approved.
10
11- **Domain:** `openvibe.report` · **Port:** 5080 · **Service id:** `report` · **Env prefix:** `REPORT_`/`OV_`
12- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development), `openvibe-shared`
13 for the frame; jobs run in-process.
14- **Read first:** [README.md](README.md), [STATUS.json](STATUS.json), [.env.example](.env.example),
15 [docs/report.metric.v1.json](docs/report.metric.v1.json).
16
17**Network context:** Network supplies the open cross-site stats, the sign-in/JWKS and the wallet history; Live supplies
18the Rundown and after-show reports; Bench supplies the AI rows (Report never derives a score, cost or time); Billing
19supplies revenue (`billing.stats.read`, `billing.balance.read`) and the AI Exchange settlements; AI/Exchange supplies
20the catalogue and offers; Design's health scan is quoted; Shared supplies the Frame. Report holds no money ledger and
21computes no price.
22
23## Commands
24
25```bash
26npm ci # install (contracts/sdk/shared pinned by tag)
27npm run dev # http://localhost:5080, PGlite under data/pglite
28npm start # node server/index.js
29npm test # every test/*.test.js, stubbed fetch, no network or running site
30npm run test:pg # the same through PostgreSQL + PgBouncer
31npm test -- shadow # only test files whose name contains the word
32node --check <file.js> # syntax check (no linter configured)
33```
34
35No build step: server-rendered pages; assets under [public/css/](public/css/) (`app`, `ai-comparison`, `arena`) and
36[public/js/app.js](public/js/app.js). Pages work without JavaScript.
37
38## Architecture at a Glance
39
40- **Entry:** [server/index.js](server/index.js) — listener and the analytics/AI-usage jobs; graceful stop in
41 `createLifecycle()`. `REPORT_JOBS=0` turns the jobs off, `REPORT_JOBS_ON_BOOT=1` runs them once after boot.
42- **App factory:** [server/app.js](server/app.js) — CSP, `/auth` (Network SSO), optional auth, `/internal/events`,
43 `/shared`+static, pages, `/api/v1`, discovery, 404/error handler. Tests pass a stub `fetchImpl`.
44- **Config:** [server/config.js](server/config.js) — `load(env)` is pure; production `/etc/openvibe/report.env`.
45- **Sources:** [server/data/](server/data/) — `network.js`, `live.js`, `bench.js`, `billing.js`, `ai-market.js`,
46 `ai-catalogue.js`, `ai-join.js`, `cache.js`, `source.js`, `design-health.js`, `network-json.js`.
47- **Analytics:** [server/analytics/](server/analytics/) — `ingest.js` (idempotent), `query.js`, `shadow.js`,
48 `ai-usage.js`, `ai-exchange.js`, `index.js`; jobs [server/jobs/](server/jobs/) (`analytics.js`, `ai-usage.js`).
49- **Pages/views:** [server/http/pages.js](server/http/pages.js), [server/http/arena.js](server/http/arena.js),
50 [server/http/discovery.js](server/http/discovery.js), [server/http/session-reports.js](server/http/session-reports.js),
51 [server/http/internal.js](server/http/internal.js); [server/views/](server/views/) (`index.js`, `short.js`,
52 `ai-comparison.js`, `design.js`). Search documents: [server/search-index.js](server/search-index.js).
53- **People's data:** [server/me-stats.js](server/me-stats.js) (private snapshot, per-section publish switches),
54 [server/session-reports.js](server/session-reports.js), [server/sharing.js](server/sharing.js),
55 [server/popularity.js](server/popularity.js).
56- **Auth:** [server/auth.js](server/auth.js) — service tokens for operator routes (`report.ai_usage.read`); the
57 person's session is `openvibe-sdk/sso` (`report` client), and staff pages judge the Network `staff.site.view` role.
58- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/): `report_inbox`, `report_events`,
59 `report_streams`, `report_frame_days`, `report_metric_days|windows|json`, `report_dimensions`,
60 `report_shadow_runs|diffs`, `report_ai_usage[_days]`, `report_ai_exchange_days|_seen|_rejected`,
61 `report_index_revisions`, `report_me_stats`, `report_sessions`, `report_session_defaults`, share tables,
62 `event_outbox`, `config_keys`, `config_snapshots`.
63- **Deploy:** [deploy/systemd/openvibe-report.service](deploy/systemd/openvibe-report.service) (127.0.0.1:5080,
64 `/etc/openvibe/report.env`, `/opt/openvibe.report`) and [deploy/nginx/openvibe.report.conf](deploy/nginx/openvibe.report.conf)
65 (TLS, www→apex, `/api/` GET-only with a coarse limit, `/internal/` refused). `sudo ovhost deploy report`; database
66 `ov_report`. `/api/ready` reports Network/Live as optional checks and is `degraded` (HTTP 200) when one is down; the
67 database becomes required only under `REPORT_SOURCE=report`.
68
69## Ownership and Boundaries
70
71Report owns its analytics tables and computation, the AI usage meter and Exchange aggregates, config snapshots,
72session reports and their share rows, the `report_me_stats` snapshots and the Search sequencer/outbox. It does **not**
73own money (Billing), the Rundown/after-show (Live), benchmark results (Bench), identity/sessions/roles (Network), the
74frame (Shared), or event delivery (Events). Never recompute a Bench score/cost/time or a Billing amount; quote the
75source and its read time. A section a source did not answer for is **absent**, never a zero.
76
77## Auth and Capabilities
78
79- The person's session is Network SSO (`report` client); `/me`, `/you` and `/u/:name` are private/noindex except the
80 sections the person publishes from their last snapshot. Staff pages need the `staff.site.view` role claim.
81- Operator routes take a Network **service token** with `report.ai_usage.read`, verified through
82 `openvibe-contracts serviceAuth`; Report's own sign-in never grants that.
83- Outbound: `billing.stats.read`/`billing.balance.read` (client-credentials token, audience `openvibe.billing`),
84 Network's open stats/history and Live's loopback reads, Bench/Exchange/AI public reads.
85- `POST /internal/events` is loopback-only, v2-HMAC (`REPORT_EVENTS_SECRET`, comma list for rotation) and idempotent
86 through `report_inbox`; nginx refuses `/internal/` from outside and the app also checks the socket is loopback.
87- The API is read-only except the person's own publish switches and session reports; other methods get 405.
88- Errors are RFC 9457 problem+json; a bad slug/id or a source that answered nothing is a 404.
89
90## API
91
92Public, CORS `*`, `Cache-Control: public, max-age=120` (popularity 300), 120 requests a minute per address:
93`GET /api/v1/stats?days=7|30|90`, `/popular`, `/popular/co-usage`, `/rundown`, `/rundown/topics`,
94`/rundown/t/:slug`, `/after-show/:streamId`, `/c/:username`, `/ai/results`, `/ai/ratings/:kind`,
95`/ai/leaderboard`, `/ai/models/:id`, `/ai/providers`, `/ai/compare`, `/ai/picks`, `/design-health`, `/ai/usage/open`.
96Person: `GET /me/stats?days=`, `PUT /me/stats/published`, `GET /u/:name/stats`, `/me/session-reports`,
97`/session-reports[/:id]`. Staff: `GET /ai/exchange?from=&to=` (only with `REPORT_AI_EXCHANGE=1`).
98Internal (loopback): `POST /internal/events`, `GET|POST /internal/shadow`, `GET /internal/metrics?scope=&id=&days=`,
99`GET /internal/channels/:channel/analytics` (creator analytics plus Billing revenue).
100
101## Conventions
102
103- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter.
104- Every dynamic value in a page goes through `esc()` in [server/views/index.js](server/views/index.js); no raw source
105 HTML is trusted.
106- New schema = a new `migrations/NNNN_name.sql`; never edit one that has run. Keep `report_` prefixes.
107- Ingest is idempotent on event id (`report_inbox`) and the shadow comparison stores both sides and their differences;
108 a replay changes nothing.
109- A source outage shows the last good copy with its read time; AI HTML is `no-store` so the outage notice stays
110 current. Public pages are `public, max-age=120`, the popularity pair 300.
111- CORS is `*` on public aggregates and never includes a person, username, subject or project; co-usage only reads
112 people who published their `sites` section and omits pairs below three people.
113- CSP: add origins to `helmet` in [server/app.js](server/app.js). No Font Awesome; shared glyphs.
114- Phone first: every page fits 390 px; charts/tables must not force horizontal scroll.
115- "Calm reduces, only Off removes": a person may publish fewer sections; only an explicit hide/delete removes data.
116- AI text may describe people; only sex acts/porn stay gated (`test/no-looks-guard.test.js`).
117
118## Key Pitfalls
119
120- **The shadow is not the cutover.** In `REPORT_SOURCE=network` the pages still read Network; Report's tables are
121 computed beside it and compared (`report_shadow_*`). Do not silently switch a page's source.
122- **Never derive** a Bench score/cost/time, a Billing price or a provider's offer; render the source row and its read
123 time. The AI Exchange aggregates use exact nano-USD integers from settlements — no floating-point money.
124- **AI meter/Exchange:** raw calls kept 90 days; settlement ids are counted once (`report_ai_exchange_seen`) and
125 invalid payloads are stored rejected, never dropped silently. `REPORT_AI_EXCHANGE=0` consumes nothing and hides the
126 staff page/API.
127- **Person's stats are private by default.** `/u/:name` copies only the published sections from the last snapshot;
128 a missing section is absent, not zero. `GET /api/v1/me/stats` is `no-store`.
129- **Session reports** belong to their owner; sharing uses `openvibe-sdk/share` and the audience default is the owner's,
130 never a public default.
131- **Staff gate:** the HTML page checks the role claim; the JSON Exchange read checks the service capability. Do not
132 replace one with the other.
133- **Internal routes** must stay loopback-only; a forwarded request or a bad HMAC is refused and changes nothing.
134- **Drill/jobs:** `REPORT_JOBS=0` stops the ingest/shadow loops; add new background work behind it.
135
136## Testing
137
138`npm test` runs each file in its own process with a stubbed fetch: data sources, ingest/idempotency, shadow, AI usage
139and Exchange, AI catalogue/join/market, pages/arena/short, me-stats/you, session reports and sharing, popularity,
140design health, discovery/Search, events route, config, readiness, metrics, auth/limits, no-looks and secret leaks.
141`npm run test:pg` for PostgreSQL. Run the narrowest file first, then the suite; always `node --check` changed files.
142
143## Documentation
144
145- [README.md](README.md) — purpose, what Report owns now, does-not-own, pages, API, ingest/idempotency, configuration,
146 run/test, cutover, deploy, security, AI market provenance.
147- [docs/report.metric.v1.json](docs/report.metric.v1.json) — the proposed flat metric contract.
148- [docs/shared-ratings.md](docs/shared-ratings.md) — shared ratings notes.
149- [STATUS.json](STATUS.json), [.env.example](.env.example), [CHANGELOG.md](CHANGELOG.md), [SECURITY.md](SECURITY.md).
150