openvibers/openvibe.reviews AGENTS.md

AGENTS.md

main 12.1 KB · 167 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.Reviews
2
3## Project Overview
4
5OpenVibe.Reviews gathers review signals from sources with provenance, resolves them to entities, and publishes honest
6aggregates and editor-written summaries. It also carries members' own reviews, the ratings other sites show (saved by
7members), and official records (inspections, complaints, violations). "Every review site, one search" is the reader
8promise; nothing is invented and no review text from a source is stored.
9
10- **Domain:** `openvibe.reviews` · **Port:** 4830 · **Service id:** `reviews`
11- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development/tests), Valkey 9
12 optional for shared per-actor limits.
13- **Read first:** [README.md](README.md), [STATUS.json](STATUS.json), [docs/threat-review.md](docs/threat-review.md),
14 [.env.example](.env.example).
15
16**Network context:** identity, sessions and roles are OpenVibe.Network (`reviews` OAuth client, PKCE; service
17principal `svc:reviews`). Source items and terms are OpenVibe.Sources (`sources.item.read`, `sources.source.read`) —
18Reviews never fetches a site. Discussion threads are OpenVibe.Community (`community.comment.write`). AI summaries are
19proposals from OpenVibe.AI (`reviews.summary.propose`); Reviews only accepts them as drafts. Search consumes the index
20events. Event delivery is OpenVibe.Events.
21
22## Commands
23
24```bash
25npm ci # install (contracts/sdk/shared/publishing pinned by tag)
26npm run dev # http://localhost:4830, PGlite under data/
27npm start # node server/index.js
28npm test # every test/*.test.js, temp PGlite, mock Network/Sources/AI
29npm run test:pg # the same through PostgreSQL + PgBouncer
30npm run subscribe # create the sources.item.* subscription (first install)
31npm test -- aggregate # only test files whose name contains the word
32node --check <file.js> # syntax check (no linter configured)
33```
34
35No build step: pages are server-rendered; CSS only under [public/css/](public/css/) (`reviews`, `everywhere`,
36`outcomes`). Every page and form works without JavaScript.
37
38## Architecture at a Glance
39
40- **Entry:** [server/index.js](server/index.js) — `start()` builds the app and (when asked) the Sources pull, import
41 queue and Events outbox relay; tests call `start({ workers: false })`. Graceful stop. `REVIEWS_DRILL` keeps
42 boot-time work off.
43- **App factory:** [server/app.js](server/app.js) — CSP, session, `/auth`, `/internal/events`, `/shared`+static,
44 pages, `/api/v1`, discovery, 404/error handler.
45- **Config:** [server/config.js](server/config.js) — `load(env)` is pure; production `/etc/openvibe/reviews.env`.
46- **Domain:** [server/reviews/](server/reviews/) — `service.js`, `source-items.js` (Sources ingest/resolution),
47 `aggregate.js` (the honest aggregate), `extract.js` (signals and values), `home.js`, `everywhere.js` (the
48 every-review-site search), `access.js`, `account-data.js`.
49- **HTTP:** [server/http/api.js](server/http/api.js), [server/http/pages.js](server/http/pages.js),
50 [server/http/events.js](server/http/events.js), [server/http/machine.js](server/http/machine.js) (robots,
51 sitemaps, feeds, llms), [server/http/actor-limits.js](server/http/actor-limits.js), [server/http/common.js](server/http/common.js).
52- **Auth/capabilities:** [server/auth/session.js](server/auth/session.js), [server/auth/viewer.js](server/auth/viewer.js),
53 [server/auth/capabilities.js](server/auth/capabilities.js) (the Contracts grant rule), [server/auth/keys.js](server/auth/keys.js).
54- **Clients/integrations:** [server/integrations/platform.js](server/integrations/platform.js), Sources through
55 `openvibe-publishing/ingest`, Community and Events through the SDK.
56- **Rendering:** [server/render/](server/render/) (`layout.js`, `views.js`, `short.js`); agent twins
57 [server/agent-pages.js](server/agent-pages.js), [server/agent-descriptor.js](server/agent-descriptor.js).
58- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/). Authority tables: `review_entities`,
59 `review_entity_aliases`, `review_sources`, `review_source_items`, `review_signals`, `review_summaries`,
60 `review_summary_revisions`, `review_trust_metadata`, `review_entity_links`. Helpers: `review_aggregates`,
61 `review_member_reviews`, `review_seen_ratings`, `review_records`, `review_summary_citations`, `review_corrections`,
62 `review_audit`, `review_import_queue`, `review_sync_state`, `review_summary_reviews`, `review_summary_drafts`,
63 `review_entity_redirects`, `review_discussion_refs`, `review_index_revisions`, `review_event_outbox`,
64 `review_event_inbox`, `account_data_events`, `telemetry_samples`.
65- **Deploy:** [deploy/systemd/openvibe-reviews.service](deploy/systemd/openvibe-reviews.service) (127.0.0.1:4830,
66 `/etc/openvibe/reviews.env`, `/opt/openvibe.reviews`) and [deploy/nginx/openvibe.reviews.conf](deploy/nginx/openvibe.reviews.conf).
67 `sudo ovhost deploy reviews` (git-checkout, database `ov_reviews`, migrations at boot add tables/columns only);
68 rollback on a failed `/api/ready`, then `sudo ovhost rollback reviews --to <sha>`.
69
70## Ownership and Boundaries
71
72Reviews owns entities/aliases, the Sources items it read (signal fields only — **never review text**), typed signals
73with provenance, summaries and immutable revisions, trust decisions with history, typed links/merges, aggregates,
74member reviews, seen ratings, official records, corrections and the append-only editorial audit. It does **not** own
75discussion (Community), source adapter policy or ingestion (Sources), AI generation (AI), the search index (Search),
76accounts/roles (Network) or event delivery (Events). A merge is a `merged_into` link with who/when/why and can be
77reversed by a split; never delete an entity to hide a merge.
78
79## Auth and Capabilities
80
81One capability per route, audience `openvibe.reviews`; a service acting for a person sends `X-OV-Subject`. Editorial
82writes need a person who is staff or in `REVIEWS_EDITORS`. A cookie write started by another site is 403
83`request.cross_site` (Bearer tokens are unaffected). Errors are RFC 9457 problem+json.
84
85| Capability | Covers |
86|---|---|
87| `reviews.entity.resolve` | `POST /resolve`, reads, history/aggregate/summary revision, items and item resolution |
88| `reviews.entity.manage` | entity CRUD, aliases, links, trust, corrections queue/resolve |
89| `reviews.entity.merge` / `.split` | merge into another entity, split a merge |
90| `reviews.signal.import` | signal import, source sync |
91| `reviews.summary.propose` | AI proposal (the only AI write) |
92| `reviews.summary.publish` | create/review/publish/unpublish a summary revision |
93| `reviews.correction.submit` | a person's correction |
94
95Outbound: `sources.item.read`, `sources.source.read`, `events.event.publish` + `events.subscription.manage`,
96`community.comment.write`. Per-actor limits ([server/http/actor-limits.js](server/http/actor-limits.js)): reads
97120/min–3000/h, entity writes 30/300, merge/split 10/100, item resolution and import 60/1200, sync now 2/20, summary
98writes 30/300, proposals 60/1200, correction submit 5/20, correction resolve 30/300, discussion comment 20/300. A
99form and its API route share one budget; forms count only a signed-in person. Never limit health/readiness/release/
100metrics, sign-in, public reads (per-address only) or signed `/internal/events`.
101
102## API and Pages
103
104Pages: `/`, `/about`, `/search`, `/e/:slug` (+ `.json`, `/history`, `/summary/:n`), `GET|POST /e/:slug/correct`,
105`POST /e/:slug/discuss`, the editor desk (`/editor/...`, `/e/:slug/edit`), and discovery
106(`/robots.txt`, `/sitemap.xml`, `/sitemaps/*.xml`, `/feed.atom`, `/feed.json`, `/llms.txt`).
107API `/api/v1`: `POST /resolve`, `GET /entities`, `/entities/:ref`, `/history`, `/aggregate`,
108`/summary/revisions/:n`, `GET /items`, `/items/:id`, `POST /items/:id/resolution`, entity/alias/link/trust writes,
109`POST /signals/import`, `POST /sources/sync`, summary proposals/revisions/review/publish/unpublish, correction
110submit/resolve, merge/split, `GET /corrections`. `/internal/events` is loopback-only and HMAC-signed; machine
111endpoints are loopback for `/metrics`.
112
113## Conventions
114
115- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter.
116- **Provenance is mandatory** on every signal/aggregate: source item, revision, observed/retrieved date, the source's
117 terms. The same item revision's signal values are immutable (database triggers); a changed value is a new signal and
118 the old one is `superseded`/`withdrawn` with a reason.
119- The aggregate never silently drops a signal: a trust exclusion needs a visible reason, limitation and verification
120 notes (`review_trust_metadata`; history kept). Official records are labelled as records, separate from ratings.
121- New schema = a new `migrations/NNNN_name.sql`; never edit one that has run. Keep `review_` prefixes.
122- Source text is never copied: store signal fields and quotes only where the licence allows. A source removal drops
123 its signals and recalculates, never leaves a stale number.
124- Summaries: editor-written or AI proposal, immutable revisions, authorship, citations to the signals; an AI draft
125 needs a person's review before publish. A correction is published at once with `correction_note`/`correction_id`.
126- CSP: add origins to `helmet` in [server/app.js](server/app.js); Community comments are rendered from its API, never
127 copied into Reviews' HTML storage. No Font Awesome; shared glyphs.
128- Phone first: entity, search, editor and home pages fit 390 px; forms work without JS.
129- Empty states use the shared kit. "Calm reduces, only Off removes": excluding a source from an aggregate is visible
130 and reversible; only a takedown/delete removes data.
131- Hostile text stays text in every page, JSON, JSON-LD, feed, sitemap and Search document.
132
133## Key Pitfalls
134
135- **Never fetch a source or a user-supplied URL.** Ingest only through Sources; an alias/URL is stored and rendered,
136 never requested. [docs/threat-review.md](docs/threat-review.md) is the threat model.
137- **Strong identifiers are unique** (GTIN/SKU/MPN, source binding, external id); a conflicting resolution is a review
138 action, not a silent overwrite. Merges/splits are audited and reversible; old slugs redirect.
139- **Signals are immutable:** a trigger rejects an UPDATE of the value fields. A re-import of the same revision is a
140 no-op; a new revision supersedes the old with `replaces`.
141- **Corrections:** anyone signed in may submit; only staff/editor resolves. Accepting with a correction also needs
142 `reviews.summary.publish`, and the resulting revision carries the correction note.
143- **Per-actor budgets** must be checked after the actor is resolved and the capability passed, before any work; a form
144 budget is not a second allowance.
145- **AI summaries** are drafts only; the stub provider's output is never public, and no AI call happens without the
146 optional AI URL.
147- **Drill mode** keeps the Sources pull, import queue, outbox relay and timers off.
148- **`review_event_outbox`** needs its rejected partial index (migration 0009); keep it.
149- Account deletion removes a person's member reviews/seen ratings and receipts through `/internal/events`; a
150 redelivery changes nothing.
151
152## Testing
153
154`npm test` runs each file in its own process on temp PGlite with mock Network/Sources/AI: resolution/aliases,
155signals/aggregates, summaries/proposals/publish, corrections, merges/splits, member reviews/seen ratings, official
156records, home/sitemap/link-header/IndexNow, no-JS/pages, capabilities/delegation, actor limits, account data, CSP,
157Ink/contrast, threat, service kit and perf budgets. `npm run test:pg` for PostgreSQL. Run the narrowest file first,
158then the suite; always `node --check` changed files.
159
160## Documentation
161
162- [README.md](README.md) — purpose, every-review-site search, member reviews, outside ratings, official records,
163 owns/does-not-own, what works, routes, API, limits, events, dependencies, capabilities, acceptance, security, deploy.
164- [docs/threat-review.md](docs/threat-review.md) — the pre-launch threat review.
165- [docs/capabilities-proposal/](docs/capabilities-proposal/) and [docs/service-manifest-proposal.json](docs/service-manifest-proposal.json).
166- [STATUS.json](STATUS.json), [.env.example](.env.example), [CHANGELOG.md](CHANGELOG.md), [SECURITY.md](SECURITY.md).
167