openvibers/openvibe.help README.md
README.md
| 1 | # OpenVibe.Help |
| 2 | |
| 3 | > Answers, support and a way to reach people. |
| 4 | |
| 5 | **Live site:** https://openvibe.help · **API docs:** https://openvibe.services · **For AI:** https://openvibe.help/llms.txt |
| 6 | **Run it:** `npm ci && npm run dev` (Node 22 or newer) · **Tests:** `npm test` |
| 7 | |
| 8 | **Status:** the product is written, tested and deployed (see [STATUS.json](STATUS.json)). |
| 9 | **Domain:** `openvibe.help` · **Port:** 5020 · **Service id:** `help` · **Env prefix:** `HELP` |
| 10 | **License:** AGPL-3.0 (same as every OpenVibe service). |
| 11 | |
| 12 | The help and support centre for every OpenVibe site: the answers each site gives about itself, search over those and |
| 13 | over the whole network, and a support ticket that stays one conversation. |
| 14 | |
| 15 | ## Purpose |
| 16 | |
| 17 | OpenVibe.Help is the help and support centre for every OpenVibe product site. At boot it builds a catalog of every |
| 18 | site from the pinned OpenVibe.Contracts manifests, generates one article per manifest FAQ entry, and searches those |
| 19 | articles alongside OpenVibe.Search's public network index. A signed-in person can also open a support ticket that |
| 20 | stays one conversation with staff. It is the service behind `openvibe.help` (port 5020, service id `help`, env |
| 21 | prefix `HELP`). |
| 22 | |
| 23 | ## Owns |
| 24 | |
| 25 | OpenVibe.Help is the authority for support tickets and their conversations, and for the receipts of the |
| 26 | account-export and account-deletion events it has applied. Its migrations create: |
| 27 | |
| 28 | | Table | Holds | Migration | |
| 29 | |---|---|---| |
| 30 | | `help_tickets` | a ticket: `tkt_<ULID>` id, requester (`user:usr_…`), site, subject, state `open \| waiting \| solved`, `page_url` (text only), created/updated | `migrations/0002_tickets.sql` | |
| 31 | | `help_messages` | one message: `msg_<ULID>` id, ticket id (`ON DELETE CASCADE`), author, role `person \| staff`, body, created_at | `migrations/0002_tickets.sql` | |
| 32 | | `account_data_events` | the export or deletion id applied, its subject, outcome and timestamps | `migrations/0003_account_data.sql` | |
| 33 | | `telemetry_samples` | request telemetry: one `http.request` sample a minute per route template, method and status class (never a raw URL), plus the active-requests, p95 latency and event-loop-lag gauges (openvibe-sdk 0.46, `platform.telemetry-sample@1`); `/shared`, static assets, health, readiness and `/metrics` are not counted; `server/app.js` builds the collector, `server/index.js` flushes it every minute and deletes rows older than 14 days daily | `migrations/0005_telemetry.sql` | |
| 34 | |
| 35 | `migrations/0001_initial.sql` creates no tables. The catalog and its articles are not stored — they are rebuilt from |
| 36 | OpenVibe.Contracts at every boot. |
| 37 | |
| 38 | ## Does not own |
| 39 | |
| 40 | OpenVibe.Help uses, but does not own: |
| 41 | |
| 42 | - **Identity, sessions and staff roles** — OpenVibe.Network. Help verifies the Network session token (OAuth client |
| 43 | `help`, PKCE S256) and its JWKS, and reads the role claim (`admin` | `global_mod`) to decide staff. It stores no |
| 44 | accounts and no credentials. |
| 45 | - **The help text** — OpenVibe.Contracts. The product and service manifests are read-only and quoted verbatim; Help |
| 46 | does not write or edit them. |
| 47 | - **The network search index** — OpenVibe.Search. The "From across OpenVibe" half of `/search` is its public API. |
| 48 | - **Event delivery** — OpenVibe.Events. Help subscribes to two topics and answers the loopback delivery; it does |
| 49 | not run the broker. |
| 50 | - **Account export and deletion** — OpenVibe.Network (ADR-033). Help answers its part of a Network-initiated export |
| 51 | or deletion and confirms it. |
| 52 | - **The host, database and Valkey** — OpenVibe.Host. Deploy, the PostgreSQL database and roles, and the shared |
| 53 | per-caller limit counters are the host's. |
| 54 | |
| 55 | ## Depends on |
| 56 | |
| 57 | Services and packages it calls, with the environment variables that point at them (see [.env.example](.env.example)): |
| 58 | |
| 59 | - **OpenVibe.Network** — sign-in, token exchange, refresh and revoke, and the JWKS: `OV_NETWORK_URL`, |
| 60 | `OV_NETWORK_INTERNAL_URL`, `OV_NETWORK_ISSUER`, `OV_OAUTH_CLIENT_ID` (`help`), `OV_OAUTH_CLIENT_SECRET`, |
| 61 | `OV_OAUTH_REDIRECT_URI`, `OV_SESSION_AUDIENCE`; tokens it accepts are for `HELP_AUDIENCE` (`openvibe.help`). |
| 62 | - **OpenVibe.Search** — the public query API: `HELP_SEARCH_URL` (default `https://search.openvibe.network`), |
| 63 | `HELP_SEARCH_LIMIT`, `HELP_SEARCH_TIMEOUT_MS`, `HELP_SEARCH_CACHE_MS`. |
| 64 | - **OpenVibe.Events** — where the two account topics are subscribed at boot: `HELP_EVENTS_URL` (or `EVENTS_URL`), |
| 65 | `HELP_EVENTS_SECRET`, `HELP_EVENTS_ENDPOINT`, `HELP_EVENTS_SUBSCRIBE=0` to turn it off. |
| 66 | - **PostgreSQL** — `DATABASE_URL` (through PgBouncer) and `DATABASE_DIRECT_URL` (migrations, owner role); an |
| 67 | embedded PGlite database in `HELP_PGLITE_DIR` for development. |
| 68 | - **Valkey** — shared per-caller limit counters: `VALKEY_URL`, `VALKEY_PREFIX`. |
| 69 | - **Packages** — `openvibe-contracts` (read at boot; pinned in `package.json`), `openvibe-sdk`, `openvibe-shared`. |
| 70 | |
| 71 | ## Capabilities |
| 72 | |
| 73 | `openvibe-contracts` declares no `help.*` capability, the service manifest's `capabilities` is `[]`, and |
| 74 | `CAPABILITIES` in [server/http/principal.js](server/http/principal.js) is empty for the same reason: every route is |
| 75 | a person acting for themself, or staff judged by the Network role claim, so no route is guarded by a capability. |
| 76 | The capabilities and scopes it uses on other services: |
| 77 | |
| 78 | - **OpenVibe.Events** — scope `events.subscription.manage` (audience `openvibe.events`) to create the two boot-time |
| 79 | subscriptions. |
| 80 | - **OpenVibe.Network** — OAuth scope `profile` for sign-in, and a client-credentials token (audience |
| 81 | `openvibe.network`) to push account-export parts and deletion confirmations. |
| 82 | - **Events** — consumed: `network.account.export_requested`, `network.account.deleted`; produced: none. |
| 83 | - **Not used yet** — `staff.console.access`, the Network staff capability |
| 84 | [server/tickets/staff.js](server/tickets/staff.js) would call in place of the role comparison if the Network |
| 85 | starts issuing it. |
| 86 | |
| 87 | ## What it does |
| 88 | |
| 89 | ### 1. Help articles, generated from real content — no database, no hand-written copy |
| 90 | |
| 91 | `server/catalog.js` reads, at boot, from `openvibe-contracts`: |
| 92 | |
| 93 | - **every product manifest** (`manifests/products/<domain>.json`) — `name`, `tagline`, `description`, `pillars`, `faq`, |
| 94 | `keywords`, `launch`, `kind`; |
| 95 | - **every service manifest whose `site` block presents a site** (`manifests/services/<id>.json`) — the same fields, |
| 96 | plus `exposure.state`. |
| 97 | |
| 98 | It builds one site per domain (a domain that is both a product and a service site is merged into one) and one article |
| 99 | per FAQ entry. Every text is the manifest's, verbatim — this service quotes, it does not write. Each page shows where |
| 100 | its text came from (the manifest paths, a link back, and when it was read). |
| 101 | |
| 102 | | Page | What it is | |
| 103 | |---|---| |
| 104 | | `/sites` | every OpenVibe site: its one-line what, its state (`live` / `coming`), and links | |
| 105 | | `/sites/:id` | one site: its pillars as "what you can do", its FAQ as anchored articles, links to the site and to its `/updates` | |
| 106 | | `/a/:site/:slug` | one question, one page (SEO title = the question), the answer verbatim, links back | |
| 107 | |
| 108 | `live` is the manifest's own `exposure.state === 'live'` on a service that exposes that domain; everything else is |
| 109 | `coming`. Today: 45 sites, 22 live, 20 questions across the 3 sites that carry an `faq`. |
| 110 | |
| 111 | ### 2. Search |
| 112 | |
| 113 | `/search?q=` shows two sections: |
| 114 | |
| 115 | - **Help articles** — `server/search.js` scores the generated articles over the question, the answer and the site's own |
| 116 | text (a phrase hit beats term hits; the question beats the answer; the answer beats the site's blurb). The same |
| 117 | ranking answers `GET /api/v1/articles?q=`. There is no index and no stemming: the same query always answers the same |
| 118 | order, and every rule is in one function so a surprising order can be explained instead of tuned. |
| 119 | - **From across OpenVibe** — OpenVibe.Search's public API, |
| 120 | `GET {HELP_SEARCH_URL}/api/v1/search?q=&limit=8` → `results[{title, summary, canonical_url, type, owner}]`. Answers |
| 121 | are cached in this process for 5 minutes. A failure (timeout, an error, an answer without a result list) is shown as |
| 122 | a failure and the page degrades to the articles alone — never to a fake empty index. |
| 123 | |
| 124 | OpenVibe.Search at `search.openvibe.network` is the **only** host `/search` calls; a person's query travels as a query |
| 125 | parameter, never as a URL, and a `canonical_url` becomes a link only when it parses as absolute http(s). |
| 126 | |
| 127 | ### 3. Support tickets |
| 128 | |
| 129 | A signed-in person opens a ticket — the site it is about (picked from the catalog), a subject, a message, and |
| 130 | optionally the page they were on. The page URL is **stored as text only and never fetched**; it is rendered as a link |
| 131 | only when it parses as absolute http(s). The first message is the description, so a ticket is never empty; the ticket |
| 132 | page is the conversation; a reply stays there. |
| 133 | |
| 134 | - **A person sees only their own tickets** — anyone else gets 404, exactly like a ticket that does not exist. |
| 135 | - **Staff** — the Network role claim on the session (`admin` or `global_mod`, `server/tickets/staff.js`) — see |
| 136 | `/staff`: every ticket, filterable by site and state, and they may reply and set `open | waiting | solved`. |
| 137 | - Bodies are plain text rendered escaped with line breaks kept, at most 10 000 characters. |
| 138 | |
| 139 | Tables (`migrations/0002_tickets.sql`): `help_tickets` (`tkt_<ULID>`, requester subject, site, subject, state, |
| 140 | `page_url`, created/updated) and `help_messages` (`ticket id, author subject, role person|staff, body, created_at`). |
| 141 | Neither is ever a credential, key or token. |
| 142 | |
| 143 | ### 4. Discovery |
| 144 | |
| 145 | `sitemap.xml` and `llms-full.txt` are two renderings of one page list, so every site and every question is crawlable in |
| 146 | both and they cannot drift apart. `llms.txt` lists the sites with their state. `/tickets`, `/staff`, `/auth/` and |
| 147 | `/api/` are disallowed; the home page's JSON-LD carries the sitelinks search action. |
| 148 | |
| 149 | ## Pages |
| 150 | |
| 151 | | Route | What it is | |
| 152 | |---|---| |
| 153 | | `/` | the help centre: a search box, "Popular questions" (the questions the sites answer), "Browse by site", and "Contact support" | |
| 154 | | `/sites`, `/sites/:id`, `/a/:site/:slug` | the generated help centre | |
| 155 | | `/search?q=` | the help articles and the network, in two sections | |
| 156 | | `/tickets/new`, `/tickets`, `/tickets/:id` | open a ticket, your tickets, one conversation (sign in required) | |
| 157 | | `/staff` | the support queue (staff only) | |
| 158 | | `/updates` | the network changelog for this service | |
| 159 | |
| 160 | All of it is server-rendered through the OpenVibe Frame and works without JavaScript: the search box is a GET, the |
| 161 | forms are plain POSTs. A signed-in write must come from openvibe.help itself (as in the API). |
| 162 | |
| 163 | ## API |
| 164 | |
| 165 | Everything under `/api/v1`. A person needs no capability (their own tickets, their own session); staff are judged by |
| 166 | their Network role claim. Writes made with the session cookie must be same-origin. Errors are RFC 9457 |
| 167 | `application/problem+json` with a stable `code`. |
| 168 | |
| 169 | | Route | Who | What | |
| 170 | |---|---|---| |
| 171 | | `GET /ping` | anyone | `{ ok: true, service: "help" }` | |
| 172 | | `GET /sites` | anyone | the whole catalog with each site's state, one line and source | |
| 173 | | `GET /sites/:id` | anyone | one site: pillars, FAQ (with article URLs), where it lives, its source | |
| 174 | | `GET /articles?q=` | anyone | the generated articles; `?q=` ranks them | |
| 175 | | `POST /tickets` | a person | open a ticket (`{ site?, subject, message, page_url? }`) → 201 `tkt_<ULID>` | |
| 176 | | `GET /tickets` | a person | your tickets, newest first (`?limit=&before=`) | |
| 177 | | `GET /tickets/:id` | a person | one ticket and its conversation (staff may read any; anyone else 404) | |
| 178 | | `POST /tickets/:id/messages` | a person | reply (`{ message }`); a staff reply is marked `staff` | |
| 179 | | `GET /staff/tickets` | staff | the queue (`?site=&state=open\|waiting\|solved\|all`) with counts | |
| 180 | | `POST /tickets/:id/state` | staff | `{ state: "open" \| "waiting" \| "solved" }` | |
| 181 | |
| 182 | Problem codes: `token.required` (401), `help.person_required` (403 — an app token cannot act as a person), |
| 183 | `request.cross_site` (403), `staff.forbidden` (403), `ticket.not_found` (404), `help.site.not_found` (404), |
| 184 | `ticket.invalid` (422), `rate_limited` (429). |
| 185 | |
| 186 | `http/principal.js`'s `CAPABILITIES` is deliberately empty: `openvibe-contracts` declares no `help.*` capability yet, |
| 187 | and every route here is a person acting for themself or staff by their role claim. The day a route exists for an app |
| 188 | (a support widget a site embeds, say), its capability is added here and in `openvibe-contracts` together. |
| 189 | |
| 190 | ## Configuration |
| 191 | |
| 192 | See [.env.example](.env.example). Required in production: `OV_OAUTH_CLIENT_SECRET` (the `help` OAuth client on the |
| 193 | Network), `BASE_URL`, `DATABASE_URL` and `DATABASE_DIRECT_URL`. Optional: `HELP_SEARCH_URL` (default |
| 194 | `https://search.openvibe.network`), `HELP_SEARCH_LIMIT`, `HELP_SEARCH_TIMEOUT_MS`, `HELP_SEARCH_CACHE_MS`, the |
| 195 | per-caller limits and Valkey. The database is the only required readiness check; the Network signing key, the OAuth |
| 196 | client and Valkey are optional (the service says so, per check, on `/api/ready`). |
| 197 | |
| 198 | ## Data sources and their terms |
| 199 | |
| 200 | | Source | What is taken | Terms as applied | |
| 201 | |---|---|---| |
| 202 | | [OpenVibe.Contracts](https://github.com/OpenVibers/OpenVibe.Contracts) `manifests/products/*.json` and `manifests/services/*.json` (pinned `v0.115.0`, read at boot from `node_modules`) | site names, taglines, descriptions, pillars, FAQ, keywords, launch notes, `exposure.state` | Read-only, from the pinned release the service already depends on; quoted verbatim with the manifest path, a link back and the time it was read shown on the page. No text is invented, rewritten or translated. | |
| 203 | | [OpenVibe.Search](https://search.openvibe.network) (`GET /api/v1/search`) | the network half of `/search`: `title`, `summary`, `canonical_url`, `type`, `owner` | Public API (anonymous queries need no token). Fetched only from the configured host, with a `User-Agent` and a timeout, cached 5 minutes, never on a URL a person typed. Failures are shown as failures. | |
| 204 | |
| 205 | Nothing else is fetched. No user-supplied URL is ever requested by this service. |
| 206 | |
| 207 | ## Development |
| 208 | |
| 209 | ```bash |
| 210 | npm install |
| 211 | fnm exec --using=22 npm test # every test/*.test.js, on temp PGlite databases with a mock Network |
| 212 | fnm exec --using=22 npm run dev # http://localhost:5020 |
| 213 | ``` |
| 214 | |
| 215 | Without `DATABASE_URL` development uses an embedded PGlite database in `data/pglite` (one process only). `npm run |
| 216 | test:pg` runs the same suite through PostgreSQL and PgBouncer (see [.github/workflows/ci.yml](.github/workflows/ci.yml)). |
| 217 | |
| 218 | ## Tests |
| 219 | |
| 220 | `npm test` runs every `test/*.test.js` in its own process, on a temp PGlite database, with an in-process mock of |
| 221 | OpenVibe.Network; outbound HTTP is stubbed (`test/helpers/search-provider.js` stands in for OpenVibe.Search), so |
| 222 | nothing calls the real internet. |
| 223 | |
| 224 | | File | What it proves | |
| 225 | |---|---| |
| 226 | | `test/help-articles.test.js` | the catalog is exactly the manifests (once per domain), every site and every question has a page, the text and the state are the manifest's, escaping | |
| 227 | | `test/help-search.test.js` | the score and its rules, the ranking over the real catalog, both sections, the 5-minute cache, and failures (error, timeout, malformed) degrading to the articles alone | |
| 228 | | `test/tickets.test.js` | the whole lifecycle: create, list own only, reply, staff state change, another person 404, anonymous 401, app tokens refused, cross-site session writes refused, escaping, the 10 000-character cap, per-caller limits | |
| 229 | |
| 230 | `test/skeleton.test.js` skips itself in a generated service: the skeleton's `__ID__`-style placeholders are already |
| 231 | substituted here, so `scripts/new-service.js` (which copies the tree it lives in) cannot rebuild the sample. A skip is |
| 232 | listed with ○ and is not counted as a pass. |
| 233 | |
| 234 | ## Deploy (for the lead) |
| 235 | |
| 236 | - **Deploy:** `sudo ovhost deploy help` on the host (git checkout at `/opt/openvibe.help`, unit |
| 237 | `openvibe-help.service` on 127.0.0.1:5020, env `/etc/openvibe/help.env`, database `ov_help` on the data role). |
| 238 | - **nginx:** [deploy/nginx/openvibe.help.conf](deploy/nginx/openvibe.help.conf), installed with `ov-vhost-install`. |
| 239 | - **Rollback:** ovhost puts the previous sha back by itself when `/api/ready` does not answer after the restart. |
| 240 | - Register the service and its capabilities in **OpenVibe.Contracts** (`contracts-service: help` in CI) and with |
| 241 | **OpenVibe.Services** before the first deploy. No new environment variable is required to run: `HELP_SEARCH_URL` has |
| 242 | a default. |
| 243 | |
| 244 | ## Account export and deletion |
| 245 | |
| 246 | A person's account at OpenVibe.Network can be exported and deleted, and every service holding their rows answers its |
| 247 | part (ADR-033). Help receives `network.account.export_requested` and `network.account.deleted` at |
| 248 | `POST /internal/events` (loopback only) — the two tables are mapped in |
| 249 | [server/identity/account-data.js](server/identity/account-data.js), and the boot-time subscriptions are created by |
| 250 | [server/events-consumer.js](server/events-consumer.js): |
| 251 | |
| 252 | - **Exported:** the tickets a person opened (`tickets.json`) and every message they wrote (`messages.json`), pushed to |
| 253 | `POST /internal/account-exports/:id/parts` with this service's own token. Nothing here is a secret — Help stores no |
| 254 | token, key or credential. |
| 255 | - **Erased:** the person's own tickets are deleted; the `ON DELETE CASCADE` on `help_messages.ticket_id` takes every |
| 256 | message under them — from either side — with them. |
| 257 | - **Anonymized:** a staff reply the person wrote in someone else's ticket keeps its body — that ticket's owner still |
| 258 | has to read their own conversation — and its `author` becomes NULL. (`help_messages.author` is nullable for exactly |
| 259 | this reason.) Help then confirms with `POST /internal/account-deletions/:id/confirmations` and the counts. |
| 260 | |
| 261 | Environment: `HELP_EVENTS_SECRET` (comma-separated for rotation, 32+ characters each; unset makes the route answer |
| 262 | 503), `HELP_EVENTS_URL` (or `EVENTS_URL`) is where the two subscriptions are created at boot (off when unset), and |
| 263 | `HELP_EVENTS_ENDPOINT` overrides the loopback endpoint; `HELP_EVENTS_SUBSCRIBE=0` turns the boot-time subscription off. |
| 264 | |
| 265 | ## Security (threat notes) |
| 266 | |
| 267 | Reporting a vulnerability: [SECURITY.md](SECURITY.md). |
| 268 | |
| 269 | - Session tokens are httpOnly cookies; a FedCM assertion or an app or service token is never a session. |
| 270 | - A ticket belongs to the person who opened it: another person gets 404, and staff are decided by the Network role |
| 271 | claim on the session, never by anything in the request body. |
| 272 | - Nothing in a request may decide a URL this service fetches. A page URL a person pastes is stored as text and never |
| 273 | opened; `/search` calls only the fixed `HELP_SEARCH_URL` host, with the query as a parameter. A `canonical_url` from |
| 274 | OpenVibe.Search becomes a link only when it parses as absolute http(s). |
| 275 | - Every dynamic value in HTML goes through the `html` tagged template; bodies are escaped and shown with their line |
| 276 | breaks. |
| 277 | - Secrets live only in the env file; only environment variable names appear in code and docs, and no secret is logged. |
| 278 | - Request bodies are never logged (a ticket body can be personal). |
| 279 | |
| 280 | --- |
| 281 | |
| 282 | Part of the [OpenVibe network](https://openvibe.network). Built in the open by [OpenVibers](https://github.com/OpenVibers). |
| 283 | |
| 284 | <!-- versions:start --> |
| 285 | - openvibe-contracts: v0.419.0 |
| 286 | - openvibe-sdk: v0.161.0 |
| 287 | - openvibe-shared: v4.241.0 |
| 288 | - openvibe-publishing: v1.7.0 |
| 289 | <!-- versions:end --> |
| 290 |