openvibers/openvibe.compare AGENTS.md
AGENTS.md
| 1 | # AGENTS.md — OpenVibe.Compare |
| 2 | |
| 3 | ## Project overview |
| 4 | |
| 5 | Public, source-linked comparisons for the OpenVibe network: AI models (price and measured Bench |
| 6 | quality/speed), AI coding agents (Codes harnesses plus verified Bench runs), OpenVibe plans |
| 7 | (Billing's public plan list) and compute prices (Exchange's reviewed GPU/computer tariffs). |
| 8 | Node.js/Express, fed entirely by other services' public APIs, **no database, no accounts, no write |
| 9 | API**. Server-rendered pages; `openvibe-shared` for the shell. |
| 10 | |
| 11 | - Origin `https://openvibe.compare`, port `5280`, unit `openvibe-compare.service`. |
| 12 | - One rule above all: **missing evidence is not zero**. A thing without a measurement is unranked, |
| 13 | never scored 0; a failed feed shows an explicit unavailable state and fills nothing in. |
| 14 | |
| 15 | ## What it owns |
| 16 | |
| 17 | - `GET /` — the comparison index and search. |
| 18 | - `GET /comparisons/:slug` — one comparison; weight overrides via `?w.<criterion>=`. |
| 19 | - `GET /things/:id` — one thing across every comparison that contains it; an AI model's thing page |
| 20 | and `/models/:org/:model` 301 to `/ai/models/:model`. |
| 21 | - `GET /compute` — compute prices from Exchange: filters, sorts and a two/three-offer side-by-side. |
| 22 | - `GET /ai`, `/ai/leaderboard`, `/ai/providers`, `/ai/models/:model`, `/ai/compare[/:a-vs-:b]`, |
| 23 | `/ai/method`, `/ai/feed.xml`, `/ai/harnesses`, `/ai/matrix`, `/ai/:ref` (a Bench task kind or |
| 24 | benchmark) and `/ai/arena` — Report's AI comparison pages, with Bench's boards through |
| 25 | `openvibe-sdk/bench`, the AI catalogue and Exchange market joined, and Report's measured first |
| 26 | token, tokens per second and success beside the prices. `/ai/harnesses` draws each harness's effect |
| 27 | against the neutral `ov-bare` with its interval; `/ai/matrix` crosses harness and model; an agent |
| 28 | pair (`<harness>+<model>-vs-<harness>+<model>`) splits its gap into model, harness and interaction. |
| 29 | `/comparisons/ai-models` and an AI model's `/things/:id` 301 to the AI pages. |
| 30 | - `GET /api/v1/comparisons[/:slug]` and `GET /api/v1/compute` — the same data as JSON, CORS-allowed |
| 31 | for the origins in [server/origins.json](server/origins.json). |
| 32 | - `robots.txt`, `sitemap.xml`, `llms.txt`, Dataset JSON-LD, and `Accept: text/markdown` negotiation — |
| 33 | all generated from the same data, never hand-written separately. |
| 34 | |
| 35 | It owns no private data, no paid execution, no builder or version-history write (reserved in the |
| 36 | schema with no API in v1). |
| 37 | |
| 38 | ## Who it reads |
| 39 | |
| 40 | The feeds in [server/data.js](server/data.js) `ENDPOINTS` and [server/ai-data.js](server/ai-data.js), |
| 41 | and the compute book in [server/compute.js](server/compute.js) `SOURCE`: |
| 42 | |
| 43 | | Feed | What | |
| 44 | |---|---| |
| 45 | | `api.openvibe.exchange/api/v1/exchange/market` | dated offers and checked prices (AI Exchange) | |
| 46 | | `bench.openvibe.services/api/v1/boards/chat.arena` | Arena Elo and median response time (OpenVibe.Bench) | |
| 47 | | `bench.openvibe.services/api/v1/runs?kind=codes.patch` / `codes.ticket` | verified Bench runs (OpenVibe.Bench) | |
| 48 | | `openvibe.codes/api/v1/harnesses` | coding harness catalog (OpenVibe.Codes) | |
| 49 | | `billing.openvibe.network/api/v1/plans` | published plans (OpenVibe.Billing) | |
| 50 | | `openvibe.report/api/v1/ai/performance?group=model&days=30` | measured AI time to first token, tokens per second and success rate, public model rows with a 20-call floor (OpenVibe.Report) | |
| 51 | | `openvibe.exchange/api/compute/prices` | reviewed compute tariffs per provider and region (OpenVibe.Exchange) | |
| 52 | | `bench.openvibe.services/api/v1/{kinds,ratings,benchmarks,rankings,runs,overview}` | AI boards, benchmarks and new runs through `openvibe-sdk/bench` (OpenVibe.Bench) | |
| 53 | | `bench.openvibe.services/api/v1/factors?task_kind=&days=` | model and harness apart: harness effects, agents and interactions, with intervals (OpenVibe.Bench) | |
| 54 | | `api.openvibe.exchange/api/v1/exchange/{catalog,market}` | AI catalogue and dated provider prices (OpenVibe.AI / Exchange) | |
| 55 | |
| 56 | It does not call AI, Live, Network or any authenticated API. Nothing here writes anywhere. |
| 57 | |
| 58 | ## Repository map |
| 59 | |
| 60 | - **Entry:** [server/index.js](server/index.js) — listens on `PORT || 5280`, graceful SIGTERM. |
| 61 | - **App/routes:** [server/app.js](server/app.js) — security headers, static `public/`, `/shared`, |
| 62 | health/ready, `/release.json`, `/api/v1`, pages, 404 and 503. |
| 63 | - **Data:** [server/data.js](server/data.js) — fetches the seven feeds (8 s timeout; 3 s for Report), builds the three |
| 64 | comparisons, coalesces concurrent reads and caches for ten minutes; a failed feed becomes `null`. |
| 65 | - **AI data:** [server/ai-data.js](server/ai-data.js) joins the catalogue/market and Bench's boards with |
| 66 | Report's measured rows and reads Bench's factor fit for the harness board, the matrix and an agent |
| 67 | pair; [server/ai-views.js](server/ai-views.js) renders the Report pages, the Bench boards and the factor |
| 68 | pages. A failed source leaves a worded status and a dash, never a zero; pages cache for two minutes. |
| 69 | - **Compute:** [server/compute.js](server/compute.js) normalizes Exchange's tariffs and derives the |
| 70 | month (730 h) and GPU GB-hour prices; [server/compute-views.js](server/compute-views.js) renders the |
| 71 | page, Markdown and Dataset JSON-LD. A failed feed is not cached. |
| 72 | - **Views:** [server/views.js](server/views.js) — HTML, markdown, escaping, JSON-LD, the method text. |
| 73 | - **Ranking:** [public/rank.js](public/rank.js) — the one ranking implementation, required by both the |
| 74 | server views and the browser. |
| 75 | - **Browser:** [public/comparison.js](public/comparison.js), [public/compare-account.js](public/compare-account.js), |
| 76 | [public/ai-factors.js](public/ai-factors.js) (an agent pair's gap split, drawn with `OVMetrics.split`), |
| 77 | [public/compare.css](public/compare.css). |
| 78 | - **Deploy:** [deploy/systemd/openvibe-compare.service](deploy/systemd/openvibe-compare.service), |
| 79 | [deploy/nginx/openvibe.compare.conf](deploy/nginx/openvibe.compare.conf). |
| 80 | |
| 81 | ## Commands |
| 82 | |
| 83 | ```bash |
| 84 | npm ci |
| 85 | npm start # node server/index.js, http://127.0.0.1:5280 |
| 86 | npm test # node --test test/*.test.js |
| 87 | ov test # the harness's sharded run of what the change reaches |
| 88 | node --check <file.js> |
| 89 | ``` |
| 90 | |
| 91 | There is no `dev` script and no build step: edit `server/` or `public/` and restart. |
| 92 | |
| 93 | ## Testing |
| 94 | |
| 95 | `test/compare.test.js` and `test/compute.test.js` use Node's built-in test runner with fixture feeds. It pins the rules that |
| 96 | matter: every measurement is finite, dated, sourced, and references a known thing and criterion; |
| 97 | missing evidence stays missing; exact model identity only; ranking handles direction, equal values, |
| 98 | zero/invalid weights and all-zero weights (unranked); pages and JSON agree. `ov test` runs the same |
| 99 | file and `test/compute.test.js` plus the repository checks the harness adds. |
| 100 | |
| 101 | ## Conventions |
| 102 | |
| 103 | - **Compact, single-line JS** is the house style here (`server/app.js`, `server/data.js`); match the |
| 104 | file you edit rather than reformatting it. |
| 105 | - CommonJS, `'use strict'`, 4 spaces in the newer data helpers, no linter. |
| 106 | - All user-visible data is escaped with `views.esc`; JSON is embedded with `json()` (escapes `<`). |
| 107 | - Reuse `openvibe-shared/kit`, `openvibe-shared/shell` and `openvibe-shared/serve` instead of new |
| 108 | layout or asset code. |
| 109 | |
| 110 | ## Data and provenance rules |
| 111 | |
| 112 | - A result needs a finite number, a parseable date, and an `https:` source URL; otherwise it is not |
| 113 | added ([server/data.js](server/data.js) `result`). |
| 114 | - Prices: the cheapest dated offer by input + output, both from the same provider. Provider feedback |
| 115 | and observation dates are labelled; no price is estimated. |
| 116 | - Arena quality: at least ten judged games and **exact** model identity (`r.model === id`); no fuzzy |
| 117 | joins, because a fuzzy join silently compares a different model revision. |
| 118 | - Measured AI columns: Report's public model rows over 30 days, matched by the full thing id or the |
| 119 | offer's provider model name. Report groups by model name across providers, so the values describe the |
| 120 | model, not one provider's serving; under 20 calls or a failed feed is a dash, never zero. |
| 121 | - Coding agents: only `provenance === 'verified'` runs; benchmark version and model form separate |
| 122 | criteria; newest verified run per harness/cohort. |
| 123 | - Plans: packages and memberships have separate units and are never ranked together. |
| 124 | - Ranking: min–max within the comparison then a weighted mean on 0–100; equal values get 50; |
| 125 | criteria with no measurements are excluded; all-zero weights leave everything unranked. |
| 126 | |
| 127 | ## Deploy |
| 128 | |
| 129 | No database and no service credentials. The unit runs `/opt/openvibe.compare/server/index.js` on |
| 130 | `127.0.0.1:5280`, env `/etc/openvibe/compare.env`. The README's launch path predates ovhost: |
| 131 | `register-compare.py` registers port 5280 with a lifecycle override until a Contracts manifest is |
| 132 | released, then `merge-deploy.sh` and `ov-vhost-install deploy/nginx/openvibe.compare.conf`. Keep the |
| 133 | vhost and unit in sync with [deploy/](deploy/); a docs-only change needs no deploy. |
| 134 | |
| 135 | ## Pitfalls |
| 136 | |
| 137 | - **Do not invent a write path.** Builder and version history are deliberately absent from v1. |
| 138 | - **Do not cache a failed feed into an old value.** It is `null` for that read; the next request |
| 139 | retries, and the comparison lists it as unavailable. |
| 140 | - **Do not hand-write `sitemap.xml`, `llms.txt` or JSON-LD.** They are generated from `getData()` and |
| 141 | `getCompute()` so they cannot drift from the pages. |
| 142 | - **Compute is read from Exchange, never copied.** A tariff's checked date and review date are shown |
| 143 | as published; an offer past `expires_at` is marked, never presented as current. |
| 144 | - **Keep `public/rank.js` framework-free and dual-use** (server `require` and browser global); the |
| 145 | test loads it both ways. |
| 146 | - **CORS is an exact allow-list** ([server/origins.json](server/origins.json)); no `*` and no |
| 147 | credentials. |
| 148 | - **`Accept: text/markdown`** on `/`, `/comparisons/:slug`, `/things/:id` and `/compute` is part of |
| 149 | the product; keep it when touching the views. |
| 150 | - **Do not remove the 503 handler**: an unexpected feed failure must answer explicitly, not 500. |
| 151 | |
| 152 | ## Documentation |
| 153 | |
| 154 | The [README.md](README.md) is the whole product brief (there is no `docs/`). Contracts, if a manifest |
| 155 | is added later, live in OpenVibe.Contracts. |
| 156 |