openvibers/openvibe.vision AGENTS.md
AGENTS.md
| 1 | # AGENTS.md — OpenVibe.Vision |
| 2 | |
| 3 | ## Project overview |
| 4 | |
| 5 | OpenVibe.Vision (`https://openvibe.vision`) answers questions about pictures and pages: ask/describe/ |
| 6 | identify/count/translate, OCR with a local Tesseract first, side-by-side model comparison, UI checks |
| 7 | (HTML, public URL or screenshot), a consented face library, and the network's read-only MCP tools. |
| 8 | |
| 9 | - **Entry:** `server/index.js`; app `server/app.js`; port `5130`, service id `vision`. |
| 10 | - **API/pages:** `server/http/` — `api.js` (`/api/v1`), `faces-api.js`, `mcp.js`, `principal.js`, |
| 11 | `pages.js`, `discovery.js`; views `server/views/index.js`, `people.js`, `review.js`, `tools.js`; |
| 12 | rendering `server/render/`; own browser script `public/js/upload-picker.js`, CSS/img in `public/`. |
| 13 | - **Engine:** `server/vision/` — `service.js`, `images.js` (normalise/limits), `ocr.js` (Tesseract 5, |
| 14 | `OMP_THREAD_LIMIT=1`), `ui-check.js` (static parser + WCAG maths + screenshot OCR/sharp), `ai.js` |
| 15 | (`ai.run.create`), `network-tools.js`, `review.js`, `media-view.js`, `local-models.js`. |
| 16 | - **Faces:** `server/vision/faces.js`, `faces-model.js` (YuNet + SFace + ONNX Runtime), |
| 17 | `face-libraries.js`, `consent-registry.js`; consent and encrypted vectors in the `face_*` tables. |
| 18 | - **Events:** `server/events/` (outbox, publisher) and `server/identity/account-data.js`. |
| 19 | - **Docs:** `README.md` ("Faces with consent", "Unsafe-image review", "MCP and events"), `SECURITY.md`. |
| 20 | |
| 21 | ## Owns, does not own |
| 22 | |
| 23 | Owns its tables: `vision_answers` (`vis_`; task, question, source, image sha256/mime/bytes, cleaned |
| 24 | text, answer jsonb, model/provider/route, cost, latency), `vision_compare_samples` (`vcm_`), |
| 25 | `vision_ui_checks` (`vuc_`), `account_data_events`, `telemetry_samples`, and the face tables — |
| 26 | `face_consents`, `face_templates`, `face_media`, `face_tags`, `face_clusters` — plus `event_outbox`. |
| 27 | General answer tables store no image bytes, token or provider key; opt-in face media and vectors are |
| 28 | AES-GCM encrypted. |
| 29 | |
| 30 | Does not own: models, routing, quotas and provider keys (AI — Vision makes first-party `ai.run.create` |
| 31 | runs and AI decides/reports the model and provenance); identity/roles (Network); the local OCR binary |
| 32 | and language data (the host: Tesseract under `/usr/bin`, `/usr/share`); Search (nothing public to |
| 33 | index); Events/Host/database. **Vision holds no provider key and calls no paid provider directly.** |
| 34 | |
| 35 | ## Who calls it, what it calls |
| 36 | |
| 37 | People use the pages/API (session cookie or Network user Bearer; a session write must come from |
| 38 | `https://openvibe.vision`, else `403 vision.cross_site`); services use a token with the route's |
| 39 | capability (`vision.image.ask`, `vision.text.read`, `vision.model.compare`, `vision.ui.check`, |
| 40 | `vision.face.cast.match`, `vision.alt.write`). Typical callers: Design's Arena for UI checks, Live for |
| 41 | consented cast matching, Pics/Media/Chat for alt text. SDK `openvibe-sdk/vision` is the client surface |
| 42 | and `/mcp` exposes the read-only tools. Vision calls Network (JWKS) and AI (`ai.run.create`); the only |
| 43 | egress is AI plus an explicitly supplied public URL for a UI check, through the guard. |
| 44 | |
| 45 | ## Database and migrations |
| 46 | |
| 47 | PostgreSQL through `openvibe-sdk/db` (PgBouncer for the app, owner role for migrations): `0001` |
| 48 | answers/compare/UI checks, `0003` the face tables, `0004` the event outbox; `0002`/later add the rest. |
| 49 | Applied at boot; never edit a migration that has run. `face_templates`, `face_media` and |
| 50 | `face_clusters` hold ciphertext, never a plaintext vector; `face_tags` disappear when a cast |
| 51 | revocation lands. Account export/deletion covers every table with a person column; a redelivery |
| 52 | changes nothing. |
| 53 | |
| 54 | ## Consent and privacy |
| 55 | |
| 56 | Faces are opt-in and **purpose- and scope-scoped**: a `face_consents` row names the feature |
| 57 | (`verify|find|cast|library`), the channel/scope, the signed text version/name and an optional |
| 58 | `purpose_ends_at`; revoked consent stops detection/matching/tagging. Identifying matches require |
| 59 | separate consent; faces are detected, blurred and matched locally — no face image or vector goes to |
| 60 | AI. An unsafe-image review exists (`server/vision/review.js`); do not bypass it for a faster answer. |
| 61 | `/internal/events` is signed and loopback-only. |
| 62 | |
| 63 | ## Deploy |
| 64 | |
| 65 | `deploy/systemd/openvibe-vision.service` (`127.0.0.1:5130`, env `/etc/openvibe/vision.env`) and |
| 66 | `deploy/nginx/openvibe.vision.conf` (TLS, www → apex, `/auth/me` on the API zone, a forms zone for |
| 67 | `/ask`, `/ocr`, `/compare`, `/ui-check`, `/internal` and `/metrics` 404). `sudo ovhost deploy vision` |
| 68 | is the production path; rollback with `sudo ovhost rollback vision --to <sha>`. |
| 69 | |
| 70 | ## Commands and tests |
| 71 | |
| 72 | ```bash |
| 73 | npm ci |
| 74 | npm run dev # http://127.0.0.1:5130 |
| 75 | npm test # every test/*.test.js, PGlite + mock Network/AI |
| 76 | npm test -- ocr ui # only matching files |
| 77 | npm run test:pg # the same on PostgreSQL |
| 78 | npm run proof # the v1 proof run |
| 79 | npm run models:faces # download the YuNet/SFace models on the host |
| 80 | npm run models:fetch # fetch the local model set |
| 81 | ``` |
| 82 | |
| 83 | `test/run.js` uses `openvibe-shared/test-runner` (90 s, serial); temp PGlite, a mock Network and AI |
| 84 | and local image/page fixtures, never the network. Coverage includes OCR/local-first and fallback, |
| 85 | compare/benchmarks, UI checks, faces/consent, libraries, network tools/MCP, unsafe-image review, |
| 86 | events/outbox, account data and the pages. Some model/OCR work skips without the binaries — a skipped |
| 87 | file is never counted as passed. |
| 88 | |
| 89 | ## Conventions and pitfalls |
| 90 | |
| 91 | - CommonJS, 4-space indent, single quotes, semicolons; problem+json with a stable code; escaped HTML. |
| 92 | - **Local first, AI second, never a direct provider:** Tesseract answers a read for free; AI's routed |
| 93 | model is the only paid path and `requested_model` is a hint, not a promise. Report the model/provider/ |
| 94 | route that AI says answered, with cost and latency. |
| 95 | - **No image bytes in the answer tables**, and no token/provider key anywhere in the repo. The |
| 96 | normalised image is what goes to AI, not the original. |
| 97 | - **Consent is checked per feature and scope, on every use**, and revoked/expired means no match; an |
| 98 | identifying match needs its own purpose. Never use a face vector outside the encrypted store. |
| 99 | - **The unsafe-image review is a gate**, not a suggestion; a route that returns image text or alt text |
| 100 | must run it. |
| 101 | - **A UI check of a URL only fetches a public HTTP(S) address** through the guard; never loopback, |
| 102 | private ranges or a redirect that lands there. |
| 103 | - **Alt text is written as text**, never markup; a caller's content is untrusted. |
| 104 |