README.md

main 30.2 KB · 332 lines · markdown

1	# OpenVibe.Vision
2
3	> See with the best model for the job.
4
5	Live site: https://openvibe.vision · API docs: https://openvibe.services · For AI: https://openvibe.vision/llms.txt
6	Run it: `npm ci && npm run dev` (Node 22 or newer) · Tests: `npm test`
7
8	Status: v1, the §5.8b faces work and the network tools written and tested (see [STATUS.json](STATUS.json)).
9	Domain: `openvibe.vision` · Port: 5130 · Service id: `vision` · Env prefix: `VISION`
10	License: AGPL-3.0-only (same as every OpenVibe service).
11
12	OpenVibe.Vision answers questions about a picture. The cheapest engine goes first: the local Tesseract reader
13	answers a read question for nothing; otherwise OpenVibe.AI's routed vision model answers. The same image can be run
14	through the local reader and up to three AI model hints side by side, with the model that actually answered, its
15	cost and its measured latency. The UI check reads HTML, a public URL or a screenshot on this host and returns
16	overflow, cut-off text, low contrast, overlap and layering faults as JSON. Face tools run locally; identifying matches require separate, purpose-scoped consent.
17
18	## v1 scope (plan §5.8)
19
20	- Ask a picture. Describe, read, identify, count, translate. Upload a file, send a data URL, or paste a public
21	image URL. Every answer reports the engine, model, provider, route, why it was chosen, the cost and the latency.
22	- OCR with a local engine first. Tesseract 5 runs on this host; no image leaves it and no model is called. An
23	AI fallback is opt-in (`fallback=1`) and runs only when the local pass found no usable text.
24	- Compare models side by side. One image, the local reader and one to three AI model hints. AI's router is the
25	authority: `requested_model` is only the hint, `model`/`provider` are what AI says answered. Each sample records
26	cost and latency; `GET /api/v1/benchmarks` aggregates them with no subject or answer text.
27	- The UI check as an API. `POST /api/v1/ui-check` is what Design's Arena will call, with a service token holding
28	`vision.ui.check`. It also works for a signed-in person and, for HTML/URL/screenshot, anonymously.
29	- Consent-scoped faces, no looks in AI text. Face detection and blur are open privacy tools. Matching requires a separate signed consent for each purpose. Public AI text is passed through `noLooks`;
30	an answer that names a person's looks is replaced by a neutral sentence.
31	- Privacy. General answers store no image bytes: a sha256, media type and byte count are stored. Opt-in face uploads, vectors and templates are encrypted and private. Answers, compare samples, UI checks and every face row are exported and erased with the person's account.
32	- Not in v1. Public Arena leaderboards, video, computer-use benchmarks and OpenCV jobs on Run (v2/v3). v1 has no
33	public records, so it writes no Search document and has no outbox.
34
35	## Faces with consent (§5.8b)
36
37	`npm run models:faces` downloads pinned OpenCV Zoo YuNet and SFace weights into `DATA_DIR/models`, checking a SHA-256 before installation. They are not in git. YuNet's directory is MIT licensed; SFace's directory is Apache 2.0 licensed. Open question before production distribution: [SFace weight and training-data provenance](https://github.com/opencv/opencv_zoo/issues/318) needs upstream clarification. The service starts without weights or `VISION_FACE_KEY`; `/api/ready` reports an optional degraded face check. Detect/blur need only YuNet; enrolment, verification and cast matching also need SFace and the key, and answer 503 until they exist. All face inference uses `onnxruntime-node` on this host, never a cloud model.
38
39	The key is `VISION_FACE_KEY` (base64 encoding of 32 random bytes). AES-256-GCM encrypts templates, Vision uploads and face vectors under the person's Network subject. Losing this key makes stored face data unreadable; rotate it only with a re-encryption migration. `VISION_FACE_MATCH_THRESHOLD` defaults to the OpenCV SFace cosine example threshold 0.363 and needs calibration before consequential use. `VISION_FACE_LIBRARY_REGIONS` is a comma-separated allowlist; empty disables library grouping everywhere. `VISION_FACE_PRIVACY_RATE_LIMIT_PER_MIN` limits anonymous detect/blur, alongside the API address limit. No face template enters Search, model prompts or Bench.
40
41	The four signed text versions are exposed by `GET /api/v1/faces/consents/text`. A Network person signs one purpose at a time by sending `signed_name` and the current `text_version` to `POST /api/v1/faces/consents/:feature` (`verify`, `find`, `cast`, `library`). Cast also needs `adult: true` and a named `channel`; library needs an allowed `region`. The signed name and server time are kept in `face_consents`. `DELETE /api/v1/faces/consents/:feature` revokes and immediately deletes that purpose's template; cast revocation also removes past tags. `/people` has the same choices and `/people/privacy` explains face retention. The site's shared legal privacy policy remains at `/privacy`.
42
43	\| Route \| Result and scope \|
44	\|---\|---\|
45	\| `POST /api/v1/faces/detect`, `/blur` \| Anyone; boxes/count or image/png redaction, no templates or tracking. \|
46	\| `POST /api/v1/faces/enrol/:feature` \| Network person only; one face image after that feature's consent (`verify`, `find`, `cast`). \|
47	\| `POST /api/v1/faces/verify` \| Person only; `{score,match,threshold,liveness:"not_checked"}` against their own verify template. A match alone must never unlock anything: use a tested liveness component, second factor and non-biometric route before any consequential use. \|
48	\| `POST /api/v1/faces/media`, `GET /api/v1/faces/media/:id`, `POST /api/v1/faces/find` \| Person only; encrypted own Vision uploads, owner-only links and self-search matches. Media objects are deferred until a source-owner grant and open-time access check exist. \|
49	\| `POST /api/v1/faces/cast/match` \| Live service token with `vision.face.cast.match` and signed `channel_id` claim; compares only that channel's enrolled roster. Requires on-stream notice from Live. Person can inspect their own tags at `GET /api/v1/faces/cast/tags`. \|
50	\| `POST /api/v1/faces/library/cluster`, `GET /api/v1/faces/library/groups`, `POST /api/v1/faces/library/correct` \| Person only, separate opt-in and allowed region. Unknown groups have no label. Corrections: `exclude`, `merge`, `split`; switching off erases groups. \|
51	\| `GET/POST /api/v1/faces/libraries`, `GET/PATCH/DELETE /api/v1/faces/libraries/:id` \| Person only; the owner's own familiar-face library and its camera ids/region. \|
52	\| `POST /api/v1/faces/libraries/:id/enroll`, `/match`, `GET /people`, `PATCH/DELETE /people/:personId` \| Person only; name a crop, match only this library, rename/mute or delete a name. Strict mode answers `pending_consent` with no template until consent. \|
53	\| `POST /api/v1/faces/libraries/:id/people/:personId/request`, `/consent-check`, `POST /api/v1/faces/consent-requests/:token/accept` · `/deny` \| Person only; the private consent link and the registry re-check. \|
54
55	### Familiar faces (F1)
56
57	A camera owner turns on Familiar faces for a camera and names the people that camera sees. `POST /api/v1/faces/libraries` creates the owner's own library (`camera_ids`, `region`), `POST /api/v1/faces/libraries/:id/enroll` names a face from the crop the camera saw, and `POST /api/v1/faces/libraries/:id/match` compares only that library. `DELETE /api/v1/faces/libraries/:id` and `DELETE /api/v1/faces/libraries/:id/people/:personId` delete the templates and the encrypted crops at once.
58
59	The region is the camera's own setting. `VISION_FACE_STRICT_REGIONS` (default `US-IL,US-TX,US-WA,EU,EU-*,GB,UK,UNKNOWN`) puts a library in strict mode: the named person's `face.recognize` choice is checked in Network's consent registry, and until it allows the `library` feature the person is `pending_consent` with no template. The owner can re-issue a private link (`POST …/people/:personId/request`), the person accepts it at `POST /api/v1/faces/consent-requests/:token/accept` (or denies it), and `POST …/consent-check` re-reads the registry for pending people. A name can be muted (still matched, alerts held). A library whose region is outside the strict list (or explicitly `region_mode: "open"`) names a face freely.
60
61	F1 adds `face_libraries`, `face_library_people`, `face_library_crops` and `face_consent_requests` in `migrations/0005_face_libraries.sql`, and `face_templates.library_id`. The templates and crops use the same AES-256-GCM key handling as every other face row; a crop is stored only so a strict-mode template can be made after consent, and a delete removes it. `VISION_FACE_CONSENT_REQUEST_DAYS` (default 14) sets the link lifetime.
62
63	`face_consents`, `face_templates`, `face_media`, `face_tags` and `face_clusters` are created by `migrations/0003_faces.sql`. A daily job (off in `VISION_DRILL`) deletes inactive templates at three years since last use and revoked consent audit rows within one year of purpose end. ADR-033 export and account deletion include all five tables. The account export carries encrypted template and upload ciphertext; it does not expose the server key.
64
65	## What runs where
66
67	\| Step \| Where \| Cost \| Leaves the host? \|
68	\|---\|---\|---\|---\|
69	\| Image decode, rotate, resize, sha256 \| this host, `sharp` \| 0 \| no \|
70	\| OCR \| this host, Tesseract 5 (`OMP_THREAD_LIMIT=1`) \| 0 \| no \|
71	\| UI check (HTML/URL) \| this host, static parser + WCAG maths \| 0 \| the URL fetch only (public HTTP) \|
72	\| UI check (screenshot) \| this host, OCR boxes + `sharp` pixels \| 0 \| no \|
73	\| Face detect, blur and consented matching \| this host, YuNet + SFace + ONNX Runtime \| 0 \| no \|
74	\| Ask / describe / identify / count / translate \| OpenVibe.AI's routed vision model \| AI's own accounting \| the normalised image, to OpenVibe.AI \|
75	\| OCR AI fallback \| OpenVibe.AI's routed vision model \| AI's own accounting \| only after an empty local pass and with consent \|
76	\| Compare \| local + AI hints \| local free; AI's own accounting \| the normalised image, to OpenVibe.AI \|
77
78	Vision holds no provider key, adds no provider and calls no paid provider directly. `ai.run.create` is the only
79	outbound capability; AI's quotas and its own route decide whether a run is allowed.
80
81	## Owns
82
83	\| Table \| Holds \| Migration \|
84	\|---\|---\|---\|
85	\| `vision_answers` \| `vis_` + ULID, subject (`usr_…` or null), requester, task `ask\\|describe\\|ocr\\|compare`, question, mode, source, public `source_url`, image sha256/mime/bytes, cleaned `text`, `answer` jsonb (engine, reason, words, samples, run id), model/provider/route, cost, latency \| `migrations/0001_initial.sql` \|
86	\| `vision_compare_samples` \| `vcm_` + ULID, answer, subject, task, engine `local\\|ai`, requested model, answered model/provider/route, ok, text, cost, latency, error code \| `0001_initial.sql` \|
87	\| `vision_ui_checks` \| `vuc_` + ULID, subject, requester, source `html\\|url\\|screenshot`, URL, image sha256, viewport, engine `static-html\\|screenshot`, findings jsonb, summary jsonb \| `0001_initial.sql` \|
88	\| `account_data_events` \| openvibe-sdk/account-data's export/deletion receipts \| `0001_initial.sql` \|
89	\| `telemetry_samples` \| aggregated request telemetry, pruned after 14 days \| `0001_initial.sql` \|
90	\| `face_consents` \| subject, feature `verify\\|find\\|cast\\|library`, channel scope, signed text version/name, granted_at, revoked_at, last_used_at, purpose_ends_at \| `migrations/0003_faces.sql` \|
91	\| `face_templates` \| subject, feature, scope, AES-GCM ciphertext of the SFace vector, created/last_used times \| `0003_faces.sql` \|
92	\| `face_media` \| owner, AES-GCM ciphertext of an opt-in Vision upload and its face vectors, media type \| `0003_faces.sql` \|
93	\| `face_tags` \| consenting subject, channel, private media reference and time; removed on cast revocation \| `0003_faces.sql` \|
94	\| `face_clusters` \| owner, media, encrypted vector, group id, optional label (null while unknown), excluded flag \| `0003_faces.sql` \|
95	\| `event_outbox` \| the events Vision publishes wait here until OpenVibe.Events takes them: the envelope, its own event id (the idempotency key), the attempts and the last error; sent rows are pruned after 7 days \| `migrations/0004_event_outbox.sql` \|
96
97	General answer tables store no image bytes, token or provider key. The opt-in face library stores encrypted uploads and vectors.
98
99	## Does not own
100
101	- Models, routing, quotas and provider keys — OpenVibe.AI. Vision makes first-party `ai.run.create` runs with
102	namespace `ai.*`; AI decides the provider/model and reports the provenance.
103	- Identity, roles and accounts — OpenVibe.Network. Vision verifies session and service tokens against its JWKS.
104	- The local OCR binary and language data — the host (Tesseract under `/usr/bin` and `/usr/share`).
105	- Search — OpenVibe.Search. v1 has no public records to index.
106	- Events, the host and the database — OpenVibe.Events, OpenVibe.Host.
107
108	## Capabilities and grants
109
110	Contracts registers Vision's capabilities; the review routes add three more. The service checks each machine token's grant on its route;
111	private media and face identity operations also require the verified owner or the person's purpose-scoped consent.
112
113	\| Capability \| Routes \| Typical caller \|
114	\|---\|---\|---\|
115	\| `vision.image.ask` \| `POST /api/v1/ask`, `GET /api/v1/answers/:id` \| a first-party service or a person \|
116	\| `vision.text.read` \| `POST /api/v1/ocr` (the AI fallback also needs a person or this grant) \| a first-party service or a person \|
117	\| `vision.model.compare` \| `POST /api/v1/compare` \| a first-party service or a person \|
118	\| `vision.ui.check` \| `POST /api/v1/ui-check`, `GET /api/v1/ui-checks/:id` \| OpenVibe.Design's Arena, or a person \|
119	\| `vision.face.cast.match` \| `POST /api/v1/faces/cast/match` (a signed `channel_id` claim must match the request) \| OpenVibe.Live, or a person inspecting their own tags \|
120	\| `vision.alt.write` \| `POST /api/v1/alt` \| Pics, Media and Chat \|
121	\| `vision.media.find` \| `POST /api/v1/index`, `GET`/`POST /api/v1/find` (owner user token) \| Media and Search \|
122	\| `vision.receipt.read` \| `POST /api/v1/receipt` \| Food \|
123	\| `vision.product.match` \| `POST /api/v1/product` \| Deals \|
124	\| `vision.camera.watch` \| `POST /api/v1/watch` (owner user token) \| Cam \|
125	\| `vision.video.moments` \| `POST /api/v1/moments` (owner user token) \| Live and Media \|
126	\| `vision.face.detect`, `vision.face.blur` \| `POST /api/v1/faces/detect`, `/blur` \| Public privacy tools; a service token needs the matching grant \|
127	\| `vision.face.consent.manage`, `vision.face.verify.self`, `vision.face.find.self`, `vision.face.library`, `vision.face.cast.read` \| `/api/v1/faces/*` \| The person themselves, with separate signed consent for each matching purpose \|
128	\| `vision.models.read`, `vision.benchmarks.read` \| `GET /api/v1/models`, `/benchmarks` \| Public reads; a service token needs the matching grant \|
129	\| `vision.image.review` \| `POST /api/v1/review` \| Media, Live and Community (the unsafe-image review) \|
130	\| `vision.review.read`, `vision.review.decide` \| `GET /api/v1/review`, `GET /api/v1/review/:id`, `POST /api/v1/review/:id/decide`, `GET /api/v1/review/reports` \| OpenVibe staff, and an operator's own service token \|
131
132	Outgoing: `ai.run.create` to OpenVibe.AI (audience `openvibe.ai`, namespace `ai.*`), minted by Network for the
133	client `vision`. No new AI capability, no provider key, no direct paid provider.
134
135	## MCP and events
136
137	`POST /mcp` serves Streamable HTTP JSON-RPC 2.0 (`initialize`, `ping`, `tools/list`, `tools/call`).
138	`/.well-known/oauth-protected-resource` advertises its Network authorization server; a private tool without a
139	token responds with a `WWW-Authenticate` challenge pointing to that metadata. The tools are `vision.ask_image`,
140	`vision.read_text`, `vision.translate_sign`, `vision.find_similar`, `vision.search_my_media`,
141	`vision.describe_clip`, `vision.detect_objects`, `vision.blur_faces`, `vision.check_ui`,
142	`vision.compare_models` and `vision.get_job`, and five that run one prompted task each through the routed model
143	(VIS-6; also `task=` on `POST /api/v1/ask`, no question needed): `vision.meal` (`meal`: foods, ingredients, portions,
144	rough calories), `vision.describe_surroundings` (`surroundings`), `vision.identify_plant` (`plant`),
145	`vision.identify_landmark` (`landmark`) and `vision.screenshot_components` (`components`). Face enrolment, verification, cast matching and consent mutation
146	are available only through their person or Live-specific API routes, never through MCP.
147
148	When configured with Events and a Network client secret, successful owner results publish `vision.result.ready`
149	with provenance, cost and retention, without image or answer text. A watch result with `source_ref` and detections
150	publishes `vision.alert` with classes and confidence. Publishing failures are logged without exposing payloads.
151
152	## API
153
154	JSON under `/api/v1`; errors are `application/problem+json` with a `code`. An image is a multipart `image` field, a
155	JSON `image` data URL, or a JSON `image_url` public URL. A person is the `vision_at` session cookie or a Network user
156	token as a Bearer. A session write must come from `https://openvibe.vision` (403 `vision.cross_site`). A service token
157	must hold the route's capability (403 `capability.denied`).
158
159	\| Route \| What it does \|
160	\|---\|---\|
161	\| `POST /api/v1/ask` (alias `POST /api/v1/describe`) \| `{ task?, question, mode?, lang?, model?, image_url? }` or multipart `image`; returns `{ answer }`. A read question is local-first, otherwise AI. \|
162	\| `POST /api/v1/ocr` \| multipart/data URL/URL; `{ fallback?, lang?, model? }`; returns the OCR row with `text`, `answer.confidence`, `answer.words`, cost 0. \|
163	\| `POST /api/v1/compare` \| `{ task?, question?, models?: string[] }` plus an image; returns `{ answer, compare }`, one row per engine. \|
164	\| `POST /api/v1/ui-check` \| `{ html }`, `{ url }`, or a screenshot plus `{ viewport: { w, h } }`; returns `{ check }` with `findings`, `counts`, `severity`, `warnings`. \|
165	\| `GET /api/v1/answers/:id` \| one saved answer: its owner, or a service holding `vision.image.ask`. \|
166	\| `GET /api/v1/ui-checks/:id` \| one saved UI check: its owner, or a service holding `vision.ui.check`. \|
167	\| `GET /api/v1/models` \| AI's offered `provider:model` hints and the local engine. No subject data. \|
168	\| `GET /api/v1/benchmarks?task=&limit=` \| aggregate compare samples: task, engine, provider, model, samples, mean latency, mean cost. No subject or answer text. \|
169	\| `GET /api/v1/ping`, `/api/health`, `/api/ready`, `/release.json`, `/metrics` \| service endpoints; readiness and metrics are truthful, metrics loopback-only. \|
170	\| `POST /api/v1/review` \| `{ image_url }`, an image, or `{ text }` plus `{ object_type, object_id, owner_subject?, preview_url? }`; returns `{ category, plain_words, action, public, status, review, report? }`. Exactly one of `fine`, `sex_acts`, `minor_sexual`. \|
171	\| `GET /api/v1/review?category=&status=` \| the operator's queue (staff or `vision.review.read`); `GET /api/v1/review/:id` is the owner's or staff's single row. \|
172	\| `POST /api/v1/review/:id/decide` \| staff or `vision.review.decide`: `{ category }` must be one of the same three. \|
173	\| `POST /api/v1/review/:id/reveal` \| staff only: one 60-second Media operator view, logged on the report. \|
174	\| `GET /api/v1/review/reports?status=` \| the operator report rows made by category 3. \|
175
176	Example — the call Design's Arena makes:
177
178	```sh
179	curl -s -X POST https://openvibe.vision/api/v1/ui-check \
180	-H "Authorization: Bearer $VISION_SERVICE_TOKEN" \
181	-H 'Content-Type: application/json' \
182	-d '{"html":"<!doctype html><style>.card{width:1400px}</style><div class=card>x</div>","viewport":{"w":390,"h":844}}'
183	```
184
185	Answer (abridged):
186
187	```json
188	{ "check": {
189	"engine": "static-html",
190	"viewport": { "w": 390, "h": 844 },
191	"counts": { "overflow": 1, "contrast": 1 },
192	"severity": { "high": 1, "medium": 1 },
193	"findings": [
194	{ "kind": "overflow", "severity": "medium", "selector": "div.card", "detail": "the box is 1400px wide and reaches 1400px in a 390px viewport", "evidence": { "width": 1400, "right": 1400, "viewport": 390 } }
195	]
196	}}
197	```
198
199	Validation refusals: `vision.invalid`, `vision.invalid_task`, `vision.invalid_mode`, `vision.question_required`,
200	`vision.image_required`, `vision.image_too_large`, `vision.image_private_url`, `vision.image_unsupported`,
201	`vision.text.read_unavailable`, `vision.ai_unavailable`, `vision.not_found`, `vision.rate_limited`, and the face codes
202	`faces.consent_required`, `faces.signature_required`, `faces.adult_required`, `faces.region_disabled`,
203	`faces.enrolment_required`, `faces.one_face_required`, `faces.model_unavailable`, `faces.key_unavailable`,
204	`faces.live_grant_required`.
205
206	## Unsafe-image review (VIS-6)
207
208	An image review sorts into exactly three categories, in the owner's words:
209
210	\| Category \| Plain words \| Action \|
211	\|---\|---\|---\|
212	\| `fine` \| Fine — OK to show, including nudity \| `allow`; nothing changes \|
213	\| `sex_acts` \| Sex acts or porn — never public; kept for its owner \| `hide`; every public surface is closed and the object stays with its owner \|
214	\| `minor_sexual` \| Sexual content involving a minor — always blocked and reported to the operator \| `quarantine_and_report`; the object is blocked, a report row is made for the operator, and nothing is sent outside OpenVibe \|
215
216	`POST /api/v1/review` accepts an image (multipart `image`, data URL, or `image_url`) or text. Text is classified by the
217	same rule Live's auto-clip guard uses: nudity alone is fine; sex acts and porn are not; a child with nudity or a sex
218	act is the hard line. An image is classified by OpenVibe.AI with one strict JSON question; the text answer can make
219	the result more severe but never less. Category 3 always creates one `vision_operator_reports` row. The staff pages
220	`/review`, `/review/:id` and `/review/reports` show the queue and let an operator decide one of the same three
221	categories; a decision writes `reviewed_by`/`reviewed_at` and dismisses a report the operator overrules. A decision on
222	a Media object also calls Media's `review-decision` route, which releases or places Media's evidence hold.
223
224	An image the reviewer cannot judge is never cleared by harmless text: if the image pass is unavailable or returns no
225	usable JSON, text may answer only when it is already `sex_acts` or `minor_sexual`; otherwise the answer is a 503 and
226	the caller keeps the object pending. `preview_url` must be `https` on `openvibe.vision`, `openvibe.network` or
227	`openvibe.media` (the origins its CSP already trusts); anything else is a 400. The page escapes it in `src` and `href`.
228
229	Category 2 and 3 pictures are withheld from the operator by default. The queue shows no picture for category 3 and
230	blurs a category-2 thumbnail; the detail page shows a blurred or withheld placeholder, not the bytes. "Show the
231	picture" asks Media for a 60-second `media.object.review.view` URL (a separate signature purpose that opens a
232	quarantine), logs the operator and time in `vision_review_reveals`, and shows that one picture. The warning on the
233	page says the picture is evidence for the report and must not be saved or shared. Child-safety hash matching
234	(Cloudflare's CSAM Scanning Tool, NCMEC lists) is a separate service and not this language model.
235
236	No image bytes are stored. The review row keeps the caller's object reference (`object_app`, and the Media
237	`media_object_id`/`media_app` when there is one), a sha256, the category, the plain words, the action, the model
238	provenance and an optional first-party preview URL. A review does not publish a `vision.result.ready` event; the
239	caller already has the row.
240
241	## Pages
242
243	All server-rendered through the OpenVibe Frame (`openvibe-shared/shell`), readable without JavaScript, semantic HTML
244	with stable `vi-` ids/classes. Public pages are `public, max-age=60` with `Vary: Cookie`; a page drawn for a signed-in
245	person is `private, no-store`.
246
247	\| Page \| What it is \|
248	\|---\|---\|
249	\| `/` \| the room: hero, ask, compare and UI-check scenes, and a try section \|
250	\| `/ask` \| upload or URL + a question (POST → `/a/:id`) \|
251	\| `/ocr` \| local OCR, with an inline reading for a guest and a saved answer for a person \|
252	\| `/compare` \| local OCR vs AI model hints (POST → `/a/:id`) \|
253	\| `/ui-check` \| HTML, URL or screenshot; inline for a guest, saved at `/ui/:id` for a person \|
254	\| `/a/:id` \| one saved answer (owner or staff) \|
255	\| `/ui/:id` \| one saved UI check (owner or staff) \|
256	\| `/find` \| private visual search over a person's own indexed images \|
257	\| `/read` \| the receipt reader \|
258	\| `/watch` \| watched objects and scene change in a camera still \|
259	\| `/people` \| the four face purposes, each with its own signed switch, enrolment and off control \|
260	\| `/people/privacy` \| the face notice: retention, erasure, liveness and what never happens \|
261	\| `/privacy` \| the shared legal privacy policy \|
262	\| `/developers` \| the API, capabilities and what leaves the host \|
263	\| `/review` \| the unsafe-image queue, staff only: three category tabs and the report count \|
264	\| `/review/:id` \| one review: the image, its provenance, the three plain choices and the operator report \|
265	\| `/review/reports` \| the operator report rows (open, resolved, dismissed) \|
266	\| 404 \| the not-found page \|
267
268	`public/css/vision.css` and `public/js/vision.js` are one-comment placeholder files: the pages work without them and
269	the main session designs them. `robots.txt`, `sitemap.xml`, `llms.txt` and `llms-full.txt` list only the public
270	pages; `/a/`, `/ui/`, `/auth/`, `/api/` and `/internal` are not indexed.
271
272	## The v1 proof
273
274	`node scripts/proof.js` is offline: it renders a receipt, reads it with local Tesseract, and checks a page with a
275	planted 1400 px overflow, a clipped nowrap line and low contrast. It exits non-zero if the local reader misses the
276	total or the UI check misses a fault. The tests do the same over the API, and `test/compare.test.js` runs two AI
277	hints and the local reader on one image with measured cost and latency.
278
279	## Run and test
280
281	```sh
282	npm install
283	npm test # PGlite; every file in its own process, no network needed
284	npm run test:pg # the same through PostgreSQL + PgBouncer (openvibe-sdk scripts/test-services.sh up)
285	OV_ALLOW_SUITE=1 npm test
286	npm start # PORT 5130, loopback; DATABASE_URL or an embedded PGlite in development
287	node scripts/proof.js # local OCR + a planted UI overflow, no network and no model
288	```
289
290	The tests use a mock Network, a mock OpenVibe.AI and local image/page fixtures: migration, SSO and readiness, the
291	API and its capability guards, local OCR and its fallback, compare and benchmark aggregation, the UI check (a
292	planted overflow, cut-off line, low contrast, overlap and layering fault), noLooks, pages/cache, account
293	export/deletion, the image SSRF guard, and faces with a fake ONNX model (all four consent gates, opt-out erasure,
294	cast-roster-only matching, AES-GCM at rest, the retention job, the region gate and ADR-033 export/deletion).
295
296	## Deploy
297
298	`deploy/systemd/openvibe-vision.service` (127.0.0.1:5130, `/etc/openvibe/vision.env`) and
299	`deploy/nginx/openvibe.vision.conf` (TLS, www → apex, `/auth/me` on the API zone, a forms zone for `/ask`, `/ocr`,
300	`/compare` and `/ui-check`, `/internal` and `/metrics` 404). Not deployed by this change.
301
302	## License
303
304	AGPL-3.0-only. See [LICENSE](LICENSE).
305
306	## Network tools, wave 1
307
308	All six tools are JSON APIs. Successful `answer` objects carry a `receipt` with engine, model, `cost_usd`, `ms`, confidence and retention. An image is multipart `image` or JSON `image` data URL unless a route says otherwise. Vision does not retain image bytes.
309
310	\| Room call \| Capability \| Notes \|
311	\|---\|---\|---\|
312	\| Pics, Media, Chat → `POST /api/v1/alt` \| `vision.alt.write` \| Routed VLM through OpenVibe.AI; short editable `alt_text`; noLooks in the prompt and output. \|
313	\| Media, Search → `POST /api/v1/index`, `GET /api/v1/find?q=`, `POST /api/v1/find` \| `vision.media.find` \| Owner's verified user token; index `{item_id,image,result_url?}`; search by text or image. Owner is in every index read predicate. \|
314	\| Food → `POST /api/v1/receipt` \| `vision.receipt.read` \| Tesseract first; `fallback=true` permits routed VLM if local extraction is incomplete. \|
315	\| Deals → `POST /api/v1/product` \| `vision.product.match` \| CLIP vector and routed product description; `allow_rerank=true` permits another routed VLM call to rerank candidate offers. \|
316	\| Cam → `POST /api/v1/watch` \| `vision.camera.watch` \| Owner's verified user token; upload still and `classes` array; optional `previous_image` data URL for scene change. \|
317	\| Live, Media → `POST /api/v1/moments` \| `vision.video.moments` \| Owner's verified user token; `{frames:[{timestamp_ms,image\|image_url}],prompts?}`. Scene histogram and CLIP prompt matching. \|
318
319	For private index, watch and moments, a service capability alone does not assert a source owner's grant. The room should call with the owner's delegated Network user token. Private index and watch reject `image_url`; the owner must supply bytes or a data URL. Moments accepts public frame URLs under Vision's existing guarded image fetch, or frame data URLs. The room that opens a returned `result_url` must check its own source grant again.
320
321	`npm run models:fetch` is an explicit, revision-pinned download into `${DATA_DIR:-data}/models`. It fetches quantized ONNX CLIP ViT-B/32 (`Xenova/clip-vit-base-patch32`, derived from OpenAI CLIP) and DETR ResNet-50 (`Xenova/detr-resnet-50`, derived from Facebook DETR). The local model loader never downloads weights during a request. Missing weights return 503 for model-dependent operations; histogram-only scene change remains available. CLIP code and model are listed as MIT, but the original checkpoint's weight licence has an [open clarification request](https://huggingface.co/openai/clip-vit-base-patch32/discussions/72); confirm this before commercial distribution. DETR model and weights are [Apache 2.0](https://huggingface.co/facebook/detr-resnet-50). We use DETR rather than Ultralytics (AGPL) or RF-DETR Nano because a supported local Transformers.js ONNX export exists for DETR. `@huggingface/transformers` is Apache 2.0. Tesseract is Apache 2.0.
322
323	Private embeddings are JSON vectors in PostgreSQL, scored in JavaScript after a mandatory `owner` filter. Vision has no pgvector migration or extension requirement. The v1 search cap is 5,000 items per owner.
324
325	Offer discovery is configured by `VISION_DEALS_SEARCH_URL` and `VISION_SHOES_SEARCH_URL`, both fixed operator URLs returning JSON `{offers:[{title,url,price?}]}` or `{results:[...]}`. Deals currently exposes public `GET /api/v1/offers` without a query filter, and its `/search?q=` is an HTML page; open.shoes has no implemented API in this workspace yet. Until those rooms expose a structured search endpoint, `/product` returns the description and embedding with an empty `offers` list and a warning. It does not fabricate matches.
326
327	<!-- versions:start -->
328	- openvibe-contracts: v0.419.0
329	- openvibe-sdk: v0.161.0
330	- openvibe-shared: v4.241.0
331	<!-- versions:end -->
332