openvibers/openvibe.run AGENTS.md

AGENTS.md

main 12.1 KB · 161 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.Run
2
3## Project Overview
4
5OpenVibe.Run is the network's job service (plan T14, ADR-034): it accepts jobs, places each on a node advertising
6the runtime class it needs, follows it to its end and streams its output. It also places **sessions** — browser desks
7one person owns and OpenVibe.Bot leases — and it carries the operator's compute-pool view. A job belongs to a
8**project**, is billed per wall-clock second it runs, and is visible only to the token that project presents.
9
10- **Site:** `openvibe.run` · **Port:** 4920 · **Service id:** `run` · **Env prefix:** `RUN`
11- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development/tests), Valkey 9.
12- **Read first:** [README.md](README.md), [STATUS.json](STATUS.json), [docs/compute-broker.md](docs/compute-broker.md),
13 [.env.example](.env.example).
14
15**Network context:** identity, tokens, projects/offers and project membership checks are OpenVibe.Network. OpenVibe.Bot
16owns the node link, sends jobs to nodes and meters their wall-clock seconds into `platform.usage-sample@1`; Run never
17posts those readings. Run mints the job id (`job_<ULID>`, passed to Bot as `platform.job@1`, so metering keys are
18`run:<id>:<n>`) and per-second usage for the external managed-browser provider only (Billing
19`billing.usage.record`, clamped). Events are OpenVibe.Events. Media owns job artifacts and inputs; Run never fetches a
20caller-chosen URL.
21
22## Commands
23
24```bash
25npm ci # install (contracts/sdk/shared pinned by tag)
26npm run dev # http://localhost:4920, PGlite under data/
27npm start # node server/index.js
28npm test # every test/*.test.js on temp PGlite with mocks, no network
29npm run test:pg # the same through PostgreSQL + PgBouncer
30npm test -- sessions # only test files whose name contains the word
31node --check <file.js> # syntax check (no linter configured)
32```
33
34No build step. Run serves mostly machine routes; the public home in [server/web/home.js](server/web/home.js) and
35[server/web/discovery.js](server/web/discovery.js) exists and is tested, but `openvibe.run` is parked (302 to
36openvibe.network) until Run has a product surface — the vhost is installed then.
37
38## Architecture at a Glance
39
40- **Entry:** [server/index.js](server/index.js) — migrations with the owner role and an advisory lock, then the
41 listener on the pooled role; starts the key refresher, stream key, events outbox relay and dispatcher poller, and
42 stops them in order. `RUN_JOBS=off` disables the background work. Nothing starts at module load.
43- **App factory:** [server/app.js](server/app.js) — helmet/CSP, `/api/v1`, the compute operator routes, `/api/health`,
44 `/api/ready`, `/release.json`, `/metrics` (direct loopback only), the public home/discovery, 404/error handler.
45- **Config:** [server/config.js](server/config.js) — `loadConfig(env)` is pure; production `/etc/openvibe/run.env`.
46- **Job API:** [server/api/v1.js](server/api/v1.js), auth/capability resolution [server/api/auth.js](server/api/auth.js),
47 job store [server/jobs/store.js](server/jobs/store.js), events [server/jobs/events.js](server/jobs/events.js),
48 stream ring/tickets [server/jobs/stream.js](server/jobs/stream.js), [server/jobs/ticket.js](server/jobs/ticket.js),
49 usage [server/jobs/usage.js](server/jobs/usage.js), poller [server/jobs/poller.js](server/jobs/poller.js).
50- **Placement/dispatch:** [server/dispatch/](server/dispatch/) — `placement.js` states requirements and calls
51 `openvibe-sdk/placement plan()` (Run has no placer of its own, ADR-036), `bot.js` sends/mirrors via
52 `bot.job.dispatch`, `browser-provider.js`, `outcomes.js`, `priors.js`.
53- **Sessions:** [server/sessions/](server/sessions/) — `service.js`, `store.js`, `desks.js`, `curtain.js`, `profiles.js`,
54 `origins.js`, `link-egress.js`; routes [server/api/sessions.js](server/api/sessions.js). A session is a person's own
55 desk placed by Run and leased by Bot (`bot.desk.session`), metered once a minute.
56- **Compute broker:** [server/compute/](server/compute/) — `broker.js`, `config.js`, `settings.js`,
57 `sandbox-providers.js`, routes [server/compute/routes.js](server/compute/routes.js); rentals default to quote-only,
58 capped per provider, and a provider API key stays in Run's environment only.
59- **Events outbox:** [server/events/outbox.js](server/events/outbox.js), [server/jobs/events.js](server/jobs/events.js);
60 account data [server/account-data.js](server/account-data.js).
61- **Network client:** [server/network.js](server/network.js) — offers and project-membership checks.
62- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/): `run_jobs`, `run_events_outbox`,
63 `run_usage_outbox`, `run_sessions`, `run_session_actions`, `run_session_control_guests`,
64 `run_session_fill_requests`, `run_actor_profiles`, compute pools/rentals/journal/charges/credits/outcomes,
65 `config_keys`, `config_snapshots`, `account_data_events`, `telemetry_samples`.
66- **Deploy:** [deploy/systemd/openvibe-run.service](deploy/systemd/openvibe-run.service) (`/etc/openvibe/run.env`,
67 `/opt/openvibe.run`, release layout, database `ov_run`) and [deploy/nginx/openvibe.run.conf](deploy/nginx/openvibe.run.conf)
68 (TLS, www→apex, rate limits, client IP from `$remote_addr` only, `/internal/` and `/metrics` never proxied).
69 `sudo ovhost deploy run`.
70
71## Ownership and Boundaries
72
73Run owns the job lifecycle record and state/timings, project scope and delegation, idempotency, the job stream and its
74tickets, requirements/project caps, placement choice, sessions/desks records and control actions, the compute broker's
75pools/rentals/journal, the events/usage outboxes and the public home copy. It does **not** own the node link, device
76credentials or per-second metering of Bot-carried jobs (Bot), identity/tokens/projects (Network), artifacts/media
77(Media), the money ledger (Billing), or event delivery (Events). Run never estimates a usage number it cannot back.
78
79## Auth and Capabilities
80
81One capability per route, from the caller's Network service token; a Network **user** token names no project and holds
82no capability here. `run.job.admin` is internal: only a first-party `svc:` token may hold it. Run verifies only tokens
83its audience names. Delegation (ADR-003 amendment): a first-party service with `X-OV-Subject` may name `X-OV-Project`;
84Run asks Network (`network.project.member.read`, cached 60 s) whether the person is admin/owner of the unarchived
85project, records the service as `via`, and limits the service to the jobs it started for that person. Another
86project's job is **404, never 403**; undelegated list/Bearer-stream is 403 `project.not_delegated`; Network
87unreachable is 503 `project.check_unavailable`.
88
89Capabilities (STATUS.json): `run.job.submit|read|list|cancel|stream|admin`; `run.session.create|read|control`.
90Outbound: `bot.job.dispatch`/`bot.desk.session` (Bot), `network.project.member.read`/`network.registry.read`,
91`events.event.publish`, `billing.usage.record` (external-browser jobs only). Contract validation is applied to every
92request and every answer; errors are RFC 9457 problem+json.
93
94## API
95
96Jobs: `POST /api/v1/jobs` (201/200 idempotent), `GET /jobs/:id`, `GET /jobs`, `POST /jobs/:id/cancel`,
97`POST /jobs/:id/stream/ticket`, `GET /jobs/:id/stream?ticket=…` (SSE; Bearer with `run.job.stream` also allowed),
98`GET /admin/jobs`. Sessions: `POST /api/v1/sessions`, `GET /sessions`, `GET /sessions/:id`,
99`POST /sessions/:id/commands`, `POST /sessions/:id/end`. Compute (operator, signed-in user in
100`RUN_COMPUTE_OPERATORS`/owner, same-origin writes): `/api/v1/compute`, `/compute/pools/:pool/quotes`,
101`POST /compute/pools/:pool/strategy`, `POST /compute/rentals/:id/bind`. Machine: `/api/health`, `/api/ready`,
102`/release.json`, `/metrics` (direct loopback only).
103
104## Conventions
105
106- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter.
107- New schema = a new `migrations/NNNN_name.sql`; never edit one that has run. Keep `run_` prefixes.
108- Every request and answer goes through the released contract (`run.job-*@1`, `run.session*@1`,
109 `platform.job@1`); an unknown field in a contract that forbids extras is a 400, never dropped silently.
110- **Idempotency is permanent:** one row per `(project_id, idempotency_key)`; a repeat answers the first job, the same
111 key with different work is 409 forever (the contract's 24 h is a floor, not an expiry). A missing key is minted
112 `auto:<ulid>` and returned.
113- **States** are `queued → placed → running → succeeded|failed|cancelled|expired`; an end state never changes. A queued
114 cancel is immediate; a placed/running cancel marks `cancel_requested`, asks Bot once per job and reaches `cancelled`
115 only when Bot says so. Run never answers `cancelled` for work a node may still be doing.
116- The stream ticket is a two-minute single-use RS256 token with audience `openvibe.run` and its own key — never
117 Network's signing key. `Last-Event-ID` resumes after a `seq`; output is best effort (a chunk lost with the node's
118 link is not resent).
119- Sessions: `ttl_s` (3600 default), `idle_s` (900), a lease check every `RUN_SESSION_LEASE_CHECK_MS` (30 s), at most
120 `RUN_SESSIONS_PER_PERSON` (3); a dropped lease ends `host_lost`; no host waits `placing` then `unplaceable`.
121- CSP: add origins to `helmet` in [server/app.js](server/app.js). Phone first for the home/compute pages; no Font
122 Awesome.
123- Never log a token, a stream ticket, a provider key or a person's args. "Calm reduces, only Off removes": a filter
124 hides, only `end`/`cancel`/`delete` removes.
125
126## Key Pitfalls
127
128- **Project scope comes from the token**, never a request body: `project_id`/`requester` are `claims.project_id`/
129 `claims.sub`. A user token cannot act here without a service grant.
130- **A job's args are limited** (512 KiB → 413 `run.args_too_large`), and `limits` are capped by the project's with
131 422 `run.limits.exceeded`. `ttl_ms` defaults to `limits.wall_ms + 600000`.
132- **No node offer:** keep the job queued until an offer appears or `ttl_ms` expires it (`run.job.unplaceable`); a
133 Network that does not answer is a retry, never a job failure. A user-owned offer is eligible only for that person's
134 own job naming `user-owned`, read from Network's internal `trust=user-owned` list — never from the public list.
135- **Metering:** Bot's wall clock is the only usage for Bot-carried jobs; a running job shows 0 until `job_exit`. The
136 browser provider's claim is clamped to Run's observed duration + 5 s. Never invent a number.
137- **Sessions are person-owned:** a session may be created only for the caller (or a first-party `svc:` naming them in
138 `X-OV-Subject`); control commands are limited to screen/input/browser and must respect the curtain.
139- **Compute rentals default to quote-only.** No key, cap or provider is inferred from credentials; a provider key never
140 enters an offer, a job, a log or a public page.
141- **Events:** the outbox row is keyed `run:<job id>:<state>` (one per job/state); the envelope carries a contract-valid
142 `evt_<ULID>`. A replay is one row.
143- **Account data** arrives at `/internal/events` (loopback, signature-verified, idempotent).
144
145## Testing
146
147`npm test` runs each file in its own process on temp PGlite with mocked Network/Bot/provider: jobs/API/placement/
148dispatch/delegation, sessions and control, compute/pools/policy/scale/settings, account data, agent steps, site/
149discovery, SSO, telemetry, link egress and security. `npm run test:pg` for PostgreSQL. Run the narrowest file first,
150then the suite; always `node --check` changed files.
151
152## Documentation
153
154- [README.md](README.md) — purpose, how a job runs, owns/does-not-own, dependencies, capabilities/routes, sessions,
155 jobs, tests, security, deploy, deploy files.
156- [docs/compute-broker.md](docs/compute-broker.md) — compute placement, pool autoscaling, rental caps, operator view.
157- [docs/compute-providers-1111.md](docs/compute-providers-1111.md), [docs/link-egress.md](docs/link-egress.md),
158 [docs/cutover-pr-openvibe-run-40.md](docs/cutover-pr-openvibe-run-40.md),
159 [docs/cutover-pr-openvibe-run-41.md](docs/cutover-pr-openvibe-run-41.md).
160- [STATUS.json](STATUS.json), [.env.example](.env.example), [CHANGELOG.md](CHANGELOG.md), [SECURITY.md](SECURITY.md).
161