openvibers/openvibe.run Commits
Run: the race play and the Arena's blinded race endpoints (#278)
Changes
@@@ -109,6 +109,19 @@ OV_OAUTH_CLIENT_SECRET=#RUN_BILLING_INTERVAL_MS=5000#RUN_BILLING_TIMEOUT_MS=8000++# ── The Arena's race (compare-arena-ecosystem.md §3) ──++# POST /api/v1/races starts one blind child job per entrant; the judgement is a bench.match@1 recorded on++# Bench (POST /api/v1/matches) with Run's Network client token. That client needs bench.judgement.cast.++# Bench has no matchmaker endpoint yet, so matchmaker: true picks two entrants of different model families++# from this list (comma-separated bench.entrant@1 ids). Unset: matchmaker: true answers 422.++#RUN_RACE_ENTRANTS=model:gpt-5-mini,model:glm-5.3,model:deepseek-flash++# The key for the keyed hash of the person in a judgement (never the person's id). Unset: Run uses its++# Network client secret; set a dedicated key to rotate it without touching the Network client.++#RUN_RACE_HASH_KEY=++#RUN_BENCH_URL=https://bench.openvibe.services++#RUN_BENCH_AUDIENCE=openvibe.bench++#RUN_BENCH_TIMEOUT_MS=8000++# ── IndexNow (search engines are told when the public pages change) ──# 8-128 letters and digits. Unset: nothing is sent and GET /<key>.txt is 404. When set, the key file is served there.#INDEXNOW_KEY=@@@ -6,6 +6,17 @@ Notable changes to OpenVibe.Run. The service is unreleased; everything below is### Added++- The Arena's race on the one engine (compare-arena-ecosystem.md §3, contracts 0.422.0): the `race` play as++ `agents.play@1` (one role per side, one parallel phase, an even budget split, `blind: true`, stop on person), listed++ by `GET /api/v1/plays`; `POST /api/v1/races` starts one child `linux` job per side (2–4) under one parent race with++ the same share of the budget each and answers the sides labelled `a, b, …` without names; `GET /api/v1/races/:id`++ returns each side's output so far and its receipts when ended, the names only after a judgement;++ `POST /api/v1/races/:id/judgement` records one `bench.match@1` on Bench (source `arena`, shown order, `blind: true`,++ the receipts, a `human_pairwise` judgement with an HMAC of the person) before revealing the names. Only the person++ who started the race, through the same first-party service, may read or judge it. `matchmaker: true` picks two++ entrants of different model families from `RUN_RACE_ENTRANTS` while Bench has no matchmaker endpoint. Migration++ 0020 adds `run_races`. The race request/answer has no released contract yet, so it is validated in++ `server/races/service.js`; everything it stores or sends is validated where a contract exists.- With `RUN_SESSION_ORIGINS_ENABLED=1`, a browser session may name exact HTTP(S) origins at creation. Run stores thelist with the session, places it on a user-owned desk that can enforce the boundary, and sends the same list whenleasing that desk. Commands cannot replace the list. The switch is off by default.@@@ -98,6 +98,9 @@ placed from the Fabric's offers and actually runs — the poller in `server/jobs(`GET /api/v1/offers`, public); Run's client `run` mints the events relay's service token, its Bot token and thetoken for project memberships (`GET /internal/projects/:project/members/:subject`, project delegation).- OpenVibe.Events: `run.job.*` events go to `events.event.publish`; unset, they wait in `run_events_outbox`.++- OpenVibe.Bench (`RUN_BENCH_URL`, default `https://bench.openvibe.services`): a race's judgement is recorded as a++ `bench.match@1` on `POST /api/v1/matches` (`bench.judgement.cast`, audience `openvibe.bench`). Bench has no++ matchmaker endpoint yet; `matchmaker: true` falls back to `RUN_RACE_ENTRANTS`.- OpenVibe.Bot (`RUN_BOT_URL`, `RUN_POLL_MS`): the node link and the meter. Run sends each job to it(`bot.job.dispatch`, audience `openvibe.bot`, minted from `OV_OAUTH_CLIENT_ID`/`OV_OAUTH_CLIENT_SECRET` against`OV_NETWORK_INTERNAL_URL` and cached until 60 s before expiry) and mirrors its answer. Unset the client secret and@@@ -206,6 +209,39 @@ first-party `svc:` token holding the capability and naming them in `X-OV-Subjectqueued until an offer appears or its `ttl_ms` ends it `expired` with `run.job.unplaceable`. Offers are cached for `RUN_OFFERS_TTL_MS`; a Network that does not answeris retried, never a reason to fail a job.++## Races (the Arena on the one engine)++++A race is the Arena's round (compare-arena-ecosystem.md §3): the same prompt, two to four entrants, one child job per++entrant on the same machine class (`linux`), the same share of the parent budget each, and no entrant named until a++person judges. The play is data — `agents.play@1`, id `race`, one role per side, one parallel phase, `blind: true`,++`stop.when: person` — listed by `GET /api/v1/plays`. Contracts 0.422.0 has no `run.race@1`, so the request and answer++are validated in `server/races/service.js`; the `bench.match@1` the judgement records is validated before it is sent.++++| Method and path | Capability | Request → answer |++|-----------------------------------|------------------|------------------|++| `GET /api/v1/plays` | `run.job.list` | → `{ plays: agents.play@1[] }` |++| `POST /api/v1/races` | `run.job.submit` | `{ prompt, kind: chat\|code\|design, entrants: [2–4 bench.entrant ids] \| matchmaker: true, budget: { usd?, wall_ms?, cpu_ms?, mem_bytes? } }` → 201 race |++| `GET /api/v1/races/:id` | `run.job.read` | each side's output so far (the job stream's ring), its state and, once ended, its receipts; names only after a judgement |++| `POST /api/v1/races/:id/judgement` | `run.job.submit` | `{ winner: a\|b\|tie\|both_bad }` → 201 race with the names revealed |++++- **Who may start and judge.** A person, through a first-party service (`X-OV-Subject` + `X-OV-Project`): Run records++ the person as the requester and the service as `via`. Only that person, through that service and project, may read++ or judge it; anyone else gets the same `404 run.race_not_found` a missing race does.++- **The split.** `budget` is the round's total and each side gets `floor(total / sides)` of wall, CPU and memory++ (memory floored to bytes); `usd` is split and recorded, not metered by Run. A total too small for every side is++ `422 run.limits.exceeded`; the per-side limits then pass the project caps like any job.++- **The child.** `class: linux`, `args: { play, role, race, kind, prompt, entrant, budget }`, with++ `job.parent = { type: 'race', id }` (dropped by `frameOf`, so Bot never receives it). Placement, metering, events++ and the stream are the ordinary job path.++- **The judgement.** One `bench.match@1`, source `arena`, the shown order (`a, b, …`, randomised when the race is++ created), `blind: true`, each ended side's receipts, `play: { id: 'race', run_id }`, and one `human_pairwise`++ judgement whose `by.id_hash` is an HMAC of the person (`RUN_RACE_HASH_KEY`, falling back to the Network client++ secret). Bench records it before Run writes the judgement and reveals the names; a Bench that cannot answer leaves++ the race blind and retryable with the same match id.++- **The matchmaker.** Bench has no matchmaker endpoint yet (checked 2026-11-13). `matchmaker: true` picks two entrants++ of different model families at random from `RUN_RACE_ENTRANTS`; unset, it answers `422++ run.race_matchmaker_unavailable` and names the setting.++## Tests```@@@ -7,11 +7,11 @@"runtime": "node 22, express 4, PostgreSQL 18 (openvibe-sdk/db) + Valkey 9","port": 4920,"domain": "openvibe.run",-- "contracts": "openvibe-contracts v0.419.0",++ "contracts": "openvibe-contracts v0.422.0","sdk": "openvibe-sdk v0.161.0","shared": "openvibe-shared v4.232.0","packages": [-- "openvibe-contracts v0.419.0",++ "openvibe-contracts v0.422.0","openvibe-sdk v0.161.0","openvibe-shared v4.232.0"],@@@ -56,7 +56,9 @@"plan T14 R4a built, not deployed: the managed-browser provider adapter (server/dispatch/browser-provider.js; RUN_BROWSER_PROVIDER_URL / _API_KEY / _TIMEOUT_MS, off while unset) places class browser jobs on kind=provider offers named browser-provider, validates every answer against browser-provider.job-create-result@1 / job-state@1 and never adopts a malformed one; Run meters those jobs itself, the provider's claimed wall time clamped to Run's observed duration + 5 s, and relays the readings from run_usage_outbox (migrations/0003_usage_outbox.sql) to Billing (RUN_BILLING_URL; unset, they wait and the jobs still run)","truthful readiness (GET /api/ready: db required; valkey, Network JWKS, Events, Bot reachability and the poller's own status reported, never claimed) and the loopback-only /metrics gauges (jobs queued, active, ended, outbox backlog, streams)","plan T11 step 4 built, not deployed: /llms.txt and /llms-full.txt (server/web/discovery.js, public pages and docs only, never the API, admin, a job or a stream) and IndexNow (INDEXNOW_KEY: serves GET /<key>.txt and pings the public pages once at boot; unset, the key route is 404 and nothing is sent)",-- "a person's own machine (user-owned offer read from Network's internal list) runs only that person's jobs naming user-owned; a delegating service cancels and streams (by ticket) the jobs it started for a person"++ "a person's own machine (user-owned offer read from Network's internal list) runs only that person's jobs naming user-owned; a delegating service cancels and streams (by ticket) the jobs it started for a person",++ "the Arena's race on the one engine (compare-arena-ecosystem.md §3, server/races/): POST /api/v1/races starts one child linux job per side under one parent race with the same share of the budget each, GET /api/v1/races/:id returns each side's output so far and its receipts, and POST /api/v1/races/:id/judgement records a bench.match@1 on Bench (source arena, shown order, blind true, human_pairwise with a keyed person hash) before the names are revealed; only the person who started it, through the same service, may read or judge",++ "the race play as data (agents.play@1 1.1.0, server/plays/race.js): id race, one role per side, one parallel phase, an even budget split, blind true and stop when person, listed by GET /api/v1/plays; the matchmaker falls back to two entrants of different families from RUN_RACE_ENTRANTS while Bench has no matchmaker endpoint (Contracts 0.422.0 has no run.race@1 yet, so the race request/answer is validated in server/races/service.js)"],"updated": "2026-11-13"}nnew file mode 100644@@@ -0,0 +1,29 @@++-- phase: expand++-- The Arena's race (compare-arena-ecosystem.md §3, 2026-11-13): one parent record per round, its sides in++-- the shown order, the entrant each side ran and the child job that answers it, the person who started it,++-- and the one human judgement that ends the round. The parent itself is Run's record; the child runs stay++-- run_jobs rows (one per side), so placement, metering and the job stream are unchanged.++--++-- SIDES is [{ slot, entrant, job_id }] in the order the sides are shown (a, b, …); ENTRANT is a++-- bench.entrant@1 id and is never answered before JUDGEMENT is written. MATCH_ID is the bench.match@1 id++-- minted with the race, so a retry after a lost answer records the same match at Bench rather than a second++-- one. JUDGEMENT is { winner, judge, at, match_id, by_hash } once Bench accepted the match.++CREATE TABLE IF NOT EXISTS run_races (++ id text PRIMARY KEY CHECK (id ~ '^rac_[0-9A-HJKMNP-TV-Z]{26}$'),++ project_id text NOT NULL,++ requester text NOT NULL, -- the person who started it: user:<subject>++ via text NOT NULL, -- the first-party service they started it through++ kind text NOT NULL CHECK (kind IN ('chat', 'code', 'design')),++ prompt text NOT NULL,++ prompt_hash text NOT NULL CHECK (prompt_hash ~ '^[a-f0-9]{64}$'),++ sides jsonb NOT NULL,++ budget jsonb NOT NULL, -- { total: {...}, per_side: {...} }; the same share for every side++ play jsonb NOT NULL, -- the race's agents.play@1 document, with the sides that ran++ match_id text NOT NULL CHECK (match_id ~ '^mat_[0-9A-HJKMNP-TV-Z]{26}$'),++ blind boolean NOT NULL DEFAULT true,++ judgement jsonb,++ judged_at bigint,++ created_at bigint NOT NULL,++ updated_at bigint NOT NULL++);++CREATE INDEX IF NOT EXISTS run_races_owner ON run_races (project_id, requester, via, created_at DESC);@@@ -15,7 +15,7 @@"express-rate-limit": "^7.5.0","helmet": "^8.1.0","iovalkey": "^0.4.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0","pg": "^8.23.0",@@@ -1229,9 +1229,9 @@}},"node_modules/openvibe-contracts": {-- "version": "0.419.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",-- "integrity": "sha512-VxrTdeHchT8gvl2wWC9ASaId92en7ZKfJ8F/0hKzCAD0f2Y00rPrqgBWF9BNPoLGARoyQvsMbhBEhSkhxGNnjA==",++ "version": "0.422.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0",++ "integrity": "sha512-xQQq7NQLM9iQve8dR4Hc2EUp75x/FZYfAsHumsmf6dGHjIy05PMX0PlYmxw5L52VXKOj+r7mRwlcehRqgMoLSw==","license": "AGPL-3.0-only","dependencies": {"ajv": "^8.17.1",@@@ -26,7 +26,7 @@"express-rate-limit": "^7.5.0","helmet": "^8.1.0","iovalkey": "^0.4.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0","pg": "^8.23.0",nnew file mode 100644@@@ -0,0 +1,68 @@++'use strict';++++/**++ * The Arena's race routes on the Run API (compare-arena-ecosystem.md §3):++ *++ * | Method & path | Capability | Request → answer |++ * |---------------------------------------|------------------|-------------------------------------------------------|++ * | GET /plays | run.job.list | common.no-body@1 → { plays: agents.play@1[] } |++ * | POST /races | run.job.submit | { prompt, kind, entrants | matchmaker, budget } → 201 race |++ * | GET /races/:id | run.job.read | sides' live output, receipts; names only after a judgement |++ * | POST /races/:id/judgement | run.job.submit | { winner } → 201 race with the names revealed |++ *++ * Contracts 0.422.0 has no run.race@1 yet, so the race request and answer are validated by++ * server/races/service.js by hand (and the bench.match@1 it records is validated against its released++ * contract). The capabilities are the job ones on purpose — Run has six, and a race is a group submit and++ * a group read; inventing run.race.* here would be a capability no manifest grants.++ *++ * A race belongs to the person who started it and the first-party service they started it through: a++ * delegated token (X-OV-Subject + X-OV-Project) is required. Another person, service or project gets the++ * same 404 a missing race does, and only that person may judge.++ */++const express = require('express');++const { fail } = require('../util');++++function racesRouter({ races, apiAuth }) {++ const r = express.Router();++ const wrap = (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next);++ const guard = apiAuth.requireCapability;++++ // ── GET /plays — run.job.list ───────────────────────────────────────────────++ // The race play is data (agents.play@1), so the same document lists here and drives a race.++ r.get('/plays', guard('run.job.list'), wrap(async (_req, res) => {++ res.json({ plays: races.plays() });++ }));++++ // ── POST /races — run.job.submit ────────────────────────────────────────────++ r.post('/races', guard('run.job.submit'), wrap(async (req, res) => {++ const scope = apiAuth.requireProject(req, { delegated: true });++ const race = await races.start({++ projectId: scope.project, requester: scope.requester, via: scope.via,++ body: req.body, traceparent: req.ov && req.ov.traceparent,++ });++ res.status(201).json(race);++ }));++++ // ── GET /races/:id — run.job.read ───────────────────────────────────────────++ r.get('/races/:id', guard('run.job.read'), wrap(async (req, res) => {++ const scope = apiAuth.requireProject(req, { delegated: true });++ const race = await races.get({ projectId: scope.project, requester: scope.requester, via: scope.via, id: req.params.id });++ if (!race) fail(404, 'run.race_not_found', 'no such race');++ res.json(race);++ }));++++ // ── POST /races/:id/judgement — run.job.submit ──────────────────────────────++ r.post('/races/:id/judgement', guard('run.job.submit'), wrap(async (req, res) => {++ const scope = apiAuth.requireProject(req, { delegated: true });++ const race = await races.judge({++ projectId: scope.project, requester: scope.requester, via: scope.via,++ id: req.params.id, body: req.body,++ });++ if (!race) fail(404, 'run.race_not_found', 'no such race');++ res.status(201).json(race);++ }));++++ return r;++}++++module.exports = { racesRouter };@@@ -63,6 +63,12 @@ function createApp(opts = {}) {const membership = opts.membership || createProjectMembership(config, { fetchImpl, now });const fillNotifier = opts.fillNotifier || createFillNotifier(config, { fetchImpl, now });const apiAuth = createApiAuth({ config, keys, userAuth, membership, log });++ // The Arena's race (compare-arena-ecosystem.md §3): one parent, one child job per side, the names after++ // a Bench judgement. A test injects its own bench recorder; production talks to OpenVibe.Bench.++ const bench = opts.bench || require('./races/bench').createBenchRecorder({ config, fetchImpl, now, log });++ const races = opts.races || require('./races/service').createRaces({++ config, db, jobs: require('./jobs/store'), events, stream, bench, now, log,++ });// The checks scheduler (server/checks/): the GitHub App's owner page, the webhook and the agreement// query. It owns no module-level state and starts nothing here; server/index.js starts its tick with// the other background jobs. Without RUN_SECRETS_KEY the page says so and the webhook answers 503.@@@ -151,6 +157,7 @@ function createApp(opts = {}) {apiAuth.middleware,require('./compute/routes').computeRouter({ broker, config }),sessionsRouter({ sessions }),++ require('./api/races').racesRouter({ races, apiAuth }),v1Router({ config, db, apiAuth, stream, tickets, events, dispatch, now, log }));// ── The public site (openvibe.run): the front page, the legal pages and the Frame's shared assets. ──@@@ -240,7 +247,7 @@ function createApp(opts = {}) {return res.status(500).type('text/plain').send('Something went wrong\n');});-- Object.assign(app.locals, { config, db, indexnow, valkey, keys, outbox, events, stream, tickets, dispatch, apiAuth, metrics, desks, sessions, broker, checks });++ Object.assign(app.locals, { config, db, indexnow, valkey, keys, outbox, events, stream, tickets, dispatch, apiAuth, metrics, desks, sessions, broker, races, bench, checks });return app;}@@@ -160,13 +160,25 @@ function loadConfig(env = process.env) {},// Placement A1: Bench's quality priors, cached an hour, used by place() when the option is a model Bench has-- // seen. Off in tests unless asked for, so a unit test never reaches the real Bench.++ // seen. Off in tests unless asked for, so a unit test never reaches the real Bench. The race judgement++ // records a bench.match@1 on the same host (POST /api/v1/matches, capability bench.judgement.cast).bench: {url: trim(env.RUN_BENCH_URL || 'https://bench.openvibe.services'),++ audience: env.RUN_BENCH_AUDIENCE || 'openvibe.bench',++ timeoutMs: Math.max(100, int(env.RUN_BENCH_TIMEOUT_MS, 8000)),priors: env.RUN_BENCH_PRIORS != null ? bool(env.RUN_BENCH_PRIORS) : nodeEnv !== 'test',priorsTtlMs: Math.max(0, int(env.RUN_BENCH_PRIORS_TTL_MS, 60 * 60_000)),},++ // The Arena's race (compare-arena-ecosystem.md §3). Bench has no matchmaker endpoint yet, so++ // `matchmaker: true` picks two entrants of different families from this configured list. `hashKey`++ // keys the person's id hash in a bench.match@1 judgement: never the person's id, and stable across++ // restarts (it falls back to Run's Network client secret, which production always has).++ races: {++ entrants: String(env.RUN_RACE_ENTRANTS || '').split(',').map((s) => s.trim()).filter(Boolean),++ hashKey: env.RUN_RACE_HASH_KEY || env.OV_OAUTH_CLIENT_SECRET || 'run-race-development-key',++ },++// The external managed-browser provider (plan T14 R4a, server/dispatch/browser-provider.js): class browser// jobs placed on a kind=provider offer named `browser-provider` go to its API instead of Bot. Off while// RUN_BROWSER_PROVIDER_URL or the key is unset: no provider offer is placed on, so a browser job waits.@@@ -156,6 +156,11 @@ async function mint(db, body, { projectId, requester, via = null, idempotencyKey/** Project-scoped get: null when the job does not exist or belongs to another project (→ 404, never 403). */const get = (db, id, projectId = null) => db.maybe(`SELECT * FROM run_jobs WHERE id = $1${projectId ? ' AND project_id = $2' : ''}`, projectId ? [id, projectId] : [id]);++/** Several jobs by id (a race's child runs); the caller maps them by id, and an absent one is simply not in the result. */++const getMany = (db, ids) => (ids && ids.length++ ? db.many('SELECT * FROM run_jobs WHERE id = ANY($1::text[])', [ids])++ : Promise.resolve([]));++/** A delegated caller's get: the job only if it is this project's, this person's, through this service (else null → 404). */const getOwn = (db, id, { projectId, requester, via }) => db.maybe('SELECT * FROM run_jobs WHERE id = $1 AND project_id = $2 AND requester = $3 AND via = $4', [id, projectId, requester, via]);@@@ -349,7 +354,7 @@ function readResult(row) {module.exports = {// The store surface the API and the dispatcher bridge (plan T14 step 6) use.-- put, mint, get, getOwn, list, adminList, cancel, mirror, settle, toJob, readResult, frameOf,++ put, mint, get, getMany, getOwn, list, adminList, cancel, mirror, settle, toJob, readResult, frameOf,// The poller's reads (waiting → placed → active → ended).waiting, active, dueTtl,};@@@ -109,6 +109,21 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {return { ended: false, unsubscribe: () => { ring.subscribers.delete(sub); ring.touched = now(); } };}++ /**++ * What a ring holds right now, without creating one: the output so far, the last state it announced++ * and whether it closed. A race's GET reads this for each side, so polling the race shows the sides'++ * answers as they stream (output is best effort, exactly as the stream contract says).++ */++ function snapshot(jobId) {++ const ring = jobs.get(jobId);++ if (!ring) return { state: null, output: '', ended: false };++ return {++ state: ring.state,++ output: ring.events.filter((e) => e.type === 'output').map((e) => e.chunk).join(''),++ ended: ring.ended,++ };++ }++const canAttach = (jobId) => ringOf(jobId).subscribers.size < maxPerJob;const status = () => ({ jobs: jobs.size, streams: [...jobs.values()].reduce((n, r) => n + r.subscribers.size, 0), events: [...jobs.values()].reduce((n, r) => n + r.events.length, 0) });@@@ -121,7 +136,7 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {}}-- return { state, output, seed, subscribe, canAttach, status, close, emit, prune, maxPerJob, ringOf };++ return { state, output, seed, subscribe, snapshot, canAttach, status, close, emit, prune, maxPerJob, ringOf };}module.exports = { createJobStream, JOB_EVENT_TYPES };nnew file mode 100644@@@ -0,0 +1,63 @@++'use strict';++++/**++ * The race play — the Arena's round on the one engine (compare-arena-ecosystem.md §3, agent-engine.md++ * §4): one child run per entrant, the same budget each, the sides blind until a person judges. The play++ * is data (agents.play@1), so the same shape lists on Run, drives the orchestrator and can be written by a++ * person or an agent.++ *++ * A race has 2 to 4 sides, so the play is built per race: one role per side (`a`, `b`, …), one parallel++ * phase that runs them at once, `blind: true`, and `stop.when: person` — the round is over when the person++ * votes, not when the last child ends. The budget split is an equal share each (the caller's total is++ * divided by the number of sides in server/races/service.js; the contract wants the shares as data).++ *++ * The roles say "Side A", never a model or harness name: a role name is visible to the person before the++ * judgement, so it must not carry the entrant.++ */++const contracts = require('openvibe-contracts');++++/** The most sides a race may have (bench.match@1 slots a–h; the Arena's own limit is 4). */++const SLOTS = ['a', 'b', 'c', 'd'];++/** Every side runs on the same machine class, so a race compares entrants on one environment. */++const MACHINE_CLASS = 'linux';++const PLAY_ID = 'race';++++/** The even share of a total budget, as the contracts' number (rounded, never a float surprise in a diff). */++const shareOf = (n) => Number((1 / n).toFixed(6));++++/**++ * racePlay(['a', 'b']) → an agents.play@1 document. `slots` is the sides this race actually has (2–4),++ * so the roles and the split describe the race that ran, not a fixed example.++ */++function racePlay(slots = SLOTS) {++ const sides = [...new Set((Array.isArray(slots) ? slots : []).filter((s) => SLOTS.includes(s)))];++ if (sides.length < 2) throw new TypeError('a race play needs at least two sides (a, b)');++ const share = shareOf(sides.length);++ const play = {++ id: PLAY_ID,++ version: 1,++ name: 'Race',++ summary: 'Several entrants answer the same ask at once, blind side by side, each on the same budget; a person judges and the names are revealed.',++ roles: sides.map((slot) => ({++ id: slot,++ name: `Side ${slot.toUpperCase()}`,++ does: 'Answer the prompt in the run\'s args as the entrant the run names. Work alone: no other side\'s answer, no judging.',++ model: 'same',++ })),++ phases: [{ id: 'answer', name: 'Answer', roles: sides, parallel: true }],++ stop: { when: 'person' },++ budget: { split: Object.fromEntries(sides.map((slot) => [slot, share])) },++ blind: true,++ visibility: 'public',++ author: { type: 'service', id: 'run' },++ };++ contracts.assertValid('agents.play@1', play);++ return play;++}++++/** The plays Run lists today: the race (the first play the engine runs) with the most sides it supports. */++function plays() {++ return [racePlay(SLOTS)];++}++++module.exports = { SLOTS, PLAY_ID, MACHINE_CLASS, racePlay, plays, shareOf };nnew file mode 100644@@@ -0,0 +1,90 @@++'use strict';++++/**++ * Run → OpenVibe.Bench: record one bench.match@1 (the race's judgement) on Bench's++ * POST /api/v1/matches (capability bench.judgement.cast, audience openvibe.bench).++ *++ * The match is validated against the released contract before it is sent, so a broken body is Run's own++ * 500/422 and never a Bench refusal. The token is a Network client-credentials service token minted from++ * OV_OAUTH_CLIENT_ID/OV_OAUTH_CLIENT_SECRET and cached until 60 s before expiry by openvibe-sdk/auth; a++ * 401 drops the cache and the call is tried once more. A Bench that cannot answer is a 503 to the caller++ * and the race stays unjudged: names are only revealed after the match is recorded.++ *++ * Bench has no matchmaker endpoint yet (checked 2026-11-13, Contracts 0.422.0), so the matchmaker fallback++ * lives in server/races/matchmaker.js.++ */++const contracts = require('openvibe-contracts');++const { createServiceTokenClient } = require('openvibe-sdk/auth');++++const BENCH_JUDGE = 'bench.judgement.cast';++++class BenchError extends Error {++ constructor(status, code, detail) {++ super(detail || code);++ this.name = 'BenchError';++ this.status = status;++ this.code = code;++ this.detail = detail;++ }++}++++/**++ * createBenchRecorder({ config, fetchImpl, now, log }) → { enabled, note, record, status }.++ * `enabled` is false without OV_OAUTH_CLIENT_SECRET (Run cannot mint the Bench token); record() then++ * answers 503 and nothing is sent.++ */++function createBenchRecorder({ config = {}, fetchImpl = globalThis.fetch, now = () => Date.now(), log = console } = {}) {++ const bench = config.bench || {};++ const url = String(bench.url || '').replace(/\/+$/, '');++ const audience = bench.audience || 'openvibe.bench';++ const timeoutMs = bench.timeoutMs || 8000;++ const { clientId, clientSecret } = config.oauth || {};++ const tokens = clientSecret++ ? createServiceTokenClient({ tokenUrl: `${config.network.internalUrl}/oauth/token`, clientId, clientSecret, audience, scope: BENCH_JUDGE, fetch: fetchImpl, now })++ : null;++ const enabled = Boolean(tokens && url);++ const note = enabled++ ? `Bench matches: ${url}, audience ${audience}, capability ${BENCH_JUDGE}`++ : 'Bench matches are off: OV_OAUTH_CLIENT_SECRET is unset, so Run cannot mint its Network client-credentials token';++++ /** One call: mint (or reuse) the token, fetch, and turn Bench's answer into a value or a BenchError. */++ async function call(match) {++ if (!enabled) throw new BenchError(503, 'run.bench_unavailable', note);++ for (let attempt = 0; ; attempt++) {++ const headers = { Accept: 'application/json', 'Content-Type': 'application/json' };++ try { Object.assign(headers, await tokens.authHeaders()); }++ catch (e) { throw new BenchError(503, 'run.bench_unavailable', 'Run could not mint its Bench token from Network'); }++ let res;++ try {++ res = await fetchImpl(`${url}/api/v1/matches`, {++ method: 'POST', headers, body: JSON.stringify(match), signal: AbortSignal.timeout(timeoutMs),++ });++ } catch (e) {++ const why = e && (e.name === 'TimeoutError' || e.name === 'AbortError') ? `no answer within ${timeoutMs} ms` : 'unreachable';++ throw new BenchError(503, 'run.bench_unavailable', `Bench did not answer (${why}); the race stays unjudged`);++ }++ // A rotated or clock-skewed token: drop the cache and try once more.++ if (res.status === 401 && attempt === 0) { tokens.invalidate(); continue; }++ const doc = await res.json().catch(() => null);++ if (res.ok && doc) return { created: res.status === 201, match: doc.match || null };++ const code = doc && typeof doc.code === 'string' ? doc.code : null;++ const detail = doc && typeof doc.detail === 'string' ? doc.detail.slice(0, 300) : null;++ const why = `Bench answered HTTP ${res.status}${code ? ` ${code}` : ''}${detail ? `: ${detail}` : ''}`;++ if (res.status >= 500) throw new BenchError(503, 'run.bench_unavailable', `${why}; the race stays unjudged`);++ throw new BenchError(502, 'run.bench_refused', `${why}; the race stays unjudged`);++ }++ }++++ return {++ enabled,++ note,++ /** Record one match. The body is validated against bench.match@1 here, before any call. */++ async record(match) {++ contracts.assertValid('bench.match@1', match);++ return call(match);++ },++ status: () => ({ enabled, note, url, audience, capability: BENCH_JUDGE, timeout_ms: timeoutMs }),++ };++}++++module.exports = { createBenchRecorder, BenchError, BENCH_JUDGE };nnew file mode 100644@@@ -0,0 +1,39 @@++'use strict';++++/**++ * bench.entrant@1 ids and their model families, for a matchmaker that must not pair two of one family++ * (bench.entrant@1: "the model family, so the matchmaker can avoid pairing two of one family").++ *++ * The id is the only thing Run is given by the race API, so the family is read from it: a model id is the++ * model itself; an agent id is agent:<harness>+<model>#<config>; a harness or a product has no model and++ * falls back to its own name. The family is the first segment of the model (gpt-5-mini → gpt,++ * deepseek-flash → deepseek, meta-llama/… → meta), which is coarse on purpose: it keeps two variants of++ * one maker's model out of a pair.++ */++++/** bench.entrant@1's own id pattern (contracts/bench/entrant.v1.json). */++const ENTRANT_RE = /^(model:[A-Za-z0-9._/:-]{1,200}|harness:[a-z0-9][a-z0-9._-]{0,60}@[A-Za-z0-9._-]{1,40}|agent:[a-z0-9][a-z0-9._-]{0,60}\+[A-Za-z0-9._/:-]{1,200}#[a-f0-9]{8,64}|product:[a-z0-9][a-z0-9-]{0,60})$/;++++const isEntrantId = (v) => typeof v === 'string' && ENTRANT_RE.test(v);++++/** The model (or, for a harness/product, the thing itself) an entrant id names. */++function modelOf(id) {++ const s = String(id || '');++ if (s.startsWith('model:')) return s.slice(6);++ if (s.startsWith('agent:')) {++ const plus = s.indexOf('+');++ const hash = s.indexOf('#');++ return plus >= 0 ? s.slice(plus + 1, hash > plus ? hash : undefined) : s.slice(6);++ }++ if (s.startsWith('harness:')) return s.slice(8).split('@')[0];++ if (s.startsWith('product:')) return s.slice(8);++ return s;++}++++/** The coarser family the matchmaker keeps apart (gpt, deepseek, glm, claude, llama …). */++function familyOf(id) {++ const model = modelOf(id).toLowerCase();++ return model.split(/[-/.:]/)[0] || model;++}++++module.exports = { ENTRANT_RE, isEntrantId, modelOf, familyOf };nnew file mode 100644@@@ -0,0 +1,42 @@++'use strict';++++/**++ * The race matchmaker's fallback (compare-arena-ecosystem.md §3: "the matchmaker (Bench) picks the++ * entrants: pairs whose outcome is most uncertain and that the person's allowance can pay for, new++ * entrants explored, two of the same family avoided, position randomised").++ *++ * Bench has no matchmaker endpoint yet (checked against Bench's main on 2026-11-13: /api/v1/matches and++ * /api/v1/factors exist, no matchmaker). Until it does, Run picks two entrants at random from++ * RUN_RACE_ENTRANTS, never two of one model family, and the race service randomises their positions.++ * When Bench ships the matchmaker this module is the one place that changes: ask Bench first, keep the++ * configured list as the fallback when it cannot answer.++ */++const { isEntrantId, familyOf } = require('./entrants');++++/** Every unordered pair from `list` (small lists only: a race has at most four sides). */++function pairs(list) {++ const out = [];++ for (let i = 0; i < list.length; i++) for (let j = i + 1; j < list.length; j++) out.push([list[i], list[j]]);++ return out;++}++++/**++ * pair({ entrants, random }) → [a, b] from two different families, at random.++ * `entrants` is the configured list (RUN_RACE_ENTRANTS); invalid ids are dropped. Throws a plain error++ * whose message the service turns into a 422, so an unconfigured deployment says what to set.++ */++function pair({ entrants = [], random = Math.random } = {}) {++ const list = [...new Set((Array.isArray(entrants) ? entrants : []).filter(isEntrantId))];++ if (list.length < 2) {++ throw Object.assign(new Error('the race matchmaker is not configured: set RUN_RACE_ENTRANTS to at least two bench.entrant ids (Bench has no matchmaker endpoint yet)'), { code: 'run.race_matchmaker_unavailable' });++ }++ const choices = pairs(list).filter(([a, b]) => familyOf(a) !== familyOf(b));++ if (!choices.length) {++ throw Object.assign(new Error('the configured race entrants are all of one model family; set RUN_RACE_ENTRANTS to entrants of at least two families'), { code: 'run.race_matchmaker_unavailable' });++ }++ const r = Number(random());++ const pick = choices[Math.min(choices.length - 1, Math.max(0, Math.floor((Number.isFinite(r) ? r : 0) * choices.length)))];++ return [...pick];++}++++module.exports = { pair, pairs, familyOf };nnew file mode 100644@@@ -0,0 +1,325 @@++'use strict';++++/**++ * The race service (compare-arena-ecosystem.md §3, agent-engine.md §4): one parent race, one child job per++ * side, the same share of the budget each, no entrant named until a person judges.++ *++ * A race is started by a first-party service acting for a person (the Arena's front door in Compare): the++ * person is the requester and the only one who may judge, through the same service. The child jobs are++ * ordinary run_jobs rows — class linux, args carrying the side, the race and the entrant the node must run++ * — so placement, metering, events and the job stream are the engine's own, unchanged. Each side's output++ * is read from the job stream's ring on every GET (best effort, as run.job-stream-event@1 says).++ *++ * The judgement is a bench.match@1 recorded on Bench BEFORE the names are revealed: source arena, the++ * shown order, blind true, each side's receipts, one human_pairwise judgement with a keyed hash of the++ * person, the play and its parent race. If Bench cannot record it, the race stays unjudged and the names++ * hidden; a retry sends the same match id, so Bench records one match rather than two.++ */++const crypto = require('crypto');++const contracts = require('openvibe-contracts');++const { fail, iso } = require('../util');++const { checkLimits } = require('../api/v1');++const { SLOTS, PLAY_ID, MACHINE_CLASS, racePlay } = require('../plays/race');++const raceStore = require('./store');++const matchmaker = require('./matchmaker');++const { isEntrantId } = require('./entrants');++++const END_STATES = new Set(['succeeded', 'failed', 'cancelled', 'expired']);++const KINDS = ['chat', 'code', 'design'];++const WINNERS = new Set([...SLOTS, 'tie', 'both_bad']);++const START_KEYS = new Set(['prompt', 'kind', 'entrants', 'matchmaker', 'budget']);++const JUDGEMENT_KEYS = new Set(['winner']);++const DEFAULT_LIMITS = { wall_ms: 300_000, cpu_ms: 300_000, mem_bytes: 512 * 1024 * 1024 };++const OUTCOME = { succeeded: 'succeeded', failed: 'failed', expired: 'timed_out', cancelled: 'cancelled' };++const RECEIPT_INTS = ['tokens_in', 'tokens_out', 'tool_calls', 'length_chars', 'markdown_items'];++const RECEIPT_NUMS = ['cost_usd', 'ttft_ms', 'tps'];++++const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : null);++const isCount = (v) => Number.isInteger(v) && v >= 1;++const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');++++/** A keyed hash of the person who judged: stable for rate limits, never the person's id (bench.match@1). */++function personHash(requester, key) {++ const subject = String(requester || '').replace(/^user:/, '');++ return crypto.createHmac('sha256', key).update(`run.race.person.v1\n${subject}`).digest('hex');++}++++/** The body of POST /api/v1/races, validated by hand (Contracts 0.422.0 has no run.race@1 yet). */++function validStart(body) {++ if (!obj(body)) fail(400, 'run.invalid_request', 'the body must be a JSON object { prompt, kind, entrants | matchmaker, budget }');++ const extra = Object.keys(body).filter((k) => !START_KEYS.has(k));++ if (extra.length) fail(400, 'run.invalid_request', `unknown field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);++ const prompt = typeof body.prompt === 'string' ? body.prompt.trim() : '';++ if (!prompt) fail(400, 'run.invalid_request', 'prompt is required and must be a non-empty string');++ if (prompt.length > 20000) fail(400, 'run.invalid_request', 'prompt is at most 20000 characters');++ if (!KINDS.includes(body.kind)) fail(400, 'run.invalid_request', `kind must be one of ${KINDS.join(', ')}`);++ if (!obj(body.budget)) fail(400, 'run.invalid_request', 'budget is required and must be an object');++ const budget = validBudget(body.budget);++ const wantsMatchmaker = body.matchmaker === true;++ if (body.matchmaker !== undefined && typeof body.matchmaker !== 'boolean') fail(400, 'run.invalid_request', 'matchmaker must be true or false');++ if (wantsMatchmaker && body.entrants !== undefined) fail(400, 'run.invalid_request', 'name entrants or ask matchmaker: true, not both');++ if (!wantsMatchmaker) {++ if (!Array.isArray(body.entrants)) fail(400, 'run.invalid_request', 'entrants must be an array of 2 to 4 bench.entrant ids, or matchmaker: true');++ if (body.entrants.length < 2 || body.entrants.length > 4) fail(400, 'run.invalid_request', 'a race has 2 to 4 entrants');++ for (const id of body.entrants) if (!isEntrantId(id)) fail(400, 'run.invalid_request', `${JSON.stringify(id)} is not a bench.entrant@1 id`);++ if (new Set(body.entrants).size !== body.entrants.length) fail(400, 'run.invalid_request', 'entrants must be unique');++ }++ return { prompt, kind: body.kind, budget, matchmaker: wantsMatchmaker, entrants: wantsMatchmaker ? null : body.entrants };++}++++/** The total budget a race asks for; an absent cap takes the default. */++function validBudget(b) {++ const keys = new Set(['usd', 'wall_ms', 'cpu_ms', 'mem_bytes']);++ const extra = Object.keys(b).filter((k) => !keys.has(k));++ if (extra.length) fail(400, 'run.invalid_request', `unknown budget field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);++ const out = { ...DEFAULT_LIMITS };++ for (const k of ['wall_ms', 'cpu_ms', 'mem_bytes']) {++ if (b[k] === undefined) continue;++ if (!isCount(b[k])) fail(400, 'run.invalid_request', `budget.${k} must be a positive integer`);++ out[k] = b[k];++ }++ if (b.usd !== undefined) {++ if (typeof b.usd !== 'number' || !Number.isFinite(b.usd) || b.usd < 0) fail(400, 'run.invalid_request', 'budget.usd must be a number of at least 0');++ out.usd = b.usd;++ }++ return out;++}++++/** Split the total evenly: every side runs on the same share, never more than the parent named. */++function splitBudget(total, n) {++ const usd = total.usd == null ? null : Math.round((total.usd / n) * 1e6) / 1e6;++ const per = {++ usd,++ wall_ms: Math.floor(total.wall_ms / n),++ cpu_ms: Math.floor(total.cpu_ms / n),++ mem_bytes: Math.floor(total.mem_bytes / n),++ machine_class: MACHINE_CLASS,++ };++ if (per.wall_ms < 1000 || per.cpu_ms < 1000) fail(422, 'run.limits.exceeded', `budget is too small for ${n} sides: each side needs at least 1000 ms of wall and CPU time`);++ if (per.mem_bytes < 1024 * 1024) fail(422, 'run.limits.exceeded', `budget is too small for ${n} sides: each side needs at least 1 MiB of memory`);++ return { total: { ...total, machine_class: MACHINE_CLASS }, per_side: per };++}++++/** Fisher–Yates: the sides are labelled a, b, … in a random order, so a position is not a name. */++function shuffled(list, random = Math.random) {++ const a = [...list];++ for (let i = a.length - 1; i > 0; i--) {++ const j = Math.floor((Number(random()) || 0) * (i + 1));++ [a[i], a[j]] = [a[j], a[i]];++ }++ return a;++}++++/** What a finished child sends Bench: its outcome and the receipts the stream/result can back. */++function receiptsOf(row) {++ const result = obj(row.result) || {};++ const src = obj(result.receipts) || result;++ const out = { outcome: OUTCOME[row.state] || 'failed' };++ for (const k of RECEIPT_NUMS) if (typeof src[k] === 'number' && Number.isFinite(src[k]) && src[k] >= 0) out[k] = src[k];++ for (const k of RECEIPT_INTS) if (Number.isInteger(src[k]) && src[k] >= 0) out[k] = src[k];++ if (row.wall_ms != null && Number(row.wall_ms) > 0) out.wall_ms = Number(row.wall_ms);++ const answer = typeof result.answer === 'string' ? result.answer : typeof result.text === 'string' ? result.text : null;++ if (out.length_chars == null && answer != null) out.length_chars = answer.length;++ return out;++}++++/** The bench.match@1 a race's judgement records. Validated before it leaves Run. */++function matchOf(row, childRows, winner, at) {++ const jobs = new Map(childRows.map((j) => [j.id, j]));++ const match = {++ id: row.match_id,++ source: 'arena',++ task: { kind: row.kind, prompt_hash: row.prompt_hash },++ slots: row.sides.map((side) => {++ const job = jobs.get(side.job_id);++ const slot = { slot: side.slot, entrant: side.entrant, run_id: side.job_id };++ if (job && END_STATES.has(job.state)) slot.receipts = receiptsOf(job);++ return slot;++ }),++ shown: row.sides.map((side) => side.slot),++ blind: true,++ budget: {++ usd: row.budget.per_side.usd == null ? null : row.budget.per_side.usd,++ ms: row.budget.per_side.wall_ms,++ machine_class: row.budget.per_side.machine_class,++ },++ judgements: [{ judge: 'human_pairwise', winner, by: { type: 'person', id_hash: personHash(row.requester, row.hash_key) }, at: iso(at) }],++ play: { id: PLAY_ID, run_id: row.id },++ at: iso(row.created_at),++ };++ contracts.assertValid('bench.match@1', match);++ return match;++}++++/** One race as the API answers it: names only once a judgement is on the row. */++function view(row, childRows, { config, stream }) {++ const jobs = new Map(childRows.map((j) => [j.id, j]));++ const revealed = row.judged_at != null;++ const sides = row.sides.map((side) => {++ const job = jobs.get(side.job_id);++ const shot = stream.snapshot(side.job_id);++ const ended = job && END_STATES.has(job.state);++ const out = {++ slot: side.slot,++ run_id: side.job_id,++ state: job ? job.state : 'missing',++ output: shot.output || null,++ receipts: ended ? receiptsOf(job) : null,++ stream_url: `${config.baseUrl}/api/v1/jobs/${side.job_id}/stream`,++ };++ // The entrant is the name: it is never in the body before the judgement. The child job's own++ // args name it for the node, and the race's sides are the only owner-scoped read that returns it.++ if (revealed) out.entrant = side.entrant;++ return out;++ });++ const allDone = sides.every((s) => END_STATES.has(s.state));++ return {++ id: row.id,++ kind: row.kind,++ prompt: row.prompt,++ state: revealed ? 'judged' : allDone ? 'done' : 'running',++ blind: true,++ revealed,++ play: row.play,++ budget: row.budget,++ shown: row.sides.map((side) => side.slot),++ sides,++ judgement: revealed ? {++ winner: row.judgement.winner,++ judge: 'human_pairwise',++ at: row.judgement.at,++ match_id: row.judgement.match_id,++ recorded_by: row.judgement.recorded_by || 'run',++ } : null,++ // The names as one mapping too, so a client does not have to walk the sides; null while blind.++ names: revealed ? Object.fromEntries(row.sides.map((side) => [side.slot, side.entrant])) : null,++ created_at: iso(row.created_at),++ updated_at: iso(row.updated_at),++ };++}++++/**++ * createRaces({ config, db, jobs, events, stream, bench, now, log, random }) → { plays, start, get, judge }.++ * Every dependency is injected; the API is its only caller.++ */++function createRaces({ config, db, jobs, events, stream, bench, now = () => Date.now(), log = console, random = Math.random } = {}) {++ const hashKey = (config.races && config.races.hashKey) || 'run-race-unsafe-default';++++ /** GET /api/v1/plays: the race play (with the most sides it supports) as agents.play@1. */++ function plays() {++ return require('../plays/race').plays();++ }++++ /** POST /api/v1/races: one parent race and one child job per side, in one transaction. */++ async function start({ projectId, requester, via, body, traceparent }) {++ if (!/^user:/.test(String(requester || ''))) {++ fail(403, 'run.race_person_required', 'a race is started by a person: a first-party service acting for them (X-OV-Subject), never a service for itself');++ }++ const req = validStart(body);++ let entrants;++ if (req.matchmaker) {++ try {++ entrants = matchmaker.pair({ entrants: (config.races && config.races.entrants) || [], random });++ } catch (e) {++ fail(422, e.code || 'run.race_matchmaker_unavailable', e.message);++ }++ } else {++ entrants = req.entrants;++ }++ // The sides are labelled a, b, … in the shown order, which is randomised (position bias: the++ // design asks for it, and bench.match@1's SHOWN records what was shown).++ const sides = shuffled(entrants, random).map((entrant, i) => ({ slot: SLOTS[i], entrant, job_id: null }));++ const budget = splitBudget(req.budget, sides.length);++ const raceId = raceStore.newRaceId();++ const matchId = raceStore.newMatchId();++ const at = now();++ const promptHash = sha256(req.prompt);++ const ttl = budget.per_side.wall_ms + 600_000;++ const play = racePlay(sides.map((side) => side.slot));++++ // Each child is an ordinary linux job: the same limits as every other side, the race and its side++ // in args for the node, and parent/race in the job record (never in the frame sent to Bot).++ const childBodies = sides.map((side) => ({++ class: MACHINE_CLASS,++ args: {++ play: PLAY_ID, role: side.slot, race: raceId, kind: req.kind, prompt: req.prompt,++ entrant: side.entrant, budget: budget.per_side,++ },++ limits: { wall_ms: budget.per_side.wall_ms, cpu_ms: budget.per_side.cpu_ms, mem_bytes: budget.per_side.mem_bytes },++ ttl_ms: ttl,++ }));++ for (const bodyOf of childBodies) {++ const checked = contracts.validate('run.job-create-request@1', bodyOf);++ if (!checked.valid) fail(422, 'run.invalid_input', `a race child is not a run.job-create-request@1: ${checked.errors.map((e) => `${e.path} ${e.message}`).join('; ')}`);++ }++ checkLimits({ ...childBodies[0], args: childBodies[0].args, inputs: [] }, config.limits);++++ const out = await db.tx(async (t) => {++ const children = [];++ for (let i = 0; i < sides.length; i++) {++ const made = await jobs.mint(t, childBodies[i], {++ projectId, requester, via, idempotencyKey: `race:${raceId}:${sides[i].slot}`, now: at,++ });++ if (made.created) await events.queued(t, made.row, { traceparent });++ // Run's own record of the child's place in the tree: platform.job@1 has no field for a++ // parent, and frameOf() drops these keys, so the node is never sent them.++ const marked = await t.maybe(++ `UPDATE run_jobs SET job = job || $2::jsonb WHERE id = $1 RETURNING *`,++ [made.row.id, JSON.stringify({ parent: { type: 'race', id: raceId }, race: { slot: sides[i].slot, play: PLAY_ID } })]);++ sides[i].job_id = made.row.id;++ children.push(marked || made.row);++ }++ const row = await raceStore.insert(t, {++ id: raceId, project_id: projectId, requester, via, kind: req.kind, prompt: req.prompt, prompt_hash: promptHash,++ sides, budget, play, match_id: matchId, blind: true, judgement: null, judged_at: null, created_at: at, updated_at: at,++ });++ return { row, children };++ });++ log.log(`[Run] race ${out.row.id}: ${sides.length} blind sides, each ${budget.per_side.wall_ms} ms`);++ return view(out.row, out.children, { config, stream });++ }++++ /** GET /api/v1/races/:id: sides' live output and receipts; names only once judged. */++ async function get({ projectId, requester, via, id }) {++ const row = await raceStore.get(db, id, { projectId, requester, via });++ if (!row) return null;++ const children = await jobs.getMany(db, row.sides.map((side) => side.job_id));++ return view(row, children, { config, stream });++ }++++ /** POST /api/v1/races/:id/judgement: record the match on Bench, then reveal the names. */++ async function judge({ projectId, requester, via, id, body }) {++ if (!obj(body)) fail(400, 'run.invalid_request', 'the body must be JSON { winner: a|b|tie|both_bad }');++ const extra = Object.keys(body).filter((k) => !JUDGEMENT_KEYS.has(k));++ if (extra.length) fail(400, 'run.invalid_request', `unknown field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);++ const winner = typeof body.winner === 'string' ? body.winner : '';++ if (!WINNERS.has(winner)) fail(400, 'run.invalid_request', 'winner must be a shown side (a, b, …), tie or both_bad');++ const row = await raceStore.get(db, id, { projectId, requester, via });++ if (!row) return null; // another person, service or project: the same 404++ if (row.judged_at != null) fail(409, 'run.race_already_judged', 'this race has already been judged');++ if (winner !== 'tie' && winner !== 'both_bad' && !row.sides.some((side) => side.slot === winner)) {++ fail(422, 'run.invalid_input', `${winner} is not a side of this race (${row.sides.map((s) => s.slot).join(', ')})`);++ }++ const children = await jobs.getMany(db, row.sides.map((side) => side.job_id));++ const at = now();++ const match = matchOf({ ...row, hash_key: hashKey }, children, winner, at);++ // Only Bench recording it reveals the names: if it cannot, the race stays blind and can be judged again.++ const recorded = await bench.record(match);++ const judgement = {++ winner, judge: 'human_pairwise', at: iso(at), match_id: row.match_id,++ by_hash: personHash(row.requester, hashKey), recorded_by: 'run',++ bench: recorded && recorded.created === false ? 'recorded' : 'created',++ };++ const updated = await db.tx((t) => raceStore.markJudged(t, row.id, { projectId, requester, via, judgement, at }));++ if (!updated) fail(409, 'run.race_already_judged', 'this race has already been judged');++ log.log(`[Run] race ${row.id}: judged ${winner}, names revealed`);++ return view(updated, children, { config, stream });++ }++++ return { plays, start, get, judge, personHash: (requester) => personHash(requester, hashKey) };++}++++module.exports = {++ createRaces, validStart, validBudget, splitBudget, receiptsOf, matchOf, view, personHash,++ KINDS, WINNERS, DEFAULT_LIMITS, END_STATES, START_KEYS, JUDGEMENT_KEYS,++};nnew file mode 100644@@@ -0,0 +1,69 @@++'use strict';++++/**++ * The race rows (migrations/0006_races.sql). Row access only: the service creates a race and its child++ * jobs in one transaction, and every read is scoped to the person who started it and the service they++ * started it through — another person's race is null, which the API answers as 404.++ */++const { ids } = require('openvibe-contracts');++++const RACE_RE = /^rac_[0-9A-HJKMNP-TV-Z]{26}$/;++const MATCH_RE = /^mat_[0-9A-HJKMNP-TV-Z]{26}$/;++const SLOT_RE = /^[a-d]$/;++++const isRaceId = (v) => typeof v === 'string' && RACE_RE.test(v);++const isMatchId = (v) => typeof v === 'string' && MATCH_RE.test(v);++const isSlot = (v) => typeof v === 'string' && SLOT_RE.test(v);++++/** A new race id: rac_<ULID> (the id pattern the race API answers; contracts has no race kind yet). */++const newRaceId = () => `rac_${ids.ulid()}`;++/** The bench.match@1 id minted with the race: a retry records the same match rather than a second one. */++const newMatchId = () => `mat_${ids.ulid()}`;++++const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : null);++const num = (v) => (v == null ? null : Number(v));++++/** One row as the store reads it: jsonb parsed, the bigints as numbers. */++function fromDb(row) {++ if (!row) return null;++ return {++ ...row,++ sides: Array.isArray(row.sides) ? row.sides.map((s) => ({ slot: s.slot, entrant: s.entrant, job_id: s.job_id })) : [],++ budget: obj(row.budget) || {},++ play: obj(row.play) || {},++ judgement: obj(row.judgement),++ blind: Boolean(row.blind),++ judged_at: num(row.judged_at),++ created_at: num(row.created_at),++ updated_at: num(row.updated_at),++ };++}++++/** Insert the parent race and its sides (the child job rows are already in this transaction). */++async function insert(t, row) {++ const made = await t.maybe(++ `INSERT INTO run_races (id, project_id, requester, via, kind, prompt, prompt_hash, sides, budget, play, match_id, blind, judgement, judged_at, created_at, updated_at)++ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16) RETURNING *`,++ [row.id, row.project_id, row.requester, row.via, row.kind, row.prompt, row.prompt_hash, JSON.stringify(row.sides),++ JSON.stringify(row.budget), JSON.stringify(row.play), row.match_id, row.blind, row.judgement == null ? null : JSON.stringify(row.judgement),++ row.judged_at, row.created_at, row.updated_at]);++ return fromDb(made);++}++++/** The starter's race: project, person and the service they used all have to match, else null (→ 404). */++async function get(db, id, { projectId, requester, via }) {++ if (!isRaceId(id)) return null;++ return fromDb(await db.maybe(++ 'SELECT * FROM run_races WHERE id = $1 AND project_id = $2 AND requester = $3 AND via = $4',++ [id, projectId, requester, via]));++}++++/** The conditional write that ends a race: one judgement, the first writer wins. → the row, or null. */++async function markJudged(t, id, { projectId, requester, via, judgement, at }) {++ return fromDb(await t.maybe(++ `UPDATE run_races SET judgement = $1, judged_at = $2, updated_at = $2++ WHERE id = $3 AND project_id = $4 AND requester = $5 AND via = $6 AND judged_at IS NULL RETURNING *`,++ [JSON.stringify(judgement), at, id, projectId, requester, via]));++}++++module.exports = { RACE_RE, MATCH_RE, SLOT_RE, isRaceId, isMatchId, isSlot, newRaceId, newMatchId, fromDb, insert, get, markJudged };nnew file mode 100644@@@ -0,0 +1,280 @@++'use strict';++/**++ * The Arena's race on Run (compare-arena-ecosystem.md §3, Contracts 0.422.0): the race play as++ * agents.play@1, N blind child jobs with one budget split evenly, names hidden until a person's judgement,++ * the bench.match@1 that judgement records, and the starter-only rule.++ *++ * node test/races.test.js++ */++const assert = require('assert');++const crypto = require('crypto');++const contracts = require('openvibe-contracts');++const { boot, check, done } = require('./helpers/app');++const { familyOf } = require('../server/races/entrants');++++const { ids } = contracts;++const sha256 = (s) => crypto.createHash('sha256').update(s).digest('hex');++++/** A Bench that records what it is sent and validates it against bench.match@1, like Bench's route does. */++function fakeBench() {++ const calls = [];++ return {++ enabled: true,++ note: 'test bench',++ calls,++ async record(match) {++ calls.push(match);++ contracts.assertValid('bench.match@1', match);++ return { created: true, match };++ },++ status: () => ({ enabled: true, note: 'test bench' }),++ };++}++++async function checks(t, bench) {++ const P = ids.newId('project');++ const Q = ids.newId('project');++ const ada = ids.newId('user');++ const bo = ids.newId('user');++ t.network.members.set(`${P} ${ada}`, { role: 'admin' });++ t.network.members.set(`${P} ${bo}`, { role: 'owner' });++++ const CAPS = ['run.job.submit', 'run.job.read', 'run.job.list'];++ // A first-party service acting for a person: the only principal a race accepts as its starter.++ const as = (subject, project, { sub = 'svc:compare', cap = CAPS } = {}) => ({++ token: t.network.signService({ sub, cap }),++ headers: { 'X-OV-Subject': subject, 'X-OV-Project': project },++ });++ const raceRows = async (raceId) => t.db.many("SELECT * FROM run_jobs WHERE job->>'parent' IS NOT NULL AND job->'parent'->>'id' = $1 ORDER BY id", [raceId]);++++ const prompt = 'Explain in two sentences why the sky is blue.';++ const entrants = ['model:gpt-5-mini', 'model:glm-5.3'];++ const body = { prompt, kind: 'chat', entrants, budget: { usd: 0.2, wall_ms: 60_000, cpu_ms: 60_000, mem_bytes: 268_435_456 } };++++ let raceId;++ let sideEntrants;++++ await check('the race play lists as agents.play@1: one role per side, one parallel phase, even split, blind, stop on person', async () => {++ const r = await t.call('GET', '/api/v1/plays', { token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.list'] }) });++ assert.strictEqual(r.status, 200, r.text);++ assert.strictEqual(r.json.plays.length, 1);++ const play = r.json.plays[0];++ assert.ok(contracts.validate('agents.play@1', play).valid, JSON.stringify(play));++ assert.strictEqual(play.id, 'race');++ assert.strictEqual(play.blind, true);++ assert.deepStrictEqual(play.stop, { when: 'person' });++ assert.strictEqual(play.phases.length, 1);++ assert.strictEqual(play.phases[0].parallel, true);++ assert.deepStrictEqual(play.phases[0].roles, play.roles.map((role) => role.id));++ const split = Object.values(play.budget.split);++ assert.strictEqual(split.length, play.roles.length);++ assert.ok(Math.abs(split.reduce((a, b) => a + b, 0) - 1) < 1e-6, JSON.stringify(play.budget.split));++ for (const role of play.roles) assert.ok(!JSON.stringify(role).includes('gpt') && !JSON.stringify(role).includes('model:'), 'a role never names an entrant');++ });++++ await check('POST /races starts N child jobs under one parent, each with the same share of the budget, and answers no names', async () => {++ const r = await t.call('POST', '/api/v1/races', { body, ...as(ada, P) });++ assert.strictEqual(r.status, 201, r.text);++ const race = r.json;++ raceId = race.id;++ assert.match(raceId, /^rac_[0-9A-HJKMNP-TV-Z]{26}$/);++ assert.strictEqual(race.kind, 'chat');++ assert.strictEqual(race.blind, true);++ assert.strictEqual(race.revealed, false);++ assert.strictEqual(race.names, null);++ assert.strictEqual(race.judgement, null);++ assert.deepStrictEqual(race.shown, ['a', 'b']);++ assert.deepStrictEqual(race.sides.map((s) => s.slot), ['a', 'b']);++ assert.ok(contracts.validate('agents.play@1', race.play).valid);++ assert.strictEqual(race.play.roles.length, 2);++ assert.strictEqual(race.budget.per_side.wall_ms, 30_000);++ assert.strictEqual(race.budget.per_side.cpu_ms, 30_000);++ assert.strictEqual(race.budget.per_side.usd, 0.1);++ assert.strictEqual(race.budget.per_side.machine_class, 'linux');++ const raw = JSON.stringify(race);++ for (const id of entrants) assert.ok(!raw.includes(id), `no entrant before the judgement: ${id} in ${raw}`);++ for (const side of race.sides) assert.ok(!('entrant' in side), 'a side carries no entrant while blind');++++ const rows = await raceRows(raceId);++ assert.strictEqual(rows.length, 2);++ for (const row of rows) {++ assert.strictEqual(row.state, 'queued');++ assert.strictEqual(row.project_id, P);++ assert.strictEqual(row.requester, `user:${ada}`);++ assert.strictEqual(row.via, 'service:compare');++ assert.strictEqual(row.class, 'linux');++ assert.strictEqual(row.job.limits.wall_ms, 30_000);++ assert.strictEqual(row.job.limits.cpu_ms, 30_000);++ assert.strictEqual(row.job.limits.mem_bytes, 134_217_728);++ assert.strictEqual(Number(row.ttl_ms), 30_000 + 600_000);++ assert.strictEqual(row.job.args.race, raceId);++ assert.strictEqual(row.job.args.play, 'race');++ assert.strictEqual(row.job.args.kind, 'chat');++ assert.deepStrictEqual(row.job.parent, { type: 'race', id: raceId });++ assert.ok(entrants.includes(row.job.args.entrant));++ assert.ok(['a', 'b'].includes(row.job.args.role));++ }++ assert.deepStrictEqual(rows.map((row) => row.job.args.role).sort(), ['a', 'b']);++ sideEntrants = Object.fromEntries(rows.map((row) => [row.job.args.role, row.job.args.entrant]));++ assert.deepStrictEqual(new Set(rows.map((row) => row.job.args.entrant)), new Set(entrants));++ assert.strictEqual((await t.outboxRows('run.job.queued')).filter((e) => e.payload.job_id === rows[0].id || e.payload.job_id === rows[1].id).length, 2);++ });++++ await check('GET /races/:id shows the sides\' output as it streams, with no names before the judgement; another person gets 404', async () => {++ const rows = await raceRows(raceId);++ t.stream.output(rows[0].id, 'The sky is blue because ');++ t.stream.output(rows[0].id, 'Rayleigh scattering favours short wavelengths.');++ t.stream.output(rows[1].id, 'Shorter wavelengths scatter more.');++ const mine = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));++ assert.strictEqual(mine.status, 200, mine.text);++ assert.strictEqual(mine.json.state, 'running');++ assert.match(mine.json.sides.find((s) => s.run_id === rows[0].id).output, /Rayleigh scattering/);++ assert.match(mine.json.sides.find((s) => s.run_id === rows[1].id).output, /Shorter wavelengths/);++ assert.strictEqual(mine.json.revealed, false);++ assert.strictEqual(mine.json.names, null);++ assert.ok(!JSON.stringify(mine.json).includes('model:gpt-5-mini'));++ const other = await t.call('GET', `/api/v1/races/${raceId}`, as(bo, P));++ assert.strictEqual(other.status, 404, other.text);++ assert.strictEqual(other.json.code, 'run.race_not_found');++ const anotherService = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P, { sub: 'svc:actor' }));++ assert.strictEqual(anotherService.status, 404, anotherService.text);++ });++++ await check('a judged side\'s receipts are the job\'s outcome and wall clock', async () => {++ const jobs = require('../server/jobs/store');++ const rows = await raceRows(raceId);++ for (const row of rows) {++ await t.db.tx(async (tx) => jobs.settle(tx, row.id, {++ state: 'succeeded', result: { answer: `answer ${row.job.args.role}` }, wallMs: 1234, at: Date.now(),++ }));++ }++ const r = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));++ assert.strictEqual(r.status, 200, r.text);++ assert.strictEqual(r.json.state, 'done');++ for (const side of r.json.sides) {++ assert.deepStrictEqual(side.receipts, { outcome: 'succeeded', wall_ms: 1234, length_chars: 8 });++ }++ });++++ await check('only the person who started it, through the same service, may judge', async () => {++ const notMine = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(bo, P) });++ assert.strictEqual(notMine.status, 404, notMine.text);++ assert.strictEqual(notMine.json.code, 'run.race_not_found');++ const otherService = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(ada, P, { sub: 'svc:actor' }) });++ assert.strictEqual(otherService.status, 404, otherService.text);++ const service = await t.call('POST', `/api/v1/races/${raceId}/judgement`, {++ body: { winner: 'a' },++ token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.submit'], project_id: P }),++ });++ assert.strictEqual(service.status, 404, service.text);++ const bad = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'e' }, ...as(ada, P) });++ assert.strictEqual(bad.status, 400, bad.text);++ assert.strictEqual(bench.calls.length, 0, 'nothing reached Bench');++ });++++ await check('the judgement records one valid bench.match@1 (arena, shown order, blind, receipts, keyed person) and reveals the names', async () => {++ const r = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(ada, P) });++ assert.strictEqual(r.status, 201, r.text);++ assert.strictEqual(bench.calls.length, 1);++ const match = bench.calls[0];++ assert.ok(contracts.validate('bench.match@1', match).valid, JSON.stringify(match));++ assert.match(match.id, /^mat_[0-9A-HJKMNP-TV-Z]{26}$/);++ assert.strictEqual(match.source, 'arena');++ assert.deepStrictEqual(match.task, { kind: 'chat', prompt_hash: sha256(prompt) });++ assert.deepStrictEqual(match.shown, ['a', 'b']);++ assert.strictEqual(match.blind, true);++ assert.deepStrictEqual(match.budget, { usd: 0.1, ms: 30_000, machine_class: 'linux' });++ assert.deepStrictEqual(match.slots.map((s) => s.slot), ['a', 'b']);++ assert.deepStrictEqual(new Set(match.slots.map((s) => s.entrant)), new Set(entrants));++ for (const slot of match.slots) {++ assert.match(slot.run_id, /^job_/);++ assert.deepStrictEqual(slot.receipts, { outcome: 'succeeded', wall_ms: 1234, length_chars: 8 });++ }++ assert.strictEqual(match.judgements.length, 1);++ assert.strictEqual(match.judgements[0].judge, 'human_pairwise');++ assert.strictEqual(match.judgements[0].winner, 'a');++ assert.deepStrictEqual(match.judgements[0].by, { type: 'person', id_hash: t.app.locals.races.personHash(`user:${ada}`) });++ assert.match(match.judgements[0].by.id_hash, /^[a-f0-9]{64}$/);++ assert.ok(!JSON.stringify(match).includes(ada), 'never the person id');++ assert.deepStrictEqual(match.play, { id: 'race', run_id: raceId });++++ assert.strictEqual(r.json.revealed, true);++ assert.strictEqual(r.json.state, 'judged');++ assert.strictEqual(r.json.judgement.winner, 'a');++ assert.strictEqual(r.json.judgement.match_id, match.id);++ assert.deepStrictEqual(r.json.names, sideEntrants);++ for (const side of r.json.sides) assert.strictEqual(side.entrant, sideEntrants[side.slot]);++ const shown = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));++ assert.strictEqual(shown.status, 200, shown.text);++ assert.ok(JSON.stringify(shown.json).includes('model:gpt-5-mini'));++ assert.strictEqual(shown.json.names.a, sideEntrants.a);++ });++++ await check('a second judgement is refused and Bench is not called again', async () => {++ const again = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'b' }, ...as(ada, P) });++ assert.strictEqual(again.status, 409, again.text);++ assert.strictEqual(again.json.code, 'run.race_already_judged');++ assert.strictEqual(bench.calls.length, 1);++ });++++ await check('matchmaker: true picks two entrants of different families from the configured list', async () => {++ const r = await t.call('POST', '/api/v1/races', {++ body: { prompt: 'Write a haiku about the sea.', kind: 'code', matchmaker: true, budget: { wall_ms: 40_000, cpu_ms: 40_000, mem_bytes: 268_435_456 } },++ ...as(ada, P),++ });++ assert.strictEqual(r.status, 201, r.text);++ const made = r.json;++ assert.strictEqual(made.sides.length, 2);++ assert.strictEqual(made.revealed, false);++ assert.ok(!JSON.stringify(made).includes('model:'));++ const judged = await t.call('POST', `/api/v1/races/${made.id}/judgement`, { body: { winner: 'tie' }, ...as(ada, P) });++ assert.strictEqual(judged.status, 201, judged.text);++ const pair = Object.values(judged.json.names);++ assert.strictEqual(new Set(pair).size, 2);++ for (const id of pair) assert.ok(t.config.races.entrants.includes(id), `${id} came from RUN_RACE_ENTRANTS`);++ assert.notStrictEqual(familyOf(pair[0]), familyOf(pair[1]), `different families: ${pair.join(' vs ')}`);++ assert.strictEqual(bench.calls.length, 2);++ assert.strictEqual(bench.calls[1].judgements[0].winner, 'tie');++ });++++ await check('a malformed race request is refused 400 and starts no child job, and a service for itself is refused', async () => {++ const before = (await t.db.many('SELECT count(*)::int AS n FROM run_jobs WHERE project_id = $1', [P]))[0].n;++ const cases = [++ { ...body, entrants: ['model:gpt-5-mini'] },++ { ...body, matchmaker: true },++ { ...body, entrants: ['not-an-entrant', 'model:glm-5.3'] },++ { prompt: '', kind: 'chat', entrants, budget: body.budget },++ { ...body, budget: undefined },++ { ...body, kind: 'poetry' },++ { ...body, extra: true },++ ];++ for (const bad of cases) {++ const r = await t.call('POST', '/api/v1/races', { body: bad, ...as(ada, P) });++ assert.strictEqual(r.status, 400, `${JSON.stringify(bad)}: ${r.text}`);++ assert.strictEqual(r.json.code, 'run.invalid_request');++ }++ const after = (await t.db.many('SELECT count(*)::int AS n FROM run_jobs WHERE project_id = $1', [P]))[0].n;++ assert.strictEqual(after, before, 'no child job was made by a refused race');++ const self = await t.call('POST', '/api/v1/races', {++ body: { ...body, entrants: ['model:gpt-5-mini', 'model:glm-5.3'] },++ token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.submit'], project_id: P }),++ });++ assert.strictEqual(self.status, 403, self.text);++ assert.strictEqual(self.json.code, 'run.race_person_required');++ });++}++++async function main() {++ const bench = fakeBench();++ const t = await boot({++ env: {++ OV_OAUTH_CLIENT_SECRET: 'run-test-secret',++ RUN_RACE_ENTRANTS: 'model:gpt-5-mini,model:glm-5.3,model:deepseek-flash,agent:codex+gpt-5-mini#0123abcd',++ },++ appOpts: { bench },++ });++ try { await checks(t, bench); } finally { await t.close(); }++ done();++}++++main().catch((e) => { console.error(e); process.exit(1); });
@@ -109,6 +109,19 @@ OV_OAUTH_CLIENT_SECRET=
@@ -109,6 +109,19 @@ OV_OAUTH_CLIENT_SECRET=
#RUN_BILLING_INTERVAL_MS=5000
#RUN_BILLING_INTERVAL_MS=5000
#RUN_BILLING_TIMEOUT_MS=8000
#RUN_BILLING_TIMEOUT_MS=8000
+# ── The Arena's race (compare-arena-ecosystem.md §3) ──
+# POST /api/v1/races starts one blind child job per entrant; the judgement is a bench.match@1 recorded on
+# Bench (POST /api/v1/matches) with Run's Network client token. That client needs bench.judgement.cast.
+# Bench has no matchmaker endpoint yet, so matchmaker: true picks two entrants of different model families
+# from this list (comma-separated bench.entrant@1 ids). Unset: matchmaker: true answers 422.
+#RUN_RACE_ENTRANTS=model:gpt-5-mini,model:glm-5.3,model:deepseek-flash
+# The key for the keyed hash of the person in a judgement (never the person's id). Unset: Run uses its
+# Network client secret; set a dedicated key to rotate it without touching the Network client.
+#RUN_RACE_HASH_KEY=
+#RUN_BENCH_URL=https://bench.openvibe.services
+#RUN_BENCH_AUDIENCE=openvibe.bench
+#RUN_BENCH_TIMEOUT_MS=8000
+
# ── IndexNow (search engines are told when the public pages change) ──
# ── IndexNow (search engines are told when the public pages change) ──
# 8-128 letters and digits. Unset: nothing is sent and GET /<key>.txt is 404. When set, the key file is served there.
# 8-128 letters and digits. Unset: nothing is sent and GET /<key>.txt is 404. When set, the key file is served there.
#INDEXNOW_KEY=
#INDEXNOW_KEY=
@@ -6,6 +6,17 @@ Notable changes to OpenVibe.Run. The service is unreleased; everything below is
@@ -6,6 +6,17 @@ Notable changes to OpenVibe.Run. The service is unreleased; everything below is
### Added
### Added
+- The Arena's race on the one engine (compare-arena-ecosystem.md §3, contracts 0.422.0): the `race` play as
+ `agents.play@1` (one role per side, one parallel phase, an even budget split, `blind: true`, stop on person), listed
+ by `GET /api/v1/plays`; `POST /api/v1/races` starts one child `linux` job per side (2–4) under one parent race with
+ the same share of the budget each and answers the sides labelled `a, b, …` without names; `GET /api/v1/races/:id`
+ returns each side's output so far and its receipts when ended, the names only after a judgement;
+ `POST /api/v1/races/:id/judgement` records one `bench.match@1` on Bench (source `arena`, shown order, `blind: true`,
+ the receipts, a `human_pairwise` judgement with an HMAC of the person) before revealing the names. Only the person
+ who started the race, through the same first-party service, may read or judge it. `matchmaker: true` picks two
+ entrants of different model families from `RUN_RACE_ENTRANTS` while Bench has no matchmaker endpoint. Migration
+ 0020 adds `run_races`. The race request/answer has no released contract yet, so it is validated in
+ `server/races/service.js`; everything it stores or sends is validated where a contract exists.
- With `RUN_SESSION_ORIGINS_ENABLED=1`, a browser session may name exact HTTP(S) origins at creation. Run stores the
- With `RUN_SESSION_ORIGINS_ENABLED=1`, a browser session may name exact HTTP(S) origins at creation. Run stores the
list with the session, places it on a user-owned desk that can enforce the boundary, and sends the same list when
list with the session, places it on a user-owned desk that can enforce the boundary, and sends the same list when
leasing that desk. Commands cannot replace the list. The switch is off by default.
leasing that desk. Commands cannot replace the list. The switch is off by default.
@@ -98,6 +98,9 @@ placed from the Fabric's offers and actually runs — the poller in `server/jobs
@@ -98,6 +98,9 @@ placed from the Fabric's offers and actually runs — the poller in `server/jobs
(`GET /api/v1/offers`, public); Run's client `run` mints the events relay's service token, its Bot token and the
(`GET /api/v1/offers`, public); Run's client `run` mints the events relay's service token, its Bot token and the
token for project memberships (`GET /internal/projects/:project/members/:subject`, project delegation).
token for project memberships (`GET /internal/projects/:project/members/:subject`, project delegation).
- OpenVibe.Events: `run.job.*` events go to `events.event.publish`; unset, they wait in `run_events_outbox`.
- OpenVibe.Events: `run.job.*` events go to `events.event.publish`; unset, they wait in `run_events_outbox`.
+- OpenVibe.Bench (`RUN_BENCH_URL`, default `https://bench.openvibe.services`): a race's judgement is recorded as a
+ `bench.match@1` on `POST /api/v1/matches` (`bench.judgement.cast`, audience `openvibe.bench`). Bench has no
+ matchmaker endpoint yet; `matchmaker: true` falls back to `RUN_RACE_ENTRANTS`.
- OpenVibe.Bot (`RUN_BOT_URL`, `RUN_POLL_MS`): the node link and the meter. Run sends each job to it
- OpenVibe.Bot (`RUN_BOT_URL`, `RUN_POLL_MS`): the node link and the meter. Run sends each job to it
(`bot.job.dispatch`, audience `openvibe.bot`, minted from `OV_OAUTH_CLIENT_ID`/`OV_OAUTH_CLIENT_SECRET` against
(`bot.job.dispatch`, audience `openvibe.bot`, minted from `OV_OAUTH_CLIENT_ID`/`OV_OAUTH_CLIENT_SECRET` against
`OV_NETWORK_INTERNAL_URL` and cached until 60 s before expiry) and mirrors its answer. Unset the client secret and
`OV_NETWORK_INTERNAL_URL` and cached until 60 s before expiry) and mirrors its answer. Unset the client secret and
@@ -206,6 +209,39 @@ first-party `svc:` token holding the capability and naming them in `X-OV-Subject
@@ -206,6 +209,39 @@ first-party `svc:` token holding the capability and naming them in `X-OV-Subject
queued until an offer appears or its `ttl_ms` ends it `expired` with `run.job.unplaceable`. Offers are cached for `RUN_OFFERS_TTL_MS`; a Network that does not answer
queued until an offer appears or its `ttl_ms` ends it `expired` with `run.job.unplaceable`. Offers are cached for `RUN_OFFERS_TTL_MS`; a Network that does not answer
is retried, never a reason to fail a job.
is retried, never a reason to fail a job.
+## Races (the Arena on the one engine)
+
+A race is the Arena's round (compare-arena-ecosystem.md §3): the same prompt, two to four entrants, one child job per
+entrant on the same machine class (`linux`), the same share of the parent budget each, and no entrant named until a
+person judges. The play is data — `agents.play@1`, id `race`, one role per side, one parallel phase, `blind: true`,
+`stop.when: person` — listed by `GET /api/v1/plays`. Contracts 0.422.0 has no `run.race@1`, so the request and answer
+are validated in `server/races/service.js`; the `bench.match@1` the judgement records is validated before it is sent.
+
+| Method and path | Capability | Request → answer |
+|-----------------------------------|------------------|------------------|
+| `GET /api/v1/plays` | `run.job.list` | → `{ plays: agents.play@1[] }` |
+| `POST /api/v1/races` | `run.job.submit` | `{ prompt, kind: chat\|code\|design, entrants: [2–4 bench.entrant ids] \| matchmaker: true, budget: { usd?, wall_ms?, cpu_ms?, mem_bytes? } }` → 201 race |
+| `GET /api/v1/races/:id` | `run.job.read` | each side's output so far (the job stream's ring), its state and, once ended, its receipts; names only after a judgement |
+| `POST /api/v1/races/:id/judgement` | `run.job.submit` | `{ winner: a\|b\|tie\|both_bad }` → 201 race with the names revealed |
+
+- **Who may start and judge.** A person, through a first-party service (`X-OV-Subject` + `X-OV-Project`): Run records
+ the person as the requester and the service as `via`. Only that person, through that service and project, may read
+ or judge it; anyone else gets the same `404 run.race_not_found` a missing race does.
+- **The split.** `budget` is the round's total and each side gets `floor(total / sides)` of wall, CPU and memory
+ (memory floored to bytes); `usd` is split and recorded, not metered by Run. A total too small for every side is
+ `422 run.limits.exceeded`; the per-side limits then pass the project caps like any job.
+- **The child.** `class: linux`, `args: { play, role, race, kind, prompt, entrant, budget }`, with
+ `job.parent = { type: 'race', id }` (dropped by `frameOf`, so Bot never receives it). Placement, metering, events
+ and the stream are the ordinary job path.
+- **The judgement.** One `bench.match@1`, source `arena`, the shown order (`a, b, …`, randomised when the race is
+ created), `blind: true`, each ended side's receipts, `play: { id: 'race', run_id }`, and one `human_pairwise`
+ judgement whose `by.id_hash` is an HMAC of the person (`RUN_RACE_HASH_KEY`, falling back to the Network client
+ secret). Bench records it before Run writes the judgement and reveals the names; a Bench that cannot answer leaves
+ the race blind and retryable with the same match id.
+- **The matchmaker.** Bench has no matchmaker endpoint yet (checked 2026-11-13). `matchmaker: true` picks two entrants
+ of different model families at random from `RUN_RACE_ENTRANTS`; unset, it answers `422
+ run.race_matchmaker_unavailable` and names the setting.
+
## Tests
## Tests
```
```
@@ -7,11 +7,11 @@
@@ -7,11 +7,11 @@
"runtime": "node 22, express 4, PostgreSQL 18 (openvibe-sdk/db) + Valkey 9",
"runtime": "node 22, express 4, PostgreSQL 18 (openvibe-sdk/db) + Valkey 9",
"port": 4920,
"port": 4920,
"domain": "openvibe.run",
"domain": "openvibe.run",
- "contracts": "openvibe-contracts v0.419.0",
+ "contracts": "openvibe-contracts v0.422.0",
"sdk": "openvibe-sdk v0.161.0",
"sdk": "openvibe-sdk v0.161.0",
"shared": "openvibe-shared v4.232.0",
"shared": "openvibe-shared v4.232.0",
"packages": [
"packages": [
- "openvibe-contracts v0.419.0",
+ "openvibe-contracts v0.422.0",
"openvibe-sdk v0.161.0",
"openvibe-sdk v0.161.0",
"openvibe-shared v4.232.0"
"openvibe-shared v4.232.0"
],
],
@@ -56,7 +56,9 @@
@@ -56,7 +56,9 @@
"plan T14 R4a built, not deployed: the managed-browser provider adapter (server/dispatch/browser-provider.js; RUN_BROWSER_PROVIDER_URL / _API_KEY / _TIMEOUT_MS, off while unset) places class browser jobs on kind=provider offers named browser-provider, validates every answer against browser-provider.job-create-result@1 / job-state@1 and never adopts a malformed one; Run meters those jobs itself, the provider's claimed wall time clamped to Run's observed duration + 5 s, and relays the readings from run_usage_outbox (migrations/0003_usage_outbox.sql) to Billing (RUN_BILLING_URL; unset, they wait and the jobs still run)",
"plan T14 R4a built, not deployed: the managed-browser provider adapter (server/dispatch/browser-provider.js; RUN_BROWSER_PROVIDER_URL / _API_KEY / _TIMEOUT_MS, off while unset) places class browser jobs on kind=provider offers named browser-provider, validates every answer against browser-provider.job-create-result@1 / job-state@1 and never adopts a malformed one; Run meters those jobs itself, the provider's claimed wall time clamped to Run's observed duration + 5 s, and relays the readings from run_usage_outbox (migrations/0003_usage_outbox.sql) to Billing (RUN_BILLING_URL; unset, they wait and the jobs still run)",
"truthful readiness (GET /api/ready: db required; valkey, Network JWKS, Events, Bot reachability and the poller's own status reported, never claimed) and the loopback-only /metrics gauges (jobs queued, active, ended, outbox backlog, streams)",
"truthful readiness (GET /api/ready: db required; valkey, Network JWKS, Events, Bot reachability and the poller's own status reported, never claimed) and the loopback-only /metrics gauges (jobs queued, active, ended, outbox backlog, streams)",
"plan T11 step 4 built, not deployed: /llms.txt and /llms-full.txt (server/web/discovery.js, public pages and docs only, never the API, admin, a job or a stream) and IndexNow (INDEXNOW_KEY: serves GET /<key>.txt and pings the public pages once at boot; unset, the key route is 404 and nothing is sent)",
"plan T11 step 4 built, not deployed: /llms.txt and /llms-full.txt (server/web/discovery.js, public pages and docs only, never the API, admin, a job or a stream) and IndexNow (INDEXNOW_KEY: serves GET /<key>.txt and pings the public pages once at boot; unset, the key route is 404 and nothing is sent)",
- "a person's own machine (user-owned offer read from Network's internal list) runs only that person's jobs naming user-owned; a delegating service cancels and streams (by ticket) the jobs it started for a person"
+ "a person's own machine (user-owned offer read from Network's internal list) runs only that person's jobs naming user-owned; a delegating service cancels and streams (by ticket) the jobs it started for a person",
+ "the Arena's race on the one engine (compare-arena-ecosystem.md §3, server/races/): POST /api/v1/races starts one child linux job per side under one parent race with the same share of the budget each, GET /api/v1/races/:id returns each side's output so far and its receipts, and POST /api/v1/races/:id/judgement records a bench.match@1 on Bench (source arena, shown order, blind true, human_pairwise with a keyed person hash) before the names are revealed; only the person who started it, through the same service, may read or judge",
+ "the race play as data (agents.play@1 1.1.0, server/plays/race.js): id race, one role per side, one parallel phase, an even budget split, blind true and stop when person, listed by GET /api/v1/plays; the matchmaker falls back to two entrants of different families from RUN_RACE_ENTRANTS while Bench has no matchmaker endpoint (Contracts 0.422.0 has no run.race@1 yet, so the race request/answer is validated in server/races/service.js)"
],
],
"updated": "2026-11-13"
"updated": "2026-11-13"
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,29 @@
@@ -0,0 +1,29 @@
+-- phase: expand
+-- The Arena's race (compare-arena-ecosystem.md §3, 2026-11-13): one parent record per round, its sides in
+-- the shown order, the entrant each side ran and the child job that answers it, the person who started it,
+-- and the one human judgement that ends the round. The parent itself is Run's record; the child runs stay
+-- run_jobs rows (one per side), so placement, metering and the job stream are unchanged.
+--
+-- SIDES is [{ slot, entrant, job_id }] in the order the sides are shown (a, b, …); ENTRANT is a
+-- bench.entrant@1 id and is never answered before JUDGEMENT is written. MATCH_ID is the bench.match@1 id
+-- minted with the race, so a retry after a lost answer records the same match at Bench rather than a second
+-- one. JUDGEMENT is { winner, judge, at, match_id, by_hash } once Bench accepted the match.
+CREATE TABLE IF NOT EXISTS run_races (
+ id text PRIMARY KEY CHECK (id ~ '^rac_[0-9A-HJKMNP-TV-Z]{26}$'),
+ project_id text NOT NULL,
+ requester text NOT NULL, -- the person who started it: user:<subject>
+ via text NOT NULL, -- the first-party service they started it through
+ kind text NOT NULL CHECK (kind IN ('chat', 'code', 'design')),
+ prompt text NOT NULL,
+ prompt_hash text NOT NULL CHECK (prompt_hash ~ '^[a-f0-9]{64}$'),
+ sides jsonb NOT NULL,
+ budget jsonb NOT NULL, -- { total: {...}, per_side: {...} }; the same share for every side
+ play jsonb NOT NULL, -- the race's agents.play@1 document, with the sides that ran
+ match_id text NOT NULL CHECK (match_id ~ '^mat_[0-9A-HJKMNP-TV-Z]{26}$'),
+ blind boolean NOT NULL DEFAULT true,
+ judgement jsonb,
+ judged_at bigint,
+ created_at bigint NOT NULL,
+ updated_at bigint NOT NULL
+);
+CREATE INDEX IF NOT EXISTS run_races_owner ON run_races (project_id, requester, via, created_at DESC);
@@ -15,7 +15,7 @@
@@ -15,7 +15,7 @@
"express-rate-limit": "^7.5.0",
"express-rate-limit": "^7.5.0",
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0",
"pg": "^8.23.0",
"pg": "^8.23.0",
@@ -1229,9 +1229,9 @@
@@ -1229,9 +1229,9 @@
}
}
},
},
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
- "version": "0.419.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",
- "integrity": "sha512-VxrTdeHchT8gvl2wWC9ASaId92en7ZKfJ8F/0hKzCAD0f2Y00rPrqgBWF9BNPoLGARoyQvsMbhBEhSkhxGNnjA==",
+ "version": "0.422.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0",
+ "integrity": "sha512-xQQq7NQLM9iQve8dR4Hc2EUp75x/FZYfAsHumsmf6dGHjIy05PMX0PlYmxw5L52VXKOj+r7mRwlcehRqgMoLSw==",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
"ajv": "^8.17.1",
"ajv": "^8.17.1",
@@ -26,7 +26,7 @@
@@ -26,7 +26,7 @@
"express-rate-limit": "^7.5.0",
"express-rate-limit": "^7.5.0",
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"iovalkey": "^0.4.0",
"iovalkey": "^0.4.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.419.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.422.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.161.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v4.232.0",
"pg": "^8.23.0",
"pg": "^8.23.0",
new file mode 100644
new file mode 100644
@@ -0,0 +1,68 @@
@@ -0,0 +1,68 @@
+'use strict';
+
+/**
+ * The Arena's race routes on the Run API (compare-arena-ecosystem.md §3):
+ *
+ * | Method & path | Capability | Request → answer |
+ * |---------------------------------------|------------------|-------------------------------------------------------|
+ * | GET /plays | run.job.list | common.no-body@1 → { plays: agents.play@1[] } |
+ * | POST /races | run.job.submit | { prompt, kind, entrants | matchmaker, budget } → 201 race |
+ * | GET /races/:id | run.job.read | sides' live output, receipts; names only after a judgement |
+ * | POST /races/:id/judgement | run.job.submit | { winner } → 201 race with the names revealed |
+ *
+ * Contracts 0.422.0 has no run.race@1 yet, so the race request and answer are validated by
+ * server/races/service.js by hand (and the bench.match@1 it records is validated against its released
+ * contract). The capabilities are the job ones on purpose — Run has six, and a race is a group submit and
+ * a group read; inventing run.race.* here would be a capability no manifest grants.
+ *
+ * A race belongs to the person who started it and the first-party service they started it through: a
+ * delegated token (X-OV-Subject + X-OV-Project) is required. Another person, service or project gets the
+ * same 404 a missing race does, and only that person may judge.
+ */
+const express = require('express');
+const { fail } = require('../util');
+
+function racesRouter({ races, apiAuth }) {
+ const r = express.Router();
+ const wrap = (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next);
+ const guard = apiAuth.requireCapability;
+
+ // ── GET /plays — run.job.list ───────────────────────────────────────────────
+ // The race play is data (agents.play@1), so the same document lists here and drives a race.
+ r.get('/plays', guard('run.job.list'), wrap(async (_req, res) => {
+ res.json({ plays: races.plays() });
+ }));
+
+ // ── POST /races — run.job.submit ────────────────────────────────────────────
+ r.post('/races', guard('run.job.submit'), wrap(async (req, res) => {
+ const scope = apiAuth.requireProject(req, { delegated: true });
+ const race = await races.start({
+ projectId: scope.project, requester: scope.requester, via: scope.via,
+ body: req.body, traceparent: req.ov && req.ov.traceparent,
+ });
+ res.status(201).json(race);
+ }));
+
+ // ── GET /races/:id — run.job.read ───────────────────────────────────────────
+ r.get('/races/:id', guard('run.job.read'), wrap(async (req, res) => {
+ const scope = apiAuth.requireProject(req, { delegated: true });
+ const race = await races.get({ projectId: scope.project, requester: scope.requester, via: scope.via, id: req.params.id });
+ if (!race) fail(404, 'run.race_not_found', 'no such race');
+ res.json(race);
+ }));
+
+ // ── POST /races/:id/judgement — run.job.submit ──────────────────────────────
+ r.post('/races/:id/judgement', guard('run.job.submit'), wrap(async (req, res) => {
+ const scope = apiAuth.requireProject(req, { delegated: true });
+ const race = await races.judge({
+ projectId: scope.project, requester: scope.requester, via: scope.via,
+ id: req.params.id, body: req.body,
+ });
+ if (!race) fail(404, 'run.race_not_found', 'no such race');
+ res.status(201).json(race);
+ }));
+
+ return r;
+}
+
+module.exports = { racesRouter };
@@ -63,6 +63,12 @@ function createApp(opts = {}) {
@@ -63,6 +63,12 @@ function createApp(opts = {}) {
const membership = opts.membership || createProjectMembership(config, { fetchImpl, now });
const membership = opts.membership || createProjectMembership(config, { fetchImpl, now });
const fillNotifier = opts.fillNotifier || createFillNotifier(config, { fetchImpl, now });
const fillNotifier = opts.fillNotifier || createFillNotifier(config, { fetchImpl, now });
const apiAuth = createApiAuth({ config, keys, userAuth, membership, log });
const apiAuth = createApiAuth({ config, keys, userAuth, membership, log });
+ // The Arena's race (compare-arena-ecosystem.md §3): one parent, one child job per side, the names after
+ // a Bench judgement. A test injects its own bench recorder; production talks to OpenVibe.Bench.
+ const bench = opts.bench || require('./races/bench').createBenchRecorder({ config, fetchImpl, now, log });
+ const races = opts.races || require('./races/service').createRaces({
+ config, db, jobs: require('./jobs/store'), events, stream, bench, now, log,
+ });
// The checks scheduler (server/checks/): the GitHub App's owner page, the webhook and the agreement
// The checks scheduler (server/checks/): the GitHub App's owner page, the webhook and the agreement
// query. It owns no module-level state and starts nothing here; server/index.js starts its tick with
// query. It owns no module-level state and starts nothing here; server/index.js starts its tick with
// the other background jobs. Without RUN_SECRETS_KEY the page says so and the webhook answers 503.
// the other background jobs. Without RUN_SECRETS_KEY the page says so and the webhook answers 503.
@@ -151,6 +157,7 @@ function createApp(opts = {}) {
@@ -151,6 +157,7 @@ function createApp(opts = {}) {
apiAuth.middleware,
apiAuth.middleware,
require('./compute/routes').computeRouter({ broker, config }),
require('./compute/routes').computeRouter({ broker, config }),
sessionsRouter({ sessions }),
sessionsRouter({ sessions }),
+ require('./api/races').racesRouter({ races, apiAuth }),
v1Router({ config, db, apiAuth, stream, tickets, events, dispatch, now, log }));
v1Router({ config, db, apiAuth, stream, tickets, events, dispatch, now, log }));
// ── The public site (openvibe.run): the front page, the legal pages and the Frame's shared assets. ──
// ── The public site (openvibe.run): the front page, the legal pages and the Frame's shared assets. ──
@@ -240,7 +247,7 @@ function createApp(opts = {}) {
@@ -240,7 +247,7 @@ function createApp(opts = {}) {
return res.status(500).type('text/plain').send('Something went wrong\n');
return res.status(500).type('text/plain').send('Something went wrong\n');
});
});
- Object.assign(app.locals, { config, db, indexnow, valkey, keys, outbox, events, stream, tickets, dispatch, apiAuth, metrics, desks, sessions, broker, checks });
+ Object.assign(app.locals, { config, db, indexnow, valkey, keys, outbox, events, stream, tickets, dispatch, apiAuth, metrics, desks, sessions, broker, races, bench, checks });
return app;
return app;
}
}
@@ -160,13 +160,25 @@ function loadConfig(env = process.env) {
@@ -160,13 +160,25 @@ function loadConfig(env = process.env) {
},
},
// Placement A1: Bench's quality priors, cached an hour, used by place() when the option is a model Bench has
// Placement A1: Bench's quality priors, cached an hour, used by place() when the option is a model Bench has
- // seen. Off in tests unless asked for, so a unit test never reaches the real Bench.
+ // seen. Off in tests unless asked for, so a unit test never reaches the real Bench. The race judgement
+ // records a bench.match@1 on the same host (POST /api/v1/matches, capability bench.judgement.cast).
bench: {
bench: {
url: trim(env.RUN_BENCH_URL || 'https://bench.openvibe.services'),
url: trim(env.RUN_BENCH_URL || 'https://bench.openvibe.services'),
+ audience: env.RUN_BENCH_AUDIENCE || 'openvibe.bench',
+ timeoutMs: Math.max(100, int(env.RUN_BENCH_TIMEOUT_MS, 8000)),
priors: env.RUN_BENCH_PRIORS != null ? bool(env.RUN_BENCH_PRIORS) : nodeEnv !== 'test',
priors: env.RUN_BENCH_PRIORS != null ? bool(env.RUN_BENCH_PRIORS) : nodeEnv !== 'test',
priorsTtlMs: Math.max(0, int(env.RUN_BENCH_PRIORS_TTL_MS, 60 * 60_000)),
priorsTtlMs: Math.max(0, int(env.RUN_BENCH_PRIORS_TTL_MS, 60 * 60_000)),
},
},
+ // The Arena's race (compare-arena-ecosystem.md §3). Bench has no matchmaker endpoint yet, so
+ // `matchmaker: true` picks two entrants of different families from this configured list. `hashKey`
+ // keys the person's id hash in a bench.match@1 judgement: never the person's id, and stable across
+ // restarts (it falls back to Run's Network client secret, which production always has).
+ races: {
+ entrants: String(env.RUN_RACE_ENTRANTS || '').split(',').map((s) => s.trim()).filter(Boolean),
+ hashKey: env.RUN_RACE_HASH_KEY || env.OV_OAUTH_CLIENT_SECRET || 'run-race-development-key',
+ },
+
// The external managed-browser provider (plan T14 R4a, server/dispatch/browser-provider.js): class browser
// The external managed-browser provider (plan T14 R4a, server/dispatch/browser-provider.js): class browser
// jobs placed on a kind=provider offer named `browser-provider` go to its API instead of Bot. Off while
// jobs placed on a kind=provider offer named `browser-provider` go to its API instead of Bot. Off while
// RUN_BROWSER_PROVIDER_URL or the key is unset: no provider offer is placed on, so a browser job waits.
// RUN_BROWSER_PROVIDER_URL or the key is unset: no provider offer is placed on, so a browser job waits.
@@ -156,6 +156,11 @@ async function mint(db, body, { projectId, requester, via = null, idempotencyKey
@@ -156,6 +156,11 @@ async function mint(db, body, { projectId, requester, via = null, idempotencyKey
/** Project-scoped get: null when the job does not exist or belongs to another project (→ 404, never 403). */
/** Project-scoped get: null when the job does not exist or belongs to another project (→ 404, never 403). */
const get = (db, id, projectId = null) => db.maybe(`SELECT * FROM run_jobs WHERE id = $1${projectId ? ' AND project_id = $2' : ''}`, projectId ? [id, projectId] : [id]);
const get = (db, id, projectId = null) => db.maybe(`SELECT * FROM run_jobs WHERE id = $1${projectId ? ' AND project_id = $2' : ''}`, projectId ? [id, projectId] : [id]);
+/** Several jobs by id (a race's child runs); the caller maps them by id, and an absent one is simply not in the result. */
+const getMany = (db, ids) => (ids && ids.length
+ ? db.many('SELECT * FROM run_jobs WHERE id = ANY($1::text[])', [ids])
+ : Promise.resolve([]));
+
/** A delegated caller's get: the job only if it is this project's, this person's, through this service (else null → 404). */
/** A delegated caller's get: the job only if it is this project's, this person's, through this service (else null → 404). */
const getOwn = (db, id, { projectId, requester, via }) => db.maybe(
const getOwn = (db, id, { projectId, requester, via }) => db.maybe(
'SELECT * FROM run_jobs WHERE id = $1 AND project_id = $2 AND requester = $3 AND via = $4', [id, projectId, requester, via]);
'SELECT * FROM run_jobs WHERE id = $1 AND project_id = $2 AND requester = $3 AND via = $4', [id, projectId, requester, via]);
@@ -349,7 +354,7 @@ function readResult(row) {
@@ -349,7 +354,7 @@ function readResult(row) {
module.exports = {
module.exports = {
// The store surface the API and the dispatcher bridge (plan T14 step 6) use.
// The store surface the API and the dispatcher bridge (plan T14 step 6) use.
- put, mint, get, getOwn, list, adminList, cancel, mirror, settle, toJob, readResult, frameOf,
+ put, mint, get, getMany, getOwn, list, adminList, cancel, mirror, settle, toJob, readResult, frameOf,
// The poller's reads (waiting → placed → active → ended).
// The poller's reads (waiting → placed → active → ended).
waiting, active, dueTtl,
waiting, active, dueTtl,
};
};
@@ -109,6 +109,21 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {
@@ -109,6 +109,21 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {
return { ended: false, unsubscribe: () => { ring.subscribers.delete(sub); ring.touched = now(); } };
return { ended: false, unsubscribe: () => { ring.subscribers.delete(sub); ring.touched = now(); } };
}
}
+ /**
+ * What a ring holds right now, without creating one: the output so far, the last state it announced
+ * and whether it closed. A race's GET reads this for each side, so polling the race shows the sides'
+ * answers as they stream (output is best effort, exactly as the stream contract says).
+ */
+ function snapshot(jobId) {
+ const ring = jobs.get(jobId);
+ if (!ring) return { state: null, output: '', ended: false };
+ return {
+ state: ring.state,
+ output: ring.events.filter((e) => e.type === 'output').map((e) => e.chunk).join(''),
+ ended: ring.ended,
+ };
+ }
+
const canAttach = (jobId) => ringOf(jobId).subscribers.size < maxPerJob;
const canAttach = (jobId) => ringOf(jobId).subscribers.size < maxPerJob;
const status = () => ({ jobs: jobs.size, streams: [...jobs.values()].reduce((n, r) => n + r.subscribers.size, 0), events: [...jobs.values()].reduce((n, r) => n + r.events.length, 0) });
const status = () => ({ jobs: jobs.size, streams: [...jobs.values()].reduce((n, r) => n + r.subscribers.size, 0), events: [...jobs.values()].reduce((n, r) => n + r.events.length, 0) });
@@ -121,7 +136,7 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {
@@ -121,7 +136,7 @@ function createJobStream({ config, now = () => Date.now(), log = console }) {
}
}
}
}
- return { state, output, seed, subscribe, canAttach, status, close, emit, prune, maxPerJob, ringOf };
+ return { state, output, seed, subscribe, snapshot, canAttach, status, close, emit, prune, maxPerJob, ringOf };
}
}
module.exports = { createJobStream, JOB_EVENT_TYPES };
module.exports = { createJobStream, JOB_EVENT_TYPES };
new file mode 100644
new file mode 100644
@@ -0,0 +1,63 @@
@@ -0,0 +1,63 @@
+'use strict';
+
+/**
+ * The race play — the Arena's round on the one engine (compare-arena-ecosystem.md §3, agent-engine.md
+ * §4): one child run per entrant, the same budget each, the sides blind until a person judges. The play
+ * is data (agents.play@1), so the same shape lists on Run, drives the orchestrator and can be written by a
+ * person or an agent.
+ *
+ * A race has 2 to 4 sides, so the play is built per race: one role per side (`a`, `b`, …), one parallel
+ * phase that runs them at once, `blind: true`, and `stop.when: person` — the round is over when the person
+ * votes, not when the last child ends. The budget split is an equal share each (the caller's total is
+ * divided by the number of sides in server/races/service.js; the contract wants the shares as data).
+ *
+ * The roles say "Side A", never a model or harness name: a role name is visible to the person before the
+ * judgement, so it must not carry the entrant.
+ */
+const contracts = require('openvibe-contracts');
+
+/** The most sides a race may have (bench.match@1 slots a–h; the Arena's own limit is 4). */
+const SLOTS = ['a', 'b', 'c', 'd'];
+/** Every side runs on the same machine class, so a race compares entrants on one environment. */
+const MACHINE_CLASS = 'linux';
+const PLAY_ID = 'race';
+
+/** The even share of a total budget, as the contracts' number (rounded, never a float surprise in a diff). */
+const shareOf = (n) => Number((1 / n).toFixed(6));
+
+/**
+ * racePlay(['a', 'b']) → an agents.play@1 document. `slots` is the sides this race actually has (2–4),
+ * so the roles and the split describe the race that ran, not a fixed example.
+ */
+function racePlay(slots = SLOTS) {
+ const sides = [...new Set((Array.isArray(slots) ? slots : []).filter((s) => SLOTS.includes(s)))];
+ if (sides.length < 2) throw new TypeError('a race play needs at least two sides (a, b)');
+ const share = shareOf(sides.length);
+ const play = {
+ id: PLAY_ID,
+ version: 1,
+ name: 'Race',
+ summary: 'Several entrants answer the same ask at once, blind side by side, each on the same budget; a person judges and the names are revealed.',
+ roles: sides.map((slot) => ({
+ id: slot,
+ name: `Side ${slot.toUpperCase()}`,
+ does: 'Answer the prompt in the run\'s args as the entrant the run names. Work alone: no other side\'s answer, no judging.',
+ model: 'same',
+ })),
+ phases: [{ id: 'answer', name: 'Answer', roles: sides, parallel: true }],
+ stop: { when: 'person' },
+ budget: { split: Object.fromEntries(sides.map((slot) => [slot, share])) },
+ blind: true,
+ visibility: 'public',
+ author: { type: 'service', id: 'run' },
+ };
+ contracts.assertValid('agents.play@1', play);
+ return play;
+}
+
+/** The plays Run lists today: the race (the first play the engine runs) with the most sides it supports. */
+function plays() {
+ return [racePlay(SLOTS)];
+}
+
+module.exports = { SLOTS, PLAY_ID, MACHINE_CLASS, racePlay, plays, shareOf };
new file mode 100644
new file mode 100644
@@ -0,0 +1,90 @@
@@ -0,0 +1,90 @@
+'use strict';
+
+/**
+ * Run → OpenVibe.Bench: record one bench.match@1 (the race's judgement) on Bench's
+ * POST /api/v1/matches (capability bench.judgement.cast, audience openvibe.bench).
+ *
+ * The match is validated against the released contract before it is sent, so a broken body is Run's own
+ * 500/422 and never a Bench refusal. The token is a Network client-credentials service token minted from
+ * OV_OAUTH_CLIENT_ID/OV_OAUTH_CLIENT_SECRET and cached until 60 s before expiry by openvibe-sdk/auth; a
+ * 401 drops the cache and the call is tried once more. A Bench that cannot answer is a 503 to the caller
+ * and the race stays unjudged: names are only revealed after the match is recorded.
+ *
+ * Bench has no matchmaker endpoint yet (checked 2026-11-13, Contracts 0.422.0), so the matchmaker fallback
+ * lives in server/races/matchmaker.js.
+ */
+const contracts = require('openvibe-contracts');
+const { createServiceTokenClient } = require('openvibe-sdk/auth');
+
+const BENCH_JUDGE = 'bench.judgement.cast';
+
+class BenchError extends Error {
+ constructor(status, code, detail) {
+ super(detail || code);
+ this.name = 'BenchError';
+ this.status = status;
+ this.code = code;
+ this.detail = detail;
+ }
+}
+
+/**
+ * createBenchRecorder({ config, fetchImpl, now, log }) → { enabled, note, record, status }.
+ * `enabled` is false without OV_OAUTH_CLIENT_SECRET (Run cannot mint the Bench token); record() then
+ * answers 503 and nothing is sent.
+ */
+function createBenchRecorder({ config = {}, fetchImpl = globalThis.fetch, now = () => Date.now(), log = console } = {}) {
+ const bench = config.bench || {};
+ const url = String(bench.url || '').replace(/\/+$/, '');
+ const audience = bench.audience || 'openvibe.bench';
+ const timeoutMs = bench.timeoutMs || 8000;
+ const { clientId, clientSecret } = config.oauth || {};
+ const tokens = clientSecret
+ ? createServiceTokenClient({ tokenUrl: `${config.network.internalUrl}/oauth/token`, clientId, clientSecret, audience, scope: BENCH_JUDGE, fetch: fetchImpl, now })
+ : null;
+ const enabled = Boolean(tokens && url);
+ const note = enabled
+ ? `Bench matches: ${url}, audience ${audience}, capability ${BENCH_JUDGE}`
+ : 'Bench matches are off: OV_OAUTH_CLIENT_SECRET is unset, so Run cannot mint its Network client-credentials token';
+
+ /** One call: mint (or reuse) the token, fetch, and turn Bench's answer into a value or a BenchError. */
+ async function call(match) {
+ if (!enabled) throw new BenchError(503, 'run.bench_unavailable', note);
+ for (let attempt = 0; ; attempt++) {
+ const headers = { Accept: 'application/json', 'Content-Type': 'application/json' };
+ try { Object.assign(headers, await tokens.authHeaders()); }
+ catch (e) { throw new BenchError(503, 'run.bench_unavailable', 'Run could not mint its Bench token from Network'); }
+ let res;
+ try {
+ res = await fetchImpl(`${url}/api/v1/matches`, {
+ method: 'POST', headers, body: JSON.stringify(match), signal: AbortSignal.timeout(timeoutMs),
+ });
+ } catch (e) {
+ const why = e && (e.name === 'TimeoutError' || e.name === 'AbortError') ? `no answer within ${timeoutMs} ms` : 'unreachable';
+ throw new BenchError(503, 'run.bench_unavailable', `Bench did not answer (${why}); the race stays unjudged`);
+ }
+ // A rotated or clock-skewed token: drop the cache and try once more.
+ if (res.status === 401 && attempt === 0) { tokens.invalidate(); continue; }
+ const doc = await res.json().catch(() => null);
+ if (res.ok && doc) return { created: res.status === 201, match: doc.match || null };
+ const code = doc && typeof doc.code === 'string' ? doc.code : null;
+ const detail = doc && typeof doc.detail === 'string' ? doc.detail.slice(0, 300) : null;
+ const why = `Bench answered HTTP ${res.status}${code ? ` ${code}` : ''}${detail ? `: ${detail}` : ''}`;
+ if (res.status >= 500) throw new BenchError(503, 'run.bench_unavailable', `${why}; the race stays unjudged`);
+ throw new BenchError(502, 'run.bench_refused', `${why}; the race stays unjudged`);
+ }
+ }
+
+ return {
+ enabled,
+ note,
+ /** Record one match. The body is validated against bench.match@1 here, before any call. */
+ async record(match) {
+ contracts.assertValid('bench.match@1', match);
+ return call(match);
+ },
+ status: () => ({ enabled, note, url, audience, capability: BENCH_JUDGE, timeout_ms: timeoutMs }),
+ };
+}
+
+module.exports = { createBenchRecorder, BenchError, BENCH_JUDGE };
new file mode 100644
new file mode 100644
@@ -0,0 +1,39 @@
@@ -0,0 +1,39 @@
+'use strict';
+
+/**
+ * bench.entrant@1 ids and their model families, for a matchmaker that must not pair two of one family
+ * (bench.entrant@1: "the model family, so the matchmaker can avoid pairing two of one family").
+ *
+ * The id is the only thing Run is given by the race API, so the family is read from it: a model id is the
+ * model itself; an agent id is agent:<harness>+<model>#<config>; a harness or a product has no model and
+ * falls back to its own name. The family is the first segment of the model (gpt-5-mini → gpt,
+ * deepseek-flash → deepseek, meta-llama/… → meta), which is coarse on purpose: it keeps two variants of
+ * one maker's model out of a pair.
+ */
+
+/** bench.entrant@1's own id pattern (contracts/bench/entrant.v1.json). */
+const ENTRANT_RE = /^(model:[A-Za-z0-9._/:-]{1,200}|harness:[a-z0-9][a-z0-9._-]{0,60}@[A-Za-z0-9._-]{1,40}|agent:[a-z0-9][a-z0-9._-]{0,60}\+[A-Za-z0-9._/:-]{1,200}#[a-f0-9]{8,64}|product:[a-z0-9][a-z0-9-]{0,60})$/;
+
+const isEntrantId = (v) => typeof v === 'string' && ENTRANT_RE.test(v);
+
+/** The model (or, for a harness/product, the thing itself) an entrant id names. */
+function modelOf(id) {
+ const s = String(id || '');
+ if (s.startsWith('model:')) return s.slice(6);
+ if (s.startsWith('agent:')) {
+ const plus = s.indexOf('+');
+ const hash = s.indexOf('#');
+ return plus >= 0 ? s.slice(plus + 1, hash > plus ? hash : undefined) : s.slice(6);
+ }
+ if (s.startsWith('harness:')) return s.slice(8).split('@')[0];
+ if (s.startsWith('product:')) return s.slice(8);
+ return s;
+}
+
+/** The coarser family the matchmaker keeps apart (gpt, deepseek, glm, claude, llama …). */
+function familyOf(id) {
+ const model = modelOf(id).toLowerCase();
+ return model.split(/[-/.:]/)[0] || model;
+}
+
+module.exports = { ENTRANT_RE, isEntrantId, modelOf, familyOf };
new file mode 100644
new file mode 100644
@@ -0,0 +1,42 @@
@@ -0,0 +1,42 @@
+'use strict';
+
+/**
+ * The race matchmaker's fallback (compare-arena-ecosystem.md §3: "the matchmaker (Bench) picks the
+ * entrants: pairs whose outcome is most uncertain and that the person's allowance can pay for, new
+ * entrants explored, two of the same family avoided, position randomised").
+ *
+ * Bench has no matchmaker endpoint yet (checked against Bench's main on 2026-11-13: /api/v1/matches and
+ * /api/v1/factors exist, no matchmaker). Until it does, Run picks two entrants at random from
+ * RUN_RACE_ENTRANTS, never two of one model family, and the race service randomises their positions.
+ * When Bench ships the matchmaker this module is the one place that changes: ask Bench first, keep the
+ * configured list as the fallback when it cannot answer.
+ */
+const { isEntrantId, familyOf } = require('./entrants');
+
+/** Every unordered pair from `list` (small lists only: a race has at most four sides). */
+function pairs(list) {
+ const out = [];
+ for (let i = 0; i < list.length; i++) for (let j = i + 1; j < list.length; j++) out.push([list[i], list[j]]);
+ return out;
+}
+
+/**
+ * pair({ entrants, random }) → [a, b] from two different families, at random.
+ * `entrants` is the configured list (RUN_RACE_ENTRANTS); invalid ids are dropped. Throws a plain error
+ * whose message the service turns into a 422, so an unconfigured deployment says what to set.
+ */
+function pair({ entrants = [], random = Math.random } = {}) {
+ const list = [...new Set((Array.isArray(entrants) ? entrants : []).filter(isEntrantId))];
+ if (list.length < 2) {
+ throw Object.assign(new Error('the race matchmaker is not configured: set RUN_RACE_ENTRANTS to at least two bench.entrant ids (Bench has no matchmaker endpoint yet)'), { code: 'run.race_matchmaker_unavailable' });
+ }
+ const choices = pairs(list).filter(([a, b]) => familyOf(a) !== familyOf(b));
+ if (!choices.length) {
+ throw Object.assign(new Error('the configured race entrants are all of one model family; set RUN_RACE_ENTRANTS to entrants of at least two families'), { code: 'run.race_matchmaker_unavailable' });
+ }
+ const r = Number(random());
+ const pick = choices[Math.min(choices.length - 1, Math.max(0, Math.floor((Number.isFinite(r) ? r : 0) * choices.length)))];
+ return [...pick];
+}
+
+module.exports = { pair, pairs, familyOf };
new file mode 100644
new file mode 100644
@@ -0,0 +1,325 @@
@@ -0,0 +1,325 @@
+'use strict';
+
+/**
+ * The race service (compare-arena-ecosystem.md §3, agent-engine.md §4): one parent race, one child job per
+ * side, the same share of the budget each, no entrant named until a person judges.
+ *
+ * A race is started by a first-party service acting for a person (the Arena's front door in Compare): the
+ * person is the requester and the only one who may judge, through the same service. The child jobs are
+ * ordinary run_jobs rows — class linux, args carrying the side, the race and the entrant the node must run
+ * — so placement, metering, events and the job stream are the engine's own, unchanged. Each side's output
+ * is read from the job stream's ring on every GET (best effort, as run.job-stream-event@1 says).
+ *
+ * The judgement is a bench.match@1 recorded on Bench BEFORE the names are revealed: source arena, the
+ * shown order, blind true, each side's receipts, one human_pairwise judgement with a keyed hash of the
+ * person, the play and its parent race. If Bench cannot record it, the race stays unjudged and the names
+ * hidden; a retry sends the same match id, so Bench records one match rather than two.
+ */
+const crypto = require('crypto');
+const contracts = require('openvibe-contracts');
+const { fail, iso } = require('../util');
+const { checkLimits } = require('../api/v1');
+const { SLOTS, PLAY_ID, MACHINE_CLASS, racePlay } = require('../plays/race');
+const raceStore = require('./store');
+const matchmaker = require('./matchmaker');
+const { isEntrantId } = require('./entrants');
+
+const END_STATES = new Set(['succeeded', 'failed', 'cancelled', 'expired']);
+const KINDS = ['chat', 'code', 'design'];
+const WINNERS = new Set([...SLOTS, 'tie', 'both_bad']);
+const START_KEYS = new Set(['prompt', 'kind', 'entrants', 'matchmaker', 'budget']);
+const JUDGEMENT_KEYS = new Set(['winner']);
+const DEFAULT_LIMITS = { wall_ms: 300_000, cpu_ms: 300_000, mem_bytes: 512 * 1024 * 1024 };
+const OUTCOME = { succeeded: 'succeeded', failed: 'failed', expired: 'timed_out', cancelled: 'cancelled' };
+const RECEIPT_INTS = ['tokens_in', 'tokens_out', 'tool_calls', 'length_chars', 'markdown_items'];
+const RECEIPT_NUMS = ['cost_usd', 'ttft_ms', 'tps'];
+
+const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : null);
+const isCount = (v) => Number.isInteger(v) && v >= 1;
+const sha256 = (s) => crypto.createHash('sha256').update(String(s)).digest('hex');
+
+/** A keyed hash of the person who judged: stable for rate limits, never the person's id (bench.match@1). */
+function personHash(requester, key) {
+ const subject = String(requester || '').replace(/^user:/, '');
+ return crypto.createHmac('sha256', key).update(`run.race.person.v1\n${subject}`).digest('hex');
+}
+
+/** The body of POST /api/v1/races, validated by hand (Contracts 0.422.0 has no run.race@1 yet). */
+function validStart(body) {
+ if (!obj(body)) fail(400, 'run.invalid_request', 'the body must be a JSON object { prompt, kind, entrants | matchmaker, budget }');
+ const extra = Object.keys(body).filter((k) => !START_KEYS.has(k));
+ if (extra.length) fail(400, 'run.invalid_request', `unknown field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);
+ const prompt = typeof body.prompt === 'string' ? body.prompt.trim() : '';
+ if (!prompt) fail(400, 'run.invalid_request', 'prompt is required and must be a non-empty string');
+ if (prompt.length > 20000) fail(400, 'run.invalid_request', 'prompt is at most 20000 characters');
+ if (!KINDS.includes(body.kind)) fail(400, 'run.invalid_request', `kind must be one of ${KINDS.join(', ')}`);
+ if (!obj(body.budget)) fail(400, 'run.invalid_request', 'budget is required and must be an object');
+ const budget = validBudget(body.budget);
+ const wantsMatchmaker = body.matchmaker === true;
+ if (body.matchmaker !== undefined && typeof body.matchmaker !== 'boolean') fail(400, 'run.invalid_request', 'matchmaker must be true or false');
+ if (wantsMatchmaker && body.entrants !== undefined) fail(400, 'run.invalid_request', 'name entrants or ask matchmaker: true, not both');
+ if (!wantsMatchmaker) {
+ if (!Array.isArray(body.entrants)) fail(400, 'run.invalid_request', 'entrants must be an array of 2 to 4 bench.entrant ids, or matchmaker: true');
+ if (body.entrants.length < 2 || body.entrants.length > 4) fail(400, 'run.invalid_request', 'a race has 2 to 4 entrants');
+ for (const id of body.entrants) if (!isEntrantId(id)) fail(400, 'run.invalid_request', `${JSON.stringify(id)} is not a bench.entrant@1 id`);
+ if (new Set(body.entrants).size !== body.entrants.length) fail(400, 'run.invalid_request', 'entrants must be unique');
+ }
+ return { prompt, kind: body.kind, budget, matchmaker: wantsMatchmaker, entrants: wantsMatchmaker ? null : body.entrants };
+}
+
+/** The total budget a race asks for; an absent cap takes the default. */
+function validBudget(b) {
+ const keys = new Set(['usd', 'wall_ms', 'cpu_ms', 'mem_bytes']);
+ const extra = Object.keys(b).filter((k) => !keys.has(k));
+ if (extra.length) fail(400, 'run.invalid_request', `unknown budget field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);
+ const out = { ...DEFAULT_LIMITS };
+ for (const k of ['wall_ms', 'cpu_ms', 'mem_bytes']) {
+ if (b[k] === undefined) continue;
+ if (!isCount(b[k])) fail(400, 'run.invalid_request', `budget.${k} must be a positive integer`);
+ out[k] = b[k];
+ }
+ if (b.usd !== undefined) {
+ if (typeof b.usd !== 'number' || !Number.isFinite(b.usd) || b.usd < 0) fail(400, 'run.invalid_request', 'budget.usd must be a number of at least 0');
+ out.usd = b.usd;
+ }
+ return out;
+}
+
+/** Split the total evenly: every side runs on the same share, never more than the parent named. */
+function splitBudget(total, n) {
+ const usd = total.usd == null ? null : Math.round((total.usd / n) * 1e6) / 1e6;
+ const per = {
+ usd,
+ wall_ms: Math.floor(total.wall_ms / n),
+ cpu_ms: Math.floor(total.cpu_ms / n),
+ mem_bytes: Math.floor(total.mem_bytes / n),
+ machine_class: MACHINE_CLASS,
+ };
+ if (per.wall_ms < 1000 || per.cpu_ms < 1000) fail(422, 'run.limits.exceeded', `budget is too small for ${n} sides: each side needs at least 1000 ms of wall and CPU time`);
+ if (per.mem_bytes < 1024 * 1024) fail(422, 'run.limits.exceeded', `budget is too small for ${n} sides: each side needs at least 1 MiB of memory`);
+ return { total: { ...total, machine_class: MACHINE_CLASS }, per_side: per };
+}
+
+/** Fisher–Yates: the sides are labelled a, b, … in a random order, so a position is not a name. */
+function shuffled(list, random = Math.random) {
+ const a = [...list];
+ for (let i = a.length - 1; i > 0; i--) {
+ const j = Math.floor((Number(random()) || 0) * (i + 1));
+ [a[i], a[j]] = [a[j], a[i]];
+ }
+ return a;
+}
+
+/** What a finished child sends Bench: its outcome and the receipts the stream/result can back. */
+function receiptsOf(row) {
+ const result = obj(row.result) || {};
+ const src = obj(result.receipts) || result;
+ const out = { outcome: OUTCOME[row.state] || 'failed' };
+ for (const k of RECEIPT_NUMS) if (typeof src[k] === 'number' && Number.isFinite(src[k]) && src[k] >= 0) out[k] = src[k];
+ for (const k of RECEIPT_INTS) if (Number.isInteger(src[k]) && src[k] >= 0) out[k] = src[k];
+ if (row.wall_ms != null && Number(row.wall_ms) > 0) out.wall_ms = Number(row.wall_ms);
+ const answer = typeof result.answer === 'string' ? result.answer : typeof result.text === 'string' ? result.text : null;
+ if (out.length_chars == null && answer != null) out.length_chars = answer.length;
+ return out;
+}
+
+/** The bench.match@1 a race's judgement records. Validated before it leaves Run. */
+function matchOf(row, childRows, winner, at) {
+ const jobs = new Map(childRows.map((j) => [j.id, j]));
+ const match = {
+ id: row.match_id,
+ source: 'arena',
+ task: { kind: row.kind, prompt_hash: row.prompt_hash },
+ slots: row.sides.map((side) => {
+ const job = jobs.get(side.job_id);
+ const slot = { slot: side.slot, entrant: side.entrant, run_id: side.job_id };
+ if (job && END_STATES.has(job.state)) slot.receipts = receiptsOf(job);
+ return slot;
+ }),
+ shown: row.sides.map((side) => side.slot),
+ blind: true,
+ budget: {
+ usd: row.budget.per_side.usd == null ? null : row.budget.per_side.usd,
+ ms: row.budget.per_side.wall_ms,
+ machine_class: row.budget.per_side.machine_class,
+ },
+ judgements: [{ judge: 'human_pairwise', winner, by: { type: 'person', id_hash: personHash(row.requester, row.hash_key) }, at: iso(at) }],
+ play: { id: PLAY_ID, run_id: row.id },
+ at: iso(row.created_at),
+ };
+ contracts.assertValid('bench.match@1', match);
+ return match;
+}
+
+/** One race as the API answers it: names only once a judgement is on the row. */
+function view(row, childRows, { config, stream }) {
+ const jobs = new Map(childRows.map((j) => [j.id, j]));
+ const revealed = row.judged_at != null;
+ const sides = row.sides.map((side) => {
+ const job = jobs.get(side.job_id);
+ const shot = stream.snapshot(side.job_id);
+ const ended = job && END_STATES.has(job.state);
+ const out = {
+ slot: side.slot,
+ run_id: side.job_id,
+ state: job ? job.state : 'missing',
+ output: shot.output || null,
+ receipts: ended ? receiptsOf(job) : null,
+ stream_url: `${config.baseUrl}/api/v1/jobs/${side.job_id}/stream`,
+ };
+ // The entrant is the name: it is never in the body before the judgement. The child job's own
+ // args name it for the node, and the race's sides are the only owner-scoped read that returns it.
+ if (revealed) out.entrant = side.entrant;
+ return out;
+ });
+ const allDone = sides.every((s) => END_STATES.has(s.state));
+ return {
+ id: row.id,
+ kind: row.kind,
+ prompt: row.prompt,
+ state: revealed ? 'judged' : allDone ? 'done' : 'running',
+ blind: true,
+ revealed,
+ play: row.play,
+ budget: row.budget,
+ shown: row.sides.map((side) => side.slot),
+ sides,
+ judgement: revealed ? {
+ winner: row.judgement.winner,
+ judge: 'human_pairwise',
+ at: row.judgement.at,
+ match_id: row.judgement.match_id,
+ recorded_by: row.judgement.recorded_by || 'run',
+ } : null,
+ // The names as one mapping too, so a client does not have to walk the sides; null while blind.
+ names: revealed ? Object.fromEntries(row.sides.map((side) => [side.slot, side.entrant])) : null,
+ created_at: iso(row.created_at),
+ updated_at: iso(row.updated_at),
+ };
+}
+
+/**
+ * createRaces({ config, db, jobs, events, stream, bench, now, log, random }) → { plays, start, get, judge }.
+ * Every dependency is injected; the API is its only caller.
+ */
+function createRaces({ config, db, jobs, events, stream, bench, now = () => Date.now(), log = console, random = Math.random } = {}) {
+ const hashKey = (config.races && config.races.hashKey) || 'run-race-unsafe-default';
+
+ /** GET /api/v1/plays: the race play (with the most sides it supports) as agents.play@1. */
+ function plays() {
+ return require('../plays/race').plays();
+ }
+
+ /** POST /api/v1/races: one parent race and one child job per side, in one transaction. */
+ async function start({ projectId, requester, via, body, traceparent }) {
+ if (!/^user:/.test(String(requester || ''))) {
+ fail(403, 'run.race_person_required', 'a race is started by a person: a first-party service acting for them (X-OV-Subject), never a service for itself');
+ }
+ const req = validStart(body);
+ let entrants;
+ if (req.matchmaker) {
+ try {
+ entrants = matchmaker.pair({ entrants: (config.races && config.races.entrants) || [], random });
+ } catch (e) {
+ fail(422, e.code || 'run.race_matchmaker_unavailable', e.message);
+ }
+ } else {
+ entrants = req.entrants;
+ }
+ // The sides are labelled a, b, … in the shown order, which is randomised (position bias: the
+ // design asks for it, and bench.match@1's SHOWN records what was shown).
+ const sides = shuffled(entrants, random).map((entrant, i) => ({ slot: SLOTS[i], entrant, job_id: null }));
+ const budget = splitBudget(req.budget, sides.length);
+ const raceId = raceStore.newRaceId();
+ const matchId = raceStore.newMatchId();
+ const at = now();
+ const promptHash = sha256(req.prompt);
+ const ttl = budget.per_side.wall_ms + 600_000;
+ const play = racePlay(sides.map((side) => side.slot));
+
+ // Each child is an ordinary linux job: the same limits as every other side, the race and its side
+ // in args for the node, and parent/race in the job record (never in the frame sent to Bot).
+ const childBodies = sides.map((side) => ({
+ class: MACHINE_CLASS,
+ args: {
+ play: PLAY_ID, role: side.slot, race: raceId, kind: req.kind, prompt: req.prompt,
+ entrant: side.entrant, budget: budget.per_side,
+ },
+ limits: { wall_ms: budget.per_side.wall_ms, cpu_ms: budget.per_side.cpu_ms, mem_bytes: budget.per_side.mem_bytes },
+ ttl_ms: ttl,
+ }));
+ for (const bodyOf of childBodies) {
+ const checked = contracts.validate('run.job-create-request@1', bodyOf);
+ if (!checked.valid) fail(422, 'run.invalid_input', `a race child is not a run.job-create-request@1: ${checked.errors.map((e) => `${e.path} ${e.message}`).join('; ')}`);
+ }
+ checkLimits({ ...childBodies[0], args: childBodies[0].args, inputs: [] }, config.limits);
+
+ const out = await db.tx(async (t) => {
+ const children = [];
+ for (let i = 0; i < sides.length; i++) {
+ const made = await jobs.mint(t, childBodies[i], {
+ projectId, requester, via, idempotencyKey: `race:${raceId}:${sides[i].slot}`, now: at,
+ });
+ if (made.created) await events.queued(t, made.row, { traceparent });
+ // Run's own record of the child's place in the tree: platform.job@1 has no field for a
+ // parent, and frameOf() drops these keys, so the node is never sent them.
+ const marked = await t.maybe(
+ `UPDATE run_jobs SET job = job || $2::jsonb WHERE id = $1 RETURNING *`,
+ [made.row.id, JSON.stringify({ parent: { type: 'race', id: raceId }, race: { slot: sides[i].slot, play: PLAY_ID } })]);
+ sides[i].job_id = made.row.id;
+ children.push(marked || made.row);
+ }
+ const row = await raceStore.insert(t, {
+ id: raceId, project_id: projectId, requester, via, kind: req.kind, prompt: req.prompt, prompt_hash: promptHash,
+ sides, budget, play, match_id: matchId, blind: true, judgement: null, judged_at: null, created_at: at, updated_at: at,
+ });
+ return { row, children };
+ });
+ log.log(`[Run] race ${out.row.id}: ${sides.length} blind sides, each ${budget.per_side.wall_ms} ms`);
+ return view(out.row, out.children, { config, stream });
+ }
+
+ /** GET /api/v1/races/:id: sides' live output and receipts; names only once judged. */
+ async function get({ projectId, requester, via, id }) {
+ const row = await raceStore.get(db, id, { projectId, requester, via });
+ if (!row) return null;
+ const children = await jobs.getMany(db, row.sides.map((side) => side.job_id));
+ return view(row, children, { config, stream });
+ }
+
+ /** POST /api/v1/races/:id/judgement: record the match on Bench, then reveal the names. */
+ async function judge({ projectId, requester, via, id, body }) {
+ if (!obj(body)) fail(400, 'run.invalid_request', 'the body must be JSON { winner: a|b|tie|both_bad }');
+ const extra = Object.keys(body).filter((k) => !JUDGEMENT_KEYS.has(k));
+ if (extra.length) fail(400, 'run.invalid_request', `unknown field${extra.length > 1 ? 's' : ''}: ${extra.join(', ')}`);
+ const winner = typeof body.winner === 'string' ? body.winner : '';
+ if (!WINNERS.has(winner)) fail(400, 'run.invalid_request', 'winner must be a shown side (a, b, …), tie or both_bad');
+ const row = await raceStore.get(db, id, { projectId, requester, via });
+ if (!row) return null; // another person, service or project: the same 404
+ if (row.judged_at != null) fail(409, 'run.race_already_judged', 'this race has already been judged');
+ if (winner !== 'tie' && winner !== 'both_bad' && !row.sides.some((side) => side.slot === winner)) {
+ fail(422, 'run.invalid_input', `${winner} is not a side of this race (${row.sides.map((s) => s.slot).join(', ')})`);
+ }
+ const children = await jobs.getMany(db, row.sides.map((side) => side.job_id));
+ const at = now();
+ const match = matchOf({ ...row, hash_key: hashKey }, children, winner, at);
+ // Only Bench recording it reveals the names: if it cannot, the race stays blind and can be judged again.
+ const recorded = await bench.record(match);
+ const judgement = {
+ winner, judge: 'human_pairwise', at: iso(at), match_id: row.match_id,
+ by_hash: personHash(row.requester, hashKey), recorded_by: 'run',
+ bench: recorded && recorded.created === false ? 'recorded' : 'created',
+ };
+ const updated = await db.tx((t) => raceStore.markJudged(t, row.id, { projectId, requester, via, judgement, at }));
+ if (!updated) fail(409, 'run.race_already_judged', 'this race has already been judged');
+ log.log(`[Run] race ${row.id}: judged ${winner}, names revealed`);
+ return view(updated, children, { config, stream });
+ }
+
+ return { plays, start, get, judge, personHash: (requester) => personHash(requester, hashKey) };
+}
+
+module.exports = {
+ createRaces, validStart, validBudget, splitBudget, receiptsOf, matchOf, view, personHash,
+ KINDS, WINNERS, DEFAULT_LIMITS, END_STATES, START_KEYS, JUDGEMENT_KEYS,
+};
new file mode 100644
new file mode 100644
@@ -0,0 +1,69 @@
@@ -0,0 +1,69 @@
+'use strict';
+
+/**
+ * The race rows (migrations/0006_races.sql). Row access only: the service creates a race and its child
+ * jobs in one transaction, and every read is scoped to the person who started it and the service they
+ * started it through — another person's race is null, which the API answers as 404.
+ */
+const { ids } = require('openvibe-contracts');
+
+const RACE_RE = /^rac_[0-9A-HJKMNP-TV-Z]{26}$/;
+const MATCH_RE = /^mat_[0-9A-HJKMNP-TV-Z]{26}$/;
+const SLOT_RE = /^[a-d]$/;
+
+const isRaceId = (v) => typeof v === 'string' && RACE_RE.test(v);
+const isMatchId = (v) => typeof v === 'string' && MATCH_RE.test(v);
+const isSlot = (v) => typeof v === 'string' && SLOT_RE.test(v);
+
+/** A new race id: rac_<ULID> (the id pattern the race API answers; contracts has no race kind yet). */
+const newRaceId = () => `rac_${ids.ulid()}`;
+/** The bench.match@1 id minted with the race: a retry records the same match rather than a second one. */
+const newMatchId = () => `mat_${ids.ulid()}`;
+
+const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : null);
+const num = (v) => (v == null ? null : Number(v));
+
+/** One row as the store reads it: jsonb parsed, the bigints as numbers. */
+function fromDb(row) {
+ if (!row) return null;
+ return {
+ ...row,
+ sides: Array.isArray(row.sides) ? row.sides.map((s) => ({ slot: s.slot, entrant: s.entrant, job_id: s.job_id })) : [],
+ budget: obj(row.budget) || {},
+ play: obj(row.play) || {},
+ judgement: obj(row.judgement),
+ blind: Boolean(row.blind),
+ judged_at: num(row.judged_at),
+ created_at: num(row.created_at),
+ updated_at: num(row.updated_at),
+ };
+}
+
+/** Insert the parent race and its sides (the child job rows are already in this transaction). */
+async function insert(t, row) {
+ const made = await t.maybe(
+ `INSERT INTO run_races (id, project_id, requester, via, kind, prompt, prompt_hash, sides, budget, play, match_id, blind, judgement, judged_at, created_at, updated_at)
+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16) RETURNING *`,
+ [row.id, row.project_id, row.requester, row.via, row.kind, row.prompt, row.prompt_hash, JSON.stringify(row.sides),
+ JSON.stringify(row.budget), JSON.stringify(row.play), row.match_id, row.blind, row.judgement == null ? null : JSON.stringify(row.judgement),
+ row.judged_at, row.created_at, row.updated_at]);
+ return fromDb(made);
+}
+
+/** The starter's race: project, person and the service they used all have to match, else null (→ 404). */
+async function get(db, id, { projectId, requester, via }) {
+ if (!isRaceId(id)) return null;
+ return fromDb(await db.maybe(
+ 'SELECT * FROM run_races WHERE id = $1 AND project_id = $2 AND requester = $3 AND via = $4',
+ [id, projectId, requester, via]));
+}
+
+/** The conditional write that ends a race: one judgement, the first writer wins. → the row, or null. */
+async function markJudged(t, id, { projectId, requester, via, judgement, at }) {
+ return fromDb(await t.maybe(
+ `UPDATE run_races SET judgement = $1, judged_at = $2, updated_at = $2
+ WHERE id = $3 AND project_id = $4 AND requester = $5 AND via = $6 AND judged_at IS NULL RETURNING *`,
+ [JSON.stringify(judgement), at, id, projectId, requester, via]));
+}
+
+module.exports = { RACE_RE, MATCH_RE, SLOT_RE, isRaceId, isMatchId, isSlot, newRaceId, newMatchId, fromDb, insert, get, markJudged };
new file mode 100644
new file mode 100644
@@ -0,0 +1,280 @@
@@ -0,0 +1,280 @@
+'use strict';
+/**
+ * The Arena's race on Run (compare-arena-ecosystem.md §3, Contracts 0.422.0): the race play as
+ * agents.play@1, N blind child jobs with one budget split evenly, names hidden until a person's judgement,
+ * the bench.match@1 that judgement records, and the starter-only rule.
+ *
+ * node test/races.test.js
+ */
+const assert = require('assert');
+const crypto = require('crypto');
+const contracts = require('openvibe-contracts');
+const { boot, check, done } = require('./helpers/app');
+const { familyOf } = require('../server/races/entrants');
+
+const { ids } = contracts;
+const sha256 = (s) => crypto.createHash('sha256').update(s).digest('hex');
+
+/** A Bench that records what it is sent and validates it against bench.match@1, like Bench's route does. */
+function fakeBench() {
+ const calls = [];
+ return {
+ enabled: true,
+ note: 'test bench',
+ calls,
+ async record(match) {
+ calls.push(match);
+ contracts.assertValid('bench.match@1', match);
+ return { created: true, match };
+ },
+ status: () => ({ enabled: true, note: 'test bench' }),
+ };
+}
+
+async function checks(t, bench) {
+ const P = ids.newId('project');
+ const Q = ids.newId('project');
+ const ada = ids.newId('user');
+ const bo = ids.newId('user');
+ t.network.members.set(`${P} ${ada}`, { role: 'admin' });
+ t.network.members.set(`${P} ${bo}`, { role: 'owner' });
+
+ const CAPS = ['run.job.submit', 'run.job.read', 'run.job.list'];
+ // A first-party service acting for a person: the only principal a race accepts as its starter.
+ const as = (subject, project, { sub = 'svc:compare', cap = CAPS } = {}) => ({
+ token: t.network.signService({ sub, cap }),
+ headers: { 'X-OV-Subject': subject, 'X-OV-Project': project },
+ });
+ const raceRows = async (raceId) => t.db.many("SELECT * FROM run_jobs WHERE job->>'parent' IS NOT NULL AND job->'parent'->>'id' = $1 ORDER BY id", [raceId]);
+
+ const prompt = 'Explain in two sentences why the sky is blue.';
+ const entrants = ['model:gpt-5-mini', 'model:glm-5.3'];
+ const body = { prompt, kind: 'chat', entrants, budget: { usd: 0.2, wall_ms: 60_000, cpu_ms: 60_000, mem_bytes: 268_435_456 } };
+
+ let raceId;
+ let sideEntrants;
+
+ await check('the race play lists as agents.play@1: one role per side, one parallel phase, even split, blind, stop on person', async () => {
+ const r = await t.call('GET', '/api/v1/plays', { token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.list'] }) });
+ assert.strictEqual(r.status, 200, r.text);
+ assert.strictEqual(r.json.plays.length, 1);
+ const play = r.json.plays[0];
+ assert.ok(contracts.validate('agents.play@1', play).valid, JSON.stringify(play));
+ assert.strictEqual(play.id, 'race');
+ assert.strictEqual(play.blind, true);
+ assert.deepStrictEqual(play.stop, { when: 'person' });
+ assert.strictEqual(play.phases.length, 1);
+ assert.strictEqual(play.phases[0].parallel, true);
+ assert.deepStrictEqual(play.phases[0].roles, play.roles.map((role) => role.id));
+ const split = Object.values(play.budget.split);
+ assert.strictEqual(split.length, play.roles.length);
+ assert.ok(Math.abs(split.reduce((a, b) => a + b, 0) - 1) < 1e-6, JSON.stringify(play.budget.split));
+ for (const role of play.roles) assert.ok(!JSON.stringify(role).includes('gpt') && !JSON.stringify(role).includes('model:'), 'a role never names an entrant');
+ });
+
+ await check('POST /races starts N child jobs under one parent, each with the same share of the budget, and answers no names', async () => {
+ const r = await t.call('POST', '/api/v1/races', { body, ...as(ada, P) });
+ assert.strictEqual(r.status, 201, r.text);
+ const race = r.json;
+ raceId = race.id;
+ assert.match(raceId, /^rac_[0-9A-HJKMNP-TV-Z]{26}$/);
+ assert.strictEqual(race.kind, 'chat');
+ assert.strictEqual(race.blind, true);
+ assert.strictEqual(race.revealed, false);
+ assert.strictEqual(race.names, null);
+ assert.strictEqual(race.judgement, null);
+ assert.deepStrictEqual(race.shown, ['a', 'b']);
+ assert.deepStrictEqual(race.sides.map((s) => s.slot), ['a', 'b']);
+ assert.ok(contracts.validate('agents.play@1', race.play).valid);
+ assert.strictEqual(race.play.roles.length, 2);
+ assert.strictEqual(race.budget.per_side.wall_ms, 30_000);
+ assert.strictEqual(race.budget.per_side.cpu_ms, 30_000);
+ assert.strictEqual(race.budget.per_side.usd, 0.1);
+ assert.strictEqual(race.budget.per_side.machine_class, 'linux');
+ const raw = JSON.stringify(race);
+ for (const id of entrants) assert.ok(!raw.includes(id), `no entrant before the judgement: ${id} in ${raw}`);
+ for (const side of race.sides) assert.ok(!('entrant' in side), 'a side carries no entrant while blind');
+
+ const rows = await raceRows(raceId);
+ assert.strictEqual(rows.length, 2);
+ for (const row of rows) {
+ assert.strictEqual(row.state, 'queued');
+ assert.strictEqual(row.project_id, P);
+ assert.strictEqual(row.requester, `user:${ada}`);
+ assert.strictEqual(row.via, 'service:compare');
+ assert.strictEqual(row.class, 'linux');
+ assert.strictEqual(row.job.limits.wall_ms, 30_000);
+ assert.strictEqual(row.job.limits.cpu_ms, 30_000);
+ assert.strictEqual(row.job.limits.mem_bytes, 134_217_728);
+ assert.strictEqual(Number(row.ttl_ms), 30_000 + 600_000);
+ assert.strictEqual(row.job.args.race, raceId);
+ assert.strictEqual(row.job.args.play, 'race');
+ assert.strictEqual(row.job.args.kind, 'chat');
+ assert.deepStrictEqual(row.job.parent, { type: 'race', id: raceId });
+ assert.ok(entrants.includes(row.job.args.entrant));
+ assert.ok(['a', 'b'].includes(row.job.args.role));
+ }
+ assert.deepStrictEqual(rows.map((row) => row.job.args.role).sort(), ['a', 'b']);
+ sideEntrants = Object.fromEntries(rows.map((row) => [row.job.args.role, row.job.args.entrant]));
+ assert.deepStrictEqual(new Set(rows.map((row) => row.job.args.entrant)), new Set(entrants));
+ assert.strictEqual((await t.outboxRows('run.job.queued')).filter((e) => e.payload.job_id === rows[0].id || e.payload.job_id === rows[1].id).length, 2);
+ });
+
+ await check('GET /races/:id shows the sides\' output as it streams, with no names before the judgement; another person gets 404', async () => {
+ const rows = await raceRows(raceId);
+ t.stream.output(rows[0].id, 'The sky is blue because ');
+ t.stream.output(rows[0].id, 'Rayleigh scattering favours short wavelengths.');
+ t.stream.output(rows[1].id, 'Shorter wavelengths scatter more.');
+ const mine = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));
+ assert.strictEqual(mine.status, 200, mine.text);
+ assert.strictEqual(mine.json.state, 'running');
+ assert.match(mine.json.sides.find((s) => s.run_id === rows[0].id).output, /Rayleigh scattering/);
+ assert.match(mine.json.sides.find((s) => s.run_id === rows[1].id).output, /Shorter wavelengths/);
+ assert.strictEqual(mine.json.revealed, false);
+ assert.strictEqual(mine.json.names, null);
+ assert.ok(!JSON.stringify(mine.json).includes('model:gpt-5-mini'));
+ const other = await t.call('GET', `/api/v1/races/${raceId}`, as(bo, P));
+ assert.strictEqual(other.status, 404, other.text);
+ assert.strictEqual(other.json.code, 'run.race_not_found');
+ const anotherService = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P, { sub: 'svc:actor' }));
+ assert.strictEqual(anotherService.status, 404, anotherService.text);
+ });
+
+ await check('a judged side\'s receipts are the job\'s outcome and wall clock', async () => {
+ const jobs = require('../server/jobs/store');
+ const rows = await raceRows(raceId);
+ for (const row of rows) {
+ await t.db.tx(async (tx) => jobs.settle(tx, row.id, {
+ state: 'succeeded', result: { answer: `answer ${row.job.args.role}` }, wallMs: 1234, at: Date.now(),
+ }));
+ }
+ const r = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));
+ assert.strictEqual(r.status, 200, r.text);
+ assert.strictEqual(r.json.state, 'done');
+ for (const side of r.json.sides) {
+ assert.deepStrictEqual(side.receipts, { outcome: 'succeeded', wall_ms: 1234, length_chars: 8 });
+ }
+ });
+
+ await check('only the person who started it, through the same service, may judge', async () => {
+ const notMine = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(bo, P) });
+ assert.strictEqual(notMine.status, 404, notMine.text);
+ assert.strictEqual(notMine.json.code, 'run.race_not_found');
+ const otherService = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(ada, P, { sub: 'svc:actor' }) });
+ assert.strictEqual(otherService.status, 404, otherService.text);
+ const service = await t.call('POST', `/api/v1/races/${raceId}/judgement`, {
+ body: { winner: 'a' },
+ token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.submit'], project_id: P }),
+ });
+ assert.strictEqual(service.status, 404, service.text);
+ const bad = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'e' }, ...as(ada, P) });
+ assert.strictEqual(bad.status, 400, bad.text);
+ assert.strictEqual(bench.calls.length, 0, 'nothing reached Bench');
+ });
+
+ await check('the judgement records one valid bench.match@1 (arena, shown order, blind, receipts, keyed person) and reveals the names', async () => {
+ const r = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'a' }, ...as(ada, P) });
+ assert.strictEqual(r.status, 201, r.text);
+ assert.strictEqual(bench.calls.length, 1);
+ const match = bench.calls[0];
+ assert.ok(contracts.validate('bench.match@1', match).valid, JSON.stringify(match));
+ assert.match(match.id, /^mat_[0-9A-HJKMNP-TV-Z]{26}$/);
+ assert.strictEqual(match.source, 'arena');
+ assert.deepStrictEqual(match.task, { kind: 'chat', prompt_hash: sha256(prompt) });
+ assert.deepStrictEqual(match.shown, ['a', 'b']);
+ assert.strictEqual(match.blind, true);
+ assert.deepStrictEqual(match.budget, { usd: 0.1, ms: 30_000, machine_class: 'linux' });
+ assert.deepStrictEqual(match.slots.map((s) => s.slot), ['a', 'b']);
+ assert.deepStrictEqual(new Set(match.slots.map((s) => s.entrant)), new Set(entrants));
+ for (const slot of match.slots) {
+ assert.match(slot.run_id, /^job_/);
+ assert.deepStrictEqual(slot.receipts, { outcome: 'succeeded', wall_ms: 1234, length_chars: 8 });
+ }
+ assert.strictEqual(match.judgements.length, 1);
+ assert.strictEqual(match.judgements[0].judge, 'human_pairwise');
+ assert.strictEqual(match.judgements[0].winner, 'a');
+ assert.deepStrictEqual(match.judgements[0].by, { type: 'person', id_hash: t.app.locals.races.personHash(`user:${ada}`) });
+ assert.match(match.judgements[0].by.id_hash, /^[a-f0-9]{64}$/);
+ assert.ok(!JSON.stringify(match).includes(ada), 'never the person id');
+ assert.deepStrictEqual(match.play, { id: 'race', run_id: raceId });
+
+ assert.strictEqual(r.json.revealed, true);
+ assert.strictEqual(r.json.state, 'judged');
+ assert.strictEqual(r.json.judgement.winner, 'a');
+ assert.strictEqual(r.json.judgement.match_id, match.id);
+ assert.deepStrictEqual(r.json.names, sideEntrants);
+ for (const side of r.json.sides) assert.strictEqual(side.entrant, sideEntrants[side.slot]);
+ const shown = await t.call('GET', `/api/v1/races/${raceId}`, as(ada, P));
+ assert.strictEqual(shown.status, 200, shown.text);
+ assert.ok(JSON.stringify(shown.json).includes('model:gpt-5-mini'));
+ assert.strictEqual(shown.json.names.a, sideEntrants.a);
+ });
+
+ await check('a second judgement is refused and Bench is not called again', async () => {
+ const again = await t.call('POST', `/api/v1/races/${raceId}/judgement`, { body: { winner: 'b' }, ...as(ada, P) });
+ assert.strictEqual(again.status, 409, again.text);
+ assert.strictEqual(again.json.code, 'run.race_already_judged');
+ assert.strictEqual(bench.calls.length, 1);
+ });
+
+ await check('matchmaker: true picks two entrants of different families from the configured list', async () => {
+ const r = await t.call('POST', '/api/v1/races', {
+ body: { prompt: 'Write a haiku about the sea.', kind: 'code', matchmaker: true, budget: { wall_ms: 40_000, cpu_ms: 40_000, mem_bytes: 268_435_456 } },
+ ...as(ada, P),
+ });
+ assert.strictEqual(r.status, 201, r.text);
+ const made = r.json;
+ assert.strictEqual(made.sides.length, 2);
+ assert.strictEqual(made.revealed, false);
+ assert.ok(!JSON.stringify(made).includes('model:'));
+ const judged = await t.call('POST', `/api/v1/races/${made.id}/judgement`, { body: { winner: 'tie' }, ...as(ada, P) });
+ assert.strictEqual(judged.status, 201, judged.text);
+ const pair = Object.values(judged.json.names);
+ assert.strictEqual(new Set(pair).size, 2);
+ for (const id of pair) assert.ok(t.config.races.entrants.includes(id), `${id} came from RUN_RACE_ENTRANTS`);
+ assert.notStrictEqual(familyOf(pair[0]), familyOf(pair[1]), `different families: ${pair.join(' vs ')}`);
+ assert.strictEqual(bench.calls.length, 2);
+ assert.strictEqual(bench.calls[1].judgements[0].winner, 'tie');
+ });
+
+ await check('a malformed race request is refused 400 and starts no child job, and a service for itself is refused', async () => {
+ const before = (await t.db.many('SELECT count(*)::int AS n FROM run_jobs WHERE project_id = $1', [P]))[0].n;
+ const cases = [
+ { ...body, entrants: ['model:gpt-5-mini'] },
+ { ...body, matchmaker: true },
+ { ...body, entrants: ['not-an-entrant', 'model:glm-5.3'] },
+ { prompt: '', kind: 'chat', entrants, budget: body.budget },
+ { ...body, budget: undefined },
+ { ...body, kind: 'poetry' },
+ { ...body, extra: true },
+ ];
+ for (const bad of cases) {
+ const r = await t.call('POST', '/api/v1/races', { body: bad, ...as(ada, P) });
+ assert.strictEqual(r.status, 400, `${JSON.stringify(bad)}: ${r.text}`);
+ assert.strictEqual(r.json.code, 'run.invalid_request');
+ }
+ const after = (await t.db.many('SELECT count(*)::int AS n FROM run_jobs WHERE project_id = $1', [P]))[0].n;
+ assert.strictEqual(after, before, 'no child job was made by a refused race');
+ const self = await t.call('POST', '/api/v1/races', {
+ body: { ...body, entrants: ['model:gpt-5-mini', 'model:glm-5.3'] },
+ token: t.network.signService({ sub: 'svc:compare', cap: ['run.job.submit'], project_id: P }),
+ });
+ assert.strictEqual(self.status, 403, self.text);
+ assert.strictEqual(self.json.code, 'run.race_person_required');
+ });
+}
+
+async function main() {
+ const bench = fakeBench();
+ const t = await boot({
+ env: {
+ OV_OAUTH_CLIENT_SECRET: 'run-test-secret',
+ RUN_RACE_ENTRANTS: 'model:gpt-5-mini,model:glm-5.3,model:deepseek-flash,agent:codex+gpt-5-mini#0123abcd',
+ },
+ appOpts: { bench },
+ });
+ try { await checks(t, bench); } finally { await t.close(); }
+ done();
+}
+
+main().catch((e) => { console.error(e); process.exit(1); });