Repositories openvibers openvibe.bench official

openvibers/openvibe.bench

Open source AI benchmark and comparison engine: blind votes, graders, tests and game outcomes, cost per solved task, open data.

Star 0 Watch 0 Fork 0
Get it

Get this repository

Clone over HTTPS

A Git token is made for this copy and shown once. It can read and write the repositories you own, and expires in an hour.

git clone https://git.openvibe.codes/openvibers/openvibe.bench.git
Clone to my computer

OpenVibe Link asks you on your computer. The folder is inside the root you chose in Link, and the request ends after two minutes.

Clone to my computer
Open in Codes

A Codes session clones from our disk: no network, no GitHub quota. When this repository has a GitHub twin, Codes pushes there too.

Open in Codes
main
Files at main
.env.exampleBench pages move to Report's AI section; the API stays (#150)4 days ago1.5 KB
.githubOpenVibe.Bench v115 days ago
.gitignoreOpenVibe.Bench v115 days ago96 B
AGENTS.mdAGENTS.md: the context an agent starts from (#215)1 days ago9.1 KB
CHANGELOG.mdAL8: score whole tickets over repeated trials (codes.ticket) (#184)3 days ago5.4 KB
CITATION.cffREADME first screen and CITATION.cff (#180)3 days ago540 B
CLAUDE.mdAGENTS.md: the context an agent starts from (#215)1 days ago11 B
LICENSEOpenVibe.Bench v115 days ago33.7 KB
README.mdopenvibe-shared v4.235.0 (was v4.234.0) (#232)19 min ago2.3 KB
SECURITY.mdOpenVibe.Bench v115 days ago2.1 KB
STATUS.jsonopenvibe-shared v4.235.0 (was v4.234.0) (#232)19 min ago563 B
deployOpen Bench messages in Dock Chats (#158)4 days ago
docsUse one fitted rating across boards and model reads (#189)3 days ago
fixturesAL8: score whole tickets over repeated trials (codes.ticket) (#184)3 days ago
migrationsMatches: every comparison as one record, and the model/harness factor board (#230)3 h ago
package-lock.jsonopenvibe-shared v4.235.0 (was v4.234.0) (#232)19 min ago65.1 KB
package.jsonopenvibe-shared v4.235.0 (was v4.234.0) (#232)19 min ago1.1 KB
publicMotion: Calm reduces, only Off stops (#183)3 days ago
scriptsAL8: score whole tickets over repeated trials (codes.ticket) (#184)3 days ago
serverMatches: every comparison as one record, and the model/harness factor board (#230)3 h ago
testMatches: every comparison as one record, and the model/harness factor board (#230)3 h ago

README.md

View file

OpenVibe.Bench

The shared AI benchmark and comparison engine for every OpenVibe room (plan 2026-10-28 §4.12).

Live site: https://bench.openvibe.services · API docs: https://openvibe.services · For AI: https://bench.openvibe.services/llms.txt
Run it: npm ci && npm run dev (Node 22 or newer) · Tests: npm test

Bench is the comparative-evaluation authority. AI stays the run authority, Report the analytics authority, Route the learner, and each room keeps its own pages. Bench owns kinds, tasks, pinned rounds, entries, judgements, ratings, boards, the open dataset and private evals.

The public pages moved to OpenVibe.Report's AI section (2026-11-08): https://openvibe.report/ai, /ai/<benchmark> and /ai/models/<model>. This origin's old HTML pages answer 301 there; its API, MCP endpoint, render origin, badges and embeds stay here. The owner's pages stay here too: /benchmarks/mine (the default audience, "apply to all", and each benchmark's Share button), /benchmarks/new, an unlisted share link (/b/<token>), and the noindex task and run pages. A benchmark's own page redirects only when it is public and the viewer is not its owner: a private benchmark, a named reader and an unlisted link have no Report page.

  • Service id: bench
  • Port: 5230
  • Public origin: https://bench.openvibe.services
  • Database: ov_bench
  • Cookies: bench_at, bench_rt, bench_oauth

Run

npm install
npm start

Without DATABASE_URL the service runs an embedded PGlite database in data/pglite (development only). Production uses PostgreSQL with DATABASE_DIRECT_URL for migrations.

Test

OV_ALLOW_SUITE=1 npm test

Tests use a temporary PGlite database, an in-process mock Network and a fake AI; none needs the network or a paid API.

API

JSON under /api/v1; errors are application/problem+json. The full list is in /openapi.json and Report's /ai/method. The MCP endpoint is POST /mcp.

The public reads Report's AI pages use are GET /api/v1/overview, GET /api/v1/benchmarks, GET /api/v1/rankings, GET /api/v1/runs, GET /api/v1/models/:model/summary and GET /api/v1/runs/:id; the long list is at /openapi.json and in Report's method page.

<!-- versions:start -->

  • openvibe-contracts: v0.422.0
  • openvibe-sdk: v0.161.0
  • openvibe-shared: v4.235.0

<!-- versions:end -->